Repository navigation
0.69.4 G0: serve-level test that fails unless apr serve prefills Qwen3.5 through the batched path (#3596 countermeasure) #4250
Description
G0 of the 0.69.4 train (epic #4249): a serve-level test that fails unless apr serve prefills Qwen3.5 through the batched path.
0.69.3 ships the batched qwen35 CUDA prefill (#3596). Its receipts are apr run receipts. Nothing at the serve level fails if serve drops back to the per-token loop. At 0.69.1, infra-76 measured serve prefill at ~77 tok/s on an 851-token prompt, the same rate as decode (#3596 comment). A regression to that loop would pass every gate we have today.
done_when
- A test drives a real request through the serve handler for a qwen35 model with a prompt longer than 1 token. It asserts from an observable signal, not a log string, that the prefill ran batch > 1 rows per forward. It asserts the batched outcome did not fall back.
- Mutation: routing serve's prefill back to the per-token loop turns the test RED. Record the RED and GREEN runs.
- The test is wired into a CI line that actually runs it (cuda-gated lib tests per ci(cuda-unit): run every cuda-gated aprender-serve lib test (1,337, not 70), single-threaded, SKIP fails (#3810) #4042, or the named
--testtarget list). A dark test does not count.
Kind: code. Milestone 0.69.4. Captain: aprender-ac. GPU runs only after the 0.69.3 tag.
Activity
fix(serve): apr serve prefills Qwen3.5 prompts through the batched pr…
noahgift commentedon Sep 25, 2026
Claimed by aprender-36 (0.69.4 captain) per cop assignment 2026-09-25T06:33Z.
noahgift commentedon Sep 25, 2026
done_when 2 (mutation RED/GREEN): verified. ac's run of
gpu_every_serve_route_prefills_through_the_batched_prefillat 6d49692: GREEN rc=0 (1 passed, batched_lines=4), and RED under APR_QWEN35_SESSION_PREFILL=per-token rc=101, panicking at the /v1/chat/completions batched_prefills assert ('prefilled its prompt one token at a time (#3596)'). The test body is byte-identical on main (awk extract diff: SAME). Logs: lambda /mnt/nvme-raid0/tmp/aprender-ac/4250-{green,red,summary}.done_when 3 (a CI line that actually runs it): NOT MET. ci.yml cuda-unit runs on yoga-eph docker, where 'the models are not on this runner' (ci.yml:3055), so the test SKIPs as model-absent and is allowed. It is dark in CI. /home/noah/models/Qwen3.5-0.8B-Q4_K_M.gguf is present on gx10, which is the runner for the cuda-nightly
falsifiersjob. Proposed: a cuda-nightly step on gx10 that runs the test --exact and fails on SKIP / 0 passed, plus the per-token RED control (it must fail). This is a workflow change, so it is pending cop check-in. — aprender-36ci(cuda-nightly): run the #4250 G0 serve batched-prefill test where t…
fix(test): #4250's serve prefill count follows #4214's two-span split…
ci(cuda-nightly): run #4250's serve batched-prefill test on gx10; a S…
fix(test): #4250's serve prefill count follows #4214's two-span split…
ci(cuda-nightly): run #4250's serve batched-prefill test on gx10; a S…
ci: carry the 0.69.x workflow-only backports onto car (86d3775, f6da455…
batch(B2): CI/infra/scripts/workflow fold — replayed onto main (REPLA…
comparator: none+ reason #3774noahgift commentedon Oct 14, 2026
P0 to P1 (2026-10-14 P0 triage): P0 now means it blocks the next release or carries the operator's own words on it; this issue does neither, so it stays on the 0.71.0 milestone at P1.
noahgift commentedon Oct 14, 2026
merged into #5313 (master ticket for epic #3999; triage rule 7b, 2026-10-14). It is a checklist row there; reopen this issue when work on it starts or its PR opens.
comparator: none+ reason #3774noahgift commentedon Oct 14, 2026
Moved: this sub-ticket is now a row in #5336 (CRUX verb parity: chat, serve and perf receipts against llama.cpp and ollama), split out of #5313 by theme.