apr prefills at decode rate on CPU: no batched prefill outside src/cuda/, 6.5x slower than llama.cpp on the same box #2787

Found by PERF-060 while calibrating intel and mini for the measurement campaign.

apr prefills at decode rate on CPU. Measured prefill-to-decode ratio: 1.07× on intel, 1.02× on mini. A GEMM-based prefill should be many times faster than token-by-token decode; these are indistinguishable.

crates/.../generate_quantized.rs:568 loops forward_single_with_cache over every prompt token. Every batched prefill_* in the tree lives under crates/aprender-serve/src/cuda/. So there is no batched prefill on the CPU path at all — it is decode, run once per prompt token.

The comparator gap, on the same box and the same file

Roughly 6.5× slower. Recorded here as a host-capability reference, not as an Arm B number — that llama.cpp build is the wrong pin (version: 1 (35bee03)) and cannot be an oracle.

What it costs the campaign

Every replicate measured, W1 shape (513-token prompt, 128 generated, c=1):

evidence/perf-054/campaign-scope.md assumed 40/6 and 60/8 tok/s. Prefill is 4.8× optimistic on intel and 25.3× on mini; t_req is 2.3× and 11.1× off. intel's apr-lane cell goes from an estimated 8.9 h to 19.2 – 21.3 h.

Note the decode estimate for intel (6 assumed vs 6.57–7.66 measured) was the one figure that held, and was slightly pessimistic. The bandwidth model of the hardware was sound. The error was assuming apr prefills with a GEMM.

Recommendation

Land batched CPU prefill before scheduling either measurement slot. It takes intel's cell to roughly 7 h — below the original estimate — and it moves mini from unmeasurable to measurable (see the timeout blocker filed separately: mini's 270 s t_req exceeds the protocol's hard 120 s per-request timeout, so every request aborts).

Scheduling the campaign first would spend ~19–21 h on intel and produce a guaranteed-empty receipt on mini, then have to be redone — and per I-9 a band may not be re-run to green, so those runs would be spent.

This is also a product defect independent of the campaign: CPU serving is 6.5× off the comparator on prefill for every user, not only for the benchmark.