Release gate: apr serve throughput vs llama.cpp on the SAME GGUF (Qwen3.5 0.8B/2B/4B/9B/27B) — a regression ratchet that blocks the tag, plus a published parity target #4218

Operator, 2026-09-24: "make the scoreboard a release gate".

Why: every current release gate checks correctness. No gate compares apr's speed with another engine (#3576: the parity basis is apr-vs-apr). That's how 0.69.1 shipped Qwen3.5 prefill at one token per step, 77 tok/s against llama.cpp/ollama's ~8.2k on the same RTX 4090 and the same weights (#3596). The prefix-reuse gap (#4214) and decode at 0.58× (#4215) got through the same hole. A one-token prefill passes every correctness gate there is.

Scoreboard source: infra PMAT-1009, lqw-bench <engine> (worker infra-76), receipts under infra docs/audits/LQW-001/. Engines: apr serve, llama.cpp llama-server, ollama, and vLLM (docker, reference only).

Gate design:

1. Comparator: llama.cpp llama-server at a pinned release, on the SAME GGUF file (sha256 recorded), the same GPU and the same prompts. It's the only comparator that takes identical bytes, so it's the one that gates. ollama and vLLM rows are published as information and don't gate.
2. Metrics per model (Qwen3.5 0.8B, 2B, 4B, 9B, 27B Q4_K_M):
   • TTFT at 850, 4k and 32k tokens
   • prefill tok/s
   • single-stream decode tok/s
   • aggregate decode at concurrency 1, 4 and 16
   • cached-prefix TTFT
   • peak VRAM and host RSS
3. Verdict is a RATIO against the comparator measured in the same run, never an absolute number, so a hotter box or another GPU doesn't flip it.
4. Two thresholds:
   • Ratchet (blocks the tag): a ratio may not fall more than a noise band (set from measured run-to-run variance, n≥3) below the last released version's committed ratio. Every release must be at least as close to llama.cpp as the previous one.
   • Target (published, reported every release): prefill ≥0.8×, decode ≥0.9×, cached TTFT ≤1.5×. Once met, a target folds into the ratchet.
   • A gate set at parity today would be red forever, and an unpassable gate trains bypass. The ratchet is what makes this one passable and still binding.
5. Correctness is a precondition of speed: temp-0 token-identical output to the previous release on the fixed prompts, or the speed rows are void.
6. No silent passes:
   • GPU-idle check before each run, taking the shared GPU lock;
   • an engine that fails to start or a model that doesn't fit is a named row, never a skipped one;
   • the gate prints how many rows it measured, and 0 rows exits non-zero;
   • self-test against a committed fixture where apr is deliberately slowed (e.g. the forced one-token prefill), which must go RED.
7. Where it runs: a GPU host at release time. lambda-labs is NOT a CI runner (fleet rule), so either a gx10 runner lane (GB10, with its own committed baseline) or a release-captain step on lambda with the receipt committed and checked by CI. Captain's call.

Done when: the 0.69.4 tag can't be cut without a committed scoreboard receipt whose ratchet verdict is PASS, and reverting #3596 makes the gate RED.