Repository navigation
FP8 batched GEMM multiplies a STALE activation on mixed-quant models: the PMAT-084 cache key (ptr, count) aliases across layers — #3602's cosine 0.4153 #3727
Description
Root cause of #3602's cosine 0.4153 (localized in #3602 (comment)), 0.69.1 row under #3710.
cublas_prefill_fp8_gemm (crates/aprender-serve/src/cuda/executor/layers/cublas_prefill/gemm.rs) reuses its converted FP8 activation whenever (packed_input_ptr, m·k) equals the last conversion's key (PMAT-084). The key is cleared only when the scratch buffer is reallocated, never when the input's contents change. Any content change behind an unchanged (ptr, count) therefore multiplies a stale activation.
It fires on every mixed-quant model. qwen2.5-coder-0.5b's hidden size of 896 is not a multiple of 256, so Q/K/V/O/gate/up are Q5_0/Q8_0 and take non-FP8 routes, and only ffn_down (Q4_K/Q6_K) is FP8. PREFILL_DETAIL_TRACE=1 shows 23 hits, all ptr=0x71aec6600000 count=238336 (49 × 4864): layers 1–23 reuse layer 0's activation. It fails on lambda (batched prefill is the default at cc 89) and identically on gx10 with BATCHED_PREFILL=1.
done_when
- A hit is only possible between GEMMs that share an input inside one phase (QKV, gate+up). Every non-FP8 route invalidates the key, and every phase that writes a fresh input opens a new scope, so the
(ptr, count)aliasing class is closed, not just this instance. - A test drives a mixed-quant layer sequence (6 non-FP8 GEMMs, then an FP8
ffn_downon the same buffer, × N layers) and asserts 0 cross-layer hits. Today's code gives N−1. An all-FP8 sequence still gives 3 hits per layer (K, V, up). A mutant that removes the invalidation turns it RED. - qwen2.5-coder-0.5b Q4_K_M: GPU backend with no fallback,
apr runcorrect,apr qagolden 3/3, on lambda (default path) AND on gx10 withBATCHED_PREFILL=1.PREFILL_DETAIL_TRACEshows 0 hits for it and 84 for qwen2.5-coder-7b.
Activity
noahgift commentedon Sep 23, 2026
milestone → 0.73.0 per docs/roadmaps/milestone-triage-2026-09-23.md (#4024), operator approval 2026-09-23 (relayed by aprender-cf). Reason: performance/decode-prefill path: llama.cpp Parity (#3999)
docs(roadmap): quorum round 10 -- 18 applied rows annotated as verify…
noahgift commentedon Oct 14, 2026
P0 to P2 (2026-10-14 P0 triage): P0 now means it blocks the next release or carries the operator's own words on it; this issue does neither, and its milestone is 0.73.0.
noahgift commentedon Oct 14, 2026
merged into #5312 (master ticket for epic #3994; triage rule 7b, 2026-10-14). It is a checklist row there; reopen this issue when work on it starts or its PR opens.
noahgift commentedon Oct 14, 2026
Moved: this sub-ticket is now a row in #5323 (CUDA quantized kernels: correct block layouts, cache keys and precision), split out of #5312 by theme.
noahgift commentedon Oct 14, 2026
Measured on
main@ 3064c77 (the fix reached main through the v0.69.3 merge-back, #4338:fp8_activation_cache.rs,share_next/begin_dispatch). Built with--features cuda.Unit tests (done_when 2).
a_mixed_quant_model_never_reuses_an_earlier_layers_activation,an_all_fp8_model_keeps_its_three_shared_input_hits_per_layerand the other 5 cache tests pass on an RTX 4090 (21/21 tests under thefp8_filter).Models (done_when 3).
apr run --gpuwithPREFILL_DETAIL_TRACE=1, thenapr run --format jsonandapr qa --json:BATCHED_PREFILL=1 FP8_PREFILL=1BATCHED_PREFILL=1 FP8_PREFILL=1apr run, at 64 and 81 tokens. The hits split as 84 atcount=229376(64×3584) and 84 atcount=290304(81×3584), which is 3 hits × 28 layers per prefill, as predicted.ffn_down× 2 prefills, and 0 hits. Before the fix it was 23 cross-layer hits per prefill.BATCHED_PREFILL=1alone now prefills in FP16 (FP8 batched prefill diverges on sm_121 (GB10): qwen2.5-coder-7b BATCHED_PREFILL=1 fails F2 at pos 1 (cos 0.872) while FP8_PREFILL=0 is green; detect_fp8_prefill's doc says 'OFF for Blackwell' but the code enables it at cc ≥ 89 #3785 default), so the FP8 path never ran there (0 FP8 GEMMs). That leg is vacuous for this bug, so it was rerun withFP8_PREFILL=1, as in the table.Every done_when item now has a measurement. This comment does not close the issue; closing is the triage call.