Repository navigation
Q4_K GEMV has no aarch64 SIMD and its "parallel" variant calls the scalar path #2567
Description
The hottest kernel in quantized inference is scalar and single-threaded on every ARM
machine, silently. Found by the aarch64 dogfood before the 0.64.0 cut (#2566).
crates/aprender-compute/src/backends/q4k/gemv/mod.rs:53 — #[cfg(target_arch = "x86_64")]
wraps the entire fast path: the >=8M-work parallel dispatch (exactly what catches
FFN 8960x1536 and lm_head 151936x1536), AVX-512, and AVX2. On aarch64 control falls
straight through to scalar::matmul_q4k_f32.
Line 203 is the part worth reading twice:
#[cfg(not(target_arch = "x86_64"))]
fn matmul_q4k_f32_parallel(...) { scalar::matmul_q4k_f32(...) }A function named parallel that directly calls the serial scalar one.
Structural census (measured on origin/main)
aprender-serve/src: 183target_arch = "x86_64"sites vs 16 aarch64backends/q4k/andbackends/q6k/: zero aarch64 cfgsgrep -rn sve: nothing
Why no gate caught it
The numbers are correct — the 339-row tensor table on GB10 is byte-identical to x86.
Only the speed is wrong. Every correctness gate passes; there is no ARM speed gate at all,
so this could not have been caught by anything currently running.
Work
- NEON path for Q4_K GEMV (and Q6_K, same shape).
- Make
matmul_q4k_f32_parallelactually parallel on non-x86 — the rayon dispatch is
ISA-independent and is being skipped for no reason. - A speed gate that runs on ARM. Without it this regresses silently again, and the
naming (_parallelcalling scalar) shows how easy that is.
Not release-blocking for 0.64.0 — correctness is unaffected — but it means ARM users get a
fraction of the advertised throughput with nothing telling them.
Refs #2566
Activity
batch(0.64.0): fold 13 reviewed PRs into one integration branch — one…
docs(release): the remaining three 0.64.0 host receipts — the matrix …
docs(release): the fleet runs ONE identical 0.64.0 — and the shadow t…
perf(compute): the aarch64 Q4_K "parallel" path is now actually paral…
perf(compute): the aarch64 Q4_K "parallel" path is now actually paral…
feat(perf): the §4.4 protocol had no caller and the gate's real mode …
spec(perf): round 8 — 9 #1 stops being a roofline ratio and becomes a…
noahgift commentedon Sep 5, 2026
0.67 lane — attaches to PP-066 spec §6 row W-G.
The parallelism half landed (PR #2689, 1.21× on GB10, measured in nightly-bench.yml); the SIMD half — a NEON Q4_K/Q6_K GEMV — is verbatim §6 W-G ('GB10: NEON Q4_K GEMV'). The ARM speed gate it asks for is the gx10 cell (master row 15) plus the #2567 nightly measurement.
Instance of class #2880 — the shipped CPU path runs a narrower ISA or algorithm than the host supports.
PP-066 triage 2026-09-05 (issue #2873); ledger
docs/audits/pp-066-triage-2026-09-05.md.noahgift commentedon Sep 10, 2026
Closed by paiml-implement — #2880
noahgift commentedon Sep 10, 2026
Reopened by the PMAT-1100 triage orchestrator: #2567 is a child (sub-issue) of epic #2880, not a duplicate of it — the lane's close was wrong. Filed to a train.
release(0.67): train A — 17 row PRs of the 0.67.0 train in one build …
noahgift commentedon Sep 23, 2026
triage 2026-09-23 (aprender-dd, standing triage owner): milestoned to 0.73.0, theme llama.cpp Parity (performance, kernels, parity benches); it moved out of
backlog(the 2026-09-17 overflow bucket, not a release).noahgift commentedon Sep 24, 2026
Closed by paiml-implement — folded into epic #3999 (triage evidence/triage-1717.md)
docs(lookahead 0.73): R3 §11 — #2567 was folded, not fixed; K10 label…