Repository navigation
AVX-512 VNNI Q4_K kernels are compiled, linked and structurally unreachable — the avx2 && fma fast path returns before the VNNI gate is ever read #2604
Description
Severity: P1 — found by the pre-release multi-platform dogfood sweep (intel/AVX-512 lane).
Measured against the published apr 0.63.0 from crates.io on the Intel receipt host
(Xeon W-3245, 16C/32T, AVX-512F/BW/CD/DQ/VL/VNNI), not a local build.
Where
crates/aprender-serve/src/quantize/q5k_q6k_matvec.rs:119-166 (early return) vs :176-186 (VNNI gate); same shape at crates/aprender-serve/src/quantize/parallel_k.rs:334-368; dead import at parallel_k.rs:16
Defect
THE ANSWER TO THE LANE'S QUESTION 3: avx512_vnni is DETECTED AND IGNORED. The AVX-512 VNNI Q4_K kernels are compiled in, retained by the linker, and execute ZERO times — not because of a cpuid miss, but because they are structurally unreachable on every x86 CPU that exists. In q5k_q6k_matvec.rs the fast path gates on use_lean = is_x86_feature_detected!("avx2") && is_x86_feature_detected!("fma") (line 119-120), calls ggml_style_q4k_q8k_dot_avx2_raw (line 159) and return Ok(()) at line 166. Only AFTER that unreachable point does line 177 compute use_4row_kernel = is_x86_feature_detected!("avx512f") && is_x86_feature_detected!("avx512vnni") and dispatch to fused_q4k_q8k_dot_4rows_avx512vnni. Since every shipping AVX-512 part also has AVX2+FMA, use_lean is ALWAYS true wherever use_4row_kernel could be true. The VNNI block can only run on a CPU with AVX-512 VNNI but WITHOUT AVX2 — which does not exist. parallel_k.rs::fused_q4k_preq8k_matvec_into has the identical if avx2 && fma { … return Ok(()) } shape, and parallel_k.rs:16 imports fused_q4k_q8k_dot_4rows_avx512vnni without ever calling it in that file. This is the #2567 failure mode (a function whose name promises an ISA that never runs) recurring on x86: 25 is_x86_feature_detected!("avx512vnni") sites and real _mm512_dpbusd_epi32 intrinsics, all dead on the inference path.
Reproduction
MEASURED on mac-server, published apr 0.63.0 sha256 2c7bc1d0eb…
. # probes verified at symbol+0 first
sudo perf probe -x /home/noah/.cargo/bin/apr --add 'k512_4row=0x182efc0' # 0x182ffc0 - 0x1000
sudo perf probe -x /home/noah/.cargo/bin/apr --add 'k512_v2=0x182e740' # 0x182f740 - 0x1000
sudo perf probe -x /home/noah/.cargo/bin/apr --add 'ggml_avx2=0x1ac58a0' # 0x1ac68a0 - 0x1000
sudo perf probe -l # MUST print 'on <symbol>' with no +NNN, else counts are meaningless
sudo perf stat -e probe_apr:k512_4row,probe_apr:k512_v2,probe_apr:ggml_avx2,probe_apr:avx2_q6k \
-- /home/noah/.cargo/bin/apr run /home/noah/models/qwen2.5-coder-1.5b-instruct-q4_k_m.gguf \
--prompt 'What is 2+2?' -n 8 --no-gpu
RESULT (exact, reproduced across 4 consecutive runs):
0 probe_apr:k512_v2 <- AVX-512 VNNI single-row
0 probe_apr:k512_4row <- AVX-512 VNNI 4-row micro-kernel
29,141,504 probe_apr:ggml_q4k_avx2 <- ggml_style_q4k_q8k_dot_avx2_raw
8,320,128 probe_apr:avx2_q6k <- fused_q6k_dot_avx2
7,896 probe_apr:mv_q4k_q8k_par <- the matvec DID run, so it is not an unreached caller
PMU corroboration, same command:
fp_arith_inst_retired.256b_packed_single 8,142,423,747
fp_arith_inst_retired.512b_packed_single 438,146 (1 per 18,585 256-bit ops)
fp_arith_inst_retired.512b_packed_double 0
core_power.lvl1_turbo_license 96,528,500,884 vs lvl2 1,436,500,632
The residual 512b/lvl2 is consistent with glibc's AVX-512 memmove, not with any FP kernel.
VARIED THE HOST (one input is an anecdote) — same experiment on lambda, AMD Threadripper 7960X, Zen 4, avx512f+avx512_vnni present, different build (2196de281), different mangling (_ZN not _RNv), addresses re-resolved:
0 L_k512_4row
0 L_k512_v2
29,141,504 L_ggml_avx2 <- identical to the Xeon, to the call
8,320,128 L_q6k_avx2 <- identical to the Xeon, to the call
Two microarchitectures, two builds, byte-identical kernel call counts: the dispatch is host-independent, confirming a structural early return rather than a cpuid difference.
Filed from the 0.64.0 pre-release sweep. Scope AVX512-SPECIFIC.
Activity
noahgift commentedon Sep 5, 2026
0.67 lane — attaches to PP-066 spec §6 row W-B kernel.
AVX-512 VNNI Q4_K kernels execute zero times because the avx2&&fma fast path returns first (perf-probe counts on two hosts). CPU kernel dispatch work is 0.67 per D-1; §6 has no CPU-x86 kernel row — nearest is W-B kernel (Q4_K GEMV family); flagged as a §6 gap. Measured on the published 0.63.0 artifact; no receipt answers it, so not E.
Instance of class #2880 — the shipped CPU path runs a narrower ISA or algorithm than the host supports.
PP-066 triage 2026-09-05 (issue #2873); ledger
docs/audits/pp-066-triage-2026-09-05.md.noahgift commentedon Sep 23, 2026
triage 2026-09-23 (aprender-dd, standing triage owner): milestoned to 0.73.0, theme llama.cpp Parity (performance, kernels, parity benches); it moved out of
backlog(the 2026-09-17 overflow bucket, not a release).noahgift commentedon Sep 24, 2026
Closed by paiml-implement — folded into epic #3999 (triage evidence/triage-1717.md)