fix(mtp): chunk scalar Q8 batched GEMM above 64 rows

[HUSRCF]

Summary

Fixes #732 by routing scalar Q8 native-MTP batches through the existing F32
chunked entry point, which splits the batch dimension at the kernel's fixed
MAX_BATCH=64 boundary.

This preserves the scalar F32 reduction contract while allowing Qwen3.8 native
MTP prefill and verification widths such as 65, 70, 76, and 256 to complete
instead of failing at runtime.

Root cause and fix

gemm_q8_0_batched is a fixed-size scalar kernel and correctly rejects
batch_size > 64. Beta already provides
gemm_q8_0_batched_f32_chunked, but several native-MTP scalar-Q8 surfaces
still called the fixed-size kernel directly.

This change routes those surfaces through the existing chunked entry point:

• native MTP batched projections and explicit scalar LM-head fallback;
• MTP trunk verification when the WMMA route is disabled;
• DFlash/MTP composition scalar-Q8 paths;
• the scalar reference arm of bench_q8_0_batched.

The scalar kernel and automatic WMMA selector are unchanged. Batches at or
below 64 retain the existing behavior; wider batches become contiguous chunks,
for example 70 = 64 + 6 and 256 = 4 x 64.

Closes #732.

Which surface(s) does this touch?

• kernel — crates/rdna-compute
• load
• serve — native MTP generation and verification
• arch crate(s): hipfire-arch-qwen35
• crates/hipfire-quantize / quant formats
• control plane
• docs / CI / scripts only
• policy files

Test plan

• ./scripts/no-gpu-ci.sh passes, or the CI jobs are green
• cargo build --release --workspace --all-targets --locked
• cargo test -p hipfire-arch-qwen35 --lib — 196 passed, 17 ignored
• cargo test -p rdna-compute --lib -- --skip dispatch::tests::upload_raw_copy_failure_hip_frees_owner — 244 passed
• isolated rerun of the skipped beta allocator test — 3/3 passed
• gfx1201 scalar-Q8 boundary matrix — B=64/65/70/76/256, all bit-exact
• Qwen3.8 native-MTP LongBench-v2 hard30 at a 32K output cap
• five Qwen3.8 native-MTP long-decode prompts at a 32K output cap
• post-rebase serve_harness.py --mode battery — pending an uncontended GPU window; the first attempt was discarded because an unrelated hardware campaign began concurrently
• speed gate — no performance claim; the change replaces a runtime error above B=64

The skipped allocator test compares process-global free VRAM for exact equality.
It passed three isolated repetitions, but failed inside the full crate once when
free VRAM decreased by 2 MiB and once when it increased by 4 MiB. This diff does
not touch allocation code; the result is disclosed as beta baseline flakiness.

Scalar-Q8 boundary matrix

Hardware: Radeon AI PRO R9700 (gfx1201), ROCm/HIP 7.14. The legacy scalar
route was forced with HIPFIRE_Q8_BATCHED_LEGACY=1.

Batch	Result	Scalar reference
64	pass	bit-exact
65	pass	bit-exact
70	pass	bit-exact
76	pass	bit-exact
256	pass	bit-exact

No MAX_BATCH error occurred.

32K-output long-context quality evidence

Target: qwen3.8-27b.mq4-xt
Target SHA-256: 9f91556f7e0431a077d03756a7102d0154108757289e6e5fe9a2d204c0c9eeb7
Native Q8 MTP sidecar SHA-256: c160bbf9f8cc4b9cee3bd3e8438263aec7946f661e2254ceecf9990ce86bd1d3
Daemon MD5: 2aee0c769188beec3af9606a9ab79bfb

Configuration: Q8 VMM KV, max_seq=65536, greedy, thinking off, native MTP
enabled, scalar-Q8 route forced, maximum output 32,768 tokens.

LongBench-v2 hard30:

• 30/30 completed with finish_reason=stop;
• 0 runtime errors and 0 MAX_BATCH failures;
• input lengths 20,412–30,228 tokens;
• 30/30 parseable choices;
• 14/30 correct (46.7%).

This is a single-arm supplementary quality smoke test, not an accuracy-equivalence
or performance claim.

Five long-decode prompts:

Prompt	Generated tokens	Finish	Runtime result
Python Snake	2,170	stop	completed; generated program has one syntax error
Python Tetris	4,312	stop	completed; Python syntax passes
Chinese science-fiction continuation	1,165	stop	coherent and fully concluded
KV Cache explanation	6,891	stop	coherent and fully concluded
Radix Attention explanation	8,572	stop	coherent and fully concluded

All five requests completed without runtime errors, empty output, or a repetition
attractor. The automated repetition heuristic flagged code/table/formula patterns;
manual inspection found structured repetition rather than decoding collapse. The
Snake syntax defect is retained in the report as model-output quality evidence,
not hidden as a pass.

local serve_harness battery.json

Pending an uncontended post-rebase hardware run. This draft will not be marked
ready until the complete per-turn JSON is attached.

Hardware validation request

{
  "routes": [
    {"mode": "battery", "tag": "qwen3.8:27b"}
  ],
  "claim": "native Qwen3.8 MTP scalar-Q8 batches above 64 rows complete without a MAX_BATCH runtime failure"
}

Architecture-trait change?

No.

[Sergey7210]

Hardware validation of PR #767 on Radeon RX 9060 XT 16 GB (gfx1200).

Environment:

• PR head: 4e1ac52f7b0dc2ff2a31b3eba9315b47544292ce

• OS/kernel: CachyOS, 7.2.6-1-cachyos

• ROCm/HIP: 7.2.53211-3d9ef42

• target: qwen3.8-27b.mq3

• target SHA-256: 09c3544690aceca29e1822d79adab6ffcc8fd9e4b58359fe8dfb185ef49811c9

• native Q8 MTP sidecar SHA-256: 401fe4b90297f2939a7df9ff3d4e54d95a1662b22e5dd5c2fe11e82963caab58

I explicitly forced the scalar F32 routes with:

• HIPFIRE_MTP_HEAD_LMHEAD_WMMA=0

• HIPFIRE_MTP_Q8_VERIFY_WMMA=0

• HIPFIRE_Q8_BATCHED_LEGACY=1

This prevents the validation from passing by routing around the changed scalar code through WMMA.

Scalar Q8 boundary matrix

bench_q8_0_batched 4096 4096 B 3:

B | Result | Exact | max_absdiff -- | -- | -- | -- 64 | PASS | YES | 0 65 | PASS | YES | 0 70 | PASS | YES | 0 76 | PASS | YES | 0 256 | PASS | YES | 0

All cases exited successfully. No SKIPPED, panic, or MAX_BATCH failure occurred.

Native-MTP serve battery

Committed serve_harness.py --mode battery, Q8/VMM KV, native MTP K=3:

• 5/5 requests completed with non-empty coherent output

• mtp=true in all five result rows

• daemon reported drafter=mtp for all five requests

• 3 requests finished with stop

• 2 reached the configured 128-token output cap with length

• empty responses: 0

• attractors: 0

• stream/runtime errors: 0

• MAX_BATCH failures: 0

Long-context native-MTP test

The 16K-configured NIAH case completed through the forced scalar route:

• actual input: 10,883 tokens

• native MTP loaded and selected

• prefill: 169.4 tok/s

• finish: stop

• recall: 1/1

• output: The secret pass code is mauve-velociraptor-7741.

• no MAX_BATCH failure

32K caveat

The original 32K-configured NIAH fixture could not complete on this 16 GB board. After the native MTP sidecar loaded and the MTP speculator was enabled, prefill stopped with:

HipError(2): hipMemCreate: out of memory

This was an allocation failure, not the previous:

gemm_q8_0_batched: batch_size 256 exceeds kernel MAX_BATCH=64

Therefore I cannot claim complete end-to-end validation of the exact original 32K fixture on this GPU. However, the scalar boundary matrix, committed native-MTP battery, and successful 10.9K-token native-MTP prefill all confirm that batches above 64 are now chunked successfully on gfx1200.

Verdict:

• scalar Q8 chunking fix on gfx1200: PASS

• exact original 32K end-to-end reproduction: BLOCKED by VRAM capacity

• overall hardware validation: positive, with the 32K limitation disclosed

[hipfire-pr767-gfx1200-validation.tar.gz](https://github.com/user-attachments/files/32335427/hipfire-pr767-gfx1200-validation.tar.gz) [summary.txt](https://github.com/user-attachments/files/32335428/summary.txt)