Conversation
Partition each live causal prefix across 2/4/8 CTAs, emit normalized output plus LSE per split, and merge with a stable max-LSE reduction. The exact gfx1201 H24/KV4/D256 eager multi-row route is opt-in and reuses the existing flash-partials allocation, so it adds no persistent VRAM.\n\nR9700 (gfx1201) evidence: B16/ctx20676 S8 attention 0.410 ms versus 2.556 ms for the direct packed FA2 path; fresh-process DFlash median 48.2 versus 39.3 tok/s (+22.6%). CPU-f64 oracle worst rel-L2 2.486e-4, minimum cosine 0.999999821. LongBench hard30 completed 30/30 with 0 errors; five long-decode fixtures completed 5/5.
HUSRCF
marked this pull request as ready for review
September 14, 2026 13:21
HUSRCF
marked this pull request as draft
September 14, 2026 13:28
HUSRCF
marked this pull request as ready for review
September 14, 2026 13:29
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacked follow-up to #758. The base is intentionally
fa2-prefill-gfx1201: #758 supplies Kaden's packed-Q8 GQA/LDS front end, while this PR only adds the split-KV verifier back end and product admission.For small DFlash verifier batches at long context, partition the live causal KV prefix across 2/4/8 CTAs. Each split emits normalized
Oaccplus LSE, then a small merge kernel combines the records with a stable max-LSE reduction. This restores parallelism without discarding #758's shared packed-KV loads inside each tile.The route is opt-in through
HIPFIRE_GFX12_FA2_SPLIT_VERIFY=1(HIPFIRE_GFX12_FA2_SPLIT_COUNT={2,4,8}, default 8). It remains fail-closed to the existing multi-row admission plus exact gfx1201, H24/KV4/D256, Q8 KV, eager sequential verification. Tree, independent-batch, graph-capture, and retained/PM4 recording routes remain on the established path. Plain AR (n=1) cannot enter this route.Performance
Radeon AI PRO R9700, gfx1201, Q8 VMM KV, Qwen3.8-27B MQ4XT, B16, context 20,676:
Fresh-process product DFlash A/B at 21,550 actual prompt tokens, 200 output tokens,
HIPFIRE_VERIFY_GRAPH=0:All six samples retained tau 1.80 and 71 cycles. A plain-AR smoke was unchanged within noise (32.6 tok/s with the env arm off, 32.5 with it on), consistent with the
n=1exclusion.Fixture identities:
b4d0b63cddcac872648ddf3cdd92cac29f91556f7e0431a077d03756a7102d0154108757289e6e5fe9a2d204c0c9eeb7d0a74a232a0e2166d889f823e91e0fbf778d21dd9668d7de055cdecb065401bc36576b9fc40345bbce6fa96c90906f56db81f2955c042c254432bc64d01437c1Correctness and long-run E2E
The CPU-f64 oracle covered S8 tail/ragged/sentinel shapes. Worst relative L2 was
2.486e-4; minimum cosine similarity was0.999999821. The partial kernel compiles at 243 VGPR / 27 SGPR and the merge at 18 VGPR / 10 SGPR, with zero spill and zero private scratch.LongBench hard30, thinking enabled (
temperature=1,top_p=.95, xhigh), Q8 VMM KV,max_seq=131072, output allowance 65,536:stop839a19be0b3b1c801a0ca58d388996faff34704e34dd196d54346adf17a1dce9Five no-think long-decode fixtures (KV Cache article, SGLang article, story, Snake, Tetris):
stop; the two 1,536-token technical articles reached their fixture-specific length capstopThe E2E binary was the equivalent current-beta port used to develop this stacked delta. The kernel source in this PR and that measured binary is byte-identical (SHA-256
7a006296173579377e69ec2a8a3947d3af40017922fb35d0d69eee91add2e394).Memory
At S8/B16/H24/D256 the split records require about 3 MiB, but they reuse the existing
flash_partialsallocation. There is no new persistent VRAM allocation.Static validation on this exact stacked commit
cargo check -p rdna-compute -p hipfire-arch-qwen35: passcargo test -p rdna-compute --lib: 246/246 passcargo test -p hipfire-arch-qwen35 --lib: 196/196 pass, 17 hardware-fixture tests ignored by designThis improves the verifier cost only. It does not claim to repair low draft/target acceptance on prompts where tau itself collapses.