[laudney]

Rows	fn_hc_dense_mix8 (main)	fn_hc_dense_mix_vec (this PR)	Saved a call
512	0.0575 ms	0.0401 ms	0.0174 ms
1,536 (default chunk)	0.3208 ms	0.3157 ms	0.0052 ms
4,218	0.8785 ms	0.8684 ms	0.0101 ms

GB10 (sm_121), nvcc 13.4.92, driver 580.178.04, on this PR's head 92e099761 (one commit on main ef9fa700a). tf-cuda-test flashnext-hc-mix-bench ROWS: alternating old/new order, CUDA events, median of nine samples after two warm-ups. The mixed rows and group sums are byte-equal at every row count.

Flash Next's HC stream mix combines four BF16 streams into 2,560 values and their FP32 group sums. fn_hc_dense_mix8 uses ten 256-thread blocks a row, one warp for each 32-column group. This PR's kernel reads BF16 pairs, so five blocks cover a row, and it writes the same bytes. The gain is small: at 1,536 rows, 94 calls in a main-model chunk save about 0.5 ms of a chunk that takes about 0.9 s at about 1,700 prompt tokens a second. No served speedup is claimed.

Refs #1754. Restacked onto main after #1764, as asked: it no longer depends on #1847.

What

• fn_hc_dense_mix_vec (flashnext_hc_mix_vec.cuh): one BF16 pair a thread, 64 columns a warp, five blocks a row. Each value keeps dense_mix8's stream order, __fmul_rn / rbf rounding points and __fdiv_rn(total, 4). Each 32-column group sums in ascending order, as dense_mix8's lane 0 does.
• Glue.hcDenseMix takes the vector kernel from 512 rows of the 2,560 × 4 shape (mixVec), and keeps dense_mix8 below that. Its one caller is flashnext cuda: prompt passes with the same bits and less work, and a prompt scratch sized by use #1764's readoutNative, so the call site does not change. Glue.hcDenseMixVec is the one launcher, and the tests use it too.
• Contract: contract_hc_mix.zig registers the vector mix and its group sums as replacements of fn_hc_dense_mix8, with a swapped-input control (Registrations; the existing four mix entries are unchanged).
• Records: flashnext_hc_mix_vec_records.zig (nvcc 13.0, sm_80/86/89/120/121), in all_cuda.
• Tests: tf-cuda-test flashnext-hc-mix, flashnext-hc-mix-contract, flashnext-hc-mix-bench [ROWS]; host step test-flashnext-hc-mix-host (stress witnesses and records); a host test of mixVec's threshold.

Gate (on 92e099761)

• Bytes: flashnext-hc-mix passes at 1, 64, 65, 511, 512, 513, 1,536 and 4,218 rows, and at 65 rows with NaN, infinite and subnormal plants: dense_mix8's bytes, five repeats, solo rows, guards and unchanged inputs.
• Contract: SUMMARY hc_mix_vectors: 2 pass, 0 fail; 1 controls red, 0 missed. Ten shapes are bit-equal; the group sums are exact at k 4 where FP16 or BF16 sums would miss.
• HC glue: flashnext-hc passes (writebacks, the read-out chain, the n-gram gate and conv, stream adds).
• Resources: kernel-contract-limits with nvcc 13.0 on sm_80/86/89/120/121: fn_hc_dense_mix_vec is 32/31/31/29/29 registers, 2,048 static shared bytes, bound 256 (record current). The run's other 226 problems are stale records that main ef9fa700a has too: the same 226 lines on both trees, none of them new (flashnext cuda: prompt passes with the same bits and less work, and a prompt scratch sized by use #1764's fn_hc_writeback_norm_moe on sm_120/121 is one).
• Host: zig build test test-kernel-contract test-golden fuzz-cortheon and compile-cuda-tests test-cuda-host test-flashnext-hc-mix-host pass; zig fmt clean; lean 424 problems, 424 on main, 0 new.
• Not run: no served model run on this head. The kernel's bytes equal dense_mix8's at every row count the pass uses, so served tokens cannot change. Other GPUs and later compiler sets are not run.

[laudney]

@ashhart This one sits on #1847, which waits for #1764. I'll rebase it after #1847 is rebased onto #1764, and A/B what is left against main.

[ashhart]

Thanks @laudney. This sits on #1846/#1847, and we took #1764 for that tile and fusion instead (it's in the landing queue now), so this stack's base won't land as it stands. Once #1764 is on main, please restack this one operator commit onto main and re-run its receipts on that head. We'll review it standalone then.

[laudney]

@ashhart #1764 is on main, so this PR is now one commit (92e099761) on main ef9fa700a, with no dependence on #1847. I force-pushed the branch, and I updated the description with the new receipts.

Rows	fn_hc_dense_mix8	Vector mix	Bytes
512	0.0575 ms	0.0401 ms	equal
1,536	0.3208 ms	0.3157 ms	equal
4,218	0.8785 ms	0.8684 ms	equal

• Port: Glue.hcDenseMix uses the vector kernel from 512 rows of the 2,560 × 4 shape. Its one caller is flashnext cuda: prompt passes with the same bits and less work, and a prompt scratch sized by use #1764's readoutNative, so that code does not change. The old version selected the kernel in Flash Next HC prompts: fuse writeback and normalization #1847's cuda_hc_fused.zig, which is not on main. I removed Flash Next HC prompts: fuse writeback and normalization #1847's checkpoint chain fixture with it.
• Gates on this head (GB10): bytes at 8 row counts plus non-finite plants, SUMMARY hc_mix_vectors: 2 pass, 0 fail; 1 controls red, 0 missed, flashnext-hc pass. The nvcc 13.0 records are current on sm_80/86/89/120/121, and the 226 stale records are the same 226 that main has. The host CI set passes, and lean gives 0 new problems.
• Size: the gain is small, about 0.5 ms in a 1,536-row chunk. No served speedup is claimed, and I did not run a served model on this head.

On #1847: I posted a kernel A/B against #1764's fuse there. Nothing in it is faster on a real pass, so I suggest that we close it.

[laudney]

@ashhart The red zig check was the lean check, and the cause was mine. main then had flashnext_hc.cu at exactly 600 lines, and this PR adds 2. On current main (f23603801) that file is at 591, but registry.zig is at 600, and this PR added a line there too.

27cdaf151 (a new commit on top, no force-push) removes that line, the test-block import of flashnext_hc_mix_vec_records.zig. No coverage is lost: all_cuda still pulls in the records, and test-flashnext-hc-mix-host, which test-cuda-host runs, runs that file's test.

Check on this PR merged into f23603801	Result
lean	425 problems, 425 on the base, 0 new
zig build test-flashnext-hc-mix-host	8/8 tests pass, the records test among them
zig fmt --check	clean

[ashhart]

Thanks @laudney. This is exactly the standalone version we asked for.

I read it on current main. The vector mix does dense_mix8's arithmetic in the same order (multiply, round, four-stream sum, divide by 4, round), and each 32-column group sum stays in ascending order. Only the loads are paired, and it's used from 512 rows of the 2,560 × 4 shape.

Two small things:

• please squash the lean follow-up into the operator commit;
• we'll run the mix tests and a served Flash Next A/B against main on our GB10 (byte-equal replies, no slower cell) before landing, since the default 1,536-row chunk takes the new kernel.

[laudney]

@ashhart Squashed, as you asked. The PR is one commit again, 1bf5be4c1, and its tree is identical to 27cdaf151, so the code did not change. The commit message now also says that the records' test runs from test-flashnext-hc-mix-host.

Check on this PR merged into main 0d43416f9	Result
merge	clean
lean	424 problems, 424 on the base, 0 new

Thanks for running the mix tests and the served A/B on your GB10.

[ashhart]

GB10 A/B of this head (tree 994008062) against its base, Flash Next FP4 on one GB10, warm then base-new-new-base:

• Exact: 0 of 20 prompt rows differ, and 64K long context passes on both arms.
• Decode: level at every width. Chat at temperature 0: 43.3 → 43.4, 81.7 → 81.5, 140.9 → 141.0 and 218.0 → 218.0 at 1/2/4/8 streams. Code: 69.0 → 69.0 … 331.6 → 331.3.
• Prompt reading is slightly slower: 8K −0.4% and 16K −0.5% to −0.7%. At 16K both new runs are below both base runs in all four cells (for example 1913.4 / 1916.2 against 1921.8 / 1925.1 tok/s).

So on this checkpoint the paired-load mix doesn't buy prompt time, and it costs a little. Your numbers were on the BF16 checkpoint, if I read them right. Could you check whether the gain is BF16-only, and if so take the new mix only where it wins (by format, or by row count if it's the 512-row and up path)? Then we'll A/B again on both. Until then this stays open: a small steady prefill loss is still a loss for us.

[laudney]

@ashhart Thanks for the A/B. The gain is not BF16-only, and no gate would keep it, so I suggest we close this PR. If you agree, I will close it.

Where	What it shows
My numbers	A kernel bench on synthetic BF16 HC rows, not a served checkpoint
Format	The mix reads the BF16 residual streams, which are the same for every weight format, so a format gate has nothing to select
1,536 rows (the default chunk)	The bench saved 0.005 ms of 0.32 ms (1.6%), within noise
512 rows	0.017 ms a launch; a served pass reaches that size only in short tail chunks
Your served A/B	Prompt reading 0.4–0.7% slower at 8K/16K, decode level

A row-count gate would add a special case for a few microseconds in tail chunks, and it would not remove the loss at 1,536 rows. A likely cause is that the vector kernel's five blocks a row share the SMs with the side-stream shared expert in a real pass, which the isolated bench does not show.

[laudney]

Closing, as offered above: the served A/B shows no prompt gain on this checkpoint, and the mix has no format or row range that would keep a gain.