[jschmied]

Found while measuring the draft head for #1870. forwardHead writes the MTP head's cache for every row of a call, but only the call's last row reaches d.out. So the attention and its write-back stay over all rows, and the experts, their write-back and the copy out now run on the last row alone. It's moved to row 0, as the scores path already does. Prompt chunks and absorbed rounds no longer run the head's MoE on rows nothing reads.

A/B (GB10, nvidia/Qwen3.8-Flash-Next-FP4-Local, --parallel 1 --context 40960 --prompt-cache-gib 0; fresh servers in the order main e8b3639a7, branch, branch, main; two time-to-first-token readings per cell):

prompt	main	this branch	change
5,996 tokens	3.013, 3.022, 3.046, 3.021 s	2.999, 2.993, 2.994, 2.994 s	−0.75 to −1.0 %
23,836 tokens	12.401, 12.393, 12.408, 12.469 s	12.333, 12.320, 12.331, 12.353 s	−0.5 to −0.7 %

Every branch reading is below every main reading. The range's first number uses the first main run only, since the last main run may have overlapped a build on the box.

Six greedy replies (temperature 0, MTP drafting on) hash the same in all four runs, and so do their drafted and accepted counts (for example 24 rounds, 127 drafted, 103 accepted for the first).

Overlap: #1786 and #1866 move this loop into cuda_mtp_pieces.zig (runPiece, captured as CUDA graphs keyed by row count). Whichever lands second needs a rebase. If #1786 goes first, I'll move the change into runPiece, with its graph key taking the experts' single row.

Checks (GB10, on main e8b3639a7): lean 0 new, zig build test and the CI set pass, git diff --check clean.

[ashhart]

Thanks @jschmied, a neat find. Reviewed: only the call's last row reaches d.out, the head's cache is still written for every row through the attention and its write-back, and the experts, their write-back and the copy out now run once, on that last row. Our GB10 run (FP4 checkpoint, main against this head, balanced order) is GREEN: 0 of 20 prompt rows differ across runs, prompts +1.9% to +3.4% at 2K, decode level (36.6 to 42.5 tok/s on both sides). It's in the landing queue. On the overlap: #1786 and #1866 aren't landing yet, so this goes first and they rebase onto it.

[ashhart]

Thank you @jschmied. This is merged into main with your commits under your own name, so you're listed as a TensorFold contributor. It ships in the next release, credited to you in the changelog and the release notes.

[ashhart]

Landed as ad98b60. Thanks @jschmied!

[ashhart]

Thanks @jschmied! This is in TensorFold 1.2.2, out now: https://github.com/ashhart/TensorFold/releases/tag/v1.2.2