[ashhart]

Draft: not ready to merge. See "Before this merges" below.

What this changes

GLM-5.3-Flash on CUDA verifies windows of up to 32 rows (was 8) and chains up to 15 MTP drafts (was 3), priced by rounds timed on the machine at load rather than by a fixed table or no table. Load timing also races the classic NVFP4 expert kernel against the existing factored one and keeps the faster, which gives the same bits. A window's greedy picks take one gather and one read, and a step's short ids travel in its first broadcast.

It also fixes two faults that wider windows expose:

• Buffers.make sized a shared window's KDA projection buffer only when seg < rows. At 32-row windows seg == rows, so the buffer shrank to 4 bytes and overwrote its neighbours, and replies at two or more streams went wrong. A new host test fails on the old condition.
• TF_GLM_KV_HOST's row cache kept its list across a call past 16 rows, which rewrites rows the list may name. It now forgets the list. Found by reading the kernels: I have not seen this one fail (see below).

No GPU kernel changes. This could split into the pick and step batching, then the windows and prices, if that reviews better.

• Issue: Refs glm cuda (native): copy drafts are missing, and the 48% acceptance the Python lane gets from them is what keeps it ahead #1910 (copies gain these wider windows and timed prices; the rest has no issue of its own).
• Stacked on: main.
• atc overlap --json (2026-11-05 13:53 UTC, base cde4fe7d99): no conflicts among those it could test (10 clean), but 450 unknown, because the board reports invalidPulls and hit its pull request and time limits. So I also ran git merge-tree against 14 open GLM pull requests on an earlier amend of this head (same base): glm cuda: TF_KV_SHARE keeps GLM's prompts in the shared prompt store, and GLM's own store goes #2211 (cuda_native.zig), GLM-5.3-Flash on CUDA keeps conversation states on disk, every rank's part (--learn-conversations) #1411 (cuda_follow.zig), glm: TF_GLM_KV_LIVE_HALF=1 keeps the live half of the KDA state alone in a kept prompt state #2181 (the recipe) and glm cuda: a prompt step takes two chunks while no other stream decodes, each one's exchanges under the other's sites #1423 (the recipe, cuda_engine.zig, cuda_state.zig) conflict in files this changes; glm cuda: TF_GLM_L2PF reads a decode window's next weights into L2 on a side stream, on by default on a GB10 #2162 and CUDA: GLM's KDA chain and saved-row replay become a shared module #2142 conflict only in files it does not touch; the rest are clean. These resolve by landing order.

Before this merges

• MTP at 2 or more streams on this head. Device runs before the last change had small dips against main (table below); window_costs_lone keeps shared MTP rounds on main's depth rule to remove them. Not yet rechecked on the cards.
• TP=2 on two GB10s, main against this head: this changes their default path too. Queued, no result yet.
• Device exactness and speed on the final rebased head. The numbers below are from before the rebase.
• Show the TF_GLM_KV_HOST fault failing, or say plainly that it rests on reading the kernels. The control without the fix passed on 5K to 7K token prompts.
• Add the changelog line once there is a pull request number to cite.

Checks on this head

• Head commit: 7f20e7bef6fbb0c08d2a220bb0f17bb6704ee92d, on main cde4fe7d99
• Lean checker on the base and on this head: 425 on the base, 425 on this head, 0 new. zig fmt --check and git diff --check pass. gate.py (secrets, private data) passes.
• Tests run, with the command and pass, fail and skip counts: zig build test test-golden test-cuda-host test-glm-copies-host test-family-copies-host test-cuda-startup-costs-host on x86_64 Linux: 552 of 555 steps, one failing test (listener: accepted IPv4 and IPv6 TCP streams disable Nagle), which fails the same way on main on this host, where IPv6 loopback is blocked. Full counts on main at 9200a3064: 7052 of 7112 passed, 59 skipped, 1 failed. The local landing gate (skill scan, gitleaks, whitespace, fmt, contract-host, HIP, dev tools, Linux CUDA cross-compile, release paths) passed every step but lean on the amend before this one; lean is fixed here and the gate was not rerun.
• CI: not read yet (draft just opened).
• Not run, and why: Windows and macOS zig build test are left to CI, because on macOS it runs Metal device checks that must not share a GPU with other runs here.

Receipt

Measured on base 9200a3064 (rebased since), comparing that main against this change.

• Environment: Zig 0.17.0, CUDA 13.4, NCCL 2.32.3, four RTX PRO 6000 Blackwell Max-Q (sm_120, 300 W) in one Threadripper PRO host, TP=4. Checkpoint nvidia/GLM-5.3-Flash-NVFP4 at 09b04e5e; drafter incoai/GLM-5.3-Flash-DFlash2 at bf582e4e. --context 245760, NCCL_P2P_LEVEL=SYS NCCL_MIN_NCHANNELS=4 NCCL_MAX_NCHANNELS=4 TF_GLM_PREFILL_REDUCE=gather on both sides.
• Paths and platforms: the default GLM CUDA drafted path, with MTP and DFlash2, on sm_120 at TP=4, which I ran. Not run yet: GB10 (TP=2), other rank counts, EXL3 and csf0 checkpoints. Metal, HIP and other families only gain an unused Config field and a moved constant. TF_GLM_COPIES=1 and TF_GLM_KV_HOST=1 were run on the DFlash2 path only.
• Exactness: tools/bench_concurrent.py --alone --serial at temperatures 0 and 1, levels 1, 4, 8 and 16: 0 unequal and 0 failed in every cell, both drafters. Separately, drafted against "draft": false on repeating text, code, counting, an edit, prose, JSON schema output and tool calls, windows to about 31 rows: all equal at temperatures 0 and 1, with TF_GLM_COPIES off and on.
• Decode speed before and after, from tools/bench_openai.py (512 tokens) and tools/bench_concurrent.py (256 tokens), tok/s. DFlash2's after column uses TF_GLM_DRAFT_CONFIDENCE=0.4, and before keeps the default 0.7; after at 0.7 was 298.7 code, 190.5 chat.

Drafter	Case	main	this change
MTP	Code, one stream, greedy	235.8	291.8
MTP	Chat, one stream, greedy	146.4	190.3
MTP	Code, one stream, T=1	191.1	219.0
MTP	Chat, one stream, T=1	146.5	190.7
DFlash2	Code, one stream, greedy	254.3	304.8
DFlash2	Chat, one stream, greedy	167.0	202.2
DFlash2	Code, one stream, T=1	187.7	215.2
DFlash2	Chat, one stream, T=1	158.3	189.1
DFlash2	Code, C=4, greedy	564.2	639.9
DFlash2	Chat, C=4, greedy	455.8	520.4
DFlash2	Code, C=4, T=1	353.6	441.1
DFlash2	Chat, C=4, T=1	301.1	333.3
DFlash2	Code, C=8, eight prompts, greedy	473.7	542.8
DFlash2	Prose, C=8, eight prompts, greedy	392.7	429.7
DFlash2	Code, C=16, eight prompts, greedy	649.2	685.4
DFlash2	Prose, C=16, eight prompts, greedy	526.0	583.6
DFlash2	Code, C=16, eight prompts, T=1	502.3	558.3
DFlash2	Prose, C=16, eight prompts, T=1	463.1	477.7

MTP at several streams, before window_costs_lone (to recheck). The dips are in bold, with tight repeats, so they look real:

Case	main	before the fix
Code, C=4, greedy	502.8	547.2
Chat, C=4, greedy	409.9	467.4
Code, C=4, T=1	396.3	384.7
Code, C=8, eight prompts, greedy	487.3	478.0
Prose, C=8, eight prompts, greedy	391.5	403.7
Code, C=8, same prompt, greedy	657.6	639.3
Code, C=8, same prompt, T=1	497.5	475.9
Code, C=16, eight prompts, greedy	559.7	618.9
Prose, C=16, eight prompts, greedy	539.2	564.3
Code, C=16, same prompt, T=1	540.8	529.6

• Prompt speed before and after, cold, tools/bench_cache.py --require-cold, tok/s at 8K, 32K and 131K: DFlash2 4388, 4457, 4434 against 4383, 4454, 4432; MTP 4431, 4476, 4446 against 4414, 4463, 4433, and 4374, 4419, 4392 in a later run on warmer cards.
• Load: a new build's first load took about 2 s longer (85.1 s against 83.5 s); later loads read the kept table (83.0 s). Timed on these cards: 9.92 ms at 1 row, 11.25 at 4, 26.02 at 32, a block draft 0.20 ms, a head step 0.40 ms.
• Memory: about 90 MiB more a rank and 26 MiB more a stream at TP=4, and 54 MiB more per sequence at TP=2 (cuda_kv8_test), about 300 MiB less room overall at --context 245760.
• Licence of any new weights or drafter: n/a. DFlash2 is the existing optional drafter (CC BY-NC-ND 4.0), and the MTP rows don't use it.

Checklist

• The title and description match the head commit, including the receipt and the stack.
• New tests fail before the change, pass after it, and skip cleanly without their dependency.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths in the diff or its commits. No AI attribution lines.
• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (Drafted and concurrent yes, before the rebase. Resumed against fresh not checked separately.)
• No precision traded for speed. No sums change: the factored and classic expert forms are the existing bit-identical pair.
• Decode and prompt processing are no slower than main at one stream and the widest admitted; with --drafter, no slower than without it, greedy and sampled. (Open: the MTP recheck, and sampled code with DFlash2 is slower than MTP at one stream, 215.2 against 219.0.)