[russellmm]

Part 3 of 3. This is the last of three changes that let GLM-5.3-Flash on CUDA use system memory for what the
device doesn't need resident, so two 96 GB cards serve the model's full 1,048,576-token context:

Part	What moves to host memory	Status
1. #1736 TF_GLM_HOST_CACHE	kept prompt states (the prompt cache)	merged
2. #1760 TF_GLM_VISION_HOST	the vision tower's weights, staged per unit	merged
3. this PR, TF_GLM_KV_HOST	the KV pool's latent rows

With all three on, the checkpoint's weights stay on the cards and the KV window goes from 512K (with a 1.4 GiB reserve,
1.5 GiB free) to 1M (with a 2.6 GiB reserve, 5.7 GiB free).

What this changes

GLM-5.3-Flash's KV pool costs every rank 13,056 bytes a token: for each of the 11 DSA layers and the MTP layer, a
1,024-byte latent row and 64 bytes of pooled index key. The sparse attention reads only the 2,048 tokens the indexer
picks per query row (512 pools of 4), and the indexer scans only the pooled keys. So the latent rows, 94% of the pool,
are read a few thousand at a time and can live off the card.

With TF_GLM_KV_HOST=1 (on every rank, with KV pages and BF16 KV):

• The pool's pages are pinned host memory, mapped into each device's address space. The pooled index keys go to
  device pages of their own (Caches.pks, their own page stride), so the indexer never leaves the card.
• Each sparse call gathers the rows it picked, once each, into device slots (tf_glm_kvh_mark, _copy) and rewrites
  its token lists to name the slots (_remap, same order); the existing attention kernels then run on the slots with
  no page table. Dense calls (contexts under ~2K) gather their visible rows in order (_rows).
• Decode and verify calls (up to 16 rows) keep each layer's last gathered rows on the device (8,192 rows a layer,
  128 MiB in all) and copy only the rows they don't have. Rows the call itself writes (a rejected draft's) never hit,
  and the cache is cleared whenever pages change hands.
• The plan counts a page's device cost as its pooled keys alone, 768 bytes a token. The gather's buffers (1 KiB per
  window token, plus row marks) are made before the ranks read their room, so the budget counts them.
• TF_GLM_KV_HOST_GIB sizes the host pages (one full window unasked). TF_GLM_KV_HOST_SHARED=1 gives the ranks of one
  host a single copy in /dev/shm: their latent rows are bit-identical, both write them, and the first KV write of a
  step comes after the step's first three all-reduces, so no rank reads a page another has already reused.
• KV pages off, or --kv-cache fp8 (which doesn't page), is refused at startup with the reason. Unset, nothing changes.

Fixes #1794. The prompt caches work unchanged on top (#1736's host cache, and #1732's TF_GLM_KV_SHARE, whose kept
states then sit in host memory as pool pages): the snapshot plan copies each page part with its own stride.

Overlap: #1779 moves attend's cache writes into writeKeys, where this PR gives the pooled-key write its own page
table, so the two conflict in that one hunk. Happy to rebase whichever lands second. The others atc overlap names
(#1411, #1423, #1545, #1576, #1577) merge cleanly.

Modules:

• cuda_kv_host.zig (new): the setting, host page count, the gather and its row cache, the shared pages.
• glm_ops.cu: the gather's kernels (mark, copy, copy_cached, remap, rows, prev_mark, unmark, keep, reset).
• cuda_dsa_block.zig: attend gathers before the dense and sparse attention when the rows are on the host.
• cuda_pages.zig, cuda_state.zig, cuda_snapshot_layout.zig, cuda_native.zig: where the pages and the pooled
  keys live, the plan's page cost, the refusal.

No attention kernel or arithmetic change.

Why this design

Measured first, on this checkpoint with a recording kernel captured in the CUDA graphs (not part of this PR):

• Picks spread over the whole context. At 459K, 45% of picked rows were more than 256K back (an even spread gives
  44%). A recent-tokens window on the card alone would leave most reads on the host.
• Consecutive steps overlap. About half of a decode step's rows were picked by the step before; the row cache
  keeps those.
• Prompt chunks touch most old rows, so they gather the union of their picks once per layer and skip the cache.

The gather copies each needed row across PCIe once a call (1 KiB rows, one warp each), rather than letting the
attention kernels read host memory directly; a first version that read the dense pass's rows directly was ~12 s slower
per prompt.

Benefits and trade-offs

Benefits:

• Device memory. The KV pool costs the card 768 bytes a token instead of 13,056. On two 96 GB cards: a 1M window
  with 5.7 GiB free on rank 0, where main fits 512K only with the reserve cut to 1.4 GiB (1.5 GiB free).
• Same answers. Nothing in the arithmetic moves; every receipt below is bit-identical to main.
• Short and mid-length work is unaffected, and prompt processing is within ~1%.

Trade-offs:

• Decode deep in a context is a few percent slower, from copying picked rows across PCIe: -2.7% after a 204,667-token prompt at 300 W.
  It grows with depth and with GPU clocks (the copy doesn't speed up with the GPU): at 400 W the same code measured
  -3.7% at 205K before rebasing.
• Host memory. One window's pages are pinned: 13.2 GB for 1M (once per host with TF_GLM_KV_HOST_SHARED=1, else per
  rank), on top of the prompt cache's.
• Device scratch. The gather needs 1 KiB per window token on each rank (1 GiB at 1M) plus a 128 MiB row cache.
• Scope. CUDA, BF16 KV on pages only; the shared copy needs every rank on one host.

Receipt

• Environment.

  • TensorFold main 1157ea5 with and without this commit, built with Zig 0.17.0 and nvcc 13.4.92 (-Dsm=120).
    The receipts' binary was built from this commit before its last amend, which changed two comments only.
  • Two RTX PRO 6000 Blackwell Workstation cards (96 GB each, PCIe Gen5 x16), TP=2 on one host, NCCL 2.32.3,
    cuBLASLt 13.4, driver 615.71.09, Linux 7.3. Host: Threadripper 9970X, 128 GB DDR5.
  • Checkpoint Mia-AiLab/GLM-5.3-Flash-EXL3-4bpw-TensorFold @ 76c0b517, --chat-template: the checkpoint's own with
    zai-org/GLM-5.3-Flash's media branch (as in glm cuda: TF_GLM_VISION_HOST keeps the vision tower's weights in pinned host memory #1760).
  • Every profile: --vision --kv-cache bf16 --parallel 2, TF_GLM_HOST_CACHE=1 --prompt-cache-gib 24,
    TF_GLM_VISION_HOST=1, KV pages on (TP=2's default).
    • main 512K: --context 524288, TENSORFOLD_MEMORY_RESERVE_GIB=1.38 (main's largest window with ≥1.5 GB free).
    • candidate 512K: the same with TF_GLM_KV_HOST=1.
    • candidate 1M: --context 1048576, reserve 2.6, TF_GLM_KV_HOST=1 TF_GLM_KV_HOST_SHARED=1.
  • GPU cap 300 W for every number below, both sides alike.

• Exactness (all three profiles):

  • tools/bench_concurrent.py --levels 1,2 --tokens 256 --temperatures 0 --alone --serial, code and chat: alone 18/18
    and serial 4/4 equal on each; 0 unequal, 0 failed.
  • Greedy 256-token replies to fixed documents of 1,693, 13,374, 61,558 and 204,206 tokens: identical SHA-256 on all
    three. The 61,558- and 204,206-token documents sent at once (two streams sharing the gather) reply as alone.
  • A 521,040-token document with 8 planted facts and 2 two-step questions: all 10 answers identical on all three.
  • Greedy 120-token replies to five test images (640x400 PNG, JPEG, WebP; 1920x1080 and 2560x1440 PNG): identical.
  • With TF_GLM_KV_SHARE=8 instead of TF_GLM_HOST_CACHE (kept states as pool pages, so in host memory too): a fixed
    four-turn conversation resumes 57,344 tokens a turn with replies identical to computing each turn fresh, a
    100,063-token conversation resumes in 0.7 s, and the fixed documents reply as on main.

• Memory (rank 0, idle after startup, driver memory.free):

  	main 512K	candidate 512K	candidate 1M
  startup	1034 pages, 6.44 GiB of KV on the card	1041 host pages (6.6 GiB), 390 MiB pooled keys	2065 shared host pages (13.2 GiB), 774 MiB pooled keys
  free on rank 0	1,531 MiB	7,019 MiB	5,787 MiB

  A 1,007,082-token prompt with a 512-token reply served on candidate 1M: rank 0's free memory never fell below
  5,785 MiB (sampled every second); 43 GB of host memory stayed available.

• Speed (fresh server each run, main and candidates alternating, two rounds; medians):

  	main 512K	candidate 512K	candidate 1M
  decode, code (bench_openai, 256 tokens)	223.2 tok/s	222.8 (-0.2%)	223.0 (-0.1%)
  decode, chat	130.3	130.1 (-0.2%)	130.2 (-0.1%)
  bench_concurrent 2 streams, code / chat	335.2 / 215.0	333.7 / 213.7 (-0.5% / -0.6%)	334.3 / 214.0 (-0.3% / -0.5%)
  cold 32K / 64K ttft_s (bench_cache --require-cold)	10.08 / 21.12 s	10.16 / 21.33 (+0.7% / +1.0%)	10.17 / 21.30 (+0.8% / +0.9%)
  a 204,667-token document: first token	69.35 s	70.60 (+1.8%)	70.65 (+1.9%)
  then 512 greedy tokens	155.3 tok/s	151.2 (-2.7%)	151.3 (-2.6%)

  The same token_sha in every bench_* cell and the same reply SHA-256 for the 204,667-token document in all six
  runs.

• Tests run.

  • zig build test --test-timeout 10m: 5,941/5,968 passed, 27 skipped, 0 failed (includes the two new tests in
    cuda_kv_host.zig: host page counts and the shared file's name).
  • test-kernel-contract: 144/144. test-golden: pass. test-server-security test-pull-security: 28/28.
    fuzz-cortheon: 19/19. Plain zig build: pass. build_options_test.py: pass. git diff --check: clean.
  • lean_new.py against main: 424 problems on the base, 424 on the head, 0 new.

• Not run, and why.

  • Metal, HIP, and GB10 or other unified-memory hosts: the change is CUDA-only, and on unified memory the pages are
    host memory already.
  • TP=3 and multi-host TP (the shared copy needs one host; per-rank copies should work but weren't measured),
    --kv-cache fp8 (refused), sm_80-sm_89 cards: not available.

Checklist

• Every token still goes through the lane rounds. Only where the latent rows live and how a call reaches them
  changes.
• Drafted output equals "draft": false, concurrent equals solo, and resumed prompts equal fresh (above).
• No precision traded for speed: the gathered rows are the same bytes, attended in the same order.
• New kernels are copies and index bookkeeping only; no attention or GEMM kernel changed.
• Prompt processing within ~1% of main (cold 32K/64K above); deep-context decode slower, disclosed above.
• CUDA TP=2 on sm_120 only; the rest is not run (above).
• The new tests need no device and fail before the change, which they don't compile without.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines.

[russellmm]

Two new commits since the receipt (no force-push):

• 9b7f8edf7: with TF_GLM_KV_HOST_SHARED=1, the page file kept its name until rank 0 closed it, so a rank stopped by
  a signal or a crash left the window's pages in /dev/shm. A count past the pages now records each rank's mapping,
  and the rank that completes it unlinks the file; the mappings stay valid. On TP=2 the file is gone from /dev/shm
  as soon as both ranks are up. New no-device test: lastToMap in cuda_kv_host.zig.
• 541e47b1a: merges main after Shared draft rounds price drafts first and skip proposals they won't verify (Nemotron, Qwen MTP, GLM, Gemma 4, LFM2; CUDA and Metal) #1779. attend's cache writes now live in writeKeys, so the pooled-key write
  takes its own page stride there (which also covers Shared draft rounds price drafts first and skip proposals they won't verify (Nemotron, Qwen MTP, GLM, Gemma 4, LFM2; CUDA and Metal) #1779's commit path); attend keeps the gathers.

Re-checked on the merged build (TP=2, 1M window, TF_GLM_KV_HOST_SHARED=1): the four fixed documents (1,693 to
204,206 tokens) and the 61,558 + 204,206 concurrent pair reply as before; bench_concurrent --alone --serial 18/18 and
4/4 equal, 0 unequal. zig build test: 5,344/5,360 passed, 16 skipped, 0 failed. lean_new.py against main: 0 new.
git diff --check: clean.

[russellmm]

73b2012f9 fixes the zig-windows failure in the last run: TF_GLM_KV_HOST_SHARED's /dev/shm code used POSIX open
flags and DynLib.close, which the Windows host-test builds can't compile. Off Linux, SharedPages is now a stub that
refuses with SharedMemoryUnavailable (each rank keeps its own pages); Linux is unchanged.

Checked locally:

• Every test-cuda-host program compiled with -target x86_64-windows-gnu --test-no-exec: glm_pages_host gives the
  run's exact error without the commit and compiles with it. Of the 116 programs, the only ones that don't compile for
  Windows are the Linux-only h3 checkpoint/cuda_lib and cli/pull tests, untouched here.
• The cuda_kv_host.zig tests and glm_pages_host run as Windows binaries under Wine: 17/17 and 12/12 passed.
• Linux, uncached: zig build test --test-timeout 10m 5,989/6,016 passed, 27 skipped, 0 failed; lean_new.py 0 new.

[ashhart]

Thanks @russellmm, the receipts are thorough and the trade-offs are stated plainly. One thing before it lands: could TF_GLM_KV_HOST and TF_GLM_KV_HOST_SHARED join cuda_ranks.zig's Fingerprint, as roce_multi does? A rank started without them would size a different pool from its peer, and _SHARED needs every rank to agree on the file. Then we'll check the unset path on our GB10s.

[russellmm]

1ff12c54a puts TF_GLM_KV_HOST and TF_GLM_KV_HOST_SHARED in cuda_ranks.zig's Settings and Fingerprint, as
roce_multi does (no force-push). kv_host is 0, 1 or 2 (2: shared), so ranks that disagree on either stop at startup
naming them. TF_GLM_KV_HOST_GIB joins too as kv_host_gib: each rank sizes its row marks from its own value while
rank 0 counts the pages. Settings refuses a value other than 0/1, _SHARED=1 without the base, and a _GIB that isn't
positive or comes without the base. Fingerprint layout 14; new test in cuda_ranks_test.zig.

Checked locally:

• TP=2, rank 0 with TF_GLM_KV_HOST=1, rank 1 without it: both stop about 2 s in, before loading weights, with
  REFUSE rank settings: rank 1's kv_host differs from rank 0's (TF_GLM_KV_HOST or TF_GLM_KV_HOST_SHARED).
• TP=2, 1M window, TF_GLM_KV_HOST_SHARED=1: the four fixed documents (1,693 to 204,206 tokens) reply the same as
  main without TF_GLM_KV_HOST.
• zig build test: 0 failed; lean_new.py against main: 0 new. The new tests also pass as Windows binaries under
  Wine (-target x86_64-windows-gnu).

[ashhart]

Thanks @russellmm, the fingerprint is just what we asked: kv_host and kv_host_gib agree across ranks, a mismatched rank stops before loading with the variable named, and the refusals are tested. Thanks also for the Windows stub and the unlink. Two things before it lands:

1. The contract: the nine tf_glm_kvh_* kernels in glm_ops.cu aren't registered. They're gathers and index maps, so a byte-exact reference is simple. Could each get its record and a planted control, such as a dropped row, a swapped slot or a stale mark, with the CUDA SUMMARY line? 2. The slot cap: Gather has capacity slots. When tf_glm_kvh_mark runs out of slots, the row keeps -2, and tf_glm_kvh_remap would write that into the token list. Could you say what bounds a call's distinct rows by capacity with several long streams in one call? If nothing does, please check the count after the mark and refuse or fall back by name.

The mbox still drops your merge of main, so we'll land from the branch. Then we'll run the unset path on our GB10s.

[russellmm]

0d28ea3ed answers both (no force-push).

1. The contract: zig/tests/cuda/contract_glm_kv_host.zig registers the nine tf_glm_kvh_* kernels, each launched as
   cuda_kv_host.zig launches it and checked exactly against the host's replay (.out = .raw, a stated rule, and a
   check): mark through each pick's slot back to its row with n, the flag and no row left at -2; copy,
   copy_cached, rows and keep byte for byte, NaN and infinities included; remap, reset, prev_mark and
   unmark id for id. Pages are out of order and the lists past their counts hold a row no call picks. Slot numbers follow
   atomicAdd's order, so mark's output is the pick-to-row map, which repeats byte for byte. Shapes: decode, verify and
   2,048-row prompt chunks at 2,051 picks a row. Each has a planted control, red at shapes: a stale mark (mark, and
   prev_mark and unmark keeping a row they must clear), a dropped row (copy, reset, keep) or swapped slots
   (remap, copy_cached, rows' table). Their launches are in glm_cuda_records.zig from nvcc 13.0 (V13.0.88) for
   sm_80/86/89/120/121; the GLM records already there come out current with the same toolchain.

CUDA SUMMARY (RTX PRO 6000, sm_120, nvcc 13.4 build):

SUMMARY 517 kernels: 517 pass gates 1-4, 0 fail, 6 known findings; 349 controls: 348 red as planted, 1 missed; 424 limit problems; gates 5710708 ms, limits 1061712 ms

All nine kernels and their nine controls are in it, and their limits lines match the records (PASS limits live glm_ops tf_glm_kvh_*). The miss is nvfp4_expert_waves<2, 2, 4>'s dropped-items control, and the limit problems are other
images built here with nvcc 13.4 against 13.0 records; this commit touches neither.

2. The slot cap: nothing lets one call's distinct rows pass capacity. A gather never spans streams: attend runs once
   per sequence (cuda_shared.zig:83 loops the segments; one sequence's path goes straight to it), and its gather marks,
   copies, remaps and resets before the next sequence's starts. Its picks are that sequence's rows: tf_glm_dsa_tokens
   gives each query row q pools it sees and its tail rows <= q, so every pick is a position below pos + rows, and each
   position is one arena row through the page table, so the distinct rows are at most pos + rows. That is within the
   sequence's capacity: a prompt chunk's rows are prompt positions, a verify window adds at most window_rows past them,
   and capacity() holds the context plus a verify window (cuda_native.zig:138), with prompts refused before that
   (cuda_lanes.zig:283, cuda_engine.zig:327). Every sequence's state is sized to e.capacity, and Gather.cap is
   e.capacity (cuda_native.zig:309, cuda_pages.zig:83).

The guard anyway: a row past the slots is now set back to -1 rather than left at -2 (before, reset never cleared it,
so it would have stayed marked into later calls), and mark sets a flag in mapped host memory. Engine.checkGathers,
which every draw already calls after its synchronize, refuses the request with error.GlmKvHostSlots naming the slots,
then clears the flag. No host read inside the step, so capture is unchanged. The contract covers it: mark at half the
slots its picks need must flag the call and leave no row at -2.

Also checked: served on two RTX PRO 6000s at 1M context with TF_GLM_KV_HOST=1, the four fixed prompts (1,693 to
204,206 tokens, 256-token replies) give the same reply SHAs as main without TF_GLM_KV_HOST, and the guard never fired;
zig build test: 451/451 steps, 387/392 tests passed (5 skipped); lean_new.py: 0 new; the changed host code compiled for
x86_64-windows-gnu.

[ashhart]

Thanks @russellmm, commit 5's title is just what we asked for. Could you add the CUDA kernel-contract SUMMARY line for the nine tf_glm_kvh_* kernels and their planted controls to the body, plus the test that shows a call past the slots is refused by name? We'll run the unset path on a GB10 for byte-identical replies once that's in. One heads-up for the rebase: #1423 has taken fingerprint layout 15, so this takes the next free one when it lands. Main is frozen for the 1.2.1 release right now, so this lands after it.

[russellmm]

Thanks @ashhart. d065689de adds both, as a new commit rather than an amended commit 5: docs/recipes/SKILL.md asks
that a branch under review isn't force-pushed, since you rebase at landing. Its body has the SUMMARY line and the
test; please fold it into commit 5 when you rebase if you'd like them there.

The refusal by name:

• tf-cuda-test glm-kv-host-slots (one GPU) runs the real Gather.gather (mark, copy, remap, reset) with 256 distinct
  rows for 64 slots, then kv_host.checkActive(), which Engine.checkGathers now calls. The call is refused as
  error.GlmKvHostSlots with the log line naming the slots, no row is left marked, and the flag is cleared: a next
  call of 48 distinct rows is accepted and each token's slot holds its row's bytes.

  error: kv host: a call picked more distinct rows than the gather's 64 slots
  PASS kv host slots: 256 distinct rows for 64 slots refused as error.GlmKvHostSlots, no row left marked
  PASS kv host slots: then 48 distinct rows for 64 slots accepted, each token's slot holding its row
  

• kv host slots: mark's flag is seen once, then cleared for the next call in cuda_kv_host.zig is the host part,
  so CI runs it.

The SUMMARY line for the nine kernels (ten cases with the past-its-slots one) and their controls, on an RTX PRO 6000 (sm_120, nvcc 13.4 build):

SUMMARY 10 kernels: 10 pass gates 1-4, 0 fail, 0 known findings; 9 controls: 9 red as planted, 0 missed; 0 limit problems; gates 405 ms, limits 0 ms

It comes from the same kernel-contract code with the kernels and the limit records narrowed to tf_glm_kvh_* by a
local name filter, which isn't in the PR. The full run is in my last comment.

Layout: noted. When #1423 lands with 15 I'll merge main and take 16 (or leave it to your rebase, as you prefer).

Also checked: served at 1M context with TF_GLM_KV_HOST=1 on two RTX PRO 6000s, the four fixed prompts still give
main's reply SHAs without it; zig build test: 458/458 steps; lean_new.py: 0 new; the changed host code compiled for x86_64-windows-gnu.

[ashhart]

Thanks @russellmm, d065689de is just what we asked for: the SUMMARY line and the refusal test are both in the body now.

We went to land it, but it no longer rebases cleanly onto current main. Main's 158a297c8 moved the latent and index-key writes in glm/cuda_dsa_block.zig's attend into a new writeKeys(). Your first commit changes those same writes (the tk table with c.pks, and the host-row path for dense attention). We'd rather not guess at that merge in a path the default build also runs.

Could you rebase onto current main and carry the tk change into writeKeys()? A new commit on top is fine, no force-push needed. There's also a small docs conflict in docs/recipes/glm-5.3-flash-cuda.md, where main added the TF_GLM_CHUNK_SOLO and TF_GLM_DENSE_BITS lines next to yours (keep both). Once it's pushed, we'll gate it and land it straight away.

[russellmm]

Thanks @ashhart. The branch already carries the tk change into writeKeys(): 541e47b1a merged main's 158a297
and moved it there, and the PR merges with current main without a conflict. What a rebase loses is that merge's
resolution. Commit 1 is replayed against a main that already has writeKeys(), so it conflicts, and the later
fingerprint commit conflicts with solo in cuda_ranks.zig the same way.

So I've done the rebase here, with no force-push: glm-kv-host-rebased
on the fork is the six commits replayed one by one onto 04ee307ae, ready to land as they are:

• Commit 1 writes the pooled keys in writeKeys() through tk (c.pks's own page stride when it's set, c.ps
  otherwise). attend keeps the dense path's host rows (g.visible) and the sparse path's tk selection and gather.
• The recipe keeps both: TF_GLM_KV_HOST and _GIB/_SHARED, then main's TF_GLM_CHUNK_SOLO and TF_GLM_DENSE_BITS.
• Commit 3's fingerprint has kv_host and kv_host_gib beside main's solo, layout 15. It takes 16 if glm cuda: a prompt step takes two chunks while no other stream decodes, each one's exchanges under the other's sites #1423 lands
  first.

Main has been moving quickly under this PR, which is good to see: writeKeys(), TF_GLM_CHUNK_SOLO taking the
fingerprint layout and the recipe's new lines all landed while it was in review, and each one reached the same few lines
this change touches. That's what has made the last couple of rounds about merging rather than the code itself. As a linear
series on today's main it should land as it is, and if main moves under it again before you land, I'll try to replay it
again, but this is a very big change and replaying it takes a lot of time on my end.

The tip's tree is byte-identical to this PR merged with current main (git diff between them is empty, tree
1d846c571), so the code you've reviewed is unchanged.

Checked on the rebased branch:

• Each commit builds on its own, and zig build test-cuda-host passes at each one, as does zig build test at the tip.
  The one test that failed here is exl3_load's allocation-failure test, at every commit. It fails only when the
  checkout is on tmpfs, the same test binary passes from ext4, and this PR doesn't touch it.
• At the tip: lean_new.py: 0 new; families/glm/cuda.zig's tests compile for
  x86_64-windows-gnu.
• tf-cuda-test glm-kv-host-slots (RTX PRO 6000): the past-the-slots call is still refused by name.
• Served at 1M context with TF_GLM_KV_HOST=1 on two RTX PRO 6000s: the four fixed prompts give main's reply SHAs
  without it, and the guard never fired.

[ashhart]

Thanks @russellmm, that's even better than what I asked for, and thanks for checking every commit on its own. We've taken glm-kv-host-rebased as is, onto current main with no conflict, and added our changelog line. It's in our gate now, and we've queued a two-GB10 GLM run at TP=2 against main to confirm the default path is unchanged. Your 1M-token run on two RTX PRO 6000s covers the flag itself. It lands as soon as both are green.

[ashhart]

@russellmm, our two-GB10 run is in: GLM-5.3-Flash NVFP4 at TP=2, main against this head (d065689) with TF_GLM_KV_HOST unset, base-new-new-base, --context 65536 --parallel 2.

• Tokens: equal in all 52 cells (52/0/0), with no unequal streams. Chat with thinking off and on qualified, with the same cached positions.
• Speed, new against base: prompt throughput +5.4% at 2K, +0.2% at 8K and +3.6% at 32K; decode is chat +0.7% (1 stream) and +1.2% (2 streams), and code +3.7% and +0.1%.

So with the switch unset, nothing changes, which is what we needed to see. It's gated and queued to land. Thanks!

[ashhart]

One more thing, @russellmm: since our run, main has moved under this, and it now conflicts in zig/src/families/glm/cuda_native.zig. Could you rebase onto current main (a force-push is fine)? We'll gate it and land it straight after. The run above stands for the unset path.

[russellmm]

Thanks @ashhart, and thanks for the two-GB10 run: good to see the unset path equal in all 52 cells.

Rebased onto current main (089a18dff) and force-pushed, as you said that's fine: the head is now c75fd0d0a, the same six
commits. The only conflict was commit 1's in cuda_native.zig, where main's TF_GLM_CHUNK_SOLO with --drafter
refusal (d229c4f) landed beside our TF_GLM_KV_HOST without KV pages refusal. Both are kept: the two checks in
openRank, their two explain lines, and both tests. Commits 2 to 6 replay unchanged (git range-diff shows them
equal), and the fingerprint stays at layout 15, since main is still at 14.

Checked on the new head:

• zig build test-cuda-host passes at each commit, and at the head zig build test passes (480/480 steps,
  6,360/6,405 tests, 45 skipped); lean_new.py: 0 new; families/glm/cuda.zig's tests compile for x86_64-windows-gnu.
• tf-cuda-test glm-kv-host-slots (RTX PRO 6000): the past-the-slots call is still refused by name.
• Served at 1M context with TF_GLM_KV_HOST=1 on two RTX PRO 6000s: the four fixed prompts give main's reply SHAs
  without it, and the guard never fired.