[jschmied]

What this changes

A Gemma 4 prompt pass takes TF_GEMMA4_CHUNK_SOLO rows (a multiple of the 512-row grid, up to 4096) while no other stream decodes, and the 512-row grid chunk once one does. Rings, prompt scratch and the stream byte count size on the larger value. Unset, nothing changes. Not beside a drafter's taps, or past 8 images in a prompt.

The rule moves to core/lanes/solo_chunk.zig, backend-free. Flash Next's TF_FLASHNEXT_CHUNK_SOLO (with its one-stream default of 8,192 kept) and GLM's TF_GLM_CHUNK_SOLO call it in place of their own copies.

• Issue: Refs Flash Next CUDA: prompt chunk size against decode waits, how other engines handle it, and options #1554.
• Stacked on: main (18ecc42).
• atc overlap --json: the rerun at posting hit its time limit and lists qwen3_5 cuda: readout streams fill in pieces of final-norm rows (#2324 step 2) #2384, qwen3_5 cuda: a plain BF16 checkpoint loads as a decision backbone, chat refused by name (#2324 step 1) #2383, moss image: serve pictures on CUDA (encoder, decoder and host for #1573) #2281, server: --unix-socket listens on an owner-only Unix socket instead of TCP #2389, hip: build and load the runtime on Windows #2178 and deepseek_v41: plan and emit a world of 3, host tests only #2325 with one shared file each, beside the file-level check: CUDA: a request's stats list its prompt pass widths for the 27B, GLM and Nemotron (stacked on #2293) #2304 conflicts in lanes.zig (adjacent line); flashnext cuda: prompt cache marks inside a chunk are kept on the way instead of cutting the pass #1866, core: a bounded ladder of intermediate prompt-cache checkpoints #1953 and the EXL3 stack EXL3 Flash Next on CUDA: one rank, offloaded experts, sm_120 gate (1/8) #1441-EXL3 Flash Next: draft lane guard for offloaded experts (8/8) #1446 and EXL3 Flash Next: host-resident K/V on the tier (6/8) #1522 conflict in Flash Next files or files this does not touch, cuda_lanes.zig being a one-word change here; profile: register the pool-page switches for Dory, Gemma 4, Spark2.5 and Nemotron's Metal paging #2306 CHANGELOG only; Living Weights on Gemma 4, step 2: the native GeGLU learner on CUDA and Metal (experimental, --slide refused) #2062, CUDA: every synchronous readback stages on integrated GPUs, as download does (#2102) #2289, lfm2: DSpark drafting with owned context and confidence scheduling #1394 clean.

Checks on this head

• Head commit: 5d5290d (timing below from 54c94ba, before the rebase; the rebase adds only main, the ledger lines and one constant)
• Lean checker on the base and on this head: 252 and 252, new findings 0.
• Tests run (sob-verify2, all rc=0): zig build -Dfixed-cpu test, test-cuda-host, test-kernel-contract test-golden test-server-security test-pull-security fuzz-cortheon, compile-cuda-tests arm and x86, check-release-paths, test-shared-lane-regressions-host test-gemma4-cuda-host, fmt, git diff --check, cuda_common_check.py.
• CI: not run yet.
• Not run, and why: Flash Next and GLM servers: the rule moved without changing its answers, covered by unit tests and the contract, not timed.

Receipt

• Environment: GB10 sm_121, clock capped 2300, Zig 0.17.0, --parallel 3 --context 49152 --no-drafts --prompt-cache-gib 0, one fresh server per arm, page cache dropped first, runs ordered base, new, new, base. main md5 f2a3b4bb, branch 395bef74.
• Paths: Gemma 4 on CUDA, 31B and E4B 4-bit. Metal not reached. qwen3_5 not touched, so the 27B MLX path is not on this change's path (the one shared file, lanes.zig, gains an import) and was not measured.
• Exactness: 15 of 15 replies equal to main on every arm, solo and unset.
• Prompt speed, TF_GEMMA4_CHUNK_SOLO=2048, median TTFT of 6 values, per-server medians in brackets:

Checkpoint	Prompt	main s	solo s
31B 4-bit	3.9k	4.220 [4.220, 4.219]	4.039 [4.025, 4.042]	-4.3 %
	15.6k	19.071 [19.053, 19.075]	18.073 [18.047, 18.103]	-5.2 %
	29.2k	40.748 [40.744, 40.797]	38.714 [38.701, 38.773]	-5.0 %
E4B 4-bit	3.9k	0.726 [0.726, 0.726]	0.653 [0.653, 0.652]	-10.1 %
	15.6k	3.269 [3.271, 3.268]	2.981 [2.985, 2.976]	-8.8 %
	29.2k	7.083 [7.085, 7.081]	6.433 [6.433, 6.433]	-9.2 %

• Decode speed: mean gap beside a filling 15.6k prompt 31B 0.6777 -> 0.6816 s (+0.6 %), E4B 0.1180 -> 0.1175 s (-0.4 %); max 0.806 -> 0.807 s, 0.142 -> 0.143 s. A busy pass stays 512 rows.
• Memory: MemAvailable after load 31B 89.3 -> 85.1 GB, E4B 116.2 -> 115.9 GB.
• NVFP4 (docs: TF_GLM_CHUNK_SOLO=4096 is slower with GLM's NVFP4 checkpoint on two GB10s #2288): no Gemma 4 NVFP4 checkpoint was tested. CUDA Gemma refuses mxfp4 at load, so this variable cannot reach one today.
• Licence of any new weights or drafter: n/a.
• Kernel contract: no kernel changed. Filtered to gemma4 Gemma glm flashnext Flash, both arms: SUMMARY 83 kernels: 83 pass gates 1-4, 0 fail, 2 known findings; 6 controls: 6 red as planted, 0 missed; 9 limit problems. Exit status 1 on both from the limit problems alone, FAIL names new 0.

Notes for the reviewer

• Nemotron gets no variable here. A prompt pass already runs whole, so decoders wait either way and no solo rule has a busy case. An oracle of 8,192-row passes against main's 4,096 (not in this PR): Lightning 4-bit +0.2, -1.9, -2.1 %, model 17.82 -> 18.69 GiB; Bolt 74B 24GB -0.1, -1.6, -1.8 %, model 22.94 -> 24.39 GiB. Replies equal. Bolt's checkpoint is mostly IQ2_XXS with NVFP4 on 80 layers, so it says little about a full NVFP4 export, and it is one GB10 at --parallel 3, unlike docs: TF_GLM_CHUNK_SOLO=4096 is slower with GLM's NVFP4 checkpoint on two GB10s #2288's two.
• Past 2048: 4096 gained nothing over 2048 on the 31B before the rebase (3.9k 4.031, 15.6k 18.045, 29.2k 38.972 s) for 1.6 GiB more a stream.

[ashhart]

Thanks @jschmied. We've read it against main, and it's right:

• Flash Next and GLM get the same answers from the shared rule as from their own copies.

  • parseSolo keeps parseChunk's parse and range (busy to max_chunk) and still returns BadChunk.
  • passRows now compares the lane map's key, and cuda_lanes.zig passes the stream that keys it instead of that stream's lane.

  Neither family needs a device run for this.

• Unset, Gemma 4 runs main's bytes. prompt_most stays 512, so the rings, the scratch, streamBytes and every chunk's end are unchanged.

• Set, it's exact by construction as well as in your 15 of 15. Sliding attention walks keys by position and maps key p to row p % ring, in prompt rows and decode alike. A windowed layer's decode chunks are counted from the window, not the ring. So a wider ring changes where keys sit, not the order they're summed in.

• The rest checks out. The ring grows by the widest pass, so a pass can't overwrite keys its first row still reads. The drafter's narrow window follows ringRows(), and streamBytes counts the wider rings for admission. The > 8 images guard keeps every chunk within the 8 slots injects has.

One ask before our run:

• The optimization ledger. Following the pre-flight (docs/pr: the pre-flight updates the optimization ledger when a family gains or drops an optimization #2243), please set Gemma 4's row 20 (Solo chunks) to done C[#2391]. Then add a gemma4 (C) line to note 20's table with your first-token gains: 31B −4..−5 %, E4B −9..−10 %, opt-in with TF_GEMMA4_CHUNK_SOLO.

A nit, take it or leave it: s.images.len > 8 could use the length of injects as a shared constant, so the two can't drift apart.

Once the ledger is in, we'll run Gemma 4 on a GB10, main against this head, unset and at 2048. Then it goes to the landing queue.

[jschmied]

Thanks @ashhart. Done on 5d5290d25, on main 18ecc42d6:

• Ledger: Gemma 4's row 20 is done C[#2391], and note 20's table has the gemma4 (C) line: 31B −4..−5 %, E4B −9..−10 %, opt-in with TF_GEMMA4_CHUNK_SOLO. The ledger wasn't on this PR's old base, so I rebased onto main. The only conflict was the CHANGELOG, and I kept both sides.
• The nit: taken. max_injects now sizes injects and guards s.images.len.
• CHANGELOG: the line now carries (gemma4 cuda: TF_GEMMA4_CHUNK_SOLO, on a solo-chunk rule shared in lanes (#1554 option C) #2391).

Lean checker, zig fmt, git diff --check, zig build test, test-cuda-host, the CI set (test-kernel-contract, test-golden, test-server-security, test-pull-security, fuzz-cortheon), compile-cuda-tests on arm and x86, check-release-paths, test-shared-lane-regressions-host and test-gemma4-cuda-host all pass on this head. The receipt's timing is from 54c94babb, before the rebase, so your run is the first on this head.