[Defilan]

What this changes

Before, deepseek_v41 refused a world of 3 and its plan, rank files and emitted calls all assumed an even head split. After, it plans three ranks with output groups split 3/3/2 by canon.partition (heads 24/24/16), writes and reads back their rank files, and every head- and group-sliced site takes the rank's own share. Serving still refuses three ranks.

• Issue: Refs deepseek_v41: three ranks on three GB10s (TP3), on the shared multi-rank layer #2270. This is the host-only first PR of the agreed gate. Still to come: the three-rank exchanges, a 24-head prefill kernel, Engram packs for a world of 3, serving, and the device gate.
• Stacked on: main.
• atc overlap --json: run 2026-11-06 19:12Z on fe8148d8e, with main at c3f64ca8a. On its own it returns complete: false, so I read the full file lists of all 224 open pull requests. Four share a code file with this branch, and atc overlap --pulls 1665,1924,2030,2379 --json is complete for them. dsv41: serve structured output with the lane core's grammar masks #1665 is clean. It reports conflicts on DeepSeek-V4.1: read the pack's weight spans past the page cache (core/direct_io) #1924, DeepSeek-V4.1 TP=2: every-rank receipts and opt-in per-layer hash files for repeat divergence (#1395) #2030 and DeepSeek-V4.1 on CUDA: refuse a malformed TF_DSV41_* setting by name before loading, and register the ones that change outputs #2379, in CHANGELOG.md, cuda_model.zig and CHANGELOG.md, and each one conflicts in the same file with main alone. A further 83 share only CHANGELOG.md, and ATC's time limit stops it finishing those, so I test-merged all 87 against this head and against main with git merge-tree: 28 are clean, 58 conflict the same way with main alone, and one, Qwen3.8-27B on a Mac: the disk staging permit takes one whole state at its context #2388, gains a CHANGELOG.md conflict from this branch's line. That's a one-line fix for whichever lands second.

Decisions to check:

• wq_b columns, wo_b rows and the attention sink follow the rank's output groups. Expert and vocabulary slices keep whole 128-column blocks (768 each, 337/337/336).
• block.emit now accepts a world of 3, because host tests can't reach the emit sites otherwise. A new block_admission.serving refuses three ranks before the pack opens, in Model.open and gate_decode_tp, and nativePrefill still refuses too. The remaining planners, gate_forward and tool_main, are fixed at world 2.
• aot_needs and calls_check take their world from the environment or a capture's metadata, so they can now emit world 3 calls. They only emit and check, with nothing running on a device.
• I also moved the two matching sites in dspark_emit.zig, which the issue's list doesn't name, because the same even split would be wrong there.
• canon is now imported by the family's module and by load_prefetch_host. Its rule sits outside the sources prepared.zig hashes, so the cache key now holds the rank's groups.
• Every world's prepared folders rebuild once after this lands, because prepared.sources hashes these files. The bytes they rebuild to are pinned unchanged. I haven't measured how long that takes at TP2 yet.
• The changelog line is its own last commit, three entries below the top of Unreleased, so new entries at the top don't conflict with it. Drop or move it as you prefer.

Review requests:

Request	Change	What proves it
Prefill pins for worlds 1 and 2	emit_pins_test.zig now pins prefill at worlds 1, 2 and 4, in the first commit. Digest captured on main's code	test-dsv41-forward passed on main's production code with the pin in place. Three new planted bugs at worlds 1 and 2 fail it by name. prefill_heads_test.zig already pinned those two worlds on main, so this overlaps it, and it can go if you'd rather not carry both
Linux compile	None needed for canon: every family module gets it through shared.moduleAt	13 of 14 Linux compile steps pass, including compile-dsv41-cuda (below). compile-dsv41-server fails on main too
Rebuild time at TP2	Not measured yet	My TP2 run used an older build without prepared folders. I'll post the number when I've run it
Changelog line	Added with your wording and the thanks you asked for, with no duration yet

Checks on this head

• Head commit: a91cd5925, base c3f64ca8a. The checks below ran on fe8148d8e; the only change since is "Thanks to @Defilan." in CHANGELOG.md.
• Lean checker, python3 tools/zig/lean_check.py, on the base and on this head: 210 and 210 by tools/zig/lean_new.py, 0 new.
• Tests run, with the command and pass, fail and skip counts, from an empty cache on this head:
  • zig build test test-golden -Dcpu=apple_m1: 11001 passed, 0 failed, 89 skipped, 720/720 steps.
  • zig build dev-tools: 234/234 steps.
  • zig build test-dsv41-forward test-dsv41-shared compile-dsv41-forward compile-dsv41-shared test-load-prefetch-host test-cuda-host: 925 passed, 0 failed, 4 skipped, 336/336 steps.
  • zig build test-dsv41-cuda-host test-dsv41-device-host test-dsv41-bounds-host test-dsv41-sm-host test-draft-dsv41-host test-dsv41-serve-line-host: 580 passed, 0 failed, 2 skipped, 40/40 steps.
  • Linux, compile only, zig build compile-dsv41-cuda compile-dsv41-server compile-dsv41-tool compile-dsv41-device-host compile-dsv41-sampler compile-dsv41-runner compile-dsv41-draft compile-dsv41-forward compile-dsv41-kv compile-dsv41-shared compile-dsv41-engram-alloc-aarch64 compile-dsv41-token-map compile-cuda-contract-consumers dsv41-prefill-heads: 13 of 14 pass. compile-dsv41-server fails with zig/src/server/clef_systemone.zig:5:25: no module named 'lanes' available within module 'root'. It fails the same way on main alone, checked at b3933ecd3, and neither that file nor the build files behind it have changed since.
  • zig fmt --check build.zig zig and git diff --check: clean.
• New tests failing before the change: the even-world digests were captured on main's production code in the first commit, so they pass before and after. For the rest, I planted 24 one-line bugs, one at a time, and test-dsv41-forward failed on every one (list below).
• CI: waiting for approval on this head; I'll classify any failure here. I can't find a workflow that names compile-dsv41-cuda or a Linux serving build, so CI may not cover those steps, which is why I ran them above.
• Not run, and why: no device runs, because the agreed first PR is host tests only. No kernel-contract, because no kernel changes. Linux is compile-only here, the 14 steps above. Two lines no test can distinguish yet: the groups lines at block_prefill_attn.zig:221 and :278. Ranks 0 and 1 stop at Unsupported first, and rank 2's two groups equal the old 8 / 3, so a planted even split there can't fail until 24-head prefill exists.

The 24 planted bugs

• Prefill pin at worlds 1 and 2: the core takes the wrong heads at world 1 and at world 2; attention takes the wrong heads at world 2.
• Planner: wq_b and wo_b ignore the output groups; world 3 stays refused; other uneven worlds admitted.
• Shares: block shares and canon each send ties to the higher rank; a rank's slices all start at zero; wo_a takes the wrong groups; wo_a's local index is off; a column split reads from the first tile.
• Refusals: serving admits three ranks; emit stays refusing three ranks; emit admits other uneven worlds; emit skips the rank's group check.
• Even splits restored at: the q group; decode attention heads and groups; prefetch groups; prefill attention heads; the prefill core's heads; DSpark attention heads and groups.

Receipt

• Environment, with the TensorFold commit, Zig and backend toolchain, machine and GPU, checkpoint and revision: fe8148d8e, Zig 0.17.0, Xcode 27.1 with Metal 32023.921, Apple M5 Max, macOS 27.0.1, 128 GB. No checkpoint: rank files are checked on a synthetic pack.
• Paths and platforms the change reaches, which I ran and which I could not: every default path at worlds 1, 2 and 4 runs the changed planner and emitters. I ran host tests on the M5 Max, including test's available Metal checks, though metal/ has no notion of a world, so those checks don't exercise this change. Not run: CUDA on any SM, including the GB10, which comes with the device gate; HIP; Windows; the desktop and mobile apps.
• Exactness, from tools/bench_concurrent.py --alone --serial or token_sha against "draft": false: not run. The evidence for this slice is byte-identical output: worlds 1, 2 and 4 keep their plan fingerprints, rank-file digests and emitted calls (decode, DSpark, prefill and prefetch), pinned on main. The TP2 token_sha comparison is gate item 1, before serving, on my GB10s.
• Decode speed before and after, from tools/bench_openai.py: not run. The calls are pinned identical at worlds 1, 2 and 4. The only added work is a host-side canon.partition at each emit site, which I haven't measured.
• Prompt speed before and after: not run, for the same reasons. The one visible cost is the one-time prepared-folder rebuild above, not measured yet.
• Licence of any new weights or drafter, or "n/a": n/a.

Checklist

• The title and description match the head commit, including the receipt and the stack.
• New tests fail before the change, pass after it, and skip cleanly without their dependency.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths in the diff or its commits. No AI attribution lines.

This change reaches inference, but with no device run yet I've left these unticked rather than claim them. They come with the device gate:

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo.
• No precision traded for speed. If the bits change, the description says which sums change and why.
• Decode and prompt processing are no slower than main at one stream and the widest admitted; with --drafter, no slower than without it, greedy and sampled.

[ashhart]

Thanks @Defilan. This is the first slice as we agreed it on #2270: world 3 plans, writes and reads back its rank
files, and emits each rank's own heads and groups, with serving still refusing three ranks. The pins taken on main's
code in the first commit, and the 21 planted bugs, are what we hoped for. Moving DSpark's two sites too was right.
Three small things:

1. Prefill pins for worlds 1 and 2. emit_pins_test.zig pins prefill only at world 4, but nativePrefill
   admits 64 and 32 heads, so emitPrefill is reachable at worlds 1 and 2 as well. World 2 is the one you serve.
   Could you add both worlds to that pin?
2. Linux compile. partition.zig now imports canon, so every module built from the family's sources needs it.
   Please paste the CI results once they're in, especially the Linux serving build and compile-dsv41-cuda. Your
   Linux evidence so far is compile-only for two steps.
3. The one-time rebuild. Every world's prepared folders rebuild once after this lands, because of the cache key
   and prepared.sources. How long does that take at TP2 on your nodes? We'd put it in the changelog line, which this
   also needs, for example: "DeepSeek-V4.1: attention heads now split by whole output groups through the cluster
   layer's shared rule, so three ranks plan and write their rank files (24/24/16 heads). Serving still refuses three
   ranks. Worlds 1, 2 and 4 plan, write and emit the same bytes as before; their prepared folders rebuild once after
   upgrading (deepseek_v41: plan and emit a world of 3, host tests only #2325, part of deepseek_v41: three ranks on three GB10s (TP3), on the shared multi-rank layer #2270)."

One note, no change needed: nothing tests that Model.open and gate_decode_tp actually call
block_admission.serving, since those paths need a device. The direct test of serving is fine for now; the
serving PR will cover it.

[Defilan]

Thanks @ashhart. The branch is now at fe8148d8e on c3f64ca8a, and the body has the full results.

1. Prefill pins: done, in the first commit, with the digest captured on main's code. I should have mentioned that prefill heads: legacy world one and two... in prefill_heads_test.zig already pins those worlds on main, with the same rows and layers plus the buffer plans, so this overlaps it. Three new planted bugs at worlds 1 and 2 fail both tests. Drop mine if you'd rather not carry both.

2. Linux compile: every family module gets canon through shared.moduleAt, and 13 of the 14 Linux compile steps pass, including compile-dsv41-cuda. I can't find a workflow that runs compile-dsv41-cuda or a Linux serving build, so I ran them here. If there's a job I've missed, tell me.

   compile-dsv41-server fails, and it fails the same way on main without this branch:

   zig/src/server/clef_systemone.zig:5:25: error: no module named 'lanes' available within module 'root'
   

   clef_systemone.zig came in with 1fb413b61, and serverOn in zig/build/dsv41_draft.zig doesn't give the server module lanes, the way zig/build/server.zig:49 does for the others. dsv41-serve goes through the same serverOn, so I'd expect it to fail too, though I haven't built it because it needs nvcc. Would you like a separate one-line PR for that?

3. Rebuild time: I don't have a number yet. My TP2 run so far used an older build without prepared folders, so measuring it means building main with nvcc and taking my nodes for a run. I'll post it when I've done that. The changelog line is in with your wording and no duration. It's its own commit, three entries below the top so new entries don't conflict with it. Drop it if you'd rather add it the usual way.

Agreed on serving: no change, and the serving PR will cover it.

Could you approve the CI runs on this head? They've been waiting on first-contributor approval. I pushed these changes first so you'd only have to approve once.

[ashhart]

Thanks @Defilan. The world 1 and 2 prefill pins are what we asked for, and keeping both tests is fine. You're right about compile-dsv41-server: serverOn in zig/build/dsv41_draft.zig doesn't give the server module lanes, so it fails on main too. A separate one-line PR for that would be welcome. On the CHANGELOG line, please add "Thanks to @Defilan." after "(#2325, part of #2270)." We'll wait for your rebuild time at TP2 on your nodes, and add it to the line then.

[Defilan]

@ashhart here's the rebuild time at TP2 on my nodes: about 10.5 minutes to weights loaded on a first boot, against about 5 on a later one. Each rank's folder is 98.4 GiB.

Boot	Rank	Prepared folder	Boot start to weights loaded	weights	Images	Writes, within images	Upload + digest
2	0	empty, 44 built	10 min 35 s	453.6 s	435.9 s	102.0 s	11.2 s
1	1	empty, 44 built	9 min 49 s	411.8 s	393.6 s	86.6 s	11.5 s
2	1	full, 44 read	4 min 48 s	108.9 s	90.5 s	none	11.1 s

The first column is wall clock from each rank's boot trace: start line to its dsv41 load line, and the rest comes from that line. On the last row, images means reading the folder.

The one-time cost I actually measured is 5 min 1 s, on rank 1. A boot waits for its slower rank, so it would be about 6 minutes if rank 0's later boot matches rank 1's, but I didn't measure that one. 177 to 181 s of every row comes before the weights loop, which is where packDigest runs, so later boots pay that too.

Setup: two GB10s, driver 580.178.04, local NVMe, page cache dropped before each boot. The pack is 197 GiB, EXL3 2.9 bpw: LibertAI's, with an overlay that keeps its layout. World 2, --context 524288, TF_DSV41_PREPARED with TF_DSV41_PREPARED_WRITE=1, drafts on. Built from fe8148d8e plus the lanes line, with zig build dsv41-serve -Doptimize=ReleaseSafe -Dnvcc=... -Ddsv41-nvcc=... -Dsm=121, Zig 0.17.0, CUDA 13.0.3. The current head differs from that commit only by the changelog thanks.

What to know about these numbers:

• There were two boots. In the first, rank 1 finished its rebuild and then stopped on my own mistake: TF_DSV41_SLOTS=2 with no pool, RowsNeedPool. The slots setting is read after the weights and isn't in the cache key, so rank 1's time stands. Fail-fast took rank 0 down at 35 of 44, so I threw that away, cleared rank 0's folder and ran the second boot with TF_TP_FAILFAST=0 and no slots setting.
• This covers loading only, not serving. In the second boot both ranks stopped with FileNotFound after the forms boot step. The log doesn't name the file. The next step reads rope.bin, and the older kit I mounted has no rope.bin or engram-token-map.bin.
• The old folder may need room too. The new key gets a new directory, and I can't find anything in the repo that removes the old key's folder. If that's right, the rebuild needs another 98.4 GiB free per rank until someone deletes the old one. I started from empty folders, so I haven't seen this on a real upgrade.

For the line, maybe: "their prepared folders rebuild once after upgrading, about 6 extra minutes at TP2 on two GB10s (#2325, part of #2270). Thanks to @Defilan." Change it however you like.

I'll send the lanes fix as its own PR.