[jschmied]

What this changes

Spark 2.5 and Dory on CUDA now read each layer past the page cache, with the next layer read while the current one packs: ledger row 22 on two more families. Each family has one call on core/checkpoint_direct.zig (#2282's Layers), which gains two helpers:

• Layers.layerAt(fmt, i, next) enters layer i and starts layer next's read (the two prefix prints every caller wrote).
• Layers.moved(v, fmt, i, n) does the same, then moves the tensor views inside v (fields, unions, optionals, slices) onto the layer's copies. It is for loaders that map every tensor before the layer loop. A view outside the layer keeps the mapping.

Spark 2.5 reads through Checkpoint.get, so it uses layerAt. Dory's names.map takes every view first, so it uses moved. Readers come from TF_LOAD_PREFETCH (default 16), as on Gemma 4 and Qwen 3.5. While direct reads are on, Spark's opt-in read-ahead gets 0 readers. TF_LOAD_DIRECT=0 or TF_LOAD_PREFETCH=0 keeps main's mapped reads.

Not here:

• Kolibri-1 and MiniMax M3: the same one call fits (moved for Kolibri-1's map, layerAt over MiniMax's rank layer list), but there's no checkpoint here to run it, so it stays out until there is.

• GLM: a TP rank cuts its rows from the mapping, while Layers reads whole tensors, so a rank would read about world-size times the bytes it uses. It needs a row-aware gather in core first.

• Issue: n/a, a ledger row 22 port (Flash-Next CUDA load reads at 1.1 GB/s on a 5-7 GB/s disk: read loaders' mapped views past the page cache (core helper) #1873 did the first family, core: Gemma 4 and Qwen 3.5 on CUDA load with direct reads, 28-31 % faster cold; Flash Next shares the read-ahead #2282 the shared loader). Memory admission on GB10: page-cache growth moves the loading minimum; a pre-launch cache release fixes it #1857 is related but held elsewhere.

• Stacked on: main 964f98b; commits 0316c8c (core), 151cb02 (Spark 2.5, Dory, ledger, CHANGELOG).

• atc overlap --json (2026-11-07 14:25, base main 964f98b): 9 conflicts, all in CHANGELOG.md's Unreleased list (CUDA: audit sm_110 cubins without widening release records #2395, docs: the end-token pair for replies that quote a template marker #2437, hip flashnext: sparse attention with a lane a key over several heads, the scalar kernel's bits #1877, hip flashnext: the MTP head absorbs a prompt chunk on the prompt tiles #1878, lfm3: add native Metal vision support above the text decoder #1502, docs: /health's memory on Metal, and which families report prompt_cache_plan #2439, LightOnOCR-3 (0.8B, 1B, 4B) on native HIP, with the model card's oracle #1965, Nemotron Parse on HIP: NVIDIA's checkpoint, token-exact against its HF example #1941, qwen3_5 cuda: readout streams fill in pieces of final-norm rows (#2324 step 2) #2384); 1 clean; the rest unknown (pullFilesLimit). dory cuda: TF_KV_SHARE keeps prompts as the pool's pages in the prompt_radix store #2430 (Dory prompt store) merges except CHANGELOG.md.

Checks on this head

• Head commit: 151cb02
• Lean checker on base and head: 116 / 116, 0 new.
• Tests run: zig build test test-golden test-cuda-host: 590/590 steps, 7702 of 7765 tests passed, 63 skipped, 0 failed. test-kernel-contract test-server-security test-pull-security fuzz-cortheon, check-native-cuda-host compile-cuda-tests -Dtarget=x86_64-linux-gnu, zig fmt --check and git diff --check: all pass. The new core test covers layerAt and moved on a two-layer checkpoint (copies equal the mapping, out-of-layer views stay mapped, get returns the same copies), and the off test covers both as no-ops.
• tf-cuda-test spark25-pages and dory-pages pass with direct reads and with TF_LOAD_DIRECT=0, and print the same RESULT lines (19 and 13).
• CI: not run yet (no push).
• Not run, and why: Kolibri-1, MiniMax M3 and GLM have no checkpoint on this box; discrete cards (we have only the GB10).

Receipt

• Main against branch: one GB10, page cache dropped before every start, a warm-up start dropped, then A-B-B-A; time to /v1/models:

  model	main (s)	branch (s)	replies	concurrent 1,4 greedy
  Spark X2.6 4B 8-bit	5.03 / 5.04	2.53 / 2.53 (−50 %)	identical	identical
  Dory NVFP4	4.53 / 4.53	3.78 / 3.78 (−17 %)	identical	identical

  Spark saves more than Dory because main reads its 3.74 GiB of layer weights through faults as stage.upload copies from the mapping, at about 1.5 GiB/s, which is about 2.5 s of main's 5.0 s. The branch reads them on reader threads while the previous layer packs, so the whole read leaves the load's critical path. Dory has 1.73 GiB in layers (its embedding and head, 1.25 GiB, stay mapped), so it saves 0.75 s. Memory at ready is level (MemAvailable within 0.26 GiB, server RSS 6.14 and 7.18 GiB in both arms), and page-cache growth is 0.09–0.10 GiB in both arms.

• Boxes it reaches: CUDA on Linux, Spark 2.5 and Dory loads.

• New switch? None; TF_LOAD_DIRECT and TF_LOAD_PREFETCH as in core: Gemma 4 and Qwen 3.5 on CUDA load with direct reads, 28-31 % faster cold; Flash Next shares the read-ahead #2282.

• Environment: TensorFold 964f98b against 151cb02; Zig 0.17.0; CUDA 13 nvcc, sm_121; GB10, Linux 7.0 aarch64, 128 GB unified; checkpoints: Spark X2.6 4B MLX 8-bit affine (converted locally from XHToken/Spark-X2.6-4B at 66e3ae7b), Dory NVFP4 (local, revision in SOURCE.json).

• Paths and platforms: the load path of both families on CUDA, direct and TF_LOAD_DIRECT=0; run on sm_121 only.

• Exactness: tools/bench_concurrent.py --tokens 128 --reps 1 --temperatures 0 --levels 1,4 --alone --serial: output hashes identical across all four arms per model.

• Decode speed: n/a, loading only. The weights uploaded are the same bytes.

• Prompt speed: n/a, loading only.

• Licence: n/a.

Checklist

• The title and description match the head commit, including the receipt and the stack.
• New tests fail before the change, pass after it, and skip cleanly without their dependency. (The new test calls the new helpers, so it can't build on main; it skips off Linux or with TF_LOAD_DIRECT=0.)
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths in the diff or its commits. No AI attribution lines.
• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo.
• No precision traded for speed.
• Decode and prompt processing are no slower than main (unchanged: loading only).