RFC: Nemotron 3.5 Bolt (nemotron_h_puzzle) on CUDA, and Nemotron Omni video #937

Proposal. Serve NVIDIA's Nemotron 3.5 Bolt 74B-A9B 24GB on the native CUDA host (one GB10), as a new family nemotron_h_puzzle, plus Nemotron Omni video input. PRs: #938 (the family, images) and #940 (video, stacked on #938). I would like to agree the gate before they are reviewed in earnest, as CONTRIBUTING.md asks.

The model. config.json is a nemotron_h_omni wrapper around a nemotron_h_puzzle text model: hidden 4096, 88 blocks (40 Mamba, 40 MoE, 8 attention), 240 to 496 routed experts a layer in widths 1536 to 2560, top-9, a 1,024-wide latent around the routed experts, ReLU². The 24GB pack is IQ2_XXS experts, NVFP4 Mamba projections and Q8_0 attention, latent projections and lm_head. Context 262,144. NVIDIA's LiLiCorr is its drafter (no MTP head). None of these shapes or packings match the Lightning or Super VL kernels, so this is not a flag on the existing family.

What I have. An engine, libbolt (about 4,800 lines of CUDA), run until now as a recipe: https://github.com/MiaAI-Lab/Nemotron-3.5-Bolt-TensorFold-DGX-Spark (private for now; I can make it public or send an archive). This change moves the Zig side into the tree:

1. PR nemotron: serve NVIDIA Nemotron 3.5 Bolt (nemotron_h_puzzle) on CUDA #938, the family. nemotron_h_puzzle as a model type, registered in the CUDA host; a lane backend families/nemotron/bolt_native.zig (prompt pieces, one shared verify window for up to 16 streams, rollback, LiLiCorr drafts, keyed draws, prompt-cache copies through snap_*); zig build bolt -Dnvcc -Dbolt-src to build libbolt.so beside the server; images through the existing RADIO tower. Two small generic changes: a family may set snap_lookahead, and --vision-image-tokens goes past 16,384.
2. PR nemotron omni: video input through the RADIO tower #940, video (stacked on nemotron: serve NVIDIA Nemotron 3.5 Bolt (nemotron_h_puzzle) on CUDA #938): video for any Omni checkpoint with a video_embedder: two frames to a tubelet, the processor's frame labels, <video> swapped for one reserved token before encoding (the tokenizer splits <video> by its neighbours), --vision-max-videos and the engine's max_videos instead of a constant.

Receipts (GB10, TensorFold 46a08c18 plus these commits, checkpoint a326822a, greedy):

tools/bench_concurrent.py --levels 1,4,8 --alone --serial, greedy, 256 tokens, 3 repetitions, against --parallel 8 --context 262144 --prompt-cache-gib 8 --vision:

Concurrent equals solo and drafted equals "draft": false in every cell. Cold TTFT (build before the exactness changes, decode-only changes since): 1.44 / 5.60 / 24.66 / 55.60 s at 2k / 8k / 32k / 64k tokens. Perplexity 5.286 on 1,499 tokens. 50 images and 4 videos in one chat, and 50 images of 2560x1920 (165,600 tokens), read correctly.

Why concurrent used to differ from solo, and what changed. An earlier version of this text blamed "GEMM kernels chosen by the rows in a block"; that was incomplete. Three things made a stream's rounding depend on the batch, and all three are fixed in libbolt:

1. The verify-window attention merged partial softmaxes over key chunks whose size followed the number of streams in the block (about 32 splits for one stream, 8 for four) and the window's width. The chunk is now win_chunk(base): 128 keys, doubled until 32 chunks reach the end of the sequence and its window, a function of the stream's own length alone. With the split count forced fixed in a scratch build, alone and 4-stream perplexity were identical, which isolated this one.
2. Dense GEMMs for blocks of up to 32 rows split K across four warps (interleaved partial sums, then added); blocks over 32 rows used a kernel that walks K in a single accumulator. Decode and verify blocks now always use the K-split kernel (64 tokens a CTA); the prompt pass keeps the faster kernel. This was the reason chat at 8 streams still differed after the attention fix.
3. The one-token plain step ran a different attention kernel from the windows; it now runs as a one-row window and is kept at once.

Harness checks (run with 32-token GEMM tiles): perplexity of four passages scored alone and in a block of 2, 3, 4, 8 and 16 streams (windows of 1, 5 and 8 rows, up to 128 rows a block) is identical to five decimals; sixteen sequences in one block equal their solo runs, 768 of 768 tokens, plain steps and 4-row windows alike. The final 64-token tile changes only the tokens a CTA holds, not the order K is summed in; it is covered by the table above.

Cost. The K-split kernel is slower in the widest rounds (shared round of N streams x R rows, ms, before / after): 8 x 8 rows 135.5 / 130.4, 16 x 4 rows 133.2 / 141.7, 16 x 8 rows (128 rows) 176.9 / 220.3. Rounds of up to 32 rows are unchanged.

What still does not meet the contract, stated plainly.

• The kernels are not under the kernel contract. libbolt is a CUDA runtime library, not fatbins; it targets sm_121 only; its sources are about 4,800 lines in files past the lean checker's 600-line and comment rules (the Zig side leaves the checker at the baseline, 517, with 0 new problems).
• The engine repository is private today.
• Tested on one GB10 only (no Metal, no other CUDA cards); Super VL not run for the video change; --alone / --serial not run on video requests.

An earlier version of #940 failed CI's lean check (engine_api.zig at 602 lines); that is fixed by moving the video offer types to core/video_offer.zig.

Questions.

1. Should the CUDA sources live in this tree (and be split and ported to the kernel contract), stay in a separate repository the PR builds with -Dbolt-src, or something else?
2. Is the 128-row-round cost above acceptable for exactness, or would you rather have a faster large-block kernel with the same summation order?
3. Is the video change welcome separately for Super VL (I can't test it there)?