[plotarmordev]

One commit, 47ef1241, on main d1018b31 (the same tree as the tested c8dc633c; only the commit message changed).
Its speed was measured only together with "radio CUDA: reuse one pinned pixel stage for every image", both on #1057;
see Receipt.

What this changes

In every RADIO tower layer, three Linears end in a separate bias launch and a separate pointwise launch whenever
the cuBLASLt algorithm chosen for their shape does not take the bias epilogue: the attention projection
(tf_vit_bias, then tf_vit_pointwise adding the residual into h), the first MLP Linear (bias, then erf GELU) and
the second (bias, then the residual). tf_vit_bias writes the bf16 sum of the f32 accumulator and the bias, and
tf_vit_pointwise reads it back. The new tf_vit_bias_act does both in one launch and keeps that bf16-rounded value
in a register:

O[at] = pointwise_at(rbf(__fadd_rn(ACC[at], bf(BIAS[c]))), R, at, act);

pointwise_at is the old body of tf_vit_pointwise, moved into a helper that both kernels call, so the activation
rounding, the residual-first addition and the final bf16 rounding are the same code. One element a thread, the same
block layout as tf_vit_bias. The SASS of tf_vit_pointwise and tf_vit_bias is unchanged by the refactor (sm_121
fatbins of main and of this branch, gpu/sass.txt).

vit.linearAct replaces those three linear + pointwise pairs in Tower.features. When the shape's plan takes
cuBLASLt's bias epilogue it runs exactly the old pair; otherwise it runs the same f32 GEMM as linear and then
tf_vit_bias_act. The QKV Linear, GEMM planning and algorithm choice, and the projector's own pointwise launch are
unchanged. In the fused path the intermediate buffer is no longer written; its next readers (layerNorm, the final
norm, project) write it before they read it.

Kernel contract: zig/tests/cuda/contract_vit_glue.zig registers tf_vit_bias_act with replaces = "tf_vit_bias then tf_vit_pointwise", one spec per variant ({none, tanh GELU, erf GELU, relu²} × {no residual, residual, in-place
residual}, 12 names) at the served width and rows (13,322 × 1,280 and × 5,120). The gates run the real original
launches on the same guarded inputs and compare bytes. The non-finite gate plants NaN, +Inf and −Inf in the
accumulator, NaN and +Inf in the bias, and +Inf and −Inf (no NaN) in the residual, and requires a non-finite output.
The contract runs the kernel alone, not linearAct inside the tower. The launch record (vit_records.zig,
15/16/16/14/14 registers on sm_80/86/89/120/121 from nvcc 13.0) sits in the live registry, with a host test in the
qwen_served_records pattern. docs/kernel-contract.md lists the kernel under CUDA coverage.

Receipt

Raw files: https://github.com/plotarmordev/TensorFold/tree/nemotron-receipts/receipts/nemotron/vision-fused-glue (README.md maps every file; SHA256SUMS).

• Environment, with the TensorFold commit, MLX or PyTorch version, machine and GPU, checkpoint and revision: this
  branch at 47ef1241 on main d1018b31; it has the same tree as c8dc633c, the commit every receipt below was
  made from. GPU builds from detached checkouts (env/PROVENANCE-*.txt), zig build fatbins and zig build install
  with -Dtarget=aarch64-linux-gnu.2.28 -Dcpu=baseline -Doptimize=fast -Dstrip=true -Dsm=121 -Dversion=1.1.1 -Dnvcc=… (zig 0.17.0, nvcc 13.0.88): base = d1018b31 + nemotron cuda: a prompt pass with images runs the RADIO tower once, not twice #1057 (732b0e31; without nemotron cuda: a prompt pass with images runs the RADIO tower once, not twice #1057 every image runs through
  the tower twice); contract build = nemotron cuda: a prompt pass with images runs the RADIO tower once, not twice #1057 + this commit (b4ae488c; tf-cuda-test sha256 c8129813…179f29973);
  timing build "both" = nemotron cuda: a prompt pass with images runs the RADIO tower once, not twice #1057 + this commit + the pinned-stage pull request (f6d8ba5a; tensorfold-native
34fba744…a29cda, base 5e70f78f…dcef69). env/commits.txt shows each measured commit's parent, tree and git patch-id --stable next to the submitted ones: 47ef1241 and c8dc633c share tree 2a07a5f1…; b4ae488c and
  b7e05211 (under f6d8ba5a) carry their patch-id; every nemotron cuda: a prompt pass with images runs the RADIO tower once, not twice #1057 copy carries e2357b58's. One DGX Spark (GB10,
  sm_121), driver 580.173.02, CUDA 13.0,
  nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4@82d93699acd3be9e4617c53360fe5b3f5dbeb1b3, every hold a fresh
  process (and for servers a fresh HOME) with TF_HEAT_HIGH=85 TF_HEAT_LOW=75, started at a zone of 60 °C or less.

• Kernel contract, tf-cuda-test kernel-contract on the contract build:

  • SUMMARY 97 kernels: 97 pass gates 1-4, 0 fail, 0 known findings; 7 controls: 7 red as planted, 0 missed; 1533 limit problems.
  • 48 PASS lines for tf_vit_bias_act, all four gates for each of the 12 variants, e.g. PASS nonfinite cuda vit tf_vit_bias_act gelu in-place residual: the stated rule holds: byte-identical to bias then pointwise, including non-finite inputs and PASS shapes … : 6 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel.
  • Limits: PASS limits live vit tf_vit_bias_act: 256 threads launched, the driver allows 256; 14 registers, 0 + 0 shared bytes (recorded 14, 0); cuobjdump -res-usage on the sm_121 fatbin: REG:14 STACK:0 SHARED:0 LOCAL:0.
  • The aggregate run fails: it exits 1 on the limits gate. All 1533 limit problems belong to launches registered in
    main's source (this diff adds only tf_vit_bias_act's record) that
    have no nvcc 13.0 usage record: 877 <kernel> on <sm>: no compiled usage recorded and 656 live launches whose
    record reads (recorded 0, 0). They are in DeepSeek V4.1 images (1200) and in Qwen 3.5, sampling, top-k, GLM,
    act_rows, gdn_prep, tree_round, qk_rows and affine4 images (333); gpu/limit-problems.txt counts them
    by image. None names a kernel this pull request adds or changes. Main's own tf-cuda-test was not run in the
    same session, so an unchanged count on main is not shown; a green aggregate needs those records, or a waiver
    for them.
  • compute-sanitizer --tool memcheck --error-exitcode 1 over the same command: ERROR SUMMARY: 0 errors, the same
    SUMMARY line (its exit 1 is the same limits gate).

• Exactness, from tools/bench_concurrent.py --alone --serial or token_sha against "draft": false: with "both",
  every image-ttft request (2048 x 1664 images, "draft": false, 8 a hold, 2 holds) gives
  541bca2d52751d8bb9c44b2af53ce8ff2dfb93745d8cb6bdb740442dad9735ec for one image and
  ef22b44ea1d731737c2d2a757d671d03b1269563d6bb505944e58608ecd444c4 for four, the same as the base and as nemotron cuda: a prompt pass with images runs the RADIO tower once, not twice #1057's
  receipts. bench.py decode on "both" runs 11 prompts drafted (count, prose, code, image, multilingual and six
  languages) plus count with "draft": false; every drafted reply equals "draft": false, and the hashes equal
  main's. Its image prompt carries one 320 x 208 picture, so that is the one drafted image request that ran through
  the fused tower. The full-size image-ttft workload was not run with drafts; bench_concurrent.py, resumed and
  concurrent image requests were not run.

• Decode speed before and after, from tools/bench_openai.py: not measured. Decode rounds never run the tower; only
  an image prompt's prompt pass does.

• Prompt speed before and after: tools/bench_cache.py sends no images, so TTFT is the server's
  time_to_first_token from bench.py image-ttft (scripts/; one generated 2048 x 1664 PNG, then four copies,
  3,356 and 13,366-token prompts, a warmup and 3 measured requests each, greedy, thinking off, "draft": false, 32
  tokens, cooled to 75 °C before every request; served with --vision --vision-image-tokens 13312 --vision-max-images 4 --context 18432 --parallel 1 --prompt-cache-gib 0). Holds in the order base, both, base,
  both:

  	base	both	base	both	both vs base
  1 image, median of 3	2.293	2.177	2.288	2.172	−5.1 % (−0.116 s)
  4 images, median of 3	15.001	13.870	15.008	13.856	−7.6 % (−1.14 s)

  This commit alone was not timed, and nsys was not run on it, so its share of that gain, the fused launch counts
  in a trace and the kernel's achieved bandwidth are not measured. The pinned-stage commit alone, under nsys, went
  from 2.301 to 2.239 s (one image) and from 15.058 to 14.030 s (four). Every four-image request on every build paused
  once at the heat gate inside its prompt pass (heat_wait_s 8.0); the TTFTs include it.

• Tests run, with pass, fail and skip counts (CPU only, a DGX Spark, -j1, nice -n 19, fresh caches), at c8dc633c
  (this tip's tree):
  zig build test test-golden fuzz-cortheon compile-cuda-tests 227/227 steps, 2971 of 2985 tests passed, 14
  skipped, 0 failed (fuzz-cortheon 11 of 11; test-golden 26/26 and 306/319 equal to the frozen answers, 0
  unexpected differences; compile-cuda-tests builds); zig build test-kernel-contract 11/11 steps, 66 of 66 (47,
  9, 3 and 7); zig fmt --check on the seven changed Zig files clean; tools/zig/lean_check.py 444 problems, 444 on
  main, 0 new. Failing receipt: the live-registry test against an earlier, stale record (expected 15, found 28; field regs incorrect); before this change the kernel is not in the registry at all.

• Not run, and why:

  • Metal: the change is CUDA-only, and there is no Mac here.
  • Tensor parallelism: ranks past 0 build no vision; a TP=2 server with images was not run.
  • tf-cuda-test radio row hashes on two image sizes: it needs float32 reference files we do not have. What is
    shown: the fused kernel's outputs equal the replaced launches' bytes on the contract's inputs, and the
    end-to-end reply hashes equal the base's on the served workloads. No intermediate tower bytes were captured, so
    whole-tower byte equality is not shown directly.
  • kernel-contract-limits on fatbins for sm_80/86/89/120: the GPU builds carry sm_121 only; the other records
    come from CPU ptxas builds.
  • Video and Bolt with images: not run on a GPU.

Checklist

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (Measured: full-size image replies byte-equal to the base with "draft": false, and drafted equal to "draft": false on 11 decode prompts, one with a 320 x 208 picture, with this commit in the build. Resumed and concurrent requests with images were not run.)
• No precision traded for speed. If the bits change, the description says which sums change and why. (Same operations and roundings by shared code; byte-identical to tf_vit_bias then tf_vit_pointwise in all 12 variants on the contract's inputs, including the planted non-finite values listed above, and equal reply hashes on the served workloads.)
• A new or changed GPU kernel is registered with the kernel contract (docs/kernel-contract.md); each backend's SUMMARY line is in the receipt. (CUDA only. All 12 variants pass gates 1-4; the aggregate run exits 1 on 1533 limit problems, none in a launch this change adds or touches, see above.)
• Prompt processing is no slower than the last release. (Not measured for this commit alone, and the base is main + nemotron cuda: a prompt pass with images runs the RADIO tower once, not twice #1057, not a release. With the pinned-stage change, image prompts are faster; text prompts never reach the tower.)
• The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not.
• New tests fail before the change, pass after it, and skip cleanly without their dependency. (The registry test's failing receipt is against a stale record, not main, where neither the kernel nor the test exists; the contract spec runs only under tf-cuda-test on a GPU.)
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines.

[ashhart]

Landed on main as 9e6e5fa with a maintainer fix-up (cb09c36) that registers the omitted BF16 bias-round control, thank you @plotarmordev. The RADIO tower's bias, activation and residual launches are fused. Served cells were exact, and the full CUDA kernel contract passes on a GB10: the fused kernel is within its limits and the new control fails as planted.

- Synthema, Ash's AI Assistant.

[ashhart]

This shipped in TensorFold 1.1.4: https://github.com/ashhart/TensorFold/releases/tag/v1.1.4

Thank you @plotarmordev for the contribution.

- Synthema, Ash's AI Assistant.