[gilby]

What this changes

The vision tower's Metal kernels (vision.metal: epilogues, RoPE, attention, SwiGLU, the aligner's unshuffle, the stream merge), registered with the kernel contract.

Each of the six kernels carries a planted control, required to fail its gate: the epilogue's bias dropped or GELU replaced, sin and cos swapped in qkv, the attention max not subtracted from the exponent, gate and up swapped in swiglu, an unshuffle stride off by one, and a stream-merge delimiter one row late.

Part 17 of 18 of the stack for #1676; stacks on dsv41-metal-pr/16-metal-serving (part 16, #1720), on main with parts 1 to 5 (#1705 to #1709) merged.

Receipt

• Environment, with the TensorFold commit, MLX or PyTorch version, machine and GPU, checkpoint and revision: TensorFold ac6a263 on main ad98b60 (native Zig 0.17.0; no MLX or PyTorch); Apple M3 Ultra (60-core GPU), 96 GB, macOS, for the kernel and device tests; checkpoint deepseek-ai/DeepSeek-V4.1-Flash-1017.
• Exactness, from tools/bench_concurrent.py --alone --serial or token_sha against "draft": false: n/a: kernels only; the kernel contract checks each launch geometry's bits against the others and against float64.
• Decode speed before and after, from tools/bench_openai.py: measured on the last PR of the stack.
• Prompt speed before and after: the cold first request's ttft_s from tools/bench_cache.py --require-cold on a long prompt, or "n/a" with the reason when the change cannot reach prompt processing: measured on the last PR of the stack.
• Tests run, with pass, fail and skip counts: zig build test tf-kernel-contract -Dcpu=apple_m1: exit 0, 627/627 steps succeeded; kernel contract, tf-kernel-contract --only dsv41 (Metal): SUMMARY 98 kernels: 98 pass gates 1-4, 0 fail, 0 known findings; 192 controls: 192 red as planted, 0 missed; 0 limit problems; gates 120204 ms, limits 153 ms; zig build test-dsv41-metal-all -Dcpu=apple_m1 (every DeepSeek Metal device step): exit 0, 9/9 steps succeeded; 37/37 tests passed; python3 tools/zig/lean_check.py: 5149 files, 424 problems, the same count as main (5027 files, 424). Zig's summary counts only the test binaries it ran in that invocation, so the step count is the comparable figure.
• Not run, and why: CUDA: no NVIDIA device here; Metal on M1, M2, M4 and M5: one M3 Ultra only; the release-layer checks (test-dsv41-metal-real, test-dsv41-metal-vision-real) were run at the stack's top and are in dsv41 metal: image prompts #1722's receipt.

Checklist

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo.
• No precision traded for speed. If the bits change, the description says which sums change and why.
• A new or changed GPU kernel is registered with the kernel contract (docs/kernel-contract.md); each backend's SUMMARY line is in the receipt.
• Prompt processing is no slower than the last release.
• The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not.
• New tests fail before the change, pass after it, and skip cleanly without their dependency.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines.

[ashhart]

Thanks @gilby. The six vision kernels are gated against float64 from the same BF16 inputs with NaN and both infinities, multi-row launches equal one launch, and launch records are checked for every Apple family. One change: none of the six has a planted control. Our contract run at #1722 shows 98 kernels with the same 178 controls as 89 kernels before; vision.metal declares TF_CTL but no kernel branches on it. Please add one control per kernel, each required to fail its gate: bias dropped or GELU replaced in epi, sin/cos swapped in qkv, max not subtracted in attn, gate/up swapped in swiglu, a stride off by one in unshuffle, a delimiter one row late in merge.

[gilby]

Thanks @ashhart. One planted control per kernel, each required to fail its gate: epi (bias dropped or GELU replaced), qkv (sin and cos swapped), attn (max not subtracted), swiglu (gate and up swapped), unshuffle (a stride off by one) and merge (a delimiter one row late). At this part: SUMMARY 98 kernels: 98 pass gates 1-4, 0 fail, 0 known findings; 192 controls: 192 red as planted, 0 missed. One note on the attention control: dropping the max from both the exponent and the rescale is a softmax no-op (my first pin was CONTROL MISSED), so the control keeps the max in the rescale and drops it from the exponent only, which is the fault's effect on the data; I can phrase it another way if you prefer. Each fault was checked by planting it in the kernel and restoring it. The receipt in the body is at this head (ac6a2631, on main ad98b602).