[gilby]

What this changes

Image prompts: the ViT and aligner on Metal (--vision), bias_vl routing for image rows, Engram skipped on image tokens, an image span prefilled in one chunk, the prompt cache keyed by image hash, and the server's image token for image parts.

Part 18 of 18 of the stack for #1676; stacks on dsv41-metal-pr/17-vision-kernels (part 17, #1721), on main with parts 1 to 5 (#1705 to #1709) merged.

Receipt

• Environment, with the TensorFold commit, MLX or PyTorch version, machine and GPU, checkpoint and revision: TensorFold 65bb881 on main ad98b60 (native Zig 0.17.0; no MLX or PyTorch); Apple M3 Ultra (60-core GPU), 96 GB, macOS, for the kernel and device tests; Apple M3 Ultra (80-core GPU), 512 GB for full-model numbers; checkpoint deepseek-ai/DeepSeek-V4.1-Flash-1017.
• Exactness, from tools/bench_concurrent.py --alone --serial or token_sha against "draft": false: the tiny model served through LaneHost in test-dsv41-metal-serve: concurrent == solo == serial, drafted == "draft": false, prompt-cache hits == fresh runs, seeded replies repeat, a cancelled prompt leaves the next request valid.
• Full model (Apple M3 Ultra, 80-core GPU, 512 GB; deepseek-ai/DeepSeek-V4.1-Flash-1017 release weights; serve --parallel 4 --context 1048576 --prompt-cache-gib 8 --vision, DSpark drafts on; measured at the stack's top before the rebase onto main 3ce4a04, 60d51fd, the same 13 commits on main 9e8c368 (git range-diff shows each patch unchanged except part 10's one planted-control line, TF_CTL == 4 only, which the served kernel compiles out)): tools/bench_concurrent.py --tokens 128 --temperatures 0,1 --levels 1,4 --alone --serial: 8 cells, 0 failed, 0 unequal (alone 1/1 or 4/4 and serial 1/1 or 4/4 equal in every cell). The memory gate, 4 cold 912K-token prompts at once in 4 slots of 1,048,576 positions (all four recall their needle, about 333 GB of GPU memory at peak), is from the 29 Oct stack's top (5db73126) and was not re-run on this head.
• Real release weights (Apple M3 Ultra, 80-core GPU, 512 GB; deepseek-ai/DeepSeek-V4.1-Flash-1017 release shards; a5e2ec9): TF_DSV41_REAL=<checkpoint> zig build test-dsv41-metal-real on layer 2 (compress ratio 2) and layer 21 (ratio 1): exit 0, 5/5 steps, 4/4 tests each; every stage at 1 and 4 rows within its float64 bound, the closest margins at 1.000 of the bound (layer 2: shared SwiGLU 6.103e-5 vs 6.104e-5, routed + shared join 2.441e-4 vs 2.441e-4, hc post 3.906e-3 vs 3.907e-3; layer 21: shared SwiGLU 1.221e-4 vs 1.221e-4, join 4.883e-4 vs 4.883e-4, hc post 1.953e-3 vs 1.953e-3); mapped weights give the same output as copied weights, bit for bit, at 4 rows.
• Real vision tower (same machine and checkpoint, a5e2ec9): TF_DSV41_REAL_VISION=<checkpoint> zig build test-dsv41-metal-vision-real: exit 0, 5/5 steps, 14/14 tests; the release tower against float64 after 32 of 32 blocks: relative RMS error 0.025, worst element 1.305 of its row's RMS (the worst stage is after 16 blocks: 0.033, worst element 2.834); tower and aligner on a 6x9 patch grid: relative RMS error 0.012, worst element 0.071; image_start, image_end and image_newline load; the prompt checks in the same step (chunk sizes 1, 7, 64, 900; cone vs full; saved and resumed; turns; grouped experts vs row kernels) show 0 differing logits, taps and state bytes.
• Decode speed before and after, from tools/bench_openai.py: no Metal DeepSeek-V4.1 backend before this stack. After (60d51fd, the same patches before this rebase, --tokens 256 --reps 3 --temperatures 0,1, median): drafts on, fibonacci-raw 53.8 tok/s at T=0 and 50.5 at T=1, gpu-chat-no-think 43.8 at T=0 and 44.2 at T=1; drafts off (serve --no-drafts, one token a round), fibonacci-raw 35.4 at T=0 and 34.8 at T=1, gpu-chat-no-think 35.4 at T=0 and 34.6 at T=1. bench_concurrent aggregate at 1 / 4 streams: code 46.9 / 78.5 at T=0 and 52.1 / 61.4 at T=1; chat 43.2 / 76.1 at T=0 and 47.5 / 62.2 at T=1.
• Prompt speed before and after: no Metal DeepSeek-V4.1 backend before this stack. After (60d51fd, the same patches before this rebase): tools/bench_cache.py --prompt-file <32,768 token ids> --expected-prompt-tokens 32768 --require-cold --require-reuse --tokens 64 --repeats 2: cold first request ttft_s 42.76 (about 766 tok/s), cached_tokens 0; repeats 0.028 and 0.029 s and two concurrent 0.029 and 0.056 s with cached_tokens 32,768, the same tokens in every row; qualified.
• Tests run, with pass, fail and skip counts: zig build test tf-kernel-contract -Dcpu=apple_m1: exit 0, 627/627 steps succeeded; kernel contract, tf-kernel-contract --only dsv41 (Metal): SUMMARY 98 kernels: 98 pass gates 1-4, 0 fail, 0 known findings; 192 controls: 192 red as planted, 0 missed; 0 limit problems; gates 126620 ms, limits 229 ms; zig build test-dsv41-metal-all -Dcpu=apple_m1 (every DeepSeek Metal device step): exit 0, 12/12 steps succeeded; 60/60 tests passed; python3 tools/zig/lean_check.py: 5159 files, 424 problems, the same count as main (5027 files, 424). Zig's summary counts only the test binaries it ran in that invocation, so the step count is the comparable figure.
• Not run, and why: CUDA: no NVIDIA device here; Metal on M1, M2 and M4; the kernel contract was not re-run on an Apple M5 Max at this head; time-dsv41-metal-real (timing only) was not part of this receipt; the two real-weights checks above ran at a5e2ec9, the head before the review round, and are not re-run at this head yet.

Checklist

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo.
• No precision traded for speed. If the bits change, the description says which sums change and why.
• A new or changed GPU kernel is registered with the kernel contract (docs/kernel-contract.md); each backend's SUMMARY line is in the receipt.
• Prompt processing is no slower than the last release.
• The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not.
• New tests fail before the change, pass after it, and skip cleanly without their dependency.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines.

[gilby]

The full-model receipt in #1720 and #1722 is now from the stack's final head, 60d51fd7 on main 9e8c368b (Apple M3 Ultra, 80-core GPU, 512 GB; deepseek-ai/DeepSeek-V4.1-Flash-1017 release weights; serve --parallel 4 --context 1048576 --prompt-cache-gib 8 --vision), replacing the numbers from the same stack before its rebase. The bodies carry the full lines; in short:

• tools/bench_concurrent.py --tokens 128 --temperatures 0,1 --levels 1,4 --alone --serial: 8 cells, 0 failed, 0 unequal.
• tools/bench_openai.py --tokens 256 --reps 3 --temperatures 0,1, medians: drafts on, fibonacci-raw 53.8 / 50.5 tok/s and gpu-chat-no-think 43.8 / 44.2 (T=0 / T=1); --no-drafts 35.4 / 34.8 and 35.4 / 34.6.
• tools/bench_cache.py --require-cold --require-reuse, 32,768 tokens: cold ttft_s 42.76 (about 766 tok/s), reuse 0.028 to 0.056 s with cached_tokens 32,768; qualified.
• Release weights at the same head: layers 2 and 21 against float64 4/4 each with mapped == copied bit for bit, and the vision tower 14/14.

Each number is within 1% of the pre-rebase run. The 4 x 912K memory gate is still the 29 Oct run, as the bodies say.

[ashhart]

Our Metal kernel contract over the whole stack (tf-kernel-contract --only dsv41 at #1722's head, M5 Ultra): 98 kernels pass gates 1-4, 0 fail, 0 known findings; 178 controls, 178 red as planted, 0 missed; 0 limit problems. With #1713's run (65 kernels, 121 controls) that covers every DeepSeek Metal kernel in the stack on device. The open asks are #1712's route_select controls, #1715's FFN controls and #1717's rebase artifact; #1719-#1722 are in review now.

[ashhart]

Thanks @gilby. The image path holds: RenderOptions.images defaults false and is set only on the DeepSeek-V4.1 path, the image token renders once and forged marks are refused, images go through core's picture.decode with token, aspect and count limits, and the identity is content bytes plus tag plus grid, never a path (opt into #1962's exact identities once it lands). One ask, on receipts: drafted = draft:false is shown only on the tiny model, and never-slower only at 1 stream. On the full model with fresh servers and --prompt-cache-gib 0, please post drafts against --no-drafts at T 0 and 1 with 1 and 4 streams (equal token_sha256), plus the cold first request both ways, with no cell slower.

[ashhart]

Thanks @gilby for the push. We re-reviewed all 13 parts on current main.

Met and ready, pending our runs: #1710-#1719 and #1721. Your asks are met: #1710's embed bound, #1712's route_select controls, #1715's four FFN faults, #1717's rebase artifact and #1721's per-kernel vision controls.

Device suite at this head (zig build test-dsv41-metal, M3 Ultra): 55/55 tests pass. Every planted fault is red: FFN 4 of 4, attention 3 of 3, forward 3 of 3, DSpark 3 of 3. The Metal kernel contract (--only dsv41, all 184 controls) and the full test suite at this head are queued on our M5 Ultra.

Still open, on #1720 and #1722, the same receipts as before. On the full DeepSeek-V4.1-Flash-1017 weights, with a fresh server per cell and --prompt-cache-gib 0:

• drafts against --no-drafts at T 0 and T 1, at 1 and 4 streams, with equal token_sha256 and tok/s for each, and no drafted cell slower;
• one cold 32,768-token first request with drafts and with --no-drafts, giving ttft_s and tok/s.

The posted bench_concurrent run (drafts on, 8 cells, 0 unequal) shows concurrent = solo = serial, but not drafted against undrafted. We don't have a 256 GB Mac with these weights, so these numbers need to come from your M3 Ultra. Once they're in, the stack lands in order.

[ashhart]

Metal kernel contract at this head (tf-kernel-contract --only dsv41, M5 Ultra): 98 kernels pass gates 1-4, 0 fail, 0 known findings. All 192 planted controls fail as planted, 0 missed, with 0 limit problems. The full test suite at this head is running now. If it passes, #1710-#1719 land in order, and #1720-#1722 follow once their receipts are in.