[laudney]

LightOnOCR-3, official BF16 weights, gfx1201	main	This PR
0.8B (qwen3_5)	refused at start: UnsupportedQuantization	served natively on HIP, with images
1B (mistral3 / LightOnOCR)	refused at start: no backend for mistral3 checkpoints	served natively on HIP, with images
4B (qwen3_5)	refused at start: UnsupportedQuantization	served natively on HIP, with images
Prompt ids vs the model card's Transformers example (grounding and transcription)	—	equal on all 3 sizes
Greedy transcription vs HF	—	equal on 1B and 4B. On 0.8B the first difference falls on an exact HF logit tie (22.25 = 22.25). There, HF with use_cache=True, or with deterministic eager attention, gives exactly the native ids
Greedy grounding vs HF	—	the text is equal, and box digits differ only at exact HF ties (margin 0.0 on all 3 sizes). HF's own repeat runs also differ at such ties
Fresh, resumed, drafted, concurrent and seeded runs	—	equal to each other on every size
Kernel contract on gfx1201	—	lighton-full 11/11 (7/7 controls), qwen-vision-full 10/10 (4/4 controls)

Part of #1927. This PR depends on #1962. It contains that PR's commit d385e8f8c (exact image identities in the prompt cache), because the 1B model has one bare <|image_pad|> per image row. Please review that commit in #1962.

Why

LightOnOCR-3 is a family of small, strong document OCR models. Serving the publisher's original BF16 weights matters for benchmarking and adoption: results then compare directly with the model card and with other engines, without a conversion step that users must trust. Before this PR, the native HIP engine refused all three sizes.

What is in it

Area	Change
zig/src/families/lighton_ocr/	1B: config, image processor and weight admission. Pixtral tower, Qwen3 decoder and the lane backend on HIP. reasoning = false, because the 1B template has no thinking switch
zig/src/families/qwen3_5/	0.8B/4B: the HF BF16 checkpoints load. The text rotary is picked by checkpoint format: HF BF16 text always uses Torch's M-RoPE, and MLX text keeps its kernels. Image prompts use Torch's M-RoPE, and an image lane's later rows decode at its M-RoPE shift
zig/src/vision/qwen_hip*.zig, grid.zig	A shared Qwen vision tower on HIP, plus Torch-exact taps for learned-position grids
zig/kernels/hip/lighton_ocr.hip, qwen_vision.hip	New HIP operators, each with a kernel-contract record and planted controls
zig/src/native/hip.zig, hip_vision_offer.zig	Registration and the image offer (a marker-free family reserves its placeholder)
tools/zig/lighton_ocr_reference.py	The pinned oracle. sources checks the checkpoint revision, the SHA-256 of every non-weight file, and the Transformers 5.22.1 model files plus the whole package tree, with no model load. examples runs the model card's Transformers example (greedy and the card's sampled call) and writes a token-hash receipt. capture writes the reference tensors for the HIP capture tools
docs/recipes/lighton-ocr.md	Build, serve, reference commands and limits

Behavior to know

• --no-thinking for 0.8B and 4B. These sizes load as qwen3_5, whose template opens a reasoning block when enable_thinking is true. The cards render with enable_thinking=False, so the recipe serves with --no-thinking. The receipt shows that the prompt ids then equal the card's. A request can still set chat_template_kwargs.
• --vision on a Qwen checkpoint whose tower the HIP engine cannot read now stops the start. Example: mlx-community/Qwen3.6-35B-A3B-4bit, whose tower has 72-wide heads. On main the server started text-only and refused image requests, because HIP had no Qwen image path. Now it prints --vision: this checkpoint's image tower is not one the HIP engine reads (64-wide heads, 16-pixel patches, 2x2 merge): serve it without --vision for text and exits 1. This is the same as the CUDA engine for this family. Text without --vision is unchanged: the receipt row for that checkpoint gives equal ids on main and on this branch.
• Known limit (MLX image lanes). For an MLX-quantized checkpoint with images, the prompt rotates with Torch's M-RoPE, and later rows use the MLX qkRope arithmetic at the M-RoPE shift. Their rounding can differ from a Torch reference after the prompt. No official MLX Qwen tower passes the tower check, and this PR qualifies no MLX image checkpoint. The follow-up is tracked in Native OCR models on HIP: dots.mocr, GLM-OCR, LightOnOCR-3 and Nemotron Parse, CUDA to follow (tracker) #1927.
• Prompt cache. The 1B resumes across requests (301/298 cached tokens in the receipt). Qwen image requests keep per-request pages and do not reuse state across requests.
• CUDA. Not in this PR. CUDA, Metal and other AMD GPUs are not qualified. Native OCR models on HIP: dots.mocr, GLM-OCR, LightOnOCR-3 and Nemotron Parse, CUDA to follow (tracker) #1927 tracks CUDA.

Receipt

The full device receipt is in the first comment: identities, pins, every row per size, the tie diagnostics, the kernel contracts and the regression rows. The receipt ran at 3f881c9a9. The two later commits change only the --vision refusal text and a usage line. Summary:

Check	0.8B	1B	4B
sources pins pass; the main binary refuses the checkpoint	yes	yes	yes
Prompt ids = HF, both modes	yes	yes	yes
Transcription greedy = HF	tie at id 89	yes	yes
Grounding greedy = HF	tie at id 14	tie at id 87	tie at id 11
HF logit margin at each first difference	0.0	0.0	0.0
Fresh = resumed = drafted = concurrent; seeded sampling equal over 4 cells	yes	yes	yes
Cancel in prefill and in decode (client-left), then an exact next reply	yes	yes	yes
Rejects (bad bytes, truncated JPEG, video_url) and the context edge (200/400)	yes	yes	yes

A native capture on 0.8B (HF prefix as teacher) gives stage cosines of 0.9993 to 0.9998 and 0 position differences. At transcription id 89, native ranks </ (22.25) over RM (22.125), and HF ranks them in the opposite order: one BF16 step.

Checks

Each commit builds native-hip. zig fmt --check, zig build test with and without -Dhip=true, and the host suites test-lighton-hip-host, test-qwen35-hip-host, check-qwen-hip-capture-host, test-lane-api-host, test-lighton-hip-contract-host, test-qwen-vision-hip-host, test-qwen-vision-contract-host and test-hip-blas-host pass. The registry test asserts reasoning for both families: false for the 1B, and true for Qwen3.5.

[laudney]

Device receipt for this PR. It ran at 3f881c9a9. The later commits d665cfe45 and 8c1eb8666 change only the --vision refusal text and the contract usage line. With them, the --vision row for mlx-community/Qwen3.6-35B-A3B-4bit exits 1 with --vision: this checkpoint's image tower is not one the HIP engine reads (64-wide heads, 16-pixel patches, 2x2 merge): serve it without --vision for text (checked on the same GPU).

LightOnOCR-3 native HIP device receipt (PR head 3f881c9a9, 2026-11-02)

One Radeon AI PRO R9700 (gfx1201), selected by ROCR_VISIBLE_DEVICES and checked by PCI address (0000:c3:00.0); other cards ran unrelated services. HF ids come from tools/zig/lighton_ocr_reference.py examples (the PR's pinned oracle), run on the same GPU. A = sha256 of comma-joined ASCII ids (the server's token_sha256). R = sha256 of little-endian u32 ids. Only 16 hex digits are shown; full values are in receipt.json.

identity	value
candidate	3f881c9a97ec (PR head, branch pr-ocr-lighton-ocr); tensorfold-native 1.2.1 sha256 96b5a5d60ded0a700d075d56c668d62305620c50d2983e2b6e752279bf5cf480
base	16d95ccec6a1 (merge base; upstream/main has since moved to 089a18dff); tensorfold-native 1.2.1 sha256 cc89e228481afbf3fa8f69bf225ebeedf2de29fe2bf6aa5e9e3bca604207bbff
build (both)	zig build native-hip -Dhip=true -Dhip-arch=gfx1201 -Dhipcc=/opt/rocm/core/bin/hipcc -Dhip-device-lib=/opt/rocm/core/lib/llvm/amdgcn/bitcode -Doptimize=ReleaseFast -j8
toolchain	Zig 0.17.0; ROCm 10.1.0; hipcc/HIP 7.16.26385; AMD clang 24.0.0git
libraries the server loaded	HIP libamdhip64.so.7.16.26385 bddbf3553ee64ae5; libamd_comgr.so.3.5.0 15f3113874725e9a; libhsa-runtime64.so.1.21.0 0457249a68424615; libjpeg-turbo 3.1.4.1 libturbojpeg.so.0.4.0 b6a5f4e6066da95c (same set for all 3 sizes; no rocBLAS, hipBLASLt or MIOpen)
checkpoint 0.8B	lightonai/LightOnOCR-3-0.8B @ 36d8636925327c671bd8e479c517a004f7566bd9; model.safetensors ae46aa6fd7136d57297ac006ed9ec345b7a2083feb2e99501bedcb014ca3efdf (equal to the HF LFS id)
checkpoint 1B	lightonai/LightOnOCR-3-1B @ 23d544085c886bbdace647bcfab91203299417e2; model.safetensors 32879861d5cdc3b2c3909d3f9a47dc8f7338685d13113c8c98c144f68b358dbb (equal to the HF LFS id)
checkpoint 4B	lightonai/LightOnOCR-3-4B @ b71010f095dc3735de7ee6b9969bff8f8f50b39a; model.safetensors 6b0dab74a2dab02d5b10ca56322255024ed8dca35c02746b42e9f296aaeb4e67 (equal to the HF LFS id)
sources --size 0.8B	exit 0, pins pass: 14 checkpoint files (config 842fafd8f0680726, chat_template 273d8e0e683b8850, tokenizer.json 06b9509352d2af50), 6 Transformers 5.22.1 files, package tree b08cead6d1b841ef
sources --size 1B	exit 0, pins pass: 12 checkpoint files (config 43b62f9ec0a7207f, chat_template 0942a43ee0ba48a6, tokenizer.json f54b55fa0c3aba0c), 8 Transformers 5.22.1 files, package tree b08cead6d1b841ef
sources --size 4B	exit 0, pins pass: 15 checkpoint files (config 99b4c363a998c2d8, chat_template d5ddb788d5f3df12, tokenizer.json 87a7830d63fcf43b), 6 Transformers 5.22.1 files, package tree b08cead6d1b841ef
image	model card SROIE-receipt.jpeg (hf-internal-testing/fixtures_ocr @ 28fe12cd) d66a3287c847823c6ddab9f8b2080582b96a66b7816d178472f697ea84e227b3
HF reference	Transformers 5.22.1, torch 2.15.0a0+rocm10.1.0a20260822, HIP 7.16.26332, SDPA, bfloat16, same GPU; greedy run 3 times per size (main + 2 repeats); 0.8B also has the older receipt from before the last pin change
server	serve <ckpt> --backend rocm --vision --context 4096 --parallel 2 --no-thinking --name lighton --host 127.0.0.1 --temperature 0 --max-tokens 1024 --no-update-check --idle-unload 0 --port <port> (the recipe flags; all exist in the native parser; serve --help prints a fixed usage line that omits --vision); each SIGINT exit 0; prompt cache default on
requests	data URL, return_token_ids: true, temperature 0, max_tokens 1024, draft set on every request (the server drafts by default); transcription = image only, grounding = image + text grounding
kernel contract	tf-hip-kernel-contract sha256 ec9cccd9ab2992cbb427a1a1db543a390e4728d962c548e797702acf49ab77ff, built in the candidate
re-run binaries	the Qwen3.5-4B re-run and the native capture used fresh builds of the same commits in clean worktrees: base ba69a43862d4e9f1cae531fdecf020c82df4d216bd5adac9df4800e179674806, candidate d788cc4e22851390f6f58d7b3c9d9f3347f1be70f808db1fddef6d2a34a006dc, tf-qwen-hip-capture 713dc41b82db474a1bcf1c2e09b26d5e8bd4550eacafbbb2baeb75de480b76d1 (these Zig builds are not byte-reproducible)
regression checkpoint Qwen/Qwen3.5-4B	@ 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a; model.safetensors-00001-of-00002.safetensors 26a93f066e1916adb13453dae5a0c707c0fbc71299ed98779571a907b8e74c61; model.safetensors-00002-of-00002.safetensors cb544bd9bfae93dc59b0f22b292f5933573854a7f9b97835c67060d7d910e188 (both equal to the HF LFS ids)

0.8B

check	native	HF / expected	equal
before: base serve, same flags	exit 1: the native HIP engine cannot load $HF_HUB/models--lightonai--LightOnOCR-3-0.8B/snapshots/36d8636925327c671bd8e479c517a004f7566bd9 (UnsupportedQuantization)	refusal before weight load	n/a
grounding prompt ids (212) via /v1/tokenize	A 2b47c54b83172aa4	A 2b47c54b83172aa4	yes
transcription prompt ids (210) via /v1/tokenize	A bf204a3fd4b810b6	A bf204a3fd4b810b6	yes
grounding greedy, fresh, stop on EOS	536 ids; A a98199524bf230d7 R 49de8eb17d18e879; first difference at output id 14; HF margin +0.000 logits (native token 24 25.375, HF token 23 25.375; HF top-2 margin 0.0); HF-vs-HF differs: older receipt at id 23; cache/eager variants at ids 22/11; 16 index-aligned positions differ: native  barcode vs HF  barcode	536 ids; A f4575dcd3693b947 R cd34b4e8596af57a; other HF greedy run(s): A 3281466b390e4ed7	no
transcription greedy, fresh, stop on EOS	358 ids; A 0247d9ae8d7cd714 R e2a093b02c9d99b0; first difference at output id 89; HF margin +0.000 logits (native token 510 22.25, HF token 29808 22.25; HF top-2 margin 0.0); HF-vs-HF differs: none (4 card runs equal); cache=True and eager variants at id 89, equal to native; 252 index-aligned positions differ: native <th>PRICE</th> vs HF <th>PRICE RM</th>	326 ids; A 6007f066a478a748 R 44568469eb755cc6	no
grounding, cache on: fresh (cached 0) then resumed (cached 0)	ids fresh = resumed: yes	HF	no
transcription, cache on: fresh (cached 0) then resumed (cached 0)	ids fresh = resumed: yes	HF	no
grounding: plain vs drafted (copy drafts, 91/352 accepted, 444 rounds vs 535)	drafted = plain: yes	HF	no
transcription: plain vs drafted (copy drafts, 80/277 accepted, 277 rounds vs 357)	drafted = plain: yes	HF	no
concurrent grounding ‖ transcription: plain / drafted	= native solo ×4: yes	HF solo	no (transcription no, grounding no)
seeded sampling grounding, T 0.2, seed 4242: solo, repeat, concurrent, concurrent drafted; 536 ids, stop	A 947f9a412ea85499 R 5d05ad61ca710303 ×4	equal across 4 cells; differs from native greedy (HF sampled ids for information only)	yes
seeded sampling transcription, T 0.2, seed 4242: solo, repeat, concurrent, concurrent drafted; 326 ids, stop	A 6007f066a478a748 R 44568469eb755cc6 ×4	equal across 4 cells; differs from native greedy; equals HF greedy (HF sampled ids for information only)	yes
client closes in prefill after the stream opens (fresh JPEG bytes), then transcription	chatcmpl-e7208a6daafc843927d7c59b80d60227 reason=client-left prompt=210 tokens=0 after=0.06s; next = native solo: yes	0 tokens, exact next = HF	no (next = native solo, which differs from HF)
client closes in decode after 8 content events, then transcription	chatcmpl-fd9c4a7bad840c3f2e6ef02058e4575e reason=client-left prompt=210 tokens=8 after=0.62s; next = native solo: yes	exact next = HF	no (next = native solo, which differs from HF)
context edge: 210 prompt + 3886 max_tokens / + 3887	200 (ids ≠ HF) / 400 context_length_exceeded	200 / 400	status yes; ids no (= native solo)
invalid image bytes / truncated JPEG	400 image 1 is not PNG, JPEG or WebP; send one of those / 400 image 1 could not be decoded; send PNG, JPEG or WebP	400 / 400	yes
video_url part, then transcription	400 content parts must be text or image_url; audio and video inputs are unsupported; next = native solo	400	yes

1B

check	native	HF / expected	equal
before: base serve, same flags	exit 1: the native HIP engine has no backend for mistral3 checkpoints yet; serve with --engine python	refusal before weight load	n/a
grounding prompt ids (304) via /v1/tokenize	A 16f80dab80d65533	A 16f80dab80d65533	yes
transcription prompt ids (301) via /v1/tokenize	A f9336b8386665ce1	A f9336b8386665ce1	yes
grounding greedy, fresh, stop on EOS	474 ids; A 0e3bc1d35e2bd7cf R 8d5fcd623d60bddd; first difference at output id 87; HF margin +0.000 logits (native token 18 18.125, HF token 16 18.125; HF top-2 margin 0.0); HF-vs-HF differs: none (3 card runs equal); cache=False / eager variants at ids 49 / 4; 5 index-aligned positions differ: native  Cashier : MANIS vs HF  Cashier : MANIS	474 ids; A 309be26bbeceaa17 R e9acca17d95b7b67	no
transcription greedy, fresh, stop on EOS	360 ids; A e471332fdc820629 R ce04b7f38ae325a4	360 ids; A e471332fdc820629 R ce04b7f38ae325a4	yes
grounding, cache on: fresh (cached 0) then resumed (cached 301)	ids fresh = resumed: yes	HF	no
transcription, cache on: fresh (cached 0) then resumed (cached 298)	ids fresh = resumed: yes	HF	yes
grounding: plain vs drafted (copy drafts, 63/211 accepted, 410 rounds vs 473)	drafted = plain: yes	HF	no
transcription: plain vs drafted (copy drafts, 62/181 accepted, 297 rounds vs 359)	drafted = plain: yes	HF	yes
concurrent grounding ‖ transcription: plain / drafted	= native solo ×4: yes	HF solo	no (transcription yes, grounding no)
seeded sampling grounding, T 0.2, seed 4242: solo, repeat, concurrent, concurrent drafted; 474 ids, stop	A 98dd63942d56d4b6 R 3288357c3f5a1b0a ×4	equal across 4 cells; differs from native greedy (HF sampled ids for information only)	yes
seeded sampling transcription, T 0.2, seed 4242: solo, repeat, concurrent, concurrent drafted; 360 ids, stop	A e471332fdc820629 R ce04b7f38ae325a4 ×4	equal across 4 cells; equals native greedy; equals HF greedy (HF sampled ids for information only)	yes
client closes in prefill after the stream opens (fresh JPEG bytes), then transcription	chatcmpl-c96fa6edd0f611d4eb33b381e0017733 reason=client-left prompt=301 tokens=0 after=0.08s; next = native solo: yes	0 tokens, exact next = HF	yes
client closes in decode after 8 content events, then transcription	chatcmpl-f662c7a02bd0d1e16319ab3444a549c5 reason=client-left prompt=301 tokens=8 after=1.77s; next = native solo: yes	exact next = HF	yes
context edge: 301 prompt + 3795 max_tokens / + 3796	200 (ids = HF) / 400 context_length_exceeded	200 / 400	yes
invalid image bytes / truncated JPEG	400 image 1 is not PNG, JPEG or WebP; send one of those / 400 image 1 could not be decoded; send PNG, JPEG or WebP	400 / 400	yes
video_url part, then transcription	400 content parts must be text or image_url; audio and video inputs are unsupported; next = native solo	400	yes

4B

check	native	HF / expected	equal
before: base serve, same flags	exit 1: the native HIP engine cannot load $HF_HUB/models--lightonai--LightOnOCR-3-4B/snapshots/b71010f095dc3735de7ee6b9969bff8f8f50b39a (UnsupportedQuantization)	refusal before weight load	n/a
grounding prompt ids (212) via /v1/tokenize	A 2b47c54b83172aa4	A 2b47c54b83172aa4	yes
transcription prompt ids (210) via /v1/tokenize	A bf204a3fd4b810b6	A bf204a3fd4b810b6	yes
grounding greedy, fresh, stop on EOS	516 ids; A e9aa2a3baa24f50a R f603eeab8d10e082; first difference at output id 11; HF margin +0.000 logits (native token 21 24.375, HF token 20 24.375; HF top-2 margin 0.0); HF-vs-HF differs: repeats 1 and 2 at id 13; variants at id 13; 6 index-aligned positions differ: native  Document No : TD01167104 vs HF  Document No : TD01167104	516 ids; A 183b9809dcff5a6a R dafbf60eaea38542; other HF greedy run(s): A 7effd8d032d7b6ec	no
transcription greedy, fresh, stop on EOS	402 ids; A aefc232f799fc06a R b1e921d81a2f7226	402 ids; A aefc232f799fc06a R b1e921d81a2f7226	yes
grounding, cache on: fresh (cached 0) then resumed (cached 0)	ids fresh = resumed: yes	HF	no
transcription, cache on: fresh (cached 0) then resumed (cached 0)	ids fresh = resumed: yes	HF	yes
grounding: plain vs drafted (copy drafts, 91/352 accepted, 424 rounds vs 515)	drafted = plain: yes	HF	no
transcription: plain vs drafted (copy drafts, 90/307 accepted, 311 rounds vs 401)	drafted = plain: yes	HF	yes
concurrent grounding ‖ transcription: plain / drafted	= native solo ×4: yes	HF solo	no (transcription yes, grounding no)
seeded sampling grounding, T 0.2, seed 4242: solo, repeat, concurrent, concurrent drafted; 516 ids, stop	A 76b6a8b0a32c52cd R f6f6f07ff5efa178 ×4	equal across 4 cells; differs from native greedy (HF sampled ids for information only)	yes
seeded sampling transcription, T 0.2, seed 4242: solo, repeat, concurrent, concurrent drafted; 402 ids, stop	A aefc232f799fc06a R b1e921d81a2f7226 ×4	equal across 4 cells; equals native greedy; equals HF greedy (HF sampled ids for information only)	yes
client closes in prefill after the stream opens (fresh JPEG bytes), then transcription	chatcmpl-2b6bca34c9f0ced27799af2f113acad8 reason=client-left prompt=210 tokens=0 after=0.07s; next = native solo: yes	0 tokens, exact next = HF	yes
client closes in decode after 8 content events, then transcription	chatcmpl-98cd9430d71281490d02d3889d07a765 reason=client-left prompt=210 tokens=8 after=5.88s; next = native solo: yes	exact next = HF	yes
context edge: 210 prompt + 3886 max_tokens / + 3887	200 (ids = HF) / 400 context_length_exceeded	200 / 400	yes
invalid image bytes / truncated JPEG	400 image 1 is not PNG, JPEG or WebP; send one of those / 400 image 1 could not be decoded; send PNG, JPEG or WebP	400 / 400	yes
video_url part, then transcription	400 content parts must be text or image_url; audio and video inputs are unsupported; next = native solo	400	yes

First-difference diagnostics (HF side diagnostics only; the card-settings run stays the reference)

Margins come from one teacher-forced HF forward (pinned model, processor, prompt; bf16, sdpa, use_cache=False) over the prompt and the HF card-run output; logits are bf16 values, so 0.125 is one step at this size.

point	HF top-2 at the point	logits of the two tokens	margin	one-pass HF argmax ≠ HF generate (whole reply)
0.8B grounding id 14 (native 24 vs HF 23)	24 9 25.375, 23 8 25.375	24: 25.375, 23: 25.375	top-2 0.0; between the two tokens 0.0	17 of 536 positions
0.8B grounding id 23 (HF older 15 vs HF card 20)	21 6 21.25, 16 1 21.125	15: 21.125, 20: 21.125	top-2 0.125; between the two tokens 0.0	17 of 536 positions
0.8B transcription id 89 (native 510 vs HF 29808)	29808 RM 22.25, 510 </ 22.25	510: 22.25, 29808: 22.25	top-2 0.0; between the two tokens 0.0	0 of 326 positions
1B grounding id 87 (native 18 vs HF 16)	17 2 18.125, 16 1 18.125	18: 18.125, 16: 18.125	top-2 0.0; between the two tokens 0.0	11 of 474 positions
4B grounding id 11 (native 21 vs HF 20)	21 6 24.375, 20 5 24.375	21: 24.375, 20: 24.375	top-2 0.0; between the two tokens 0.0	9 of 516 positions
4B grounding id 13 (HF repeats 24 vs HF card 23)	23 8 22.375, 22 7 22.375	24: 22.375, 23: 22.375	top-2 0.0; between the two tokens 0.0	9 of 516 positions

HF greedy variant (2 repeats each)	stable	first difference vs card run	equals native	first difference vs native
0.8B grounding: use_cache=true, sdpa	yes	id 22	no	id 14
0.8B grounding: deterministic algorithms, eager, card use_cache	yes	id 11	no	id 11
0.8B transcription: use_cache=true, sdpa	yes	id 89	yes (A 0247d9ae8d7cd714)	—
0.8B transcription: deterministic algorithms, eager, card use_cache	yes	id 89	yes (A 0247d9ae8d7cd714)	—
1B grounding: use_cache=false, sdpa	yes	id 49	no	id 49
1B grounding: deterministic algorithms, eager, card use_cache	yes	id 4	no	id 4
4B grounding: use_cache=false, sdpa	yes	id 13	no	id 11
4B grounding: deterministic algorithms, eager, card use_cache	yes	id 13	no	id 11

native capture, 0.8B (tf-qwen-hip-capture, HF prefix as teacher)	report	top-token rows that differ	native vs HF logits there
0.8B-transcription, 89 teacher ids	positions: 0/897 differences; logits: 1/90 top token differences; logits max_abs=1.234375e0 rms=6.961322e-2 cosine=0.999811004 different=20065964/22348800	id 89: native 510, HF 29808	see note
0.8B-grounding, 14 teacher ids	positions: 0/678 differences; logits: 3/15 top token differences; logits max_abs=8.437500e-1 rms=7.529624e-2 cosine=0.999516241 different=3471256/3724800	id 4: native 16, HF 17; id 5: native 22, HF 20; id 14: native 20, HF 23	see note

Kernel contract on gfx1201 (candidate)

contract	result	controls	equal
lighton-full	exit 0; SUMMARY lighton_ocr HIP full: 11 kernels, 11 passed, 0 failed; 7/7 controls caught; 0 limit problems	7 lines CONTROL …: red at …, as planted	yes
qwen-vision-full	exit 0; SUMMARY qwen_vision HIP full: 10 kernels, 10 passed, 0 failed; 4/4 controls caught; 0 limit problems	4 lines CONTROL …: red at …, as planted	yes

Regression rows (shared Qwen3.5 HIP code)

checkpoint	base	candidate	equal
Qwen/Qwen3.5-0.8B-Base @ dc7cdfe2 text, 3 prompts × 64 greedy	exit 1 the native HIP engine cannot load $HF_HUB/models--Qwen--Qwen3.5-0.8B-Base/snapshots/dc7cdfe2ee4154fa7e30f5b51ca41bfa40174e68 (UnsupportedQuantization)	exit 1 the native HIP engine cannot load $HF_HUB/models--Qwen--Qwen3.5-0.8B-Base/snapshots/dc7cdfe2ee4154fa7e30f5b51ca41bfa40174e68 (UnsupportedQuantization)	n/a (neither loads)
Qwen/Qwen3.5-4B-Base @ 1001bb4d text, 3 prompts × 64 greedy	exit 1 the native HIP engine cannot load $HF_HUB/models--Qwen--Qwen3.5-4B-Base/snapshots/1001bb4d826a52d1f399e183466143f4da7b741b (UnsupportedQuantization)	exit 1 the native HIP engine cannot load $HF_HUB/models--Qwen--Qwen3.5-4B-Base/snapshots/1001bb4d826a52d1f399e183466143f4da7b741b (UnsupportedQuantization)	n/a (neither loads)
Qwen/Qwen3.5-4B @ 851bf6e8, first run (shard 2 not yet on disk) text, 3 prompts × 64 greedy	exit 1 the native HIP engine cannot load $HF_HUB/models--Qwen--Qwen3.5-4B/snapshots/851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a (UnsupportedQuantization)	exit 1 the native HIP engine cannot load $HF_HUB/models--Qwen--Qwen3.5-4B/snapshots/851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a (UnsupportedQuantization)	n/a (neither loads)
Qwen/Qwen3.5-4B @ 851bf6e8, re-run with both shards (re-run binaries) text, 3 prompts × 64 greedy	exit 1 the native HIP engine cannot load $HF_HUB/models--Qwen--Qwen3.5-4B/snapshots/851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a (UnsupportedQuantization)	exit 1 the native HIP engine cannot load $HF_HUB/models--Qwen--Qwen3.5-4B/snapshots/851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a (UnsupportedQuantization)	n/a (neither loads)
mlx-community/Qwen3.6-35B-A3B-4bit (MLX affine 4-bit, qwen3_5_moe) text, 3 prompts × 64 greedy, /v1/completions	A e586c2cab4104019; A 13be50dc19feb1f2; A 2c51008dc9e04cde	A e586c2cab4104019; A 13be50dc19feb1f2; A 2c51008dc9e04cde	yes
same checkpoint, --vision, receipt image + text, greedy	server starts; image request 400 this server accepts text parts only; image, audio and video inputs are unsupported	exit 1 the native HIP engine cannot load $HF_HUB/models--mlx-community--Qwen3.6-35B-A3B-4bit/snapshots/38740b847e4cb78f352aba30aa41c76e08e6eb46 (UnsupportedQwenVision)	no: behavior change
Qwen/Qwen3.5-4B image row, candidate vs HF Transformers 5.22.1	n/a	not run: the candidate refuses the checkpoint at config parse (UnsupportedQuantization, mtp_num_hidden_layers 1; text serve above), so no image request can reach it	skipped

Notes

• Prompt ids equal HF for every size and mode, so --no-thinking renders the card template (enable_thinking=False) on 0.8B and 4B, and the 1B template needs no switch.
• Transcription greedy equals HF on 1B and 4B in every cell (fresh, resumed, drafted, concurrent, after both cancels, after the rejects).
• Every native-vs-HF first difference is at an exact HF logit tie: 0.8B grounding id 14 (25.375 = 25.375), 0.8B transcription id 89 (22.25 = 22.25), 1B grounding id 87 (18.125 = 18.125), 4B grounding id 11 (24.375 = 24.375). No margin at a native difference is above 0; none meets the 0.5-logit defect rule.
• HF itself moves at the same kind of ties: 0.8B grounding older receipt vs card run at id 23 (both tokens 21.125), 4B grounding card run vs both repeats at id 13 (22.375 = 22.375). In one teacher-forced pass HF does not reproduce its own generated token at 17/536 (0.8B grounding), 11/474 (1B grounding) and 9/516 (4B grounding) positions, almost all ties.
• 0.8B transcription: HF with use_cache=True, and HF with deterministic algorithms and eager attention, both give exactly the native ids (358 ids, A 0247d9ae8d7cd714), stable over 2 repeats each. Only the card setting (use_cache=False from the 0.8B generation_config, sdpa) gives the 326-id reply.
• Grounding: every HF variant is stable over its 2 repeats, but every variant differs from the card run (0.8B at ids 22/11, 1B at ids 49/4, 4B at id 13) and none equals native. Grounding box digits are tie-dense, so greedy grounding ids are not a stable cross-implementation target on this GPU.
• Native capture (0.8B, HF prefix as teacher): stage cosines 0.9993 (vision encoder) to 0.9998 (logits), 0 position differences. At transcription id 89 the native logits are </ 22.25, RM 22.125 and HF capture gives RM 22.25, </ 22.125: one bf16 step apart, in opposite order. At grounding id 14 native has a 4-way tie at 25.125 (20, 21, 23, 24) and HF a 2-way tie at 25.375 (23, 24).
• On 0.8B, native seeded sampling (T 0.2, seed 4242) gives the HF card-run transcription ids, while native greedy gives the HF use_cache=True ids; both are tie outcomes at id 89.
• Native results are self-consistent on every size: within a mode, fresh, resumed, drafted, concurrent plain and concurrent drafted greedy ids are equal, and the 4 seeded sampling cells are equal.
• Prompt cache: the 1B resumes 301/298 cached tokens. The 0.8B and 4B (Qwen3.5 path) report cached 0 on the resumed request; the recipe states that Qwen image requests keep per-request pages and bypass cross-request reuse.
• Drafts: the server reports drafts: copies only for all sizes (no MTP head in these checkpoints); drafted ids equal plain ids in every cell.
• Cancel: both cuts log reason=client-left (prefill: 0 tokens; decode: 8 tokens) and the next request equals the native solo reply.
• Regression: both binaries refuse the official HF BF16 Qwen3.5 checkpoints, also Qwen/Qwen3.5-4B with both shards (UnsupportedQuantization): the base reads only MLX-quantized Qwen3.5, and the candidate admits HF BF16 only with mtp_num_hidden_layers 0 (these have 1). The MLX Qwen3.6-35B-A3B-4bit text row is the only local checkpoint both binaries load; its ids are equal. The Qwen3.5-4B image row could not run for the same reason.
• Behavior change: with --vision, the base serves the MLX Qwen3.6-35B-A3B-4bit checkpoint text-only (image request 400), while the candidate refuses to start (UnsupportedQwenVision, from vision/qwen_hip.zig check: the tower needs head dim 64).
• The server prints warning: libtfgrammar.so is required for structured output ... at startup (0.8B, 4B); no request here uses structured output. tf-hip-kernel-contract usage text omits the qwen-vision-smoke and qwen-vision-full modes, which exist and pass.

[ashhart]

Thanks @laudney. We reviewed this independently, and the family itself holds up. The HF BF16 Qwen 3.5 loader is HIP-only and refuses MLX BF16 by tensor name. LightOnOCR refuses other mistral3 checkpoints by name, and every shape is checked before the budget and allocation. The Pixtral processor matches your account, and both contracts pass with their controls. Three changes:

1. Lean is 432 against main's 424: 5 problems in tools/zig/lighton_ocr_reference.py, plus core/qwen_vl/grid.zig:19 and :58, and qwen3_5 forward/window.zig:13.
2. A regression row. Every Qwen 3.5 HIP decode round now carries rope plan words, and MLX text reads qkRope positions from them. The values equal the old position words for text lanes, but the receipt's only regression row is the MLX Qwen3.6-35B-A3B MoE. Please add equal-ids rows for a dense MLX Qwen 3.5 (27B or 9B), drafts on and off, at 1 and 4 streams.
3. Shared plumbing once. The HIP vision and cache wiring overlaps dots.mocr on HIP: the publisher's BF16 OCR checkpoint, token-exact against its model card #1938 in seven shared files, with different mechanisms. Please land one shared HIP-OCR plumbing PR (with core: keep exact image identities beside prompt-cache words #1962 and core/cancel.zig) before the families build on it.

And our call on --vision: this PR makes it stop the start for an unreadable Qwen tower, and for Flash Next and Nemotron on HIP, where main starts text-only. Refusing an unreadable tower is right, since that checkpoint is broken. But for families with no HIP vision, please keep main's behaviour: start text-only, with a clear log line saying images are refused. Refusing to start would break users who pass --vision routinely.

[laudney]

Thanks, Ash. Status per point:

Point	State
1. Lean	Fixed in e549cba9d: each new comment and docstring is one line. lean_check.py now reports 424, the same as main.
2. Regression rows	Running now on gfx1201 with mlx-community/Qwen3.5-9B-4bit (dense, MLX 4-bit): base vs this branch, drafts on and off, at 1 and 4 streams. I will post the table here when it finishes.
3. Shared plumbing once	Agreed. I am building one HIP-OCR plumbing PR from main. More than #1938 and #1965 overlap: #1940 has its own "share native open and image offers" refactor, and #1941 has shared hip_serve load options and dense vision operators that dots also uses. So the plumbing PR takes one mechanism per concern from all four: one image open and offer path, family refusals and model-type aliases, the prompt-cache pieces (#1962 and the warm rule), core/cancel.zig, and the shared HIP vision operators with their contract records.
--vision	Fixed in a3db28505. A HIP family with no image path starts text-only again and logs --vision: the HIP engine has no image tower for <model_type>; serving text only, and image requests are refused; image requests get main's 400. The condition follows the family's Loaded type, not a list. Checked on gfx1201 with a nemotron_h checkpoint: text 200, image 400. An unreadable Qwen tower still stops the start, with a message.

On history: I will not force-push the model PRs. When the plumbing PR is open, I will say which commits of each model PR it replaces, so you can rebase them as you did after #1933. If you prefer that I push rebased branches instead, tell me and I will do it.

[ashhart]

Thanks @laudney. The 21:41 push answers two of our asks: lean is back to 424, and --vision on a HIP family with no image tower now starts text-only with the log line. The new flagRefused test covers Nemotron, Flash Next, Qwen 3.5, Qwen 3.5 MoE and LightOnOCR. An unreadable Qwen tower still refuses the start by name (UnsupportedQwenVision), as we asked.

Still open:

1. Equal-ids rows for a dense MLX Qwen 3.5 (27B or 9B), drafts on and off, at 1 and 4 streams. The only regression row so far is the Qwen3.6-35B-A3B MoE, and this PR changes the shared Qwen 3.5 HIP path.
2. One shared HIP-OCR plumbing PR before the families. The vision and prompt-cache wiring is still in this PR, and dots.mocr on HIP: the publisher's BF16 OCR checkpoint, token-exact against its model card #1938, GLM-OCR on HIP: the publisher's BF16 checkpoint, exact against its six model-card tasks #1940 and core: keep exact image identities beside prompt-cache words #1962 touch the same shared files (build/hip.zig, native/hip.zig, core/prompt_cache.zig). It's 38 commits behind main, and those are the files a rebase will conflict on. GLM-OCR on HIP: the publisher's BF16 checkpoint, exact against its six model-card tasks #1940 lands first, so this can rebase onto it.
3. One point on --vision: a Qwen 3.5 checkpoint with no tower at all now stops the start (NoQwenVision). On main it started text-only, and a text-only checkpoint isn't broken, so it should get the same text-only start and log line as the no-tower families. Keep the refusal for UnsupportedQwenVision only.

[laudney]

Thanks, Ash.

Point	State
3. --vision on a Qwen 3.5 checkpoint with no tower	Fixed in 490dc39b4. The engine decides once whether config.json declares a tower. With vision_config absent or null, it serves text and logs --vision: the HIP engine has no image tower for qwen3_5; serving text only, and image requests are refused. A declared tower it cannot read, or a malformed vision_config, still refuses with the UnsupportedQwenVision message. Checked on gfx1201: a copy of mlx-community/Qwen3.5-9B-4bit without vision_config starts and answers text 200 / image 400. The real 9B (72-wide tower heads) refuses.
1. Dense MLX Qwen 3.5 rows	Still running on mlx-community/Qwen3.5-9B-4bit: base vs this branch, drafts on and off, 1 and 4 streams. I will post the table here.
2. Shared plumbing	In progress, and I will base it on main after #1940 lands, as you suggest. GLM's Loaded.vision/.cache fields become one defaulted Offer value that the host reads once. A family refusal hook, model-type aliases, a cancellation helper on the existing CancelCheck, and a contract registration list complete it. Then #1938, #1941 and this PR adopt it, and this PR drops its own copies (hip_vision_offer.zig, core/cancel.zig, the native/hip.zig hunks).

[laudney]

Point 1: dense MLX Qwen 3.5 rows. Base 16d95ccec (this PR's merge base) and this PR's head 490dc39b4 give equal greedy ids in every cell, on one Radeon AI PRO R9700 (gfx1201).

identity	value
checkpoint	mlx-community/Qwen3.5-9B-4bit @ 8b2b98c00a6b (dense, MLX affine 4-bit); shards a68b87558c6ef43f, b0a770bf8469c7f3
binaries	base 95939d4ea9023048, candidate d8e90d2155630daa (zig build native-hip -Dhip=true -Dhip-arch=gfx1201 ... -Doptimize=ReleaseFast)
server	serve <ckpt> --backend rocm --context 4096 --parallel {1,4} --temperature 0 plus --no-drafts for the plain cells; the server reports drafts: copies only (the MLX checkpoint ships no MTP weights) or drafts: off, and 1 stream at once / 4 streams at once
requests	/v1/chat/completions, three prompts, temperature 0, max_tokens 64 (every reply hits the cap, 64 ids), return_token_ids: true; at 4 streams the three prompts plus a repeat of the first go in at once

prompt	A (comma-joined ids)	R (u32 LE)	base: plain, drafted × 1, 4 streams	candidate: plain, drafted × 1, 4 streams
sky	e419a57dfa7fbfd7	ee740cb42e46f940	4/4 equal	4/4 equal
Fibonacci	5d9afa7be90f6fb6	58131651d4ddfdcb	4/4 equal	4/4 equal
TCP/UDP	e910bc7c42653cc9	0878c498f255926f	4/4 equal	4/4 equal
sky, repeated beside the others at 4 streams	e419a57dfa7fbfd7	ee740cb42e46f940	2/2 equal	2/2 equal

All 32 id arrays are equal across binaries, draft settings and stream counts. So the round's rope plan words feed MLX text lanes the same positions as before.

[ashhart]

Thanks @laudney: the dense Qwen3.5-9B rows equal base in every cell, and NoQwenVision now takes the text-only start with the log line when vision_config is absent or null, while an unreadable tower still refuses. One ask left, the one you agreed: the shared HIP-OCR plumbing as its own PR after #1940, with #1941 and this taking its form (#1941 still carries the older text-only check). There's one mechanical build/hip.zig conflict on current main.

[laudney]

The shared HIP-OCR plumbing is up as #2040, on main c7952f398, after #1940.

what it gives	one hip_serve.Offer in a family's Loaded, a refusal(o) hook, model-type aliases, hip_serve.vision.offer, hip_serve.cancel, per-source strict flags, and family contract entries run by tf-hip-kernel-contract image <group>
GLM-OCR	converted in the same PR (offer, refusal hook, and the first contract entry), so main keeps one path
families on main	the same 52 code objects (sha256), the same contract summary, and equal greedy ids for Qwen3.5-9B and Nemotron Lightning at 1 and 4 streams; the GLM-OCR receipt driver gives 65/65 on both binaries, with all 62 id hashes equal

Once #2040 lands, this PR merges it and takes its form: .offer with hip_serve.vision.offer for lighton and qwen3_5, hip_serve.cancel instead of its own cancel type, refusal(o), and image entries for its contract modes. #1941 does the same. No force-push.