[jayleaton]

Stacked on #1766 (Idol's denoiser on Metal). This branch holds #1766's commits too; its own change is the last two commits. Independent of #1767, #1773 and #1774.

What this changes

Idol's DINOv3 ViT-H+/16 image encoder on Metal: the SDK's 1024 x 1024 preprocessed image to the denoiser's feature1 [4101, 1280] (class token, four registers, 64 x 64 patches), as idol-sdk's ImageEncoder builds it, with tf-idol-dino and an idol_case.py dino case. For #1757. Details: the "DINOv3 encoder" part of docs/recipes/idol.md.

• What the SDK feeds the denoiser: transformers' DINOv3ViTModel in bf16 (dinov3_dtype='bf16', the pipeline's default), its last_hidden_state (the model's own final LayerNorm), then a plain fp32 F.layer_norm (eps 1e-5, no affine).
• Patch embedding as one GEMM; 32 blocks of width 1,280, 20 heads of 64, LayerScale, SwiGLU of 5,120 (gate and up as one 10,240-column GEMM); RoPE on the 4,096 patches with transformers' fp32-generated periods (not the bf16 periods in Meta's checkpoint), tables equal to torch's bit for bit.
• zig/kernels/metal/idol_dino.metal, eight kernels of Idol's own. No shared kernel changes. Attention keeps fp32 scores and sums and rounds P to bf16 itself before P V, summing the denominator over those values (M5 tensor units truncate fp32 operands to 10 mantissa bits: gemm_nax<float> is not fp32, and attention_nax's outputs are biased low #1772).

Weights and licence

timm/vit_huge_plus_patch16_dinov3.lvd1689m at dd0addc09788111fa893d24799a44744ba022eee, ungated, rounded to bf16 at load as from_pretrained(dtype=bfloat16) rounds. These weights are under Meta's DINOv3 License, not Apache 2.0 (its LICENSE.md ships with them); the recipe says so.

The SDK reads Meta's gated facebook/dinov3-vith16plus-pretrain-lvd1689m. idol_case.py dino --official DIR checks that timm's file is the same model: all 550 mapped tensors (the 32 fused qkv weights as 96 thirds) equal Meta's byte for byte, Meta's q and v biases are all zero, and its only other unmapped tensor is mask_token. feature1 from timm's weights equals feature1 from Meta's from_pretrained bit for bit on MPS, and both equal the features the SDK pipeline recorded for the image.

Precision

The kernels round where torch on MPS rounds, measured on random bf16 operands against float64: a linear once after its bias, LayerNorm once, silu(g) * u twice, the rotary three times, LayerScale with the residual twice; fp32 sums and statistics. torch's bf16 SDPA on MPS matches no candidate rounding scheme (it sits about three times a bf16 rounding from the exact output), so the attention is gated on distance, not bits.

Kernel contract

zig/tests/contract/metal_idol_dino*.zig: dn_linear_0, dn_linear_1, dn_norm, dn_final, dn_heads, dn_scale_add, dn_swiglu, dn_attention, each at transformers' bf16 points, with eleven controls (a bf16 rounding added or dropped, fp16 sums in the norms, P V rounded to bf16 each key block, the attention's tail keys unmasked).

SUMMARY 8 kernels: 8 pass gates 1-4, 0 fail, 0 known findings; 11 controls: 11 red as planted, 0 missed; 0 limit problems

Receipt

• Environment: Mac Studio M5 Ultra 256 GB, macOS 27; this branch on idol: Idol's denoiser as a native Metal family #1766's 94a039249; idol-sdk 4702a7f0 on torch 2.13.0 (MPS, ATTN_BACKEND=sdpa), transformers 5.22.1 (sha256-pinned DINOv3 files); the SDK's preprocessed avocado example.
• Exactness, against a float32 CPU ruler (the M5's fp32 GEMM is not fp32, M5 tensor units truncate fp32 operands to 10 mantissa bits: gemm_nax<float> is not fp32, and attention_nax's outputs are biased low #1772), each gate within twice torch bf16's own distance, worst row and 1 - cos:
  • feature1 from the image: our worst row 0.1353, torch's 0.1686 (both row 4038, the bottom-left corner patch); 1 - cos 5.37e-5, torch's 5.57e-5. Against the SDK's own features: cosine 0.999954, worst row 0.109;
  • each of the 32 blocks alone on torch's bf16 input: all pass, our worst row torch's times 0.82 to 1.24, our 1 - cos torch's times 0.84 to 1.06;
  • the embedding and the last norm alone: equal to torch's at the printed digits (our embedding is not byte-identical to torch's; I did not find why);
  • 10 encodes repeat byte for byte; dino_reference.zig's test plants a 2% error in one row and the gate fails it, though cosine alone would pass.
• Speed: 0.120 s an encode (median of 10; first 0.150 s) against the SDK's 0.261 s on MPS (first 0.385 s): 2.2x. Load 0.2 to 0.6 s.
• Memory: 2.28 GB peak footprint (/usr/bin/time -l), 1.68 GB of it the bf16 weights; the SDK's MPS encode holds 2.24 GB of driver memory.
• Tests: zig build test test-golden -Dcpu=apple_m1: 585 of 585 steps, 0 fail; goldens 55 of 55 and 322 of 335, 0 unexpected differences (before the rebase onto 94a039249, a docs-only change; after it, plain zig build 248 of 248). zig fmt --check build.zig zig: clean. lean_check.py: 424, the same as main.
• Not run: CUDA (none in this PR); Metal M1 to M4.

Checklist

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off. (No token path.)
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (n/a.)
• No precision traded for speed. If the bits change, the description says which sums change and why.
• A new or changed GPU kernel is registered with the kernel contract (docs/kernel-contract.md); each backend's SUMMARY line is in the receipt.
• Prompt processing is no slower than the last release. (Not touched.)
• The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not.
• New tests fail before the change, pass after it, and skip cleanly without their dependency.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines.

[ashhart]

Thanks @jayleaton. All 32 blocks pass alone, and the check that timm's file equals Meta's byte for byte is exactly the evidence we needed for the weights. The DINOv3 License terms are fine for us, as the recipe documents them. It waits on #1766; please rebase after it lands, and we'll run tf-idol-dino and the contract on an Apple M5 Ultra.