Skip to content

Contract: masked non-finite values across every Metal prompt attention, the findings registered as known (refs #1009) - #2308

Merged
ashhart merged 4 commits into
mainfrom
lead/masked-nonfinite-census
Nov 6, 2026
Merged

ashhart merged 4 commits into
mainfrom
lead/masked-nonfinite-census

Conversation

@ashhart

@ashhart ashhart commented Nov 6, 2026

Copy link
Copy Markdown
Owner

Refs #1009 (the finding from 28 Oct). #2106 fixed it for Qwen3.8-27B. This finds where else it holds.

A row gives probability 0 to a key it must not see, so a masked key whose value is +Inf or NaN should add nothing to that row. In many prompt-attention kernels, 0 × Inf gives NaN instead. This adds a contract census of that case across every Metal family with its own prompt attention:

  • one cache coordinate poisoned with +Inf or NaN;
  • the rows that must not see it checked by class against an independent float64 formula;
  • every finite reference output held to its existing bound;
  • finite shape panels at 8 and 32 rows, five-repeat stability, and planted controls.

The findings are registered as known, as #2106 registered the 27B's before fixing it, so nothing new goes red on main. Flash Next is left out (its own line).

On an M5 Ultra, 45 panels are red and 37 green on main:

red: hidden rows see NaN green
Gemma 4 (12B, 31B, E4B; global, sliding, wrap), Spark2.5 (full, sliding, wrap), MiniMax-M3 prompt, Llama (Nanbeige 3B and 33B), Nemotron prompt, LFM2 prompt (nax), GLM tensor direct (64 and 32 heads, padding, block split), Step5 tensor (+Inf only) Qwen 3.5 small dense, GLM hybrid dense latent and generated indexed, GLM EXLR8 MLA, Kimi K3/K3.1 MLA, Step5 scalar, LFM3 on its PR head

Fixes follow family by family, the 27B's way: finite prompts run main's kernel unchanged, and the masked read sits only where a masked non-finite value can occur.

  • Build: zig build tf-masked-values and zig build test-masked-values-host (macOS, outside the default install). The census also runs inside tf-kernel-contract.
  • Checks:
    • test-masked-values-host: 148 of 148 tests;
    • a plain zig build: 255 of 255 steps;
    • zig fmt --check;
    • no new lean finding (metal_main.zig stays at 600 lines).

Check masked Inf and NaN separately against binary64, including visible IEEE classes, sparse padding and wrapped windows. Retain the 45 measured inherited failures as known #1009 findings and catch both planted GPU controls. Register the suite in the Metal contract and provide a source-only focused runner. Production kernels and dispatch remain unchanged.
@ashhart
ashhart force-pushed the lead/masked-nonfinite-census branch from 811e6d2 to 158b5eb Compare November 6, 2026 14:36
@ashhart
ashhart merged commit eaf79c5 into main Nov 6, 2026
7 checks passed
ashhart added a commit that referenced this pull request Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant