Skip to content

contract: qwen27 attention_io at the other two shapes Geometry admits, GQA16's 32 over 2 and the 9B's 16 over 4 - #2063

Merged
ashhart merged 2 commits into
ashhart:mainfrom
cshintov:bugfix/qwen27-keep-contract-shapes
Nov 4, 2026
Merged

ashhart merged 2 commits into
ashhart:mainfrom
cshintov:bugfix/qwen27-keep-contract-shapes

Conversation

@cshintov

@cshintov cshintov commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

What this changes

The review on #2013 asked for keep at the other two shapes Geometry.check admits. The qwen27 attention_io contract now runs pack, append, host keep and device keep at all four: the 27B (24 over 4), its two-Mac rank (12 over 2), GQA16 on one Mac (32 over 2) and the 9B (16 over 4). Test only; no kernel or engine change.

Commit:

  • contract: qwen27 attention_io at the other two shapes Geometry admits, GQA16's 32 over 2 and the 9B's 16 over 4

Checks on this head

  • Head commit: 6b7ae97
  • Lean checker, python3 tools/zig/lean_check.py: 424 problems on the base, 424 on this head, 0 new.
  • Tests run: zig build test-qwen27, test-qwen27-keep-host, test-qwen27-plans-host and test-kernel-contract pass; zig fmt --check build.zig zig passes.
  • CI: pending.
  • Not run, and why: no model runs; the change adds contract cases only.

Receipt

  • Environment: TensorFold 37c2241 plus this commit, Zig 0.17.0, macOS 26.5, Apple M4 Pro (applegpu_g16, no tensor units). No checkpoint.
  • Paths and platforms the change reaches: the kernel contract only.
  • Exactness, decode speed, prompt speed: n/a, no engine change.
  • Licence of any new weights or drafter: n/a.
Kernel contract, only when a GPU kernel is new or changed

No kernel changes; the existing qwen27 attention_io kernels gain two registered shapes each.

  • tf-kernel-contract --only qwen27 on this head: SUMMARY 16 kernels: 16 pass gates 1-4, 0 fail, 0 known findings; 2 controls: 2 red as planted, 0 missed; 0 limit problems.
  • With main's pre-qwen27: keep's gather and scatter stage the binding's KV heads a row, not 4 #2013 attention_io.metal put back, the GQA16 keep and device keep fail ("staging keys row 1 head 0 is not stream 0's row 66" at 3x32x64) and the 9B's pass, as they should at 4 KV heads.
  • Chips I could not run: M5 (tensor units).

Checklist

  • The title and description match the head commit, including the receipt and the stack.
  • New tests fail before the change, pass after it, and skip cleanly without their dependency.
  • Comments and docstrings are one line. No measurements or history in the source.
  • No personal data, machine names, internal hosts or local paths in the diff or its commits. No AI attribution lines.

@ashhart

ashhart commented Nov 3, 2026

Copy link
Copy Markdown
Owner

Thanks @cshintov, this is the follow-up we asked for on #2013, and the check with the old attention_io.metal put back is exactly the evidence it needs: GQA16's keep fails where it should and the 9B's passes. The new shapes reach only the attention_io registration loop, so the contract goes from 8 to 16 cases with nothing else touched. We'll run the qwen27 contract on an M5 Ultra, the chip you couldn't, and land it after that.

@ashhart
ashhart force-pushed the bugfix/qwen27-keep-contract-shapes branch from 6b7ae97 to c29658a Compare November 4, 2026 00:59
@ashhart
ashhart merged commit d4ee89f into ashhart:main Nov 4, 2026
4 of 7 checks passed
@ashhart

ashhart commented Nov 4, 2026

Copy link
Copy Markdown
Owner

Thank you @cshintov. This is merged into main with your commits under your own name, so you're listed as a TensorFold contributor. It ships in the next release, credited to you in the changelog and the release notes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants