[jschmied]

Qwen3.8-27B on Metal takes its keys and values from pages of the core pool, behind TF_QWEN27_METAL_KV_PAGES=1. It is off by default. This is the 27B step of #1293.

What changes

1. metal kv pages, shared step: the head-major page row (tf_paged_row), the page tensor (core/kv_page_tensor.h) and the compile-time twin source edits (core/source_edit.zig) move into kernels/metal/core. Nemotron and Qwen 3.5 are rewired to them, with the same kernels and bits.
2. qwen27 metal: paged twins of the 7 K/V kernels, derived from main's sources through source_edit. Cache and binding carry the page table, and admission goes through metal_admit.pages (kv_pool.plan). Snapshot spans and the digest follow the pages.
   • Nemotron's Zig lease glue (grow/keep/census) is not lifted yet, because metal nemotron: TF_KV_SHARE keeps prompt states as core pool pages (stacked on #1837) #1850 edits it. q27 calls core Pages through a thin grow/keep, to be folded in once metal nemotron: TF_KV_SHARE keeps prompt states as core pool pages (stacked on #1837) #1850 lands.
3. tf-qwen27-pages-resume: on the synthetic model, a kept state is saved through pages, its slot overwritten, then restored and decoded; then the slot moves to 1 and back. The restored, moved and fresh digests (KV rows included) equal main's.

Checks

• M3 Pro, real Qwen3.8-27B MLX 4-bit at 70ae7fac:
  • Greedy replies are equal on main, flag off and flag on, alone and with 2 streams.
  • The pool is idle at close, and speed is within noise.
  • tf-kernel-contract --only qwen27 passes 24 kernels at gates 1-4 with 0 fails, and 2/2 controls are red.
  • tf-kernel-contract --only nemotron gives the same SUMMARY as main. zig build test passes 691/691 steps.
• GB10 (Linux): lean finds 0 new problems. zig build test, the no-GPU CI set, git diff --check and zig fmt --check pass.
• The branch merges cleanly into today's main.

[ashhart]

Thanks @jschmied. Lifting the page row, page tensor and twin source edits into kernels/metal/core, and deriving the 27B's paged kernels from main's sources rather than copying them, is the right shape for #1293's later steps. Off by default, with greedy replies equal on main, flag off and flag on, is what we need. A few things before it queues:

1. A CHANGELOG line naming TF_QWEN27_METAL_KV_PAGES=1 as opt-in.
2. The two tools' lines from your M3 Pro: tf-qwen27-attention-pages, paged against contiguous on shuffled pages, and tf-qwen27-pages-resume, plus prefill and decode numbers with the flag on and off against main.
3. The M5 path: if you can reach one, both tools and the greedy check with the flag on. If not, please refuse the flag by name on chips with tensor units until it's run.
4. Qwen 3.5's paged kernels unchanged after the move: its contract SUMMARY against main, as you posted for Nemotron.

Kept states as pool pages stay with #2215's walk, as planned on #1293.

[jschmied]

Thanks @ashhart. All four are in, as new commits on top. The head is 35e3d98.

1. CHANGELOG: one line under Unreleased names TF_QWEN27_METAL_KV_PAGES=1 as opt-in.

2. Tools and speed, M3 Pro:

   attention_pages on Apple M3 Pro, tensor units no: 16 rounds, 197 pages of 1049088 bytes shuffled
   PASS 16 rounds: 4675584 output values and every cache row the same bits, paged and contiguous
   PASS kept states through pages: restored equal to fresh, and restored, moved and fresh slots the same digests as main's
   info: qwen27: KV pages idle, 8 of 8 free
   

   Real 27B, two ABBA rounds per config, each round a median of 3:

   	3,000-word cold prompt (s)	decode (tok/s)
   main	45.04 / 43.77	8.55 / 8.71
   flag off	44.49 / 43.96	8.62 / 8.72
   flag on	45.80 / 44.77	8.51 / 8.65

   Flag on against main is +2.0 % on prefill and −0.6 % on decode. A config's own two rounds differ by up to 3 %, so both are inside the noise. That Mac was also in daily use, under memory pressure.

3. M5: we can't reach one, so 6474203 refuses the flag on a chip with tensor units before anything loads: error.KvPagesUntested, with a message naming the flag. The two tools still run there, so an M5 can validate before the refusal is lifted.

4. Qwen 3.5: there's no contract SUMMARY to post. The kernel contract registers no Qwen 3.5 Metal kernels, on main or on this branch: --only qwen3_5 gives SUMMARY 0 kernels on both. So "unchanged" rests on these instead:

   • The paged library's LLVM IR at the runtime flags, main against branch: all 7 functions match except one 8-line block in the key write, where the same three u64 terms are added in another order. Integer adds are exact, so the addresses are the same.
   • test-metal-kv-pages: Qwen 3.5's paged test passes, paged equal to contiguous bit for bit on shuffled pages.
   • Qwen3.5-2B with TF_QWEN35_METAL_KV_PAGES=1, --parallel 2: greedy 64 tokens at 40, 600 and 3,000 words, main equals branch, and the pool closes with 64 of 64 free.

   Registering those kernels would be a follow-up, like Nemotron's.

On this head: zig build test 691/691, lean the same 425 problems as main 54336bc, zig fmt --check and git diff --check clean.

[jschmied]

Rebased onto main 754c3e9; the head is 615d153.

The rebase needed code, not only a merge. main's masking work added V reads to the same 27B kernels the paged twins are derived from, and those reads address V by head stride and key, not through the page table. The twins would still have built and passed, then read the wrong rows whenever the masked path ran. 615d153 is on its own for review: q27_nonfinite_paged, the masked prefix's value rows, the prompt and prompt_sg scans and q27_attn_prompt_scan now read through the table. It also passes paged=false from main's two new grouped() callers, which the rebased tree needs to build. The other commits are the same work; only CHANGELOG conflicted.

tf-qwen27-attention-pages now has 21 rounds, 5 of them planted with Inf, -Inf and NaN:

attention_pages on Apple M3 Pro, tensor units no, shared prefix no: 21 rounds, 197 pages of 1049088 bytes shuffled
PASS 21 rounds: 6696960 output values and every cache row the same bits, paged and contiguous
attention_pages on Apple M3 Pro, tensor units no, shared prefix yes: 21 rounds, 197 pages of 1049088 bytes shuffled
PASS 21 rounds: 6696960 output values and every cache row the same bits, paged and contiguous
PASS kept states through pages: restored equal to fresh, and restored, moved and fresh slots the same digests as main's

Dropping each new edit alone fails the tool: both prompt scans at round 19; the masked V row and the prefix scan with --shared-prefix at round 16, the first planted round. The prefix tile edit with --shared-prefix fails at round 0.

Three limits:

1. Prefix kernels: a Mac without tensor units has no shared prefix (limit 0), so the prefix kernels never run there, on main either. Until now the paged prefix twins, masked or not, were only compiled. --shared-prefix runs the M5's plan on the M3 to check their addressing bit for bit. It doesn't check their M5 result layout, so the M5 refusal stays.
2. Greedy check: 27B greedy equals main with the flag off and on, solo and on two streams, and resumed equals cold, but it doesn't reach the masked path because real values are finite. Only the planted rounds exercise it.
3. Shared code: non-finite masking exists only in qwen27, so the paged scan stays in its twin file, on the shared tf_paged_row. Nemotron's and Qwen 3.5's paged kernels on main have no such reads.

Timings re-run, since the scan runs on every prompt pass. ABBA, medians of 3, same Mac, in daily use:

	3,000-word cold prompt (s)	decode (tok/s)
main	48.84 / 47.12	7.85 / 7.89
flag off	46.86 / 46.83	8.22 / 8.33
flag on	47.34 / 47.50	8.16 / 8.16

Flag on against flag off is +1.2 % on prefill and −1.4 % on decode. main ran slower than both branch builds, which I put down to the box, so I wouldn't read a gain into that gap.

On this head: zig build test 711/711, lean 325 problems as on main with none new, tf-kernel-contract --only qwen27 30 of 30 kernels pass and 3 controls red as planted, zig fmt --check and git diff --check clean.

[ashhart]

Thanks @jschmied. All four are answered: the CHANGELOG line, both tools and the ABBA numbers, the M5 refusal by
name before the paged runner starts, and Qwen 3.5's paged path shown unchanged without a contract SUMMARY.
Catching the masked V reads in the rebase, with planted Inf/NaN rounds that fail when any one edit is dropped, is
the right check for kernels only planted inputs reach. It goes in the landing queue once our own cells on this head are in: main against it, with tokens equal and speed level.

[ashhart]

/atc exchange b54e2fe8fc759cc36fc4ddb5877d1ec216b3dc32abae3000e190ac3b3de659e5

[ashhart]

Our cells on this head (615d153b91) are in, main against it on an M5 Ultra, runs ordered base, new, new, base, with the first run dropped. Replies equal main's in each, and speeds are level:

• Qwen3.8-27B MLX 4-bit: -1.3% to +0.6%.
• Nemotron 3.5 Lightning 30B-A3B MLX 4-bit: -0.3% to +0.6%.

It's in the landing queue now.