[akol1]

What this changes

Step 2 of 6, stacked on step 1 (qwen27/pair-seam) and main (5897e0bc). Step 3 stacks on this commit.

The 27B family becomes rank-aware behind one interface: forward.Reduce (zig/src/families/qwen27/forward.zig:23) carries rowsIn, rowsOut, pick, broken and next. It defaults to null, so a one-Mac model compiles the seam out and executes the code it executed before this series. projection.prepare derives a rank's row slice from the checkpoint matrix (gn), so the slice keeps the checkpoint's K split, tile and prompt tables (zig/src/families/qwen27/projection.zig); prompt rows take promptSplit of the whole matrix, so the pair's prompt reduction is the reduction tree one Mac uses rather than a half-width tree.

The split is by output rows with the full K on the rank that owns them: 12 of 24 attention heads, 2 of 4 KV heads, half of the linear attention's heads and value channels, 8,704 of the MLP's 17,408 rows, and 124,160 of the 248,320-row lm_head. o_proj and down_proj stage their own output rows, gather the peer's as whole bf16 rows, and add in the fp32 stream, so no two bf16 partials are summed. Embeddings and norms stay whole on every rank, so a token id means the same thing on both, and the embedding kernel's gather bound stays the whole 248,320-row table rather than a shard's width (config.embedBound). The head seam is argmax over each rank's 124,160-logit half and pick over the pair, so the decided token is one Mac's argmax over all 248,320 logits; session.zig keeps the merged top two for its cross-block tie test, and reads the pick's maxInt(u32) sentinel as error.PairFailed rather than as a logit.

The prompt attention merge writes the rank's KV node rows (kv_heads in Q27PromptArgs) instead of assuming the model's four. config_test pins the spans (intermediate 8,704, down columns 8,704, vocab 124,160, output rows 2,560) and the single-Mac spans stay 0; synthetic_graph runs the same graph on both ranks.

Receipt

• Environment, with the TensorFold commit, MLX or PyTorch version, machine and GPU, checkpoint and revision: 5897e0bc plus steps 1-2 (0b79f9c9, 7d57101a); no MLX or PyTorch; macOS 27.0.1 on an Apple M4 Pro (Mac16,11), Metal only (applegpu_g16, no tensor units), Apple metal version 32023.921. No checkpoint: this commit's gates build the graph from a synthetic config, they do not read weights.
• Exactness, from tools/bench_concurrent.py --alone --serial or token_sha against "draft": false: n/a, no serving path at this commit; step 4 records the token fixtures. The kernel change here is checked at the series head with tf-kernel-contract --rotary-accuracy <rotary.json> --only pair: SUMMARY 16 kernels: 16 pass gates 1-4, 0 fail, 0 known findings; 30 controls: 30 red as planted, 0 missed; 0 limit problems. The split itself is pinned by config_test and by projection.zig's tests: a rank's slice keeps the checking matrix's K split, a fused gate-up keeps the member split rather than recomputing it from the stacked width, and a local qkv block keys neither m5Split nor promptSplit, so the fallback would stand against an unsplit table.
• Decode speed before and after, from tools/bench_openai.py: n/a, nothing serves yet; step 4 measures the pair and its one-Mac reference.
• Prompt speed before and after: n/a, no prompt table changes here; prompt projections gain the full matrix's split, which is the same reduction one Mac runs.
• Tests run, with pass, fail and skip counts: zig build test: 557/557 steps, exit 0. zig build test-kernel-contract: 18/18 steps, 120/120 tests. zig build dev-tools: 213/213 steps. zig fmt --check build.zig zig: clean. Lean rule: 428 problems on both 5897e0bc and this tree, 0 new (tools/zig/lean_new.py, exit 0). The series head's fresh-cache suite, which includes these test steps: 565/565 steps, 8,157 passed, 47 skipped, 0 failed of 8,204.
• Not run, and why: the model gates (they need a checkpoint and arrive with tf-qwen27-tp-check in step 4); a two-Mac run (the serving link arrives in step 3); the full kernel corpus (stops at UNSUPPORTED gemma4 31B rounded dense: tensor units required); CUDA, HIP/RDNA, M1-M3, M5 and Windows are not here.

Checklist

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off. — the split rides the rounds the lanes already execute; no serving path is added in this commit.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. — no serving path at this commit; step 3 refuses drafts, and step 4's fixtures are greedy so there is nothing drafted to compare.
• No precision traded for speed. If the bits change, the description says which sums change and why. — the split moves bf16 rows and adds in fp32 once per seam; no bf16 partial sum is created, and the one-Mac path is unchanged because its Reduce is null.
• A new or changed GPU kernel is registered with the kernel contract (docs/kernel-contract.md); each backend's SUMMARY line is in the receipt. — attention_prompt keeps its registration with the new kv_heads argument; the filtered pair SUMMARY above is the run that exercises it.
• Prompt processing is no slower than the last release. — n/a, prompt projections keep the checkpoint matrix's split, so their reduction tree is unchanged; a throughput comparison belongs with step 4's pair measurement.
• The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not. — Metal on an M4 Pro (applegpu_g16); no CUDA, HIP, M1-M3, M5 or Windows.
• New tests fail before the change, pass after it, and skip cleanly without their dependency. — config_test's shard spans and the split tests fail on the parent commit (no shard), and the model-requiring paths skip without a checkpoint as before.
• Comments and docstrings are one line. No measurements or history in the source. — the lean check reports 428 problems on the base too, 0 new.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines. — no user path, hostname or address is in the diff.

[ashhart]

The embedBound fix is a real bug fix with a test. Thank you.

Two places change the arithmetic, so the pair can't match the single Mac yet:

• The row-parallel split of o_proj/down_proj reduces across ranks. See core: add the row-split pair seam and split simd_qmm over K #1432 for an exact layout that keeps each sum on one rank.
• The rank-local K-split. projection.prepare picks the K-split from split(weight.shape[0], k), and rowView sets shape[0] to the rank's rows. So a rank's q/k/v, gate/up and z/b/a get a different split from the single Mac's. For example, gate goes from 17,408 to 8,704 rows and sk from 2 to 4; q goes from 12,288 to 6,144 and sk from 4 to 8. On M5, the m5Split/promptSplit tables are keyed on the global shapes, so a rank misses them. A different sk is a different fp32 order. Please carry the global matrix's split into the rank slice.

Stage 1 compares loadShard(ranks=1) against load on the same new code. It needs a comparison against main's binary too.

[akol1]

Refreshed head 7d57101a carries the global split into the rank slice, and the arithmetic now matches one Mac's:

• No cross-rank reduction. Each output projection is split by output rows; rank r owns 2,560 of the 5,120 hidden columns (8,704 of 17,408 for down_proj) and reads the full K. Every sum stays on one rank, and pair_gather copies whole bf16 rows with no arithmetic between them, so no cross-rank partial sum and no tie policy exists.
• The K split comes from the checkpoint matrix. projection.prepare and Linear.gn carry the global width into quant_gpu, so a rank's q/k/v, gate/up and z/b/a keep the split one Mac uses instead of recomputing it from the slice's rows: gate-up stays at 2 for 8,704 of 17,408 (not 4), q stays 4 and 8 for 6,144 of 12,288 (not 8 and 16), and a local 2,560-row down slice is keyed on neither the m5Split nor the promptSplit table. projection.zig and config_test.zig pin both directions.
• Stage 1 against main's binary. tp_check --stage 3 still compares loadShard(ranks=1) against load on the new code, and the separate cross-check in qwen27: pin the pair against the single Mac in the test suite #1435's receipt compares against main's own tensorfold-native built from 5897e0bc: it reproduces the same 256-token sha (a06fab34…) for the same 31-id chat prompt.

[ashhart]

Thanks @akol1. Carrying the checkpoint matrix's width (gn) into a rank's slice keeps the 4-bit K splits one Mac uses. The embedding-bound fix and config_test are good. On a local merge with main, fmt and lean are clean, config_test.zig passes 17/17, and the qwen27 host steps we ran pass. Four asks:

1. q27_attn_merge still strides 24 heads a row (zig/kernels/metal/qwen27/attention_mpp.metal:320,323). You fixed the same stride in the prompt merge. This kernel serves every multi-row round that is not a prompt chunk (attention.zig:343,410-413). A served chat feeds each earlier reply that way (qwen27_slots.zig:81,122, decodeChunk). On a rank, rows 1 and up land at twice their offset and read another row's gate. Please pass the rank's head count, as Q27PromptArgs now does.
2. One Mac's split changes. in_proj_qkv now loads as three blocks (gpu_weights.zig:222), and projection.zig:61 keys its split on the q block's 2,048 rows: 8 K slices, where main gives 4 (:145 pins 8). Widths other than 4-bit g64 on the M5 use that split (:71). Please key it on the tensor's rows. Add a host test that every one-Mac stack keeps main's split.
3. Before the M5, other widths take simd_qmm_bits, which chunks by the rank's width (quant_gpu.zig:217, simd_qmm_bits.zig:68). That gives 16 chunks for q, qkv and zba where one Mac takes 8. Please pass gn through, or refuse those widths on a pair by name, with a test.
4. Small: Part.k0/k, Triple.k/k0 and Shard.cols/*Cols are unused outside tests. Please drop them. Also, no contract record names q27_attn_prompt_merge, so please correct the body.

This also needs #1432's rebase.