Skip to content

dsv41 metal: hyper-connection mix, pre and post, and the fused and folded decode glue - #1714

Closed
gilby wants to merge 3 commits into
ashhart:mainfrom
gilby:dsv41-metal-pr/10-hc-fused-fold
Closed

gilby wants to merge 3 commits into
ashhart:mainfrom
gilby:dsv41-metal-pr/10-hc-fused-fold

Conversation

@gilby

@gilby gilby commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

What this changes

The hyper-connection boundary (hc.metal: hc_mixes with Sinkhorn, hc_pre, hc_post) and the decode launches that fuse or fold neighbouring glue at the same bits (fused.metal, fold.metal), all registered with the kernel contract: hc.metal (3 launch records), fold.metal (3) and fused.metal (11, with planted controls in every kernel that compile away at TF_CTL 0, checked by five gate suites metal_dsv41_fused*.zig against the unfused kernels and float64).

Main's shared files change only by one-line hooks; this part's records, suites and limits go into the family's own files (dsv41_metal_records.zig, metal_dsv41_main.zig, metal_dsv41_limits.zig).

Part 10 of 18 of the stack for #1676; stacks on dsv41-metal-pr/09-compressor-engram (part 9, #1713), on main with parts 1 to 5 (#1705 to #1709) merged.

Receipt

  • Environment, with the TensorFold commit, MLX or PyTorch version, machine and GPU, checkpoint and revision: TensorFold f1eea29 on main ad98b60 (native Zig 0.17.0; no MLX or PyTorch); Apple M3 Ultra (60-core GPU), 96 GB, macOS, for the kernel and device tests; checkpoint deepseek-ai/DeepSeek-V4.1-Flash-1017.
  • Exactness, from tools/bench_concurrent.py --alone --serial or token_sha against "draft": false: n/a: kernels only; the kernel contract checks each launch geometry's bits against the others and against float64.
  • Decode speed before and after, from tools/bench_openai.py: measured on the PR that turns serving on.
  • Prompt speed before and after: the cold first request's ttft_s from tools/bench_cache.py --require-cold on a long prompt, or "n/a" with the reason when the change cannot reach prompt processing: measured on the PR that turns serving on.
  • Tests run, with pass, fail and skip counts: zig build test tf-kernel-contract -Dcpu=apple_m1: exit 0, 621/621 steps succeeded; kernel contract, tf-kernel-contract --only dsv41 (Metal): SUMMARY 89 kernels: 89 pass gates 1-4, 0 fail, 0 known findings; 186 controls: 186 red as planted, 0 missed; 0 limit problems; gates 119418 ms, limits 161 ms; python3 tools/zig/lean_check.py: 5085 files, 424 problems, the same count as main (5027 files, 424). Zig's summary counts only the test binaries it ran in that invocation, so the step count is the comparable figure.
  • Not run, and why: CUDA: no CUDA code changes; Metal on M1, M2 and M4 (the contract's limits check covers the M1 to M5 pipeline limits). The kernel contract was not re-run on an Apple M5 Max at this head.

Checklist

  • Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
  • Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo.
  • No precision traded for speed. If the bits change, the description says which sums change and why.
  • A new or changed GPU kernel is registered with the kernel contract (docs/kernel-contract.md); each backend's SUMMARY line is in the receipt.
  • Prompt processing is no slower than the last release.
  • The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not.
  • New tests fail before the change, pass after it, and skip cleanly without their dependency.
  • Comments and docstrings are one line. No measurements or history in the source.
  • No personal data, machine names, internal hosts or local paths. No AI attribution lines.

@ashhart

ashhart commented Nov 1, 2026

Copy link
Copy Markdown
Owner

Thanks @gilby, this answers both of our 31 Oct points for this part: fused.metal's kernels now have their launch records, and every one has planted controls, checked by the five fused suites against the unfused kernels and float64. 82 kernels and 164 controls red, and Kimi untouched. With #1711-#1713, the kernel parts are done. They land in order after #1706-#1710, once our M3 Ultra and M5 Ultra runs pass. Part 11 (#1715) is next once it's rebased, with test-dsv41-metal in its own step, as we asked.

@gilby

gilby commented Nov 2, 2026

Copy link
Copy Markdown
Contributor Author

A fix to one of this part's planted controls, found while re-running the contract on an Apple M5 Max: dsv41 fused control: kv writes past its window rows missed there every time (10 of 10 runs; on the M3 Ultra it went red only because of scheduling). Its plant wrote WIN[r * dim + dim], which is row r+1's first slot inside the ring, and the repeat gate only sees that if the two writes race differently between runs. It now writes the first bf16 past the ring from threadgroup 0 (WIN[rows * dim]), as the family's other write-past controls do, so it goes red through the guard check on every run: FAIL repeat ...: a run wrote outside its outputs then red at repeat, as planted, 5 of 5 on the M5 Max. The line is under TF_CTL == 4, so the shipped kernel at TF_CTL 0 is unchanged.

Parts 10 to 18 are rebuilt on it (the only difference from the previous heads is that line) and pushed. tf-kernel-contract --only dsv41 is 178 of 178 controls red at every head on the M3 Ultra, and on the M5 Max at #1714 (89 kernels) and #1722 (98 kernels); zig build test tf-kernel-contract exit 0 and lean 424 at every head. The bodies name the new heads.

@gilby
gilby force-pushed the dsv41-metal-pr/10-hc-fused-fold branch from 01352d8 to f1eea29 Compare November 3, 2026 05:30
gilby added 3 commits November 3, 2026 10:35
index.metal scores FP4 queries against FP4 keys in one stated order with causal and block masks, topk512.metal
keeps each row's top 512, cand.metal picks the candidate blocks as a byte mask, and route.metal takes the top-k of
sqrt(softplus(logit)) + bias for the backbone and the drafter. Each is registered with the contract.
compress.metal pools the ratio-2 and ratio-4 compressed KV and round-trips it through FP4; engram.metal dequantises
the FP8 Engram rows, projects them, gates them and adds them to the residual. Registered with the contract against
float64 and bit-equal to the phase-1 glue where they replace it.
…lded decode glue

hc.metal runs hc_mixes with Sinkhorn, hc_pre and hc_post in fixed orders; fused.metal and fold.metal split or fold
neighbouring glue launches at the same bits. Registered with the contract against float64 and bit-equal to the
phase-1 glue.
@gilby
gilby force-pushed the dsv41-metal-pr/10-hc-fused-fold branch from f1eea29 to 371bf9e Compare November 3, 2026 09:39
@ashhart

ashhart commented Nov 3, 2026

Copy link
Copy Markdown
Owner

Thanks @gilby. This part landed on main with #1719's merge (cfcbecb), which carries #1712-#1719 as your own commits in order; closing it as included. The same Metal contract, full test and DeepSeek device suite cleared the whole stack.

@ashhart ashhart closed this Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants