Skip to content

dsv41 metal: DSpark drafts - #1718

Closed
gilby wants to merge 7 commits into
ashhart:mainfrom
gilby:dsv41-metal-pr/14-dspark
Closed

gilby wants to merge 7 commits into
ashhart:mainfrom
gilby:dsv41-metal-pr/14-dspark

Conversation

@gilby

@gilby gilby commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

What this changes

DSpark on Metal: the drafter's weights, the ingest of committed rows, proposals for several slots in one launch and the pass's plan.

Part 14 of 18 of the stack for #1676; stacks on dsv41-metal-pr/13-prompt-suffix (part 13, #1717), on main with parts 1 to 5 (#1705 to #1709) merged.

Receipt

  • Environment, with the TensorFold commit, MLX or PyTorch version, machine and GPU, checkpoint and revision: TensorFold 9f16848 on main ad98b60 (native Zig 0.17.0; no MLX or PyTorch); Apple M3 Ultra (60-core GPU), 96 GB, macOS, for the kernel and device tests; checkpoint deepseek-ai/DeepSeek-V4.1-Flash-1017.
  • Exactness, from tools/bench_concurrent.py --alone --serial or token_sha against "draft": false: n/a for served output; R rows ingested at once == one at a time, 2 and 3 slots in one pass == each alone (greedy and sampled).
  • Decode speed before and after, from tools/bench_openai.py: measured on the last PR of the stack.
  • Prompt speed before and after: the cold first request's ttft_s from tools/bench_cache.py --require-cold on a long prompt, or "n/a" with the reason when the change cannot reach prompt processing: measured on the last PR of the stack.
  • Tests run, with pass, fail and skip counts: zig build test tf-kernel-contract -Dcpu=apple_m1: exit 0, 625/625 steps succeeded; kernel contract, tf-kernel-contract --only dsv41 (Metal): SUMMARY 89 kernels: 89 pass gates 1-4, 0 fail, 0 known findings; 186 controls: 186 red as planted, 0 missed; 0 limit problems; gates 119115 ms, limits 159 ms; zig build test-dsv41-metal -Dcpu=apple_m1 (the device suite, its own step): exit 0, 7/7 steps succeeded; 28/28 tests passed; python3 tools/zig/lean_check.py: 5131 files, 424 problems, the same count as main (5027 files, 424). Zig's summary counts only the test binaries it ran in that invocation, so the step count is the comparable figure.
  • Not run, and why: CUDA: no NVIDIA device here; Metal on M1, M2, M4 and M5: one M3 Ultra only; the release-layer checks (test-dsv41-metal-real, test-dsv41-metal-vision-real) were run at the stack's top and are in dsv41 metal: image prompts #1722's receipt.

Checklist

  • Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
  • Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo.
  • No precision traded for speed. If the bits change, the description says which sums change and why.
  • A new or changed GPU kernel is registered with the kernel contract (docs/kernel-contract.md); each backend's SUMMARY line is in the receipt.
  • Prompt processing is no slower than the last release.
  • The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not.
  • New tests fail before the change, pass after it, and skip cleanly without their dependency.
  • Comments and docstrings are one line. No measurements or history in the source.
  • No personal data, machine names, internal hosts or local paths. No AI attribution lines.

@ashhart

ashhart commented Nov 2, 2026

Copy link
Copy Markdown
Owner

Thanks @gilby: nothing owed for this part. It adds the drafter (Metal pass, plan and key rings, stage encoders, weights) with no new Metal source, only composing kernels the earlier parts gate. It lands after #1717.

@gilby
gilby force-pushed the dsv41-metal-pr/14-dspark branch from 69bcd3d to 9f16848 Compare November 3, 2026 05:30
gilby added 7 commits November 3, 2026 10:35
index.metal scores FP4 queries against FP4 keys in one stated order with causal and block masks, topk512.metal
keeps each row's top 512, cand.metal picks the candidate blocks as a byte mask, and route.metal takes the top-k of
sqrt(softplus(logit)) + bias for the backbone and the drafter. Each is registered with the contract.
compress.metal pools the ratio-2 and ratio-4 compressed KV and round-trips it through FP4; engram.metal dequantises
the FP8 Engram rows, projects them, gates them and adds them to the residual. Registered with the contract against
float64 and bit-equal to the phase-1 glue where they replace it.
…lded decode glue

hc.metal runs hc_mixes with Sinkhorn, hc_pre and hc_post in fixed orders; fused.metal and fold.metal split or fold
neighbouring glue launches at the same bits. Registered with the contract against float64 and bit-equal to the
phase-1 glue.
… by stage against float64

The Metal package over the family's neutral host files: the kernels' pipelines, release weights (copied or mapped
from the shards), a slot's KV regions, RoPE tables, scratch and one layer's encoders (attention with the indexer and
compressor, MoE, hyper-connections). test-dsv41-metal checks every stage of every layer kind against float64 and
rows at once against one at a time; test-dsv41-metal-real runs one release layer under TF_DSV41_REAL.
…d the DSpark taps

forward.zig runs the stack for 1-16 rows of one or several slots in one launch: Engram before its layers, the
hc mean into the taps, the head. model_weights loads a whole checkpoint, its shards mapped zero-copy where aligned.
The tiny model's stages are checked against float64 and R rows in a launch against one at a time.
The prompt runs the encoder layers over every row in chunks, then the decoder's suffix cone: layer L of the decoder
computes only the rows the prompt's end depends on (kept + (39 - L) x 127), with the bits of running every row.
ced gains the cone's rules and the snapshot replay; tests check chunk sizes, the cone against every row, resumes
and the grouped experts against the decode rows.
The drafter on Metal: its weights, the ingest of committed rows, the propose launches for several slots at once
and the pass's plan. Its stages are checked against float64, R rows ingested at once against one at a time, and
several slots in one pass against each alone.
@gilby
gilby force-pushed the dsv41-metal-pr/14-dspark branch from 9f16848 to e859dbd Compare November 3, 2026 09:39
@ashhart

ashhart commented Nov 3, 2026

Copy link
Copy Markdown
Owner

Thanks @gilby. This part landed on main with #1719's merge (cfcbecb), which carries #1712-#1719 as your own commits in order; closing it as included. The same Metal contract, full test and DeepSeek device suite cleared the whole stack.

@ashhart ashhart closed this Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants