Skip to content

deepseek: build CPU asset generators before CUDA images - #1640

Merged
ashhart merged 3 commits into
ashhart:mainfrom
jayleaton:dsv41-up/asset-tools
Oct 30, 2026
Merged

ashhart merged 3 commits into
ashhart:mainfrom
jayleaton:dsv41-up/asset-tools

Conversation

@jayleaton

Copy link
Copy Markdown
Contributor

What this changes

DeepSeek-V4.1 serving needs prepared assets: the token map, RoPE tables, signatures, the AOT needs list and SM121 cubins. Producing them meant booting a CUDA model. This installs the token-map exporter and the real AOT planner as a CPU tool, dsv41-assets, that needs no GPU. It parses the complete release metadata for preparation, while serving keeps its checkpoint representation refusals.

A second commit lets the same tool list Engram shard paths from an index-only source. It keeps repository-relative paths and requires an explicit prefix for bare filenames. Missing or unsafe index data is refused with an actionable message. No model shards or device are needed.

Docs: docs/recipes/deepseek-v41-cuda.md describes the CPU preparation steps.

Issue: none; this is tooling for the existing experimental family. Stacked on: nothing (based on main 3f0a715c4; the 28 later main commits, to f350cd3ea, touch none of its files).

Receipt

  • Environment, with the TensorFold commit, MLX or PyTorch version, machine and GPU, checkpoint and revision: head ddde40de8812 on main 3f0a715c4; Zig 0.17.0; host checks on a Linux aarch64 DGX Spark (GB10, CUDA 13.0). Real-pack boot on two GB10s, TP=2 over RoCE, checkpoint jayleaton/DeepSeek-V4.1-Flash-1017-EXL3-2.8bpw revision 859396591c34, TF_EXPERIMENTAL=deepseek.
  • Exactness: n/a; this is a preparation tool, and serving arithmetic does not change.
  • Decode speed before and after: n/a.
  • Prompt speed before and after: n/a.
  • Evidence: from the public 1017 config and metadata, the tool produces the canonical token map, RoPE, signatures, needs and an SM121 cubin. Host checks cover CLI admission, a refusal to select a GPU operation, the public 1017 config fixture, and Engram index refusals.
  • Tests run, with pass, fail and skip counts:
    zig fmt --check build.zig zig                      exit 0
    git diff --check 3f0a715c4..HEAD                    exit 0
    python3 tools/zig/lean_check.py                     base 425 problems, head 425, new 0 (lean_new.py)
    zig build test-cuda-host --summary all              exit 0, 271/271 steps, 3906/3915 tests passed, 9 skipped (with test-server-cpu)
    zig build test-dsv41-shared test-dsv41-state test-dsv41-forward test-dsv41-kv test-dsv41-device-host test-dsv41-draft test-dsv41-cuda-host test-draft-dsv41-host test-dsv41-assets-host test-dsv41-token-map dsv41-assets  exit 0, 63/63 steps, 552/554 tests passed, 2 skipped
    
  • Not run, and why: Metal; macOS host build of the tool (no Mac available).
  • Overlap: atc overlap reports a one-line conflict in build.zig with draft lfm3: serve LFM3-1.7B on CUDA: NVFP4 and FP8 weight-only, MLX 4-bit (stacked on #1513, #1492 and #1503) #1508 and with lfm3 CUDA: serve the block FP8 export weight-only and the MLX 4-bit export as stored (stacked on #1508) #1634, which is stacked on it. Each PR adds its own family host-test line, and the merge takes both.

Checklist

  • Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
  • Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (Not measured for this change alone.)
  • No precision traded for speed. If the bits change, the description says which sums change and why.
  • A new or changed GPU kernel is registered with the kernel contract (docs/kernel-contract.md); each backend's SUMMARY line is in the receipt. (No kernel changes.)
  • Prompt processing is no slower than the last release. (Tooling only.)
  • The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not.
  • New tests fail before the change, pass after it, and skip cleanly without their dependency. (No reverted-fix control: the tool and its tests are new.)
  • Comments and docstrings are one line. No measurements or history in the source.
  • No personal data, machine names, internal hosts or local paths. No AI attribution lines.

jayleaton and others added 3 commits October 30, 2026 16:15
Install the token-map exporter and the real AOT planner without booting a CUDA model. Parse complete release metadata for CPU preparation while retaining checkpoint representation refusals in serving.

Host checks cover CLI admission and the public 1017 config. CPU qualification on a Linux CUDA host passes formatting, zero new lean identities, native/plain builds, the full host and CUDA subsets, kernel contracts, security, golden, fuzz and CI tool checks. Actual metadata produces the canonical token map and the reference generators produce RoPE, signatures, needs and an SM121 cubin.

(cherry picked from commit 2765377)
List Engram shard paths from an index-only source, preserving repository-relative paths and requiring an explicit prefix for bare filenames. Missing or unsafe index data gives an actionable refusal; no model shards or device are needed for discovery.

Ryan3040 reported the missing-index traceback and shard-discovery mismatch in issue jayleaton/deepseek-v41-tensorfold-spark#68. This already-started native change is isolated for the recipe owner after the scope split; pack generation, real download paths and recipe wiring remain separate work.

(cherry picked from commit d5da559)
@ashhart
ashhart force-pushed the dsv41-up/asset-tools branch from ddde40d to ed97c59 Compare October 30, 2026 16:15
@ashhart
ashhart merged commit 6698974 into ashhart:main Oct 30, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants