Skip to content

glm cuda: TF_GLM_CHUNK_SOLO=4096 beside --drafter is refused by name - #1842

Merged
ashhart merged 2 commits into
ashhart:mainfrom
Bizuayeu:pr/glm-solo-drafter
Nov 2, 2026
Merged

ashhart merged 2 commits into
ashhart:mainfrom
Bizuayeu:pr/glm-solo-drafter

Conversation

@Bizuayeu

@Bizuayeu Bizuayeu commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

The refusal you suggested on #1823 (comment): TF_GLM_CHUNK_SOLO=4096 beside --drafter holds 1.6 to 1.9 GiB of prompt buffers that no pass uses. It was the last commit on that branch (a27444ebb) and the head that merged doesn't have it, so here it is on its own. A server without both settings starts and runs as it does on main.

What this changes

  • prompt_solo.refuse runs before the weights load, on every rank, as TF_GLM_KV_SHARE without pages does, with error.GlmSoloWithDrafter and its line in explain.
  • prompt_solo.at and allowed lose e.drafter != null, which nothing reaches now.
  • The recipe's ## Prompt chunk size says so.

Receipts

  • Tests (Linux x86_64, at 0f67c9da9): test-cuda-host 276/276 steps (4,236 of 4,246 tests, 10 skipped), zig build test 456/456 steps (1,894 of 1,913 tests, 19 skipped), test-kv-pool 33/33, compile-cuda-tests 10/10 (the CUDA runner, which CI does not build, also for aarch64), compile-config-server 7/7, zig fmt --check, and tools/zig/lean_check.py (424 against main's 424, 0 new).
  • Not run: a start with a real DFlash2 drafter (the refusal is in the host tests), and any GPU: no step changes.

Checklist

  • Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off. (A refusal at load; no round changes.)
  • Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (No step changes.)
  • No precision traded for speed. If the bits change, the description says which sums change and why. (No sum changes.)
  • A new or changed GPU kernel is registered with the kernel contract (docs/kernel-contract.md); each backend's SUMMARY line is in the receipt. (No kernel changes.)
  • Prompt processing is no slower than the last release. (No step changes.)
  • The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not. (The host tests on Linux; no GPU run, as above.)
  • New tests fail before the change, pass after it, and skip cleanly without their dependency. (The host test reads refuse, which main does not have.)
  • Comments and docstrings are one line. No measurements or history in the source.
  • No personal data, machine names, internal hosts or local paths. No AI attribution lines.

Bizuayeu and others added 2 commits November 2, 2026 04:57
A drafter's steps take one chunk, so the wider buffers would hold 1.6 to
1.9 GiB that no pass uses. Refused before the weights load, as
TF_GLM_KV_SHARE without pages is, on every rank; with that, the step's
own checks no longer need the drafter.
@ashhart
ashhart force-pushed the pr/glm-solo-drafter branch from 0f67c9d to 16a756c Compare November 2, 2026 04:57
@ashhart
ashhart merged commit c637b80 into ashhart:main Nov 2, 2026
9 checks passed
@ashhart

ashhart commented Nov 2, 2026

Copy link
Copy Markdown
Owner

Thank you @Bizuayeu. This is merged into main with your commits under your own name, so you're listed as a TensorFold contributor. It ships in the next release, credited to you in the changelog and the release notes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants