[harrisonfriia]

What this changes

One Mac moves GLM-5.3-Flash's depth rule estimates at 0.08, the rate two Macs already use. Before this change, one Mac left the rate at 0 and took the core's 0.15.

At 0.15, a short run of missed drafts puts a stream's acceptance estimate under the line where plain rounds win. The stream goes plain, and its probes back off to one draft in 128 rounds, so a reply can stay plain after its drafts would land again. At 0.08, a stream takes about twice the misses in a row to go plain. The constant moves from timing.zig to backend.zig, beside the one-Mac costs.

This is the served lanes' depth rule, separate from #1114's standalone controller. The speed receipts are single-stream, on main before #1059's shared-round fix.

Rebase validation

Rebased onto c0b0afce at 79a6ea09; the patch is unchanged. Static integration review found no blocker. On macOS with Zig 0.17.0:

zig build test-kernel-contract test-glm-scheduling-host test-glm-expiry fuzz-cortheon native -Dcpu=apple_m1 -j4 --summary all

All 76 build steps and 140 tests passed. Formatting and diff checks passed; lean reports zero new findings (438 on both branches). Model measurements were not repeated; the receipt retains its original source commits.

ATC coverage is incomplete (pullFilesLimit). The reported overlap with #911 already conflicts against the base, with the same conflict-file set.

Receipt

• Environment: TensorFold d1018b31 (main) and a4ad73e1 (this PR), each built with zig build native -Dcpu=apple_m1 (Zig 0.17.0, Xcode 27.1). Mac Studio, M3 Ultra with the 80-core GPU, 256 GB, macOS 27.0.1. Checkpoints: TensorFold/GLM-5.3-Flash-MLX-oQ4-MTP at 98725463 and ddalcu/GLM-5.3-Flash-MLX-Serve-iQ-MLX-5.1bpw at 2b1c4539. Served with --context 262144 --max-tokens 32768 --parallel 3 --thinking --reasoning-effort high --thinking-budget 15360 --temperature 1 --top-p 0.95 --prompt-cache-gib 24 and GLM_LOAD_LIMIT_GB=220.

• Exactness: tools/bench_concurrent.py --levels 3 --alone --serial (256 tokens, 3 reps, temperatures 1 and 0) on both checkpoints, main and this PR: 36 of 36 alone runs equal their concurrent replies, and 8 of 8 serial checks equal "draft": false. Every bench_openai.py reply in the following tables, seeded at both temperatures, has the same output_sha256 on main and this PR. Resumed prompts: a greedy two-turn chat over TensorFold's README.md, 64-token replies with thinking off, sent as turn 1, turn 2, turn 2. With --prompt-cache-gib 24, both turn 2 requests (10,056 prompt tokens) resumed at 10,021 cached tokens from turn 1's state and gave the same token_sha256 as fresh runs with --prompt-cache-gib 0 on main, on both checkpoints.

• Decode speed before and after, from tools/bench_openai.py, tok/s, the median of 3 replies, two runs each. Long replies, --tokens 2048 --reps 3, on oQ4-MTP, in the order main, this PR, this PR, main:

  Prompt	main	This PR	Change
  fibonacci-raw, temperature 0	60.8, 61.1	61.0, 61.2	+0.2%
  gpu-chat-no-think, temperature 0	54.4, 54.0	54.2, 54.3	+0.1%
  fibonacci-raw, temperature 1	81.4, 81.9	82.1, 81.5	+0.2%
  gpu-chat-no-think, temperature 1	48.4, 51.1	51.9, 51.7	+4.2%

  The chat replies at temperature 1 are the same tokens in all four runs. Each reply's tok/s:

  Seed	main	This PR
  1234	46.8, 44.3	50.8, 51.2
  1235	48.4, 52.5	52.7, 53.0
  1236	50.6, 51.1	51.9, 51.7

  512-token replies, --tokens 512 --reps 3, each build twice on each checkpoint, main first, then in reverse order:

  oQ4-MTP

  Prompt	main	This PR	Change
  fibonacci-raw, temperature 0	62.9, 62.9	62.3, 62.5	-0.8%
  gpu-chat-no-think, temperature 0	55.2, 55.1	55.4, 55.4	+0.5%
  fibonacci-raw, temperature 1	58.1, 59.0	59.0, 59.1	+0.8%
  gpu-chat-no-think, temperature 1	51.1, 51.2	52.6, 52.7	+3.0%

  iQ-MLX-5.1bpw

  Prompt	main	This PR	Change
  fibonacci-raw, temperature 0	59.5, 58.7	58.3, 58.3	-1.4%
  gpu-chat-no-think, temperature 0	52.7, 51.9	51.6, 51.8	-1.2%
  fibonacci-raw, temperature 1	52.5, 52.8	53.3, 52.8	+0.8%
  gpu-chat-no-think, temperature 1	47.6, 47.1	48.2, 48.5	+2.1%

  On iQ at temperature 0 this PR is 1.2 to 1.4% slower, within the 1.4 to 1.5% between main's own two runs.

• Prompt speed: n/a. The depth rule runs only in decode. The median cold ttft_s from tools/bench_cache.py --require-cold --repeats 1 --streams 1 on a 16,384-token prompt: oQ4-MTP 32.30 and 32.25 s on main, 32.31 and 32.26 s with this PR; iQ-MLX-5.1bpw 70.65 and 70.64 s on main, 70.64 and 70.62 s with this PR.

• Tests run: zig build test test-golden -Dcpu=apple_m1 with an empty cache: 4,927 of 4,948 passed, 0 failed, 21 skipped, including the JPEG and WebP pixel cases, whose libraries are not installed here. python3 tools/zig/lean_check.py: 444 problems, the same as main.

• Not run: CUDA, because the change is in the Metal backend only. Metal on M1, M2, M4 and M5, because I have no access to those chips. Two Macs, because I have one; a pair's rank 0 already used 0.08. The kernel contract, because no kernel changes.

Checklist

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo.
• No precision traded for speed. If the bits change, the description says which sums change and why. No bits change.
• A new or changed GPU kernel is registered with the kernel contract (docs/kernel-contract.md); each backend's SUMMARY line is in the receipt. No kernel changes.
• Prompt processing is no slower than the last release. Compared with main.
• The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not.
• New tests fail before the change, pass after it, and skip cleanly without their dependency. No new tests: zig/src/core/lanes/config.zig already tests a family rate of 0.08.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines.

[ashhart]

Thanks @harrisonfriia. The 0.08 depth rate on one Mac is on main (1b44f31). With GLM-5.3-Flash against main, tokens were identical in every cell on both an M5 Ultra (MLX 4-bit MTP) and an M3 Ultra (the mixed-width oQ4-MTP pack). Prefill was unchanged, and the sweep was within 0.2% on the M5 and up 2.0% and 2.5% at one and two streams on the M3, where two short decode tails read 3.6% lower. Closing as landed; it ships in 1.1.4.

- Synthema, Ash's AI Assistant.

[ashhart]

This shipped in TensorFold 1.1.4: https://github.com/ashhart/TensorFold/releases/tag/v1.1.4

Thank you @harrisonfriia for the contribution.

- Synthema, Ash's AI Assistant.