Skip to content

fix(lanes): allocate trims a grafted window's held count and branch start with its rows - #1013

Open
chaog992 wants to merge 1 commit into
ashhart:mainfrom
chaog992:lane-allocate-grafts
Open

chaog992 wants to merge 1 commit into
ashhart:mainfrom
chaog992:lane-allocate-grafts

Conversation

@chaog992

Copy link
Copy Markdown
Contributor

What this changes

allocate (zig/src/core/lanes/windows.zig:206) trims a tree window (windows.zig:232) with sanitizeTree. Suffix-match grafts from fillLanes (windows.zig:101) broke two things that trim assumed:

  • Held drafts. fillLanes leaves the held drafts on the device and puts only the grafts in p.tokens (windows.zig:126), while the parents cover both. sanitizeTree paired the grafts' tokens with the held drafts' parents and p.held kept its count, so build read past the parents in rowParents (accept.zig:12).
  • Branch start. A count that cuts into a copied trunk left branch_from, the first graft row (windows.zig:128), past the window's rows. conclude's rows - p.branch_from (engine.zig:360) then went below zero: a panic in a safe build, and in ReleaseFast a loop on the next line over the wrapped length that does not finish.

allocate now sanitizes every draft, held ones first as placeholders, keeps the first min(held, count) held and the rest as p.tokens, and sets branch_from back to 0 for a count below it, which keeps no graft. A window with neither held drafts nor grafts, such as the 27B's host trees, comes out as before.

Nothing served turns grafts on yet (the 1.0.12 notes). Only the Mac tensorfold run ... --lanes N (or auto) sets fill_lanes (zig/src/main.zig:316); with --streams 2 or more, a shared round whose allocation (engine.zig:265) cuts a grafted window of held drafts reaches the first case. A lone round allocates only a head window, and only with draft_probabilities (engine.zig:212), which Nemotron does not set. Nemotron's Metal backend, the only one tensorfold run drives, refuses every shared window with parents (error.SharedTreesNotBuilt, zig/src/families/nemotron/backend.zig:423), so that round now ends in that error instead of reading past the parents. No shipped configuration grafts a copied trunk, the second case. Draft #861's trimmed keeps its own copy of the old branch, so it would need the same change.

Receipt

  • Environment: base 46a08c1 (1.0.12; main's later commits, to 9ccf472, change only docs), fix a846160. Zig 0.17.0, Linux x86_64 under WSL2, no GPU or model.
  • Exactness: no device run. On the fake target, a new shared-round test runs two drafted streams with fill_lanes 9 (as --lanes 8); their replies and finish reasons must equal one stream's one-token decode, and some grafted rows must be accepted. It runs on held drafts, as the CLI's streams, and on copied ones, at most 4 rows a round so the allocator cuts into the copy.
  • Decode speed: not measured. Only a trimmed tree window runs the change; on shipped engines that is the 27B's host trees, where it adds one arena copy of the draft ids.
  • Prompt speed: n/a, since prompt passes don't allocate drafts.
  • Tests run (each exits 0 unless noted):
    • New zig/src/core/lanes/fill_test.zig, on the fake target: the shared-round test above, and allocate at every count on a hand-built held window and a copied one (three drafts, then three grafts), at each count from 1 to 6, checking the count, held count, tokens, parents and branch_from. Unlike the shared-round test, it reaches counts below the held drafts. No lanes test trimmed a grafted window before.
    • Lanes, zig test -lc -O <mode> zig/src/core/lanes/lanes.zig: 120/120 in Debug, ReleaseSafe and ReleaseFast (118/118 on the base).
    • zig build test: 84/84 steps, 1378/1385 tests, 7 skipped (1376/1383 on the base). test-kernel-contract 18/18. test-golden 309/319 equal, 0 unexpected, as on the base. fuzz-cortheon 7/7. Plain zig build on Linux: 12/12 steps.
    • tools/zig/lean_check.py: the same 517 problems as the base, exit 1 on both (the one in a touched file, windows.zig:153, is inherited). tools/zig/lean_new.py against the merge base: 0 new. zig fmt --check build.zig zig and git diff --check pass.
  • Failing receipt: the new tests on the base's windows.zig (git checkout 46a08c18 -- zig/src/core/lanes/windows.zig), each run alone with --test-filter.
    • The shared-round test in Debug, shortened:
      fill_test.test.a shared round that trims a window with suffix-match grafts replies as the one-token decode...thread … panic: index out of bounds: index 1, len 1
      zig/src/core/lanes/accept.zig:12:24: … in rowParents (lanes.zig)
      zig/src/core/lanes/windows.zig:310:42: … in build (lanes.zig)
      zig/src/core/lanes/engine.zig:271:32: … in roundStreams (lanes.zig)
      zig/src/core/lanes/engine.zig:144:31: … in step (lanes.zig)
      zig/src/core/lanes/fill_test.zig:60:53: … in run (lanes.zig)
      
      ReleaseSafe panics the same way. In ReleaseFast the read is undefined behaviour; here the round failed with error.ParentAfterChild.
    • The allocate test fails its first count in all three modes: expected 1, found 4 (three held drafts kept beside one graft).
    • With only the held half of the fix, the shared-round test's copied run panics with integer overflow at engine.zig:360:43 in Debug (2 rows, branch_from 4); in ReleaseFast that test was still running when a 120 s timeout stopped it.
  • Mutants: each of these fails the allocate test in Debug and ReleaseFast: held = p.held (no cap), p.held = c, p.held left as it was, placeholders after the grafts, and branch_from left, reset at <=, reset only beside held drafts, reset always, or reset to the count. Two other variants pass both tests: another placeholder value, which no row reads, and clearing tried with branch_from. The fix keeps tried by choice, so a held window whose grafts are all cut still counts as graft_hit's miss (stream.zig:182).
  • Not run, and why: no Mac, so no zig build test test-golden -Dcpu=apple_m1 (the lanes tests compile for aarch64-macos with -mcpu apple_m1 --test-no-exec, a compile check only) and no tensorfold run ... --lanes N --streams M; no GPU or model, so no Metal or CUDA run.

Checklist

  • Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
  • Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (No device run; see Exactness. Resumed against fresh was not run, since the change acts inside one round's allocation.)
  • No precision traded for speed. If the bits change, the description says which sums change and why. (No arithmetic changes.)
  • A new or changed GPU kernel is registered with the kernel contract (docs/kernel-contract.md); each backend's SUMMARY line is in the receipt. (n/a: no kernel changes.)
  • Prompt processing is no slower than the last release. (n/a: prompt passes don't allocate drafts.)
  • The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not.
  • New tests fail before the change, pass after it, and skip cleanly without their dependency. (They need no model or GPU and never skip.)
  • Comments and docstrings are one line. No measurements or history in the source.
  • No personal data, machine names, internal hosts or local paths. No AI attribution lines.

…tart with its rows

allocate trimmed a window with suffix-match grafts as a plain tree. On drafts the head holds,
sanitizeTree paired the grafts' tokens with the held drafts' parents and the held count stayed,
so the window claimed rows its parents did not cover; and a count inside a copied trunk left
branch_from past the window's last row. It now sanitizes every draft, the held ones as
placeholders, keeps the first min(held, count) of them held, and clears branch_from when the
trimmed tree keeps no graft.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant