Repository navigation
Conversation
…they allocated it drop() freed the expert list and the gate and up scale rows through sliceAsBytes, at byte alignment. The safe allocator refuses a free whose alignment differs from the allocation's, so the repeat gate's second prepare of "flashnext factored experts down 64 pairs" aborted the contract run before every later kernel.
… the scores' first n, not of all three of its scratch rows
attend() kept the scores, their bounds and the probabilities in one scratch of 3n and looped the probabilities over the
whole of it, so the two loop operands differed in length, which Zig forbids; a ReleaseSafe runner stopped there ("for
loop over objects with non-equal lengths").
|
Updated the description: it now opens with what the change buys (a ReleaseSafe runner reaches its |
|
Thanks, Bizuayeu. All three are real ReleaseSafe faults in the test adapters, and each fix is the smallest one that
We reproduced each panic on the host from the adapters' own code: the misaligned free, The last check is ours to run. We're running the ReleaseSafe kernel contract |
|
This is in TensorFold 1.1.8, released today. Thank you, @Bizuayeu! |
What this changes
What it buys: a ReleaseSafe build of the contract's runner reaches its
SUMMARY, with the default build's results,instead of stopping three times. Only the contract's test adapters change, so prompt and decode speed and the served
bytes cannot move.
zig build kernel-contractbuilds its runner at the build's default, fast. Built ReleaseSafe instead (as one does tocheck a runner), the CUDA contract stops three times before its
SUMMARY, each in a test adapter's host code:Fexp.drop()inzig/tests/cuda/contract_flashnext_fexp.zigfreed its expert list ([]usize) and its gate and upscale rows (
[]f32) throughstd.mem.sliceAsBytes, at byte alignment, so the repeat gate's secondprepareof"flashnext factored experts down 64 pairs" panics:
free of [addr: ef747402f070, len: 64 (0x40) align: 1] mismatches allocation of [addr: ef747402f070, len: 64 (0x40) align: 8](contract_flashnext_fexp.zig:108inprepare, fromgates.zig:172inrepeat).drop()now frees each slice at its own type; freeing an empty slicedoes nothing, so the length check goes too.
Rows.stress()inzig/tests/cuda/contract_window_rows.zigcopied the first cache row onto every row startingwith the first, so its first
@memcpyhad the same source and destination:@memcpy arguments alias(
contract_window_rows.zig:249instress, fromgates.zig:284insums, the window_rows decode case). The loopnow starts at the second row; every cache holds at least one row (
crowis the ring, the prompt's keys and rows,or streams × ring).
Chunks.attend()inzig/tests/cuda/contract_glm_attn.zig(mine, from glm/cuda: KV pages from the core pool, kept states sized by their position (#1142) #1163) keeps a row's scores, their boundsand its probabilities in one scratch of 3n and looped the probabilities over the whole scratch:
for (p, s)withpn long ands3n,for loop over objects with non-equal lengths(the GLM chunk attention's nonfinite gate, ona page table out of order). It now loops over the scores' first n,
s[0..n], which is what the fast build happenedto read.
None changes what a gate computes: the first frees scratch at the end of a gate, the second skips a copy of a row
onto itself, the third reads the n scores the probabilities come from.
Receipt
fa735852d(main1b4f2f979and the three commits) against main1b4f2f979; GB10 (sm_121,128 GB unified memory); driver 580.178.04; nvcc 13.3.1;
tf-cuda-testbuilt withzig build compile-cuda-tests -Doptimize=ReleaseSafe -Dnvcc=..., run astf-cuda-test kernel-contract.controls going red, and Qwen 3.5's two decode attention nonfinite lines the run marks KNOWN, CUDA kernel contract: the Qwen 3.5 decode attention cases launch
tf_q35_attn_partialandtf_q35_attn_partial8at 192 threads, past their__launch_bounds__(128), so every gate fails withCUDA_ERROR_INVALID_VALUE#1267). With the fexpfix alone it stopped at the window_rows panic after 744 PASS, the same 76 FAIL lines and no
SUMMARY; with both,at the GLM attention panic after 1,286 PASS.
SUMMARY(5,821 s):SUMMARY 340 kernels: 340 pass gates 1-4, 0 fail, 5 known findings; 94 controls: 94 red as planted, 0 missed; 231 limit problems. Its 293 FAIL lines are,numbers aside, the same lines as a default-build run's on glm cuda: a decode step's select gives each row a 1,024-thread block #1424's branch, whose
SUMMARYdiffers only by thatbranch's one added kernel, and both runs exit 1 on the same 231 limit problems (
contract_tests.zig:594).zig build compile-cuda-tests10 of 10 steps,zig fmt --check, the lean check (no new problemsagainst main).
Checklist
code changes.)
"draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (Noengine code changes.)
arithmetic changes.)
SUMMARYline is in the receipt. (No kernel changes; the CUDASUMMARYline is in the receipt.)repeat and sums gates that stopped before run through after.)