Repository navigation
Conversation
On RDNA 3.5, v_dot2_f32_bf16 and v_dot2_f32_f16 round a sum differently when one lane's product is -0 instead of +0. The 64-row prompt tile multiplies P by V two keys at a time and zeroed V only past the span, so when a row saw an odd number of keys in its last tile, the partner key fed +0 in a run whose chunk ended there but p = 0 times its real V, a -0 for a negative value, in a run whose chunk ended later, and a row's bytes hung on where its chunk began (#2317). Each row now zeroes every value past its own causal edge, so all runs feed the same dot2 lanes on every chip. This also drops the 0 x inf NaN past the causal edge from the prefill contract's known finding, adds cut 1 to the chunk-start test, moves the gfx1151 pin to the new bits (gfx1152 reproduced the old pin exactly), and refreshes the prompt and image tiles' VGPR records on every chip.
RDNA 3.5's dot2 rounds a sum with a -0 lane apart from the same sum with +0, which made #2317's prompt rows depend on their chunk start, while gfx1201's does not. A caps field, dot2_signed_zero, records that fact per generation and reaches the kernels as TF_DOT2_SIGNED_ZERO, so the prompt tile can keep main's code on the chips that do not need the per-row mask.
The per-row V mask changes no bits on gfx1201 but cost 0.8 to 0.9% of its prompt speed (#2357 review). The tile's body moves to prefill.hpp with MASKED and EDGE instances: the served tile is masked only where dot2_signed_zero holds and is main's code elsewhere, the same instructions on gfx1201. There a new edge launch follows each tile: a block scans the V rows its rows are masked from, page ids read once so it does not spill, and reruns masked only when one is non-finite. A kernel test checks float64's classes, the tile's own bytes in every unplanted dim, and chunk starts, paged and flat.
The prompt tile's gate-1 plant mixed an infinity past a row's causal edge with one under fp32's least weight, which still fails, so the contract stayed green with no mask or edge pass at all (#2357 review). Four entries, flat and paged in both cache types, plant +inf, -inf and NaN only past rows' edges over a span a page past the last row and expect a clean gate 1. Each has a planted control, main's tile without the edge pass, which must go red at that gate.
…or returns instead of aborting the process Runner.keep opens its command encoder before the per-layer commits and keep uploads; any error after that left the encoder open, and the objc pool's release then asserted "Command encoder released without endEncoding", so the process aborted and the error was lost. A guarded errdefer ends it, as qwen27_keep's other keep sites do, so the errors after the encoder's own end (a GPU failure, a refused path) don't end it twice. tf-qwen27-keep-error fails each of the keep's allocations in turn on a synthetic model: on main it aborts, here each returns OutOfMemory and the same round keeps on retry; then a refused path after the end returns BadPath. Found on the pipeline stage path (#2005).
… cuda_snaps.zig reads for TF_GLM_PROMPT_TAIL
On TF_FLASHNEXT_KV_PAGES, a shared round with two or more drafting streams ran the MTP head over one grouped pass whose lanes had no page table, so the head wrote and read its rows as if its cache were contiguous from the first page: into other streams' pages. With three equal 16K prompts at once, one reply of three differed from its solo run. The lone head's pass already carries it. The grouped lane is built in headLane now, and a host test checks that it carries its draft's page table, length and rows (without `.pg` it fails: expected 28672, found 0).
…s, each one's exchanges and glue under the other's sites
…and counts it once (#1404) Engine.prefillChunk reads and clears the split flag before it hands a paired step to prompt_pair.prefill, which never set it on its two forwards, so with #1404 a paired step ran the indexer whole on every rank. The step's flag now reaches both chunks: the header carries the one choice rank 0 makes at the step's start, and index_splits counts the step once. The twin buffers already share the first chunk's select scratch, whose pools hold #1404's world rows more.
…prompt pairs a step past the split's position The check called Leader.chunk without the pair argument, so compile-cuda-tests failed on the stacked tree. It now steps by Chunk.rows, pairs where it can and counts the splits a step at a time; pairing leaves the first four prompts fewer split steps, so a fifth prompt of three chunks and min_rows pairs its second step past TF_GLM_INDEX_SPLIT.
…t a time's do prompt_pair.prefill readies the index rings once, at the pair's start. Where a chunk at a time rebases them only at the second chunk (from chunk 1 of rings based at 0: after a resume from a state kept at an odd chunk, or once decoders finish mid-prompt), the pair rebased at the first: the tokens agree, but the rings' bytes and base do not. pairedAt now pairs only where both rebase at the first chunk or neither rebases, judging a resumed step by its kept state's base (restored after the choice), so one unpaired chunk realigns a prompt; prefill refuses any other pair on every rank. glm-pair-check adds a prompt of four chunks with a mark at chunk 1, counts the paired steps with the rings' base followed on the host, and compares both twins' bases.
At TP=1 a pair has no exchanges to hide and no receipt, yet the twin was allocated and the side stream ran at top priority. prompt_pair.wanted(on, world) is now on and world > 1, for both; pairedAt already pairs only with a twin. glm-pair-check needs other ranks.
… size added or taken away stops the build twin_sizes tied the fields a twin holds apart to make's size indexes by hand, beside a list of their names. twin_at now maps each field to its index: make reads those fields from it, the twin's allocation and copy take it too, and make's sizes are an array of twin_checked entries, so a future size entry fails the build until the indexes are checked again.
…nd Cancel with the engine's types, so the engine and the lanes keep room under the line limit
…er holds no twin A solo pass and a paired step both reach the other ranks as a part of two chunks, so one server cannot mix them. With the wider buffers set, the pair stands aside and makes no twin.
Step: test-gemma4-affine-host. Error: "landed Metal source bytes are pinned without shader compilation" fails expectEqualStrings: expected d8f47cbb..., found a91a3539... Cause: cf10a6b rewrapped one comment in core/affine_mm.metal to fit 120 columns and left the pin at the old bytes (the file at 6d412e7 hashes to the old pin; that comment is the only change since). Not macOS-only: the step is built on every OS but is not in zig build test, so no CI job runs it. The lane_projection pin already matches.
Steps: compile-fsr-metal-contract-host and compile-gemma4-contract-host. Error: "no module
named 'masked_sources' available within module 'contract_main'" ('runner' for Gemma), from
tests/contract/masked_attention_more.zig:5. Cause: 17e6242 made metal_main.zig import
masked_sources and gave it to tf-kernel-contract (kernel_contract.zig) but not to these two
compile-only copies of the same runner. Not macOS-only by logic: both steps exist only on
macOS, and no CI job builds them.
Step: compile-metal-residency-host. Error: "no module named 'kimi_k3' available within module 'host'" at native/kimi_metal.zig:4. Cause: 2c4fa0c made native/metal.zig open Kimi K3 through kimi_metal.zig, and 57ff96d wired kimi_k3 and its inert metallib into slide_host.zig's copy of that module but not into this one. The inert bytes are now one module shared by Drama and Kimi (a file roots one module). Not macOS-only by logic: the step exists only on macOS and no CI job builds it. test-metal-formats-host, on the same module, still passes.
Step: test-shared-head-cost-metal-host. Error: "shared head costs: GLM Metal MTP retains costed copy grants" terminated with signal ABRT, "Segmentation fault at address 0x10" in objc_msgSend from Device.responds <- generation <- tensorUnits <- Backend.facts. Cause: ae320aa made facts() ask e.device.tensorUnits(), but this test and "zero grant: GLM Metal MTP resumes after a long priced skip" (test-drafter-skip-metal-host) leave the Engine undefined, so facts() messages an uninitialized id. The outcome changes from process to process: on origin/main the shared-head-cost binary aborted in 15 of 20 direct runs, and the zero-grant binary segfaulted in 20 of 20 direct runs while passing under the build runner. Both tests now set the NSObject host object that draft_sources_metal_test.zig already uses: it names no Metal architecture, so tensorUnits() is false without a GPU. Fixed binaries: 20 of 20 each. Not macOS-only by logic: the tests build only on macOS.
`tensorfold models` prints one line a checkpoint and takes no argument, so a program that wants the inventory
parses the prose or reads the Hugging Face cache itself; `models --json` printed the usage and exited 2.
`models --json` now prints the same checkpoints as one document, {"schema": 1, "models": [...]}, each entry the
line's repo, family and model_type; a cache that does not exist yet is an empty list. The lines are unchanged,
and any other word after `models` is still the usage. The README's Models on disk names the flag.
From the review: the CHANGELOG line under Unreleased, `models [--json]` in the native binary's usage text (both
lines in main.zig), and the README says that `family` is a display title and `model_type` the key to match on.
…Metal and CUDA tf-moss-image-blocks runs one step's branch through Model.steps (on both backends): the embedding, each kept block alone on a float32 run's stream rounded to bf16, then the final layer. Each must be within twice torch bf16's distance from float32 in cosine error and worst row, and a second run must repeat the first byte for byte. moss_image_case.py blocks records the float32 stream and torch bf16's steps alone, on the case's device and on the CPU; the limit is the farther of the two (reference.wider), since torch's own bf16 runs disagree at a massive-activation row (block 0 of the fox's step 1, negative branch: worst row 0.00687 on MPS, 0.00960 on the CPU). tf-moss-image's whole forwards are reported beside torch bf16's and not gated, as the encoder's whole prompt is not; a repeat that differs still fails it. Seven steps' branches: 305 of 305 gates pass on Metal (at most 0.50 of the limit) and on an RTX 5090 (at most 0.60), each run repeated byte for byte; a row planted 5% off in block 20 fails at blocks 20 and 21. Model.forward's modulation and embedding moved into helpers that Model.steps shares; both cases' latents are byte-identical before and after on both backends.
…o zig/build/cuda.zig stays under 600 lines With main at 600 lines, this branch's tf-moss-image-blocks took zig/build/cuda.zig to 602. Both MOSS-Image tools move, unchanged, to one file that cuda.zig calls once.
6 of 7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI for a landing train: #2357 #2376 #2403 #2244 #1423 #2409 #2284 #2412 #2364 #2248 #2294 stacked on main in that order. Nothing merges from this PR: each PR above lands on its own once this tip is green, and this PR is then closed.