Repository navigation
Conversation
Keep the binary accumulate operation explicit and cover three- and four-rank host reductions with literal rounding and order witnesses. Device checks retain their actual NCCL oracle above two ranks.
- drop the duplicate Clef-Flash on HIP bullet; keep the one carrying the current `decisions`-key wording and the full PR list (zig/src/server/cli_capabilities.zig:30, docs/api.md:184) - Metal prefill widths: Flash Next's engine reports none, so 'every Metal text family' gains the exception (no prefill_widths reference under zig/src/families/flashnext/) - native FP decode: bit-equality holds unless a weight carries an e4m3 NaN code, which only a broken checkpoint has (docs/kernel-contract.md:367-370) - cache-tool test trigger list: the tool's test and build.zig.zon also trigger it (.github/workflows/no-gpu.yml:93-94)
…one Mac it no longer happens (#2237)
…they run from v0.6.6, and the vocab command puts its glob first
…hing The fresh runs of check_invariants' resumed check and check_radix's stress ran inside the resumed runs' keepPrompts scope, on the same prompts, so they resumed too and each check compared two resumed runs. Each fresh run now evicts every kept prompt first, and its check fails by name when a fresh reply still restored tokens.
…at any checkout path length
…so a mistyped or missing option never opens a real endpoint (#1594 step 2a) The commands looked their flags up by name and ignored the rest, so `node ... --dry-run` (node has no dry run) opened the real MCDMA endpoint and waited for its peer. node and serve now refuse, before reading the cluster file, an unknown flag, a flag missing its value, a value given to a switch and a stray argument, each by name, with the usage (exit 2). serve's usage line lists --probe, which it already took.
…index (#1594 step 2a) A CUDA node was read with the macOS probes, so a GB10 failed them. It is now read with meminfo, nvidia-smi, ibv_devinfo -v, df and machine-id. nvidia-smi asks only for name and memory, so a GB10 stays NVIDIA GB10. A memory column of [N/A] stays gpu_limit 0, and a declared gpu_limit_gb still fills it. A missing nvidia-smi or ibv_devinfo is an empty report, not a failed probe. ibv_devinfo keeps the first RoCE v2 GID. In the captured ibv_devinfo -v output that row is index 1. vendor_part_id 4129 is ConnectX-7. ::ffff:a.b.c.d parses as the same GID as the colon form. The inventory wire version is 2 because each port carries that GID index. A CUDA host probed this way reports no Thunderbolt peers, so the probed topology never joins a Mac to a GB10, and serve's fake bring-up of a disaggregated model (serve --cluster --probes) could not settle. It now cables the links the file declares, as the real nodes open them, and the probed topology only when the file declares none. Its test gives the GB10 Linux probe output (the captured ibv_devinfo -v, now testdata/gb10-ibv.txt) and checks both. No test names a machine. test-cluster: 110 passed, 6 skipped. test-tp: 70 passed, 3 skipped. tools/zig/test_fabric.sh: every file passes (fabric 40 passed, 1 skipped; the cluster's MCDMA stub bring-up 3 of 3).
… 2a) A probed GB10 with gpu_limit still 0 used to plan anyway. weight() treats 0 as 1 byte and the fit then fails with a Mac wired-limit sentence. launch.inventory now returns GpuLimitMissing before any plan, and names the node: refused: NAME is a GB10 and nvidia-smi printed no memory. Set gpu_limit_gb for NAME in the cluster file. init, check, serve and node exit 1 with that line and no stack trace; serve of a disaggregated model catches it too, in its fake hosts' inventories and in its own, and its fake-host test checks that. A declared gpu_limit_gb is kept, including an explicit 0, which is still refused. MemTotal is not used as a stand-in. test-cluster: 111 passed, 6 skipped. test-tp: 70 passed, 3 skipped. tools/zig/test_fabric.sh: every file passes (fabric 40 passed, 1 skipped; the cluster's MCDMA stub bring-up 3 of 3).
The CUDA binary now imports the cluster module and handles cluster init, check and status, serve --cluster, and node --name before it looks for a model. Those commands need no driver and no GPU. On top of step 2a, the CUDA binary's node takes the cluster CLI's MCDMA path, through the library each node names in the cluster file. That binary is one compilation, and Zig allows a file in only one module. canon.zig stays the canon module GLM already uses, and cluster imports it. verbs.zig and verbs_abi.zig stay the verbs module TP already uses, and fabric imports it. The CUDA CLI passes those same modules through. Other builds make their own. Those modules' tests do not run inside cluster or fabric. test-cluster also roots the canon module. test_fabric.sh and zig build test both root the verbs module, so those tests stay in the suite upstream CI runs. The verbs.h size check is the arm64 macOS table, so it skips on other systems. test-cluster: 111 passed, 6 skipped. test-tp: 70 passed, 3 skipped. test_fabric.sh: fabric 37 passed; verbs 3 passed, 1 skipped; cluster stub 3 of 3. verbs on zig build test: 3 passed, 1 skipped.
…so the script runs on main again (#1594 step 2a)
…t 600 lines (#1594 step 2a)
…when its plan fails (#1594 step 2a)
…to main's one-line wording
… 16-byte chunk hpattn_kernel and hpattn8_kernel staged each chunk through PT[row / PTOK], a runtime divide and a dependent table load per chunk. An item's 32 keys sit in one page whenever PTOK is a multiple of 32, so its first page is read once and a row's offset is row - lo; a row past that page (PTOK 48) still looks up its own. Addresses only: the same bytes land in the same slots.
…, with the page size a constant TF_QWEN_KV_PAGES's default page holds 256 positions. hpattn_p256_kernel is hpattn_kernel with PTOK a compile-time 256: an item's keys sit in one page (static_assert), so it reads the table once and finds its base once, with no runtime divide or modulo. cuda_prompt.zig takes it for bf16 pages of 256; every other page size keeps hpattn_kernel. Addresses only: the same bytes land in the same slots. The kernel contract checks it against contiguous rows twice, with a swapped page table as its control, and registers its usage.
…e, not once per key Bf16Tiles::fetch and Fp8Tiles::fetch found each key's page with a runtime pos / ptok and pos % ptok. A thread's keys run in order, so it keeps the page it is in (its first position and base) and looks a new one up only when a key leaves it. Addresses only: the same bytes are copied. On a GB10 this brings paged decode on Qwen3.8-27B to the window path's speed (attn_partial was 0.327 against 0.266 ms a call before).
TF_QWEN_KV_PAGES unset now pages at 256 positions (a divisor of the 4096-row prompt chunk, as TF_QWEN_KV_SHARE needs) and TF_QWEN_KV_PAGES_ALL unset puts the engine's stream on pages too. 0 keeps a window each (PAGES) or the engine's window (ALL). Beside --slide, whose learner reads whole windows, an unset TF_QWEN_KV_PAGES keeps windows; set explicitly it is still refused. Both are still read through the profile registry, which supplies no value. Its reporting follows the new default: a count entry names what its reader takes when unset (256 for TF_QWEN_KV_PAGES), and TF_QWEN_KV_PAGES_ALL takes the `off` syntax, on unless exactly 0, so the startup line reports 0 and not 256 or 1.
…les, so its first row is looked up once a page, not once a tile
…ids in shared memory (up to 768 pages), hpattn_p256_kernel past that
…page, not one run a page and piece
…; a dense one pages at 256 positions TF_QWEN_KV_PAGES unset now pages a dense Qwen3.5 (the 27B) and keeps a MoE (the 35B-A3B) on windows: the 35B's warm turn measured slower on pages, the 27B's no different. The reader takes the MoE bit from the loaded geometry, so a malformed value is still refused before the load. The profile registry reports a count against what this family takes unset (unset_in), and TF_QWEN_KV_PAGES_ALL takes 1 or 0, refusing anything else by name.
…set off again, both as on main TF_QWEN_KV_PAGES unset keeps a window each on the 27B and the 35B-A3B alike: the 27B measured about 0.3 % slower on 8k and 32k prompts and the warm turn on pages. The reader still takes the MoE bit from the loaded geometry, so a per-model default is one constant (unset_pages) once #2043 closes the gap. With pages unset there is no pool, so --parallel reserves a window a stream as on main; set, it is a cap and a request can wait for pages. TF_QWEN_KV_PAGES_ALL is on only at exactly 1 again, so setting only TF_QWEN_KV_PAGES gives what it gave on main. The profile registry, its startup line and docs/profiles.md are main's again.
…oE), _ALL unset off, --parallel a cap only with pages test-qwen35-pages-default-host runs them, part of test-cuda-host. A planted wrong admit count fails the cap test.
… unset_pages a plain 0 With no per-model default left, TF_QWEN_KV_PAGES no longer depends on the model or --slide, so paging() is main's again and open() reads it, and refuses QwenPagesNoLearner and QwenShareNeedsPages, before Engine.init.
…21, measured with the fp8 tiles' one page lookup
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI for a landing train: #2366 #2371 #2309 #2374 #2377 #2381 #2361 #2380 #2365 #2088 #2011 stacked on main in that order. Nothing merges from this PR: each PR above lands on its own once this tip is green, and this PR is then closed.