[garvrao80]

What this changes

Use zig build -Dfixed-cpu test-host --test-timeout 20m in Linux no-GPU CI, preserving main's cache restore, restat and save policy. The aggregate collects test, test-* and check-* after build registration, including compile-only checks. It rejects empty named checks, stale exclusions and aliases that would run excluded work.

Run the graph regression suite only when build.zig, zig/build/host_tests.zig or tools/zig/host_tests_test.py changes. When Git cannot resolve a comparison base, run the suite. The Linux job retains a 60-minute cap and a 20-minute per-test timeout for the cold DeepSeek enumeration.

• Issue: Refs CI never reaches 246 of the 393 named test steps, and nine of them fail on main #2031.
• Stacked on: main at 964f98b0ba640dee5c139b3d567e597964373bca. Merged main into the published branch to pick up the fixture fixes.
• atc overlap --json: incomplete (pullFilesLimit, pullFilesUnavailable, timeLimit). The GitHub inventory covered 182 open PRs and 31 overlaps. No new conflicts relative to main. ATC reported a build.zig overlap with lfm3 CUDA: serve the block FP8 export weight-only and the MLX 4-bit export as stored (stacked on #1508) #1634; the conflict hunk is identical on the pinned main and this branch.

Reviewed exclusions

Two optional xgrammar checks need a separate checkout and flag. check-release-paths runs in its own CI job. Qwen27 residency (#2331) and GLM snapshots (#2403) passed on pinned main and are now part of test-host. The former reported 60 passed and 2 skipped; the latter passed all 21 tests.

Each remaining temporary exclusion failed on pinned Linux main. Remove the entry and its matching graph fixture entry when the repair lands and the named check passes.

Check	Baseline failure and repair
test-exl3-prompt-host	The recording fixture omits the b32 initializer. Tracked in #2094.
test-qwen35-draft-frame-host, test-qwen35-frame-invalidation-host	Both fail to compile on Linux at zig/qwen35_draft_frame_host.zig:239 because cuda_draft_tree.nodes now returns an error union. The draft-frame step also has the same problem at zig/qwen35_draft_equality.zig:140. The previous Linux revision aborted in affine4_kernels.zig:249 through the recording fixture's uninitialized seg_fn. Tracked in #2031.
test-qwen35-draft-trees-host	Fails to compile at zig/qwen35_draft_equality.zig:140 for the same draft tree return type change. Tracked in #2031.
test-qwen27-prompt-rows-host	The malformed/over-cap fixture expected error.InvalidEnvironment and got success. Pinned main reports 12 passed and 1 failed. Tracked in #2031.
test-qwen-serving-host	Both the reviewed CUDA source hash and the launch-sequence digest differ after #2418. The launch-sequence digest is d21cd8152c54, expected 265a34f9f0ae. Tracked in #2031.
test-gradient-serving	The first mismatch after #2418 is zig/src/cuda/aot.zig. Tracked in #2031.
test-cluster-loader-host	Fails to compile on pinned main because the fixture cannot import canon, which zig/src/cluster/plan.zig now uses. Tracked in #2031.
test-gemma4-affine-host	The Metal affine_mm SHA-256 changed again after #2409. Actual c959eb1f5ac8, expected a91a353981c4. Pinned main reports 138 passed, 6 skipped and 1 failed. Tracked in #2031.

The maintainer's hosted trace found no additional tracked-file declarations needed for the new checks. The inherited portable-names case was fixed separately in #2311.

Checks on this head

• Head commit: 5cf22489dfc9a01f25bcbbd058be534ac3c37f52.
• Lean checker, python3 tools/zig/lean_check.py: 116 inherited findings on the base and head, 0 new. zig fmt --check build.zig zig and git diff --check passed.
• Tests: python3 -m unittest discover -s tools/zig -p host_tests_test.py -v passed 9/9. The configured -Dfixed-cpu graph reaches all 418 eligible checks, 540 run steps and 536 compile steps, with zero guard failures. The workflow guard previously passed 12 cases covering both event types, watched paths, deletion and missing bases. The wider local sweep reproduced the three additional failures later verified directly on main, then aborted on WSL storage I/O errors. The focused tests and configured graph completed.
• CI: GitHub Actions is starting on this revision.
• Not run locally: Windows, macOS/Xcode and device or checkpoint tests. The revised aggregate is a native Linux CI step.

Receipt

n/a for inference. This changes build-graph registration, Linux CI, documentation and graph tests. Local validation uses Zig 0.17.0 on Linux x86_64 (WSL). GPU toolchain, checkpoint, token exactness, speed and weight licensing do not apply.

Checklist

• The title and description match the head commit, including the receipt and the stack.
• New tests fail before the change, pass after it, and skip cleanly without their dependency.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths in the diff or its commits. No AI attribution lines.

[ashhart]

Thanks @garvrao80, this is the shape we agreed on #2031, and the guards against stale exclusions and aliases are a good addition. One change so it can land green: please put the checks still red on main on the exclusion list, each with the reason "red on main, #2031" and the PR that fixes it, so each comes off when its fix lands. #2098 has already fixed two of the eight. Of the rest, the two source-pin checks (test-qwen-serving-host and test-gradient-serving) are ours to review, and we'll do that next.

[ashhart]

@garvrao80, the two source pins are ours to settle. We'll re-baseline both to main's bytes in one reviewed commit that says no device re-qualification is claimed, and drop ck_experts.cu from the gradient manifest, since its own note says the learners don't read it. Until that lands, please put test-qwen-serving-host and test-gradient-serving on the exclusion list with the reason "source pins awaiting re-baseline (#2031)", beside the six fixture prerequisites, so the gate can land green.

[ashhart]

Thanks @garvrao80. CI changed under this PR today, so here's what it costs now and what's needed to land it. In short, it's worth having: it catches the 246 named steps CI never ran. With main's cache it costs a few minutes on most pull requests, and there are four asks.

What it costs

These are measured on hosted Linux runners, with this PR's test-host and exclusions on current main, one run each. Runners differ, so read them as rough sizes.

	test (today)	test-host
Cold, as every main commit runs	11.7 min	28.4 min (12,328 passed, 94 skipped, all green)
Restored cache, nothing changed	5.3 min	8.6 min
Restored, one family file changed	6.2 min	14.7 min
Restored, one zig/src/core file changed	5.8 min	over 21 min (cancelled, unfinished)

The restored rows used main's cache with test passes reused, as #2274 makes CI do. They also predate the tool that stops C and C++ from recompiling, so both columns should come down a little. A cold test-host run spends 12 minutes on one binary, deepseek_v41/device_host_test.zig, which is why it needs the 20-minute test timeout. Then qwen4_family (4 min) and rows_startup_scale_test (3 min).

The saved cache grows from 0.84 GB to 1.87 GB a save, against GitHub's 10 GB for the repository. That fits, but fewer main saves stay in the budget.

What's needed

1. Rebase on main. CI: main's build cache fits every runner, and release-paths and macOS use it too #2252 is on main, and CI: a pull request reuses main's test passes and C compiles, so it runs only what its diff changed #2274 is next in the landing queue. no-gpu.yml has changed a lot: the new line becomes zig build -Dfixed-cpu test-host --test-timeout 20m, without -Drerun-tests, which CI: a pull request reuses main's test passes and C compiles, so it runs only what its diff changed #2274 removes.
2. Run Host test graph regressions only when it can change. It takes about 2 minutes on every run, and only zig/build/host_tests.zig, tools/zig/host_tests_test.py and build.zig affect it. A git diff --quiet guard like CI: a pull request reuses main's test passes and C compiles, so it runs only what its diff changed #2274's Restored cache regressions step would do.
3. Ours: the tests' file reads. With CI: a pull request reuses main's test passes and C compiles, so it runs only what its diff changed #2274, a pull request reuses a test's pass unless the test's binary or a declared input changed. A test that reads a tracked file while it runs has to declare it in zig/build/test_reads.zig. The 246 new steps haven't been traced yet; I'll trace them and add what's needed. This shouldn't land before that.
4. Optional: the DeepSeek enumeration. Building device_host_test in ReleaseSafe, as Build: qwen4's and GGUF's host tests compile with ReleaseSafe #2240 does for the heaviest binaries, would likely bring its 12 minutes down a long way and let the timeout go back to 10 minutes.

Once 1 and 2 are in and the trace is done, it can go in the landing queue.

[ashhart]

@garvrao80, an update on ask 3 (the file-read trace). It's settled on our side. We traced a cold test-host on a hosted runner: the trace saw every one of the 618 runs the build summary lists, across 273 binaries. None of the 246 new steps reads tracked files without either declaring them or always running, so this PR needs nothing added to zig/build/test_reads.zig. The one gap the trace found was already on main (the portable-names test), and #2311 fixes it.

So what's left is yours: ask 1 (the rebase) and ask 2 (guarding the two-minute Host test graph regressions step). Ask 4 stays optional.

[ashhart]

/atc exchange 4659daa814b96b2b72c65474fef9b39704cbe31fab546e38f616b4fc0a2a626f