Skip to content

host steps: four that no CI job runs build and pass again (part of #2031) - #2409

Merged
ashhart merged 5 commits into
mainfrom
lead/mac-host-steps
Nov 7, 2026
Merged

ashhart merged 5 commits into
mainfrom
lead/mac-host-steps

Conversation

@ashhart

@ashhart ashhart commented Nov 7, 2026

Copy link
Copy Markdown
Owner

Part of #2031.

CI's jobs run zig build test and a few named steps, so a host step outside them can break and stay broken unseen. We ran every -host step on a Mac from a clean main: 405 steps, 395 passing. This fixes four of the failures. Each has a small cause, and every fix is in a build file or a test.

  • test-gemma4-affine-host: the test pins core/affine_mm.metal's bytes. A comment in that file was rewrapped to fit 120 columns, and the pin kept the old hash. The pin now holds the current bytes; lane_projection.metal's pin already matched.
  • compile-fsr-metal-contract-host and compile-gemma4-contract-host: the Metal contract runner now imports masked_sources. Only tf-kernel-contract was given that module, not these two compile-only copies of the runner, so both failed with "no module named 'masked_sources'". Both now import it the same way.
  • compile-metal-residency-host: native/metal.zig now opens Kimi K3, and kimi_k3 was wired into slide_host.zig's copy of the native module but not this one. This adds it, as slide_host does. A file can be the root of only one module, so Drama and Kimi now share one inert-metallib module.
  • test-shared-head-cost-metal-host (and "zero grant" in test-drafter-skip-metal-host): Backend.facts() now asks the device for tensor units. These two tests left the engine's device undefined, so facts() sent a message to an uninitialized pointer. It crashed in objc_msgSend in some runs and not others: 15 of 20 direct runs of one test aborted, 20 of 20 of the other. Both tests now set the host object that draft_sources_metal_test.zig already uses. It names no Metal architecture, so without a GPU tensorUnits() is false. The fixed binaries pass 20 of 20 runs each.

Two more failures from the same sweep are already open elsewhere: test-glm-snapshots-host is #2403, and test-exl3-prompt-host's missing b32 field goes with #2094. test-cuda-host's exl3_load allocation count on macOS is #2398. test-qwen-serving-host is a reviewed-bytes gate, and we've left it alone.

Tests

All six steps above pass on this branch, and the landing gate's host checks pass. No engine source changes, so there's no model or GPU run.

Step: test-gemma4-affine-host. Error: "landed Metal source bytes are pinned without
shader compilation" fails expectEqualStrings: expected d8f47cbb..., found a91a3539...
Cause: cf10a6b rewrapped one comment in core/affine_mm.metal to fit 120 columns and
left the pin at the old bytes (the file at 6d412e7 hashes to the old pin; that comment
is the only change since). Not macOS-only: the step is built on every OS but is not in
zig build test, so no CI job runs it. The lane_projection pin already matches.
Steps: compile-fsr-metal-contract-host and compile-gemma4-contract-host. Error: "no module
named 'masked_sources' available within module 'contract_main'" ('runner' for Gemma), from
tests/contract/masked_attention_more.zig:5. Cause: 17e6242 made metal_main.zig import
masked_sources and gave it to tf-kernel-contract (kernel_contract.zig) but not to these two
compile-only copies of the same runner. Not macOS-only by logic: both steps exist only on
macOS, and no CI job builds them.
Step: compile-metal-residency-host. Error: "no module named 'kimi_k3' available within
module 'host'" at native/kimi_metal.zig:4. Cause: 2c4fa0c made native/metal.zig open
Kimi K3 through kimi_metal.zig, and 57ff96d wired kimi_k3 and its inert metallib into
slide_host.zig's copy of that module but not into this one. The inert bytes are now one
module shared by Drama and Kimi (a file roots one module). Not macOS-only by logic: the
step exists only on macOS and no CI job builds it. test-metal-formats-host, on the same
module, still passes.
Step: test-shared-head-cost-metal-host. Error: "shared head costs: GLM Metal MTP retains
costed copy grants" terminated with signal ABRT, "Segmentation fault at address 0x10" in
objc_msgSend from Device.responds <- generation <- tensorUnits <- Backend.facts.
Cause: ae320aa made facts() ask e.device.tensorUnits(), but this test and "zero grant:
GLM Metal MTP resumes after a long priced skip" (test-drafter-skip-metal-host) leave the
Engine undefined, so facts() messages an uninitialized id. The outcome changes from process
to process: on origin/main the shared-head-cost binary aborted in 15 of 20 direct runs, and
the zero-grant binary segfaulted in 20 of 20 direct runs while passing under the build
runner. Both tests now set the NSObject host object that draft_sources_metal_test.zig
already uses: it names no Metal architecture, so tensorUnits() is false without a GPU.
Fixed binaries: 20 of 20 each. Not macOS-only by logic: the tests build only on macOS.
@ashhart
ashhart force-pushed the lead/mac-host-steps branch from 7608567 to 52cc350 Compare November 7, 2026 05:09
@ashhart
ashhart merged commit ba82536 into main Nov 7, 2026
0 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant