Skip to content

cli: run the pull tests' fake hub on std.Io.net so Windows builds them - #2284

Merged
ashhart merged 3 commits into
ashhart:mainfrom
ElderOrb:cli-pull-tests-portable
Nov 7, 2026
Merged

ashhart merged 3 commits into
ashhart:mainfrom
ElderOrb:cli-pull-tests-portable

Conversation

@ElderOrb

@ElderOrb ElderOrb commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

What this changes

zig build test-pull-security now compiles and passes on Windows, and the zig-windows job runs it. The fake hub in zig/src/cli/pull_test.zig runs on std.Io.net (listen, accept, streams, shutdown) instead of std.posix.system.socket/bind/listen/poll/accept/read/write on raw integer fds. That raw layer does not exist on Windows: the step links libc, so std.posix.system is std.c, whose fd_t there is windows.HANDLE (*anyopaque), and std.posix.socket_t follows it, so @intCast of the socket result cannot compile. The file was POSIX-only by construction and CI ran it on ubuntu alone, which is why this never showed.

  • Issue: n/a - test plumbing; no engine, kernel or weight-format change.
  • Stacked on: main
  • atc overlap --json: 2 files. .github/workflows/no-gpu.yml is shared with ci: collect native Linux host checks and guard exclusions #2095 (a different job in the same file, so the merge rebases the step); no open pull request changes zig/src/cli/pull_test.zig. atc reported complete: false (pullFilesUnavailable, timeLimit).

Checks on this head

  • Head commit: c491e4a (rebased onto current main; only no-gpu.yml conflicted, resolved by keeping main's -Dfixed-cpu on the step we add)
  • Lean checker, python3 tools/zig/lean_check.py: 0 new problems.
  • zig fmt --check build.zig zig: clean.
  • Windows x86_64, Zig 0.17.0: zig build test-pull-security - 7/7 tests pass. Before this change the same command failed to compile the test binary.
  • Linux x86_64 (WSL2 Ubuntu, Zig 0.17.0): zig build test-pull-security -j4 - 7/7 tests pass, so the portable hub keeps the POSIX behaviour.
  • CI: not run; this pull request comes from a fork.

Receipt

n/a: tests and CI only. The change reaches no weights, kernels or inference path.

Checklist

  • The title and description match the head commit.
  • The affected tests did not compile on Windows before the change and pass there after it; on Linux they still pass.
  • Comments and docstrings are one line. No measurements or history in the source.
  • No personal data, machine names, internal hosts or local paths in the diff or its commits. No AI attribution lines.

@ElderOrb
ElderOrb force-pushed the cli-pull-tests-portable branch from 2cc0963 to c491e4a Compare November 5, 2026 22:03
@ashhart

ashhart commented Nov 5, 2026

Copy link
Copy Markdown
Owner

Thanks @ElderOrb, moving the fake hub onto std.Io.net is the right fix, and it keeps the same seven tests on a portable hub instead of special-casing Windows. Three small things before it queues:

  1. A rebase of the workflow step. no-gpu.yml's Windows job has changed on main since your base: every step now passes -Dfixed-cpu, and there's no -Drerun-tests option in today's build, so the step would be zig build -Dfixed-cpu test-pull-security.
  2. The file's first line gained a byte-order mark before name: No GPU; please drop it.
  3. A CHANGELOG line under Unreleased, since Windows now builds and runs these tests, for example: "Tests: the pull client's security tests build and run on Windows, on a portable fake hub, and CI runs them there."

Once it's rebased, CI's Windows run on the new head is the check we'll look for.

@ElderOrb

ElderOrb commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

All three are in, head e1267e502.

  1. Rebased onto current main: the step is zig build -Dfixed-cpu test-pull-security, matching the rest of the Windows job. The no-gpu.yml conflict was the only one, and it is gone.
  2. The byte-order mark is dropped; the diff on that file is now exactly one line, from a BOM-prefixed name: No GPU to name: No GPU.
  3. CHANGELOG.md under Unreleased: "Tests: the pull client's security tests build and run on Windows, on a portable fake hub, and CI runs them there."

Re-checked on the rebased base: zig build test-pull-security passes 7/7 on Windows x86_64, and zig build test-pull-security -j4 passes 7/7 on Linux x86_64 (WSL2 Ubuntu, Zig 0.17.0). zig fmt --check is clean and the lean checker finds nothing new. No workflow has run on this fork pull request yet, so if you approve the run, the Windows job on e1267e502 is the check to look at.

@ashhart

ashhart commented Nov 6, 2026

Copy link
Copy Markdown
Owner

Thanks @ElderOrb, all three are in: the step matches the rest of the Windows job, the first line is clean again, and
the CHANGELOG line is there. One small thing: the last hunk of no-gpu.yml adds two blank lines at the end of the
file (git apply warns about them). If you can drop them, good; otherwise we'll trim them, and add the PR number and
thanks to the CHANGELOG line, at landing. We'll approve the workflow run, and with the Windows job green on this
head it's ready.

@ashhart

ashhart commented Nov 6, 2026 •

Copy link
Copy Markdown
Owner

@ElderOrb, a correction and one question. I've trimmed the blank lines myself.

  1. The Windows step does run (correction). I had this wrong in an earlier version of this comment. test-pull-security is defined in zig/build/golden.zig and registered on Windows beside test-golden and fuzz-cortheon, so your 7 of 7 on Windows stands. The early return at zig/build/pull_tests.zig:5 drops only test-pull-host, the whole pull suite inside zig build test, and the reason its comment gives ("the fake hub listens on a POSIX socket") isn't true after this PR. Dropping that skip, if the whole suite passes on Windows, or fixing the comment can be a follow-up. It doesn't hold this PR.
  2. Connection: keep-alive after a closed connection (a question, not blocking). The fake hub now closes the connection after a whole-file answer (the new return false after whole() for the whole weights and the nested shard), but still sends Connection: keep-alive, and the pull client's std.http.Client keeps connections to reuse them. If Windows needed the close, Connection: close would tell the client.
  3. Blank lines: I pushed a one-line commit that drops the two blank lines at the end of no-gpu.yml.

It's back in the landing queue. It lands once the Windows job is green on the new head, and these tests take seconds there.

@ashhart
ashhart force-pushed the cli-pull-tests-portable branch 2 times, most recently from 86b8618 to c5ff3fa Compare November 6, 2026 08:37
ashhart added a commit to ElderOrb/TensorFold that referenced this pull request Nov 6, 2026
@ElderOrb
ElderOrb force-pushed the cli-pull-tests-portable branch from c5ff3fa to 7fb1375 Compare November 6, 2026 13:37
ElderOrb pushed a commit to ElderOrb/TensorFold that referenced this pull request Nov 6, 2026
@ElderOrb

ElderOrb commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main, head 7fb13759a: the branch is mergeable again, and the only conflict was the CHANGELOG bullet, which now carries the PR number and thanks you added.

On the keep-alive point you are right: two whole-file answers (/Org/Whole/.../weights.safetensors and the nested shard) return false from the handler, so the connection closes, while the header still says Connection: keep-alive; the no-range /Org/Flash/.../weights.safetensors answer keeps the connection open. I will send the header from the same keep decision the handler returns, in a follow-up so this pull request stays as landed. The alternative is to keep the connection open on all three, but the close is what exercises the client's reconnect path.

On zig/build/pull_tests.zig:5, agreed: the comment is stale and the skip can go once the whole pull suite passes on Windows. I will run it there first, and either drop the skip or add test-pull-host as its own Windows step, whichever the suite's runtime argues for. Also a follow-up, as you said.

Windows and Linux are re-checked on 7fb13759a: zig build test-pull-security passes 7/7 on Windows, and -j4 passes 7/7 on Linux (WSL2 Ubuntu, Zig 0.17.0). The workflow has not started on this head yet; once you approve the run, the Windows job is the one to read.

@ashhart

ashhart commented Nov 6, 2026

Copy link
Copy Markdown
Owner

/atc exchange 237b6665cc0d973fef36b63752311631fa0f5b3351c85b456c36e86e184c24e0

@ElderOrb

ElderOrb commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

/atc exchange 8c043b962cfbb8367401cc9db7c56b2aa18117e87e28755312686bc93bc346ed

@ElderOrb

ElderOrb commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor Author

@ashhart The two runs on 7fb13759a (No GPU, Zig format) are in action_required, so the Windows job has not started yet. Could you approve the run for this head? Everything else on the rebased branch is unchanged: 7/7 on test-pull-security locally on Windows and on Linux, and only the CHANGELOG bullet conflicted in the rebase.

@ashhart

ashhart commented Nov 6, 2026

Copy link
Copy Markdown
Owner

Thanks @ElderOrb, the runs are approved. It lands once the Windows job is green.

@ashhart

ashhart commented Nov 7, 2026

Copy link
Copy Markdown
Owner

Thanks @ElderOrb. #2402 changed the same workflow block tonight, so I rebased the branch onto main (cbd76e3e43). The test-pull-security step sits where you put it, beside main's new cache-save condition. The byte-order-mark and trailing-blank-line fixes are now already true on main.

I also set your two commits' author email to your GitHub noreply address. Our push checks refuse any other address, and GitHub still credits the commits to your account. The code is unchanged, and the landing gate's host checks pass. It's in our landing queue.

@ashhart
ashhart force-pushed the cli-pull-tests-portable branch from cbd76e3 to 35f233a Compare November 7, 2026 05:10
@ashhart
ashhart merged commit 1b58f2e into ashhart:main Nov 7, 2026
@ashhart

ashhart commented Nov 7, 2026

Copy link
Copy Markdown
Owner

Thank you @ElderOrb. This is merged into main with your commits under your own name, so you're listed as a TensorFold contributor. It ships in the next release, credited to you in the changelog and the release notes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants