[devopsfortunato]

What this changes

• Issue: Refs One reason code for every wait or refusal, and an audit of queue limits #2135 (the reason-code half; the prompt/decode family matrix and the queue-limit audit in that issue stay open)
• Stacked on: main (c588759e7)
• atc overlap --json, after the latest rebase: 4 entries carry conflict: "conflicts" — tokenizer: read a Qwen2 slow-tokenizer export when tokenizer.json is absent #2225, qwen CUDA: a faster opt-in paged path (TF_QWEN_KV_PAGES=256), --parallel a cap with pages set #2011 and Metal: the 27B verifies 32 rows a pass on pre-M5 Macs #1841 are CHANGELOG.md alone (the shared top-bullet slot), and kimi: experimental K3.1 index operators on Metal, native serving stays capped #2228 also reports zig/src/server/chat.zig. kimi: experimental K3.1 index operators on Metal, native serving stays capped #2228 is based on a main older than cached_tokens and tool_parse, so its whole chat.zig diverges from current main; my part of that file is one import line plus the three capacity arm lines, so once kimi: experimental K3.1 index operators on Metal, native serving stays capped #2228 rebases onto current main the overlap disappears (it does not touch those arms). No rebase of this branch is needed for any of them; it is on current main (c588759e7), rebased twice while the PR was open.

A parked engine's reload wait and the retryable capacity refusals now name their reason in one fixed code from a bounded table (zig/src/server/wait_reason.zig): server_restarting (the drain, unchanged), engine_loading and engine_reload_failed (a parked model still unloading, or whose reload failed — both used to answer with the restart body, "code": "server_restarting" and "server restarting: retry shortly", so a client could not tell a shutdown from a model load), and engine_busy / engine_unavailable / memory_pressure on the capacity refusals that carried no code at all.

Status codes, the drain body's bytes, Retry-After values and every 400 refusal are unchanged; the code is additive. A capacity refusal keeps the frozen Python engine's shape (type invalid_request_error, its message, and no Retry-After). The Messages routes keep their Anthropic envelope, which has no code field, and the Sliding Weights learn/delete and imprint routes keep their own 503 bodies; docs/api.md says so.

Review fix on this head (maintainer's two points)

• embeddings.zig answered a busy engine with engine_unavailable, so a client could wait for a restart that was not coming. Busy now answers engine_busy and says "the embedding engine is busy; retry shortly", matching chat and speech; only Closed/Stale stay engine_unavailable. A new host test in embedding_service_test.zig pins both codes and both messages through the same envelope the route sends.
• Rebased onto current main (c588759e7); the only conflict each time was CHANGELOG.md's top bullet, resolved by keeping both, and chat.zig's import merge was clean.

Checks on this head

• Head commit: 881dadb32 (two commits, rebased onto c588759e7)
• Lean checker, python3 tools/zig/lean_check.py, on this tree: 425 problems, 0 new (the file count moved to 5458 with the rebased tree; the problem count is the same as main's).
• Tests run, pass/fail/skip:
  • zig build test-server-cpu --summary all — 1046 passed, 0 failed, 1 skipped of 1047 (the skip is the pre-existing libtfgrammar.so grammar-backend test). Includes the five wait_reason_test cases (bounded table, frozen drain body, reload mapping, capacity code with a control, and the two end-to-end route tests) and the new embeddings busy/closed test.
  • zig build test-rerank-host --summary all — 37 passed, 1 skipped (run on the previous head; the review fix does not touch rerank).
  • zig build test-golden --summary all — 0 unexpected differences (322/335 and 55/55 equal).
  • zig build test-server-security --summary all — 24 passed, 0 failed; 20/20 host tests.
  • zig fmt --check build.zig zig and git diff --check — clean.
• CI on this head (881dadb32): all 7 checks pass — zig, zig-windows, zig-macos-dev-tools, fmt, release-paths (ubuntu-24.04), benchmark-clients (ubuntu-latest) and benchmark-clients (windows-latest). Both earlier heads were 7/7 green too.
• Not run, and why: no GPU, checkpoint, model or kernel path is touched, so no device or checkpoint gate applies; the Windows and macOS jobs are CI's.

Receipt

n/a: this change cannot reach inference. It only widens the JSON error object of retryable 503s and gives the idle-reload wait its own reason; no kernel, weight, sampling, scheduling or arithmetic path changes, so exactness, decode/prompt speed and kernel-contract receipts do not apply. What does change on the wire is pinned by the tests: the drain body is byte-frozen, the capacity envelope keeps message and type and adds code, and the route tests assert the full status line, Retry-After: 30 and the code.

Checklist

• The title and description match the head commit.
• New tests: the two route tests assert the new codes and the absence of server_restarting, which main's draining() answer necessarily contains; the embeddings test pins engine_busy for a busy engine and engine_unavailable for a closed one. They skip cleanly without a GPU or grammar library.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths in the diff or its commits. No AI attribution lines.

[ashhart]

Thanks @devopsfortunato, splitting the reload wait from the restart drain is a good fix: a client could not tell a model load from a shutdown before. Two small things. First, embeddings.zig answers a busy engine with engine_unavailable, while chat and speech say engine_busy; could embeddings use engine_busy for Busy too, so a client doesn't wait for a restart that isn't coming? Second, main moved under chat.zig's imports (a cached_tokens line), so it needs a small rebase.

On the names: #2196 counts the same refusals under its own, and clients will key on one set. We'll take your engine_* codes for anything a client sees, and #2196's counters will reuse them.

[devopsfortunato]

Both points are done, head is now bbbd150b3.

• Embeddings busy code: embeddings.zig now answers engine_busy + "the embedding engine is busy; retry shortly" for Busy, and keeps engine_unavailable only for Closed/Stale — the same split chat and speech use, so a client no longer waits for a restart that is not coming. A new host test in embedding_service_test.zig drives embeddings.reply with a fake engine returning Busy and Closed and pins both codes and both messages through the route's own envelope.
• Rebase: onto current main (359bc9b09). The only conflict was CHANGELOG.md's top bullet (resolved by keeping both); chat.zig's import merge was clean.

Checks on this head: test-server-cpu 1046 passed / 0 failed / 1 skipped (pre-existing grammar skip), test-golden 0 unexpected differences, test-server-security 24/0, zig fmt and git diff --check clean, lean 425 problems / 0 new. CI is running on the new head; the previous head was 7/7 green.

Glad to hear #2196's counters will reuse the engine_* codes — the six names are in zig/src/server/wait_reason.zig and the table is documented in docs/api.md under ### Waits.

[ashhart]

Thanks @devopsfortunato. The embeddings split now matches chat and speech, and the test pins it through the route's own envelope. Ready to land, and it's in the queue. #2196's counters will follow your names.

[ashhart]

Thank you @devopsfortunato. This is merged into main with your commits under your own name, so you're listed as a TensorFold contributor. It ships in the next release, credited to you in the changelog and the release notes.

[ashhart]

Thanks @devopsfortunato! This is in TensorFold 1.2.3, out now: https://github.com/ashhart/TensorFold/releases/tag/v1.2.3