[devopsfortunato]

What this changes

With parallel_tool_calls: false, a streamed reply now sends the same content, arguments and finish reason as the whole reply in every case #1396 lists. The whole reply's behavior is unchanged.

I rechecked all of #1396's open items on current main first, as asked in the issue. Item 3 was fixed in 1.1.7. Of what remained:

• Item 1 still reproduced (namespaced and mixed-case envelopes): the whole reply kept Before.<foo:tool_call>{"name":"other",…}</foo:tool_call>Tail. as content, the stream sent Before.Tail. — also for <TOOL_CALL> and <Tool_Call>; <tool_call>{invalid}</tool_call> already agreed. Cause: in one-call mode the streamed prose goes through Policy.text → TextFilter, which hides every callTag envelope (any case, any NAME: prefix) and keeps emitting the text after it, while the whole reply keeps a rejected whole envelope in its content (Policy.parsedContent), and Policy.kept at the finish can only append a suffix, so an interior envelope could never reach the stream. Fix: once the filter has hidden a closed envelope it holds all later prose (deferred), and the finish sends the reply's final content past what streamed.
• Item 2 still reproduced: the stream's end-of-reply prose took the cut-call branch (cutProse, which rstrips) in one-call mode, while the whole reply turns that branch off in one-call mode (openai.zig !policy.single). A rejected envelope before a cut call (<tool_call>{"name":"other",…}</tool_call>Between. then a cut GLM call) and the cut prose's trailing newline were lost. Fix: endProse takes the single flag and uses finishedProse in one-call mode.
• Item 4 already agreed on main; it is kept as a regression case. The trailing-newline variant of it did differ in one-call mode (whole content pyStripped by the parse, stream keeping the newline); finishedProse now always rstrips and Policy.kept delivers any non-empty rest, so the finish sends exactly the final content's tail. Default-mode wire bytes are unchanged.

Behavior change worth naming: in one-call mode, prose written after a hidden call envelope now arrives at the finish, as one content delta from the reply's final content, instead of streaming past a dropped envelope. Default mode is byte-identical to main: both changed predicates (endProse's rstrip and Policy.kept's whitespace-only rest) are gated on the one-call flag, finishedProse is untouched, and the unrelated shared reformat in chat.zig from the first revision was reverted for a smaller #1665 rebase. A new test pins the default-mode content chunk sequence for a tool-offering, call-free reply that ends with text plus a newline, in both modes and both engine batching shapes.

Tokens, the engine and requests without tools are untouched.

• Issue: Fixes Streamed chat: four more cases where the stream and the whole reply disagree (single-call envelopes, max_tokens cuts, DSML whitespace) #1396
• Stacked on: main a6c5b97eb0b352eccdfa1703958bc068df0eeee8 (one commit, 9467a1db8). Independent of Streamed chat: GLM and DSML tool calls arrive in one piece, only XML calls stream their arguments #1975's GLM streaming work; the two only share the test-import line in zig/src/server/root.zig, a trivial rebase either way.
• atc overlap --json: 16 open pull requests share files; 10 merge clean, 6 report unknown (pullFilesUnavailable). Of those, dsv41: serve structured output with the lane core's grammar masks #1665 (dsv41 structured output) is a real textual conflict in zig/src/server/chat.zig — git merge-tree confirms it. Both changes edit the same Generation lines in a file at the 600-line limit, so this needs a small rebase against whichever lands first. The other five (2009, 1980, 1871, 1295, 1336) touch docs/api.md or unrelated regions and merge clean by inspection.

Checks on this head

• Head commit: 7484be91b (base a6c5b97eb0b)
• Lean checker, python3 tools/zig/lean_check.py, on the base and on this head: base 424 problems, head 424 problems, lean_new.py 0 new.
• Tests run, with the command and pass, fail and skip counts (DGX Spark, GB10, Linux aarch64, Zig 0.17.0, empty local .zig-cache per run; these are the no-GPU CI steps from .github/workflows/no-gpu.yml):
  • zig build test --test-timeout 10m → 494/494 steps; 6567 passed, 48 skipped, 0 failed
  • zig build test-kernel-contract → 22/22 steps; 177/177
  • zig build test-golden → 5/5 steps, no golden file changed
  • zig build test-server-security test-pull-security → 14/14 steps; 28/28
  • zig build fuzz-cortheon → 3/3 steps; 19/19
  • zig build test-cuda-host → 293/293 steps; 2652/2655 (3 skipped)
  • zig build (plain) → 31/31 steps
  • zig build test-server-cpu → 5/5 steps; 955/955
  • python3 -m unittest discover -s tools/tests -p "test_bench*.py" → 74 OK; tools/zig flashnext_batch_serve → 8 OK; build_options → 5 OK (2 skipped)
  • zig fmt --check build.zig zig → clean; git diff --check → clean
• CI: will run on this pull request (no-gpu.yml covers every step above); nothing here should move it.
• Not run, and why: the Windows and macOS CI legs need those runners; the NFKD-table check needs Python 3.14's unicodedata and this change doesn't touch the tables; GPU and model checks don't apply to a server-only parsing change.

New tests: zig/src/server/tool_single_stream_test.zig drives every case whole vs streamed through the real chat path (openai.run with a byte-token engine, batched and one character per round): items 1a–1d, 2a–2c, 4a/4b with and without a trailing newline in both modes, an accepted call whose prose streams before it, and the default-mode chunk-sequence pin (a tool-offering, call-free reply ending abc\n: per-token a,b,c,\n and batched abc,\n, identical in default and one-call mode, joined content equal to the whole reply's). 1a–1c, 2a–2c and the single-mode trailing-newline case were observed failing before this change; 1d and the no-newline 4a/4b are regression assertions only.

Review: two rounds by GLM-5.3 on the first revision (one medium finding on the GLM PR, none here), plus the maintainer's review that led to this revision; a final GLM-5.3 round on this revision approved with no correctness findings.

Receipt

n/a — the change is in the HTTP server's one-call streaming filter and prose handling (tool_stream.zig, chat.zig, reply_text.zig); it never reaches inference, so no exactness or speed receipt applies. Drafted output, resumed prompts and concurrency are unaffected for the same reason.

Checklist

• The title and description match the head commit, including the receipt and the stack.
• New tests fail before the change, pass after it, and skip cleanly without their dependency (host-only; no dependency).
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths in the diff or its commits. No AI attribution lines.
• Inference unaffected: no token path, kernel, precision or scheduling change.

[devopsfortunato]

The cancelled zig-macos-dev-tools job never started: no steps ran and no runner was assigned (the job's step list is empty), so it looks like a macos-26 runner was lost rather than a build failure. The same job passed on the sibling PR #2050, and the zig job with the full no-GPU suite passed here. I can't re-run it without rights on this repository; gh run rerun --failed from a maintainer, or the next push, should clear it.

[ashhart]

Thanks @devopsfortunato. Rechecking #1396 on current main first made this easy to review. The one-call fix reads right. Holding prose after a hidden envelope until the finish is a clean way to keep the stream equal to the whole reply. Requests without tools are untouched.

One thing before it lands. finishedProse now always strips trailing whitespace, and kept now sends a whitespace-only rest, in both modes. So a default-mode stream that offers tools and ends a call-free reply with text plus a newline sends abc and \n as two chunks where main sends one. The joined content is the same, but the wire bytes aren't. Could you limit both changes to one-call mode, and add a test pinning the default-mode chunk sequence? Dropping the unrelated shared reformat in chat.zig would also keep the #1665 rebase smaller.

[devopsfortunato]

Done in 7484be91b; thank you for the careful read.

• Default mode is byte-identical to main again. finishedProse is restored exactly (zero diff on reply_text.zig), endProse's default branch and Policy.kept's default rule are main's statements untouched, and both previous changes now sit behind the one-call flag: endProse rstrips only in one-call mode, and kept allows a whitespace-only rest only there. TextFilter.deferred is only reachable through Policy.text/content/flush, all gated on policy.single.
• Default-mode chunk sequence pinned. New test in tool_single_stream_test.zig: a tool-offering, call-free reply ending in text plus a newline, recording every content delta — per-token a","b","c","\n and batched abc","\n, asserted for default and one-call mode, joined content equal to the whole reply's. To make the pin live I forced finishedProse back to unconditional rstrip in a throwaway tree: the test fails (expected 4, found 3).
• shared reformat reverted to main's 3-line form (chat.zig is now 596 lines).
• Re-ran everything on the final head (DGX Spark, Zig 0.17.0, empty local cache): test 6567/6615 passed, 48 skipped, 0 failed; kernel contract, goldens (no file changed), security, fuzz and the plain build all green; test-server-cpu 955/955; zig fmt clean; lean checker 424 = baseline, 0 new. CI is running on this head now.

One honest note, in case it changes what you'd like pinned: I measured the chunk sequence on upstream main before changing anything (temporary worktree on the same machine), and main also sends abc then \n for that shape — and the first revision of this PR measured byte-identical to main there, because in default mode the old finishedProse rstrip and kept's whitespace rule cancel out (the tail is withheld during the reply and re-sent by kept at the same position). So that legacy pair was wire-neutral in default mode; the revision still does exactly what you asked (default mode now shares no code path with the one-call change), and the new pin guards the framing from here on. If you'd like the one-call cut cases' chunk order pinned too (in a leading-whitespace cut reply the trailing prose now arrives via kept after the cut-argument deltas), say so and I'll add it.

[ashhart]

Thanks @devopsfortunato. Both asks are covered:

• the strip and the whitespace-only rest now sit behind the one-call flag, and default mode's code is untouched;
• the new test pins the default-mode chunks, per token and batched.

Thanks too for measuring main's own sequence. You're right that main already sends abc and then \n there, so our earlier note overstated the difference. Keeping default mode off the new path is still the cleaner form. Approved. Whichever of this and #2050 lands second will need its root.zig test import rebased.

[ashhart]

Thank you @devopsfortunato. This is merged into main with your commits under your own name, so you're listed as a TensorFold contributor. It ships in the next release, credited to you in the changelog and the release notes.