[devopsfortunato]

What this changes

GLM tool calls stream their arguments while the model writes them, as Qwen-style XML calls already do. Today a GLM call (<tool_call>NAME<arg_key>K</arg_key><arg_value>V</arg_value>…</tool_call>) aborts the streamer (.call sees text that does not start with <function=), so the whole call goes out in one piece at the finish: one delta with the header, one with all the arguments.

Now, when no DSML opener has been seen:

• the header goes out once the name is whole (the first <arg_key> after <tool_call>) and names an offered tool;
• string values stream as JSON-escaped pieces while they decode, holding back only a tail that could start </arg_value>;
• typed values go out as compact JSON at their closer;
• the joined pieces equal the whole reply's arguments byte for byte.

tool_glm_stream.zig mirrors tool_parse.glmCall's reading incrementally — valueKey's key pairing (a key runs to the first </arg_key> an <arg_value> follows) and repairGlmKeys' lone-closer repair (a repaired key streams only when the counts seen so far say repairGlmKeys would repair it) — over resumed indexes, so a long call costs linear time. DSML calls are unchanged: see the open question below.

On the issue's probe (a 2,000-character Write content fed 3 bytes at a time): main gives headers 0, argument deltas 0, state abort; this head gives headers 1, argument deltas 677, the first argument bytes arrive while the value is still being written, and the joined arguments equal the whole reply's.

Two stream/whole differences found on the way, same bar (#1396's), fixed here:

• Calls lost after a call had streamed. Any envelope the streamer could not stream — an unoffered <function=other>, a GLM or JSON payload, <|tool_call>, <NAME:tool_call> — set it to .abort, so every later call was missing from the stream while the whole reply still returned it (openai.zig does not replay calls when tool_calls_streamed is set). Example: XML lookup(a), unoffered other(b), XML lookup(c) — whole 2 calls, stream 1. Now such an envelope is parsed at its closer and its calls sent; the streamer aborts only while nothing has streamed, which keeps today's "finish sends every call" path.

• Typed values were not compact. XML typed values streamed as ["a", "b"] while the whole reply's arguments are ["a","b"] (json.stringify without .compact).

• Issue: Refs Streamed chat: GLM and DSML tool calls arrive in one piece, only XML calls stream their arguments #1975 (the GLM half; the DSML half is an open question below)

• Stacked on: main a6c5b97eb0b352eccdfa1703958bc068df0eeee8 (one commit, f13d002d0). Independent of the Streamed chat: four more cases where the stream and the whole reply disagree (single-call envelopes, max_tokens cuts, DSML whitespace) #1396 one-call fix; the two only share the test-import line in zig/src/server/root.zig, a trivial rebase either way.

• atc overlap --json: 16 open pull requests share files; 10 merge clean, 6 report unknown (pullFilesUnavailable). By git merge-tree, dsv41: serve structured output with the lane core's grammar masks #1665 merges clean against this branch; the other five (2009, 1980, 1871, 1295, 1336) touch unrelated regions or docs.

Open question — DSML. The issue also asks for DSML calls to stream their arguments. I left that out deliberately: DSML blocks parse all or nothing (tool_dsml.calls rejects a whole block on any malformed or unoffered invoke, and envelopes masks what is inside), and the existing containment contract in stream_dsml_test.zig ("unfinished and malformed DSML never emit their complete nested call") requires that an unclosed block stream no call at all. Streaming a DSML invoke before its block closes would break that contract and could send a call a later malformed invoke then rejects, which cannot be taken back. That trade-off needs a maintainer's decision, so this PR keeps DSML exactly as it is. zig/src/families/deepseek_v41/serve/tool_stream.zig is a related DSML streamer on the served-engine side.

Checks on this head

• Head commit: fb8d9fcff (base a6c5b97eb0b)
• Lean checker, python3 tools/zig/lean_check.py, on the base and on this head: base 424 problems, head 424 problems, lean_new.py 0 new.
• Tests run, with the command and pass, fail and skip counts (DGX Spark, GB10, Linux aarch64, Zig 0.17.0, empty local .zig-cache per run; these are the no-GPU CI steps from .github/workflows/no-gpu.yml):
  • zig build test --test-timeout 10m → 494/494 steps; 6572 passed, 48 skipped, 0 failed
  • zig build test-kernel-contract → 22/22 steps; 177/177
  • zig build test-golden → 5/5 steps, no golden file changed
  • zig build test-server-security test-pull-security → 14/14 steps; 28/28
  • zig build fuzz-cortheon → 3/3 steps; 19/19
  • zig build test-cuda-host → 293/293 steps; 2967/2975 (8 skipped)
  • zig build (plain) → 31/31 steps
  • zig build test-server-cpu → 5/5 steps; 960/960
  • python3 -m unittest discover -s tools/tests -p "test_bench*.py" → 74 OK; tools/zig flashnext_batch_serve → 8 OK; build_options → 5 OK (2 skipped)
  • zig fmt --check build.zig zig → clean; git diff --check → clean
• CI: will run on this pull request (no-gpu.yml covers every step above); nothing here should move it.
• Not run, and why: the Windows and macOS CI legs need those runners; the NFKD-table check needs Python 3.14's unicodedata and this change doesn't touch the tables; GPU and model checks don't apply to a server-only parsing change. No /v1/messages end-to-end test was added: Anthropic's input_json_delta is built from the same argument deltas (anthropic.zig drain), so GLM calls now produce many of them instead of one.

New tests: zig/src/server/tool_glm_stream_test.zig (8 tests) — the 2,000-byte value streaming before it closes and joining to the whole reply's arguments; every feed boundary for three replies (typed values, non-ASCII, a repaired key, and the lone-closer case the first review round caught); ten GLM shapes; unstreamable envelopes sending nothing; a GLM call after a DSML block staying buffered as today; the end-token closeCall path; cut replay equal to tool_glm_cut; and a whole-vs-streamed chat table through the real chat path, batched and per character (prose plus GLM, two GLM calls, the unoffered-middle case, XML+GLM+XML, typed values, a max_tokens cut inside a value, zero-argument, key without a value). Every test that names a bug was red before the change; the unstreamable and DSML-first cases are regression assertions.

A follow-up adds the two checks the maintainer asked for: a test pinning the repeated-key gap in both GLM and XML spellings, and a /v1/messages streaming test that drives the real HTTP route with a scripted engine and asserts the Anthropic input_json_delta partial_json pieces (19 of them) join byte-for-byte to the whole reply's tool_use input; with GLM streaming disabled in a throwaway copy that test fails with one piece, so it is live. Production code was not touched by the follow-up.

Reviewed over two rounds on the production change; round 1 found one real bug (the lone-</arg_key> repair was optimistic where repairGlmKeys is all-or-nothing, so the stream could invent an argument — counter-example now a test), round 2 approved with no correctness findings.

Receipt

n/a — the change is in the HTTP server's tool-call streaming (tool_stream.zig, new tool_glm_stream.zig); it never reaches inference, so no exactness or speed receipt applies. Drafted output, resumed prompts and concurrency are unaffected for the same reason.

Known limitations, named so they are decisions rather than surprises: a </tool_call> written inside an <arg_value> is not special-cased (XML behaves the same way); an end token inside a GLM key leaves the stream's call open where the whole reply returns none (XML has the same case for <parameter=); a call with two key malformations, where later text changes repairGlmKeys' whole-payload decision, can still differ (single-malformation cases are guarded and tested; noted in the module header); a repeated key streams both pairs while the whole reply's argument object keeps the last one — XML has the same gap today, and both formats are pinned by a test so it stays visible ({"q":"a","q":"b"} streamed, {"q":"b"} whole).

Checklist

• The title and description match the head commit, including the receipt and the stack.
• New tests fail before the change, pass after it, and skip cleanly without their dependency (host-only; no dependency).
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths in the diff or its commits. No AI attribution lines.
• Inference unaffected: no token path, kernel, precision or scheduling change.

[ashhart]

Thanks @devopsfortunato, this is careful work. Mirroring glmCall's key pairing and repairGlmKeys' all-or-nothing repair over resumed indexes is the right shape. The two stream/whole fixes you found on the way (calls lost after a streamed call, and non-compact typed values) are welcome too.

Two small things before we land it:

1. A repeated key streams both pairs, while the whole reply keeps one. XML has the same gap today, so please list it under the known limitations or pin it with a test.
2. Please add one /v1/messages streaming test showing a GLM call's input_json_delta pieces join to the whole reply's input.

Leaving DSML out is the right call for now. Its containment test is a contract we want to keep, so that one will need a maintainer decision. #2049 only touches the same root.zig line, so whichever lands second needs a one-line rebase.

[devopsfortunato]

Done in fb8d9fcff; production code untouched by this revision (tests only).

1. Repeated key pinned. New test in tool_glm_stream_test.zig runs the GLM spelling and the XML spelling of the same key twice and asserts both sides exactly: one header, streamed joined arguments {"q":"a","q":"b"}, whole reply {"q":"b"}. GLM now visibly behaves like XML here, and the gap fails the test if either side changes. It is also listed under the PR's known limitations.
2. /v1/messages streaming test added. tool_glm_messages_test.zig starts the real server (serve.run on loopback, scripted engine emitting a GLM call one byte a token), POSTs the Anthropic body whole and streamed, and joins the input_json_delta partial_json pieces from the SSE stream: 19 pieces that concatenate byte-for-byte to the whole reply's tool_use input, {"path":"notes.txt","content":"hello"}, which the test also pins. It fails with a single piece when GLM streaming is disabled in a throwaway copy (1 input_json_delta pieces at the len > 1 assert), so it is not vacuous. drain_test.zig is untouched; the test carries its own byte tokenizer.

Full no-GPU set re-run on this head (DGX Spark, Zig 0.17.0, empty local cache): test 6572/6620 passed, 48 skipped, 0 failed (+2 new tests); kernel contract, goldens (unchanged), security, fuzz, CUDA host subset and the plain build all green; test-server-cpu 960/960; zig fmt clean; lean checker 424 = baseline, 0 new. CI is running on this head now.

On the landing order: agreed — #2049 and #2050 share only the root.zig test-import line, so whichever lands second needs that one-line rebase, and DSML stays out pending your decision.

[ashhart]

Thanks @devopsfortunato. Both points are covered: the repeated-key test pins the stream and the whole reply for both the GLM and XML spellings, and the /v1/messages test joins the input_json_delta pieces back to the whole reply's input byte for byte, through the real server. The production code is unchanged from the version we reviewed. This is good to land; whichever of this and #2049 goes second just needs the one-line root.zig rebase.

[ashhart]

Thank you @devopsfortunato. This is merged into main with your commits under your own name, so you're listed as a TensorFold contributor. It ships in the next release, credited to you in the changelog and the release notes.