[jayleaton]

What this changes

With thinking on, the same greedy request gave a reasoning_content that differed by one trailing newline depending on stream. The whole reply pyStrips the reasoning, but the stream sent it as decoded. The joined deltas therefore kept the \n before </think>, and any leading whitespace, which the whole reply drops. Content, tool calls, finish reason and usage were already equal.

The stream now sends the same stripped reasoning (chat.zig, the streamed think split):

• Leading whitespace is never sent.
• Trailing whitespace is held until more reasoning follows it.
• It is never sent when the block closes or the reply ends.

Only whitespace is ever held back. Every thinking split goes through this code: Qwen's </think> (close token or written), a reply-written <think>, MiniMax <mm:think>, Gemma's thought channel and the WordPiece close. /v1/responses and Anthropic streams build on these deltas. Harmony with thinking off already matched.

Found by LocalRouter's chat qualification, which compares stream and whole replies byte for byte. It has carried a named exception for this since v1.1.1 and still needed it in v1.1.4. No issue is open upstream.

Overlap: the one line this changes in chat.zig is also edited by #1336 (recipient split). It merges by keeping #1336's split and wrapping its split.reasoning in reply_text.pyStrip as here. No other open pull request conflicts with this one.

Receipt

• Environment, with the TensorFold commit, MLX or PyTorch version, machine and GPU, checkpoint and revision:
  • Mac Studio M5 Ultra, 96 GiB, macOS 27.2, Metal. Zig 0.17.0, zig build native -Dcpu=apple_m1 -Dversion=1.0.0.
  • Baseline is main acea3b52; candidate is 4ecefc32.
  • TensorFold/Qwen3.8-27B-MLX-4bit with --drafter z-lab/Qwen3.8-27B-DFlash2, served with --context 32768 --prompt-cache-gib 4 --parallel 1.
• Exactness:

  • Stream against whole on the same server, temperature 0, seed 7, max_tokens 1024:

    Build	Prompt	whole reasoning_content	stream (joined)	equal	content equal	completion tokens
    main	"What is the capital of Australia? Answer in one sentence."	598 B	599 B, ends \n	no	yes	155 / 155
    main	"A train leaves at 09:40 and arrives at 13:15 the same day. How long is the trip in minutes? Give the number only." with chat_template_kwargs.enable_thinking: true	351 B	352 B, ends \n	no	yes	159 / 159
    this PR	the same	598 B	598 B	yes	yes	155 / 155
    this PR	the same	351 B	351 B	yes	yes	159 / 159

  • The candidate's streamed reasoning is byte-identical (cmp) to main's whole reply, so only the stream changed.

  • LocalRouter saw the same 1-byte difference on CUDA (GB10, v1.1.1 and v1.1.4 release binaries).

• Decode speed before and after, from tools/bench_openai.py: n/a. The change holds only whitespace bytes of reasoning deltas and does nothing on the decode path.
• Prompt speed before and after: n/a. The change is in the server's stream formatting, after prompt processing.
• Tests run, with pass, fail and skip counts:
  • A new case set in think_split_test.zig covers leading, interior and trailing whitespace, a reply-written opener, a reply cut at max_tokens, and whitespace-only reasoning (no field, no deltas). It runs at strides 1, 2, 3, half and 256, and fails on main (main streams "\n Plan\n\nstep two \n" where the whole reply has "Plan\n\nstep two").
  • The WordPiece stream check and the loop-stop test trimmed the stream's trailing whitespace before comparing; they now compare exactly.
  • Nine frozen stream goldens recorded the old whitespace in reasoning_content deltas. They now match their non-streamed counterparts (stops/*, tools/bad_tool*, tools/xml_tool*, responses/tools-stream); only reasoning deltas changed.
  • zig fmt --check on the touched files passes.
  • python3 tools/zig/lean_check.py reports 424 on main and on this branch, so it adds nothing.
• Not run, and why:
  • zig build test / test-golden, which this pull request's CI runs: I don't run the zig suites on the machines I use.
  • The server tests were compiled (not run) on Linux, and the native build ran on macOS.
  • The new behaviour was not run on CUDA: the change is in the backend-neutral server.

Checklist

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (Unchanged: tokens and content are identical; drafts were on in the receipt.)
• No precision traded for speed. If the bits change, the description says which sums change and why. (No arithmetic change.)
• n/a: no GPU kernel.
• Prompt processing is no slower than the last release. (Not reached.)
• The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not.
• New tests fail before the change, pass after it, and skip cleanly without their dependency.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines.

[ashhart]

Thanks @jayleaton, a clean one. Rebased onto main with a changelog line; test-server-cpu and test-golden pass with your golden updates, and lean is unchanged. It's host-only, so it lands on our gate in the next batch.

[ashhart]

Thank you @jayleaton. This is merged into main with your commits under your own name, so you're listed as a TensorFold contributor. It ships in the next release, credited to you in the changelog and the release notes.

[ashhart]

Thanks @jayleaton! Streamed reasoning that equals the finished reply byte for byte is in 1.2.1: https://github.com/ashhart/TensorFold/releases/tag/v1.2.1