Skip to content

docs: what a no-thinking GLM-5.3 prompt is on each engine (#1911) - #2297

Merged
ashhart merged 1 commit into
ashhart:mainfrom
devopsfortunato:docs/glm-no-thinking-prompt
Nov 6, 2026
Merged

ashhart merged 1 commit into
ashhart:mainfrom
devopsfortunato:docs/glm-no-thinking-prompt

Conversation

@devopsfortunato

@devopsfortunato devopsfortunato commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

What this changes

docs/api.md's Reasoning section said a no-thinking GLM-5.3 prompt has "no reasoning-effort line". Native renders the checkpoint's template unchanged and appends </think> once to close its dangling think block (zig/src/server/prompt.zig), so the template's Reasoning Effort head stays and the prompt is the render plus one token. The released Python 0.6 engine instead rewrites GLM's render (thinking_off strips the head and closes the block inline), which is what its smaller usage.prompt_tokens reports. The section now states both engines' behavior and what follows: a prompt count, a cache key or a kept state from one engine does not match the other's for enable_thinking: false. The CHANGELOG carries one bullet.

Checks on this head

  • Head commit: 3648d035b (rebased on 8993a3cca; the approved head was dd28d3360 on 2e81d5df1). The rebase changed no content: docs/api.md is byte-identical to that approved head, and the only conflict was the CHANGELOG.md top bullet, resolved by putting this change's bullet at the top of ## Unreleased.
  • Lean checker, python3 tools/zig/lean_check.py: base 8993a3cca 425 problems, this head 425 problems; lean_new.py reports 425 problems, 425 on the base, 0 new. The nonzero count is inherited from main.
  • Tests run, with the command and pass, fail and skip counts: no code changed, so no test counts apply. On this head zig fmt --check build.zig zig exit 0 and git diff --check exit 0.
  • CI: all seven checks pass on this head — zig 15m03s, zig-windows 22m52s, zig-macos-dev-tools 9m19s, release-paths 1m17s, fmt 13s, benchmark-clients 13s and 22s.
  • Not run, and why: zig build test and the other suites locally — the diff is two Markdown files (docs/api.md, CHANGELOG.md); the build graph is unchanged, and CI builds and runs the unchanged tree on this head.

Receipt

n/a — documentation only. The change cannot reach inference: no source, build-graph or configuration file is touched, so prompts, tokens and caches are byte for byte as main's.

Checklist

  • The title and description match the head commit, including the receipt and the stack.
  • New tests fail before the change, pass after it, and skip cleanly without their dependency. — n/a, no new tests; no code changed.
  • Comments and docstrings are one line. No measurements or history in the source.
  • No personal data, machine names, internal hosts or local paths in the diff or its commits. No AI attribution lines.

@ashhart

ashhart commented Nov 6, 2026

Copy link
Copy Markdown
Owner

Thanks @devopsfortunato. This matches what we measured on #1911, and what zig/src/server/prompt.zig does: native renders the template unchanged and appends </think> once. It also replaces the old sentence, which was wrong for native. It's queued to land, and #1911 closes with it.

@devopsfortunato
devopsfortunato force-pushed the docs/glm-no-thinking-prompt branch 4 times, most recently from dd28d33 to 3648d03 Compare November 6, 2026 06:49
@ashhart

ashhart commented Nov 6, 2026

Copy link
Copy Markdown
Owner

Thanks @devopsfortunato. The push still says what we measured on #1911 and what prompt.zig does: the render is unchanged, </think> is appended once when the render ends in <think>, and the template's own head stays. It lands from the queue. At landing we'll make the CHANGELOG line cite "(#2297, #1911)" with your thanks. The body still names ccb635cfd, so please point it at the current head if you touch it again.

The quality question you left open on #1911 (whether the kept head changes greedy no-thinking output) stays there: we'll keep #1911 open for it and note that the docs half is done.

@devopsfortunato

Copy link
Copy Markdown
Contributor Author

Done — the body now names head 3648d035b (rebased on 8993a3cca), not ccb635cfd: head commit, base, the lean checker on both (425 and 425, lean_new.py 0 new), the post-rebase atc overlap --json and the CI run are all recorded under "Checks on this head". CI is 7/7 green on that head — zig 15m03s, zig-windows 22m52s, zig-macos-dev-tools 9m19s, release-paths 1m17s, fmt 13s, benchmark-clients 13s and 22s. Content is unchanged from dd28d3360; the only resolution was moving this bullet to the top of ## Unreleased. Main has moved again since, so the changelog line reads CONFLICTING again — leaving that to your landing flow as you described, and no further pushes unless you want one. Thanks for the landing note and the "(#2297, #1911)" citation.

The Reasoning section said a no-thinking GLM-5.3 prompt has no
reasoning-effort line. Native renders the checkpoint template unchanged
and appends </think> once to close its dangling think block, so the
template's Reasoning Effort head stays and the prompt is the render plus
one token. The released 0.6 Python engine's thinking_off instead strips
that head and closes the block inline, which is why its no-thinking
prompts report 6-7 fewer prompt_tokens than native's on the same
request. State both engines' behavior and what follows for prompt
counts, cache keys and kept states.
@ashhart
ashhart force-pushed the docs/glm-no-thinking-prompt branch from 3648d03 to e0a230b Compare November 6, 2026 09:39
@ashhart
ashhart merged commit ebb67e6 into ashhart:main Nov 6, 2026
7 checks passed
@ashhart

ashhart commented Nov 6, 2026

Copy link
Copy Markdown
Owner

Thank you @devopsfortunato. This is merged into main with your commits under your own name, so you're listed as a TensorFold contributor. It ships in the next release, credited to you in the changelog and the release notes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants