Repository navigation
cluster: reasoning turns end in finish_reason=length with empty content and no usage block (~60k chars ceiling on deepseek-v4-flash, glm5.3-flash, qwen3.8-flash) #90
Description
Summary
On reasoning-heavy requests, the cluster's OpenAI-compatible endpoint returns a turn that ends with finish_reason: "length", empty content, and no usage object, even when max_tokens: 384000 is requested and the reasoning trace stops at ~59,994 characters — well below the requested budget. The cut happens at roughly the same point on every attempt, so the caller receives a truncated turn with no answer and no accounting.
Reported by a member; every figure below is measured from the client side against https://api.nan.builders/v1, and is reproducible with no client in the middle.
Impact
- A truncated turn is indistinguishable from a completed one for anything that reads
finish_reasonwithout ausageblock: agents treat a dead end as an answer. - Token accounting breaks: with
stream_options.include_usageset and no usage chunk emitted, the caller cannot price or cap the attempt. - Each reproduction costs a long reasoning generation and returns nothing, so this is expensive to investigate from the member side — it consumed a full morning of bisecting and a large amount of budget.
Reproduction (direct, no client)
POST https://api.nan.builders/v1/chat/completions
Authorization: Bearer ***
Content-Type: application/json
User-Agent: opencode/1.18.31
Body:
{
"model": "deepseek-v4-flash",
"max_tokens": 384000,
"stream": true,
"stream_options": { "include_usage": true },
"messages": [
{
"role": "user",
"content": "OUTPUT BUDGET CALIBRATION RUN. Produce at least 500 distinct numbered findings about robustness, error handling, and edge cases in a Binance websocket/REST connector, each finding at least 100 words. Do NOT summarize or shorten. Do NOT stop early. Completeness is the only success criterion. End with a JSON line: {\"calibration\":true,\"findings_count\":<n>}"
}
]
}Note on the proxy: requests without a User-Agent are rejected with 403 "error code: 1010", so the reproduction above carries one.
Observed
finish_reason: "length"
reasoning_content: ~59,994 chars
content: (empty)
usage: (absent — no usage chunk even with include_usage: true)
The ceiling reproduces across three models from OpenCode sessions, landing in the same ~60k-character window every time:
| model | reasoning_content chars |
|---|---|
deepseek-v4-flash |
59,991 – 59,999 |
qwen3.8-flash |
59,996 – 59,997 |
glm5.3-flash |
59,994 |
Controls that complete normally
Same endpoint, same max_tokens: 384000:
| prompt | completion tokens | finish_reason |
|---|---|---|
| integer-list prompt | 93,429 | stop |
| review-shaped prompt with tools | 38,222 | stop |
So this is not a blanket output cap: a run of 93,429 completion tokens finishes normally. The failure looks prompt-shaped — it triggers on certain long reasoning generations rather than at a fixed token count — and when it triggers, the turn is cut without a usage block.
What this rules out
- Not a client bug. It reproduces with
curldirectly, no client involved. - Not a context-window limit. The prompt is short and the requested budget is far above where generation stops.
- Not a global output cap. The integer-list control returns 93,429 completion tokens.
- Not one model. Three different models show the same ~60k-character ceiling.
Related
- cloud-api: modelRateLimits publishes a 1M window for qwen3.8-flash while the proxy enforces 262144 #53 — published vs enforced limits (
modelRateLimitsvs the proxy) onqwen3.8-flash: the same class of disagreement between what the platform publishes and what it enforces. - Error-body SHAPE partitions the catalogue by upstream, and our vLLM 400s are not JSON #85 — error-body / vLLM behaviour around
max_tokens too large. This is the inverse case: the request is accepted, generation starts, and the turn dies mid-flight.
Questions
- Is there an undeclared reasoning/output budget (~32k reasoning tokens) on
deepseek-v4-flash,glm5.3-flashandqwen3.8-flash? - Why is
usageomitted when a turn is cut withfinish_reason=length? Clients such as OpenCode and the gentle-ai review agents need it to tell "done" apart from "truncated" and to account spend. - Can the ceiling be raised or removed, or is it a model limit members should design around — in which case, should it be published alongside the per-model limits already documented?
Environment
- Endpoint:
https://api.nan.builders/v1/chat/completions - Models:
deepseek-v4-flash,glm5.3-flash,qwen3.8-flash - Clients that show it: OpenCode 1.18.31, gentle-ai review agents
- Reproduced: 2026-09-17, from the EU, member API key
I can provide the full raw SSE stream of a failing run, or run any specific test you need — the reproduction above is self-contained.
Activity
dakotad74 commentedon Sep 21, 2026
Resolution
Helmcode/NaN confirmed the report and shipped a fix. Their reply (email, 2026-09-21) answers all three questions, and every measurement in the original report was confirmed correct — the attribution was the only thing off.
1. The ceiling is a guard in front of the endpoint, not a model limit
A protection closes any streaming turn that has produced only reasoning — no
content, no tool calls — at 60,000 reasoning characters (~15k tokens) or 420 seconds, whichever comes first.It counts characters, not tokens, which is exactly why the cut lands in the same ~60k-character window across
deepseek-v4-flash,qwen3.8-flashandglm5.3-flashdespite their different tokenizers.It was added after observing thinking models plan for 10–25 minutes without converging and burn their entire output budget with nothing visible to show for it. The limit disarms on the first visible token, which is why the long healthy completions in the controls section finish normally.
2. The missing
usageblock was a defect on their side, and it is fixedBecause the closure aborts the request mid-flight, the turn never completed and no usage chunk was emitted, even with
stream_options.include_usage: true.Deployed 2026-09-21. Closes now carry an estimated usage chunk:
usage.completion_tokens/usage.prompt_tokenswith"estimated": true"billed": false— turns closed by this limit are not charged against the monthly quotacompletion_tokens_details.reasoning_tokensbreakdownnan_truncation.reasonmarker, so a client can tell this closure apart from a real context-length cutThe estimates are approximations, since the request is aborted mid-generation.
3. The ceiling stays
It is tunable on their side but they are keeping it: it converts a dead 20-minute stream burning ~131k tokens into a bounded, signalled, un-billed close.
Practical notes for reasoning-heavy prompts
glm5.3-flashhonoursreasoning_effort: "low" | "medium" | "high" | "max"end to end, which shortens the planning phase considerably.deepseek-v4-flashis adaptive — there is no lever there.Documentation
Their reply points at the model docs (
nan.builders/docs/models, "Reasoning-only stream limit"). As of 2026-09-21 that section is not visible at that URL, so it is worth confirming before relying on it.Closing
Credit where it is due: they answered the specifics, called the
usageomission a defect rather than a quirk, and had it deployed within a day of the follow-up.Closing as resolved. The accounting gap is fixed, and the ceiling is declared and signalled instead of silent.
dakotad74 commentedon Sep 21, 2026
Follow-up: one piece still outstanding
Recording this for anyone who finds the issue later — the resolution above is real, but it is not yet complete on their side.
The reply states the guard is now documented in the model docs (
nan.builders/docs/models, "Reasoning-only stream limit"). As of 2026-09-21 that section is not published at that URL: going through the page text (model table, "Controlling reasoning", rate limits, quotas) it is not there.Status: the behaviour is understood, the
usagefix is deployed, and this issue stays closed. The only outstanding item is the documentation, and the guard cannot be treated as fully declared until it is published.We are waiting for that update. No action requested.
sre-helmcode commentedon Sep 22, 2026
Root cause found — it was platform-side, not model-side. Fix deployed 2026-09-21.
The ~60k ceiling is not a model budget. It's our reasoning-only stream guard in front of the community endpoint: any streaming turn that emits only reasoning (no
content, no tool calls) for 60,000 characters — or 420 seconds — is closed with a syntheticfinish_reason: "length". It's character-based, not token-based, which is why the cut lands at the same ~60k chars on all three models despite different tokenizers (59,99x < 60,000 because the crossing chunk is swallowed). The reason the integer-list control completed 93,429 tokens: the guard disarms on the first visible output, so turns that write as they go are never cut.The missing
usagewas a genuine defect in that guard (reported accurately here): closing the stream abandoned the upstream request, so the provider never emitted usage and the synthetic close carried none. Fixed and deployed 2026-09-21 — guarded streams now end with:Estimates are approximate (the upstream request was aborted mid-generation) and
"billed": falseis literal: turns closed by the limit do not consume monthly quota. Verified live onglm5.3-flashwithreasoning_effort: "max"(tripped at 164s / 60,009 chars) after adeepseek-v4-flashrun with the same calibration prompt converged normally (31k tokens,finish_reason: stop) — early content disarms the guard.Answers to the three questions: (1) no model-side reasoning budget exists — the ceiling is ours, character-based; (2) usage was missing because the abort happened before the provider could report it — now replaced by the estimated, not-billed chunk above; (3) the ceiling is tunable platform-side and stays, but it is no longer undeclared: documented in the model docs ("Reasoning-only stream limit"), with mitigation levers (
reasoning_effortonglm5.3-flash, early-content prompt design).Members affected by dead-stream stalls were being protected by the same guard since 2026-09-14; this issue made it visible and made truncated turns accountable. Thanks @dakotad74 for the measurements that pinned it down.
docs(models): document the reasoning-only stream limit (#94)
sre-helmcode commentedon Sep 22, 2026
Follow-up closed: the section is now published.
It documents the 60,000-character / 420-second tripwires, the disarm-on-first-output behavior, the estimated / not-billed usage chunk, and the mitigation levers. Deployed 2026-09-22; the same guard now also protects the enterprise endpoint. Nothing else outstanding on our side — thanks again for the clean report.