Skip to content

docs(models): update reasoning-only stream limit to input-aware bounds #143

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sre-helmcode merged 1 commit into from
Oct 2, 2026
Merged

docs(models): update reasoning-only stream limit to input-aware bounds #143

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 13 additions & 8 deletions src/content/docs-es/models.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -365,11 +365,15 @@ curl https://api.nan.builders/v1/chat/completions \

Los prompts que exigen mucho razonamiento pueden dejar al modelo pensando
mucho tiempo antes de escribir nada. Cuando un turno en streaming produjo
solo razonamiento — sin `content` ni tool calls — durante **60.000 caracteres
de razonamiento** (unos 15.000 tokens) o **420 segundos**, la plataforma
cierra el turno en lugar de dejarlo correr hacia un callejón sin salida:
recibís `finish_reason: "length"` sobre un delta vacío, seguido de un chunk
de usage.
solo razonamiento — sin `content` ni tool calls — durante **420 segundos**,
la plataforma cierra el turno en lugar de dejarlo correr hacia un callejón
sin salida. Un turno que demuestra que está pensando — ya produjo más de
45.000 caracteres de razonamiento — tiene hasta **600 segundos**. El
presupuesto de caracteres de razonamiento también escala con el tamaño de
tu prompt: unos 150.000 caracteres de margen más la mitad del tamaño del
prompt en caracteres, con un tope de 300.000. Cuando la plataforma cierra
el turno recibís `finish_reason: "length"` sobre un delta vacío, seguido de
un chunk de usage.

Ese chunk de usage es una estimación de lo que consumió el intento hasta el
corte (tokens de prompt y de razonamiento), marcada con
Expand All @@ -389,8 +393,9 @@ Dos cosas que conviene saber:

El límite existe para acotar un modo de fallo conocido de los modelos con
thinking: a veces planifican sin converger durante 10–25 minutos y queman
todo su presupuesto de output sin output visible. Está por encima del
razonamiento legítimo más grande que medimos (~44.000 caracteres) y por
debajo de toda corrida estancada.
todo su presupuesto de output sin output visible. Los topes están por
encima de las trazas de razonamiento de los turnos sanos que convergen en
nuestro tráfico de producción, y solo se endurecen cuando un stream está
atascado.

<RateLimits />
13 changes: 2 additions & 11 deletions src/content/docs/models.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -361,12 +361,7 @@ curl https://api.nan.builders/v1/chat/completions \

## Reasoning-only stream limit.

Reasoning-heavy prompts can keep a model thinking for a long time before it
writes anything. When a streaming turn has produced only reasoning — no
`content`, no tool calls — for **60,000 reasoning characters** (roughly
15,000 tokens) or **420 seconds**, the platform closes the turn instead of
letting it run into a dead end: you receive `finish_reason: "length"` on an
empty delta, followed by a usage chunk.
Reasoning-heavy prompts can keep a model thinking for a long time before it writes anything. When a streaming turn has produced only reasoning — no `content`, no tool calls — for **420 seconds**, the platform closes the turn instead of letting it run into a dead end. A turn that is demonstrably thinking — it has already produced more than 45,000 reasoning characters — gets until **600 seconds**. The reasoning-character budget also scales with the size of your prompt: roughly 150,000 characters of headroom plus half the prompt's size in characters, capped at 300,000. When the platform closes the turn you receive `finish_reason: "length"` on an empty delta, followed by a usage chunk.

That usage chunk is an estimate of what the attempt consumed up to the cut
(prompt and reasoning tokens), marked `"estimated": true, "billed": false` —
Expand All @@ -383,10 +378,6 @@ Two things worth knowing:
ceiling. Lower `reasoning_effort` where the model supports it (table
above), or restructure the prompt so the model starts writing early.

The limit exists to bound a known failure mode of thinking models: they
occasionally plan without converging for 10–25 minutes and burn their whole
output budget with zero visible output. It sits above the largest legitimate
reasoning trace we have measured (~44,000 characters) and below every
stalled run.
The limit exists to bound a known failure mode of thinking models: they occasionally plan without converging for 10–25 minutes and burn their whole output budget with zero visible output. The bounds sit above the reasoning traces of healthy converging turns in our production traffic, and only tighten when a stream is stuck.

<RateLimits />
Loading