Repository navigation
docs(models): long outputs need stream=true (120s non-stream limit) - #192
Merged
Merged
Conversation
Non-streaming requests return nothing until the whole answer is ready and must be ready within 120 seconds; agents asking for 32k-131k output tokens without streaming hit a 524. Document the limit and that a stream has no total time limit while it keeps producing. EN and ES. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tu register The "no total time limit" line now points at the reasoning-only stream limit right below it, the 10-15k token estimate is stated for deepseek-v4-flash speed, streaming is claimed for chat models only, and the Spanish copy uses tu like the rest of the page (two older voseo lines too). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
New section in the models page (EN + ES), right before "Reasoning-only stream limit":
stream: truereturns nothing until the answer is complete and has to be complete within 120 seconds, else a524(or an error a few seconds earlier);max_tokens, agents writing whole files) usestream: true: a stream has no total time limit while it keeps producing (linked to the reasoning-only stream limit right below, the one exception), on both endpoints, for every chat model.Review round: ES copy aligned to tu like the rest of the page (incl. two older voseo lines), anchors checked in the build (
#reasoning-only-stream-limit,#límite-de-stream-solo-razonamiento).Why
The 2026-10-15 Novita timeout investigation found ~17% of the timeout-driven fallbacks were non-stream requests asking for 32k-131k tokens: they cannot finish in 120s on any provider, and the member got a 524. The infra side (split
stream_timeout/timeouton the Novita rows, and no paid fallback for a non-stream request with no time left before the wall) is in helmcode/nan-devops (companion PR). The "(or an error a few seconds earlier)" covers that refusal, which returns the primary's own timeout at ~115s.No provider names, no internal details.
Tests
npm run build: OKnpx vitest run: 86 files, 2490 tests passed (re-run on 8424403)Deploy
Docs only, no VERSION file in this repo.
🤖 Generated with Claude Code