From 6497b3c019aa9597582a38d6356f6fadcfe640ef Mon Sep 17 00:00:00 2001 From: barckcode Date: Mon, 19 Oct 2026 12:06:33 +0200 Subject: [PATCH 1/6] docs(models): output budget is passed through as sent, no reasoning floor The platform no longer raises a deepseek-v4-flash output budget below 16384 while reasoning is on (owner decision 2026-10-19). max_tokens, max_completion_tokens and max_output_tokens are now used exactly as sent on every model and route. - deepseek-v4-flash card (en + es): drop the floor and the "prompt within 16384 tokens of the window is rejected" sentence; say reasoning counts inside the budget, how a reasoning-exhausted answer looks (empty content, finish_reason "length", reasoning_tokens == completion_tokens) and the remedies (reasoning off, lower effort, generous budget plus streaming). - Controlling reasoning (en + es): same guidance, linking the existing long-outputs streaming section instead of duplicating it. - examples (en + es): replace the "max_tokens >= 300" advice. - OpenAPI max_tokens description: pass-through contract. - Tests pin the new contract and assert the floor wording is gone. Co-Authored-By: Claude Opus 5.5 --- src/content/docs-es/examples.md | 2 +- src/content/docs-es/models.mdx | 14 +++++- src/content/docs/examples.md | 2 +- src/content/docs/models.mdx | 13 +++++- src/data/openapi.json | 2 +- src/lib/openapiSpec.test.ts | 29 ++++++------ src/tests/lib/docsReasoning.test.ts | 72 +++++++++++++++++++++++------ 7 files changed, 101 insertions(+), 33 deletions(-) diff --git a/src/content/docs-es/examples.md b/src/content/docs-es/examples.md index 9c8ae05..754ab30 100644 --- a/src/content/docs-es/examples.md +++ b/src/content/docs-es/examples.md @@ -434,7 +434,7 @@ curl https://api.nan.builders/v1/chat/completions \ }' ``` -Con el razonamiento activado se recomienda `max_tokens ≥ 300`, para dejarle sitio. +Los tokens de razonamiento cuentan dentro de `max_tokens`, que se usa tal cual lo envías. Con el razonamiento activado, un presupuesto pequeño puede gastarse entero en razonar y llegar con `content` vacío y `finish_reason: "length"`. Si solo necesitas la respuesta, pon `"reasoning_effort": "none"`; si no, da un `max_tokens` generoso (los prompts densos en `deepseek-v4-flash` pueden razonar más de 16k tokens) y usa `stream: true`. Mira [Controlar el razonamiento](/es/docs/models#controlar-el-razonamiento). ### visión (curl) diff --git a/src/content/docs-es/models.mdx b/src/content/docs-es/models.mdx index 27e615f..3f3d046 100644 --- a/src/content/docs-es/models.mdx +++ b/src/content/docs-es/models.mdx @@ -50,7 +50,7 @@ OpenAI y la misma `base URL`. tag="305B MoE" leftLabel="generación de texto, chat y visión" rightLabel="capacidades" - description={'Modelo MoE de 305B parámetros, servido en su variante Vision-Exp: acepta imágenes de entrada. Contexto de 1M tokens. Tool calling y razonamiento. Cuota de 3B tokens al mes por miembro. Salida estructurada: response_format json_object es compatible (en api.nan.builders el prompt debe contener la palabra JSON; si no, se rechaza con un 400). json_schema se acepta en peticiones a /chat/completions sin streaming y sin tools: la respuesta se comprueba contra tu esquema y, si no lo cumple, normalmente la vuelve a servir un proveedor que lo impone; una respuesta que no se pudo comprobar, o un fallo que no se pudo reemplazar a tiempo, se devuelve tal cual (valídala si tu código depende de ella). En api.nan.builders el resto de peticiones json_schema se rechazan con un 400: con streaming, con tools, a /responses, con n mayor que 1, con un esquema de más de 16 KB, que no sea un JSON Schema válido o que el gateway no valida (un $ref fuera del esquema, $dynamicRef o $anchor, un $id anidado o demasiado grande al resolver sus $ref); en api.helmcode.com se aceptan. json_object solo garantiza JSON sintácticamente válido, no su forma: describe el esquema en el prompt. Para salida restringida a un esquema en esas otras peticiones, usa qwen3.6 o gemma4, donde json_schema con strict restringe campos, tipos y claves obligatorias; o, en este modelo, fuerza una única herramienta de tipo función cuyo parameters sea tu esquema (ver Ejemplos). Con el razonamiento activado, un presupuesto de salida por debajo de 16384 se sube a 16384 para que quepa el razonamiento (max_tokens y max_completion_tokens en /chat/completions, max_output_tokens en /responses, igual en las dos rutas), así que un prompt a menos de 16384 tokens de la ventana de 1.048.576 se rechaza con un 400 aunque pidas un presupuesto pequeño (normalmente Context length exceeded, a veces un Invalid request genérico). Con el razonamiento apagado (reasoning_effort "none" o cualquiera de sus alias) tu presupuesto se usa tal cual. Un presupuesto por encima de 16384 nunca se baja.'} + description={'Modelo MoE de 305B parámetros, servido en su variante Vision-Exp: acepta imágenes de entrada. Contexto de 1M tokens. Tool calling y razonamiento. Cuota de 3B tokens al mes por miembro. Salida estructurada: response_format json_object es compatible (en api.nan.builders el prompt debe contener la palabra JSON; si no, se rechaza con un 400). json_schema se acepta en peticiones a /chat/completions sin streaming y sin tools: la respuesta se comprueba contra tu esquema y, si no lo cumple, normalmente la vuelve a servir un proveedor que lo impone; una respuesta que no se pudo comprobar, o un fallo que no se pudo reemplazar a tiempo, se devuelve tal cual (valídala si tu código depende de ella). En api.nan.builders el resto de peticiones json_schema se rechazan con un 400: con streaming, con tools, a /responses, con n mayor que 1, con un esquema de más de 16 KB, que no sea un JSON Schema válido o que el gateway no valida (un $ref fuera del esquema, $dynamicRef o $anchor, un $id anidado o demasiado grande al resolver sus $ref); en api.helmcode.com se aceptan. json_object solo garantiza JSON sintácticamente válido, no su forma: describe el esquema en el prompt. Para salida restringida a un esquema en esas otras peticiones, usa qwen3.6 o gemma4, donde json_schema con strict restringe campos, tipos y claves obligatorias; o, en este modelo, fuerza una única herramienta de tipo función cuyo parameters sea tu esquema (ver Ejemplos). Tu presupuesto de salida se usa tal cual lo envías, con el razonamiento activado o apagado (max_tokens y max_completion_tokens en /chat/completions, max_output_tokens en /responses). Los tokens de razonamiento cuentan dentro de ese presupuesto: con el razonamiento activado, un presupuesto pequeño puede gastarse entero en razonar, y la respuesta llega con content vacío y finish_reason "length" (el razonamiento está en reasoning_content, y usage.completion_tokens_details.reasoning_tokens es igual a completion_tokens). Los prompts densos pueden razonar más de 16k tokens: apaga el razonamiento (reasoning_effort "none"), baja el esfuerzo, o da un presupuesto generoso y usa streaming.'} specs={[ { label: 'Tipo', value: 'MoE (305B total)' }, { label: 'Cuantización', value: 'FP8' }, @@ -369,6 +369,18 @@ cuentan dentro de `max_tokens`. Usa solo los valores de esta página. Cualquier otro valor se envía tal cual y queda fuera de este contrato: según el modelo, se rechaza con un `400` o se ignora. +Tu presupuesto de salida (`max_tokens` o `max_completion_tokens` en Chat +Completions, `max_output_tokens` en Responses) se usa tal cual lo envías, en +todos los modelos. El razonamiento sale de ese mismo presupuesto, así que con +el razonamiento activado un presupuesto pequeño puede agotarse antes de que +empiece la respuesta: recibes un `content` vacío con `finish_reason: "length"`, +el razonamiento en `message.reasoning_content` y +`usage.completion_tokens_details.reasoning_tokens` igual a `completion_tokens`. +En `deepseek-v4-flash` un prompt denso puede razonar más de 16k tokens. Si solo +necesitas la respuesta, apaga el razonamiento (`reasoning_effort: "none"`) o +baja el esfuerzo; si quieres el razonamiento, da un presupuesto generoso y +[usa streaming](#respuestas-largas-usa-streaming). + ```bash curl https://api.nan.builders/v1/chat/completions \ -H "Authorization: Bearer $NAN_API_KEY" \ diff --git a/src/content/docs/examples.md b/src/content/docs/examples.md index 13795c4..c43a433 100644 --- a/src/content/docs/examples.md +++ b/src/content/docs/examples.md @@ -434,7 +434,7 @@ curl https://api.nan.builders/v1/chat/completions \ }' ``` -With reasoning enabled, `max_tokens ≥ 300` is recommended to leave room for reasoning. +Reasoning tokens count inside `max_tokens`, which is used exactly as sent. With reasoning on, a small budget can be spent entirely on reasoning and come back with an empty `content` and `finish_reason: "length"`. If you only need the answer, set `"reasoning_effort": "none"`; otherwise give a generous `max_tokens` (dense prompts on `deepseek-v4-flash` can reason past 16k tokens) and use `stream: true`. See [Controlling reasoning](/docs/models#controlling-reasoning). ### vision (curl) diff --git a/src/content/docs/models.mdx b/src/content/docs/models.mdx index eebc628..650a3a1 100644 --- a/src/content/docs/models.mdx +++ b/src/content/docs/models.mdx @@ -50,7 +50,7 @@ with the same `base URL`. tag="305B MoE" leftLabel="text generation, chat & vision" rightLabel="capabilities" - description={'305B parameter MoE model, served as the Vision-Exp variant: it takes images as input. 1M token context. Tool calling and reasoning. 3B token monthly quota per member. Structured output: response_format json_object is supported (on api.nan.builders the prompt must contain the word JSON, otherwise it is rejected with a 400). json_schema is accepted on non-streamed /chat/completions requests without tools: the answer is checked against your schema and, when it does not match, usually re-served by a provider that enforces it; an answer that could not be checked, or a miss the re-serve could not replace in time, is returned as is (validate it if your code depends on it). On api.nan.builders other json_schema requests are rejected with a 400: streamed, with tools, on /responses, with n above 1, with a schema over 16 KB, not a valid JSON Schema, or one the gateway will not validate (a $ref outside the schema, $dynamicRef or $anchor, a nested $id, or too large once its $refs are inlined); on api.helmcode.com they are accepted. json_object guarantees syntactically valid JSON only, not its shape: describe the schema in the prompt. For schema-constrained output in those other requests, use qwen3.6 or gemma4, where json_schema with strict constrains fields, types and required keys; or, on this model, force one function tool whose parameters is your schema (see Examples). While reasoning is on, an output budget below 16384 is raised to 16384 so the reasoning fits (max_tokens and max_completion_tokens on /chat/completions, max_output_tokens on /responses, the same on both routes), so a prompt within 16384 tokens of the 1,048,576-token window is rejected with a 400 even with a small budget (usually Context length exceeded, sometimes a generic Invalid request). With reasoning off (reasoning_effort "none" or any of its aliases) your budget is used as sent. A budget above 16384 is never lowered.'} + description={'305B parameter MoE model, served as the Vision-Exp variant: it takes images as input. 1M token context. Tool calling and reasoning. 3B token monthly quota per member. Structured output: response_format json_object is supported (on api.nan.builders the prompt must contain the word JSON, otherwise it is rejected with a 400). json_schema is accepted on non-streamed /chat/completions requests without tools: the answer is checked against your schema and, when it does not match, usually re-served by a provider that enforces it; an answer that could not be checked, or a miss the re-serve could not replace in time, is returned as is (validate it if your code depends on it). On api.nan.builders other json_schema requests are rejected with a 400: streamed, with tools, on /responses, with n above 1, with a schema over 16 KB, not a valid JSON Schema, or one the gateway will not validate (a $ref outside the schema, $dynamicRef or $anchor, a nested $id, or too large once its $refs are inlined); on api.helmcode.com they are accepted. json_object guarantees syntactically valid JSON only, not its shape: describe the schema in the prompt. For schema-constrained output in those other requests, use qwen3.6 or gemma4, where json_schema with strict constrains fields, types and required keys; or, on this model, force one function tool whose parameters is your schema (see Examples). Your output budget is used exactly as sent, with reasoning on or off (max_tokens and max_completion_tokens on /chat/completions, max_output_tokens on /responses). Reasoning tokens count inside that budget: with reasoning on, a small budget can be spent entirely on reasoning, and the answer comes back with an empty content and finish_reason "length" (the reasoning is in reasoning_content, and usage.completion_tokens_details.reasoning_tokens equals completion_tokens). Dense prompts can reason past 16k tokens: turn reasoning off (reasoning_effort "none"), lower the effort, or give a generous budget and stream the answer.'} specs={[ { label: 'Type', value: 'MoE (305B total)' }, { label: 'Quantization', value: 'FP8' }, @@ -366,6 +366,17 @@ With no parameter every model reasons (`qwen3.6` and `gemma4` with a Any other value is passed through unchanged and is not part of this contract: depending on the model it is rejected with a `400` or ignored. +Your output budget (`max_tokens` or `max_completion_tokens` on Chat Completions, +`max_output_tokens` on Responses) is used exactly as you send it, on every +model. Reasoning is spent from that same budget, so with reasoning on a small +budget can run out before the answer starts: you get an empty `content` with +`finish_reason: "length"`, the reasoning in `message.reasoning_content`, and +`usage.completion_tokens_details.reasoning_tokens` equal to `completion_tokens`. +On `deepseek-v4-flash` a dense prompt can reason past 16k tokens. If you only +need the answer, turn reasoning off (`reasoning_effort: "none"`) or lower the +effort; if you want the reasoning, give a generous budget and +[stream the answer](#long-outputs-stream-them). + ```bash curl https://api.nan.builders/v1/chat/completions \ -H "Authorization: Bearer $NAN_API_KEY" \ diff --git a/src/data/openapi.json b/src/data/openapi.json index cd890eb..748cb12 100644 --- a/src/data/openapi.json +++ b/src/data/openapi.json @@ -200,7 +200,7 @@ "max_tokens": { "type": "integer", "minimum": 1, - "description": "Maximum number of tokens to generate. A value above what the model can generate is rejected with a generic `400`. A request that does not fit the model's context window is rejected with a `400`: \"Context length exceeded for model '...'\" on `qwen3.6` and `gemma4`, usually the same text (sometimes the generic \"Invalid request\") on the other models. On `deepseek-v4-flash`, while reasoning is on, a smaller value is raised to 16384 so the reasoning fits, so a prompt within 16384 tokens of its 1,048,576-token window is rejected with a `400` even with a small `max_tokens`; with reasoning off (`reasoning_effort: \"none\"` or an alias) the value is used as sent.", + "description": "Maximum number of tokens to generate. A value above what the model can generate is rejected with a generic `400`. A request that does not fit the model's context window is rejected with a `400`: \"Context length exceeded for model '...'\" on `qwen3.6` and `gemma4`, usually the same text (sometimes the generic \"Invalid request\") on the other models. The value is used exactly as sent, on every model, with reasoning on or off. Reasoning tokens count inside it: with reasoning on, a small value can be spent entirely on reasoning, and the answer comes back with an empty `content` and `finish_reason: \"length\"` (the reasoning is in `message.reasoning_content`, and `usage.completion_tokens_details.reasoning_tokens` equals `completion_tokens`). Turn reasoning off (`reasoning_effort: \"none\"`), lower the effort, or send a generous value and use `stream: true`.", "example": 512 }, "temperature": { diff --git a/src/lib/openapiSpec.test.ts b/src/lib/openapiSpec.test.ts index 674e8f1..25db678 100644 --- a/src/lib/openapiSpec.test.ts +++ b/src/lib/openapiSpec.test.ts @@ -874,9 +874,10 @@ describe('documented context: qwen3.8-flash is 1M', () => { * automatically. The exact "Context length exceeded for model '...'" text was * measured on the hosted models (gemma4); on deepseek-v4-flash an overflow * measured 2026-09-29 came back as the GENERIC "Invalid request. Check your - * request parameters.", so the docs promise the text only where measured. On - * deepseek-v4-flash a small `max_tokens` is raised to 16384 so the reasoning - * fits, which is why a prompt close to the window overflows with a small one. + * request parameters.", so the docs promise the text only where measured. + * Since 2026-10-19 there is no output-budget floor: `max_tokens` is passed + * through as sent on every model, so a small one no longer overflows a prompt + * close to the window. */ describe('documented 400s: tool parameters and context overflow', () => { const schemas = spec.components.schemas as any; @@ -914,12 +915,12 @@ describe('documented 400s: tool parameters and context overflow', () => { expect(bad).toContain('Invalid request. Check your request parameters.'); }); - it('explains the deepseek-v4-flash 16384 floor on max_tokens, in the spec and on both cards', () => { + it('max_tokens is passed through as sent: no deepseek-v4-flash floor, in the spec and on both cards', () => { const maxTokens = (spec.paths as any)['/chat/completions'].post.requestBody.content['application/json'] .schema.properties.max_tokens.description; - expect(maxTokens).toContain('On `deepseek-v4-flash`, while reasoning is on, a smaller value is raised to 16384'); - expect(maxTokens).toContain('with reasoning off (`reasoning_effort: "none"` or an alias) the value is used as sent'); - expect(maxTokens).toContain('1,048,576-token window'); + expect(maxTokens).toContain('The value is used exactly as sent, on every model, with reasoning on or off.'); + expect(maxTokens).toContain('Reasoning tokens count inside it'); + expect(maxTokens).not.toMatch(/raised to 16384|within 16384 tokens/); // Not promised as the exact text on deepseek-v4-flash: measured generic there. expect(maxTokens).not.toMatch(/deepseek-v4-flash[^.]*Context length exceeded/); expect(maxTokens).toMatch(/above what the model can generate is rejected with a generic `400`/); @@ -932,14 +933,12 @@ describe('documented 400s: tool parameters and context overflow', () => { const start = page.indexOf('id="deepseek-v4-flash"'); return page.slice(start, page.indexOf('/>', start)); }; - expect(card('')).toContain('While reasoning is on, an output budget below 16384 is raised to 16384 so the reasoning fits'); - expect(card('')).toContain('max_output_tokens on /responses, the same on both routes'); - expect(card('')).toContain('rejected with a 400 even with a small budget (usually Context length exceeded, sometimes a generic Invalid request)'); - expect(card('')).toContain('A budget above 16384 is never lowered.'); - expect(card('-es')).toContain('Con el razonamiento activado, un presupuesto de salida por debajo de 16384 se sube a 16384'); - expect(card('-es')).toContain('max_output_tokens en /responses, igual en las dos rutas'); - expect(card('-es')).toContain('se rechaza con un 400 aunque pidas un presupuesto pequeño (normalmente Context length exceeded, a veces un Invalid request genérico)'); - expect(card('-es')).toContain('Un presupuesto por encima de 16384 nunca se baja.'); + expect(card('')).toContain('Your output budget is used exactly as sent'); + expect(card('')).toContain('max_output_tokens on /responses'); + expect(card('')).not.toMatch(/raised to 16384|never lowered|within 16384 tokens/); + expect(card('-es')).toContain('Tu presupuesto de salida se usa tal cual lo envías'); + expect(card('-es')).toContain('max_output_tokens en /responses'); + expect(card('-es')).not.toMatch(/se sube a 16384|nunca se baja|a menos de 16384 tokens/); }); }); diff --git a/src/tests/lib/docsReasoning.test.ts b/src/tests/lib/docsReasoning.test.ts index b53adc2..3e3ff5e 100644 --- a/src/tests/lib/docsReasoning.test.ts +++ b/src/tests/lib/docsReasoning.test.ts @@ -98,10 +98,14 @@ describe('API reference: one documented way to turn reasoning off', () => { } }); - it('max_tokens: the deepseek floor applies only while reasoning is on', () => { - expect(chatBody.max_tokens.description).toContain( - 'On `deepseek-v4-flash`, while reasoning is on, a smaller value is raised to 16384 so the reasoning fits, so a prompt within 16384 tokens of its 1,048,576-token window is rejected with a `400` even with a small `max_tokens`; with reasoning off (`reasoning_effort: "none"` or an alias) the value is used as sent.', - ); + it('max_tokens: passed through as sent, reasoning counts inside it (no deepseek floor)', () => { + const d = chatBody.max_tokens.description; + expect(d).toContain('The value is used exactly as sent, on every model, with reasoning on or off.'); + expect(d).toContain('an empty `content` and `finish_reason: "length"`'); + expect(d).toContain('`usage.completion_tokens_details.reasoning_tokens` equals `completion_tokens`'); + expect(d).toContain('`reasoning_effort: "none"`'); + expect(d).toContain('`stream: true`'); + expect(d).not.toMatch(/16384|raised/); }); it('lists the accepted aliases and says the explicit value wins', () => { @@ -242,23 +246,65 @@ describe('model cards agree with the table (en + es)', () => { } }); -describe('deepseek-v4-flash card: the max_tokens floor only applies while reasoning is on (en + es)', () => { +/** + * NO OUTPUT-BUDGET FLOOR (owner decision 2026-10-19, nan-devops + * docs/community-report-2026-10-19-output-budget.md). Until then a + * deepseek-v4-flash budget below 16384 was silently raised while reasoning was + * on. Now the budget is passed through as sent on every model and route, so + * the docs must say that reasoning counts inside it and how a reasoning- + * exhausted answer looks, and must not describe the floor any more. + */ +describe('output budget: passed through as sent, no floor (en + es)', () => { const card = (locale: '' | '-es') => { const text = page(locale); const start = text.indexOf('name="deepseek-v4-flash"'); return text.slice(start, text.indexOf('/>', start)); }; + const examples = (locale: '' | '-es') => + readFileSync(resolve(here, `../../content/docs${locale}/examples.md`), 'utf-8'); + const FLOOR = /raised to 16384|se sube a 16384|16384 nunca se baja|never lowered|within 16384 tokens|a menos de 16384 tokens|≥ ?300/; + + for (const locale of ['', '-es'] as const) { + it(`${locale || 'en'}: no page describes the floor or the old max_tokens >= 300 advice`, () => { + expect(page(locale)).not.toMatch(FLOOR); + expect(examples(locale)).not.toMatch(FLOOR); + }); + + it(`${locale || 'en'}: the reasoning section and the card name every field and the empty-answer signature`, () => { + for (const text of [section(locale), card(locale)]) { + for (const needle of ['max_tokens', 'max_completion_tokens', 'max_output_tokens', 'finish_reason', '"length"', + 'reasoning_content', 'usage.completion_tokens_details.reasoning_tokens', 'completion_tokens', 'reasoning_effort', '"none"']) { + expect(text, needle).toContain(needle); + } + } + }); + + it(`${locale || 'en'}: examples point at reasoning off or a generous budget plus streaming`, () => { + expect(examples(locale)).toContain('`finish_reason: "length"`'); + expect(examples(locale)).toContain('`"reasoning_effort": "none"`'); + expect(examples(locale)).toContain('`stream: true`'); + }); + } + + it('en wording', () => { + expect(card('')).toContain('Your output budget is used exactly as sent, with reasoning on or off'); + expect(card('')).toContain('with reasoning on, a small budget can be spent entirely on reasoning, and the answer comes back with an empty content and finish_reason "length"'); + expect(card('')).toContain('turn reasoning off (reasoning_effort "none"), lower the effort, or give a generous budget and stream the answer.'); + expect(section('')).toContain('is used exactly as you send it, on every\nmodel.'); + expect(section('')).toContain('[stream the answer](#long-outputs-stream-them)'); + }); - it('en', () => { - expect(card('')).toContain( - 'While reasoning is on, an output budget below 16384 is raised to 16384 so the reasoning fits (max_tokens and max_completion_tokens on /chat/completions, max_output_tokens on /responses, the same on both routes), so a prompt within 16384 tokens of the 1,048,576-token window is rejected with a 400 even with a small budget (usually Context length exceeded, sometimes a generic Invalid request). With reasoning off (reasoning_effort "none" or any of its aliases) your budget is used as sent. A budget above 16384 is never lowered.', - ); + it('es wording', () => { + expect(card('-es')).toContain('Tu presupuesto de salida se usa tal cual lo envías, con el razonamiento activado o apagado'); + expect(card('-es')).toContain('con el razonamiento activado, un presupuesto pequeño puede gastarse entero en razonar, y la respuesta llega con content vacío y finish_reason "length"'); + expect(card('-es')).toContain('apaga el razonamiento (reasoning_effort "none"), baja el esfuerzo, o da un presupuesto generoso y usa streaming.'); + expect(section('-es')).toContain('se usa tal cual lo envías, en\ntodos los modelos.'); + expect(section('-es')).toContain('[usa streaming](#respuestas-largas-usa-streaming)'); }); - it('es', () => { - expect(card('-es')).toContain('Con el razonamiento activado, un presupuesto de salida por debajo de 16384 se sube a 16384'); - expect(card('-es')).toContain('Con el razonamiento apagado (reasoning_effort "none" o cualquiera de sus alias) tu presupuesto se usa tal cual.'); - expect(card('-es')).toContain('Un presupuesto por encima de 16384 nunca se baja.'); + it('both anchors the prose links to exist as headings', () => { + expect(page('')).toContain('\n## Long outputs: stream them.\n'); + expect(page('-es')).toContain('\n## Respuestas largas: usa streaming.\n'); }); }); From 645c9c48690491a478b21a6a5a9fc47ab05f5b44 Mon Sep 17 00:00:00 2001 From: barckcode Date: Mon, 19 Oct 2026 12:26:10 +0200 Subject: [PATCH 2/6] docs(models): name the glm5.3-flash ceiling, drop mimo max_tokens >= 300 QA follow-up on the output-budget pass-through docs: - The budget is used as sent on every model except glm5.3-flash, where a value above 131072 is lowered to 131072 (its upstream limit). Said in the reasoning section (en + es) and the OpenAPI max_tokens description, whose generic-400 rule now names that exception. - mimo-v2.6-flash card (en + es): replace "max_tokens ≥ 300" with "reasoning counts inside max_tokens"; the floor regex now also catches ≥ / &ge;. - Reasoning location scoped per route (reasoning_content on Chat Completions, a reasoning output item on Responses). - examples.md paragraph under the mimo example made model-generic. Co-Authored-By: Claude Opus 5.5 --- src/content/docs-es/examples.md | 2 +- src/content/docs-es/models.mdx | 9 ++++--- src/content/docs/examples.md | 2 +- src/content/docs/models.mdx | 9 ++++--- src/data/openapi.json | 2 +- src/lib/openapiSpec.test.ts | 5 ++-- src/tests/lib/docsReasoning.test.ts | 39 ++++++++++++++++++++++++++--- 7 files changed, 51 insertions(+), 17 deletions(-) diff --git a/src/content/docs-es/examples.md b/src/content/docs-es/examples.md index 754ab30..b9cc40f 100644 --- a/src/content/docs-es/examples.md +++ b/src/content/docs-es/examples.md @@ -434,7 +434,7 @@ curl https://api.nan.builders/v1/chat/completions \ }' ``` -Los tokens de razonamiento cuentan dentro de `max_tokens`, que se usa tal cual lo envías. Con el razonamiento activado, un presupuesto pequeño puede gastarse entero en razonar y llegar con `content` vacío y `finish_reason: "length"`. Si solo necesitas la respuesta, pon `"reasoning_effort": "none"`; si no, da un `max_tokens` generoso (los prompts densos en `deepseek-v4-flash` pueden razonar más de 16k tokens) y usa `stream: true`. Mira [Controlar el razonamiento](/es/docs/models#controlar-el-razonamiento). +Los tokens de razonamiento cuentan dentro de `max_tokens`, que se usa tal cual lo envías. Con el razonamiento activado, un presupuesto pequeño puede gastarse entero en razonar y llegar con `content` vacío y `finish_reason: "length"`. Si solo necesitas la respuesta, pon `"reasoning_effort": "none"`; si no, da un `max_tokens` generoso (los prompts densos pueden razonar miles de tokens) y usa `stream: true`. Mira [Controlar el razonamiento](/es/docs/models#controlar-el-razonamiento). ### visión (curl) diff --git a/src/content/docs-es/models.mdx b/src/content/docs-es/models.mdx index 3f3d046..a441cac 100644 --- a/src/content/docs-es/models.mdx +++ b/src/content/docs-es/models.mdx @@ -134,7 +134,7 @@ OpenAI y la misma `base URL`. ]} items={[ 'Tool calling (function calling)', - 'Control del razonamiento (reasoning_effort: none lo apaga · de low a max; se recomienda max_tokens ≥ 300) · detalles', + 'Control del razonamiento (reasoning_effort: none lo apaga · de low a max; el razonamiento cuenta dentro de max_tokens) · detalles', 'Visión (entrada de imagen)', 'Audio (entrada de audio)', 'Contexto de 1M tokens', @@ -370,11 +370,12 @@ Cualquier otro valor se envía tal cual y queda fuera de este contrato: según e se rechaza con un `400` o se ignora. Tu presupuesto de salida (`max_tokens` o `max_completion_tokens` en Chat -Completions, `max_output_tokens` en Responses) se usa tal cual lo envías, en -todos los modelos. El razonamiento sale de ese mismo presupuesto, así que con +Completions, `max_output_tokens` en Responses) se usa tal cual lo envías en todos los modelos, +salvo en `glm5.3-flash`, donde un valor por encima de 131072 se baja a 131072, lo máximo que acepta su proveedor. +El razonamiento sale de ese mismo presupuesto, así que con el razonamiento activado un presupuesto pequeño puede agotarse antes de que empiece la respuesta: recibes un `content` vacío con `finish_reason: "length"`, -el razonamiento en `message.reasoning_content` y +el razonamiento en `message.reasoning_content` en Chat Completions (un elemento de salida `reasoning` en Responses) y `usage.completion_tokens_details.reasoning_tokens` igual a `completion_tokens`. En `deepseek-v4-flash` un prompt denso puede razonar más de 16k tokens. Si solo necesitas la respuesta, apaga el razonamiento (`reasoning_effort: "none"`) o diff --git a/src/content/docs/examples.md b/src/content/docs/examples.md index c43a433..6c0e930 100644 --- a/src/content/docs/examples.md +++ b/src/content/docs/examples.md @@ -434,7 +434,7 @@ curl https://api.nan.builders/v1/chat/completions \ }' ``` -Reasoning tokens count inside `max_tokens`, which is used exactly as sent. With reasoning on, a small budget can be spent entirely on reasoning and come back with an empty `content` and `finish_reason: "length"`. If you only need the answer, set `"reasoning_effort": "none"`; otherwise give a generous `max_tokens` (dense prompts on `deepseek-v4-flash` can reason past 16k tokens) and use `stream: true`. See [Controlling reasoning](/docs/models#controlling-reasoning). +Reasoning tokens count inside `max_tokens`, which is used exactly as sent. With reasoning on, a small budget can be spent entirely on reasoning and come back with an empty `content` and `finish_reason: "length"`. If you only need the answer, set `"reasoning_effort": "none"`; otherwise give a generous `max_tokens` (dense prompts can reason for thousands of tokens) and use `stream: true`. See [Controlling reasoning](/docs/models#controlling-reasoning). ### vision (curl) diff --git a/src/content/docs/models.mdx b/src/content/docs/models.mdx index 650a3a1..63af172 100644 --- a/src/content/docs/models.mdx +++ b/src/content/docs/models.mdx @@ -134,7 +134,7 @@ with the same `base URL`. ]} items={[ 'Tool calling (function calling)', - 'Reasoning control (reasoning_effort: none turns it off · low to max; recommended max_tokens ≥ 300) · details', + 'Reasoning control (reasoning_effort: none turns it off · low to max; reasoning counts inside max_tokens) · details', 'Vision (image input)', 'Audio (audio input)', '1M token context', @@ -367,10 +367,11 @@ Any other value is passed through unchanged and is not part of this contract: de on the model it is rejected with a `400` or ignored. Your output budget (`max_tokens` or `max_completion_tokens` on Chat Completions, -`max_output_tokens` on Responses) is used exactly as you send it, on every -model. Reasoning is spent from that same budget, so with reasoning on a small +`max_output_tokens` on Responses) is used exactly as you send it on every model, +except on `glm5.3-flash`, where a value above 131072 is lowered to 131072, the most its upstream accepts. +Reasoning is spent from that same budget, so with reasoning on a small budget can run out before the answer starts: you get an empty `content` with -`finish_reason: "length"`, the reasoning in `message.reasoning_content`, and +`finish_reason: "length"`, the reasoning in `message.reasoning_content` on Chat Completions (a `reasoning` output item on Responses), and `usage.completion_tokens_details.reasoning_tokens` equal to `completion_tokens`. On `deepseek-v4-flash` a dense prompt can reason past 16k tokens. If you only need the answer, turn reasoning off (`reasoning_effort: "none"`) or lower the diff --git a/src/data/openapi.json b/src/data/openapi.json index 748cb12..5933347 100644 --- a/src/data/openapi.json +++ b/src/data/openapi.json @@ -200,7 +200,7 @@ "max_tokens": { "type": "integer", "minimum": 1, - "description": "Maximum number of tokens to generate. A value above what the model can generate is rejected with a generic `400`. A request that does not fit the model's context window is rejected with a `400`: \"Context length exceeded for model '...'\" on `qwen3.6` and `gemma4`, usually the same text (sometimes the generic \"Invalid request\") on the other models. The value is used exactly as sent, on every model, with reasoning on or off. Reasoning tokens count inside it: with reasoning on, a small value can be spent entirely on reasoning, and the answer comes back with an empty `content` and `finish_reason: \"length\"` (the reasoning is in `message.reasoning_content`, and `usage.completion_tokens_details.reasoning_tokens` equals `completion_tokens`). Turn reasoning off (`reasoning_effort: \"none\"`), lower the effort, or send a generous value and use `stream: true`.", + "description": "Maximum number of tokens to generate. A value above what the model can generate is rejected with a generic `400`, except on `glm5.3-flash` (see below). A request that does not fit the model's context window is rejected with a `400`: \"Context length exceeded for model '...'\" on `qwen3.6` and `gemma4`, usually the same text (sometimes the generic \"Invalid request\") on the other models. The value is used exactly as sent, with reasoning on or off, on every model except `glm5.3-flash`, where a value above 131072 is lowered to 131072, the most its upstream accepts. Reasoning tokens count inside it: with reasoning on, a small value can be spent entirely on reasoning, and the answer comes back with an empty `content` and `finish_reason: \"length\"` (the reasoning is in `message.reasoning_content`, and `usage.completion_tokens_details.reasoning_tokens` equals `completion_tokens`). Turn reasoning off (`reasoning_effort: \"none\"`), lower the effort, or send a generous value and use `stream: true`.", "example": 512 }, "temperature": { diff --git a/src/lib/openapiSpec.test.ts b/src/lib/openapiSpec.test.ts index 25db678..e13caf7 100644 --- a/src/lib/openapiSpec.test.ts +++ b/src/lib/openapiSpec.test.ts @@ -918,12 +918,13 @@ describe('documented 400s: tool parameters and context overflow', () => { it('max_tokens is passed through as sent: no deepseek-v4-flash floor, in the spec and on both cards', () => { const maxTokens = (spec.paths as any)['/chat/completions'].post.requestBody.content['application/json'] .schema.properties.max_tokens.description; - expect(maxTokens).toContain('The value is used exactly as sent, on every model, with reasoning on or off.'); + expect(maxTokens).toContain('on every model except `glm5.3-flash`, where a value above 131072 is lowered to 131072'); expect(maxTokens).toContain('Reasoning tokens count inside it'); expect(maxTokens).not.toMatch(/raised to 16384|within 16384 tokens/); // Not promised as the exact text on deepseek-v4-flash: measured generic there. expect(maxTokens).not.toMatch(/deepseek-v4-flash[^.]*Context length exceeded/); - expect(maxTokens).toMatch(/above what the model can generate is rejected with a generic `400`/); + // glm5.3-flash lowers instead of rejecting, so the generic-400 rule names its exception. + expect(maxTokens).toContain('A value above what the model can generate is rejected with a generic `400`, except on `glm5.3-flash` (see below).'); const card = (locale: string) => { const page = readFileSync( diff --git a/src/tests/lib/docsReasoning.test.ts b/src/tests/lib/docsReasoning.test.ts index 3e3ff5e..0d95d10 100644 --- a/src/tests/lib/docsReasoning.test.ts +++ b/src/tests/lib/docsReasoning.test.ts @@ -100,7 +100,9 @@ describe('API reference: one documented way to turn reasoning off', () => { it('max_tokens: passed through as sent, reasoning counts inside it (no deepseek floor)', () => { const d = chatBody.max_tokens.description; - expect(d).toContain('The value is used exactly as sent, on every model, with reasoning on or off.'); + expect(d).toContain('The value is used exactly as sent, with reasoning on or off, on every model except `glm5.3-flash`, where a value above 131072 is lowered to 131072, the most its upstream accepts.'); + expect(d).not.toMatch(/above what the model can generate is rejected with a generic `400`\./); + expect(d).toContain('the reasoning is in `message.reasoning_content`'); expect(d).toContain('an empty `content` and `finish_reason: "length"`'); expect(d).toContain('`usage.completion_tokens_details.reasoning_tokens` equals `completion_tokens`'); expect(d).toContain('`reasoning_effort: "none"`'); @@ -262,7 +264,8 @@ describe('output budget: passed through as sent, no floor (en + es)', () => { }; const examples = (locale: '' | '-es') => readFileSync(resolve(here, `../../content/docs${locale}/examples.md`), 'utf-8'); - const FLOOR = /raised to 16384|se sube a 16384|16384 nunca se baja|never lowered|within 16384 tokens|a menos de 16384 tokens|≥ ?300/; + // `≥` / `&ge;` too: card strings are HTML, and the entity slipped past `≥` once. + const FLOOR = /raised to 16384|se sube a 16384|16384 nunca se baja|never lowered|within 16384 tokens|a menos de 16384 tokens|(?:≥|≥|&ge;)\s*(?:<\/?code>)?\s*300|max_tokens\s*(?:≥|≥|&ge;)/; for (const locale of ['', '-es'] as const) { it(`${locale || 'en'}: no page describes the floor or the old max_tokens >= 300 advice`, () => { @@ -286,11 +289,39 @@ describe('output budget: passed through as sent, no floor (en + es)', () => { }); } + it(`the glm5.3-flash ceiling is the one exception, said in both locales`, () => { + expect(section('')).toContain('except on `glm5.3-flash`, where a value above 131072 is lowered to 131072, the most its upstream accepts.'); + expect(section('-es')).toContain('salvo en `glm5.3-flash`, donde un valor por encima de 131072 se baja a 131072, lo máximo que acepta su proveedor.'); + expect(section('')).not.toContain('on every\nmodel.'); + }); + + it('reasoning location is scoped per route (Chat Completions field, Responses output item)', () => { + expect(section('')).toContain('the reasoning in `message.reasoning_content` on Chat Completions (a `reasoning` output item on Responses)'); + expect(section('-es')).toContain('el razonamiento en `message.reasoning_content` en Chat Completions (un elemento de salida `reasoning` en Responses)'); + }); + + it('mimo-v2.6-flash card: no number, points at Controlling reasoning (en + es)', () => { + const mimo = (locale: '' | '-es') => cardReasoning(locale, 'mimo-v2.6-flash').join(' '); + expect(mimo('')).toContain('reasoning counts inside max_tokens'); + expect(mimo('-es')).toContain('el razonamiento cuenta dentro de max_tokens'); + expect(mimo('')).not.toMatch(FLOOR); + expect(mimo('-es')).not.toMatch(FLOOR); + }); + + for (const locale of ['', '-es'] as const) { + it(`${locale || 'en'}: the examples paragraph is model-generic`, () => { + const ex = examples(locale); + const i = ex.indexOf('`finish_reason: "length"`'); + const para = ex.slice(ex.lastIndexOf('\n\n', i), ex.indexOf('\n\n', i)); + expect(para).not.toContain('deepseek-v4-flash'); + }); + } + it('en wording', () => { expect(card('')).toContain('Your output budget is used exactly as sent, with reasoning on or off'); expect(card('')).toContain('with reasoning on, a small budget can be spent entirely on reasoning, and the answer comes back with an empty content and finish_reason "length"'); expect(card('')).toContain('turn reasoning off (reasoning_effort "none"), lower the effort, or give a generous budget and stream the answer.'); - expect(section('')).toContain('is used exactly as you send it, on every\nmodel.'); + expect(section('')).toContain('is used exactly as you send it on every model'); expect(section('')).toContain('[stream the answer](#long-outputs-stream-them)'); }); @@ -298,7 +329,7 @@ describe('output budget: passed through as sent, no floor (en + es)', () => { expect(card('-es')).toContain('Tu presupuesto de salida se usa tal cual lo envías, con el razonamiento activado o apagado'); expect(card('-es')).toContain('con el razonamiento activado, un presupuesto pequeño puede gastarse entero en razonar, y la respuesta llega con content vacío y finish_reason "length"'); expect(card('-es')).toContain('apaga el razonamiento (reasoning_effort "none"), baja el esfuerzo, o da un presupuesto generoso y usa streaming.'); - expect(section('-es')).toContain('se usa tal cual lo envías, en\ntodos los modelos.'); + expect(section('-es')).toContain('se usa tal cual lo envías en todos los modelos'); expect(section('-es')).toContain('[usa streaming](#respuestas-largas-usa-streaming)'); }); From 32a95dc4e4a9be6c6d68690bdcd2fef9118c3b8d Mon Sep 17 00:00:00 2001 From: barckcode Date: Mon, 19 Oct 2026 12:46:27 +0200 Subject: [PATCH 3/6] docs(models): accurate remedies for a reasoning-exhausted budget Measured 2026-10-19 on deepseek-v4-flash (dense prompt, budget 16384): none reasons 0 tokens; minimal 13k and still hits length; low 11k; high spends all 16384 with empty content. So: - reasoning off is the reliable fix, named per route (reasoning_effort on Chat Completions, reasoning.effort on Responses); GLM cannot turn it off (minimal is its shortest). - a lower effort is a hint on deepseek-v4-flash and qwen3.8-flash, not a cap; only qwen3.6/gemma4 cap it, on Chat Completions. - finish_reason "length" and completion_tokens_details are scoped to Chat Completions; on Responses only the reasoning output item is promised. Applied to the reasoning section and deepseek-v4-flash card (en + es) and the OpenAPI max_tokens description, with tests. Co-Authored-By: Claude Opus 5.5 --- src/content/docs-es/models.mdx | 12 +++++---- src/content/docs/models.mdx | 12 +++++---- src/data/openapi.json | 2 +- src/tests/lib/docsReasoning.test.ts | 38 +++++++++++++++++++++++------ 4 files changed, 46 insertions(+), 18 deletions(-) diff --git a/src/content/docs-es/models.mdx b/src/content/docs-es/models.mdx index a441cac..d1acdbe 100644 --- a/src/content/docs-es/models.mdx +++ b/src/content/docs-es/models.mdx @@ -50,7 +50,7 @@ OpenAI y la misma `base URL`. tag="305B MoE" leftLabel="generación de texto, chat y visión" rightLabel="capacidades" - description={'Modelo MoE de 305B parámetros, servido en su variante Vision-Exp: acepta imágenes de entrada. Contexto de 1M tokens. Tool calling y razonamiento. Cuota de 3B tokens al mes por miembro. Salida estructurada: response_format json_object es compatible (en api.nan.builders el prompt debe contener la palabra JSON; si no, se rechaza con un 400). json_schema se acepta en peticiones a /chat/completions sin streaming y sin tools: la respuesta se comprueba contra tu esquema y, si no lo cumple, normalmente la vuelve a servir un proveedor que lo impone; una respuesta que no se pudo comprobar, o un fallo que no se pudo reemplazar a tiempo, se devuelve tal cual (valídala si tu código depende de ella). En api.nan.builders el resto de peticiones json_schema se rechazan con un 400: con streaming, con tools, a /responses, con n mayor que 1, con un esquema de más de 16 KB, que no sea un JSON Schema válido o que el gateway no valida (un $ref fuera del esquema, $dynamicRef o $anchor, un $id anidado o demasiado grande al resolver sus $ref); en api.helmcode.com se aceptan. json_object solo garantiza JSON sintácticamente válido, no su forma: describe el esquema en el prompt. Para salida restringida a un esquema en esas otras peticiones, usa qwen3.6 o gemma4, donde json_schema con strict restringe campos, tipos y claves obligatorias; o, en este modelo, fuerza una única herramienta de tipo función cuyo parameters sea tu esquema (ver Ejemplos). Tu presupuesto de salida se usa tal cual lo envías, con el razonamiento activado o apagado (max_tokens y max_completion_tokens en /chat/completions, max_output_tokens en /responses). Los tokens de razonamiento cuentan dentro de ese presupuesto: con el razonamiento activado, un presupuesto pequeño puede gastarse entero en razonar, y la respuesta llega con content vacío y finish_reason "length" (el razonamiento está en reasoning_content, y usage.completion_tokens_details.reasoning_tokens es igual a completion_tokens). Los prompts densos pueden razonar más de 16k tokens: apaga el razonamiento (reasoning_effort "none"), baja el esfuerzo, o da un presupuesto generoso y usa streaming.'} + description={'Modelo MoE de 305B parámetros, servido en su variante Vision-Exp: acepta imágenes de entrada. Contexto de 1M tokens. Tool calling y razonamiento. Cuota de 3B tokens al mes por miembro. Salida estructurada: response_format json_object es compatible (en api.nan.builders el prompt debe contener la palabra JSON; si no, se rechaza con un 400). json_schema se acepta en peticiones a /chat/completions sin streaming y sin tools: la respuesta se comprueba contra tu esquema y, si no lo cumple, normalmente la vuelve a servir un proveedor que lo impone; una respuesta que no se pudo comprobar, o un fallo que no se pudo reemplazar a tiempo, se devuelve tal cual (valídala si tu código depende de ella). En api.nan.builders el resto de peticiones json_schema se rechazan con un 400: con streaming, con tools, a /responses, con n mayor que 1, con un esquema de más de 16 KB, que no sea un JSON Schema válido o que el gateway no valida (un $ref fuera del esquema, $dynamicRef o $anchor, un $id anidado o demasiado grande al resolver sus $ref); en api.helmcode.com se aceptan. json_object solo garantiza JSON sintácticamente válido, no su forma: describe el esquema en el prompt. Para salida restringida a un esquema en esas otras peticiones, usa qwen3.6 o gemma4, donde json_schema con strict restringe campos, tipos y claves obligatorias; o, en este modelo, fuerza una única herramienta de tipo función cuyo parameters sea tu esquema (ver Ejemplos). Tu presupuesto de salida se usa tal cual lo envías, con el razonamiento activado o apagado (max_tokens y max_completion_tokens en /chat/completions, max_output_tokens en /responses). Los tokens de razonamiento cuentan dentro de ese presupuesto: con el razonamiento activado, un presupuesto pequeño puede gastarse entero en razonar. En /chat/completions la respuesta llega entonces con content vacío y finish_reason "length" (el razonamiento está en reasoning_content, y usage.completion_tokens_details.reasoning_tokens es igual a completion_tokens); en /responses el razonamiento llega como un elemento de salida reasoning sin texto de mensaje. Los prompts densos pueden razonar más de 16k tokens. La solución fiable es apagar el razonamiento (reasoning_effort "none" en /chat/completions, reasoning.effort "none" en /responses); un esfuerzo menor es una pista en este modelo, no un tope. O da un presupuesto generoso y usa streaming.'} specs={[ { label: 'Tipo', value: 'MoE (305B total)' }, { label: 'Cuantización', value: 'FP8' }, @@ -374,12 +374,14 @@ Completions, `max_output_tokens` en Responses) se usa tal cual lo envías en tod salvo en `glm5.3-flash`, donde un valor por encima de 131072 se baja a 131072, lo máximo que acepta su proveedor. El razonamiento sale de ese mismo presupuesto, así que con el razonamiento activado un presupuesto pequeño puede agotarse antes de que -empiece la respuesta: recibes un `content` vacío con `finish_reason: "length"`, -el razonamiento en `message.reasoning_content` en Chat Completions (un elemento de salida `reasoning` en Responses) y +empiece la respuesta. En Chat Completions recibes un `content` vacío con +`finish_reason: "length"`, el razonamiento en `message.reasoning_content` y `usage.completion_tokens_details.reasoning_tokens` igual a `completion_tokens`. +En Responses el razonamiento llega como un elemento de salida `reasoning` y no hay texto de mensaje. En `deepseek-v4-flash` un prompt denso puede razonar más de 16k tokens. Si solo -necesitas la respuesta, apaga el razonamiento (`reasoning_effort: "none"`) o -baja el esfuerzo; si quieres el razonamiento, da un presupuesto generoso y +necesitas la respuesta, la solución fiable es apagar el razonamiento: `reasoning_effort: "none"` en Chat Completions, `reasoning: {"effort": "none"}` en Responses (GLM no puede apagarlo; `minimal` es lo más corto). +Un esfuerzo menor suele razonar menos, pero en `deepseek-v4-flash` y `qwen3.8-flash` es una pista, no un tope; solo `qwen3.6` y `gemma4` lo acotan, en Chat Completions. +Si quieres el razonamiento, da un presupuesto generoso y [usa streaming](#respuestas-largas-usa-streaming). ```bash diff --git a/src/content/docs/models.mdx b/src/content/docs/models.mdx index 63af172..b017edd 100644 --- a/src/content/docs/models.mdx +++ b/src/content/docs/models.mdx @@ -50,7 +50,7 @@ with the same `base URL`. tag="305B MoE" leftLabel="text generation, chat & vision" rightLabel="capabilities" - description={'305B parameter MoE model, served as the Vision-Exp variant: it takes images as input. 1M token context. Tool calling and reasoning. 3B token monthly quota per member. Structured output: response_format json_object is supported (on api.nan.builders the prompt must contain the word JSON, otherwise it is rejected with a 400). json_schema is accepted on non-streamed /chat/completions requests without tools: the answer is checked against your schema and, when it does not match, usually re-served by a provider that enforces it; an answer that could not be checked, or a miss the re-serve could not replace in time, is returned as is (validate it if your code depends on it). On api.nan.builders other json_schema requests are rejected with a 400: streamed, with tools, on /responses, with n above 1, with a schema over 16 KB, not a valid JSON Schema, or one the gateway will not validate (a $ref outside the schema, $dynamicRef or $anchor, a nested $id, or too large once its $refs are inlined); on api.helmcode.com they are accepted. json_object guarantees syntactically valid JSON only, not its shape: describe the schema in the prompt. For schema-constrained output in those other requests, use qwen3.6 or gemma4, where json_schema with strict constrains fields, types and required keys; or, on this model, force one function tool whose parameters is your schema (see Examples). Your output budget is used exactly as sent, with reasoning on or off (max_tokens and max_completion_tokens on /chat/completions, max_output_tokens on /responses). Reasoning tokens count inside that budget: with reasoning on, a small budget can be spent entirely on reasoning, and the answer comes back with an empty content and finish_reason "length" (the reasoning is in reasoning_content, and usage.completion_tokens_details.reasoning_tokens equals completion_tokens). Dense prompts can reason past 16k tokens: turn reasoning off (reasoning_effort "none"), lower the effort, or give a generous budget and stream the answer.'} + description={'305B parameter MoE model, served as the Vision-Exp variant: it takes images as input. 1M token context. Tool calling and reasoning. 3B token monthly quota per member. Structured output: response_format json_object is supported (on api.nan.builders the prompt must contain the word JSON, otherwise it is rejected with a 400). json_schema is accepted on non-streamed /chat/completions requests without tools: the answer is checked against your schema and, when it does not match, usually re-served by a provider that enforces it; an answer that could not be checked, or a miss the re-serve could not replace in time, is returned as is (validate it if your code depends on it). On api.nan.builders other json_schema requests are rejected with a 400: streamed, with tools, on /responses, with n above 1, with a schema over 16 KB, not a valid JSON Schema, or one the gateway will not validate (a $ref outside the schema, $dynamicRef or $anchor, a nested $id, or too large once its $refs are inlined); on api.helmcode.com they are accepted. json_object guarantees syntactically valid JSON only, not its shape: describe the schema in the prompt. For schema-constrained output in those other requests, use qwen3.6 or gemma4, where json_schema with strict constrains fields, types and required keys; or, on this model, force one function tool whose parameters is your schema (see Examples). Your output budget is used exactly as sent, with reasoning on or off (max_tokens and max_completion_tokens on /chat/completions, max_output_tokens on /responses). Reasoning tokens count inside that budget: with reasoning on, a small budget can be spent entirely on reasoning. On /chat/completions the answer then comes back with an empty content and finish_reason "length" (the reasoning is in reasoning_content, and usage.completion_tokens_details.reasoning_tokens equals completion_tokens); on /responses the reasoning arrives as a reasoning output item with no message text. Dense prompts can reason past 16k tokens. The reliable fix is reasoning off (reasoning_effort "none" on /chat/completions, reasoning.effort "none" on /responses); a lower effort is a hint on this model, not a cap. Or give a generous budget and stream the answer.'} specs={[ { label: 'Type', value: 'MoE (305B total)' }, { label: 'Quantization', value: 'FP8' }, @@ -370,12 +370,14 @@ Your output budget (`max_tokens` or `max_completion_tokens` on Chat Completions, `max_output_tokens` on Responses) is used exactly as you send it on every model, except on `glm5.3-flash`, where a value above 131072 is lowered to 131072, the most its upstream accepts. Reasoning is spent from that same budget, so with reasoning on a small -budget can run out before the answer starts: you get an empty `content` with -`finish_reason: "length"`, the reasoning in `message.reasoning_content` on Chat Completions (a `reasoning` output item on Responses), and +budget can run out before the answer starts. On Chat Completions you get an empty `content` with +`finish_reason: "length"`, the reasoning in `message.reasoning_content`, and `usage.completion_tokens_details.reasoning_tokens` equal to `completion_tokens`. +On Responses the reasoning arrives as a `reasoning` output item and there is no message text. On `deepseek-v4-flash` a dense prompt can reason past 16k tokens. If you only -need the answer, turn reasoning off (`reasoning_effort: "none"`) or lower the -effort; if you want the reasoning, give a generous budget and +need the answer, the reliable fix is to turn reasoning off: `reasoning_effort: "none"` on Chat Completions, `reasoning: {"effort": "none"}` on Responses (GLM cannot turn it off; `minimal` is its shortest). +A lower effort usually reasons less, but on `deepseek-v4-flash` and `qwen3.8-flash` it is a hint, not a cap; only `qwen3.6` and `gemma4` cap it, on Chat Completions. +If you want the reasoning, give a generous budget and [stream the answer](#long-outputs-stream-them). ```bash diff --git a/src/data/openapi.json b/src/data/openapi.json index 5933347..b90ed81 100644 --- a/src/data/openapi.json +++ b/src/data/openapi.json @@ -200,7 +200,7 @@ "max_tokens": { "type": "integer", "minimum": 1, - "description": "Maximum number of tokens to generate. A value above what the model can generate is rejected with a generic `400`, except on `glm5.3-flash` (see below). A request that does not fit the model's context window is rejected with a `400`: \"Context length exceeded for model '...'\" on `qwen3.6` and `gemma4`, usually the same text (sometimes the generic \"Invalid request\") on the other models. The value is used exactly as sent, with reasoning on or off, on every model except `glm5.3-flash`, where a value above 131072 is lowered to 131072, the most its upstream accepts. Reasoning tokens count inside it: with reasoning on, a small value can be spent entirely on reasoning, and the answer comes back with an empty `content` and `finish_reason: \"length\"` (the reasoning is in `message.reasoning_content`, and `usage.completion_tokens_details.reasoning_tokens` equals `completion_tokens`). Turn reasoning off (`reasoning_effort: \"none\"`), lower the effort, or send a generous value and use `stream: true`.", + "description": "Maximum number of tokens to generate. A value above what the model can generate is rejected with a generic `400`, except on `glm5.3-flash` (see below). A request that does not fit the model's context window is rejected with a `400`: \"Context length exceeded for model '...'\" on `qwen3.6` and `gemma4`, usually the same text (sometimes the generic \"Invalid request\") on the other models. The value is used exactly as sent, with reasoning on or off, on every model except `glm5.3-flash`, where a value above 131072 is lowered to 131072, the most its upstream accepts. Reasoning tokens count inside it: with reasoning on, a small value can be spent entirely on reasoning, and the answer comes back with an empty `content` and `finish_reason: \"length\"` (the reasoning is in `message.reasoning_content`, and `usage.completion_tokens_details.reasoning_tokens` equals `completion_tokens`). Turn reasoning off (`reasoning_effort: \"none\"`) for a reliable fix (GLM cannot turn it off; `minimal` is its shortest); a lower effort is a hint, not a cap, on `deepseek-v4-flash` and `qwen3.8-flash`. Or send a generous value and use `stream: true`.", "example": 512 }, "temperature": { diff --git a/src/tests/lib/docsReasoning.test.ts b/src/tests/lib/docsReasoning.test.ts index 0d95d10..f576ecf 100644 --- a/src/tests/lib/docsReasoning.test.ts +++ b/src/tests/lib/docsReasoning.test.ts @@ -107,6 +107,8 @@ describe('API reference: one documented way to turn reasoning off', () => { expect(d).toContain('`usage.completion_tokens_details.reasoning_tokens` equals `completion_tokens`'); expect(d).toContain('`reasoning_effort: "none"`'); expect(d).toContain('`stream: true`'); + expect(d).toContain('Turn reasoning off (`reasoning_effort: "none"`) for a reliable fix (GLM cannot turn it off; `minimal` is its shortest); a lower effort is a hint, not a cap, on `deepseek-v4-flash` and `qwen3.8-flash`.'); + expect(d).not.toMatch(/lower the effort,/); expect(d).not.toMatch(/16384|raised/); }); @@ -295,9 +297,31 @@ describe('output budget: passed through as sent, no floor (en + es)', () => { expect(section('')).not.toContain('on every\nmodel.'); }); - it('reasoning location is scoped per route (Chat Completions field, Responses output item)', () => { - expect(section('')).toContain('the reasoning in `message.reasoning_content` on Chat Completions (a `reasoning` output item on Responses)'); - expect(section('-es')).toContain('el razonamiento en `message.reasoning_content` en Chat Completions (un elemento de salida `reasoning` en Responses)'); + it('empty-answer signals are scoped to Chat Completions; Responses only promises the reasoning item', () => { + expect(section('')).toContain('On Chat Completions you get an empty `content` with\n`finish_reason: "length"`, the reasoning in `message.reasoning_content`, and'); + expect(section('')).toContain('On Responses the reasoning arrives as a `reasoning` output item and there is no message text.'); + expect(section('-es')).toContain('En Chat Completions recibes un `content` vacío con\n`finish_reason: "length"`, el razonamiento en `message.reasoning_content` y'); + expect(section('-es')).toContain('En Responses el razonamiento llega como un elemento de salida `reasoning` y no hay texto de mensaje.'); + // finish_reason / completion_tokens_details must not be promised for Responses. + for (const locale of ['', '-es'] as const) { + expect(section(locale)).not.toMatch(/Responses[^.]*finish_reason|Responses[^.]*completion_tokens_details/); + expect(card(locale)).not.toMatch(/\/responses[^.;]*finish_reason|\/responses[^.;]*completion_tokens_details/); + } + expect(card('')).toContain('on /responses the reasoning arrives as a reasoning output item with no message text'); + expect(card('-es')).toContain('en /responses el razonamiento llega como un elemento de salida reasoning sin texto de mensaje'); + }); + + it('remedies: none is the reliable fix, on both routes; lower effort is a hint on deepseek/qwen3.8-flash; GLM has no off switch', () => { + expect(section('')).toContain('the reliable fix is to turn reasoning off: `reasoning_effort: "none"` on Chat Completions, `reasoning: {"effort": "none"}` on Responses (GLM cannot turn it off; `minimal` is its shortest).'); + expect(section('')).toContain('A lower effort usually reasons less, but on `deepseek-v4-flash` and `qwen3.8-flash` it is a hint, not a cap; only `qwen3.6` and `gemma4` cap it, on Chat Completions.'); + expect(section('-es')).toContain('la solución fiable es apagar el razonamiento: `reasoning_effort: "none"` en Chat Completions, `reasoning: {"effort": "none"}` en Responses (GLM no puede apagarlo; `minimal` es lo más corto).'); + expect(section('-es')).toContain('Un esfuerzo menor suele razonar menos, pero en `deepseek-v4-flash` y `qwen3.8-flash` es una pista, no un tope; solo `qwen3.6` y `gemma4` lo acotan, en Chat Completions.'); + expect(card('')).toContain('The reliable fix is reasoning off (reasoning_effort "none" on /chat/completions, reasoning.effort "none" on /responses); a lower effort is a hint on this model, not a cap.'); + expect(card('-es')).toContain('La solución fiable es apagar el razonamiento (reasoning_effort "none" en /chat/completions, reasoning.effort "none" en /responses); un esfuerzo menor es una pista en este modelo, no un tope.'); + for (const locale of ['', '-es'] as const) { + // "lower the effort" is never offered as an unqualified fix. + expect(section(locale) + card(locale)).not.toMatch(/lower the effort|baja el esfuerzo/); + } }); it('mimo-v2.6-flash card: no number, points at Controlling reasoning (en + es)', () => { @@ -319,16 +343,16 @@ describe('output budget: passed through as sent, no floor (en + es)', () => { it('en wording', () => { expect(card('')).toContain('Your output budget is used exactly as sent, with reasoning on or off'); - expect(card('')).toContain('with reasoning on, a small budget can be spent entirely on reasoning, and the answer comes back with an empty content and finish_reason "length"'); - expect(card('')).toContain('turn reasoning off (reasoning_effort "none"), lower the effort, or give a generous budget and stream the answer.'); + expect(card('')).toContain('On /chat/completions the answer then comes back with an empty content and finish_reason "length"'); + expect(card('')).toContain('Or give a generous budget and stream the answer.'); expect(section('')).toContain('is used exactly as you send it on every model'); expect(section('')).toContain('[stream the answer](#long-outputs-stream-them)'); }); it('es wording', () => { expect(card('-es')).toContain('Tu presupuesto de salida se usa tal cual lo envías, con el razonamiento activado o apagado'); - expect(card('-es')).toContain('con el razonamiento activado, un presupuesto pequeño puede gastarse entero en razonar, y la respuesta llega con content vacío y finish_reason "length"'); - expect(card('-es')).toContain('apaga el razonamiento (reasoning_effort "none"), baja el esfuerzo, o da un presupuesto generoso y usa streaming.'); + expect(card('-es')).toContain('En /chat/completions la respuesta llega entonces con content vacío y finish_reason "length"'); + expect(card('-es')).toContain('O da un presupuesto generoso y usa streaming.'); expect(section('-es')).toContain('se usa tal cual lo envías en todos los modelos'); expect(section('-es')).toContain('[usa streaming](#respuestas-largas-usa-streaming)'); }); From 10951dd4e2a046ce2e1c932583ed8d987a44cd58 Mon Sep 17 00:00:00 2001 From: barckcode Date: Mon, 19 Oct 2026 13:20:16 +0200 Subject: [PATCH 4/6] docs(models): Responses truncation signal, bold lead, budget with reasoning off Non-blocking review follow-ups: - Responses signal measured live: status "incomplete" with incomplete_details.reason "max_output_tokens", only a reasoning output item, output_tokens_details.reasoning_tokens == output_tokens. In the reasoning section; the deepseek-v4-flash card says it briefly. - Bold lead "Empty answer with finish_reason "length"?" (en + es). - Note that with reasoning off the answer can still stop at the budget. - OpenAPI max_output_tokens: pass-through, glm5.3-flash exception, points to max_tokens. - Neutral wording for the glm5.3-flash ceiling ("the model's maximum"). - Drop the now-duplicated "its tokens count toward max_tokens". Co-Authored-By: Claude Opus 5.5 --- src/content/docs-es/models.mdx | 16 +++++++++------- src/content/docs/models.mdx | 14 ++++++++------ src/data/openapi.json | 4 ++-- src/lib/openapiSpec.test.ts | 5 +++++ src/tests/lib/docsReasoning.test.ts | 27 ++++++++++++++++++++------- 5 files changed, 44 insertions(+), 22 deletions(-) diff --git a/src/content/docs-es/models.mdx b/src/content/docs-es/models.mdx index d1acdbe..9dcd717 100644 --- a/src/content/docs-es/models.mdx +++ b/src/content/docs-es/models.mdx @@ -50,7 +50,7 @@ OpenAI y la misma `base URL`. tag="305B MoE" leftLabel="generación de texto, chat y visión" rightLabel="capacidades" - description={'Modelo MoE de 305B parámetros, servido en su variante Vision-Exp: acepta imágenes de entrada. Contexto de 1M tokens. Tool calling y razonamiento. Cuota de 3B tokens al mes por miembro. Salida estructurada: response_format json_object es compatible (en api.nan.builders el prompt debe contener la palabra JSON; si no, se rechaza con un 400). json_schema se acepta en peticiones a /chat/completions sin streaming y sin tools: la respuesta se comprueba contra tu esquema y, si no lo cumple, normalmente la vuelve a servir un proveedor que lo impone; una respuesta que no se pudo comprobar, o un fallo que no se pudo reemplazar a tiempo, se devuelve tal cual (valídala si tu código depende de ella). En api.nan.builders el resto de peticiones json_schema se rechazan con un 400: con streaming, con tools, a /responses, con n mayor que 1, con un esquema de más de 16 KB, que no sea un JSON Schema válido o que el gateway no valida (un $ref fuera del esquema, $dynamicRef o $anchor, un $id anidado o demasiado grande al resolver sus $ref); en api.helmcode.com se aceptan. json_object solo garantiza JSON sintácticamente válido, no su forma: describe el esquema en el prompt. Para salida restringida a un esquema en esas otras peticiones, usa qwen3.6 o gemma4, donde json_schema con strict restringe campos, tipos y claves obligatorias; o, en este modelo, fuerza una única herramienta de tipo función cuyo parameters sea tu esquema (ver Ejemplos). Tu presupuesto de salida se usa tal cual lo envías, con el razonamiento activado o apagado (max_tokens y max_completion_tokens en /chat/completions, max_output_tokens en /responses). Los tokens de razonamiento cuentan dentro de ese presupuesto: con el razonamiento activado, un presupuesto pequeño puede gastarse entero en razonar. En /chat/completions la respuesta llega entonces con content vacío y finish_reason "length" (el razonamiento está en reasoning_content, y usage.completion_tokens_details.reasoning_tokens es igual a completion_tokens); en /responses el razonamiento llega como un elemento de salida reasoning sin texto de mensaje. Los prompts densos pueden razonar más de 16k tokens. La solución fiable es apagar el razonamiento (reasoning_effort "none" en /chat/completions, reasoning.effort "none" en /responses); un esfuerzo menor es una pista en este modelo, no un tope. O da un presupuesto generoso y usa streaming.'} + description={'Modelo MoE de 305B parámetros, servido en su variante Vision-Exp: acepta imágenes de entrada. Contexto de 1M tokens. Tool calling y razonamiento. Cuota de 3B tokens al mes por miembro. Salida estructurada: response_format json_object es compatible (en api.nan.builders el prompt debe contener la palabra JSON; si no, se rechaza con un 400). json_schema se acepta en peticiones a /chat/completions sin streaming y sin tools: la respuesta se comprueba contra tu esquema y, si no lo cumple, normalmente la vuelve a servir un proveedor que lo impone; una respuesta que no se pudo comprobar, o un fallo que no se pudo reemplazar a tiempo, se devuelve tal cual (valídala si tu código depende de ella). En api.nan.builders el resto de peticiones json_schema se rechazan con un 400: con streaming, con tools, a /responses, con n mayor que 1, con un esquema de más de 16 KB, que no sea un JSON Schema válido o que el gateway no valida (un $ref fuera del esquema, $dynamicRef o $anchor, un $id anidado o demasiado grande al resolver sus $ref); en api.helmcode.com se aceptan. json_object solo garantiza JSON sintácticamente válido, no su forma: describe el esquema en el prompt. Para salida restringida a un esquema en esas otras peticiones, usa qwen3.6 o gemma4, donde json_schema con strict restringe campos, tipos y claves obligatorias; o, en este modelo, fuerza una única herramienta de tipo función cuyo parameters sea tu esquema (ver Ejemplos). Tu presupuesto de salida se usa tal cual lo envías, con el razonamiento activado o apagado (max_tokens y max_completion_tokens en /chat/completions, max_output_tokens en /responses). Los tokens de razonamiento cuentan dentro de ese presupuesto: con el razonamiento activado, un presupuesto pequeño puede gastarse entero en razonar. En /chat/completions la respuesta llega entonces con content vacío y finish_reason "length" (el razonamiento está en reasoning_content, y usage.completion_tokens_details.reasoning_tokens es igual a completion_tokens); en /responses el status es incomplete, con solo un elemento de salida reasoning. Los prompts densos pueden razonar más de 16k tokens. La solución fiable es apagar el razonamiento (reasoning_effort "none" en /chat/completions, reasoning.effort "none" en /responses); un esfuerzo menor es una pista en este modelo, no un tope. O da un presupuesto generoso y usa streaming.'} specs={[ { label: 'Tipo', value: 'MoE (305B total)' }, { label: 'Cuantización', value: 'FP8' }, @@ -364,22 +364,24 @@ el valor. | `glm5.3` · `glm5.3-flash` | No es posible: GLM siempre razona. `none` se aplica como `minimal`, su razonamiento más corto (unos pocos tokens). | De `minimal` a `max`; los valores más altos razonan más antes de responder. | Sin parámetro, todos los modelos razonan (`qwen3.6` y `gemma4` con un -presupuesto de 16.384 tokens). El razonamiento cuesta latencia y sus tokens -cuentan dentro de `max_tokens`. Usa solo los valores de esta página. +presupuesto de 16.384 tokens). El razonamiento cuesta latencia. Usa solo los valores de esta página. Cualquier otro valor se envía tal cual y queda fuera de este contrato: según el modelo, se rechaza con un `400` o se ignora. Tu presupuesto de salida (`max_tokens` o `max_completion_tokens` en Chat Completions, `max_output_tokens` en Responses) se usa tal cual lo envías en todos los modelos, -salvo en `glm5.3-flash`, donde un valor por encima de 131072 se baja a 131072, lo máximo que acepta su proveedor. -El razonamiento sale de ese mismo presupuesto, así que con -el razonamiento activado un presupuesto pequeño puede agotarse antes de que +salvo en `glm5.3-flash`, donde un valor por encima de 131072 se baja a 131072, el máximo del modelo. +Los tokens de razonamiento salen de ese mismo presupuesto. + +**¿Respuesta vacía con `finish_reason: "length"`?** Con el razonamiento activado, +un presupuesto pequeño puede agotarse antes de que empiece la respuesta. En Chat Completions recibes un `content` vacío con `finish_reason: "length"`, el razonamiento en `message.reasoning_content` y `usage.completion_tokens_details.reasoning_tokens` igual a `completion_tokens`. -En Responses el razonamiento llega como un elemento de salida `reasoning` y no hay texto de mensaje. +En Responses la respuesta tiene `status: "incomplete"` con `incomplete_details.reason: "max_output_tokens"`, solo un elemento de salida `reasoning` y ningún texto de mensaje, y `usage.output_tokens_details.reasoning_tokens` es igual a `output_tokens`. En `deepseek-v4-flash` un prompt denso puede razonar más de 16k tokens. Si solo necesitas la respuesta, la solución fiable es apagar el razonamiento: `reasoning_effort: "none"` en Chat Completions, `reasoning: {"effort": "none"}` en Responses (GLM no puede apagarlo; `minimal` es lo más corto). +Con el razonamiento apagado la respuesta también puede cortarse en el presupuesto, con texto, así que el presupuesto tiene que caberle a la respuesta. Un esfuerzo menor suele razonar menos, pero en `deepseek-v4-flash` y `qwen3.8-flash` es una pista, no un tope; solo `qwen3.6` y `gemma4` lo acotan, en Chat Completions. Si quieres el razonamiento, da un presupuesto generoso y [usa streaming](#respuestas-largas-usa-streaming). diff --git a/src/content/docs/models.mdx b/src/content/docs/models.mdx index b017edd..3582840 100644 --- a/src/content/docs/models.mdx +++ b/src/content/docs/models.mdx @@ -50,7 +50,7 @@ with the same `base URL`. tag="305B MoE" leftLabel="text generation, chat & vision" rightLabel="capabilities" - description={'305B parameter MoE model, served as the Vision-Exp variant: it takes images as input. 1M token context. Tool calling and reasoning. 3B token monthly quota per member. Structured output: response_format json_object is supported (on api.nan.builders the prompt must contain the word JSON, otherwise it is rejected with a 400). json_schema is accepted on non-streamed /chat/completions requests without tools: the answer is checked against your schema and, when it does not match, usually re-served by a provider that enforces it; an answer that could not be checked, or a miss the re-serve could not replace in time, is returned as is (validate it if your code depends on it). On api.nan.builders other json_schema requests are rejected with a 400: streamed, with tools, on /responses, with n above 1, with a schema over 16 KB, not a valid JSON Schema, or one the gateway will not validate (a $ref outside the schema, $dynamicRef or $anchor, a nested $id, or too large once its $refs are inlined); on api.helmcode.com they are accepted. json_object guarantees syntactically valid JSON only, not its shape: describe the schema in the prompt. For schema-constrained output in those other requests, use qwen3.6 or gemma4, where json_schema with strict constrains fields, types and required keys; or, on this model, force one function tool whose parameters is your schema (see Examples). Your output budget is used exactly as sent, with reasoning on or off (max_tokens and max_completion_tokens on /chat/completions, max_output_tokens on /responses). Reasoning tokens count inside that budget: with reasoning on, a small budget can be spent entirely on reasoning. On /chat/completions the answer then comes back with an empty content and finish_reason "length" (the reasoning is in reasoning_content, and usage.completion_tokens_details.reasoning_tokens equals completion_tokens); on /responses the reasoning arrives as a reasoning output item with no message text. Dense prompts can reason past 16k tokens. The reliable fix is reasoning off (reasoning_effort "none" on /chat/completions, reasoning.effort "none" on /responses); a lower effort is a hint on this model, not a cap. Or give a generous budget and stream the answer.'} + description={'305B parameter MoE model, served as the Vision-Exp variant: it takes images as input. 1M token context. Tool calling and reasoning. 3B token monthly quota per member. Structured output: response_format json_object is supported (on api.nan.builders the prompt must contain the word JSON, otherwise it is rejected with a 400). json_schema is accepted on non-streamed /chat/completions requests without tools: the answer is checked against your schema and, when it does not match, usually re-served by a provider that enforces it; an answer that could not be checked, or a miss the re-serve could not replace in time, is returned as is (validate it if your code depends on it). On api.nan.builders other json_schema requests are rejected with a 400: streamed, with tools, on /responses, with n above 1, with a schema over 16 KB, not a valid JSON Schema, or one the gateway will not validate (a $ref outside the schema, $dynamicRef or $anchor, a nested $id, or too large once its $refs are inlined); on api.helmcode.com they are accepted. json_object guarantees syntactically valid JSON only, not its shape: describe the schema in the prompt. For schema-constrained output in those other requests, use qwen3.6 or gemma4, where json_schema with strict constrains fields, types and required keys; or, on this model, force one function tool whose parameters is your schema (see Examples). Your output budget is used exactly as sent, with reasoning on or off (max_tokens and max_completion_tokens on /chat/completions, max_output_tokens on /responses). Reasoning tokens count inside that budget: with reasoning on, a small budget can be spent entirely on reasoning. On /chat/completions the answer then comes back with an empty content and finish_reason "length" (the reasoning is in reasoning_content, and usage.completion_tokens_details.reasoning_tokens equals completion_tokens); on /responses the status is incomplete, with only a reasoning output item. Dense prompts can reason past 16k tokens. The reliable fix is reasoning off (reasoning_effort "none" on /chat/completions, reasoning.effort "none" on /responses); a lower effort is a hint on this model, not a cap. Or give a generous budget and stream the answer.'} specs={[ { label: 'Type', value: 'MoE (305B total)' }, { label: 'Quantization', value: 'FP8' }, @@ -361,21 +361,23 @@ at the top level of the body or inside `extra_body`. Each one means exactly | `glm5.3` · `glm5.3-flash` | Not possible: GLM always reasons. `none` is applied as `minimal`, its shortest reasoning (a few tokens). | `minimal` to `max`; higher values reason longer before answering. | With no parameter every model reasons (`qwen3.6` and `gemma4` with a -16,384-token budget). Reasoning costs latency and its tokens count toward -`max_tokens`. Use only the values listed here. +16,384-token budget). Reasoning costs latency. Use only the values listed here. Any other value is passed through unchanged and is not part of this contract: depending on the model it is rejected with a `400` or ignored. Your output budget (`max_tokens` or `max_completion_tokens` on Chat Completions, `max_output_tokens` on Responses) is used exactly as you send it on every model, -except on `glm5.3-flash`, where a value above 131072 is lowered to 131072, the most its upstream accepts. -Reasoning is spent from that same budget, so with reasoning on a small +except on `glm5.3-flash`, where a value above 131072 is lowered to 131072, the model's maximum. +Reasoning tokens are spent from that same budget. + +**Empty answer with `finish_reason: "length"`?** With reasoning on, a small budget can run out before the answer starts. On Chat Completions you get an empty `content` with `finish_reason: "length"`, the reasoning in `message.reasoning_content`, and `usage.completion_tokens_details.reasoning_tokens` equal to `completion_tokens`. -On Responses the reasoning arrives as a `reasoning` output item and there is no message text. +On Responses the response has `status: "incomplete"` with `incomplete_details.reason: "max_output_tokens"`, only a `reasoning` output item and no message text, and `usage.output_tokens_details.reasoning_tokens` equals `output_tokens`. On `deepseek-v4-flash` a dense prompt can reason past 16k tokens. If you only need the answer, the reliable fix is to turn reasoning off: `reasoning_effort: "none"` on Chat Completions, `reasoning: {"effort": "none"}` on Responses (GLM cannot turn it off; `minimal` is its shortest). +With reasoning off the answer can still stop at the budget, with text, so the budget must fit the answer. A lower effort usually reasons less, but on `deepseek-v4-flash` and `qwen3.8-flash` it is a hint, not a cap; only `qwen3.6` and `gemma4` cap it, on Chat Completions. If you want the reasoning, give a generous budget and [stream the answer](#long-outputs-stream-them). diff --git a/src/data/openapi.json b/src/data/openapi.json index b90ed81..fbfc081 100644 --- a/src/data/openapi.json +++ b/src/data/openapi.json @@ -200,7 +200,7 @@ "max_tokens": { "type": "integer", "minimum": 1, - "description": "Maximum number of tokens to generate. A value above what the model can generate is rejected with a generic `400`, except on `glm5.3-flash` (see below). A request that does not fit the model's context window is rejected with a `400`: \"Context length exceeded for model '...'\" on `qwen3.6` and `gemma4`, usually the same text (sometimes the generic \"Invalid request\") on the other models. The value is used exactly as sent, with reasoning on or off, on every model except `glm5.3-flash`, where a value above 131072 is lowered to 131072, the most its upstream accepts. Reasoning tokens count inside it: with reasoning on, a small value can be spent entirely on reasoning, and the answer comes back with an empty `content` and `finish_reason: \"length\"` (the reasoning is in `message.reasoning_content`, and `usage.completion_tokens_details.reasoning_tokens` equals `completion_tokens`). Turn reasoning off (`reasoning_effort: \"none\"`) for a reliable fix (GLM cannot turn it off; `minimal` is its shortest); a lower effort is a hint, not a cap, on `deepseek-v4-flash` and `qwen3.8-flash`. Or send a generous value and use `stream: true`.", + "description": "Maximum number of tokens to generate. A value above what the model can generate is rejected with a generic `400`, except on `glm5.3-flash` (see below). A request that does not fit the model's context window is rejected with a `400`: \"Context length exceeded for model '...'\" on `qwen3.6` and `gemma4`, usually the same text (sometimes the generic \"Invalid request\") on the other models. The value is used exactly as sent, with reasoning on or off, on every model except `glm5.3-flash`, where a value above 131072 is lowered to 131072, the model's maximum. Reasoning tokens count inside it: with reasoning on, a small value can be spent entirely on reasoning, and the answer comes back with an empty `content` and `finish_reason: \"length\"` (the reasoning is in `message.reasoning_content`, and `usage.completion_tokens_details.reasoning_tokens` equals `completion_tokens`). Turn reasoning off (`reasoning_effort: \"none\"`) for a reliable fix (GLM cannot turn it off; `minimal` is its shortest); a lower effort is a hint, not a cap, on `deepseek-v4-flash` and `qwen3.8-flash`. Or send a generous value and use `stream: true`.", "example": 512 }, "temperature": { @@ -1087,7 +1087,7 @@ "max_output_tokens": { "type": "integer", "minimum": 1, - "description": "The maximum number of output tokens. Default 65536 on `qwen3.6`." + "description": "The maximum number of output tokens. Default 65536 on `qwen3.6`. Used exactly as sent (except `glm5.3-flash`, lowered to 131072). Reasoning tokens count inside it; see `max_tokens`." }, "reasoning": { "type": "object", diff --git a/src/lib/openapiSpec.test.ts b/src/lib/openapiSpec.test.ts index e13caf7..f5e21e8 100644 --- a/src/lib/openapiSpec.test.ts +++ b/src/lib/openapiSpec.test.ts @@ -919,6 +919,11 @@ describe('documented 400s: tool parameters and context overflow', () => { const maxTokens = (spec.paths as any)['/chat/completions'].post.requestBody.content['application/json'] .schema.properties.max_tokens.description; expect(maxTokens).toContain('on every model except `glm5.3-flash`, where a value above 131072 is lowered to 131072'); + expect(maxTokens).toContain("lowered to 131072, the model's maximum."); + expect(maxTokens).not.toContain('upstream accepts'); + const maxOut = (spec.paths as any)['/responses'].post.requestBody.content['application/json'].schema.properties + .max_output_tokens.description; + expect(maxOut).toContain('Used exactly as sent (except `glm5.3-flash`, lowered to 131072). Reasoning tokens count inside it; see `max_tokens`.'); expect(maxTokens).toContain('Reasoning tokens count inside it'); expect(maxTokens).not.toMatch(/raised to 16384|within 16384 tokens/); // Not promised as the exact text on deepseek-v4-flash: measured generic there. diff --git a/src/tests/lib/docsReasoning.test.ts b/src/tests/lib/docsReasoning.test.ts index f576ecf..4c180bf 100644 --- a/src/tests/lib/docsReasoning.test.ts +++ b/src/tests/lib/docsReasoning.test.ts @@ -100,7 +100,7 @@ describe('API reference: one documented way to turn reasoning off', () => { it('max_tokens: passed through as sent, reasoning counts inside it (no deepseek floor)', () => { const d = chatBody.max_tokens.description; - expect(d).toContain('The value is used exactly as sent, with reasoning on or off, on every model except `glm5.3-flash`, where a value above 131072 is lowered to 131072, the most its upstream accepts.'); + expect(d).toContain('The value is used exactly as sent, with reasoning on or off, on every model except `glm5.3-flash`, where a value above 131072 is lowered to 131072, the model\'s maximum.'); expect(d).not.toMatch(/above what the model can generate is rejected with a generic `400`\./); expect(d).toContain('the reasoning is in `message.reasoning_content`'); expect(d).toContain('an empty `content` and `finish_reason: "length"`'); @@ -292,23 +292,36 @@ describe('output budget: passed through as sent, no floor (en + es)', () => { } it(`the glm5.3-flash ceiling is the one exception, said in both locales`, () => { - expect(section('')).toContain('except on `glm5.3-flash`, where a value above 131072 is lowered to 131072, the most its upstream accepts.'); - expect(section('-es')).toContain('salvo en `glm5.3-flash`, donde un valor por encima de 131072 se baja a 131072, lo máximo que acepta su proveedor.'); + expect(section('')).toContain('except on `glm5.3-flash`, where a value above 131072 is lowered to 131072, the model\'s maximum.'); + expect(section('-es')).toContain('salvo en `glm5.3-flash`, donde un valor por encima de 131072 se baja a 131072, el máximo del modelo.'); expect(section('')).not.toContain('on every\nmodel.'); }); it('empty-answer signals are scoped to Chat Completions; Responses only promises the reasoning item', () => { expect(section('')).toContain('On Chat Completions you get an empty `content` with\n`finish_reason: "length"`, the reasoning in `message.reasoning_content`, and'); - expect(section('')).toContain('On Responses the reasoning arrives as a `reasoning` output item and there is no message text.'); + expect(section('')).toContain('On Responses the response has `status: "incomplete"` with `incomplete_details.reason: "max_output_tokens"`, only a `reasoning` output item and no message text, and `usage.output_tokens_details.reasoning_tokens` equals `output_tokens`.'); expect(section('-es')).toContain('En Chat Completions recibes un `content` vacío con\n`finish_reason: "length"`, el razonamiento en `message.reasoning_content` y'); - expect(section('-es')).toContain('En Responses el razonamiento llega como un elemento de salida `reasoning` y no hay texto de mensaje.'); + expect(section('-es')).toContain('En Responses la respuesta tiene `status: "incomplete"` con `incomplete_details.reason: "max_output_tokens"`, solo un elemento de salida `reasoning` y ningún texto de mensaje, y `usage.output_tokens_details.reasoning_tokens` es igual a `output_tokens`.'); // finish_reason / completion_tokens_details must not be promised for Responses. for (const locale of ['', '-es'] as const) { expect(section(locale)).not.toMatch(/Responses[^.]*finish_reason|Responses[^.]*completion_tokens_details/); expect(card(locale)).not.toMatch(/\/responses[^.;]*finish_reason|\/responses[^.;]*completion_tokens_details/); } - expect(card('')).toContain('on /responses the reasoning arrives as a reasoning output item with no message text'); - expect(card('-es')).toContain('en /responses el razonamiento llega como un elemento de salida reasoning sin texto de mensaje'); + expect(card('')).toContain('on /responses the status is incomplete, with only a reasoning output item'); + expect(card('-es')).toContain('en /responses el status es incomplete, con solo un elemento de salida reasoning'); + }); + + it('the empty-answer paragraph has a bold lead, and reasoning off still respects the budget (en + es)', () => { + expect(section('')).toContain('\n\n**Empty answer with `finish_reason: "length"`?** '); + expect(section('-es')).toContain('\n\n**¿Respuesta vacía con `finish_reason: "length"`?** '); + expect(section('')).toContain('With reasoning off the answer can still stop at the budget, with text, so the budget must fit the answer.'); + expect(section('-es')).toContain('Con el razonamiento apagado la respuesta también puede cortarse en el presupuesto, con texto, así que el presupuesto tiene que caberle a la respuesta.'); + // The old one-line mention is folded into the paragraph, not repeated. + expect(section('')).not.toContain('its tokens count toward'); + expect(section('-es')).not.toMatch(/sus tokens\s+cuentan dentro de/); + for (const locale of ['', '-es'] as const) { + expect(section(locale)).not.toMatch(/upstream accepts|acepta su proveedor/); + } }); it('remedies: none is the reliable fix, on both routes; lower effort is a hint on deepseek/qwen3.8-flash; GLM has no off switch', () => { From 29a409ba401ead226cc71ff20669da0702bd42cf Mon Sep 17 00:00:00 2001 From: barckcode Date: Mon, 19 Oct 2026 14:47:14 +0200 Subject: [PATCH 5/6] docs(models): name the deepseek-v4-flash 393216 output ceiling nan-devops#902 lowers any deepseek-v4-flash output budget above 393216 to 393216 (DeepSeek's own maximum), on max_tokens, max_completion_tokens and max_output_tokens. The pass-through contract now names both per-model ceilings (131072 on glm5.3-flash, 393216 on deepseek-v4-flash) in the reasoning section (en + es), the deepseek-v4-flash card and the OpenAPI max_tokens / max_output_tokens descriptions. Tests updated first; the Responses-signals test is renamed to what it checks. Co-Authored-By: Claude Opus 5.5 --- src/content/docs-es/models.mdx | 4 ++-- src/content/docs/models.mdx | 4 ++-- src/data/openapi.json | 4 ++-- src/lib/openapiSpec.test.ts | 7 +++---- src/tests/lib/docsReasoning.test.ts | 17 ++++++++++------- 5 files changed, 19 insertions(+), 17 deletions(-) diff --git a/src/content/docs-es/models.mdx b/src/content/docs-es/models.mdx index 9dcd717..9364541 100644 --- a/src/content/docs-es/models.mdx +++ b/src/content/docs-es/models.mdx @@ -50,7 +50,7 @@ OpenAI y la misma `base URL`. tag="305B MoE" leftLabel="generación de texto, chat y visión" rightLabel="capacidades" - description={'Modelo MoE de 305B parámetros, servido en su variante Vision-Exp: acepta imágenes de entrada. Contexto de 1M tokens. Tool calling y razonamiento. Cuota de 3B tokens al mes por miembro. Salida estructurada: response_format json_object es compatible (en api.nan.builders el prompt debe contener la palabra JSON; si no, se rechaza con un 400). json_schema se acepta en peticiones a /chat/completions sin streaming y sin tools: la respuesta se comprueba contra tu esquema y, si no lo cumple, normalmente la vuelve a servir un proveedor que lo impone; una respuesta que no se pudo comprobar, o un fallo que no se pudo reemplazar a tiempo, se devuelve tal cual (valídala si tu código depende de ella). En api.nan.builders el resto de peticiones json_schema se rechazan con un 400: con streaming, con tools, a /responses, con n mayor que 1, con un esquema de más de 16 KB, que no sea un JSON Schema válido o que el gateway no valida (un $ref fuera del esquema, $dynamicRef o $anchor, un $id anidado o demasiado grande al resolver sus $ref); en api.helmcode.com se aceptan. json_object solo garantiza JSON sintácticamente válido, no su forma: describe el esquema en el prompt. Para salida restringida a un esquema en esas otras peticiones, usa qwen3.6 o gemma4, donde json_schema con strict restringe campos, tipos y claves obligatorias; o, en este modelo, fuerza una única herramienta de tipo función cuyo parameters sea tu esquema (ver Ejemplos). Tu presupuesto de salida se usa tal cual lo envías, con el razonamiento activado o apagado (max_tokens y max_completion_tokens en /chat/completions, max_output_tokens en /responses). Los tokens de razonamiento cuentan dentro de ese presupuesto: con el razonamiento activado, un presupuesto pequeño puede gastarse entero en razonar. En /chat/completions la respuesta llega entonces con content vacío y finish_reason "length" (el razonamiento está en reasoning_content, y usage.completion_tokens_details.reasoning_tokens es igual a completion_tokens); en /responses el status es incomplete, con solo un elemento de salida reasoning. Los prompts densos pueden razonar más de 16k tokens. La solución fiable es apagar el razonamiento (reasoning_effort "none" en /chat/completions, reasoning.effort "none" en /responses); un esfuerzo menor es una pista en este modelo, no un tope. O da un presupuesto generoso y usa streaming.'} + description={'Modelo MoE de 305B parámetros, servido en su variante Vision-Exp: acepta imágenes de entrada. Contexto de 1M tokens. Tool calling y razonamiento. Cuota de 3B tokens al mes por miembro. Salida estructurada: response_format json_object es compatible (en api.nan.builders el prompt debe contener la palabra JSON; si no, se rechaza con un 400). json_schema se acepta en peticiones a /chat/completions sin streaming y sin tools: la respuesta se comprueba contra tu esquema y, si no lo cumple, normalmente la vuelve a servir un proveedor que lo impone; una respuesta que no se pudo comprobar, o un fallo que no se pudo reemplazar a tiempo, se devuelve tal cual (valídala si tu código depende de ella). En api.nan.builders el resto de peticiones json_schema se rechazan con un 400: con streaming, con tools, a /responses, con n mayor que 1, con un esquema de más de 16 KB, que no sea un JSON Schema válido o que el gateway no valida (un $ref fuera del esquema, $dynamicRef o $anchor, un $id anidado o demasiado grande al resolver sus $ref); en api.helmcode.com se aceptan. json_object solo garantiza JSON sintácticamente válido, no su forma: describe el esquema en el prompt. Para salida restringida a un esquema en esas otras peticiones, usa qwen3.6 o gemma4, donde json_schema con strict restringe campos, tipos y claves obligatorias; o, en este modelo, fuerza una única herramienta de tipo función cuyo parameters sea tu esquema (ver Ejemplos). Tu presupuesto de salida se usa tal cual lo envías (por encima de 393216, su máximo, se baja a 393216), con el razonamiento activado o apagado (max_tokens y max_completion_tokens en /chat/completions, max_output_tokens en /responses). Los tokens de razonamiento cuentan dentro de ese presupuesto: con el razonamiento activado, un presupuesto pequeño puede gastarse entero en razonar. En /chat/completions la respuesta llega entonces con content vacío y finish_reason "length" (el razonamiento está en reasoning_content, y usage.completion_tokens_details.reasoning_tokens es igual a completion_tokens); en /responses el status es incomplete, con solo un elemento de salida reasoning. Los prompts densos pueden razonar más de 16k tokens. La solución fiable es apagar el razonamiento (reasoning_effort "none" en /chat/completions, reasoning.effort "none" en /responses); un esfuerzo menor es una pista en este modelo, no un tope. O da un presupuesto generoso y usa streaming.'} specs={[ { label: 'Tipo', value: 'MoE (305B total)' }, { label: 'Cuantización', value: 'FP8' }, @@ -370,7 +370,7 @@ se rechaza con un `400` o se ignora. Tu presupuesto de salida (`max_tokens` o `max_completion_tokens` en Chat Completions, `max_output_tokens` en Responses) se usa tal cual lo envías en todos los modelos, -salvo en `glm5.3-flash`, donde un valor por encima de 131072 se baja a 131072, el máximo del modelo. +salvo por encima del máximo del modelo en dos modelos, donde se baja a ese máximo: 131072 en `glm5.3-flash`, 393216 en `deepseek-v4-flash`. Los tokens de razonamiento salen de ese mismo presupuesto. **¿Respuesta vacía con `finish_reason: "length"`?** Con el razonamiento activado, diff --git a/src/content/docs/models.mdx b/src/content/docs/models.mdx index 3582840..0481353 100644 --- a/src/content/docs/models.mdx +++ b/src/content/docs/models.mdx @@ -50,7 +50,7 @@ with the same `base URL`. tag="305B MoE" leftLabel="text generation, chat & vision" rightLabel="capabilities" - description={'305B parameter MoE model, served as the Vision-Exp variant: it takes images as input. 1M token context. Tool calling and reasoning. 3B token monthly quota per member. Structured output: response_format json_object is supported (on api.nan.builders the prompt must contain the word JSON, otherwise it is rejected with a 400). json_schema is accepted on non-streamed /chat/completions requests without tools: the answer is checked against your schema and, when it does not match, usually re-served by a provider that enforces it; an answer that could not be checked, or a miss the re-serve could not replace in time, is returned as is (validate it if your code depends on it). On api.nan.builders other json_schema requests are rejected with a 400: streamed, with tools, on /responses, with n above 1, with a schema over 16 KB, not a valid JSON Schema, or one the gateway will not validate (a $ref outside the schema, $dynamicRef or $anchor, a nested $id, or too large once its $refs are inlined); on api.helmcode.com they are accepted. json_object guarantees syntactically valid JSON only, not its shape: describe the schema in the prompt. For schema-constrained output in those other requests, use qwen3.6 or gemma4, where json_schema with strict constrains fields, types and required keys; or, on this model, force one function tool whose parameters is your schema (see Examples). Your output budget is used exactly as sent, with reasoning on or off (max_tokens and max_completion_tokens on /chat/completions, max_output_tokens on /responses). Reasoning tokens count inside that budget: with reasoning on, a small budget can be spent entirely on reasoning. On /chat/completions the answer then comes back with an empty content and finish_reason "length" (the reasoning is in reasoning_content, and usage.completion_tokens_details.reasoning_tokens equals completion_tokens); on /responses the status is incomplete, with only a reasoning output item. Dense prompts can reason past 16k tokens. The reliable fix is reasoning off (reasoning_effort "none" on /chat/completions, reasoning.effort "none" on /responses); a lower effort is a hint on this model, not a cap. Or give a generous budget and stream the answer.'} + description={'305B parameter MoE model, served as the Vision-Exp variant: it takes images as input. 1M token context. Tool calling and reasoning. 3B token monthly quota per member. Structured output: response_format json_object is supported (on api.nan.builders the prompt must contain the word JSON, otherwise it is rejected with a 400). json_schema is accepted on non-streamed /chat/completions requests without tools: the answer is checked against your schema and, when it does not match, usually re-served by a provider that enforces it; an answer that could not be checked, or a miss the re-serve could not replace in time, is returned as is (validate it if your code depends on it). On api.nan.builders other json_schema requests are rejected with a 400: streamed, with tools, on /responses, with n above 1, with a schema over 16 KB, not a valid JSON Schema, or one the gateway will not validate (a $ref outside the schema, $dynamicRef or $anchor, a nested $id, or too large once its $refs are inlined); on api.helmcode.com they are accepted. json_object guarantees syntactically valid JSON only, not its shape: describe the schema in the prompt. For schema-constrained output in those other requests, use qwen3.6 or gemma4, where json_schema with strict constrains fields, types and required keys; or, on this model, force one function tool whose parameters is your schema (see Examples). Your output budget is used exactly as sent (above 393216, its maximum, it is lowered to 393216), with reasoning on or off (max_tokens and max_completion_tokens on /chat/completions, max_output_tokens on /responses). Reasoning tokens count inside that budget: with reasoning on, a small budget can be spent entirely on reasoning. On /chat/completions the answer then comes back with an empty content and finish_reason "length" (the reasoning is in reasoning_content, and usage.completion_tokens_details.reasoning_tokens equals completion_tokens); on /responses the status is incomplete, with only a reasoning output item. Dense prompts can reason past 16k tokens. The reliable fix is reasoning off (reasoning_effort "none" on /chat/completions, reasoning.effort "none" on /responses); a lower effort is a hint on this model, not a cap. Or give a generous budget and stream the answer.'} specs={[ { label: 'Type', value: 'MoE (305B total)' }, { label: 'Quantization', value: 'FP8' }, @@ -367,7 +367,7 @@ on the model it is rejected with a `400` or ignored. Your output budget (`max_tokens` or `max_completion_tokens` on Chat Completions, `max_output_tokens` on Responses) is used exactly as you send it on every model, -except on `glm5.3-flash`, where a value above 131072 is lowered to 131072, the model's maximum. +except above the model's maximum on two models, where it is lowered to that maximum: 131072 on `glm5.3-flash`, 393216 on `deepseek-v4-flash`. Reasoning tokens are spent from that same budget. **Empty answer with `finish_reason: "length"`?** With reasoning on, a small diff --git a/src/data/openapi.json b/src/data/openapi.json index fbfc081..ff819e9 100644 --- a/src/data/openapi.json +++ b/src/data/openapi.json @@ -200,7 +200,7 @@ "max_tokens": { "type": "integer", "minimum": 1, - "description": "Maximum number of tokens to generate. A value above what the model can generate is rejected with a generic `400`, except on `glm5.3-flash` (see below). A request that does not fit the model's context window is rejected with a `400`: \"Context length exceeded for model '...'\" on `qwen3.6` and `gemma4`, usually the same text (sometimes the generic \"Invalid request\") on the other models. The value is used exactly as sent, with reasoning on or off, on every model except `glm5.3-flash`, where a value above 131072 is lowered to 131072, the model's maximum. Reasoning tokens count inside it: with reasoning on, a small value can be spent entirely on reasoning, and the answer comes back with an empty `content` and `finish_reason: \"length\"` (the reasoning is in `message.reasoning_content`, and `usage.completion_tokens_details.reasoning_tokens` equals `completion_tokens`). Turn reasoning off (`reasoning_effort: \"none\"`) for a reliable fix (GLM cannot turn it off; `minimal` is its shortest); a lower effort is a hint, not a cap, on `deepseek-v4-flash` and `qwen3.8-flash`. Or send a generous value and use `stream: true`.", + "description": "Maximum number of tokens to generate. A value above what the model can generate is rejected with a generic `400`, except on `glm5.3-flash` and `deepseek-v4-flash` (see below). A request that does not fit the model's context window is rejected with a `400`: \"Context length exceeded for model '...'\" on `qwen3.6` and `gemma4`, usually the same text (sometimes the generic \"Invalid request\") on the other models. The value is used exactly as sent, with reasoning on or off, except above the model's maximum on two models, where it is lowered to that maximum: 131072 on `glm5.3-flash`, 393216 on `deepseek-v4-flash`. Reasoning tokens count inside it: with reasoning on, a small value can be spent entirely on reasoning, and the answer comes back with an empty `content` and `finish_reason: \"length\"` (the reasoning is in `message.reasoning_content`, and `usage.completion_tokens_details.reasoning_tokens` equals `completion_tokens`). Turn reasoning off (`reasoning_effort: \"none\"`) for a reliable fix (GLM cannot turn it off; `minimal` is its shortest); a lower effort is a hint, not a cap, on `deepseek-v4-flash` and `qwen3.8-flash`. Or send a generous value and use `stream: true`.", "example": 512 }, "temperature": { @@ -1087,7 +1087,7 @@ "max_output_tokens": { "type": "integer", "minimum": 1, - "description": "The maximum number of output tokens. Default 65536 on `qwen3.6`. Used exactly as sent (except `glm5.3-flash`, lowered to 131072). Reasoning tokens count inside it; see `max_tokens`." + "description": "The maximum number of output tokens. Default 65536 on `qwen3.6`. Used exactly as sent, except above the model's maximum: lowered to 131072 on `glm5.3-flash`, 393216 on `deepseek-v4-flash`. Reasoning tokens count inside it; see `max_tokens`." }, "reasoning": { "type": "object", diff --git a/src/lib/openapiSpec.test.ts b/src/lib/openapiSpec.test.ts index f5e21e8..2beae8a 100644 --- a/src/lib/openapiSpec.test.ts +++ b/src/lib/openapiSpec.test.ts @@ -918,18 +918,17 @@ describe('documented 400s: tool parameters and context overflow', () => { it('max_tokens is passed through as sent: no deepseek-v4-flash floor, in the spec and on both cards', () => { const maxTokens = (spec.paths as any)['/chat/completions'].post.requestBody.content['application/json'] .schema.properties.max_tokens.description; - expect(maxTokens).toContain('on every model except `glm5.3-flash`, where a value above 131072 is lowered to 131072'); - expect(maxTokens).toContain("lowered to 131072, the model's maximum."); + expect(maxTokens).toContain("except above the model's maximum on two models, where it is lowered to that maximum: 131072 on `glm5.3-flash`, 393216 on `deepseek-v4-flash`."); expect(maxTokens).not.toContain('upstream accepts'); const maxOut = (spec.paths as any)['/responses'].post.requestBody.content['application/json'].schema.properties .max_output_tokens.description; - expect(maxOut).toContain('Used exactly as sent (except `glm5.3-flash`, lowered to 131072). Reasoning tokens count inside it; see `max_tokens`.'); + expect(maxOut).toContain("Used exactly as sent, except above the model's maximum: lowered to 131072 on `glm5.3-flash`, 393216 on `deepseek-v4-flash`. Reasoning tokens count inside it; see `max_tokens`."); expect(maxTokens).toContain('Reasoning tokens count inside it'); expect(maxTokens).not.toMatch(/raised to 16384|within 16384 tokens/); // Not promised as the exact text on deepseek-v4-flash: measured generic there. expect(maxTokens).not.toMatch(/deepseek-v4-flash[^.]*Context length exceeded/); // glm5.3-flash lowers instead of rejecting, so the generic-400 rule names its exception. - expect(maxTokens).toContain('A value above what the model can generate is rejected with a generic `400`, except on `glm5.3-flash` (see below).'); + expect(maxTokens).toContain('A value above what the model can generate is rejected with a generic `400`, except on `glm5.3-flash` and `deepseek-v4-flash` (see below).'); const card = (locale: string) => { const page = readFileSync( diff --git a/src/tests/lib/docsReasoning.test.ts b/src/tests/lib/docsReasoning.test.ts index 4c180bf..350d7f4 100644 --- a/src/tests/lib/docsReasoning.test.ts +++ b/src/tests/lib/docsReasoning.test.ts @@ -100,7 +100,8 @@ describe('API reference: one documented way to turn reasoning off', () => { it('max_tokens: passed through as sent, reasoning counts inside it (no deepseek floor)', () => { const d = chatBody.max_tokens.description; - expect(d).toContain('The value is used exactly as sent, with reasoning on or off, on every model except `glm5.3-flash`, where a value above 131072 is lowered to 131072, the model\'s maximum.'); + expect(d).toContain('The value is used exactly as sent, with reasoning on or off, except above the model\'s maximum on two models, where it is lowered to that maximum: 131072 on `glm5.3-flash`, 393216 on `deepseek-v4-flash`.'); + expect(d).toContain('rejected with a generic `400`, except on `glm5.3-flash` and `deepseek-v4-flash` (see below).'); expect(d).not.toMatch(/above what the model can generate is rejected with a generic `400`\./); expect(d).toContain('the reasoning is in `message.reasoning_content`'); expect(d).toContain('an empty `content` and `finish_reason: "length"`'); @@ -291,13 +292,15 @@ describe('output budget: passed through as sent, no floor (en + es)', () => { }); } - it(`the glm5.3-flash ceiling is the one exception, said in both locales`, () => { - expect(section('')).toContain('except on `glm5.3-flash`, where a value above 131072 is lowered to 131072, the model\'s maximum.'); - expect(section('-es')).toContain('salvo en `glm5.3-flash`, donde un valor por encima de 131072 se baja a 131072, el máximo del modelo.'); + it(`the two per-model ceilings (glm5.3-flash 131072, deepseek-v4-flash 393216) are the only exceptions, in both locales`, () => { + expect(section('')).toContain('except above the model\'s maximum on two models, where it is lowered to that maximum: 131072 on `glm5.3-flash`, 393216 on `deepseek-v4-flash`.'); + expect(section('-es')).toContain('salvo por encima del máximo del modelo en dos modelos, donde se baja a ese máximo: 131072 en `glm5.3-flash`, 393216 en `deepseek-v4-flash`.'); + expect(card('')).toContain('Your output budget is used exactly as sent (above 393216, its maximum, it is lowered to 393216), with reasoning on or off'); + expect(card('-es')).toContain('Tu presupuesto de salida se usa tal cual lo envías (por encima de 393216, su máximo, se baja a 393216), con el razonamiento activado o apagado'); expect(section('')).not.toContain('on every\nmodel.'); }); - it('empty-answer signals are scoped to Chat Completions; Responses only promises the reasoning item', () => { + it('empty-answer signals are scoped to Chat Completions; Responses signals are status, incomplete_details, the reasoning item and usage', () => { expect(section('')).toContain('On Chat Completions you get an empty `content` with\n`finish_reason: "length"`, the reasoning in `message.reasoning_content`, and'); expect(section('')).toContain('On Responses the response has `status: "incomplete"` with `incomplete_details.reason: "max_output_tokens"`, only a `reasoning` output item and no message text, and `usage.output_tokens_details.reasoning_tokens` equals `output_tokens`.'); expect(section('-es')).toContain('En Chat Completions recibes un `content` vacío con\n`finish_reason: "length"`, el razonamiento en `message.reasoning_content` y'); @@ -355,7 +358,7 @@ describe('output budget: passed through as sent, no floor (en + es)', () => { } it('en wording', () => { - expect(card('')).toContain('Your output budget is used exactly as sent, with reasoning on or off'); + expect(card('')).toContain('Your output budget is used exactly as sent (above 393216'); expect(card('')).toContain('On /chat/completions the answer then comes back with an empty content and finish_reason "length"'); expect(card('')).toContain('Or give a generous budget and stream the answer.'); expect(section('')).toContain('is used exactly as you send it on every model'); @@ -363,7 +366,7 @@ describe('output budget: passed through as sent, no floor (en + es)', () => { }); it('es wording', () => { - expect(card('-es')).toContain('Tu presupuesto de salida se usa tal cual lo envías, con el razonamiento activado o apagado'); + expect(card('-es')).toContain('Tu presupuesto de salida se usa tal cual lo envías (por encima de 393216'); expect(card('-es')).toContain('En /chat/completions la respuesta llega entonces con content vacío y finish_reason "length"'); expect(card('-es')).toContain('O da un presupuesto generoso y usa streaming.'); expect(section('-es')).toContain('se usa tal cual lo envías en todos los modelos'); From 1c909edbb1d2498470758669380116c554e21b91 Mon Sep 17 00:00:00 2001 From: barckcode Date: Mon, 19 Oct 2026 15:25:02 +0200 Subject: [PATCH 6/6] docs(openapi): name the Responses off switch in max_output_tokens The description pointed Responses users at max_tokens' remedy, which uses the Chat Completions form (reasoning_effort). Responses needs reasoning: {"effort": "none"}; say it in place, with the GLM caveat. Co-Authored-By: Claude Opus 5.5 --- src/data/openapi.json | 2 +- src/lib/openapiSpec.test.ts | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/src/data/openapi.json b/src/data/openapi.json index 9269708..133bf08 100644 --- a/src/data/openapi.json +++ b/src/data/openapi.json @@ -1087,7 +1087,7 @@ "max_output_tokens": { "type": "integer", "minimum": 1, - "description": "The maximum number of output tokens. Default 65536 on `qwen3.6`. Used exactly as sent, except above the model's maximum: lowered to 131072 on `glm5.3-flash`, 393216 on `deepseek-v4-flash`. Reasoning tokens count inside it: a budget spent entirely on reasoning returns `status: \"incomplete\"` with `incomplete_details.reason: \"max_output_tokens\"` and only a `reasoning` output item; see `max_tokens` for the remedies." + "description": "The maximum number of output tokens. Default 65536 on `qwen3.6`. Used exactly as sent, except above the model's maximum: lowered to 131072 on `glm5.3-flash`, 393216 on `deepseek-v4-flash`. Reasoning tokens count inside it: a budget spent entirely on reasoning returns `status: \"incomplete\"` with `incomplete_details.reason: \"max_output_tokens\"` and only a `reasoning` output item; the reliable fix is reasoning off with `reasoning: {\"effort\": \"none\"}` (GLM cannot turn it off; `minimal` is its shortest), or a larger value." }, "reasoning": { "type": "object", diff --git a/src/lib/openapiSpec.test.ts b/src/lib/openapiSpec.test.ts index eb40469..5251a4c 100644 --- a/src/lib/openapiSpec.test.ts +++ b/src/lib/openapiSpec.test.ts @@ -924,6 +924,9 @@ describe('documented 400s: tool parameters and context overflow', () => { .max_output_tokens.description; expect(maxOut).toContain("Used exactly as sent, except above the model's maximum: lowered to 131072 on `glm5.3-flash`, 393216 on `deepseek-v4-flash`."); expect(maxOut).toContain('a budget spent entirely on reasoning returns `status: "incomplete"` with `incomplete_details.reason: "max_output_tokens"` and only a `reasoning` output item'); + // Responses has its own off switch: the remedy must name it, not the Chat form. + expect(maxOut).toContain('the reliable fix is reasoning off with `reasoning: {"effort": "none"}` (GLM cannot turn it off; `minimal` is its shortest)'); + expect(maxOut).not.toContain('reasoning_effort'); // The header and finish_reason are Chat Completions signals (nan-devops#901 reads `choices`). expect(maxOut).not.toMatch(/finish_reason|x-nan-truncated-in-reasoning/); expect(maxTokens).toContain('non-streamed calls also carry the `x-nan-truncated-in-reasoning: true` response header.');