diff --git a/src/content/docs-es/examples.md b/src/content/docs-es/examples.md index 9c8ae05..b9cc40f 100644 --- a/src/content/docs-es/examples.md +++ b/src/content/docs-es/examples.md @@ -434,7 +434,7 @@ curl https://api.nan.builders/v1/chat/completions \ }' ``` -Con el razonamiento activado se recomienda `max_tokens ≥ 300`, para dejarle sitio. +Los tokens de razonamiento cuentan dentro de `max_tokens`, que se usa tal cual lo envías. Con el razonamiento activado, un presupuesto pequeño puede gastarse entero en razonar y llegar con `content` vacío y `finish_reason: "length"`. Si solo necesitas la respuesta, pon `"reasoning_effort": "none"`; si no, da un `max_tokens` generoso (los prompts densos pueden razonar miles de tokens) y usa `stream: true`. Mira [Controlar el razonamiento](/es/docs/models#controlar-el-razonamiento). ### visión (curl) diff --git a/src/content/docs-es/models.mdx b/src/content/docs-es/models.mdx index 913a26a..5140725 100644 --- a/src/content/docs-es/models.mdx +++ b/src/content/docs-es/models.mdx @@ -50,7 +50,7 @@ OpenAI y la misma `base URL`. tag="305B MoE" leftLabel="generación de texto, chat y visión" rightLabel="capacidades" - description={'Modelo MoE de 305B parámetros, servido en su variante Vision-Exp: acepta imágenes de entrada. Contexto de 1M tokens. Tool calling y razonamiento. Cuota de 3B tokens al mes por miembro. Salida estructurada: response_format json_object es compatible (en api.nan.builders el prompt debe contener la palabra JSON; si no, se rechaza con un 400). json_schema se acepta en peticiones a /chat/completions sin streaming y sin tools: la respuesta se comprueba contra tu esquema y, si no lo cumple, normalmente la vuelve a servir un proveedor que lo impone; una respuesta que no se pudo comprobar, o un fallo que no se pudo reemplazar a tiempo, se devuelve tal cual (valídala si tu código depende de ella). En api.nan.builders el resto de peticiones json_schema se rechazan con un 400: con streaming, con tools, a /responses, con n mayor que 1, con un esquema de más de 16 KB, que no sea un JSON Schema válido o que el gateway no valida (un $ref fuera del esquema, $dynamicRef o $anchor, un $id anidado o demasiado grande al resolver sus $ref); en api.helmcode.com se aceptan. json_object solo garantiza JSON sintácticamente válido, no su forma: describe el esquema en el prompt. Para salida restringida a un esquema en esas otras peticiones, usa qwen3.6 o gemma4, donde json_schema con strict restringe campos, tipos y claves obligatorias; o, en este modelo, fuerza una única herramienta de tipo función cuyo parameters sea tu esquema (ver Ejemplos). Tu presupuesto de salida se usa exactamente como lo mandás en las dos rutas (max_tokens y max_completion_tokens en /chat/completions, max_output_tokens en /responses): nunca lo subimos. El razonamiento se factura contra ese mismo presupuesto, así que un presupuesto chico con un prompt que exige deliberar puede gastarse entero pensando, y la respuesta llega como HTTP 200 con finish_reason "length" y contenido vacío. Detectalo con finish_reason "length" Y contenido vacío Y usage.completion_tokens_details.reasoning_tokens igual a usage.completion_tokens; las llamadas sin streaming además traen el header x-nan-truncated-in-reasoning: true. Subí el presupuesto, o usá streaming.'} + description={'Modelo MoE de 305B parámetros, servido en su variante Vision-Exp: acepta imágenes de entrada. Contexto de 1M tokens. Tool calling y razonamiento. Cuota de 3B tokens al mes por miembro. Salida estructurada: response_format json_object es compatible (en api.nan.builders el prompt debe contener la palabra JSON; si no, se rechaza con un 400). json_schema se acepta en peticiones a /chat/completions sin streaming y sin tools: la respuesta se comprueba contra tu esquema y, si no lo cumple, normalmente la vuelve a servir un proveedor que lo impone; una respuesta que no se pudo comprobar, o un fallo que no se pudo reemplazar a tiempo, se devuelve tal cual (valídala si tu código depende de ella). En api.nan.builders el resto de peticiones json_schema se rechazan con un 400: con streaming, con tools, a /responses, con n mayor que 1, con un esquema de más de 16 KB, que no sea un JSON Schema válido o que el gateway no valida (un $ref fuera del esquema, $dynamicRef o $anchor, un $id anidado o demasiado grande al resolver sus $ref); en api.helmcode.com se aceptan. json_object solo garantiza JSON sintácticamente válido, no su forma: describe el esquema en el prompt. Para salida restringida a un esquema en esas otras peticiones, usa qwen3.6 o gemma4, donde json_schema con strict restringe campos, tipos y claves obligatorias; o, en este modelo, fuerza una única herramienta de tipo función cuyo parameters sea tu esquema (ver Ejemplos). Tu presupuesto de salida se usa tal cual lo envías (por encima de 393216, su máximo, se baja a 393216), con el razonamiento activado o apagado (max_tokens y max_completion_tokens en /chat/completions, max_output_tokens en /responses). Los tokens de razonamiento cuentan dentro de ese presupuesto: con el razonamiento activado, un presupuesto pequeño puede gastarse entero en razonar. En /chat/completions la respuesta llega entonces con content vacío y finish_reason "length" (el razonamiento está en reasoning_content, y usage.completion_tokens_details.reasoning_tokens es igual a completion_tokens; las llamadas sin streaming además traen el header de respuesta x-nan-truncated-in-reasoning: true); en /responses el status es incomplete, con solo un elemento de salida reasoning. Los prompts densos pueden razonar más de 16k tokens. La solución fiable es apagar el razonamiento (reasoning_effort "none" en /chat/completions, reasoning.effort "none" en /responses); un esfuerzo menor es una pista en este modelo, no un tope. O da un presupuesto generoso y usa streaming.'} specs={[ { label: 'Tipo', value: 'MoE (305B total)' }, { label: 'Cuantización', value: 'FP8' }, @@ -134,7 +134,7 @@ OpenAI y la misma `base URL`. ]} items={[ 'Tool calling (function calling)', - 'Control del razonamiento (reasoning_effort: none lo apaga · de low a max; se recomienda max_tokens ≥ 300) · detalles', + 'Control del razonamiento (reasoning_effort: none lo apaga · de low a max; el razonamiento cuenta dentro de max_tokens) · detalles', 'Visión (entrada de imagen)', 'Audio (entrada de audio)', 'Contexto de 1M tokens', @@ -364,11 +364,29 @@ el valor. | `glm5.3` · `glm5.3-flash` | No es posible: GLM siempre razona. `none` se aplica como `minimal`, su razonamiento más corto (unos pocos tokens). | De `minimal` a `max`; los valores más altos razonan más antes de responder. | Sin parámetro, todos los modelos razonan (`qwen3.6` y `gemma4` con un -presupuesto de 16.384 tokens). El razonamiento cuesta latencia y sus tokens -cuentan dentro de `max_tokens`. Usa solo los valores de esta página. +presupuesto de 16.384 tokens). El razonamiento cuesta latencia. Usa solo los valores de esta página. Cualquier otro valor se envía tal cual y queda fuera de este contrato: según el modelo, se rechaza con un `400` o se ignora. +Tu presupuesto de salida (`max_tokens` o `max_completion_tokens` en Chat +Completions, `max_output_tokens` en Responses) se usa tal cual lo envías en todos los modelos, +salvo por encima del máximo del modelo en dos modelos, donde se baja a ese máximo: 131072 en `glm5.3-flash`, 393216 en `deepseek-v4-flash`. +Los tokens de razonamiento salen de ese mismo presupuesto. + +**¿Respuesta vacía con `finish_reason: "length"`?** Con el razonamiento activado, +un presupuesto pequeño puede agotarse antes de que +empiece la respuesta. En Chat Completions recibes un `content` vacío con +`finish_reason: "length"`, el razonamiento en `message.reasoning_content` y +`usage.completion_tokens_details.reasoning_tokens` igual a `completion_tokens`; +solo las tres juntas significan que el presupuesto se agotó razonando, y una llamada sin streaming además trae el header de respuesta `x-nan-truncated-in-reasoning: true`. +En Responses la respuesta tiene `status: "incomplete"` con `incomplete_details.reason: "max_output_tokens"`, solo un elemento de salida `reasoning` y ningún texto de mensaje, y `usage.output_tokens_details.reasoning_tokens` es igual a `output_tokens`. +En `deepseek-v4-flash` un prompt denso puede razonar más de 16k tokens. Si solo +necesitas la respuesta, la solución fiable es apagar el razonamiento: `reasoning_effort: "none"` en Chat Completions, `reasoning: {"effort": "none"}` en Responses (GLM no puede apagarlo; `minimal` es lo más corto). +Con el razonamiento apagado la respuesta también puede cortarse en el presupuesto, con texto, así que el presupuesto tiene que caberle a la respuesta. +Un esfuerzo menor suele razonar menos, pero en `deepseek-v4-flash` y `qwen3.8-flash` es una pista, no un tope; solo `qwen3.6` y `gemma4` lo acotan, en Chat Completions. Para un tope estricto en `qwen3.6`, `gemma4` y `qwen3.8-flash`, mira [Limitar el razonamiento con un presupuesto de tokens](#limitar-el-razonamiento-con-un-presupuesto-de-tokens). +Si quieres el razonamiento, da un presupuesto generoso y +[usa streaming](#respuestas-largas-usa-streaming). + ```bash curl https://api.nan.builders/v1/chat/completions \ -H "Authorization: Bearer $NAN_API_KEY" \ diff --git a/src/content/docs/examples.md b/src/content/docs/examples.md index 13795c4..6c0e930 100644 --- a/src/content/docs/examples.md +++ b/src/content/docs/examples.md @@ -434,7 +434,7 @@ curl https://api.nan.builders/v1/chat/completions \ }' ``` -With reasoning enabled, `max_tokens ≥ 300` is recommended to leave room for reasoning. +Reasoning tokens count inside `max_tokens`, which is used exactly as sent. With reasoning on, a small budget can be spent entirely on reasoning and come back with an empty `content` and `finish_reason: "length"`. If you only need the answer, set `"reasoning_effort": "none"`; otherwise give a generous `max_tokens` (dense prompts can reason for thousands of tokens) and use `stream: true`. See [Controlling reasoning](/docs/models#controlling-reasoning). ### vision (curl) diff --git a/src/content/docs/models.mdx b/src/content/docs/models.mdx index 0a42cfb..9a4bdd7 100644 --- a/src/content/docs/models.mdx +++ b/src/content/docs/models.mdx @@ -50,7 +50,7 @@ with the same `base URL`. tag="305B MoE" leftLabel="text generation, chat & vision" rightLabel="capabilities" - description={'305B parameter MoE model, served as the Vision-Exp variant: it takes images as input. 1M token context. Tool calling and reasoning. 3B token monthly quota per member. Structured output: response_format json_object is supported (on api.nan.builders the prompt must contain the word JSON, otherwise it is rejected with a 400). json_schema is accepted on non-streamed /chat/completions requests without tools: the answer is checked against your schema and, when it does not match, usually re-served by a provider that enforces it; an answer that could not be checked, or a miss the re-serve could not replace in time, is returned as is (validate it if your code depends on it). On api.nan.builders other json_schema requests are rejected with a 400: streamed, with tools, on /responses, with n above 1, with a schema over 16 KB, not a valid JSON Schema, or one the gateway will not validate (a $ref outside the schema, $dynamicRef or $anchor, a nested $id, or too large once its $refs are inlined); on api.helmcode.com they are accepted. json_object guarantees syntactically valid JSON only, not its shape: describe the schema in the prompt. For schema-constrained output in those other requests, use qwen3.6 or gemma4, where json_schema with strict constrains fields, types and required keys; or, on this model, force one function tool whose parameters is your schema (see Examples). Your output budget is used exactly as sent on every route (max_tokens and max_completion_tokens on /chat/completions, max_output_tokens on /responses): we never raise it. Reasoning is billed against that same budget, so a small budget on a prompt that needs deliberation can be spent entirely thinking, and the answer then arrives as HTTP 200 with finish_reason "length" and empty content. Detect it with finish_reason "length" AND empty content AND usage.completion_tokens_details.reasoning_tokens equal to usage.completion_tokens; non-streamed calls also carry the response header x-nan-truncated-in-reasoning: true. Raise the budget, or stream.'} + description={'305B parameter MoE model, served as the Vision-Exp variant: it takes images as input. 1M token context. Tool calling and reasoning. 3B token monthly quota per member. Structured output: response_format json_object is supported (on api.nan.builders the prompt must contain the word JSON, otherwise it is rejected with a 400). json_schema is accepted on non-streamed /chat/completions requests without tools: the answer is checked against your schema and, when it does not match, usually re-served by a provider that enforces it; an answer that could not be checked, or a miss the re-serve could not replace in time, is returned as is (validate it if your code depends on it). On api.nan.builders other json_schema requests are rejected with a 400: streamed, with tools, on /responses, with n above 1, with a schema over 16 KB, not a valid JSON Schema, or one the gateway will not validate (a $ref outside the schema, $dynamicRef or $anchor, a nested $id, or too large once its $refs are inlined); on api.helmcode.com they are accepted. json_object guarantees syntactically valid JSON only, not its shape: describe the schema in the prompt. For schema-constrained output in those other requests, use qwen3.6 or gemma4, where json_schema with strict constrains fields, types and required keys; or, on this model, force one function tool whose parameters is your schema (see Examples). Your output budget is used exactly as sent (above 393216, its maximum, it is lowered to 393216), with reasoning on or off (max_tokens and max_completion_tokens on /chat/completions, max_output_tokens on /responses). Reasoning tokens count inside that budget: with reasoning on, a small budget can be spent entirely on reasoning. On /chat/completions the answer then comes back with an empty content and finish_reason "length" (the reasoning is in reasoning_content, and usage.completion_tokens_details.reasoning_tokens equals completion_tokens; non-streamed calls also carry the response header x-nan-truncated-in-reasoning: true); on /responses the status is incomplete, with only a reasoning output item. Dense prompts can reason past 16k tokens. The reliable fix is reasoning off (reasoning_effort "none" on /chat/completions, reasoning.effort "none" on /responses); a lower effort is a hint on this model, not a cap. Or give a generous budget and stream the answer.'} specs={[ { label: 'Type', value: 'MoE (305B total)' }, { label: 'Quantization', value: 'FP8' }, @@ -134,7 +134,7 @@ with the same `base URL`. ]} items={[ 'Tool calling (function calling)', - 'Reasoning control (reasoning_effort: none turns it off · low to max; recommended max_tokens ≥ 300) · details', + 'Reasoning control (reasoning_effort: none turns it off · low to max; reasoning counts inside max_tokens) · details', 'Vision (image input)', 'Audio (audio input)', '1M token context', @@ -361,11 +361,28 @@ at the top level of the body or inside `extra_body`. Each one means exactly | `glm5.3` · `glm5.3-flash` | Not possible: GLM always reasons. `none` is applied as `minimal`, its shortest reasoning (a few tokens). | `minimal` to `max`; higher values reason longer before answering. | With no parameter every model reasons (`qwen3.6` and `gemma4` with a -16,384-token budget). Reasoning costs latency and its tokens count toward -`max_tokens`. Use only the values listed here. +16,384-token budget). Reasoning costs latency. Use only the values listed here. Any other value is passed through unchanged and is not part of this contract: depending on the model it is rejected with a `400` or ignored. +Your output budget (`max_tokens` or `max_completion_tokens` on Chat Completions, +`max_output_tokens` on Responses) is used exactly as you send it on every model, +except above the model's maximum on two models, where it is lowered to that maximum: 131072 on `glm5.3-flash`, 393216 on `deepseek-v4-flash`. +Reasoning tokens are spent from that same budget. + +**Empty answer with `finish_reason: "length"`?** With reasoning on, a small +budget can run out before the answer starts. On Chat Completions you get an empty `content` with +`finish_reason: "length"`, the reasoning in `message.reasoning_content`, and +`usage.completion_tokens_details.reasoning_tokens` equal to `completion_tokens`; +only the three together mean the budget ran out while reasoning, and a non-streamed call also carries the response header `x-nan-truncated-in-reasoning: true`. +On Responses the response has `status: "incomplete"` with `incomplete_details.reason: "max_output_tokens"`, only a `reasoning` output item and no message text, and `usage.output_tokens_details.reasoning_tokens` equals `output_tokens`. +On `deepseek-v4-flash` a dense prompt can reason past 16k tokens. If you only +need the answer, the reliable fix is to turn reasoning off: `reasoning_effort: "none"` on Chat Completions, `reasoning: {"effort": "none"}` on Responses (GLM cannot turn it off; `minimal` is its shortest). +With reasoning off the answer can still stop at the budget, with text, so the budget must fit the answer. +A lower effort usually reasons less, but on `deepseek-v4-flash` and `qwen3.8-flash` it is a hint, not a cap; only `qwen3.6` and `gemma4` cap it, on Chat Completions. For a hard cap on `qwen3.6`, `gemma4` and `qwen3.8-flash`, see [Capping reasoning with a token budget](#capping-reasoning-with-a-token-budget). +If you want the reasoning, give a generous budget and +[stream the answer](#long-outputs-stream-them). + ```bash curl https://api.nan.builders/v1/chat/completions \ -H "Authorization: Bearer $NAN_API_KEY" \ diff --git a/src/data/openapi.json b/src/data/openapi.json index ab19ea4..133bf08 100644 --- a/src/data/openapi.json +++ b/src/data/openapi.json @@ -200,7 +200,7 @@ "max_tokens": { "type": "integer", "minimum": 1, - "description": "Maximum number of tokens to generate. A value above what the model can generate is rejected with a generic `400`. A request that does not fit the model's context window is rejected with a `400`: \"Context length exceeded for model '...'\" on `qwen3.6` and `gemma4`, usually the same text (sometimes the generic \"Invalid request\") on the other models. The value is used exactly as sent on every model: it is never raised. Reasoning is billed against this same budget, so a small value on a prompt that needs deliberation can be spent entirely thinking, and the answer then arrives as HTTP 200 with `finish_reason` `\"length\"` and empty content; non-streamed calls carry the `x-nan-truncated-in-reasoning: true` response header. Raise the budget, or stream.", + "description": "Maximum number of tokens to generate. A value above what the model can generate is rejected with a generic `400`, except on `glm5.3-flash` and `deepseek-v4-flash` (see below). A request that does not fit the model's context window is rejected with a `400`: \"Context length exceeded for model '...'\" on `qwen3.6` and `gemma4`, usually the same text (sometimes the generic \"Invalid request\") on the other models. The value is used exactly as sent, with reasoning on or off, except above the model's maximum on two models, where it is lowered to that maximum: 131072 on `glm5.3-flash`, 393216 on `deepseek-v4-flash`. Reasoning tokens count inside it: with reasoning on, a small value can be spent entirely on reasoning, and the answer comes back with an empty `content` and `finish_reason: \"length\"` (the reasoning is in `message.reasoning_content`, and `usage.completion_tokens_details.reasoning_tokens` equals `completion_tokens`); non-streamed calls also carry the `x-nan-truncated-in-reasoning: true` response header. Turn reasoning off (`reasoning_effort: \"none\"`) for a reliable fix (GLM cannot turn it off; `minimal` is its shortest); a lower effort is a hint, not a cap, on `deepseek-v4-flash` and `qwen3.8-flash`. Or send a generous value and use `stream: true`.", "example": 512 }, "temperature": { @@ -1087,7 +1087,7 @@ "max_output_tokens": { "type": "integer", "minimum": 1, - "description": "The maximum number of output tokens. Used exactly as sent: it is never raised. Reasoning is billed against this same budget, so the answer can come back as 200 with `finish_reason` `\"length\"` and empty content; non-streamed calls then carry the `x-nan-truncated-in-reasoning: true` response header. Default 65536 on `qwen3.6`." + "description": "The maximum number of output tokens. Default 65536 on `qwen3.6`. Used exactly as sent, except above the model's maximum: lowered to 131072 on `glm5.3-flash`, 393216 on `deepseek-v4-flash`. Reasoning tokens count inside it: a budget spent entirely on reasoning returns `status: \"incomplete\"` with `incomplete_details.reason: \"max_output_tokens\"` and only a `reasoning` output item; the reliable fix is reasoning off with `reasoning: {\"effort\": \"none\"}` (GLM cannot turn it off; `minimal` is its shortest), or a larger value." }, "reasoning": { "type": "object", diff --git a/src/lib/openapiSpec.test.ts b/src/lib/openapiSpec.test.ts index 9e2d352..5251a4c 100644 --- a/src/lib/openapiSpec.test.ts +++ b/src/lib/openapiSpec.test.ts @@ -874,9 +874,10 @@ describe('documented context: qwen3.8-flash is 1M', () => { * automatically. The exact "Context length exceeded for model '...'" text was * measured on the hosted models (gemma4); on deepseek-v4-flash an overflow * measured 2026-09-29 came back as the GENERIC "Invalid request. Check your - * request parameters.", so the docs promise the text only where measured. On - * deepseek-v4-flash a small `max_tokens` is raised to 16384 so the reasoning - * fits, which is why a prompt close to the window overflows with a small one. + * request parameters.", so the docs promise the text only where measured. + * Since 2026-10-19 there is no output-budget floor: `max_tokens` is passed + * through as sent on every model, so a small one no longer overflows a prompt + * close to the window. */ describe('documented 400s: tool parameters and context overflow', () => { const schemas = spec.components.schemas as any; @@ -914,16 +915,27 @@ describe('documented 400s: tool parameters and context overflow', () => { expect(bad).toContain('Invalid request. Check your request parameters.'); }); - it('explains the deepseek-v4-flash 16384 floor on max_tokens, in the spec and on both cards', () => { + it('max_tokens is passed through as sent: no deepseek-v4-flash floor, in the spec and on both cards', () => { const maxTokens = (spec.paths as any)['/chat/completions'].post.requestBody.content['application/json'] .schema.properties.max_tokens.description; - expect(maxTokens).toContain('The value is used exactly as sent on every model: it is never raised.'); - expect(maxTokens).toContain('Reasoning is billed against this same budget'); - expect(maxTokens).toContain('x-nan-truncated-in-reasoning: true'); - expect(maxTokens).not.toContain('raised to 16384'); + expect(maxTokens).toContain("except above the model's maximum on two models, where it is lowered to that maximum: 131072 on `glm5.3-flash`, 393216 on `deepseek-v4-flash`."); + expect(maxTokens).not.toContain('upstream accepts'); + const maxOut = (spec.paths as any)['/responses'].post.requestBody.content['application/json'].schema.properties + .max_output_tokens.description; + expect(maxOut).toContain("Used exactly as sent, except above the model's maximum: lowered to 131072 on `glm5.3-flash`, 393216 on `deepseek-v4-flash`."); + expect(maxOut).toContain('a budget spent entirely on reasoning returns `status: "incomplete"` with `incomplete_details.reason: "max_output_tokens"` and only a `reasoning` output item'); + // Responses has its own off switch: the remedy must name it, not the Chat form. + expect(maxOut).toContain('the reliable fix is reasoning off with `reasoning: {"effort": "none"}` (GLM cannot turn it off; `minimal` is its shortest)'); + expect(maxOut).not.toContain('reasoning_effort'); + // The header and finish_reason are Chat Completions signals (nan-devops#901 reads `choices`). + expect(maxOut).not.toMatch(/finish_reason|x-nan-truncated-in-reasoning/); + expect(maxTokens).toContain('non-streamed calls also carry the `x-nan-truncated-in-reasoning: true` response header.'); + expect(maxTokens).toContain('Reasoning tokens count inside it'); + expect(maxTokens).not.toMatch(/raised to 16384|within 16384 tokens/); // Not promised as the exact text on deepseek-v4-flash: measured generic there. expect(maxTokens).not.toMatch(/deepseek-v4-flash[^.]*Context length exceeded/); - expect(maxTokens).toMatch(/above what the model can generate is rejected with a generic `400`/); + // glm5.3-flash lowers instead of rejecting, so the generic-400 rule names its exception. + expect(maxTokens).toContain('A value above what the model can generate is rejected with a generic `400`, except on `glm5.3-flash` and `deepseek-v4-flash` (see below).'); const card = (locale: string) => { const page = readFileSync( @@ -933,12 +945,14 @@ describe('documented 400s: tool parameters and context overflow', () => { const start = page.indexOf('id="deepseek-v4-flash"'); return page.slice(start, page.indexOf('/>', start)); }; - expect(card('')).toContain('Your output budget is used exactly as sent on every route'); - expect(card('')).toContain('non-streamed calls also carry the response header x-nan-truncated-in-reasoning: true'); - expect(card('')).not.toContain('raised to 16384'); - expect(card('-es')).toContain('Tu presupuesto de salida se usa exactamente como lo mandás en las dos rutas'); - expect(card('-es')).toContain('las llamadas sin streaming además traen el header x-nan-truncated-in-reasoning: true'); - expect(card('-es')).not.toContain('se sube a 16384'); + expect(card('')).toContain('Your output budget is used exactly as sent'); + expect(card('')).toContain('max_output_tokens on /responses'); + expect(card('')).toContain('non-streamed calls also carry the response header x-nan-truncated-in-reasoning: true); on /responses'); + expect(card('')).not.toMatch(/raised to 16384|never lowered|within 16384 tokens/); + expect(card('-es')).toContain('Tu presupuesto de salida se usa tal cual lo envías'); + expect(card('-es')).toContain('max_output_tokens en /responses'); + expect(card('-es')).toContain('las llamadas sin streaming además traen el header de respuesta x-nan-truncated-in-reasoning: true); en /responses'); + expect(card('-es')).not.toMatch(/se sube a 16384|nunca se baja|a menos de 16384 tokens/); }); }); diff --git a/src/tests/lib/docsReasoning.test.ts b/src/tests/lib/docsReasoning.test.ts index 668bfbf..2f33bc6 100644 --- a/src/tests/lib/docsReasoning.test.ts +++ b/src/tests/lib/docsReasoning.test.ts @@ -98,12 +98,20 @@ describe('API reference: one documented way to turn reasoning off', () => { } }); - it('max_tokens: the budget is used as sent, and reasoning counts against it', () => { - expect(chatBody.max_tokens.description).toContain( - 'The value is used exactly as sent on every model: it is never raised.', - ); - expect(chatBody.max_tokens.description).toContain('x-nan-truncated-in-reasoning: true'); - expect(chatBody.max_tokens.description).not.toContain('raised to 16384'); + it('max_tokens: passed through as sent, reasoning counts inside it (no deepseek floor)', () => { + const d = chatBody.max_tokens.description; + expect(d).toContain('The value is used exactly as sent, with reasoning on or off, except above the model\'s maximum on two models, where it is lowered to that maximum: 131072 on `glm5.3-flash`, 393216 on `deepseek-v4-flash`.'); + expect(d).toContain('rejected with a generic `400`, except on `glm5.3-flash` and `deepseek-v4-flash` (see below).'); + expect(d).not.toMatch(/above what the model can generate is rejected with a generic `400`\./); + expect(d).toContain('the reasoning is in `message.reasoning_content`'); + expect(d).toContain('an empty `content` and `finish_reason: "length"`'); + expect(d).toContain('`usage.completion_tokens_details.reasoning_tokens` equals `completion_tokens`'); + expect(d).toContain('`reasoning_effort: "none"`'); + expect(d).toContain('`stream: true`'); + expect(d).toContain('non-streamed calls also carry the `x-nan-truncated-in-reasoning: true` response header.'); + expect(d).toContain('Turn reasoning off (`reasoning_effort: "none"`) for a reliable fix (GLM cannot turn it off; `minimal` is its shortest); a lower effort is a hint, not a cap, on `deepseek-v4-flash` and `qwen3.8-flash`.'); + expect(d).not.toMatch(/lower the effort,/); + expect(d).not.toMatch(/16384|raised/); }); it('lists the accepted aliases and says the explicit value wins', () => { @@ -270,25 +278,141 @@ describe('model cards agree with the table (en + es)', () => { } }); -describe('deepseek-v4-flash card: the max_tokens floor only applies while reasoning is on (en + es)', () => { +/** + * NO OUTPUT-BUDGET FLOOR (owner decision 2026-10-19, nan-devops + * docs/community-report-2026-10-19-output-budget.md). Until then a + * deepseek-v4-flash budget below 16384 was silently raised while reasoning was + * on. Now the budget is passed through as sent on every model and route, so + * the docs must say that reasoning counts inside it and how a reasoning- + * exhausted answer looks, and must not describe the floor any more. + */ +describe('output budget: passed through as sent, no floor (en + es)', () => { const card = (locale: '' | '-es') => { const text = page(locale); const start = text.indexOf('name="deepseek-v4-flash"'); return text.slice(start, text.indexOf('/>', start)); }; + const examples = (locale: '' | '-es') => + readFileSync(resolve(here, `../../content/docs${locale}/examples.md`), 'utf-8'); + // `≥` / `&ge;` too: card strings are HTML, and the entity slipped past `≥` once. + const FLOOR = /raised to 16384|se sube a 16384|16384 nunca se baja|never lowered|within 16384 tokens|a menos de 16384 tokens|(?:≥|≥|&ge;)\s*(?:<\/?code>)?\s*300|max_tokens\s*(?:≥|≥|&ge;)/; + + for (const locale of ['', '-es'] as const) { + it(`${locale || 'en'}: no page describes the floor or the old max_tokens >= 300 advice`, () => { + expect(page(locale)).not.toMatch(FLOOR); + expect(examples(locale)).not.toMatch(FLOOR); + }); + + it(`${locale || 'en'}: the reasoning section and the card name every field and the empty-answer signature`, () => { + for (const text of [section(locale), card(locale)]) { + for (const needle of ['max_tokens', 'max_completion_tokens', 'max_output_tokens', 'finish_reason', '"length"', + 'reasoning_content', 'usage.completion_tokens_details.reasoning_tokens', 'completion_tokens', 'reasoning_effort', '"none"']) { + expect(text, needle).toContain(needle); + } + } + }); + + it(`${locale || 'en'}: examples point at reasoning off or a generous budget plus streaming`, () => { + expect(examples(locale)).toContain('`finish_reason: "length"`'); + expect(examples(locale)).toContain('`"reasoning_effort": "none"`'); + expect(examples(locale)).toContain('`stream: true`'); + }); + } + + it(`the two per-model ceilings (glm5.3-flash 131072, deepseek-v4-flash 393216) are the only exceptions, in both locales`, () => { + expect(section('')).toContain('except above the model\'s maximum on two models, where it is lowered to that maximum: 131072 on `glm5.3-flash`, 393216 on `deepseek-v4-flash`.'); + expect(section('-es')).toContain('salvo por encima del máximo del modelo en dos modelos, donde se baja a ese máximo: 131072 en `glm5.3-flash`, 393216 en `deepseek-v4-flash`.'); + expect(card('')).toContain('Your output budget is used exactly as sent (above 393216, its maximum, it is lowered to 393216), with reasoning on or off'); + expect(card('-es')).toContain('Tu presupuesto de salida se usa tal cual lo envías (por encima de 393216, su máximo, se baja a 393216), con el razonamiento activado o apagado'); + expect(section('')).not.toContain('on every\nmodel.'); + }); + + it('empty-answer signals are scoped to Chat Completions; Responses signals are status, incomplete_details, the reasoning item and usage', () => { + expect(section('')).toContain('On Chat Completions you get an empty `content` with\n`finish_reason: "length"`, the reasoning in `message.reasoning_content`, and'); + expect(section('')).toContain('only the three together mean the budget ran out while reasoning, and a non-streamed call also carries the response header `x-nan-truncated-in-reasoning: true`.'); + expect(section('-es')).toContain('solo las tres juntas significan que el presupuesto se agotó razonando, y una llamada sin streaming además trae el header de respuesta `x-nan-truncated-in-reasoning: true`.'); + // The header is a Chat Completions signal: never promised on Responses. + for (const locale of ['', '-es'] as const) { + expect(section(locale)).not.toMatch(/Responses[^.]*x-nan-truncated-in-reasoning/); + } + expect(section('')).toContain('On Responses the response has `status: "incomplete"` with `incomplete_details.reason: "max_output_tokens"`, only a `reasoning` output item and no message text, and `usage.output_tokens_details.reasoning_tokens` equals `output_tokens`.'); + expect(section('-es')).toContain('En Chat Completions recibes un `content` vacío con\n`finish_reason: "length"`, el razonamiento en `message.reasoning_content` y'); + expect(section('-es')).toContain('En Responses la respuesta tiene `status: "incomplete"` con `incomplete_details.reason: "max_output_tokens"`, solo un elemento de salida `reasoning` y ningún texto de mensaje, y `usage.output_tokens_details.reasoning_tokens` es igual a `output_tokens`.'); + // finish_reason / completion_tokens_details must not be promised for Responses. + for (const locale of ['', '-es'] as const) { + expect(section(locale)).not.toMatch(/Responses[^.]*finish_reason|Responses[^.]*completion_tokens_details/); + expect(card(locale)).not.toMatch(/\/responses[^.;]*finish_reason|\/responses[^.;]*completion_tokens_details/); + } + expect(card('')).toContain('on /responses the status is incomplete, with only a reasoning output item'); + expect(card('-es')).toContain('en /responses el status es incomplete, con solo un elemento de salida reasoning'); + }); + + it('the empty-answer paragraph has a bold lead, and reasoning off still respects the budget (en + es)', () => { + expect(section('')).toContain('\n\n**Empty answer with `finish_reason: "length"`?** '); + expect(section('-es')).toContain('\n\n**¿Respuesta vacía con `finish_reason: "length"`?** '); + expect(section('')).toContain('With reasoning off the answer can still stop at the budget, with text, so the budget must fit the answer.'); + expect(section('-es')).toContain('Con el razonamiento apagado la respuesta también puede cortarse en el presupuesto, con texto, así que el presupuesto tiene que caberle a la respuesta.'); + // The old one-line mention is folded into the paragraph, not repeated. + expect(section('')).not.toContain('its tokens count toward'); + expect(section('-es')).not.toMatch(/sus tokens\s+cuentan dentro de/); + for (const locale of ['', '-es'] as const) { + expect(section(locale)).not.toMatch(/upstream accepts|acepta su proveedor/); + } + }); + + it('remedies: none is the reliable fix, on both routes; lower effort is a hint on deepseek/qwen3.8-flash; GLM has no off switch', () => { + expect(section('')).toContain('the reliable fix is to turn reasoning off: `reasoning_effort: "none"` on Chat Completions, `reasoning: {"effort": "none"}` on Responses (GLM cannot turn it off; `minimal` is its shortest).'); + expect(section('')).toContain('A lower effort usually reasons less, but on `deepseek-v4-flash` and `qwen3.8-flash` it is a hint, not a cap; only `qwen3.6` and `gemma4` cap it, on Chat Completions.'); + expect(section('-es')).toContain('la solución fiable es apagar el razonamiento: `reasoning_effort: "none"` en Chat Completions, `reasoning: {"effort": "none"}` en Responses (GLM no puede apagarlo; `minimal` es lo más corto).'); + expect(section('-es')).toContain('Un esfuerzo menor suele razonar menos, pero en `deepseek-v4-flash` y `qwen3.8-flash` es una pista, no un tope; solo `qwen3.6` y `gemma4` lo acotan, en Chat Completions.'); + expect(card('')).toContain('The reliable fix is reasoning off (reasoning_effort "none" on /chat/completions, reasoning.effort "none" on /responses); a lower effort is a hint on this model, not a cap.'); + expect(card('-es')).toContain('La solución fiable es apagar el razonamiento (reasoning_effort "none" en /chat/completions, reasoning.effort "none" en /responses); un esfuerzo menor es una pista en este modelo, no un tope.'); + for (const locale of ['', '-es'] as const) { + // "lower the effort" is never offered as an unqualified fix. + expect(section(locale) + card(locale)).not.toMatch(/lower the effort|baja el esfuerzo/); + } + }); + + it('mimo-v2.6-flash card: no number, points at Controlling reasoning (en + es)', () => { + const mimo = (locale: '' | '-es') => cardReasoning(locale, 'mimo-v2.6-flash').join(' '); + expect(mimo('')).toContain('reasoning counts inside max_tokens'); + expect(mimo('-es')).toContain('el razonamiento cuenta dentro de max_tokens'); + expect(mimo('')).not.toMatch(FLOOR); + expect(mimo('-es')).not.toMatch(FLOOR); + }); + + for (const locale of ['', '-es'] as const) { + it(`${locale || 'en'}: the examples paragraph is model-generic`, () => { + const ex = examples(locale); + const i = ex.indexOf('`finish_reason: "length"`'); + const para = ex.slice(ex.lastIndexOf('\n\n', i), ex.indexOf('\n\n', i)); + expect(para).not.toContain('deepseek-v4-flash'); + }); + } + + it('en wording', () => { + expect(card('')).toContain('Your output budget is used exactly as sent (above 393216'); + expect(card('')).toContain('On /chat/completions the answer then comes back with an empty content and finish_reason "length"'); + expect(card('')).toContain('Or give a generous budget and stream the answer.'); + expect(section('')).toContain('is used exactly as you send it on every model'); + expect(section('')).toContain('[stream the answer](#long-outputs-stream-them)'); + }); - it('en', () => { - expect(card('')).toContain( - 'Your output budget is used exactly as sent on every route (max_tokens and max_completion_tokens on /chat/completions, max_output_tokens on /responses): we never raise it.', - ); - expect(card('')).toContain('non-streamed calls also carry the response header x-nan-truncated-in-reasoning: true'); - expect(card('')).not.toContain('raised to 16384'); + it('es wording', () => { + expect(card('-es')).toContain('Tu presupuesto de salida se usa tal cual lo envías (por encima de 393216'); + expect(card('-es')).toContain('En /chat/completions la respuesta llega entonces con content vacío y finish_reason "length"'); + expect(card('-es')).toContain('O da un presupuesto generoso y usa streaming.'); + expect(section('-es')).toContain('se usa tal cual lo envías en todos los modelos'); + expect(section('-es')).toContain('[usa streaming](#respuestas-largas-usa-streaming)'); }); - it('es', () => { - expect(card('-es')).toContain('Tu presupuesto de salida se usa exactamente como lo mandás en las dos rutas'); - expect(card('-es')).toContain('las llamadas sin streaming además traen el header x-nan-truncated-in-reasoning: true'); - expect(card('-es')).not.toContain('se sube a 16384'); + it('both anchors the prose links to exist as headings', () => { + expect(page('')).toContain('\n## Long outputs: stream them.\n'); + expect(page('-es')).toContain('\n## Respuestas largas: usa streaming.\n'); + expect(page('')).toContain('\n### Capping reasoning with a token budget.\n'); + expect(page('-es')).toContain('\n### Limitar el razonamiento con un presupuesto de tokens.\n'); + expect(section('')).toContain('[Capping reasoning with a token budget](#capping-reasoning-with-a-token-budget)'); + expect(section('-es')).toContain('[Limitar el razonamiento con un presupuesto de tokens](#limitar-el-razonamiento-con-un-presupuesto-de-tokens)'); }); });