diff --git a/src/content/docs-es/models.mdx b/src/content/docs-es/models.mdx index 28ece00..913a26a 100644 --- a/src/content/docs-es/models.mdx +++ b/src/content/docs-es/models.mdx @@ -50,7 +50,7 @@ OpenAI y la misma `base URL`. tag="305B MoE" leftLabel="generación de texto, chat y visión" rightLabel="capacidades" - description={'Modelo MoE de 305B parámetros, servido en su variante Vision-Exp: acepta imágenes de entrada. Contexto de 1M tokens. Tool calling y razonamiento. Cuota de 3B tokens al mes por miembro. Salida estructurada: response_format json_object es compatible (en api.nan.builders el prompt debe contener la palabra JSON; si no, se rechaza con un 400). json_schema se acepta en peticiones a /chat/completions sin streaming y sin tools: la respuesta se comprueba contra tu esquema y, si no lo cumple, normalmente la vuelve a servir un proveedor que lo impone; una respuesta que no se pudo comprobar, o un fallo que no se pudo reemplazar a tiempo, se devuelve tal cual (valídala si tu código depende de ella). En api.nan.builders el resto de peticiones json_schema se rechazan con un 400: con streaming, con tools, a /responses, con n mayor que 1, con un esquema de más de 16 KB, que no sea un JSON Schema válido o que el gateway no valida (un $ref fuera del esquema, $dynamicRef o $anchor, un $id anidado o demasiado grande al resolver sus $ref); en api.helmcode.com se aceptan. json_object solo garantiza JSON sintácticamente válido, no su forma: describe el esquema en el prompt. Para salida restringida a un esquema en esas otras peticiones, usa qwen3.6 o gemma4, donde json_schema con strict restringe campos, tipos y claves obligatorias; o, en este modelo, fuerza una única herramienta de tipo función cuyo parameters sea tu esquema (ver Ejemplos). Con el razonamiento activado, un presupuesto de salida por debajo de 16384 se sube a 16384 para que quepa el razonamiento (max_tokens y max_completion_tokens en /chat/completions, max_output_tokens en /responses, igual en las dos rutas), así que un prompt a menos de 16384 tokens de la ventana de 1.048.576 se rechaza con un 400 aunque pidas un presupuesto pequeño (normalmente Context length exceeded, a veces un Invalid request genérico). Con el razonamiento apagado (reasoning_effort "none" o cualquiera de sus alias) tu presupuesto se usa tal cual. Un presupuesto por encima de 16384 nunca se baja.'} + description={'Modelo MoE de 305B parámetros, servido en su variante Vision-Exp: acepta imágenes de entrada. Contexto de 1M tokens. Tool calling y razonamiento. Cuota de 3B tokens al mes por miembro. Salida estructurada: response_format json_object es compatible (en api.nan.builders el prompt debe contener la palabra JSON; si no, se rechaza con un 400). json_schema se acepta en peticiones a /chat/completions sin streaming y sin tools: la respuesta se comprueba contra tu esquema y, si no lo cumple, normalmente la vuelve a servir un proveedor que lo impone; una respuesta que no se pudo comprobar, o un fallo que no se pudo reemplazar a tiempo, se devuelve tal cual (valídala si tu código depende de ella). En api.nan.builders el resto de peticiones json_schema se rechazan con un 400: con streaming, con tools, a /responses, con n mayor que 1, con un esquema de más de 16 KB, que no sea un JSON Schema válido o que el gateway no valida (un $ref fuera del esquema, $dynamicRef o $anchor, un $id anidado o demasiado grande al resolver sus $ref); en api.helmcode.com se aceptan. json_object solo garantiza JSON sintácticamente válido, no su forma: describe el esquema en el prompt. Para salida restringida a un esquema en esas otras peticiones, usa qwen3.6 o gemma4, donde json_schema con strict restringe campos, tipos y claves obligatorias; o, en este modelo, fuerza una única herramienta de tipo función cuyo parameters sea tu esquema (ver Ejemplos). Tu presupuesto de salida se usa exactamente como lo mandás en las dos rutas (max_tokens y max_completion_tokens en /chat/completions, max_output_tokens en /responses): nunca lo subimos. El razonamiento se factura contra ese mismo presupuesto, así que un presupuesto chico con un prompt que exige deliberar puede gastarse entero pensando, y la respuesta llega como HTTP 200 con finish_reason "length" y contenido vacío. Detectalo con finish_reason "length" Y contenido vacío Y usage.completion_tokens_details.reasoning_tokens igual a usage.completion_tokens; las llamadas sin streaming además traen el header x-nan-truncated-in-reasoning: true. Subí el presupuesto, o usá streaming.'} specs={[ { label: 'Tipo', value: 'MoE (305B total)' }, { label: 'Cuantización', value: 'FP8' }, diff --git a/src/content/docs/models.mdx b/src/content/docs/models.mdx index 693cc09..0a42cfb 100644 --- a/src/content/docs/models.mdx +++ b/src/content/docs/models.mdx @@ -50,7 +50,7 @@ with the same `base URL`. tag="305B MoE" leftLabel="text generation, chat & vision" rightLabel="capabilities" - description={'305B parameter MoE model, served as the Vision-Exp variant: it takes images as input. 1M token context. Tool calling and reasoning. 3B token monthly quota per member. Structured output: response_format json_object is supported (on api.nan.builders the prompt must contain the word JSON, otherwise it is rejected with a 400). json_schema is accepted on non-streamed /chat/completions requests without tools: the answer is checked against your schema and, when it does not match, usually re-served by a provider that enforces it; an answer that could not be checked, or a miss the re-serve could not replace in time, is returned as is (validate it if your code depends on it). On api.nan.builders other json_schema requests are rejected with a 400: streamed, with tools, on /responses, with n above 1, with a schema over 16 KB, not a valid JSON Schema, or one the gateway will not validate (a $ref outside the schema, $dynamicRef or $anchor, a nested $id, or too large once its $refs are inlined); on api.helmcode.com they are accepted. json_object guarantees syntactically valid JSON only, not its shape: describe the schema in the prompt. For schema-constrained output in those other requests, use qwen3.6 or gemma4, where json_schema with strict constrains fields, types and required keys; or, on this model, force one function tool whose parameters is your schema (see Examples). While reasoning is on, an output budget below 16384 is raised to 16384 so the reasoning fits (max_tokens and max_completion_tokens on /chat/completions, max_output_tokens on /responses, the same on both routes), so a prompt within 16384 tokens of the 1,048,576-token window is rejected with a 400 even with a small budget (usually Context length exceeded, sometimes a generic Invalid request). With reasoning off (reasoning_effort "none" or any of its aliases) your budget is used as sent. A budget above 16384 is never lowered.'} + description={'305B parameter MoE model, served as the Vision-Exp variant: it takes images as input. 1M token context. Tool calling and reasoning. 3B token monthly quota per member. Structured output: response_format json_object is supported (on api.nan.builders the prompt must contain the word JSON, otherwise it is rejected with a 400). json_schema is accepted on non-streamed /chat/completions requests without tools: the answer is checked against your schema and, when it does not match, usually re-served by a provider that enforces it; an answer that could not be checked, or a miss the re-serve could not replace in time, is returned as is (validate it if your code depends on it). On api.nan.builders other json_schema requests are rejected with a 400: streamed, with tools, on /responses, with n above 1, with a schema over 16 KB, not a valid JSON Schema, or one the gateway will not validate (a $ref outside the schema, $dynamicRef or $anchor, a nested $id, or too large once its $refs are inlined); on api.helmcode.com they are accepted. json_object guarantees syntactically valid JSON only, not its shape: describe the schema in the prompt. For schema-constrained output in those other requests, use qwen3.6 or gemma4, where json_schema with strict constrains fields, types and required keys; or, on this model, force one function tool whose parameters is your schema (see Examples). Your output budget is used exactly as sent on every route (max_tokens and max_completion_tokens on /chat/completions, max_output_tokens on /responses): we never raise it. Reasoning is billed against that same budget, so a small budget on a prompt that needs deliberation can be spent entirely thinking, and the answer then arrives as HTTP 200 with finish_reason "length" and empty content. Detect it with finish_reason "length" AND empty content AND usage.completion_tokens_details.reasoning_tokens equal to usage.completion_tokens; non-streamed calls also carry the response header x-nan-truncated-in-reasoning: true. Raise the budget, or stream.'} specs={[ { label: 'Type', value: 'MoE (305B total)' }, { label: 'Quantization', value: 'FP8' }, diff --git a/src/data/openapi.json b/src/data/openapi.json index cd890eb..ab19ea4 100644 --- a/src/data/openapi.json +++ b/src/data/openapi.json @@ -200,7 +200,7 @@ "max_tokens": { "type": "integer", "minimum": 1, - "description": "Maximum number of tokens to generate. A value above what the model can generate is rejected with a generic `400`. A request that does not fit the model's context window is rejected with a `400`: \"Context length exceeded for model '...'\" on `qwen3.6` and `gemma4`, usually the same text (sometimes the generic \"Invalid request\") on the other models. On `deepseek-v4-flash`, while reasoning is on, a smaller value is raised to 16384 so the reasoning fits, so a prompt within 16384 tokens of its 1,048,576-token window is rejected with a `400` even with a small `max_tokens`; with reasoning off (`reasoning_effort: \"none\"` or an alias) the value is used as sent.", + "description": "Maximum number of tokens to generate. A value above what the model can generate is rejected with a generic `400`. A request that does not fit the model's context window is rejected with a `400`: \"Context length exceeded for model '...'\" on `qwen3.6` and `gemma4`, usually the same text (sometimes the generic \"Invalid request\") on the other models. The value is used exactly as sent on every model: it is never raised. Reasoning is billed against this same budget, so a small value on a prompt that needs deliberation can be spent entirely thinking, and the answer then arrives as HTTP 200 with `finish_reason` `\"length\"` and empty content; non-streamed calls carry the `x-nan-truncated-in-reasoning: true` response header. Raise the budget, or stream.", "example": 512 }, "temperature": { @@ -1087,7 +1087,7 @@ "max_output_tokens": { "type": "integer", "minimum": 1, - "description": "The maximum number of output tokens. Default 65536 on `qwen3.6`." + "description": "The maximum number of output tokens. Used exactly as sent: it is never raised. Reasoning is billed against this same budget, so the answer can come back as 200 with `finish_reason` `\"length\"` and empty content; non-streamed calls then carry the `x-nan-truncated-in-reasoning: true` response header. Default 65536 on `qwen3.6`." }, "reasoning": { "type": "object", diff --git a/src/lib/openapiSpec.test.ts b/src/lib/openapiSpec.test.ts index 674e8f1..9e2d352 100644 --- a/src/lib/openapiSpec.test.ts +++ b/src/lib/openapiSpec.test.ts @@ -917,9 +917,10 @@ describe('documented 400s: tool parameters and context overflow', () => { it('explains the deepseek-v4-flash 16384 floor on max_tokens, in the spec and on both cards', () => { const maxTokens = (spec.paths as any)['/chat/completions'].post.requestBody.content['application/json'] .schema.properties.max_tokens.description; - expect(maxTokens).toContain('On `deepseek-v4-flash`, while reasoning is on, a smaller value is raised to 16384'); - expect(maxTokens).toContain('with reasoning off (`reasoning_effort: "none"` or an alias) the value is used as sent'); - expect(maxTokens).toContain('1,048,576-token window'); + expect(maxTokens).toContain('The value is used exactly as sent on every model: it is never raised.'); + expect(maxTokens).toContain('Reasoning is billed against this same budget'); + expect(maxTokens).toContain('x-nan-truncated-in-reasoning: true'); + expect(maxTokens).not.toContain('raised to 16384'); // Not promised as the exact text on deepseek-v4-flash: measured generic there. expect(maxTokens).not.toMatch(/deepseek-v4-flash[^.]*Context length exceeded/); expect(maxTokens).toMatch(/above what the model can generate is rejected with a generic `400`/); @@ -932,14 +933,12 @@ describe('documented 400s: tool parameters and context overflow', () => { const start = page.indexOf('id="deepseek-v4-flash"'); return page.slice(start, page.indexOf('/>', start)); }; - expect(card('')).toContain('While reasoning is on, an output budget below 16384 is raised to 16384 so the reasoning fits'); - expect(card('')).toContain('max_output_tokens on /responses, the same on both routes'); - expect(card('')).toContain('rejected with a 400 even with a small budget (usually Context length exceeded, sometimes a generic Invalid request)'); - expect(card('')).toContain('A budget above 16384 is never lowered.'); - expect(card('-es')).toContain('Con el razonamiento activado, un presupuesto de salida por debajo de 16384 se sube a 16384'); - expect(card('-es')).toContain('max_output_tokens en /responses, igual en las dos rutas'); - expect(card('-es')).toContain('se rechaza con un 400 aunque pidas un presupuesto pequeño (normalmente Context length exceeded, a veces un Invalid request genérico)'); - expect(card('-es')).toContain('Un presupuesto por encima de 16384 nunca se baja.'); + expect(card('')).toContain('Your output budget is used exactly as sent on every route'); + expect(card('')).toContain('non-streamed calls also carry the response header x-nan-truncated-in-reasoning: true'); + expect(card('')).not.toContain('raised to 16384'); + expect(card('-es')).toContain('Tu presupuesto de salida se usa exactamente como lo mandás en las dos rutas'); + expect(card('-es')).toContain('las llamadas sin streaming además traen el header x-nan-truncated-in-reasoning: true'); + expect(card('-es')).not.toContain('se sube a 16384'); }); }); diff --git a/src/tests/lib/docsReasoning.test.ts b/src/tests/lib/docsReasoning.test.ts index b9fbd4a..668bfbf 100644 --- a/src/tests/lib/docsReasoning.test.ts +++ b/src/tests/lib/docsReasoning.test.ts @@ -98,10 +98,12 @@ describe('API reference: one documented way to turn reasoning off', () => { } }); - it('max_tokens: the deepseek floor applies only while reasoning is on', () => { + it('max_tokens: the budget is used as sent, and reasoning counts against it', () => { expect(chatBody.max_tokens.description).toContain( - 'On `deepseek-v4-flash`, while reasoning is on, a smaller value is raised to 16384 so the reasoning fits, so a prompt within 16384 tokens of its 1,048,576-token window is rejected with a `400` even with a small `max_tokens`; with reasoning off (`reasoning_effort: "none"` or an alias) the value is used as sent.', + 'The value is used exactly as sent on every model: it is never raised.', ); + expect(chatBody.max_tokens.description).toContain('x-nan-truncated-in-reasoning: true'); + expect(chatBody.max_tokens.description).not.toContain('raised to 16384'); }); it('lists the accepted aliases and says the explicit value wins', () => { @@ -277,14 +279,16 @@ describe('deepseek-v4-flash card: the max_tokens floor only applies while reason it('en', () => { expect(card('')).toContain( - 'While reasoning is on, an output budget below 16384 is raised to 16384 so the reasoning fits (max_tokens and max_completion_tokens on /chat/completions, max_output_tokens on /responses, the same on both routes), so a prompt within 16384 tokens of the 1,048,576-token window is rejected with a 400 even with a small budget (usually Context length exceeded, sometimes a generic Invalid request). With reasoning off (reasoning_effort "none" or any of its aliases) your budget is used as sent. A budget above 16384 is never lowered.', + 'Your output budget is used exactly as sent on every route (max_tokens and max_completion_tokens on /chat/completions, max_output_tokens on /responses): we never raise it.', ); + expect(card('')).toContain('non-streamed calls also carry the response header x-nan-truncated-in-reasoning: true'); + expect(card('')).not.toContain('raised to 16384'); }); it('es', () => { - expect(card('-es')).toContain('Con el razonamiento activado, un presupuesto de salida por debajo de 16384 se sube a 16384'); - expect(card('-es')).toContain('Con el razonamiento apagado (reasoning_effort "none" o cualquiera de sus alias) tu presupuesto se usa tal cual.'); - expect(card('-es')).toContain('Un presupuesto por encima de 16384 nunca se baja.'); + expect(card('-es')).toContain('Tu presupuesto de salida se usa exactamente como lo mandás en las dos rutas'); + expect(card('-es')).toContain('las llamadas sin streaming además traen el header x-nan-truncated-in-reasoning: true'); + expect(card('-es')).not.toContain('se sube a 16384'); }); });