Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions internal/translate/default_max_tokens_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -178,3 +178,22 @@ func TestOpenAISameFormat_ExplicitMaxTokensClampsToKimiK3Ceiling(t *testing.T) {
out := parseAndEmit(t, body, "openai", opts)
assert.Equal(t, float64(32000), out["max_tokens"])
}

// Regression: qwen/qwen3.8-max was absent from modelMaxOutputTokens, so a
// Claude Code turn requesting max_tokens=64000 was clamped to the 8192
// fallback. Qwen3-Max serves 32k output on Fireworks (qwen3p8-max); every
// agentic turn ended in finish_reason=length and an auto-resume loop until
// the client surfaced "response exceeded the 64000 output token maximum".
func TestOpenAISameFormat_ExplicitMaxTokensClampsToQwen38MaxCeiling(t *testing.T) {
Comment on lines +183 to +187

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Claude Code turn requesting max_tokens=64000 was clamped to the 8192
// fallback. Qwen3-Max serves 32k output on Fireworks (qwen3p8-max); every
// agentic turn ended in finish_reason=length and an auto-resume loop until
// the client surfaced "response exceeded the 64000 output token maximum".
func TestOpenAISameFormat_ExplicitMaxTokensClampsToQwen38MaxCeiling(t *testing.T) {
// Regression: absent modelMaxOutputTokens entry clamped max_tokens=64000 to
// the 8192 fallback; every turn truncated (finish_reason=length loop).

Was 5 lines narrating the full incident; the essential WHY fits in two.

body := []byte(`{"model":"gpt-4o","messages":[{"role":"user","content":"hi"}],"max_tokens":64000}`)
opts := translate.EmitOptions{
TargetModel: "qwen/qwen3.8-max",
Capabilities: router.Lookup("qwen/qwen3.8-max"),
}
out := parseAndEmit(t, body, "openai", opts)
assert.Equal(t, float64(32768), out["max_tokens"])

body = []byte(`{"model":"gpt-4o","messages":[{"role":"user","content":"hi"}],"max_tokens":16384}`)
out = parseAndEmit(t, body, "openai", opts)
assert.Equal(t, float64(16384), out["max_tokens"])
}
4 changes: 4 additions & 0 deletions internal/translate/envelope.go
Original file line number Diff line number Diff line change
Expand Up @@ -1201,6 +1201,10 @@ var modelMaxOutputTokens = map[string]int{
// Keyed by full catalog ID, since decision.Model keeps the vendor prefix.
// Other OSS rows are still absent and so inherit the 8192 fallback.
"moonshotai/kimi-k3": 131072,
// Qwen3-Max family serves 32k output on Fireworks (qwen3p8-max). Without
// this entry a Claude Code max_tokens=64000 turn was clamped to the 8192
// fallback and truncated every agentic turn (finish_reason=length loop).
"qwen/qwen3.8-max": 32768,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrong Qwen output-token ceiling

Medium Severity

The new qwen/qwen3.8-max entry in modelMaxOutputTokens sets the ceiling to 32768, but the Qwen3.8-Max model card (and Fireworks listings) document a 131072 output limit. Claude Code’s max_tokens=64000 requests still clamp below the real ceiling, so thinking-heavy turns can keep hitting finish_reason=length and the same truncate→resume loop this change aimed to stop.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 8ff3f7d. Configure here.

}

const defaultMaxOutputTokenCap = 8192
Expand Down
Loading