Repository navigation
litellm fork: emit OpenAI error code context_length_exceeded on context-window 400s #119
Description
Context
nan-devops#629 makes community return a clear 400 on context-window overflow, with no fallback. Message-based detection works: opencode's patterns and litellm's own detector are tested against every variant.
What's still missing is the OpenAI error code: "context_length_exceeded", which clients that only read code rely on (OpenAI SDK users, Codex-style clients). It can't be set from the usage hook, in litellm 1.83.14 or 1.102.1:
ProxyException.__init__stores the HTTP status inself.code(str(code)).to_dict()emitscode, but never emitsopenai_code.openai_exception_handlerderives the HTTP status fromint(exc.code), so a non-numeric code would 500.
Options
- Patch the fork: have
ProxyException.to_dict()emitopenai_codeascodewhen set, keeping the HTTP status separate. - Add a custom exception handler.
Either way, add a test in the real-litellm harness (tests/usage-hook/test_context_window_refusal_real_litellm.py).
🤖 Generated with Claude Code
Metadata
Metadata
Assignees
Labels
No labels
Activity
sre-helmcode commentedon Sep 30, 2026
Reopening: nan-devops#641 is deployed, but the code is not effective on community yet.
code: "context_length_exceeded", streamed and non-streamed. ✅type: invalid_request_error, butcode: "400", streamed and non-streamed. The same happens via api.nan.builders, and the edge Worker is ruled out (the direct pod test gives the same result).litellm.proxy._types.StatusWithOpenAICodeexists, and/etc/litellm/usage_hook.pycontains_cwe_status.Hypothesis: in prod, community registers several callbacks (usage_hook, rate_limit_hook, credits/stream-integrity/…). A later failure hook, or litellm 1.83.14's handler on the multi-callback path, rebuilds or replaces the exception or its status as a plain int, and the marker is lost. The real-litellm harness only registers the usage hook. The next step is a harness with the production callback list from
values.yaml, on 1.83.14.There is no regression: the behaviour is identical to before #641 (clear message,
code "400").