You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
inference-router: substituted model names leak into the client's session state and fail to restore #1597
When the inference-router sends a request to a server that refuses the agent's model, it resends the request with the server's substitute (chooseModel: the server's main for a request with tools, helper without). The server then answers with model: naming the substitute it served — and at least Claude Code persists that name as the session's model.
On the conversation's next restart, the client cannot restore it:
⏺ Session model rits/zai-org/glm-5-3 could not be restored (not a model this
version of Claude Code recognizes) — using opus instead.
Live trace, 2026-10-29 (laptop install, agent routed to a GLM server behind LiteLLM):
Session 1439e16a: requests carried requestedModel: claude-opus-5-5, sent model rits/zai-org/glm-5-3 (router substitution on glm); every response reported model: rits/zai-org/glm-5-3.
The conversation continued into a new session id (ca3c9830) at 22:01; Claude Code failed to restore rits/zai-org/glm-5-3 and silently fell back to opus.
Impact
Every restart of a continued conversation whose server substitutes models shows the restore error and falls back to the default model. Substitution is the default behavior for any server that does not serve the agent's own model names, so this hits every routed agent on such a server.
The substitute name only exists on the wire because the router rewrote the request; the server answered honestly with what it served. The leak is ours.
Compounds the already-documented continued-in behavior: the new session id is judged a continuation and not routed, so the conversation lands on the terminal's own base URL and on the default model.
Fix options (decision needed, not presuming one)
Rewrite the response's model field back to the name the client asked for. This is a WritesResponseBody, which is the SSE streaming predicate — declaring it would force the buffered relay path for all traffic. Probably too expensive for a cosmetic field.
Accept and document — the client's fallback already works (it uses opus); the cost is the error banner and the model reset. Doc note in docs/agents/claude-code.md goes with this issue.
Something cheaper than 1 that still stops the name reaching client session state — none identified yet.
Closing without action. Fix options assessed: rewriting the response's model field needs WritesResponseBody, which forces the buffered relay path for all streaming traffic; a streaming-frame rewrite would need a new framework capability (frames are read-only today). Neither is worth it for this. The client's opus fallback works and is visible, so leaving this as accepted behavior.
Activity
huang195 commentedon Oct 30, 2026
Doc note: #1598
huang195 commentedon Oct 30, 2026
Closing without action. Fix options assessed: rewriting the response's
modelfield needsWritesResponseBody, which forces the buffered relay path for all streaming traffic; a streaming-frame rewrite would need a new framework capability (frames are read-only today). Neither is worth it for this. The client's opus fallback works and is visible, so leaving this as accepted behavior.