Skip to content

feat: providers — any model from any provider in every mode (LLM bridge) - #22

Open
Its-My-Work wants to merge 23 commits into
mainfrom
feat/multi-provider
Open

feat: providers — any model from any provider in every mode (LLM bridge)#22
Its-My-Work wants to merge 23 commits into
mainfrom
feat/multi-provider

Conversation

@Its-My-Work

@Its-My-Work Its-My-Work commented Sep 24, 2026

Copy link
Copy Markdown
Owner

Summary

The studio could reach exactly one endpoint (ANTHROPIC_BASE_URL at boot) and offered four hardcoded Claude aliases. This adds a provider registry: every run — chat, multi-agent, rooms, @@bots, Kanban tasks, schedules, Telegram, utility calls — picks its own provider and model. Every run is still the claude CLI (tools, MCP, hooks, resume, compaction unchanged); a new LLM bridge translates for OpenAI-compatible endpoints. Design and operating notes: docs/providers.md.

  • Provider types: Claude CLI login (subscription), Anthropic API, Anthropic-compatible endpoints (gateways such as kilo-gateway, DeepSeek/Kimi/GLM/MiniMax), OpenAI-compatible endpoints (OpenAI, OpenRouter, Gemini, Qwen, Groq, Mistral, xAI, Ollama, LM Studio…), with presets.
  • Model = one string (provider::model; bare values resolve on the default provider), so it travels through every existing column, WS frame, MCP tool and plan file — no schema change to existing tables.
  • Routing has one choke point: ClaudeCLI.send() asks resolveRunTarget() for the model id, the child's env and extra flags; /api/translate asks the same hook.
  • llm-bridge/ — a separate node process on 127.0.0.1 (spawned like the MCP helpers, restarted on crash on the same port). Passthrough for Anthropic-type providers, Anthropic Messages ⇄ Chat Completions translation for OpenAI-compatible ones (tools, tool results with images, reasoning echo, schema sanitising, effort per dialect, token clamps, keep-alive pings, stop reasons, cached-token usage, error canonicalisation the CLI understands), usage ledger.
  • Keys never enter the agent's environment: the CLI gets a short-lived run token for the loopback bridge; keys are encrypted at rest with the SSH-host key and never returned to the browser.
  • UI: Providers sidebar section + editor (test before save, catalogue import, capabilities/prices, probe, utility model, 30-day usage); a ⋯ model picker next to the toolbar chips; bot editor, Kanban and Schedule list every provider's models; Telegram /model and /effort. All strings in uk/en/ru/fr/he.
  • Migration: on first start ANTHROPIC_BASE_URL (+ token, ANTHROPIC_DEFAULT_*_MODEL, ANTHROPIC_CUSTOM_HEADERS) becomes the default provider gateway; a bare sonnet in an existing row keeps landing on the same endpoint and model. Verified end to end: the gateway receives the same auth header, betas, query and model as before.
  • Fixed on the way: the Subscription engine passes --effort and honours a bot's model on tasks, and its spawn script unsets provider vars tmux could hand the pane; /api/translate no longer inherits server secrets; set_ui_state knows fable; Dispatch and Telegram send the chat's effort; a routing refusal stops the turn instead of burning auto-continues.
  • Pinned CLI: Dockerfile CLAUDE_CODE_VERSION=2.1.281; CI installs the same version so test/llm-bridge-contract.test.js drives the real binary through the bridge.

Not covered: SSH projects still run on the remote host's own config; the tmux Subscription engine stays Claude-login only (a non-Claude model runs on the API engine and says so); external agents keep their own CLIs.

Test plan

  • npm test suites run file by file: all pass except mcp-web.test.js and path-guard.test.js, which fail identically on main in this environment (DNS / host path assumptions).
  • New suites: providers.test.js (94, on both node:sqlite and better-sqlite3), llm-bridge-translate (185), llm-bridge-server (77), llm-bridge-host (35), llm-bridge-contract (23, real claude 2.1.281: read → edit → finish through the bridge against a fake OpenAI endpoint).
  • Manual, in a browser against a real server: chat on an OpenAI-compatible model through the real CLI and the bridge (system prompt, tools, reasoning_effort, max_completion_tokens, usage + cost recorded); chat on the default gateway through passthrough (same Authorization, anthropic-beta, ?beta=true, model as before); provider create/test/catalogue/probe; picker, engine and effort dials.
  • Before deploying: back up the chat-data volume (the first start creates the provider tables and seeds gateway from the env).

🤖 Generated with Claude Code

Its-My-Work and others added 23 commits September 24, 2026 08:49
@Its-My-Work @claude
feat(providers): model refs, resolution, run env, SQLite store, catal…
1c057d0 
…ogue fetch

A model is one string 'provider::model' (bare values keep resolving on the
default provider), so it travels through every existing column and frame.
Store: providers/provider_models/llm_usage tables, keys encrypted with the
SSH-host AES-GCM key, env ANTHROPIC_BASE_URL seeded as the default provider.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Its-My-Work @claude
feat(providers): route every CLI run through the registry
47b097e 
ClaudeCLI.send() asks one hook where a run goes (model id, ANTHROPIC_* for the
child, extra flags), so chat, bots, rooms, multi-agent, dispatch, Kanban and the
utility calls cannot forget to route. /api/translate asks the same hook and no
longer inherits server secrets. SSH strips the provider prefix; the tmux engine
takes claude:: refs, now passes --effort, and unsets every provider variable in
the spawn script (tmux hands panes its SERVER's env). A subscription-engine run
whose model belongs to another provider runs headless and says so. Task runs on
the subscription engine now honour the bot's model, as the card always claimed.
Result meta takes cost/window from the bridge ledger off Anthropic. REST API for
providers, models, probes, choices and usage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Its-My-Work @claude
feat(providers): accept provider refs wherever a model is chosen
89832cd 
chat-defaults takes 'provider::model' besides the four aliases; a bot saved
against an unknown/disabled provider is refused; create_task/create_chain take
any configured model (listed to the agent via env, re-validated by the
endpoint); set_ui_state gains fable and refs. A provider's rejected key gets its
own auth kind pointing at Providers instead of 'claude login'.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Its-My-Work @claude
feat(llm-bridge): Anthropic<->OpenAI bridge for the claude CLI
ee16b1e 
A self-contained package the studio can run as a supervised child: it
authenticates a per-run token, looks up the run's provider/model/effort,
and either passes Anthropic requests through (credential swap + usage tap)
or translates them to OpenAI Chat Completions with per-provider dialects.

Includes the pure translation suite (a real captured Claude Code 2.1.281
request is the golden input) and the end-to-end contract test that drives
the real claude CLI through the bridge against a fake provider.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Its-My-Work @claude
test(llm-bridge): server and host suites; guard per-request failures
5dbc389 
- test/llm-bridge-server.test.js boots createBridgeServer against fake
  upstreams: auth, byte-identical passthrough with credential swap and usage
  tap, the full error mapping table, retry before the first client byte and
  never after, keep-alive pings, idle watchdog, client abort, per-provider
  FIFO concurrency, count_tokens, /v1/models, /health, usage records, log.
- test/llm-bridge-host.test.js drives the real child over IPC: immediate
  requests after registerRun, SIGKILL -> restart on the same port with runs
  re-sent, releaseRun grace, updateProviders, stop(), inProcess, give-up.
- A translator exception on one odd chunk now fails that request instead of
  escaping as an uncaught exception in the child.
- listen() refuses non-loopback hosts; fallbackModel also applies to
  anthropic-compatible providers for unknown Claude-looking ids.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Its-My-Work @claude
feat(providers): providers UI — sidebar section, editor, model picker…
e310656 
… everywhere in the chat

Providers section (same pattern as SSH hosts / agents): status dot from the last
test and the last live request, default star, test, edit, delete, utility-model
select and 30-day usage. Editor with General / Models / Roles / Advanced tabs:
presets, draft test before save, key never pre-filled (blank keeps), catalogue
import with bulk on/off, per-model capability and price corrections, probe.
Toolbar: the four alias chips stay; a fifth '⋯' opens a searchable picker of every
provider's models and names the chosen one. Subscription is off for a model of
another provider, effort for a model that cannot reason. Bot editor lists every
provider's models. Dispatch sends effort. A routing refusal stops the turn at once
(auth-errors kind provider_unavailable) instead of burning the auto-continues.
Strings in all five languages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Its-My-Work @claude
feat(providers): Kanban and Schedule offer every provider's model
b77ed6e 
The four alias buttons stay; an 'Other models…' select next to them lists every
provider's models and carries its choice as a hidden .sc-btn.on, so every reader
of the segment keeps working unchanged. Cards show 'Provider · model' for a ref.
The subscription task path now receives effort (pin updated on purpose).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Its-My-Work @claude
fix(llm-bridge): provider quirks found in review
1570adc 
- OpenRouter's mid-stream failure chunk carries both `error` and a choice
  with finish_reason "error"; it was translated as a clean end_turn. Either
  signal is now an error event (or a real HTTP status for non-stream).
- Passthrough drops thinking blocks signed by this bridge: a session that
  ran on a translated provider and resumes on Anthropic would otherwise 400
  on "invalid signature" forever. Real Anthropic signatures are kept.
- New dialect field sendUser (off for mistral/gemini); mistral also stops
  sending stream_options — its validator 422s on unknown fields.
- The child reports a synchronous startup failure as `fatal` instead of
  leaving the host waiting for its ready timeout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Its-My-Work @claude
feat(llm-bridge): RunCtx.effort 'auto' = provider default
fcb9e43 
Measured on Claude Code 2.1.281: the CLI sends output_config.effort "high"
even when no --effort flag is given, so effort:null in the run context always
ended up forwarding "high". 'auto' now means "send no effort parameter",
whatever the request carries. The documented values behave as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Its-My-Work @claude
feat(providers): Telegram /model and /effort
da7a982 
A chat's model and effort could only be changed from the web UI. /model lists
providers, then their models (indexes in callback_data — a ref can outgrow the
64-byte limit), and writes sessions.model; /effort writes sessions.effort. /info
names the model. Help and strings in all five languages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Its-My-Work @claude
Merge llm-bridge package
17e07cf 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Its-My-Work @claude
feat(providers): wire the bridge in; pin the CLI; contract test in CI
1fa9eda 
- 'Auto' effort reaches the bridge as 'auto' (send none): measured on 2.1.281, the
  CLI sends effort 'high' by itself when no flag is given. CLAUDE_CODE_ALWAYS_ENABLE_EFFORT
  is not needed — the CLI sends output_config.effort for unknown ids too.
- The bridge child gets a minimal env (keys arrive over IPC).
- llm-bridge/** unpacked from the asar for the desktop build.
- Dockerfile pins @anthropic-ai/claude-code@2.1.281 (ARG); CI installs the same
  version so test/llm-bridge-contract.test.js runs instead of self-skipping.
- providers + llm-bridge suites join npm test; two harnesses updated for the new
  engine plumbing (room-run injects engineForModel, interrupt-idle-config reads
  the engineDial write).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Its-My-Work @claude
docs(providers): docs/providers.md, CLAUDE.md conventions, changelog;…
52b675e 
… hints updated

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Its-My-Work @claude
fix(providers): review findings
59ec35f 
- Kanban tasks now see a routing refusal (the run's error text joins the stop and
  overload checks) instead of burning auto-continues; a bridge that is RESTARTING is
  a transient back-off (overloaded_error), one that gave up is a stop.
- Upgrade keeps the env's ANTHROPIC_DEFAULT_*_MODEL / SMALL_FAST / SUBAGENT remaps
  (as roles, new 'fable' role) and ANTHROPIC_CUSTOM_HEADERS (as headers); removing
  ANTHROPIC_BASE_URL switches the env row off and hands the default back, putting
  it back re-enables it; an untouched Save no longer detaches a row from .env.
- A Claude model behind a Claude gateway keeps WebSearch.
- /api/providers-test reuses a stored key only for the stored endpoint.
- Cleared provider options are removed; changing a provider's API type is stored.
- Bridge child env: CA bundle, proxies, Windows system vars.
- Routed runs unset Bedrock/Vertex/Foundry switches and CLAUDE_CODE_OAUTH_TOKEN.
- Live runs get rotated keys (updateProviders with the real map); a synchronous
  throw in send() releases the run token; direct fallback sends both headers and
  the provider's custom headers; a bare unknown id is not a valid agent model; a
  fallback keeps an alias's tier; bot refs validated by the ref grammar; Telegram
  bots follow the effective engine; room/bot tmux seats get effort; fallback
  notices reach every mode; usage pruned daily; Kanban shows 'Subscription' only
  for a Claude model; the toolbar restores Subscription after a forced switch;
  SPA state read during boot is var (no TDZ).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Its-My-Work @claude
feat(ui): one model button instead of four chips + '⋯'
47bc5cd 
The toolbar's model group showed four alias chips next to a '⋯' that named any
other model — so 'Sonnet' on the CLI login and 'Sonnet' on the default gateway
appeared side by side and looked the same. It is now a single button naming the
chat's current model (plus a provider tag when it is not the default provider's);
a click opens the existing picker, default provider first. Same on the mobile sheet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Its-My-Work @claude
feat(providers): the model's provider decides the engine; no engine dial
4e6f327 
There is no API/Subscription switch any more — not in the toolbar (nor its ★
default), the mobile sheet, Kanban, Schedule or the bot editor. A model of the
CLI-login provider runs on the tmux Subscription engine when tmux exists; every
other provider (an Anthropic key, a gateway, OpenAI-compatible) runs headless.
One rule for chats, tasks, bots and Telegram: server.js runEngineFor() over
providers.engineForModel().

- Every list marks each provider "subscription" or "API" (toolbar picker,
  Providers sidebar, bot editor, Kanban/Schedule optgroups, Telegram /model).
- Bot editor: one Model button that borrows the toolbar picker, with "Same as
  chat" on top; the starter-prompt templates are removed (they overwrote the
  bot's own prompt). The picker now fits the viewport and widens to its anchor.
- One-time boot migration (engineDialMigrated): an explicit "Subscription" on a
  session/task/bot becomes a claude::<model> pin (providers.toClaudeRef);
  tasks/bots.run_engine are cleared. Older clients and export files that still
  send runEngine:"subscription" get the same conversion on save/import.
- Removed: the defaultEngine setting, PUT /api/default-engine, botsLogic.botEngine
  and the dead i18n keys. Anthropic preset renamed "Claude API (Anthropic key)".
- Tests: engine-by-model and toClaudeRef in providers.test.js; room-run,
  kanban-run-badges, interrupt-idle-config, bots(-api), config-resolve updated;
  render/bot-engine.test.mjs -> bot-model.test.mjs for the new editor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Its-My-Work @claude
test: pin an API default provider where runs go through a fake headle…
c1530f8 
…ss claude

With no gateway the default provider is the CLI login, and since the engine now
follows the model's provider, its models run on the tmux Subscription engine
whenever tmux exists. CI has tmux and no ANTHROPIC_BASE_URL, so the suites that
drive chats/tasks through a fake headless `claude` (running-sessions,
session-liveness, session-restart, subscribe-nocatchup,
task-blocked-and-empty-response, ssh-secret) ran on the wrong engine there —
and passed on a dev box whose shell exports ANTHROPIC_BASE_URL. They now spread
test/_api-provider-env.js (a gateway default that is never contacted) into the
server env. Reproduced and verified with ANTHROPIC_BASE_URL unset and a tmux
stand-in on PATH. CLAUDE.md states the rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Its-My-Work @claude
fix(ui): plan review card no longer spreads out when restored from hi…
39efb19 
…story

A plan_review message restored from the DB renders its card inside
.mw.assistant .msg, which is white-space: pre-wrap for plain text. The card
inherited it, so the newlines and indentation of its HTML template showed up
as large blank gaps. Reset white-space on the card (and on the agent team
card, which is restored the same way).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Its-My-Work @claude
ui: hide Remote Access and Telegram sections from the sidebar
fd36eeb 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Its-My-Work @claude
ui: keep the ".." row in the file panel for an empty folder
da6ebe7 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Its-My-Work @claude
ui: tidy the project switcher dropdown, remove the GitHub header link
74b6313 
Compact inset rows, path aligned under the name, a single check mark on the
active project (it had both a CSS dot and a JS dot). The upstream GitHub link
is dropped from the chat, Kanban and Schedule headers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Its-My-Work @claude
fix(compose): mount hh from projects/main/hh, not the nonexistent pro…
becc4dc 
…jects/hh/hh

Docker auto-created an empty root-owned dir at the old path, so the hh
workspace was empty and git init failed with Permission denied.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VHRcmYMJg7Dx2Rhh8FXeZn
@Its-My-Work
Merge pull request #23 from Its-My-Work/fix/hh-workspace-mount
069c8b7 
fix(compose): mount hh workspace from projects/main/hh
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@Its-My-Work