feat(mcp): custom HTTP headers for bring-your-own MCP servers - #2186
Conversation
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository: raphaeltm/simple-agent-manager/.coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@coderabbitai review |
1 similar comment
|
@coderabbitai review |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…to dedicated files Pure moves, no behaviour change. Prepares for custom MCP headers without growing files already over the rule-18 ceiling: - acp/mcp_servers.go: McpServerEntry + ACP/Amp server builders (from gateway.go, session_host.go) - acp/codex_config.go: Codex config.toml generation and writers (from gateway.go) - acp/vibe_config.go: Vibe config.toml generation and writer (from gateway.go) - server/mcp_servers.go: normalizeMcpServers + registerSessionMcpServers (from workspaces.go) - persistence/session_mcp_servers.go: session_mcp_servers table + CRUD (from store.go) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MCP connections can now carry custom HTTP headers alongside (or instead of)
a bearer token — Composio requires x-api-key / x-consumer-api-key.
Control plane:
- D1 0175 adds header_names (display projection) + encrypted_headers/headers_iv
(AES-GCM sealed [{name,value}] list). Values are never returned; responses
expose headerNames only.
- services/mcp-connection-headers.ts owns the rules: [A-Za-z0-9_-]{1,64} names
(mcp-remote + TOML bare-key safe), case-insensitive uniqueness, transport
headers reserved, Authorization only when authType is none, trimmed values
without control characters, configurable MAX_MCP_CONNECTION_HEADERS and
MCP_CONNECTION_HEADER_VALUE_MAX_BYTES.
- PATCH headers is the full desired set; an entry without a value keeps the
stored value, so one header can be rotated without re-sending the others.
- Resolution re-validates decrypted headers and skips a bad row (rules 41/50);
node-agent sends headers only when present (rule 54).
vm-agent:
- McpServerEntry.Headers, validated in normalizeMcpServers, persisted
(migrateV18) and restored on restart through one pair of converters.
- ACP HTTP header list, Codex env_http_headers with SAM_MCP_<NAME>_HEADER_<i>_SECRET
env vars, Vibe headers table, Amp mcp-remote --header name:${ENV}. Secret
values never reach config.toml (Codex) or argv (Amp).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…agent - API: validation, encryption at rest, never-returned values, PATCH keep/rotate/ remove/clear semantics, bearer vs Authorization conflict, unreadable-ciphertext recovery, rule-50 list tolerance (real SQLite). - Vertical slice: a connection saved through the write path authorizes against a live x-api-key MCP server using only what session resolution injects; corrupt or vm-agent-unsafe rows are skipped without logging secrets. - Shared wire fixture mcp-server-entry-wire.json: node-agent must serialize it exactly; the real vm-agent create-agent-session handler must accept it. - vm-agent: ACP/Amp/Codex/Vibe output (TOML parsed, mcp-remote regex pinned), env var collision + secret classification, boundary validation, full normalize -> SQLite -> restart backfill round trip, migrateV18 upgrade. - Every guard verified discriminating by mutation (15 mutations, all red). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- McpServerForm (add + inline Edit) and McpServerHeadersField replace the inline create form in McpServersManager, which keeps list/toggle/delete. - Headers are name + masked value rows. A saved header shows its name and a blank value that keeps the saved one (the API never returns values); editing likewise keeps the URL and token unless new ones are typed. - Rows list their header names; one form is open at a time. - Payloads are built in mcp-server-form-state.ts: create omits empty headers, update always sends the full desired header set (value-less = keep). - Unit tests drive the real form: Composio-shape create, unsaved-row removal, edit keep/rotate/remove/add, bearer token requirement, rejected save keeps the editor. Playwright audit adds header data, add/edit form scenarios and a measured-coordinate check of the responsive header row. - Docs: MCP servers guide (headers, Composio, editing, Amp note), limits in configuration reference and .env.example, changelog skill entry. - Formatting: prettier on the new files and on the MCP-owned files touched. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Visual audit findings, fixed: - header rows are bordered cards below sm so each value visibly belongs to its name (they read as one undifferentiated stack before) - server row actions wrap under the text on phones instead of squeezing long hosts into a ~24-character column now that there are three of them - the None auth option no longer truncates at 375px - Settings help card: Composio takes an x-api-key header, not only a pre-signed URL Audit: long header-name row and Project Settings -> Runtime scenarios added. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… deploy - The wire-contract test drove the real create-agent-session handler, whose message reporter wrote messages-ws.db* into the package directory, and those files were committed. Point the test's persistence path at a temp dir and remove them (found by the constitution-validator review). - The deploy publishes the web UI before the API Worker, so listMcpConnections defaults a missing headerNames to [] for that window instead of crashing the MCP settings page. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…scope
- gitignore packages/vm-agent/**/*.db{,-shm,-wal} so a test run cannot commit
message-reporter databases again (task-completion + security review).
- ValidateMcpHeaders: say that policy rules (reserved names, duplicates,
Authorization vs bearer, limits) live in the control plane, the only writer.
- Task file: acceptance criteria 1-6 and the Playwright audit ticked with the
tests that prove them; staging stays open.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ames - updateMcpConnection writes only if updated_at still matches the row it read (409 otherwise); nextUpdatedAt keeps the column strictly increasing, so a header-only edit racing a switch to bearer can no longer persist an Authorization header beside a bearer token. Resolution refuses that pair as a backstop. - vm-agent McpServerEntry.ValidateHeaders rejects reserved transport names, case-insensitive duplicates and Authorization beside a bearer token: one repeated key made the whole Codex/Vibe TOML unparseable, sam-mcp included. The reserved list is pinned TS<->Go by headerNames.reserved. - GetSessionMcpServers skips a row whose headers cannot be decoded instead of failing the read, which the restore path treated as no MCP servers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… routes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… front Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main claimed 0175 in #2181, and a sibling branch's 0176 is already in staging's D1 ledger. This file had never been applied anywhere, so it moves to the next free prefix before its first staging deploy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Open sibling branches have already applied 0176 and 0177 to staging, so this migration takes the next prefix no open branch uses, before its first staging deploy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A literal "x-api-key": "SAM_MCP_..._SECRET" pair reads as an API key assignment to Gitleaks; the values are environment variable names. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
f096d7e to
6550382
Compare
|



Summary
Bring-your-own MCP servers could only authenticate with a bearer token or a credential in the URL. Composio's MCP endpoints need a custom API-key header (
x-api-keyfor single-toolkit MCP,x-consumer-api-keyfor Composio Connect), and the public guide listed custom headers as unsupported. This PR lets users manage custom HTTP headers on any MCP server, in both personal settings and project settings, and injects them into every agent harness.authType(none | bearer). Composio isnoneplusx-api-key. A customAuthorizationheader is allowed only withnone, for non-Bearer schemes.^[A-Za-z0-9_-]{1,64}$, which is exactly what the Amp bridgemcp-remote@0.1.38parses in--header name:valueand what TOML accepts as a bare key. Names are unique case-insensitively, and transport-managed names (host,content-length,mcp-session-id, …) are reserved. The rule and the reserved list are pinned between TypeScript and Go bypackages/shared/src/fixtures/mcp-server-name-contract.json.MCP_CONNECTION_HEADER_VALUE_MAX_BYTES(8192) bytes, and at mostMAX_MCP_CONNECTION_HEADERS(10) per server.[{name,value}]list (D10178:encrypted_headers,headers_iv), plus a plaintextheader_namesdisplay projection.headerNamesonly.PATCHtreatsheadersas the full desired set, and an entry without avaluekeeps the stored one. The new inline Edit form can therefore add, remove or rotate one header without the browser ever holding the others. The update only writes ifupdated_atstill matches the row it read (a lost race returns 409), so a header-only edit racing a switch to bearer can never storeAuthorizationbeside a bearer token.McpServerEntry.Headersis additive and sent only when non-empty (rule 54). The vm-agent validates it (ValidateHeaders: charset, reserved names, duplicates,Authorizationbeside a token), persists it (SQLitemigrateV18) and injects it per harness:env_http_headerspointing atSAM_MCP_<NAME>_HEADER_<i>_SECRET, so values never land inconfig.toml.headerstable.mcp-remote --header name:${SAM_MCP_HEADER_<i>}, with values in the stdio env rather than argv.refactor(vm-agent)) only moves code. It extracts the MCP/Codex/Vibe code fromgateway.go,session_host.go,workspaces.goandstore.gointo dedicated files (rule 18), and the go-specialist verified it is behaviour-neutral line by line.Task:
tasks/archive/2026-09-29-mcp-connection-custom-headers.md(SAM task01M3NTSE4PAGHVDJPKKZ0AZ2DH).Validation
pnpm lint—pnpm check:fastexit 0 (format ratchet, oxlint shadow, ESLint workspace, type-boundary ratchet)pnpm typecheck— api / web / sharedtsc --noEmitclean;go vetclean oninternal/acp,internal/persistence,internal/serverpnpm testinternal/acp,internal/persistenceandinternal/serverpass in full.pnpm buildexit 0,quality:migration-safetyexit 0,quality:file-sizesexit 0.updated_atpredicate, the strictly-increasing timestamp, the resolution backstop and both env-limit wirings), the wire tag and the reserved-name contract."x-api-key"with an environment-variable name. It is not a credential.82674ca69removes that literal from the current tree; the test now builds the map in a loop and was mutation-checked.655038210baselines the one historical PR-range occurrence as an exact, expiringsynthetic-test-fixturedigest.Staging Verification (REQUIRED for all code changes — merge-blocking)
Deploy Stagingrun 36558820009 atc1c9084a4: Validate Configuration, Deploy to Cloudflare and smoke-tests all green/dashboard,/projects,/settings/mcp-serversand a project's Runtime settings at 1280 and 375 px: pages load, no horizontal overflow, zero console errors, zero failed requestsStaging Verification Evidence
Staging is shared, so I waited in line behind three other agents' staging verifications and coordinated handoffs by durable message. While waiting I renumbered this PR's D1 migration twice,
0175→0177→0178, before any environment had applied it: main claimed0175(#2181), and open sibling branches had already put0176and0177into staging's ledger. Staging's ledger now shows0178_mcp_connection_headers.sql(id 206) on top of them.Test endpoint. A minimal streamable-HTTP MCP server that answers only when
x-api-keymatches, as Composio does, and otherwise returns 401.probetool returns a per-run nonce and echoes anyx-probe-*headers it received.header-probewith authentication "None" and headerx-api-key.POSTreturned 201 withheaderNames: ["x-api-key"]; the value was not echoed.header_names = ["x-api-key"], a 108-byteencrypted_headersand a 16-byteheaders_iv.instr()finds the key in neither column.01M3PE6VWYS2DWWRRKSHGB6PEEwas created at 11:18:43Z and first heartbeat as healthy at 11:21:19Z. Itsagent_versionisa089e14f…, the last commit on this branch that touchedpackages/vm-agent. The workspace was ready at 11:23:20Z.server/discover→initialize→tools/list→tools/call, and every request was authorized byx-api-key.PROBE RESULT: probe-ok nonce-db0f0b13 extra-headers=[].x-probe-mode = editedand left the savedx-api-keyblank.{"headers":[{"name":"x-api-key"},{"name":"x-probe-mode","value":"edited"}]}, so it never held the stored value.headerNames: ["x-api-key","x-probe-mode"].initialize→tools/list→tools/call, each carrying both headers.PROBE RESULT: probe-ok nonce-db0f0b13 extra-headers=[x-probe-mode=edited]. This shows the value kept through the edit still authorizes, and the Codexenv_http_headerspath works in the container runtime.headerNames: ["x-api-key"], no value echoed) and the server was listed. Deleting it from the mobile view returned 200, and the D1 row is gone./api/nodesreturns[]and D1 has 0 active nodes. The test MCP connections are deleted and the tunnel is stopped. Staging was then handed to the next agent in line.Staging ran
c1c9084a4. Since then:internal/server/workspaces.go,env.tsand.env.example, and none of that changes this PR's code paths.The Go
acp/server/persistencesuites, the TS typechecks, the MCP suites (175/175), migration ordering and the Gitleaks current-tree scan were all re-run green on the rebased head.UI Compliance Checklist (Required for UI changes)
Header N name,<name> value,Remove <name>,Edit <server>,Delete <server>).Add MCP server/Edit <server>).Button,Input,Selectfrom@simple-agent-manager/ui; the app'sConfirmDialogfor delete).apps/web/tests/playwright/mcp-servers-audit.spec.ts, 80/80 pass, withassertNoOverflow(including the clipped-overflow walk) in every scenario.UI Screenshot Evidence
All screenshots below were taken with Playwright by the local audit (
apps/web/tests/playwright/mcp-servers-audit.spec.ts) against mocked API data. The staging screenshots under Staging Verification Evidence were taken with Playwright against app.sammy.party.Surface: Settings → MCP servers — server list with header names
Desktop evidence:

Mobile evidence:

Mock/stress data used: Playwright mock data with long text (a row with five header names, two of them 64 unbroken characters; hosts with 90+ character subdomains), many items (30 servers, every fifth with two header names), special characters (a punycode emoji host, a
<script>host, a Unicode host), and the empty and error (API 500) states.Screenshot quality review: I reviewed every screenshot at both viewports for layout quality, overflow, clipping, readability and responsive behavior. The first audit found three mobile issues, all fixed in
b63461d3fbefore handoff:Long header names now wrap inside the card, and there is no horizontal overflow anywhere.
Surface: MCP server add form — headers editor
Surface: MCP server edit form — inline edit that keeps saved secrets
Surface: Project Settings → Runtime → MCP servers (shared)
http://127.0.0.1:4788), a no-auth server carryingx-api-key, and a disabled server, below the runtime env-var and file sections.Headers: x-api-keyand the Edit/Delete actions. Result: no visual issues found.End-to-End Verification (Required for multi-component changes)
mcp-connection-headers-injection.test.tssaves a connection through the service, resolves it for a session, and calls a live mock MCP server that answers only withx-api-key.mcp-server-entry-wire.jsonis serialized bynode-agent.tsand posted to the real GohandleCreateAgentSessionhandler.mcp-remoteheader regex comes from the 0.1.38 tarball source.env_http_headerscomes from the Codex config reference.isSecretEnvVarsuffix handling was read and tested.Data Flow Trace
McpServerForm→toCreateRequest/toUpdateRequest(apps/web/src/components/mcp-servers/mcp-server-form-state.ts) →createMcpConnection/updateMcpConnection(apps/web/src/lib/api/mcp-connections.ts).userMcpConnectionRoutes/projectMcpConnectionRoutes(apps/api/src/routes/mcp-connections.ts, ValibotCreateMcpConnectionSchema/UpdateMcpConnectionSchema) →createMcpConnection/updateMcpConnection(apps/api/src/services/mcp-connections.ts) →validateMcpConnectionHeaders+sealMcpConnectionHeaders(apps/api/src/services/mcp-connection-headers.ts) → D1mcp_connections.header_names,encrypted_headers,headers_iv.agent-session-bootstrap.ts→buildSessionMcpServers→resolveMcpServersForSession→toEntry→openMcpConnectionHeaders(apps/api/src/services/mcp-connection-resolution.ts) →McpServerEntry.headers.serializeMcpServers(apps/api/src/services/node-agent.ts) →handleCreateAgentSession(packages/vm-agent/internal/server/workspaces.go) →normalizeMcpServers+McpServerEntry.ValidateHeaders(internal/server/mcp_servers.go,internal/acp/mcp_servers.go) →registerSessionMcpServers→UpsertSessionMcpServers(internal/persistence/session_mcp_servers.go).buildAcpMcpServers/buildAmpMcpServer(internal/acp/mcp_servers.go),generateCodexMcpConfig(internal/acp/codex_config.go),generateVibeConfig(internal/acp/vibe_config.go) → the MCP server receives the header.Untested Gaps
Post-Mortem (Required for bug fix PRs)
N/A: not a bug fix (new capability).
Specialist Review Evidence (Required for agent-authored PRs)
needs-human-reviewlabel added and merge deferred to human — N/A, all completed.dbfiles were committed; removed and gitignored. MEDIUM: acceptance criteria were left unticked; ticked with evidence. LOW: reserved-name scope; commented. Fixed in1daf06264,6f2bbec46.bbacb6cff(5 Go mutations red):- HIGH: a repeated header name made the whole Codex/Vibe TOML unparseable. Duplicates, and
- MEDIUM: reserved names are now enforced in Go and pinned by the contract fixture.
- MEDIUM:
Accepted and documented: MEDIUM, Vibe writes header values intoAuthorizationbeside a bearer token, are now rejected.GetSessionMcpServersnow skips one corrupt row instead of dropping every server.~/.vibe/config.toml. This is the existing bearer-token pattern: the directory is 0700 and the file is excluded from snapshots. Declined: LOW, mirroring the operator-configurable limits in Go.updateMcpConnectionderives every field from the row it read, so an edit racing a switch to bearer could persistAuthorizationbeside a bearer token. Fixed inbbacb6cffwith anupdated_atcompare-and-set (409 on a lost race), strictly increasing timestamps, and a resolution-time backstop. All three were mutation-proven under a frozen clock..dbfiles; fixed and gitignored. Info: vm-agent SQLite stores values like the existing token.fd5c1b18d; Playwright 80/80 and screenshots re-reviewed.bbacb6cffand9ae91f735: route-level POST/PATCH tests on real SQLite; env-driven header count/size limits (hardcoding either in the route turns a test red); Vibe TOML escaping; per-row persistence skip; authType switch keeping a non-conflicting header; bearer +x-api-key; update-time limit; UTF-8 byte limit.MCP_CONNECTION_*pattern (optional Worker vars, not synced from GitHub Environment), which is pre-existing for all five.DEFAULT_*constants. LOW: stray artifacts; removed.gateway.go,session_host.goandworkspaces.gostay over 800 lines. They are on thecheck-file-sizesEXEMPT list, and this PR only shrinks them. LOW: consolidate other byte-length helpers ontolib/utf8.ts, tracked as SAM idea01M3P26DV2N1FF59AZ021TX6MQ.01M3PHPH90P58AXT4326PRQ6WE)01M3PHPH90P58AXT4326PRQ6WE)TestShutdown_CancelsSizeFallbackRequestis outside this PR's diff and is byte-identical to main; the exact test passed 10/10 locally under Go 1.26.6, while the affected ACP/persistence/server packages also passed.01M3PHPH90P58AXT4326PRQ6WE)CodeRabbit Review Evidence (Required for agent-authored PRs)
coderabbit-reviewlabel applied after local review, staging, and CI gates passedCodeRabbit Notes
coderabbit-reviewlabel was applied at 13:00:58Z, after CI went fully green: CI run 36568729320, attempt 2.TestShutdown_CancelsSizeFallbackRequest, which is outside this diff, byte-identical to main, and passed 10/10 locally.coderabbit-bot-review.ymlrun 36572061511 posted@coderabbitai reviewthrough the human-identity bridge at 13:01:09Z.coderabbit-bot-review.ymlrun 36576317519 frommain, posted@coderabbitai reviewagain at 13:36:24Z.Exceptions (If any)
packages/vm-agent/internal/acp/gateway.go,internal/acp/session_host.go,internal/server/workspaces.goremain over 800 lines.check-file-sizesEXEMPT list. This PR moves MCP, Codex and Vibe code out of them (net shrink) and adds nothing to them.Agent Preflight (Required)
Classification
External References
Official documentation and primary sources consulted before coding:
x-api-key(single-toolkit MCP) andx-consumer-api-key(Composio Connect). Sources: https://docs.composio.dev/docs/single-toolkit-mcp and https://composio.dev/toolkits/composio/framework/codex.mcp_servers.<id>.env_http_headers(map of header name to env var name) andbearer_token_env_var. Source: https://learn.chatgpt.com/docs/config-file/config-reference.mcp-remote@0.1.38(the Amp bridge), read from the published tarball:--headerparsing uses/^([A-Za-z0-9_-]+):\s*(.*)$/(dist/chunk-65X3S4HB.js:20713); a non-matching name is silently dropped.${ENV}is expanded in values (:20851).acpsdk.HttpHeaderfor inline HTTP MCP servers.[A-Za-z0-9_-]and the duplicate-key rule, checked withpelletier/go-toml/v2(the pinned parser).Codebase Impact Analysis
packages/shared: MCP connection types and header constants,McpServerEntrySchema.headers, contract fixtures.apps/api:0178(renumbered past main's0175and the staged sibling migrations0176/0177).services/mcp-connection-headers.ts(new),services/mcp-connections.ts(create/update and the concurrency guard),services/mcp-connection-resolution.ts,services/node-agent.ts(serializer).packages/vm-agent:internal/acp(entry, validation, ACP/Amp/Codex/Vibe builders).internal/server(normalize, register, converters, agent_ws prefetch).internal/persistence(migrateV18, per-row isolation).apps/web:McpServersManager, the newMcpServerForm,McpServerHeadersField,mcp-server-form-state.ts,lib/api/mcp-connections.ts, Settings help copy.Documentation & Specs
apps/www/src/content/docs/docs/guides/mcp-servers.md: headers, the Composio row, editing, and the per-harness note. The "custom headers not supported" limitation is removed.apps/www/src/content/docs/docs/reference/configuration.mdandapps/api/.env.example:MAX_MCP_CONNECTION_HEADERSandMCP_CONNECTION_HEADER_VALUE_MAX_BYTES..claude/skills/changelog/SKILL.md: themcp-connection-custom-headersentry.Constitution & Risk Check
DEFAULT_*constants inpackages/shared. The 64-character name length is a fixed safety ceiling, like the server-name length.headerNames ?? []).🤖 Generated with Claude Code