Skip to content

Latest commit

 

History

54 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

api-key-rotator

A multi-provider API key rotation reverse proxy. Supports Firecrawl, Tavily, Apify, and Exa profiles — each with its own key pool, upstream, route prefix, and rotation policy.

Why

firecrawl-mcp forwards all upstream calls to FIRECRAWL_API_URL. Pointing that at this proxy adds key rotation with zero changes to firecrawl-mcp - run the stock npx -y firecrawl-mcp.

Tavily integration works by sed-replacing the Tavily API URL inside OpenWebUI's source so Tavily requests flow through the proxy, picking keys from a separate pool (see "OpenWebUI + Tavily" below).

Run

docker compose up -d

With docker-compose.yml:

api-key-rotator:
  build: .
  environment:
    FIRECRAWL_API_KEYS: "fc-key1,fc-key2,fc-key3"
    UPSTREAM: "https://api.firecrawl.dev"
    PORT: "8788"
    MAX_PASSES: "2"

firecrawl:                     # your existing mcpo + firecrawl-mcp service
  environment:
    FIRECRAWL_API_URL: "http://api-key-rotator:8788/firecrawl"
    # FIRECRAWL_API_KEY removed - the rotator injects it

The rotator exposes each provider on its own URL prefix (/firecrawl, /tavily, /exa, /fmp, /alphavantage, /marketaux, /apify for Apify). During the migration window, requests to a bare path (e.g. http://api-key-rotator:8788/v2/scrape with no /firecrawl prefix) still reach the firecrawl profile — that's the legacy fallback, designed to keep old callers working while they migrate. Flip FIRECRAWL_LEGACY_FALLBACK=false to make unrecognized paths return 404.

Configuration (env vars)

Var Default Purpose
FIRECRAWL_API_KEYS (required) Comma-separated key pool.
UPSTREAM https://api.firecrawl.dev Upstream Firecrawl API base.
FIRECRAWL_ROUTE_PREFIX /firecrawl Route prefix that selects the Firecrawl profile. Stripped before forwarding. Set to empty to restore the historic catch-all behaviour.
FIRECRAWL_LEGACY_FALLBACK true When true, any path no other profile claims still routes to Firecrawl (kept on during the caller migration). When false, unmatched paths return 404. Ignored if FIRECRAWL_ROUTE_PREFIX is empty.
UPSTREAM_PROXY (unset) Explicit forward proxy for egress (http/https/socks5/socks5h). With socks5h, DNS is resolved through the SOCKS5 server (same as socks5; the h form is accepted for curl/SearXNG-style configs). Wins over system vars.
HTTPS_PROXY / HTTP_PROXY / NO_PROXY (unset) System/curl-style proxy env, honored when UPSTREAM_PROXY is unset.
PORT 8788 Listen port.
HOST 0.0.0.0 Listen address.
MAX_PASSES 2 Full passes over the pool before giving up.
MAX_BODY_BYTES 16777216 (16 MiB) Cap on a buffered response body. Above it, forwarded untouched. 0 = no cap.
PROXY_BASE_URL (from Host header) Base used when rewriting next URLs.
CREDIT_RESET_DAY 1 Fallback day-of-month (1-31, UTC) when a key's billing reset can't be auto-detected. See "Credit disabling" below.
LOW_CREDIT_THRESHOLD 10 Switch off a key (rotate to the next) when its predicted remainingCredits drops to/below this.
STOP_CREDIT_THRESHOLD 2 Stop accepting requests when every key is below this. Must be <= LOW_CREDIT_THRESHOLD.
CREDIT_REFRESH_INTERVAL 300 Seconds; minimum interval between credit refreshes of a low-balance key. See "Credit-aware selection".
FIRECRAWL_MAX_CONCURRENT_PER_KEY 1 Max in-flight requests on a single Firecrawl key. Free keys are ~2 concurrent browsers, so a sub-agent burst on one key would otherwise trigger 429 "concurrency limit reached". 0 = unlimited. See "Concurrency control" below.
FIRECRAWL_CONCURRENCY_SATURATION queue What to do when every Firecrawl key is at its concurrency cap: queue (wait for a slot) or reject (return 429 immediately).
FIRECRAWL_CONCURRENCY_QUEUE_MS 15000 How long a queued request waits for a free slot before returning 503. Capped per request, canceled if the client disconnects.
FIRECRAWL_403_RETRIES 1 Same-key retries on a 403 before rotating, for the Firecrawl and Tavily profiles. Firecrawl documents 403 as non-retryable (permission/edge block), and Tavily's 403 is likewise an AWS-edge block of the egress IP (bare awselb/2.0 HTML body) rather than a key problem, so the default caps it at 1 retry (2 hits/key) instead of the legacy 5. 0 = legacy full backoff.
TAVILY_API_KEYS (unset) Comma-separated Tavily key pool. When set, enables the Tavily profile.
TAVILY_UPSTREAM https://api.tavily.com Upstream Tavily API base for the Tavily profile.
TAVILY_ROUTE_PREFIX /tavily Route prefix that selects the Tavily profile. Stripped before forwarding.
TAVILY_LOW_CREDIT_THRESHOLD 10 Same as LOW_CREDIT_THRESHOLD but for the Tavily key pool.
TAVILY_STOP_CREDIT_THRESHOLD 2 Same as STOP_CREDIT_THRESHOLD but for the Tavily key pool.
EXA_API_KEYS (unset) Comma-separated Exa key pool. When set, enables the Exa profile.
EXA_UPSTREAM https://api.exa.ai Upstream Exa API base.
EXA_ROUTE_PREFIX /exa Route prefix that selects the Exa profile. Stripped before forwarding.
EXA_BUDGET_USD 10.00 Monthly credit budget per key (the free tier's $10). Exa exposes no balance endpoint, so the rotator accounts spend against this.
EXA_LOW_CREDIT_USD 0.50 Rotate off a key when its accounted balance drops below this.
EXA_STOP_CREDIT_USD 0.10 Stop serving (503) when every Exa key is below this.
APIFY_API_KEYS (unset) Comma-separated Apify token pool. When set, enables the Apify profile.
APIFY_UPSTREAM https://api.apify.com Upstream Apify API base.
APIFY_ROUTE_PREFIX /apify Route prefix that selects the Apify profile. Stripped before forwarding (callers hit /apify/v2/acts/... and Apify receives /v2/acts/..., keeping every profile prefixed).
APIFY_TIMEOUT_SEC 180 Upstream client timeout for the Apify profile. Synchronous actor runs take 30-120s, so this must comfortably exceed timeout= in the request URL.
APIFY_FREE_CREDIT_USD (unset = auto) Override for the plan's included monthly credit. When unset, the rotator reads it fresh from the account's limits.maxMonthlyUsageUsd on every fetch (so a plan change is picked up automatically). Set it only to force a different value. remaining = credit − current.monthlyUsageUsd.
APIFY_LOW_CREDIT_USD 0.10 Rotate off a token when its remaining balance drops below this.
APIFY_STOP_CREDIT_USD 0.05 Stop serving (503) when every Apify token is below this, until the usage cycle resets.
LOG_LEVEL info info already logs one req line per completed request (profile=… matched=… method=… path=… key=… status=… dur_ms=… cost=…); debug adds the rotation internals.

Endpoints

  • GET /healthz -> 200 {"ok":true} if at least one key is usable, else 503. Docker healthcheck target.
  • GET /status -> pool size, current index, per-key stats, disabled state, and remainingCredits (keys masked to last 4 chars; -1 = unmeasured).

Credit-aware key selection

The rotator picks the key with the highest remaining credits that is above the stop threshold, so traffic concentrates on the healthiest account. It tracks each key's remainingCredits from GET /v2/team/credit-usage (read-only, costs no credits):

  • Startup: fetches every key's real balance in the background (the server starts immediately; until this completes, unmeasured keys are assumed plentiful).
  • On success: decrements the used key's predicted balance by the response's creditsUsed (or by 1 if that field is absent).
  • On rotation: refreshes the keys we switched off and onto (only when the rotation reason could have changed the balance, and skipped entirely for the Exa profile, which has no balance endpoint).
  • Low balance: when a key's predicted balance drops below 100, it is refreshed at most once per CREDIT_REFRESH_INTERVAL (default 5 min) to correct estimation drift.
  • Daily: every key is refreshed once per 24h as a catch-all.

When the current key's predicted balance hits LOW_CREDIT_THRESHOLD (default 10), it is rotated off (cooled down ~30s) and the next-richest key takes over. When every key is below STOP_CREDIT_THRESHOLD (default 2), /healthz returns 503 and new requests return 503 {"success":false,"error":"all keys credit-exhausted until billing reset"}.

Concurrency control (Firecrawl)

Each Firecrawl key has a concurrent-browser cap (Free plan = 2). When OpenWebUI spins up a sub-agent it fires many scrapes at once; if they all land on one key, the upstream returns 429 "concurrency limit reached", and the old behavior (retry each key 5× then rotate) turned that into a churn storm that risks account flags. The rotator now respects per-key concurrency:

  • Per-key in-flight cap (FIRECRAWL_MAX_CONCURRENT_PER_KEY, default 1). Key selection skips keys already at their cap, so a burst is spread across the pool instead of piling onto one key. Each key is a separate account here, so different keys have independent concurrency budgets.
  • Saturation behavior — when every usable key is busy: FIRECRAWL_CONCURRENCY_SATURATION=queue (default) waits up to FIRECRAWL_CONCURRENCY_QUEUE_MS (default 15s) for a slot to free, then serves the request; =reject returns 429 {"error":"all keys busy"} immediately. Queued waits do not consume rotation passes and are canceled if the client disconnects. On queue timeout the request gets 503 {"error":"all keys busy, queue timeout"}.
  • 429 "concurrency limit" rotates to a key with a free slot (different account = different budget). It never disables the key.
  • 403 retry is capped (FIRECRAWL_403_RETRIES, default 1) because Firecrawl documents 403 as non-retryable (permission/edge block) - retrying it 5× per key was the main churn driver. It still never disables the key. The same cap now applies to Tavily (see "Tavily profile" below).

These apply to the Firecrawl profile only; Tavily, Apify, and Exa are unchanged (their pools run unlimited, as before).

Rotation & retry behavior

Two kinds of failure are handled differently:

Key-level rejection -> rotate to another key: HTTP 402 (credits), 429 (rate limit), 401 (bad key), and failure envelopes ({"success":false,...}) whose text matches insufficient credits, rate limit, exceeded, payment required, unauthorized, forbidden. The key is cooled down ~30s (or disabled if credit-exhausted - see below) and the next key is tried, up to MAX_PASSES full sweeps.

Transient error -> backoff on the SAME key: HTTP 403 (edge/WAF), 408, 5xx, and network errors are retried on the same key with exponential backoff 500ms -> 1s -> 2s -> 4s -> 8s (5 attempts, ~15s total) before rotating. A 403 is usually a network/edge-layer issue, not a per-key problem, so it does NOT disable or rotate immediately. 403s are additionally capped by FIRECRAWL_403_RETRIES (default 1, so 2 hits per key) on the Firecrawl and Tavily profiles.

A successful response (status < 400 with success:true, or no success field) never rotates - even if the scraped content mentions "rate limit" or "payment required". The denylist is checked against the Firecrawl failure envelope only, not the response body as a whole.

  • The next field's absolute upstream URL is rewritten to the proxy so crawl pagination stays under rotation. Other occurrences of the host in response bodies are never rewritten (they may be real scraped content).

Credit disabling

A key that returns a genuine credit-exhaustion signal (HTTP 402, or a success:false envelope mentioning insufficient credits / payment required / exceeded) is disabled and skipped on all subsequent requests until its credits reset - it is not retried every pass (which would waste upstream calls and risk account flags).

  • The reset instant is read per key from that key's own GET /v2/team/credit-usage -> billingPeriodEnd (a read-only endpoint that costs no credits). This matters because each key belongs to a separate account and resets on that account's billing anniversary, which is often a different day per key - not a universal date.

  • If the credit-usage call fails, the key is disabled until the next occurrence of CREDIT_RESET_DAY (UTC) as a fallback.

  • 429 (rate limit) and 401 (auth) rotate but do NOT disable - they are transient or account-global, and disabling on them would take a good key offline. 403 is retried with backoff, never disabled.

  • A background loop re-enables each key at its own reset instant; restarting the container also clears all disables.

    Exception for daily-capped finance (FMP/Alpha Vantage/Marketaux): their free tiers reset daily, not on a billing anniversary, so a 429 from FMP or a 402 from Marketaux (and Alpha Vantage's {"Note":...} 200 body) means "the daily cap ran out", which does disable until the next CREDIT_RESET_DAY. See Finance profiles below.

Tavily profile

When TAVILY_API_KEYS is set, the proxy creates a separate key pool for Tavily. Requests whose path starts with TAVILY_ROUTE_PREFIX (default /tavily) are routed to the Tavily pool:

  • Prefix stripping: the leading /tavily is removed before forwarding. A request to /tavily/search hits {TAVILY_UPSTREAM}/search.
  • Rotation policy: HTTP 401 and 429 rotate the key (cool down ~30s) but do not disable. HTTP 432 and 433 disable the key until its credit reset (same reset mechanism as Firecrawl - per-key /api/usage or fallback CREDIT_RESET_DAY).
  • Body-based rejection detection: Tavily rejects are detected purely by status code, never by scanning response body text. The Firecrawl denylist never applies to Tavily responses.
  • 403 retry is capped (shares FIRECRAWL_403_RETRIES, default 1). Tavily's 403 comes from its AWS edge (server: awselb/2.0, bare HTML body) and blocks the egress IP, not the key - so retrying it 6× per key across all keys turned one client request into ~48 upstream hits, deepened the block, and made a failing request take ~106s to return. With the cap, a failing request returns in ~4s.
  • Retry-After is honored as a cooldown. Tavily returns retry-after: 60 on its 429s. Since each key is its own account, rotating to the next key is the correct response - but the rejected key is then deprioritized for as long as upstream asked, instead of coming straight back after the usual ~30s cooldown. The cooldown is only a deprioritization: the pool still serves from a cooling-down key when no other key is available, so a long Retry-After can never cause a 503. Without the header (all other providers), behavior is the legacy ~30s cooldown.
  • Usage tracking: the proxy calls GET /usage on the Tavily upstream (per-key) to read key.usage / key.limit, account.plan_usage / plan_limit, and account.paygo_usage / paygo_limit. Effective remaining credits = min over layers of (limit - usage), skipping any layer whose limit <= 0 (unlimited/unmeasured). If all layers are unlimited the key stays unmeasured. Tavily returns no billing-period end, so a disabled key re-enables at the CREDIT_RESET_DAY fallback, same as the Firecrawl fallback path.
  • Credit thresholds: TAVILY_LOW_CREDIT_THRESHOLD and TAVILY_STOP_CREDIT_THRESHOLD mirror the Firecrawl thresholds but apply to the Tavily pool independently.

Exa profile

When EXA_API_KEYS is set, the proxy creates a separate key pool for Exa. Requests whose path starts with EXA_ROUTE_PREFIX (default /exa) are routed to https://api.exa.ai:

  • Prefix stripping: the leading /exa is removed before forwarding. A request to /exa/search hits {EXA_UPSTREAM}/search.
  • Auth is the x-api-key header, not Authorization: Bearer. The key is sent as the bare header value, and any client-sent value is replaced. (Bearer happens to work upstream but is undocumented, so the rotator does not rely on it.)
  • Rotation policy (status codes only, body never scanned): HTTP 401 (invalid key) and 429 (rate limit) rotate but do not disable. HTTP 402 (credits exhausted - NO_MORE_CREDITS / API_KEY_BUDGET_EXCEEDED / TEAM_BUDGET_EXCEEDED) disables the key until the CREDIT_RESET_DAY fallback. 403 stays transient: retried on the same key with backoff, never disabled. Exa search results contain arbitrary scraped page text (which can contain "insufficient credits", "rate limit", ...), so the body is never consulted.
  • Balances are accounted, not read. Exa exposes no balance endpoint (probed /billing, /usage, /account, /credits, /me, /key on api.exa.ai - all 404; the only usage API lives on admin-api.exa.ai, needs a separate service key, and reports spend rather than remaining credits). So each key starts at EXA_BUDGET_USD and is decremented by every response's costDollars.total. Consequences:
    • Restarting the container resets each key to the full budget. A mid-month restart therefore over-estimates the remaining balance. This cannot cause a false 503 - only extra spend until a real 402 arrives, which is authoritative.
    • costDollars is an official estimate ("billing is computed from usage counters rather than this response object"), so drift is expected.
  • Money is tracked in micro-USD (millionths of a dollar) internally. A search costs $0.007, and int(0.007 * 100) == 0, so a cents-denominated balance would never decrement at all. The env vars stay in USD.
  • No next-URL rewriting. Exa's /search returns no pagination cursor, and a real response contains zero api.exa.ai URLs (result URLs point at third-party pages), so there is nothing to rewrite.
  • No per-key concurrency cap. Exa's documented limits are QPS-level (/search 10 QPS, /contents 100 QPS), not concurrency-level. The existing 429-rotate and Retry-After cooldown handle throttling.

Client usage:

curl -X POST http://localhost:8788/exa/search \
  -H 'Content-Type: application/json' \
  -d '{"query":"kubernetes best practices","numResults":5,"type":"fast"}'
# no auth header needed - the rotator injects x-api-key from EXA_API_KEYS.

Apify profile

When APIFY_API_KEYS is set, the proxy creates a separate token pool for Apify. Requests whose path starts with APIFY_ROUTE_PREFIX (default /apify) are routed to https://api.apify.com:

  • Auth is a query param, not a header. Clients put ?token=... in the URL (or omit it); the rotator replaces/adds the token query parameter with the pooled token on every attempt. Any client-sent Authorization header is dropped. The rest of the query string (timeout=120, ...) is preserved verbatim.
  • Prefix is stripped, like every other profile. A request to /apify/v2/acts/{user}~{actor}/run-sync-get-dataset-items is forwarded to {APIFY_UPSTREAM}/v2/acts/{user}~{actor}/run-sync-get-dataset-items - point APIFY_BASE_URL (or your SDK's base URL) at http://api-key-rotator:8788/apify and append the usual /v2/acts/... path.
  • Long timeout. The Apify profile gets its own HTTP client with APIFY_TIMEOUT_SEC (default 180s) instead of the shared 30s, because synchronous actor runs take 30-120s.
  • Rotation policy (status codes only, body never scanned - a success is a bare dataset-items array with no envelope): HTTP 401 and 429 rotate (cool down ~30s) but do not disable. HTTP 402 disables the token until its real usage-cycle end (see below). 403 stays transient: retried on the same token with backoff like every profile.
  • Credit / balance tracking (USD, in cents). The proxy calls GET /v2/users/me/limits?token=... (read-only) and computes remaining = includedCredit − current.monthlyUsageUsd, tracked in cents so sub-dollar thresholds are exact. The included credit is the account's own limits.maxMonthlyUsageUsd read fresh each fetch (a plan upgrade is picked up automatically), unless APIFY_FREE_CREDIT_USD overrides it. The real reset instant is monthlyUsageCycle.endAt (the account's billing anniversary, not the 1st of the month), so a 402-disabled token re-enables exactly when the credit renews. If the limits call fails, it falls back to CREDIT_RESET_DAY.
  • Auto-stop near zero. A token is rotated off when its remaining balance drops below APIFY_LOW_CREDIT_USD (default $0.10), and the whole profile returns 503 once every token is below APIFY_STOP_CREDIT_USD (default $0.05). The balance is measured on the first request and refreshed on rotation / when low / daily, so a token stops serving before it hits $0 and comes back after the monthly reset - no manual intervention.
  • Note on the included credit: it comes from the account's limits.maxMonthlyUsageUsd (the monthly usage cap reported by Apify), read fresh every fetch. APIFY_FREE_CREDIT_USD exists only as an escape hatch to force a different value (e.g. if the account's reported cap isn't the real included credit).

Client usage (no SDK, plain HTTP):

curl -X POST "http://localhost:8788/apify/v2/acts/apimaestro~linkedin-posts-search-scraper-no-cookies/run-sync-get-dataset-items?timeout=120" \
  -H "Content-Type: application/json" \
  -d '{"searchUrl":"https://www.linkedin.com/search/results/content/?keywords=ai"}'
# ?token= is optional - the rotator injects/replaces it from APIFY_API_KEYS.

Finance profiles: FMP / Alpha Vantage / Marketaux

Three daily-capped finance data providers, each with a very small free tier. All authenticate via query param (apikey or api_token), are routed by strip prefix, and share the same daily-cap behavior: when a key's quota is exhausted the key is disabled until the next CREDIT_RESET_DAY at 00:00 UTC (not merely cooled down), and when its predicted balance falls to the stop threshold it is held back locally so a real upstream 429/402 never has to fire.

FMP Alpha Vantage Marketaux
Route prefix /fmp (stripped) /alphavantage (stripped) /marketaux (stripped)
Upstream https://financialmodelingprep.com https://www.alphavantage.co https://api.marketaux.com
Auth query param apikey apikey api_token
Free tier 250 req/day 25 req/day 100 req/day
Daily-cap signal 429 "Limit Reach" 200 OK + body {"Note":"..."}/{"Information":"..."} 402 usage_limit_reached
Rotate (transient) 401, 402¹ 401 401, 402, 429
Credit-exhausted (disables) 429 rate-limit 200 body 402

¹ FMP 402 is special-cased. FMP returns 402 for two very different reasons, and we use the body prefix to tell them apart:

  • Plan-tier block — body starts with Restricted Endpoint: This endpoint is not available under your current subscription. The plan (not this key) forbids the endpoint; rotating the other keys would burn their daily counter on the same refusal. The rotator does NOT rotate, does NOT disable, does NOT decrement — it forwards the 402 to the client so you can pick a different (free-tier) endpoint. Free-tier endpoints: /stable/profile, /stable/quote, /stable/historical-price-eod/{light,full}, /stable/income-statement, /stable/balance-sheet-statement, /stable/cash-flow-statement, /stable/search-name. Plan-blocked endpoints: /stable/news/stock, /stable/news/stock-latest, /fmp-articles, and most of the legacy /api/v3/* and /api/v4/* news/household endpoints.
  • Key-tier quota/billing refusal — any other 402 body (rare, not currently observed) rotates like a 401, so the next key can corroborate.

Alpha Vantage is the only profile allowed to credit-disable on a 200 body. Its daily-cap envelope is {"Note":"Thank you for using Alpha Vantage! ..."} (or {"Information":"..."}) - never a 4xx. Genuine data payloads never contain those keys, so the body scan is safe for this single provider. Every other profile treats a 200 body as opaque success.

Per-provider env vars (FMP_*, ALPHAVANTAGE_*, MARKETAUX_*):

Variable FMP default AlphaVantage default Marketaux default Description
*_API_KEYS — — — Comma-separated keys; enables the profile when set
*_UPSTREAM https://financialmodelingprep.com https://www.alphavantage.co https://api.marketaux.com Upstream base URL
*_ROUTE_PREFIX /fmp /alphavantage /marketaux Path prefix routed here (stripped)
*_DAILY_BUDGET 250 25 100 Predicted requests/day, seeded at startup
*_LOW_CREDIT 10 5 10 Below this a key refreshes more aggressively
*_STOP_CREDIT 1 1 1 Hard floor; at or below this the key is held back, so the last request of the day is never served to the upstream

Client examples (keys are injected by the rotator - do NOT put a key in the URL; any client-supplied apikey/api_token value is overwritten by the pool):

FMP_BASE_URL=http://api-key-rotator:8788/fmp
ALPHAVANTAGE_BASE_URL=http://api-key-rotator:8788/alphavantage
MARKETAUX_BASE_URL=http://api-key-rotator:8788/marketaux

curl "http://localhost:8788/fmp/api/v3/profile/AAPL"
curl "http://localhost:8788/alphavantage/query?function=GLOBAL_QUOTE&symbol=IBM"
curl "http://localhost:8788/marketaux/v1/news/all?language=en&limit=1"

OpenWebUI + Tavily

To route OpenWebUI's Tavily searches through the proxy, override the Tavily API URL at container startup with sed:

services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    # ... your existing config ...
    command: >
      bash -c "
        sed -i \"s|https://api.tavily.com|http://api-key-rotator:8788/tavily|g\"
          /app/backend/open_webui/retrieval/web/tavily.py
          /app/backend/open_webui/retrieval/loaders/tavily.py
        && bash start.sh"

The /tavily prefix is stripped by the proxy before forwarding to https://api.tavily.com, so no other changes are needed.

Migration note

This project was originally named firecrawl-rotator. The rename to api-key-rotator affects:

Item Old New
Docker image ghcr.io/<you>/firecrawl-rotator ghcr.io/<you>/api-key-rotator
Compose service name firecrawl-rotator api-key-rotator
FIRECRAWL_API_URL host http://firecrawl-rotator:8788 http://api-key-rotator:8788/firecrawl
Binary / healthcheck /rotator -healthcheck /api-key-rotator -healthcheck

Update your docker-compose.yml service name, image tag, healthcheck path, and depends_on / FIRECRAWL_API_URL references accordingly.

Develop

go test ./...
go build -o api-key-rotator .
FIRECRAWL_API_KEYS=fc-x ./api-key-rotator

See docs/superpowers/specs/2026-07-09-firecrawl-token-rotation-design.md for the full design.

About

Reverse proxy in front of firecrawl-mcp that rotates Firecrawl API keys on 402/429/401/403, rewrites crawl next URLs, and exposes /healthz + /status. Go stdlib only, scratch image.

Topics

Resources

Stars

14 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages