Skip to content

fix(console): distinguish DB-down from rate-limiter-unavailable in 503 error#1420

Draft
vklimontovich wants to merge 2 commits into
newjitsufrom
fix/rate-limiter-db-error-message
Draft

fix(console): distinguish DB-down from rate-limiter-unavailable in 503 error#1420
vklimontovich wants to merge 2 commits into
newjitsufrom
fix/rate-limiter-db-error-message

Conversation

@vklimontovich

@vklimontovich vklimontovich commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Problem

When the database is down/unreachable, API requests returned a misleading 503:

{ "error": "rate_limit_unavailable",
  "message": "Rate limiter store is unavailable; request rejected." }

This points developers at the wrong subsystem. And it only surfaced on
rate-limited routes — a DB outage on any other route returned a generic 500.

Root cause

The console rate limiter is Postgres-backed (PgRateLimiter) — it runs an
INSERT ... ON CONFLICT on every rate-limited request. The limiter check in
lib/api.ts is intentionally fail-closed: if check() throws for any reason,
the request is rejected with 503. When Postgres is down, that catch swallowed
the real cause behind the generic "rate limiter unavailable" message.

Fix

Detect a DB outage by actively probing the database, and report it clearly
and consistently across every route.

  • lib/server/db.ts: new isDatabaseReachable(timeoutMs = 1500) — runs a
    trivial SELECT 1 bounded by a timeout. Prisma exposes no "am I connected?"
    flag (connections are lazy + pooled), so a probe is the only reliable signal,
    and there is no Prisma error-code list to keep in sync across versions.
  • lib/api.ts: DB-down handling is centralized in the shared top-level catch
    (respondIfDatabaseDown), so every route — not just rate-limited ones —
    returns error: "database_unavailable" / "Database is unavailable; request
    rejected." when Postgres is down. The rate-limiter catch reuses the same probe.
  • Fail-closed behavior is unchanged (still 503, still Retry-After: 5).

Note

The probe runs one SELECT 1 on the error path (unexpected 500s and limiter
failures). Cheap and exception-path-only; if a real outage's probe stampede ever
matters, a short negative-result cache on isDatabaseReachable would collapse it.

Replace the hardcoded P1xxx code list with an active `SELECT 1` probe
(`isDatabaseReachable`). Prisma exposes no "am I connected?" flag, so probing
is the only reliable signal and there is no error-code list to keep in sync.

Move DB-down handling into the shared top-level catch so every route, not just
rate-limited ones, returns `database_unavailable` when Postgres is down. The
rate-limiter catch reuses the same probe. Fail-closed behavior is unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant