[rennf93]

Add an optional fastapi-guard security layer

Relates to discussion #463

Summary

WhisperLiveKit is often run where other people can reach it, and every WebSocket session it accepts costs a decoder. This adds an opt-in application-layer security layer built on fastapi-guard, off by default:

• Attack-pattern screening of the path, query string and headers (path traversal, injection, XSS, scanner probes) on the HTTP routes and on the WebSocket handshake of /asr and /v1/listen, with automatic bans for clients that keep probing. A banned or refused client is turned away before accept(), so no AudioProcessor, FFmpeg process or decoder is started for it.
• Handshake protection and per-route budgets: a per-client budget on new WebSocket sessions, and on POST /v1/audio/transcriptions a stricter request budget plus a size limit and a content-type filter.
• A rule that bans a client after repeated 401 responses from the HTTP API.
• IP and CIDR allow and block lists, country allow or block lists, cloud-provider range blocking and user-agent filtering, all enforced at the WebSocket handshake as well.
• Emergency mode (lock the server down to a short allowlist during an incident) and passive mode (log only, to try the rules before enforcing them).
• Rate limiting is one item in that list, not the whole feature.

Enable with --security-guard or WLK_SECURITY_GUARD=1, after installing the security extra. Settings are WhisperLiveKitConfig fields with matching CLI flags (--security-guard-*), documented in the new docs/security.md. The guard runs inside the process and complements a reverse proxy, WAF or firewall in front of the server; where there is none, it is the protection. TLS stays at the proxy.

On #455: the guard limits how fast one client can open sessions and shuts out abusers, but it does not cap concurrent sessions or close idle ones. Those two stay server fixes.

User impact

• Off by default. With it off, guard is never imported, no middleware is added and create_app builds the same routes. The only structural change is that the two WebSocket routes now live on a second router (ws_router), so the guard can run its handshake dependency on exactly those routes.
• With it on and the package missing, the server stops at startup with an error that names the extra to install.
• Everything the layer enables is a safe default for this app: state is in memory (enable_redis=False), no agent, no response headers and no CORS handling from the guard (CORSMiddleware stays outermost, so a 403 or 429 from the guard still carries CORS headers), request bodies are never scanned, and the free-text context query parameter is excluded from screening. The ssrf detection category is disabled because, with guard-core 4.4.0, a Referer pointing at localhost or a LAN address is flagged and the browser UI sends exactly that.
• Default budget is 60 requests and, separately, 60 new WebSocket sessions per client per 60 seconds, and 10 uploads per 60 seconds. A cold load of the UI is one GET / plus the browser's /favicon.ico (the page and its assets are inline), and the UI reconnects with a 1 s to 30 s backoff, so 60 leaves roughly ten times the headroom a single user needs. The reasoning is in docs/security.md.
• Client address: Uvicorn rewrites the client from X-Forwarded-For before any middleware runs. With the guard on, main() passes --forwarded-allow-ips to both Uvicorn and the guard, and turns Uvicorn's proxy headers off when none is configured (even with FORWARDED_ALLOW_IPS in the environment), so a client cannot rotate its identity to dodge a budget or a ban. The manual uvicorn whisperlivekit.basic_server:app launch is documented.
• Things to know, all with fastapi-guard 8.1.1 and guard-core 4.4.0 and all in the docs: bans are in memory, per process, and never applied to a loopback address or a trusted proxy; the 401 rule counts the HTTP API only (a wrong token on a WebSocket closes with 4401 after the handshake); the engine's handshake dependency does not enforce user-agent rules or emergency mode, so the handshake dependency here applies them; in passive mode a handshake from a blocklisted or banned address, or over its session budget, is still refused.
• Dependencies: new optional extra security = ["fastapi-guard>=8.1.1"]. It is also in the test extra so CI (pip install -e ".[test]") and uv sync --extra test run the new tests. uv.lock gains fastapi-guard, guard-core, maxminddb, redis, async-timeout and nothing else changes. Both packages are MIT.

Validation

Hardware and tools: Apple Silicon Mac, Python 3.12.15, uv 0.11.1, ruff 0.16, fastapi 0.143.3, starlette 1.8.0, uvicorn 0.54.0, fastapi-guard 8.1.1, guard-core 4.4.0.

The CI gate on main before the change and on the branch after it:

Step	main	branch
ruff check .	pass	pass
uv lock --check	pass	pass
pytest -q tests/ --ignore=tests/test_pipeline.py --ignore=tests/test_asr_coalescing_pipeline.py	314 passed, 15 skipped	338 passed, 15 skipped (e8f6780; d5a9b9f adds one more passing test)
pytest -q tests/test_asr_coalescing_pipeline.py (Whisper tiny, real audio)	4 passed	4 passed
python -m build and twine check	not run	pass

tests/test_security_guard.py (25 tests) goes through create_app() with real HTTP requests and real WebSocket handshakes (Starlette TestClient, plus one real Uvicorn socket with a websockets client). It covers: off adds nothing and never imports guard (clean subprocess); the missing-package error; middleware order and CORS headers on 429 and 403; handshakes from banned, blocklisted, emergency-blocked and attack-payload clients are refused before accept and the test counts AudioProcessor and FFmpeg starts (zero); the session budget; HTTP attack query returning 400 and repeated attacks banning; the upload budget, size limit and content-type filter; more than five 401s banning the client on HTTP and at the handshake; spoofed X-Forwarded-For ignored from an untrusted peer and honored from a trusted proxy; uvicorn.Config(...).load() with FORWARDED_ALLOW_IPS=* and rotating X-Forwarded-For still tripping the budget; localhost and LAN Referer returning 200 and a <script> Referer still refused; free text in context; allow, block and user-agent rules; emergency and passive modes; normal streaming through /asr; and the warning logged when --forwarded-allow-ips '*' is combined with the guard, since * would let any client choose the address that is rate limited and banned. Each test uses its own client address. I ran the file five times and the whole tree three more times in random order (pytest-randomly, not added to the project) with no failure.

I also ran the real server on 127.0.0.1 (--no-transcription --no-vac --pcm-input, so no model is needed) with the guard on and off and probed it with curl-style HTTP requests and a websockets client: with the guard on, an attack query and a script Referer get 400, WebSocket handshakes with an attack query or a blocked user-agent get 403 with no session, the HTTP and session budgets trip while a normal /asr session still works, localhost and LAN Referer get 200, and a client with more than five 401s is banned and then refused on HTTP and at the handshake. With it off, all of those are accepted as before.

One note on the gate: one of three runs of the real-audio Whisper test on the branch exited with status 139 after reporting 4 passed (interpreter teardown); two reruns exited 0. That test does not touch this code.

Disclosure: I maintain fastapi-guard and guard-core, the engine underneath it.

I used AI assistance to prepare this and reviewed and verified everything myself.

Checklist

• I searched for an existing issue or discussion and linked it when relevant.
• I added or updated tests for behavior changes.
• I updated documentation for user-facing changes.
• I ran ruff check ..
• I ran the relevant pytest suite.
• I did not commit credentials, model weights, generated caches, or private data.

Project background: Guard Core's first library is fastapi-guard, public since August 2024 (first PyPI release, 0.1.0, on 2024-08-26). It has 825 GitHub stars, 65 releases and about 62,000 PyPI downloads in the last month (pypistats.org). The framework-agnostic engines and the adapters for other frameworks and languages came later and run the same detection rules, checked in CI against shared conformance fixtures.