From ed8bfe1af80d467d40d384f9413ab77f4898e195 Mon Sep 17 00:00:00 2001 From: 0xDeon Date: Tue, 20 Oct 2026 12:54:44 +0100 Subject: [PATCH] docs(backend): add operational endpoints, prekey replenishment, key material and socket event guides --- .../backend/docs/api-operational-endpoints.md | 247 ++++++++++++++++++ .../docs/concepts-prekey-replenishment.md | 114 ++++++++ apps/backend/docs/contracts-key-material.md | 194 ++++++++++++++ apps/backend/docs/howto-add-socket-event.md | 236 +++++++++++++++++ 4 files changed, 791 insertions(+) create mode 100644 apps/backend/docs/api-operational-endpoints.md create mode 100644 apps/backend/docs/concepts-prekey-replenishment.md create mode 100644 apps/backend/docs/contracts-key-material.md create mode 100644 apps/backend/docs/howto-add-socket-event.md diff --git a/apps/backend/docs/api-operational-endpoints.md b/apps/backend/docs/api-operational-endpoints.md new file mode 100644 index 0000000..24b7106 --- /dev/null +++ b/apps/backend/docs/api-operational-endpoints.md @@ -0,0 +1,247 @@ +# Operational and utility endpoints + +Source: [`src/app.ts`](../src/app.ts), +[`src/routes/security.ts`](../src/routes/security.ts), +[`src/routes/localStorage.ts`](../src/routes/localStorage.ts), +[`src/middleware/transportSecurity.ts`](../src/middleware/transportSecurity.ts), +[`src/middleware/rateLimit.ts`](../src/middleware/rateLimit.ts). + +These routes are registered directly in `app.ts` (or mounted from a tiny router) rather than in +a feature router. They serve orchestrators, scrapers, mobile bootstrap and local development, not +product features. + +## Summary + +| Route | Auth | Global IP rate limit | Plaintext over `http` when TLS is enforced | +| ---------------------------------- | ------------------------------------- | -------------------- | ------------------------------------------ | +| `GET /health` | none | **exempt** | allowed | +| `GET /metrics` | none (see [protection](#get-metrics)) | applies | rejected (`403 tls_required`) | +| `GET /me` | `requireAuth` (Bearer JWT) | applies | rejected | +| `GET /security/transport-policy` | none | applies | rejected | +| `/local-storage/*` (dev/test only) | signed URL (HMAC + expiry) | applies | n/a (not mounted in production) | + +### Rate-limiter ordering + +`app.ts` registers `app.use(rateLimit('global_ip', { identifier: ipIdentifier }))` **after** +`GET /health` and **before** every other route. Express runs handlers in registration order, so: + +- `/health` is registered first and answers before the limiter runs. It is never counted and never + returns `429`. The comment in `app.ts` states the intent: orchestrator probes must not be rate + limited. +- `/metrics`, `/me`, `/security/transport-policy` and `/local-storage/*` are all registered after the + limiter, so each request is charged to the caller's IP in the `global_ip` bucket + (default 600 requests per minute, `config/rateLimits.ts`). A `429` response is + `{ "error": "Too many requests", "bucket": "global_ip", "retryAfterSeconds": n }` with + `RateLimit-*` and `Retry-After` headers. + +Consequence for scrapers: a Prometheus server scraping every 15 s uses 4 requests per minute per +instance, far below the ceiling, but a shared egress IP (for example many pods behind one NAT scraping +the same gateway) shares one budget. + +### Transport-security middleware + +`enforceTransportSecurity` runs before any route. When TLS is enforced (the default outside +`development`/`test`, see [`ENFORCE_TLS`](../../../docs/environment-variables.md)) it rejects +non-HTTPS requests with: + +```json +{ + "error": "tls_required", + "message": "This endpoint is only available over https. Reconnect using https:// or wss://." +} +``` + +The only path exempt is `/health` (`PLAINTEXT_ALLOWED_PATHS`). Load balancers health-check the pod +directly before TLS termination and a `403` would pull a healthy gateway out of rotation. +`enforceOriginPolicy` also runs first on every route: a request carrying an `Origin` header that is not +on the `ALLOWED_ORIGINS` allowlist gets `403 origin_not_allowed`. Requests without an `Origin` +header (curl, Prometheus, orchestrator probes) are not affected. + +## `GET /health` + +Liveness and readiness probe. + +**Auth:** none. **Rate limit:** exempt. **TLS:** allowed over plaintext. + +**Response `200`:** + +```json +{ + "status": "ok", + "db": "connected", + "node": "v22.11.0", + "version": "1.0.0" +} +``` + +**Response `503`** (degraded): + +```json +{ + "status": "error", + "db": "unreachable", + "node": "v22.11.0", + "version": "1.0.0" +} +``` + +### Semantics for orchestrators + +- The only dependency checked is **PostgreSQL**, via `SELECT 1` through the Drizzle `db` client. Any + error from that query produces the `503`. There is no query timeout of its own, so a hung database + connection holds the probe until the orchestrator's own probe timeout fires. +- **Redis, the object store, Stellar/Horizon and the Socket.IO server are not checked.** Redis is + treated as degradable throughout the gateway (rate limiting, presence and latches fall back to + in-process state), so a Redis outage leaves `/health` at `200`. +- `status` and `db` are fixed literals: `ok`/`connected` on success, `error`/`unreachable` on failure. + `node` is `process.version` and `version` is `version` from `apps/backend/package.json`, so both are + always present, including in the degraded body. +- The route is cheap and unauthenticated, so use it for both liveness and readiness. Because it + depends on the database, a pure **liveness** probe will restart pods during a database outage, + which will not fix the outage. If that is a concern, point the liveness probe at a TCP check and + keep `/health` for readiness. + +## `GET /metrics` + +Prometheus scrape endpoint (#393). + +**Auth:** none in the application. **Rate limit:** applies (`global_ip`). +**Response `200`:** Prometheus text exposition, `Content-Type` taken from +`registry.contentType` (`text/plain; version=0.0.4; charset=utf-8`). It contains the application +counters, gauges and histograms defined in `lib/metrics.ts` plus the default Node process metrics. +The metric list is maintained in [`docs/observability.md`](../../../docs/observability.md#metrics-reference); +it is not repeated here. + +Metrics carry no message content: `lib/metrics.ts` takes no ciphertext or free-text labels. +They still reveal traffic volume, instance count, version/runtime details and per-instance load, so +they should not be public. + +### Protecting `/metrics` in production + +The application does **not** authenticate this route: there is no `requireAuth`, token or IP allowlist +on it. Protection must come from the deployment: + +1. **Do not expose it on the public ingress.** Route only the API paths you intend to publish, or + explicitly block `/metrics` at the edge (load balancer, ingress or WAF rule returning `404`). +2. **Scrape over the private network** (cluster-internal Service, VPC, service mesh), with a network + policy or security group limiting the source to the Prometheus server. +3. **TLS enforcement still applies.** `/metrics` is not in `PLAINTEXT_ALLOWED_PATHS`, so a scraper + hitting the pod over plain `http` when TLS is enforced gets `403 tls_required`. Either scrape + through the TLS-terminating proxy (so `X-Forwarded-Proto: https` is present and `TRUST_PROXY` covers + that hop), or scrape over mTLS. Setting `ENFORCE_TLS=false` for this reason weakens every other + route and is not recommended. +4. **The global limiter applies.** Keep the scrape interval modest and make sure the scraper's IP is + not shared with unrelated traffic (see above). + +## `GET /me` + +Returns the identity carried by the caller's token. + +**Auth:** `requireAuth`. The request needs `Authorization: Bearer `. Beyond signature and expiry, the +middleware checks that the `(userId, deviceId)` pair exists in `devices` and is not revoked. See +[JWT and auth contract](contracts-jwt-auth.md). **Rate limit:** applies (`global_ip`, by IP). + +**Response `200`:** + +```json +{ + "user": { + "userId": "9f1c6e2a-...", + "walletAddress": "GABC...", + "deviceId": "b6b6c3b0-...", + "iat": 1767225600, + "exp": 1767830400 + } +} +``` + +`user` is the verified JWT payload (`userId`, `walletAddress`, `deviceId`, plus the standard `iat` +and `exp` claims added by the JWT library). It is read from the token, not from the database, so +it does not reflect later profile edits. + +**Errors** (all `401`): + +| Body `error` | Cause | +| ----------------------------------------- | --------------------------------------------- | +| `Missing or invalid Authorization header` | no `Bearer ` header | +| `Invalid or expired token` | bad signature, or the 7-day token has expired | +| `Token missing deviceId` | legacy token without a `deviceId` | +| `Device not found or has been revoked` | device row missing or `revokedAt` set | + +## `GET /security/transport-policy` + +Public TLS and certificate-pinning policy for mobile clients, so pin rotation does not need an app +release (#374). See [`docs/security/`](../../../docs/security) for how clients consume it. + +**Auth:** none, deliberately: a client must read it before it holds a token, and every value is public. +**Rate limit:** applies (`global_ip`). +**Headers:** `Cache-Control: public, max-age=`. + +**Response `200`:** + +```json +{ + "tls": { + "required": true, + "minimumVersion": "TLSv1.2", + "hsts": "max-age=31536000; includeSubDomains; preload" + }, + "pinning": { + "enforced": true, + "hosts": ["api.example.com"], + "pins": ["sha256/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="], + "backupPins": ["sha256/BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB="], + "maxAgeSeconds": 5184000, + "reportUri": null + } +} +``` + +- `tls.required` is `isTlsEnforced()`; `tls.hsts` is the header value, or `null` when HSTS is disabled. +- `pinning.enforced` is `true` only when **both** `pins` and `backupPins` are non-empty, so a client never + hard-fails against a set with no spare key. +- Pins must match `sha256/<44-char base64>`; malformed entries are logged and dropped. +- `maxAgeSeconds` defaults to 60 days (`TLS_PIN_MAX_AGE_SECONDS`); `reportUri` comes from + `TLS_PIN_REPORT_URI`, else `null`. +- Values come from `TLS_PINNED_SPKI_SHA256`, `TLS_BACKUP_SPKI_SHA256`, `TLS_PINNED_HOSTS`, + `TLS_PIN_MAX_AGE_SECONDS` and `TLS_PIN_REPORT_URI`. + +## `/local-storage/*` (development and test only) + +File server for the filesystem-backed object store, so presigned URLs issued locally are real working +URLs. See [Local Object Store](concepts-local-object-store.md). + +**Auth:** none from `requireAuth`. Like an S3 presigned URL, each request carries its own +`expires` (Unix seconds) and `sig` (HMAC-SHA256 hex over `METHOD:key:expires`) query parameters, +method-bound and time-limited. **Rate limit:** applies (`global_ip`), as it is registered after the limiter. + +| Method | Path | Success | Failure | +| ------ | ---------------------- | ----------------------------------------- | ----------------------------------------------------------------------------------------------- | +| `PUT` | `/local-storage/` | `200` empty body (raw body up to `150mb`) | `403 { "error": "Invalid or expired signed URL" }`, `500 { "error": "Failed to store object" }` | +| `GET` | `/local-storage/` | `200`, stored bytes and `Content-Type` | `403` as above, `404 { "error": "Object not found" }` | + +The HMAC secret is random per process, so a URL signed by one process cannot be verified by another or after a +restart. + +### When it is mounted + +```ts +if (process.env['NODE_ENV'] !== 'production') { + app.use('/local-storage', localStorageRouter); +} +``` + +It is mounted whenever `NODE_ENV` is anything other than the exact string `production`, including +`development`, `test` and **unset**. It is not mounted when `NODE_ENV === 'production'`. + +### Can it be enabled in production? + +Not by any flag, header or configuration option: there is no environment variable that forces the mount, +and `APP_ENV` is not consulted for it. With `NODE_ENV=production`, the route does not exist (requests +fall through to Express's default `404`), and `storage.ts` and `objectStore.ts` use the real S3 store +instead of the local one. + +The guarantee is only as good as `NODE_ENV` itself. If a production deployment forgets to set +`NODE_ENV=production`, the route **is** mounted and uploads go to local disk (this is also called out in +[environment-variables.md](../../../docs/environment-variables.md)). Set `NODE_ENV=production` in every +production environment and verify with `curl -i https:///local-storage/x` returning `404`, not `403`. diff --git a/apps/backend/docs/concepts-prekey-replenishment.md b/apps/backend/docs/concepts-prekey-replenishment.md new file mode 100644 index 0000000..1b86514 --- /dev/null +++ b/apps/backend/docs/concepts-prekey-replenishment.md @@ -0,0 +1,114 @@ +# One-time prekey low-water signalling + +Source: [`src/services/prekeyLowSignal.ts`](../src/services/prekeyLowSignal.ts), +[`src/routes/users.ts`](../src/routes/users.ts) (bundle fetch), +[`src/routes/devices.ts`](../src/routes/devices.ts) (prekey upload, revocation). + +See also: [E2EE key management](concepts-e2ee-key-management.md), +[Devices API](api-devices.md#post-devicesidprekeys), [WebSocket events](api-websocket-events.md). + +## Why the server signals + +A device's one-time prekeys (OPKs) are consumed one per X3DH bundle fetch and are **never regenerated +server-side**, because the private halves exist only on the device. Once a device has none left, an +initiator receives a bundle without a one-time key and falls back from 4-DH to 3-DH, losing the forward +secrecy contribution of that key. The server cannot create keys, so it tells the device to top up +_before_ the pool reaches zero. + +## When the signal fires + +| Item | Value | +| ------------ | ------------------------------------------------------------------------------------------------------------------------------ | +| Threshold | `PREKEY_LOW_THRESHOLD`, default **20**. Read once at process start; must be a positive integer, otherwise the default is used. | +| Condition | unconsumed OPKs remaining **below** the threshold (`remaining < threshold`) | +| Trigger path | the bundle-fetch route in `routes/users.ts`, **only when a one-time key was actually claimed** | + +The count only ever drops on a bundle fetch, so that is the single place the check runs. After the +claim transaction the route recounts the device's unconsumed OPKs and calls +`signalPrekeysLowIfNeeded(deviceId, remaining)` fire-and-forget: it never blocks, delays or fails the +bundle response, and never throws. A fetch that finds no OPK to claim (the pool is already empty) does not +re-run the check. + +Nothing else emits it. There is no periodic sweep, no check on connect, and no check on upload other than +releasing the latch (below). + +## The latch + +Without a latch, a device sitting at 12 keys would be signalled on every one of the next dozen fetches. The +signal is therefore latched per device: + +- **Redis key:** `prekeys:low:`, value `1`, **TTL 30 days** (`LATCH_TTL_SECONDS`). +- **Acquire:** one atomic `SET key 1 EX 2592000 NX`. Exactly one caller sees `OK` and emits, so two gateways racing + on concurrent fetches still produce a single event. +- **Without Redis** (not configured, as in single-node dev and unit tests), or if the Redis call throws, an + in-process `Set` provides the same semantics for that process only. It is not shared across nodes, so a + multi-node deployment running without Redis may signal once per node, and the set is lost on restart. +- **Emit:** only the caller that acquired the latch emits. Everyone else returns silently. + +### What releases the latch + +| Event | Where | +| ----------------------------------------------------------------------------------------- | --------------------------------- | +| A `signalPrekeysLowIfNeeded` call that sees `remaining >= threshold` | `prekeyLowSignal.ts` | +| A successful `POST /devices/:id/prekeys` after which the recounted pool is `>= threshold` | `routes/devices.ts` | +| Device revocation (its prekeys are deleted, so there is nothing left to replenish) | `revokeDeviceRow` in `devices.ts` | +| The 30-day TTL expiring (a device that has ignored the signal this long is told again) | Redis | + +Releasing deletes the Redis key and removes the id from the in-process set. A device that replenishes only +partway (for example from 5 to 15, still under 20) keeps the latch and is **not** signalled again until it +climbs back to the threshold and drops under it again, or the TTL passes. The upload route recounts +rather than adding up `current + uploaded`, because duplicate `keyId`s are dropped by `ON CONFLICT DO NOTHING`. + +## The `prekeys_low` event + +Delivered to the Socket.IO room `device:`, which each socket joins on connect +(`index.ts`). Only that device's own sockets get it, on whichever gateway holds the connection (via the +Redis adapter). Other devices on the same account do not, since only the owning device holds the private +keys it can replenish. + +```json +{ + "deviceId": "b6b6c3b0-...", + "oneTimePreKeysRemaining": 19, + "threshold": 20 +} +``` + +It is a plain server push: it is not an envelope, carries no `eventId`, and is not acknowledged. + +### What the client should do + +Upload a fresh batch with `POST /devices/:id/prekeys`, which takes `signedPreKey` and a non-empty +`oneTimePreKeys` array (see the [prekey upload contract](api-devices.md#prekey-upload-contract)): + +- The server enforces a **cap of 200 unconsumed one-time prekeys per device** (`OTP_CAP`). An upload is + trimmed to the remaining room (`200 - current`) and the response says so with `capped: true`; if the device is + already at 200 the request fails with `422`. +- The server sets **no batch-size limit** other than that cap and requires at least one key. A client should + generate enough to refill to the cap, `200 - oneTimePreKeysRemaining` keys, using new `keyId`s that do not + collide with ones already stored, and send a fresh signed prekey signed by the identity key (the endpoint + verifies the signature). +- Uploading fewer keys than needed to reach the threshold leaves the latch set and the device quiet, so top up + well above `threshold`, not just to it. +- The `200` response returns `oneTimePreKeysRemaining`, the recount after the insert, so the client can confirm + it is back above the threshold. + +At the time of writing, no client code in this repository listens for `prekeys_low`; the contract above is +what the server expects. + +## Offline devices and reconnecting + +Socket.IO delivers `prekeys_low` only to sockets present in the room at the moment of the emit. It is **not +queued, buffered or replayed**. A device that is offline when the latch is acquired never receives the event, +and because the latch stays set, the server will not send another until it is released (replenish to the +threshold, revocation, or the 30-day TTL). + +Therefore the event is a hint, not the source of truth. A client should check its own count: + +1. On every (re)connect, and on app start, call `GET /devices` and read `oneTimePreKeysRemaining` for the entry + with `current: true` (the count of unconsumed OPKs for that device). +2. If it is below the threshold (use 20, or the `threshold` from the last event), generate and upload a batch as + above. +3. Keep listening for `prekeys_low` while connected, to react before the next reconnect. + +Uploading releases the latch, which re-arms the signal for the next crossing. diff --git a/apps/backend/docs/contracts-key-material.md b/apps/backend/docs/contracts-key-material.md new file mode 100644 index 0000000..305b981 --- /dev/null +++ b/apps/backend/docs/contracts-key-material.md @@ -0,0 +1,194 @@ +# Public key material: encoding and validation + +Source: [`src/lib/keys.ts`](../src/lib/keys.ts) (the single validator), +[`src/schemas/auth.schemas.ts`](../src/schemas/auth.schemas.ts), +[`src/routes/devices.ts`](../src/routes/devices.ts), +[`apps/web/src/lib/x3dh.ts`](../../web/src/lib/x3dh.ts) (reference client). + +This document gives the exact encodings and byte lengths the server accepts. The shape of each request is in the +[REST schema reference](contracts-rest-schemas.md); only key formats are covered here. The server stores and +verifies **public** material only: every schema below is `.strict()`, so a stray `privateKey` or `sessionState` +field fails validation instead of being dropped (see [crypto invariants](concepts-crypto-invariants.md)). + +## Encoding rules (all key types) + +- Standard base64 (alphabet `A-Za-z0-9+/`), **padded**. `isValidBase64` requires `^[A-Za-z0-9+/]*={0,2}$` and a + length that is a multiple of 4. Base64url (`-`, `_`), unpadded and whitespace-containing values are rejected. +- Length is checked on the **decoded** bytes, computed from the string length and padding. +- A bad value returns `400 { "error": "Validation failed", "issues": [{ "field": "...", "message": "..." }] }`. + +## Accepted key types + +| Key type | JSON field | Encoding | Form | Decoded length | Accepted at | +| --------------- | ----------------------------------------------------- | -------- | ------------------------------- | -------------------------------- | ---------------------------------------------------------- | +| Identity key | `identityPublicKey` (also `device.identityPublicKey`) | base64 | Ed25519, **SPKI DER** | **44** (`ED25519_SPKI_BYTES`) | `POST /auth/verify`, device registration | +| Signed prekey | `signedPreKey.publicKey` | base64 | X25519, **raw** | **32** (`ED25519_RAW_KEY_BYTES`) | `POST /devices/:id/prekeys` | +| One-time prekey | `oneTimePreKeys[].publicKey` | base64 | X25519, **raw** | **32** | `POST /devices/:id/prekeys` | +| Signature | `signedPreKey.signature` | base64 | Ed25519 signature, raw | **64** (`ED25519_SIG_BYTES`) | `POST /devices/:id/prekeys` | +| MLS key package | `keyPackages[].keyPackage` | base64 | TLS-serialised MLS `KeyPackage` | **32 to 4096** (inclusive) | `POST /devices/:id/mls-key-packages` (max 100 per request) | + +Notes: + +- The constant name `ED25519_RAW_KEY_BYTES` is reused for prekeys, but the prekeys are **X25519** public keys. Both + are 32 bytes; only the identity key is an Ed25519 key. +- `keyId` is a non-negative integer (`z.number().int().nonnegative()`). +- For MLS the server cannot parse a `KeyPackage` without an MLS implementation, so the size window is the only + structural check. The endpoint adds device-ownership checks and rejects an `expiresAt` that is not in the future + with `400 { "error": "expiresAt must be in the future" }`. See [MLS key packages](mls-key-packages.md). +- `POST /auth/verify` requires `device.identityPublicKey` to equal the top-level `identityPublicKey`. + +## Raw versus SPKI + +Node's `crypto.verify` needs the identity key as SPKI DER, so that is what the server stores and what clients +send. The web client works with raw 32-byte Ed25519 keys internally (`@noble/curves`), and converts at the wire +boundary in `apps/web/src/lib/x3dh.ts`: + +| Function | Does | +| --------------------------------- | ---------------------------------------------------------------------------------------------- | +| `rawEd25519PublicKeyToSpki(raw)` | prepends the 12-byte RFC 8410 header `30 2a 30 05 06 03 2b 65 70 03 21 00` to the 32 raw bytes | +| `spkiToRawEd25519PublicKey(spki)` | throws unless the input is exactly 44 bytes (`12 + 32`), then returns the last 32 bytes | + +`toBase64` / `fromBase64` in the same file wrap `btoa` / `atob` (standard, padded). The client uses +`spkiToRawEd25519PublicKey` on a bundle's `identityPublicKey` before verifying the signed prekey or deriving X25519 +keys; the prekeys in a bundle are already raw and need no conversion. + +## What the signed-prekey signature covers + +On `POST /devices/:id/prekeys` the server calls +`verifyEd25519Signature(device.identityPublicKey, signedPreKey.publicKey, signedPreKey.signature)`, which: + +1. Base64-decodes the stored identity key (44-byte SPKI DER). +2. Base64-decodes `signedPreKey.publicKey` (32 raw bytes). **These bytes are the signed message.** +3. Base64-decodes the signature (64 bytes). +4. Runs `crypto.verify(null, message, { key, format: 'der', type: 'spki' }, signature)`, a pure Ed25519 verification + (no pre-hash). + +So the signature is over the **32 raw bytes of the signed prekey's public key only**. It does not cover `keyId`, a +prefix, or the base64 text. The client produces it with `ed25519.sign(publicKey, identity.privateKey)` +(`generateSignedPreKey`). A verification failure, or any exception, returns +`400 { "error": "Signed prekey signature is invalid" }`. The verifier is whichever identity key is registered +for the device in the URL, not one supplied in the body. Initiators repeat the check on the fetched bundle before +starting X3DH (`initiateSession`). + +## Examples + +The valid values below come from one real keypair: the signature is a real Ed25519 signature by the identity key +over the signed prekey bytes, so the combined payload passes the signature check too. + +### Identity key (44-byte SPKI) + +Valid: + +```json +{ "identityPublicKey": "MCowBQYDK2VwAyEAH6JwScWbg0BzKfoxkggh2dgC9gpQkcAsvbDNGLEQ+jo=" } +``` + +Rejected, because a raw 32-byte key was sent where SPKI is required: + +```json +{ "identityPublicKey": "H6JwScWbg0BzKfoxkggh2dgC9gpQkcAsvbDNGLEQ+jo=" } +``` + +```text +identityPublicKey must be 44 bytes (got 32) +``` + +### Signed prekey (32-byte raw key + 64-byte signature) + +Valid: + +```json +{ + "signedPreKey": { + "keyId": 1, + "publicKey": "x645YvSqiDkxrn0ZnWSXZbugQa1pVtGEa1/PpTrgUVU=", + "signature": "0k3GQRAthm1VjJEUuMyvGep1tG1Mi3QphlopRmyDK48K+Aa8I0kXICubLUUtBkZ0TYhNrjdnwfl5Lgv0v5OuAg==" + } +} +``` + +Rejected, because the signature is 32 bytes instead of 64 (a truncated value): + +```json +{ + "signedPreKey": { + "keyId": 1, + "publicKey": "x645YvSqiDkxrn0ZnWSXZbugQa1pVtGEa1/PpTrgUVU=", + "signature": "x645YvSqiDkxrn0ZnWSXZbugQa1pVtGEa1/PpTrgUVU=" + } +} +``` + +```text +signature must be 64 bytes (got 32) +``` + +Rejected at the signature step (`400 Signed prekey signature is invalid`): a well-formed 64-byte signature made by +a different identity key, or over different bytes (for example the base64 text, or `keyId || publicKey`). + +### One-time prekey (32-byte raw key) + +Valid: + +```json +{ + "oneTimePreKeys": [{ "keyId": 100, "publicKey": "x645YvSqiDkxrn0ZnWSXZbugQa1pVtGEa1/PpTrgUVU=" }] +} +``` + +Rejected, because the SPKI-wrapped form was sent (44 bytes): + +```json +{ + "oneTimePreKeys": [ + { "keyId": 100, "publicKey": "MCowBQYDK2VwAyEAH6JwScWbg0BzKfoxkggh2dgC9gpQkcAsvbDNGLEQ+jo=" } + ] +} +``` + +```text +publicKey must be 32 bytes (got 44) +``` + +Also rejected: any extra field, for example `{ "keyId": 100, "publicKey": "...", "privateKey": "..." }` fails the +`.strict()` object check with `Validation failed`. + +### Signature (64 bytes) + +Valid: `0k3GQRAthm1VjJEUuMyvGep1tG1Mi3QphlopRmyDK48K+Aa8I0kXICubLUUtBkZ0TYhNrjdnwfl5Lgv0v5OuAg==` + +Rejected, because it is not valid base64 (base64url alphabet and no padding): + +```text +0k3GQRAthm1VjJEUuMyvGep1tG1Mi3QphlopRmyDK48K-Aa8I0kXICubLUUtBkZ0TYhNrjdnwfl5Lgv0v5OuAg +``` + +```text +signature must be valid base64 +``` + +### MLS key package (32 to 4096 bytes) + +Valid, for a 48-byte package (placeholder bytes; the server only checks size): + +```json +{ + "cipherSuite": 1, + "keyPackages": [ + { "keyPackage": "BwcHBwcHBwcHBwcHBwcHBwcHBwcHBwcHBwcHBwcHBwcHBwcHBwcHBwcHBwcHBwcH" } + ] +} +``` + +Rejected, because the decoded package is 16 bytes (below the 32-byte minimum): + +```json +{ "cipherSuite": 1, "keyPackages": [{ "keyPackage": "BwcHBwcHBwcHBwcHBwcHBw==" }] } +``` + +```text +keyPackage must be 32–4096 bytes (got 16) +``` + +An empty string fails earlier with `keyPackage is required`, and anything above 4096 decoded bytes fails with the +same range message. diff --git a/apps/backend/docs/howto-add-socket-event.md b/apps/backend/docs/howto-add-socket-event.md new file mode 100644 index 0000000..9ed1ad7 --- /dev/null +++ b/apps/backend/docs/howto-add-socket-event.md @@ -0,0 +1,236 @@ +# How to add a socket event end-to-end + +Source: [`src/lib/eventEnvelope.ts`](../src/lib/eventEnvelope.ts), +[`src/socket/dispatcher.ts`](../src/socket/dispatcher.ts), +[`src/socket/messaging.ts`](../src/socket/messaging.ts), +[`src/config/rateLimits.ts`](../src/config/rateLimits.ts), +[`src/index.ts`](../src/index.ts) (per-socket middleware). + +Adding an inbound event touches several places, and **missing one fails silently**. An event that is not in +`KNOWN_EVENT_TYPES` is rejected with an error. An event that is known but has no registered handler is dropped +with only a server-side `console.warn`: the client gets no error, no `dispatch_ack`, nothing. There is no raw +`socket.on(type, ...)` fallback (#342), so a bare `socket.emit('my_event', ...)` never reaches a handler either. + +## Checklist + +Work through these in order. + +1. **Register the type** in `KNOWN_EVENT_TYPES` in `src/lib/eventEnvelope.ts`. Put inbound (client to server) types + under the inbound comment and outbound ones under the outbound comment. Without this the dispatcher answers with + `Unknown event type: ` and never looks for a handler. +2. **Write and register the handler** inside `registerMessagingHandlers` in `src/socket/messaging.ts` with + `dispatcher.register('', async (payload) => { ... })`. Register before the `dispatcher.listen()` call at + the end of the function. A handler that is never registered is the silent-drop case above. +3. **Run the payload invariant checks** as the first statement of the handler if it carries a client payload that + could hold key or session material (see [Invariants](#invariants-every-payload-carrying-handler-must-run)). +4. **Map the event to a rate-limit bucket** in `SOCKET_EVENT_BUCKETS` in `src/config/rateLimits.ts` if it should not use + `socket_default`, adding a new bucket to `RATE_LIMIT_DEFAULTS` when none fits (see + [Rate limiting](#rate-limiting-and-the-socket_event_buckets-mapping)). +5. **Emit errors** in the shape the [error catalog](contracts-error-catalog.md#socket-errors-come-in-two-shapes) documents. +6. **Add a unit test** with the dispatcher harness (see [Testing](#testing-the-handler)). +7. **Update the docs** (see [Update the docs](#update-the-docs)). + +If the event is outbound only (server to client), step 1 is still required so the registry stays the single source of +truth, but there is no handler to register. Emit it with `socket.emit(...)` / `io.to(room).emit(...)` or, for the +enveloped form, `dispatcher.emit(type, payload)`. + +## Worked example: `star_message` + +A client marks a message as starred. Minimal, for illustration only. + +**1. `src/lib/eventEnvelope.ts`** + +```ts +export const KNOWN_EVENT_TYPES = new Set([ + // Inbound (client → server) + 'auth', + // ...existing types... + 'star_message', +``` + +**2 and 3. `src/socket/messaging.ts`**, next to the other `dispatcher.register` calls: + +```ts +dispatcher.register('star_message', async (payload) => { + const forbiddenField = findForbiddenSessionStateField(payload); + if (forbiddenField) { + socket.emit('error', { + event: 'star_message', + code: 400, + message: `Field "${forbiddenField}" is not permitted: the server never stores session or private-key state`, + }); + return; + } + + const { conversationId, messageId } = payload as { conversationId?: string; messageId?: string }; + if (!conversationId?.trim() || !messageId?.trim()) { + socket.emit('error', { + event: 'star_message', + message: 'conversationId and messageId are required', + }); + return; + } + + const membership = await db.query.conversationMembers.findFirst({ + where: and( + eq(conversationMembers.conversationId, conversationId), + eq(conversationMembers.userId, userId), + ), + }); + if (!membership) { + socket.emit('error', { event: 'star_message', message: 'Not a member of this conversation' }); + return; + } + + // ...persist the star... + socket.emit('star_message_ack', { conversationId, messageId }); +}); +``` + +`star_message_ack` would also need to be added to `KNOWN_EVENT_TYPES`, as an outbound type. + +**4. `src/config/rateLimits.ts`** + +```ts +const SOCKET_EVENT_BUCKETS: Record = { + send_message: 'socket_send_message', + // ... + star_message: 'socket_typing', // pick, or add, a bucket whose budget fits +}; +``` + +## What the dispatcher does before a handler runs + +Clients send `socket.emit('dispatch', envelope)`. `EventDispatcher.listen()` runs these checks in order and stops at +the first failure. A failed check returns before any handler code executes. Failures 1 to 4 emit an enveloped +`error` (Shape B, see the [error catalog](contracts-error-catalog.md)). + +| # | Check | Detail | On failure | +| --- | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- | +| 0 | Authenticated socket | `socket.auth` must be set | `error` payload `Unauthenticated` | +| 1 | Envelope schema | `EventEnvelopeSchema`: `eventId` non-empty string, `type` non-empty string, `timestamp` positive integer, `payload` an object (defaults to `{}`) | `Malformed envelope` with Zod `details` | +| 2 | Known type | `type` must be in `KNOWN_EVENT_TYPES` | `Unknown event type: ` (also `console.warn`) | +| 3 | Timestamp freshness | `now - SOCKET_EVENT_MAX_AGE_MS <= timestamp <= now + SOCKET_EVENT_MAX_FUTURE_SKEW_MS`. Defaults: **300000 ms (300 s)** into the past, **30000 ms (30 s)** into the future | `Stale or invalid envelope timestamp` | +| 4 | Replay rejection | Redis `SET NX EX` on a key scoped to **`(deviceId, eventId)`**, TTL `REPLAY_PROTECTION_TTL_SECONDS` (default **300**, valid range 1 to 86400). Fails open when Redis is unavailable (or with no Redis client) | `dispatch_ack` `{ eventId, duplicate: true }`; the handler is **not** run | +| 5 | Handler lookup | `handlers.get(type)` | `console.warn` only, **silent drop** (no ack, no error) | +| 6 | Run handler | `await handler(payload)` | success: `dispatch_ack` `{ eventId, duplicate: false }` | + +Consequences when writing a handler: + +- Clients must generate a fresh `eventId` per emission and a current `Date.now()` `timestamp`. Reusing an `eventId` + within the TTL from the same device produces `duplicate: true` and does not re-run the handler, so retry logic should + reuse an `eventId` only when it is deliberately retrying the same event. +- The replay window should be at least as long as the freshness window to be a meaningful replay defence. With the + defaults both are 300 s. +- If the handler **throws**, the dispatcher only logs `handler error for ""` and swallows the exception. No + `dispatch_ack` is sent and no error is emitted to the client. A handler therefore has to catch its own failures and + emit the error itself (see [Errors](#emitting-errors)). Do not rely on a throw to tell the client something went wrong. +- Three checks happen **before** the dispatcher, in the per-socket middleware in `src/index.ts`, on every incoming + Socket.IO event: a revoked-device check, the payload size cap (`MAX_PAYLOAD_SIZE`, default 16384 bytes), and the + per-event rate limit. + +## Rate limiting and the `SOCKET_EVENT_BUCKETS` mapping + +`socketEventBucket(event)` returns the mapped bucket or `socket_default` (10 per second). The charge is made per +device (`device:`), shared across nodes through Redis. Existing buckets: `socket_default`, +`socket_send_message`, `socket_typing`, `socket_ask_assistant`. + +The limiter runs in `socket.use(...)`, which sees the **Socket.IO event name**, not the envelope `type`. For +enveloped traffic that name is always `dispatch`, so `socketEventBucket('dispatch')` currently resolves to +`socket_default` for every enveloped event, regardless of the `SOCKET_EVENT_BUCKETS` entry for its `type`. Keep the +mapping up to date anyway, because it documents the intended budget and applies if the limiter is moved to see the +envelope type, but do not assume it is the effective limit today. Confirm in `src/index.ts` before relying on a +specific bucket, and see [rate limits](../../../docs/security/rate-limits.md). + +## Invariants every payload-carrying handler must run + +The server relays ciphertext and public routing metadata only. See +[server-side crypto invariants](concepts-crypto-invariants.md). + +- **No session state.** Run `findForbiddenSessionStateField(payload)` from `lib/signalInvariants.ts` as the + **first statement**, before destructuring the payload, any membership check, or any `db` call. If it returns a name, + emit the error below and `return`. It checks the top level and every entry of `payload.envelopes`, and matches on + key presence, so `null` and empty values are still rejected. +- **Ciphertext only.** Never accept, store or log plaintext, private keys, ratchet or session state. Message bodies must + arrive as ciphertext, and the persistence path applies the ciphertext invariant (`lib/ciphertextInvariant.ts`). + Do not add fields to a payload that would carry anything else. +- **Size.** If the payload contains per-device envelopes, also use `checkEnvelopeSizes` from `services/rateLimit.ts` + (`MAX_ENVELOPE_SIZE`, default 4096 bytes per ciphertext), as `send_message` does. +- Do not log payload contents; log ids only. + +A handler with no client-supplied payload (such as `heartbeat`) does not need the invariant checks. + +## Emitting errors + +Errors from a handler use the **bare** Shape A documented in the +[error catalog](contracts-error-catalog.md#socket-errors-come-in-two-shapes): an `error` event whose payload names the +failing client event. + +```ts +socket.emit('error', { + event: 'star_message', // the client event that failed + message: 'Not a member of this conversation', // always present + code: 400, // optional; see the catalog for the codes used +}); +``` + +Rules: + +- Always include `event` and `message`. Add `code` only for the cases the catalog already defines it for, such as the + `400` for forbidden session-state fields. +- Reuse existing `message` strings for the same failure (`Not a member of this conversation`, `Invalid conversationId`) + so clients can handle them uniformly. +- Do not use `createEnvelope('error', ...)` in a handler. That is Shape B and belongs to the dispatcher layer. +- `return` after emitting, and never throw to signal a client error (see above). +- Add any new error string to the catalog. + +## Testing the handler + +Dispatcher behaviour has its own harness in [`src/__tests__/dispatcher.test.ts`](../src/__tests__/dispatcher.test.ts) +(and `src/socket/dispatcher.spec.ts`, which uses `ioredis-mock`). The helpers are local to those files; copy them +into a new test file: + +- `makeSocket(auth)` builds an `EventEmitter`-backed fake `AuthSocket` with a mocked `emit`, `join`, `disconnect`, an + `emitted` array recording every emit, and a `trigger(event, data)` that fires listeners through the real emitter. +- `makeIo()` builds a minimal `Server` double and `makeRedis('OK' | null)` a stub whose `set` result drives the replay check. + +```ts +import { describe, it, expect, vi } from 'vitest'; +import { EventDispatcher } from '../socket/dispatcher.js'; + +it('runs the star_message handler through the envelope path', async () => { + const { socket, trigger } = makeSocket(); + const dispatcher = new EventDispatcher(makeIo(), socket, makeRedis('OK') as never); + const handler = vi.fn().mockResolvedValue(undefined); + + dispatcher.register('star_message', handler); + dispatcher.listen(); + + trigger('dispatch', { + eventId: 'evt-1', + type: 'star_message', + timestamp: Date.now(), + payload: { conversationId: 'c1', messageId: 'm1' }, + }); + + await new Promise((r) => setTimeout(r, 10)); + expect(handler).toHaveBeenCalledWith({ conversationId: 'c1', messageId: 'm1' }); +}); +``` + +Test handler logic (not just routing) by calling `registerMessagingHandlers(io, socket)` with `vi.mock`ed `db`, as +[`signalInvariants.socket.test.ts`](../src/__tests__/signalInvariants.socket.test.ts) does, then `trigger('dispatch', ...)` and +assert on `emitted`. Cover at least: the happy path, the forbidden-field rejection (and that no `db` call happened), the +membership failure, and a `duplicate: true` replay. Also assert the type is in `KNOWN_EVENT_TYPES` so step 1 cannot be forgotten. Run with +`pnpm --filter backend test`. + +## Update the docs + +Finally, document the event so clients can use it: + +- Add it to the **WebSocket event catalog**, [`api-websocket-events.md`](api-websocket-events.md) (inbound or outbound + section, plus the table of contents). +- Add its payload schema to the **payload reference**, [`contracts-websocket-payloads.md`](contracts-websocket-payloads.md), + including the `KNOWN_EVENT_TYPES` list in section 2. +- Add any new error strings to [`contracts-error-catalog.md`](contracts-error-catalog.md), and any new bucket to the + [rate limit docs](../../../docs/security/rate-limits.md).