Note: this issue was drafted by Claude via back-and-forth with @njbrake. The reasoning and decisions are his; the prose is Claude's.
Context
#1333 moved the inbound media staging cache from an in-process Python dict to a disk + Postgres layout: bytes live on the deployment's persistent volume at `MEDIA_STAGING_BASE_DIR` (default `data/staged_media`), metadata lives in the `staged_media` table. This gives us 7-day retention that survives process restarts on a single-instance deployment.
It does not survive horizontal scaling. If we ever run more than one Clawbolt replica behind a load balancer, a webhook delivered to replica A writes bytes to A's local volume; a follow-up turn routed to replica B finds the DB row but cannot read the bytes (different volume).
When this becomes a problem
When we deploy to a multi-replica topology. Today Railway runs one instance so this is a deferred concern, not an active bug.
Options for the fix
- Postgres `BYTEA` on the existing `staged_media` row. Simplest: single source of truth, no new infrastructure. Adds DB load: a `stage()` call uploads several MB into a transaction, and reads pull them back. With the 50-per-user cap and 7-day TTL the table can grow substantially. Probably fine up to a few hundred active users.
- Object store (S3, GCS, R2). Bytes live in a bucket keyed by `(user_id, handle)`; DB row keeps the object URL instead of `disk_path`. Scales further, costs a small amount per request, adds a new dependency and credentials surface.
- Hybrid: keep disk-backed staging but co-locate the agent loop with the channel webhook (sticky routing by user_id). Avoids object-store cost but adds infra complexity (sharded sessions, drained-host handling).
(2) is the conventional answer; (1) buys time and is cheap.
Where the warning lives in code
- `backend/app/agent/media_staging.py` module docstring ("MULTI-REPLICA WARNING")
- `backend/app/config.py` next to the `media_staging_base_dir` setting
- `docs/self-host/configuration.md` in the "Inbound media staging" section
When this issue is fixed, those references should be updated.
Definition of done
- Bytes are reachable from any replica without depending on local disk.
- Migration plan: existing on-disk entries either drained (TTL is 7 days, so a window-deferred deploy works) or copied into the new backing store at boot.
- The warnings in the three locations above are removed.
Note: this issue was drafted by Claude via back-and-forth with @njbrake. The reasoning and decisions are his; the prose is Claude's.
Context
#1333 moved the inbound media staging cache from an in-process Python dict to a disk + Postgres layout: bytes live on the deployment's persistent volume at `MEDIA_STAGING_BASE_DIR` (default `data/staged_media`), metadata lives in the `staged_media` table. This gives us 7-day retention that survives process restarts on a single-instance deployment.
It does not survive horizontal scaling. If we ever run more than one Clawbolt replica behind a load balancer, a webhook delivered to replica A writes bytes to A's local volume; a follow-up turn routed to replica B finds the DB row but cannot read the bytes (different volume).
When this becomes a problem
When we deploy to a multi-replica topology. Today Railway runs one instance so this is a deferred concern, not an active bug.
Options for the fix
(2) is the conventional answer; (1) buys time and is cheap.
Where the warning lives in code
When this issue is fixed, those references should be updated.
Definition of done