Add muse2api media provider for muse.ai rendering

README.md

@@ -260,6 +260,13 @@ token. Step 3 is not optional: Flow's project-creation endpoint went with the
 migration, so without a pinned project every request fails `NO_FLOW_PROJECT`.
 You can also pass `flow_project_id` per project on `POST /api/projects`.
 
+**Using the Muse provider (`muse2api`)?** It needs a second service: the
+[muse2api](https://github.com/crisng95/muse2api) gateway, cloned from its own
+repo and started separately (default `http://127.0.0.1:18610`), with
+`MUSE2API_URL` / `MUSE2API_KEY` exported before step 4. The Flow tab and
+extension are not needed for muse2api requests. Full setup:
+[Muse Media Provider](#muse-media-provider-provider-muse2api).
+
 ### Configuration
 
 | Env var | Default | What it does |
@@ -270,6 +277,14 @@ You can also pass `flow_project_id` per project on `POST /api/projects`.
 | `MEDIA_PROVIDER` | `flow` | `flow` (Google Flow via extension) or `assistant` (route all generation to the AI assistant). |
 | `ASSISTANT_PROVIDER_TIMEOUT_S` | `1800` | How long a request waits for a worker to complete its provider job before failing. |
 | `ASSISTANT_PROVIDER_POLL_S` | `15` | How often to poll the provider-job row while waiting for completion. |
+| `MUSE2API_URL` | — | Base URL of a running [muse2api](https://github.com/crisng95/muse2api) gateway — a separate service you start from its own repo (see [Muse Media Provider](#muse-media-provider-provider-muse2api)). Setting it makes the `muse2api` provider available. |
+| `MUSE2API_KEY` | — | The gateway's `MUSE2API_API_KEY` (sent as `Authorization: Bearer`). |
+| `MUSE2API_IMAGE_MODEL` / `MUSE2API_VIDEO_MODEL` | `muse-image` / `muse-video` | Model ids sent to the gateway. |
+| `MUSE2API_VIDEO_SECONDS` | `8` | Clip length requested for i2v. |
+| `MUSE2API_TIMEOUT_S` | `1800` | Max wait for one image call or video task (covers the gateway's own account failover). |
+| `MUSE2API_POLL_S` | `5` | Video task poll interval. |
+| `MUSE2API_MAX_CONCURRENT` / `MUSE2API_COOLDOWN_S` | `2` / `0` | Worker throttling for this provider. |
+| `MUSE2API_ALLOW_DEGRADED` | `0` | `1` renders chained scenes and r2v as plain i2v (muse.ai takes a first frame only). |
 
 ### Assistant Media Provider (`MEDIA_PROVIDER=assistant`)
 
@@ -299,6 +314,101 @@ See `agent/worker/assistant_worker.py` for a reference worker implementing the
 worker side of the protocol. Video upscale is not supported on this provider and
 fails loudly.
 
+### Muse Media Provider (`provider: muse2api`)
+
+Renders through a [muse2api](https://github.com/crisng95/muse2api) gateway, which
+exposes the muse.ai web app as an OpenAI-compatible API:
+
+```
+Flow Kit worker ──> Muse2APIProvider ──HTTP──> muse2api service ──> muse.ai
+     (this repo, :8100)                      (separate repo, :18610)
+```
+
+> **muse2api is a separate service — Flow Kit does not ship or start it.**
+> Flow Kit only contains the client (`agent/services/muse2api_client.py`) and
+> the provider adapter. Before using `provider: muse2api` you must clone the
+> [muse2api repo](https://github.com/crisng95/muse2api), start its server, and
+> keep it running alongside the Flow Kit server. With no gateway running, every
+> muse2api request fails with `muse2api connection_error`.
+
+#### 1. Start the muse2api service (from its own repo)
+
+```bash
+git clone https://github.com/crisng95/muse2api.git
+cd muse2api
+python3 -m venv .venv && source .venv/bin/activate    # Python 3.10+
+pip install -e .
+cp .env.example .env
+```
+
+Edit `.env`:
+
+| Variable | Set to | Why |
+|----------|--------|-----|
+| `MUSE2API_DRIVER` | `browser` | The default `mock` returns fake media — fine for a dry run, useless for real renders. |
+| `MUSE2API_CHROMIUM_PATH` | path to Chromium/Chrome (or leave empty to auto-detect) | The browser driver drives muse.ai through Chromium. |
+| `MUSE2API_API_KEY` | any secret | Flow Kit sends it as `MUSE2API_KEY`. Left empty, one is generated into `data/api_key`. |
+| `MUSE2API_HOST` | `0.0.0.0` only if Flow Kit runs on another machine | Default `127.0.0.1` accepts local connections only. |
+
+Start it and leave it running (its own terminal, tmux, or a service manager):
+
+```bash
+python -m muse2api                         # → http://127.0.0.1:18610
+# or, from the same folder: docker compose up -d --build   (binds 0.0.0.0:18610)
+```
+
+Import at least one logged-in muse.ai account — the gateway renders nothing
+without one. Export the cookies `hatch_sess`, `hatch_gw`, `hatch_vml` and
+`hatch_native_auth_device` from a browser signed in to muse.ai:
+
+```bash
+KEY=$(cat data/api_key)   # or your MUSE2API_API_KEY
+curl http://127.0.0.1:18610/admin/accounts \
+  -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
+  -d '{"label":"acc1","cookies":{"hatch_sess":"...","hatch_gw":"...","hatch_vml":"...","hatch_native_auth_device":"..."}}'
+
+curl -s http://127.0.0.1:18610/readyz
+# {"ready": true, "driver": {...}, "accounts": {"available": 1, ...}}
+```
+
+`ready: false` or `accounts.available: 0` means Flow Kit requests will fail with
+`no_account_available` — fix the gateway first. Account management, renewal and
+the rest of the gateway's options are documented in the muse2api README.
+
+#### 2. Point Flow Kit at it
+
+Set these before starting the Flow Kit server (they are read once at startup —
+restart after changing them):
+
+```bash
+export MUSE2API_URL=http://127.0.0.1:18610          # where the gateway listens
+export MUSE2API_KEY=<the gateway's MUSE2API_API_KEY>
+export DEFAULT_PROVIDER=muse2api                    # optional: make it the default
+```
+
+Check that Flow Kit sees it:
+
+```bash
+curl -s http://127.0.0.1:8100/api/providers/status   # muse2api → "available": true
+```
+
+`available: true` only means `MUSE2API_URL` is set — Flow Kit does not probe the
+gateway. Use `/readyz` above to confirm the service itself is up.
+
+#### 3. Use it
+
+Pass `"provider": "muse2api"` on any request (or rely on `DEFAULT_PROVIDER`).
+Flow Kit calls `POST /v1/images/generations` for images and `POST /v1/videos` +
+`GET /v1/videos/{id}` for i2v, sends input frames inline as data URLs (so the
+gateway may run on another host), and downloads every result into
+`output/_shared/muse2api/` as a `file://` URL with a minted UUID `media_id` —
+so `/fk-refresh-urls` is never needed for it.
+
+Supports image and i2v; no edit, upscale or TTS, and no end frame or r2v unless
+`MUSE2API_ALLOW_DEGRADED=1`. Entity reference images are not sent (muse.ai takes
+none), so character consistency across scenes is weaker than on Flow.
+Troubleshooting lives in `/fk-provider` and `/fk-doctor`.
+
 ### Post-production (provider-neutral)
 
 Concat, finalize, and review work on clips from any provider — each clip is

agent/config.py

@@ -60,7 +60,7 @@
 # /api/provider-jobs. See agent/sdk/services/assistant_provider.py and
 # agent/worker/assistant_worker.py for the full protocol.
 # Default media provider for requests that don't specify one ("flow" |
-# "assistant" | any registered provider name). MEDIA_PROVIDER is kept as a
+# "assistant" | "muse2api" | any registered provider name). MEDIA_PROVIDER is kept as a
 # legacy alias — DEFAULT_PROVIDER wins if both are set.
 DEFAULT_PROVIDER = os.environ.get(
     "DEFAULT_PROVIDER", os.environ.get("MEDIA_PROVIDER", "flow")
@@ -71,6 +71,24 @@
 ASSISTANT_MAX_CONCURRENT = int(os.environ.get("ASSISTANT_MAX_CONCURRENT", "2"))
 ASSISTANT_COOLDOWN_S = float(os.environ.get("ASSISTANT_COOLDOWN_S", "0"))
 
+# "muse2api": render through a muse2api gateway (https://github.com/crisng95/muse2api),
+# which fronts the muse.ai web app with an OpenAI-compatible API. The provider
+# is registered always and becomes available once MUSE2API_URL is set; the key
+# is the gateway's own MUSE2API_API_KEY. See agent/sdk/services/muse2api_provider.py.
+MUSE2API_URL = os.environ.get("MUSE2API_URL", "").strip().rstrip("/")
+MUSE2API_KEY = os.environ.get("MUSE2API_KEY", "")
+MUSE2API_IMAGE_MODEL = os.environ.get("MUSE2API_IMAGE_MODEL", "muse-image")
+MUSE2API_VIDEO_MODEL = os.environ.get("MUSE2API_VIDEO_MODEL", "muse-video")
+MUSE2API_VIDEO_SECONDS = int(os.environ.get("MUSE2API_VIDEO_SECONDS", "8"))
+# Covers the gateway's own failover (up to 3 accounts x its 240s/600s timeouts).
+MUSE2API_TIMEOUT_S = float(os.environ.get("MUSE2API_TIMEOUT_S", "1800"))
+MUSE2API_POLL_S = float(os.environ.get("MUSE2API_POLL_S", "5"))
+MUSE2API_MAX_CONCURRENT = int(os.environ.get("MUSE2API_MAX_CONCURRENT", "2"))
+MUSE2API_COOLDOWN_S = float(os.environ.get("MUSE2API_COOLDOWN_S", "0"))
+# muse.ai takes a first frame only. With this on, chained scenes drop the end
+# frame and r2v renders as i2v; off, both fail loudly (same rule as Flow).
+MUSE2API_ALLOW_DEGRADED = os.environ.get("MUSE2API_ALLOW_DEGRADED", "0") == "1"
+
 # ─── Worker ──────────────────────────────────────────────────
 POLL_INTERVAL = int(os.environ.get("POLL_INTERVAL", "5"))
 VIDEO_POLL_INTERVAL = int(os.environ.get("VIDEO_POLL_INTERVAL", "10"))  # polling interval for video/upscale status

agent/db/schema.py

@@ -8,6 +8,11 @@
 
 _db_connection: aiosqlite.Connection | None = None
 _db_lock = asyncio.Lock()
+# Guards get_db()'s lazy connect. Separate from _db_lock because crud calls
+# get_db() while holding that one. Rebuilt per event loop: a lock that has
+# been waited on is bound to its loop, and pytest runs each test in a new one.
+_db_init_lock: asyncio.Lock | None = None
+_db_init_loop: asyncio.AbstractEventLoop | None = None
 
 SCHEMA = """
 CREATE TABLE IF NOT EXISTS character (
@@ -356,17 +361,38 @@ async def init_db():
     logger.info("Database initialized at %s", DB_PATH)
 
 
+def _get_db_init_lock() -> asyncio.Lock:
+    global _db_init_lock, _db_init_loop
+    loop = asyncio.get_running_loop()
+    if _db_init_lock is None or _db_init_loop is not loop:
+        _db_init_lock = asyncio.Lock()
+        _db_init_loop = loop
+    return _db_init_lock
+
+
 async def get_db() -> aiosqlite.Connection:
-    """Return the shared database connection, creating it if needed."""
+    """Return the shared database connection, creating it if needed.
+
+    The first two callers used to race here: both saw None, both connected,
+    the second overwrote the global, and the first then ran its PRAGMAs on
+    the second's connection while a statement was open on it — failing with
+    "database table is locked" and leaking a connection (whose worker thread
+    keeps the interpreter alive). Connect under a lock, set up a local, and
+    publish it only once it is ready.
+    """
     global _db_connection
-    if _db_connection is None:
-        _db_connection = await aiosqlite.connect(str(DB_PATH))
-        _db_connection.row_factory = aiosqlite.Row
-        await _db_connection.execute("PRAGMA journal_mode=WAL")
-        await _db_connection.execute("PRAGMA foreign_keys=ON")
-        # Force WAL checkpoint so this connection sees all committed writes
-        # from previous processes (e.g. after hot-reload)
-        await _db_connection.execute("PRAGMA wal_checkpoint(PASSIVE)")
+    if _db_connection is not None:
+        return _db_connection
+    async with _get_db_init_lock():
+        if _db_connection is None:
+            conn = await aiosqlite.connect(str(DB_PATH))
+            conn.row_factory = aiosqlite.Row
+            await conn.execute("PRAGMA journal_mode=WAL")
+            await conn.execute("PRAGMA foreign_keys=ON")
+            # Force WAL checkpoint so this connection sees all committed writes
+            # from previous processes (e.g. after hot-reload)
+            await conn.execute("PRAGMA wal_checkpoint(PASSIVE)")
+            _db_connection = conn
     return _db_connection