Merge #48: restore Omni frame/first+last/reference on the batch API

CLAUDE.md

@@ -30,11 +30,13 @@ that change how you work:
 
 - **Projects are not created by Flow Kit any more.** Make one in the Flow UI and
   pin its uuid as `FLOW_PROJECT_ID`, or pass `flow_project_id` to `POST /api/projects`.
-- **Four capabilities are unported** because their payloads were never captured:
-  4K upscale, r2v, start+end-frame chaining, and Omni Flash. They fail with
+- **Three capabilities are unported**, all on the Veo path, because their
+  payloads were never captured: **video** upscale (not image export, which
+  works), Veo r2v, and Veo start+end-frame chaining. They fail with
   `UNSUPPORTED_ON_BATCH_API` rather than silently producing the wrong thing.
-  `FLOW_ALLOW_DEGRADED=1` drops chaining and r2v to plain i2v; upscale has no
-  fallback. To restore one properly, see `docs/CAPTURE.md`.
+  Omni covers frame, first+last and reference modes — use
+  `model_family=omni_flash`. `FLOW_ALLOW_DEGRADED=1` drops Veo chaining and r2v
+  to plain i2v; video upscale has no fallback. See `docs/CAPTURE.md`.
 - **A poll saying "Media not found." is not a failure.** Finished jobs report it.
 
 ## Skills

README.md

@@ -24,12 +24,15 @@
 > from an older Flow Kit: reload the extension (v0.3.2+) and pin
 > `FLOW_PROJECT_ID`; Flow Kit can no longer create the project for you.
 >
-> Four capabilities are still unported because their payloads were never
-> captured off the new UI — **video upscale**, **reference-to-video (r2v)**,
-> **start+end-frame chaining**, and **Omni Flash frame/reference**. They fail
-> loudly with `UNSUPPORTED_ON_BATCH_API` instead of quietly producing the wrong
-> thing. `FLOW_ALLOW_DEGRADED=1` drops chaining and r2v to plain i2v; video
-> upscale has no fallback. To restore one properly see [`docs/CAPTURE.md`](docs/CAPTURE.md).
+> Every Omni 1.1 Flash video mode is ported and live-verified: text-to-video,
+> first frame, first+last frame, and references. Three capabilities remain
+> unported on the **Veo** path because their payloads were never captured off
+> the new UI — **video upscale**, **Veo reference-to-video**, and **Veo
+> start+end-frame chaining**. They fail loudly with `UNSUPPORTED_ON_BATCH_API`
+> instead of quietly producing the wrong thing. For the latter two, Omni covers
+> the same shot with `model_family=omni_flash`, or `FLOW_ALLOW_DEGRADED=1` drops
+> them to plain i2v; video upscale has no fallback. To restore one properly see
+> [`docs/CAPTURE.md`](docs/CAPTURE.md).
 
 ---
 
@@ -276,10 +279,10 @@ Three capabilities have no captured payload, so they fail with
 | Capability | Status | Workaround |
 |---|---|---|
 | 4K/1080p upscale (`/fk-pipeline` last step) | unported | none — keep the 1080p render |
-| Reference-to-video (r2v) | unported | `FLOW_ALLOW_DEGRADED=1` → i2v off the first reference |
-| Start+end-frame chaining (`/fk-gen-chain-videos`) | unported | `FLOW_ALLOW_DEGRADED=1` → i2v off the start frame |
+| Veo reference-to-video (r2v) | unported | Omni r2v (`model_family=omni_flash`), or `FLOW_ALLOW_DEGRADED=1` → i2v off the first reference |
+| Veo start+end-frame chaining (`/fk-gen-chain-videos`) | unported | Omni first+last (`model_family=omni_flash`), or `FLOW_ALLOW_DEGRADED=1` → i2v off the start frame |
 | Omni Flash text-to-video | ported | `POST /api/flow/generate-video-omni-text` (4/6/8/10s) |
-| Omni Flash frame/reference modes | unported | use Veo or text-to-video until their batch payloads are captured |
+| Omni Flash frame / first+last / reference modes | ported | `eb1hJf`, `nprQif`, `MZZa6b` — `POST /api/flow/generate-video` with `model_family=omni_flash` |
 
 Restoring one starts with a capture, not a guess: [`docs/CAPTURE.md`](docs/CAPTURE.md).
 

agent/api/flow.py

@@ -39,6 +39,7 @@ class GenerateVideoRequest(BaseModel):
     # Backward compatible: legacy requests remain Veo unless explicitly set.
     model_family: Literal["veo", "omni_flash"] = "veo"
     duration_s: int = 8
+    resolution: Literal["360p", "720p"] = "720p"
 
 
 class GenerateVideoRefsRequest(BaseModel):
@@ -52,6 +53,7 @@ class GenerateVideoRefsRequest(BaseModel):
     # explicitly opt into Omni Flash.
     model_family: Literal["veo", "omni_flash"] = "veo"
     duration_s: int = 8
+    resolution: Literal["360p", "720p"] = "720p"
 
 
 class GenerateOmniFlashVideoRequest(BaseModel):
@@ -60,6 +62,7 @@ class GenerateOmniFlashVideoRequest(BaseModel):
     project_id: str
     scene_id: str = ""
     duration_s: int = 8
+    resolution: Literal["360p", "720p"] = "720p"
     aspect_ratio: str = "VIDEO_ASPECT_RATIO_PORTRAIT"
     user_paygate_tier: str = "PAYGATE_TIER_ONE"
 
@@ -175,8 +178,8 @@ async def generate_video(body: GenerateVideoRequest):
     request uses Omni First frame. When ``end_image_media_id`` is also present,
     it uses Omni First+Last frames.
 
-    Omni responses include ``flowkitPolling.workflows`` and must use workflow
-    media polling rather than legacy operation polling.
+    On the migrated batch transport, Omni frame-conditioned responses return
+    ``flowkitPolling.mode=batch_operation`` and are polled through ``/check-status``.
     """
     client = get_flow_client()
     if not client.connected:
@@ -190,6 +193,7 @@ async def generate_video(body: GenerateVideoRequest):
                 project_id=body.project_id,
                 scene_id=body.scene_id,
                 duration_s=body.duration_s,
+                resolution=body.resolution,
                 aspect_ratio=body.aspect_ratio,
                 user_paygate_tier=body.user_paygate_tier,
             )
@@ -204,7 +208,7 @@ async def generate_video(body: GenerateVideoRequest):
             raise HTTPException(400, str(exc)) from exc
     else:
         result = await client.generate_video(
-            **body.model_dump(exclude={"model_family", "duration_s"}, exclude_none=True)
+            **body.model_dump(exclude={"model_family", "duration_s", "resolution"}, exclude_none=True)
         )
 
     if result.get("error") or (isinstance(result.get("status"), int) and result["status"] >= 400):
@@ -218,8 +222,8 @@ async def generate_video_refs(body: GenerateVideoRefsRequest):
 
     Existing requests default to ``model_family=veo``. Set
     ``model_family=omni_flash`` and ``duration_s`` to 4/6/8/10 to use Omni.
-    Omni responses include ``flowkitPolling.workflows``; poll those workflows,
-    not the operation-looking handles in the raw Flow response.
+    Migrated Omni Ingredients/R2V returns ``flowkitPolling.mode=batch_operation``;
+    poll its operations through ``/check-status``.
     """
     client = get_flow_client()
     if not client.connected:
@@ -233,14 +237,15 @@ async def generate_video_refs(body: GenerateVideoRefsRequest):
                 project_id=body.project_id,
                 scene_id=body.scene_id,
                 duration_s=body.duration_s,
+                resolution=body.resolution,
                 aspect_ratio=body.aspect_ratio,
                 user_paygate_tier=body.user_paygate_tier,
             )
         except ValueError as exc:
             raise HTTPException(400, str(exc)) from exc
     else:
         result = await client.generate_video_from_references(
-            **body.model_dump(exclude={"model_family", "duration_s"})
+            **body.model_dump(exclude={"model_family", "duration_s", "resolution"})
         )
 
     if result.get("error") or (isinstance(result.get("status"), int) and result["status"] >= 400):

agent/services/flow_batch.py

@@ -37,6 +37,8 @@
 RPC_GEN_IMAGE = "ogiZ0b"
 RPC_GEN_VIDEO = "eb1hJf"
 RPC_GEN_VIDEO_TEXT = "YhhmEf"
+RPC_GEN_VIDEO_FIRST_LAST = "nprQif"
+RPC_GEN_VIDEO_REFERENCES = "MZZa6b"
 RPC_OPERATION = "jwpduf"
 RPC_PROJECT_MEDIA = "Zzl0ze"
 RPC_MEDIA = "as29s"
@@ -411,6 +413,101 @@ def video_request(prompt: str, project_id: str, source_media_id: str,
     return build_envelope(RPC_GEN_VIDEO, inner)
 
 
+def omni_first_frame_request(prompt: str, project_id: str, source_media_id: str,
+                             *, duration_s: int = 8, resolution: str = "720p",
+                             aspect: Any = VIDEO_ASPECT_LANDSCAPE,
+                             crop: Optional[list] = None) -> str:
+    """Build current Omni first-frame I2V (RPC ``eb1hJf``).
+
+    Live-captured from Flow on 2026-09-14. 720p uses ``abra_i2v_<N>s``;
+    360p appends ``_360p`` and carries the UI's low-resolution option slot.
+    """
+    if duration_s not in (4, 6, 8, 10):
+        raise ValueError("Omni duration must be 4, 6, 8 or 10 seconds")
+    res = str(resolution).strip().lower()
+    if res not in {"360p", "720p"}:
+        raise ValueError("Omni resolution must be 360p or 720p")
+    model = f"abra_i2v_{duration_s}s" + ("_360p" if res == "360p" else "")
+    request = [
+        [None, None, [[[prompt]]]],
+        model,
+        resolve_video_aspect(aspect),
+        None,
+        [None, source_media_id, None, None, None,
+         FULL_FRAME_CROP if crop is None else crop],
+        [None, None, None, None, _client_uuid(), _client_uuid()],
+    ]
+    if res == "360p":
+        request.extend([None, None, None, [4]])
+    return build_envelope(RPC_GEN_VIDEO, [
+        [request],
+        _context(project_id),
+        [_client_uuid(), 2],
+    ])
+
+
+def omni_first_last_request(prompt: str, project_id: str,
+                            start_media_id: str, end_media_id: str,
+                            *, duration_s: int = 8, resolution: str = "720p",
+                            aspect: Any = VIDEO_ASPECT_LANDSCAPE,
+                            start_crop: Optional[list] = None,
+                            end_crop: Optional[list] = None) -> str:
+    """Build Omni First+Last frames submit (RPC ``nprQif``)."""
+    if duration_s not in (4, 6, 8, 10):
+        raise ValueError("Omni duration must be 4, 6, 8 or 10 seconds")
+    res = str(resolution).strip().lower()
+    if res not in {"360p", "720p"}:
+        raise ValueError("Omni resolution must be 360p or 720p")
+    model = f"omni_flash_i2v_{duration_s}s_first_last" + ("_360p" if res == "360p" else "")
+    request = [
+        [None, None, [[[prompt]]]],
+        model,
+        resolve_video_aspect(aspect),
+        None,
+        [None, start_media_id, None, None, None,
+         FULL_FRAME_CROP if start_crop is None else start_crop],
+        [None, end_media_id, None, None, None,
+         FULL_FRAME_CROP if end_crop is None else end_crop],
+        [None, None, None, None, _client_uuid(), _client_uuid()],
+    ]
+    return build_envelope(RPC_GEN_VIDEO_FIRST_LAST, [
+        [request],
+        _context(project_id),
+        [_client_uuid(), 2],
+    ])
+
+
+def omni_reference_video_request(prompt: str, project_id: str,
+                                 reference_media_ids: list[str],
+                                 *, duration_s: int = 8, resolution: str = "720p",
+                                 aspect: Any = VIDEO_ASPECT_LANDSCAPE) -> str:
+    """Build Omni Ingredients/reference-to-video submit (RPC ``MZZa6b``)."""
+    refs = [str(mid) for mid in reference_media_ids if str(mid)]
+    if not refs:
+        raise ValueError("Omni reference-to-video requires at least one reference image")
+    if duration_s not in (4, 6, 8, 10):
+        raise ValueError("Omni duration must be 4, 6, 8 or 10 seconds")
+    res = str(resolution).strip().lower()
+    if res not in {"360p", "720p"}:
+        raise ValueError("Omni resolution must be 360p or 720p")
+    model = f"abra_r2v_{duration_s}s" + ("_360p" if res == "360p" else "")
+    request = [
+        [None, None, [[[prompt]]]],
+        [[None, mid] for mid in refs],
+        model,
+        resolve_video_aspect(aspect),
+        None,
+        [None, None, None, None, _client_uuid(), _client_uuid()],
+    ]
+    if res == "360p":
+        request.extend([None, None, None, None, None, [4]])
+    return build_envelope(RPC_GEN_VIDEO_REFERENCES, [
+        [request],
+        _context(project_id),
+        [_client_uuid(), 2],
+    ])
+
+
 def text_video_request(prompt: str, project_id: str,
                        aspect: Any = VIDEO_ASPECT_LANDSCAPE,
                        model: str = "abra_t2v_4s") -> str: