Video API: allow frame_images and input_references in one request

[marcospgp]

The video generation cookbook documents that a request carrying both frame_images and input_references is routed as image-to-video and the references are dropped. Seedance 2.0 itself accepts a pinned first frame plus reference images together (BytePlus direct, fal.ai, Higgsfield all pass both), so the single-mode restriction loses a model capability rather than reflecting one.

Use case: a clip whose opening frame is an approved still, with character portraits as identity references for people entering after frame one.

Is the mutual exclusion an OpenRouter routing decision that could be lifted, or a provider-integration gap?

[tahabahrami]

Confirmed with paid POST /api/v1/videos requests on both Seedance variants. This is not a prompt, payload-size, or media-fetch issue; it is the documented mode precedence dropping a valid input.

Reproduction

Models:

• bytedance/seedance-2.0-fast — OpenRouter video job ClQ1uaF8S9CGu38on62v
• bytedance/seedance-2.0 — OpenRouter video job n1vcJpBfFTVYoxrF8dmU

Both requests used an empty-stage image as a first-frame anchor and a separate character plate that should appear only in the middle of the shot:

{
  "model": "bytedance/seedance-2.0-fast",
  "prompt": "[Image1] is the empty opening stage. [Image2] is the exact character identity; introduce it after the shot begins...",
  "duration": 10,
  "resolution": "480p",
  "aspect_ratio": "9:16",
  "frame_images": [
    {
      "type": "image_url",
      "image_url": { "url": "<same first-frame JPEG>" },
      "frame_type": "first_frame"
    }
  ],
  "input_references": [
    {
      "type": "image_url",
      "image_url": { "url": "<same character-reference JPEG>" }
    }
  ]
}

Actual result: HTTP submission and generation both succeeded and were billed, but the character reference was not followed. The model generated a visibly different creature. The same behavior occurred on Fast and Standard.

Controlled reference-only result

I changed only the routing shape: both images were placed in ordered input_references, frame_images was omitted, and the prompt bound [Image1] as opening composition and [Image2] as identity reference.

{
  "model": "bytedance/seedance-2.0-fast",
  "prompt": "[Image1] is the opening composition. [Image2] is an identity reference, not an opening or ending frame...",
  "duration": 4,
  "resolution": "480p",
  "aspect_ratio": "9:16",
  "input_references": [
    { "type": "image_url", "image_url": { "url": "<same first-frame JPEG>" } },
    { "type": "image_url", "image_url": { "url": "<same character-reference JPEG>" } }
  ]
}

OpenRouter video job: T6osAeVryjk4B9CiWJTX.

That output followed the character plate correctly. Human QC matched body shape, face, eyes, limbs, texture, and color. An objective middle-frame sample measured hue 325.714° / saturation 0.7778 against reference hue 324° / saturation 0.7143. The earlier mixed-field outputs measured around 350–352° and produced the wrong identity.

The exact same small JPEG bytes were used in both paths, so this also rules out URL accessibility and payload size. However, the reference-only control no longer had a hard first frame: the character was already entering at the opening edge. That confirms the practical capability loss described in this issue.

Requested behavior

1. For Seedance providers that support mixed multimodal controls, forward frame_images and input_references together instead of applying the global precedence rule.
2. If mixed controls cannot currently be honored, reject the request before generation with a 4xx explaining which field conflicts. Silently accepting, billing, and dropping input_references is especially difficult to detect.
3. Ideally expose the effective generation mode and any ignored fields in the submit response/status metadata.

Happy to provide additional job IDs or run a minimal provider-specific regression if the team needs one.