Port remaining demos to the FlashDreams application/session API
Summary
The newer public demo API is the flashdreams.demo application/session contract (IFlashDreamsApplication, IFlashDreamsApplicationSession, run_application) with host-owned MP4, null, local-window, and WebRTC I/O. The host invokes the unified runtime internally; application/session API coverage is the only migration status tracked by this issue.
Port the two remaining plain T2V integration families and add application adapters for the already-runtime-migrated OmniDreams and Lingbot demos. Do not include OmniDreams Interactive Drive in this issue.
Current status
| Area |
Application/session API |
Evidence |
| Cosmos Predict2 T2V |
Complete |
t2v-cosmos-predict2 application entry point |
| Causal Forcing T2V |
Complete |
t2v-causal-forcing application entry point |
| Self Forcing T2V |
Complete |
t2v-self-forcing application entry point |
| Wan 2.1 T2V |
Missing |
wan21-t2v-1.3b-480p has no application entry point |
| FastVideo CausalWan 2.2 T2V |
Missing |
fastvideo-causal-wan2.2-t2v-14b has no application entry point |
| OmniDreams demo (excluding Interactive Drive) |
Missing |
No flashdreams.applications entry point |
| Lingbot demo |
Missing |
No flashdreams.applications entry point |
The three existing T2V application packages are the intended reference: their integration owns a small t2v/app.py factory, depends on flashdreams-t2v, and registers a flashdreams.applications entry point.
Work
1. Add the missing plain T2V applications
- Add
t2v application factories and flashdreams.applications registrations for Wan 2.1 T2V and FastVideo CausalWan 2.2 T2V.
- Reuse
apps/t2v's T2VApplication / T2VApplicationSession where the model's pipeline contracts permit it; keep integration-specific defaults in the integration package.
- Seed defaults from the existing runner configs:
RUNNER_WAN21_T2V_1PT3B_480P and RUNNER_WAN22_T2V_14B.
- Preserve the current prompt, resolution, FPS, block-count, decoder layout, compile override, and checkpoint/pipeline behavior. The new applications must stream every autoregressive block through the shared output path rather than call the legacy runner's
run() method.
- Add these slugs to
apps/t2v/README.md and its integration-entry-point CPU coverage. Update the app's documented package-selection commands.
Suggested slugs: t2v-wan21 and t2v-fastvideo-causal-wan22 (confirm naming against the project's application-slug convention before landing).
2. Port the OmniDreams demo to an application adapter
- Add a dedicated OmniDreams
IFlashDreamsApplication and session implementation plus a flashdreams.applications entry point. Do not force it into the plain T2VApplication: it has world/scene conditioning and live controls, not prompt-only input.
- Implement the application/session adapter as the supported OmniDreams demo path. Reuse useful scenario, provider, model-session, and output logic from the current implementation, but do not retain a parallel direct demo adapter as part of the target architecture.
- Define the canonical input schema for the supported OmniDreams demo controls and map the application's canonical input windows to the current conditioning inputs. Preserve finite replay (MP4 and null) and WebRTC behavior, including model warmup and session lifecycle.
- Keep scope to the standalone OmniDreams replay/WebRTC demo. Exclude
omnidreams.interactive_drive, its local desktop loop, controller setup, and physics work.
- Register/document the application and replace or clearly demote the temporary
DemoApplication user-facing launch path only after feature and parity coverage is in place.
3. Port the Lingbot demo to an application adapter
- Add a dedicated Lingbot
IFlashDreamsApplication and session implementation plus a flashdreams.applications entry point. Lingbot is I2V with live camera pose controls, so it also should not use the prompt-only T2VApplication.
- Implement the application/session adapter as the supported Lingbot demo path. Reuse useful example-data setup, input mapping, provider, and session logic from the current implementation, while keeping first-frame image, prompt, camera poses, intrinsics, example selection, and context-parallel setup functional.
- Declare canonical controls for Lingbot's camera input and verify that their time windows produce the expected conditioning/pose sequence.
- Support bounded MP4/null replay and WebRTC through the shared application host, including the current warmup and reconnect/reset semantics.
- Register/document the application and retire the
lingbot-demo development shortcut from user-facing guidance only once the application path is covered.
Acceptance criteria
flashdreams-run discovers five T2V-style application slugs: the existing Cosmos Predict2, Causal Forcing, and Self Forcing applications plus Wan 2.1 and FastVideo CausalWan 2.2; the OmniDreams and Lingbot application slugs are also discoverable through the same flashdreams.applications mechanism.
- Each application owns its integration-specific factory and is independently installable with the minimum package selection documented in its README.
- CPU tests cover entry-point registration, factory construction, argument validation, canonical schema validation, session reset/lifecycle, and MP4/null adapter routing. Extend the existing
apps/t2v test style rather than adding untested registration-only code.
- GPU/manual smoke coverage runs one finite generation for every new T2V application and exercises WebRTC startup for OmniDreams and Lingbot. For the controlled demos, include short deterministic trace/fixture coverage for canonical-input-to-conditioning mapping.
- Update
apps/t2v/README.md, the two integration READMEs, relevant model docs, and runner/application slug inventories so the documented commands use the new API.
- No Interactive Drive files or behavior are changed as part of this work.
Notes for implementation
- Existing T2V applications expose one integration-level default, not every legacy runner preset. Keep that precedent unless product requirements call for preset selection; do not silently drop a supported variant.
- The application host already bridges applications onto the unified runtime internally. This port is therefore an API/ownership migration, not a request to replace the runtime engine or duplicate WebRTC/output infrastructure.
Port remaining demos to the FlashDreams application/session API
Summary
The newer public demo API is the
flashdreams.demoapplication/session contract (IFlashDreamsApplication,IFlashDreamsApplicationSession,run_application) with host-owned MP4, null, local-window, and WebRTC I/O. The host invokes the unified runtime internally; application/session API coverage is the only migration status tracked by this issue.Port the two remaining plain T2V integration families and add application adapters for the already-runtime-migrated OmniDreams and Lingbot demos. Do not include OmniDreams Interactive Drive in this issue.
Current status
t2v-cosmos-predict2application entry pointt2v-causal-forcingapplication entry pointt2v-self-forcingapplication entry pointwan21-t2v-1.3b-480phas no application entry pointfastvideo-causal-wan2.2-t2v-14bhas no application entry pointflashdreams.applicationsentry pointflashdreams.applicationsentry pointThe three existing T2V application packages are the intended reference: their integration owns a small
t2v/app.pyfactory, depends onflashdreams-t2v, and registers aflashdreams.applicationsentry point.Work
1. Add the missing plain T2V applications
t2vapplication factories andflashdreams.applicationsregistrations for Wan 2.1 T2V and FastVideo CausalWan 2.2 T2V.apps/t2v'sT2VApplication/T2VApplicationSessionwhere the model's pipeline contracts permit it; keep integration-specific defaults in the integration package.RUNNER_WAN21_T2V_1PT3B_480PandRUNNER_WAN22_T2V_14B.run()method.apps/t2v/README.mdand its integration-entry-point CPU coverage. Update the app's documented package-selection commands.Suggested slugs:
t2v-wan21andt2v-fastvideo-causal-wan22(confirm naming against the project's application-slug convention before landing).2. Port the OmniDreams demo to an application adapter
IFlashDreamsApplicationand session implementation plus aflashdreams.applicationsentry point. Do not force it into the plainT2VApplication: it has world/scene conditioning and live controls, not prompt-only input.omnidreams.interactive_drive, its local desktop loop, controller setup, and physics work.DemoApplicationuser-facing launch path only after feature and parity coverage is in place.3. Port the Lingbot demo to an application adapter
IFlashDreamsApplicationand session implementation plus aflashdreams.applicationsentry point. Lingbot is I2V with live camera pose controls, so it also should not use the prompt-onlyT2VApplication.lingbot-demodevelopment shortcut from user-facing guidance only once the application path is covered.Acceptance criteria
flashdreams-rundiscovers five T2V-style application slugs: the existing Cosmos Predict2, Causal Forcing, and Self Forcing applications plus Wan 2.1 and FastVideo CausalWan 2.2; the OmniDreams and Lingbot application slugs are also discoverable through the sameflashdreams.applicationsmechanism.apps/t2vtest style rather than adding untested registration-only code.apps/t2v/README.md, the two integration READMEs, relevant model docs, and runner/application slug inventories so the documented commands use the new API.Notes for implementation