feat(www): automate asset capture and refresh previews

.agents/skills/cliparr-www-assets/SKILL.md

@@ -1,95 +1,125 @@
 ---
 name: cliparr-www-assets
-description: Workflow for updating homepage marketing videos and static first-frame images. Includes ffmpeg and ImageMagick commands for extracting first frames and optimizing videos for web (MP4/H.264 and WebM/VP9) at specific resolutions for desktop and mobile.
+description: Automatically capture Cliparr's homepage hero and mobile editor videos and matching posters with Playwright, a disposable Jellyfin instance, and FFmpeg. Use when refreshing apps/www marketing previews or maintaining their capture workflow.
 ---
 
-# Cliparr Website Assets Workflow
+# Cliparr Website Asset Capture
 
-Use this skill when you need to replace the preview videos or static screenshots on the Cliparr homepage (`apps/www`).
+Use the repository capture pipeline rather than manually recording the desktop.
+The source media plays through Jellyfin and Cliparr's real media adapter. Playwright
+controls the editor through its capture-only bridge; no mouse interaction is needed.
 
-## Assets and Resolutions
+## Inputs and scene configuration
 
-| Asset Type  | Resolution | Image Path                                    | Video Paths                                                                         |
-| :---------- | :--------- | :-------------------------------------------- | :---------------------------------------------------------------------------------- |
-| **Desktop** | 1600x886   | `apps/www/src/assets/screenshot.webp`         | `apps/www/public/preview.mp4`, `apps/www/public/preview.webm`                       |
-| **Mobile**  | 402x874    | `apps/www/src/assets/mobile-pwa-preview.webp` | `apps/www/public/mobile-pwa-preview.mp4`, `apps/www/public/mobile-pwa-preview.webm` |
+Expect **two original media assets**, one for the hero and one for the mobile
+workspace. Both must retain their original timelines through the out point. Do not
+pre-trim the sources to the selected clips. Supply embedded text subtitles or
+Jellyfin-compatible sidecars alongside the media, such as `hero.en.srt`.
 
-## Preparation
+The canonical configuration is `tools/www-assets/src/scenes.ts`:
 
-1.  **Record Source:** Capture high-quality screen recordings at the target resolution or higher.
-    - Ensure no mouse cursor is visible if possible.
-    - Use a consistent theme (Cliparr dark theme is preferred).
-    - Keep the recording focused and short (under 10 seconds).
-2.  **Input Files:** Name your source files `desktop-source.mov` (or `.mp4`) and `mobile-source.mov` (or `.mp4`).
+| Scene  | Cliparr in/out  | Selected duration | Browser/video size | Default recording duration |
+| ------ | --------------- | ----------------- | ------------------ | -------------------------- |
+| Hero   | 8:16.07–8:19.01 | 2.94 seconds      | 1600×886           | 82/30 seconds (~2.733)     |
+| Mobile | 23:22–23:32     | 10 seconds        | 402×874            | 3 seconds                  |
 
-## Processing Workflow
+The fractional hero timecodes are **decimal seconds, not frame numbers**.
+Cliparr selection and website recording length are separate settings. Never infer
+output length from the selected clip duration. Recording begins at the in point;
+its duration can be overridden without changing the selection.
 
-### 1. Extract First Frame (Static Placeholder)
+## Run
 
-Extract the first frame as a PNG, then convert it to optimized WebP.
-
-**Desktop:**
-
-```bash
-ffmpeg -i desktop-source.mov -frames:v 1 desktop-frame.png
-magick desktop-frame.png -quality 72 apps/www/src/assets/screenshot.webp
-rm desktop-frame.png
-```
-
-**Mobile:**
-
-```bash
-ffmpeg -i mobile-source.mov -frames:v 1 mobile-frame.png
-magick mobile-frame.png -quality 72 apps/www/src/assets/mobile-pwa-preview.webp
-rm mobile-frame.png
-```
-
-### 2. Optimize Videos for Web
-
-Generate optimized MP4 and WebM versions.
-
-#### Desktop (1600x886)
-
-**MP4 (H.264):**
+Requires Node 24+, the repository's pinned pnpm, and Docker with Compose. The image
+installs the pinned Playwright browser and FFmpeg; no host ImageMagick is needed.
+Install workspace dependencies with `pnpm install --frozen-lockfile` first.
 
 ```bash
-ffmpeg -i desktop-source.mov -vf "scale=1600:886" -c:v libx264 -crf 23 -preset slow -pix_fmt yuv420p -an apps/www/public/preview.mp4
+pnpm assets:capture --media-dir /absolute/path/to/media \
+  --hero 'hero.mkv' --mobile 'mobile.mkv'
 ```
 
-**WebM (VP9):**
-
-```bash
-ffmpeg -i desktop-source.mov -vf "scale=1600:886" -c:v libvpx-vp9 -crf 30 -b:v 0 -deadline good -cpu-used 1 -an apps/www/public/preview.webm
-```
+Paths for `--hero` and `--mobile` are relative to `--media-dir` and may include
+subdirectories. Defaults are `hero.mkv` and `mobile.mkv`. Mount a directory dedicated
+to these two assets; the capture library indexes that directory recursively.
 
-#### Mobile (402x874)
+The command builds a production-mode Cliparr with the capture bridge enabled,
+starts a disposable Jellyfin library, registers playback through Jellyfin's API,
+and records each scene sequentially. It uses unique project volumes and no host
+ports, so it can run alongside development services. The recorder shares Cliparr's
+network namespace and uses loopback, which keeps WebCodecs available in a secure
+browser context. Sources are read-only.
+The existing Jellyfin bootstrap script is reused with remote metadata disabled.
 
-**MP4 (H.264):**
+Recordings and review artifacts go into a new gitignored `.asset-capture/run-*`
+directory. Open its `review.html` and inspect `report.json`. To regenerate and
+replace the website's six assets after both captures validate, add `--write`:
 
 ```bash
-ffmpeg -i mobile-source.mov -vf "scale=402:874" -c:v libx264 -crf 23 -preset slow -pix_fmt yuv420p -an apps/www/public/mobile-pwa-preview.mp4
+pnpm assets:capture --media-dir /absolute/path/to/media \
+  --hero 'hero.mkv' --mobile 'mobile.mkv' --write
 ```
 
-**WebM (VP9):**
-
-```bash
-ffmpeg -i mobile-source.mov -vf "scale=402:874" -c:v libvpx-vp9 -crf 30 -b:v 0 -deadline good -cpu-used 1 -an apps/www/public/mobile-pwa-preview.webm
-```
-
-## Validation
-
-1.  **Check File Sizes:** They should be relatively small (under 500KB for desktop, under 100KB for mobile).
-    ```bash
-    ls -lh apps/www/public/preview.* apps/www/public/mobile-pwa-preview.* apps/www/src/assets/*.webp
-    ```
-2.  **Run Preview:** Start the dev server and check the homepage.
-    ```bash
-    pnpm dev:web
-    ```
-3.  **Visual Check:**
-    - Hover over the preview areas to ensure the videos play correctly and align with the static images.
-    - Verify that the "reduced motion" check in `apps/www/src/pages/index.astro` still shows the static image.
-
-## Integration
-
-After updating the assets, ensure the `Picture` component in `apps/www/src/pages/index.astro` still has matching `width` and `height` attributes to avoid layout shifts.
+Optional `--hero-seconds` and `--mobile-seconds` control recording duration only.
+They must be positive and fit within the selected range. `--hero-subtitle` and
+`--mobile-subtitle` accept Cliparr subtitle track keys to override its preferred
+supported text track. Failure diagnostics include the available track keys.
+
+## Capture invariants
+
+- Preserve the exact browser and output dimensions above; use device scale 1.
+- Subtitles must be enabled, loaded, and overlap the selected range. Do not silently
+  capture without them. Image-only subtitles are not a substitute for supported
+  text subtitles.
+- Do not show a mouse cursor, pointer annotations, or Playwright overlays.
+- Use `window.cliparrAssetCapture` to configure selection, seek, fit the timeline,
+  inspect readiness, and start/pause playback. It wraps existing Cliparr hooks and
+  `TimelineEngine.setInOutRange`; do not bypass the media adapter with engine-only
+  playback or introduce a separate fake editor.
+- The bridge exists only when built with `VITE_CLIPARR_ASSET_CAPTURE=true`. Never
+  enable this flag for a deployed application. Verify normal bundles exclude it.
+- Wait for metadata, the decoded starting frame, subtitle cues, fonts, and layout.
+  Start Playwright's explicit screencast and trigger playback programmatically;
+  trim the pre-play lead-in using captured frame timestamps. Do not record login,
+  loading screens, or setup interactions.
+- After fitting the selection, zoom to 65% of the fitted scale, centered on the
+  viewport, so surrounding source media makes the selected clip boundaries clear.
+- Do not run tracing with screenshots or context-level video alongside the final
+  screencast: they can override its frame size.
+- Mobile is a responsive browser viewport capture, not proof of native PWA behavior.
+
+## Outputs and verification
+
+| Scene  | Poster                                        | Videos                                                              |
+| ------ | --------------------------------------------- | ------------------------------------------------------------------- |
+| Hero   | `apps/www/src/assets/screenshot.webp`         | `apps/www/public/preview.mp4`, `preview.webm`                       |
+| Mobile | `apps/www/src/assets/mobile-pwa-preview.webp` | `apps/www/public/mobile-pwa-preview.mp4`, `mobile-pwa-preview.webm` |
+
+The pipeline encodes silent H.264 MP4 and VP9 WebM at 30 fps, with fast-start MP4.
+It extracts each WebP poster from the delivered MP4's first frame. FFprobe and a
+full decode validate dimensions, codecs, absent audio, and recording duration.
+Playback telemetry verifies decoded frame advancement and rejects capture stalls.
+
+The existing 500 KB hero / 100 KB mobile video budgets are advisory. The report
+flags larger files; inspect quality before changing encoding settings. Do not
+silently lower resolution or compress unreadable UI to meet a byte target.
+
+After replacing assets, run `pnpm build:web`, preview the homepage with
+`pnpm dev:web`, and check poster/video alignment, hover playback, looping, and
+reduced-motion static behavior. Keep the Picture dimensions in
+`apps/www/src/pages/index.astro` consistent with the source assets.
+
+For pipeline changes, run `pnpm test:assets`, relevant package type/lint checks,
+and `pnpm --filter @cliparr/frontend test` when editing the capture bridge. Exercise
+both scenes end to end. Synthetic media can validate the pipeline before supplied
+assets arrive, but must never replace the marketing assets. Report separately
+whether supplied-media visual review has been completed.
+
+## Troubleshooting
+
+Inspect `*-failure.png`, `*-failure.json`, `*-playback.json`, and `containers.log` in the run directory.
+Missing media or an insufficient source duration is a source-input error; do not
+change the requested in/out points to make it pass. Missing subtitles require a
+supported embedded track or sidecar. Decoder failures should be diagnosed through
+Cliparr's source/codec handling. Docker resources owned by the run are removed on
+completion or failure; review files are retained.

.gitignore

@@ -11,3 +11,5 @@ apps/server/.cliparr-data/
 *.tsbuildinfo
 .env*
 !.env.example
+
+.asset-capture/

apps/frontend/src/components/editor/EditorScreen.tsx

@@ -27,6 +27,7 @@ import {
   resolveRelativeSeekTime,
 } from "@/components/editor/editorShortcutCommands";
 import type { PlaybackFallbackInfo } from "@/components/editor/editorPlaybackSources";
+import { useEditorAssetCapture } from "@/components/editor/useEditorAssetCapture";
 import { useEditorExport } from "@/components/editor/useEditorExport";
 import { useEditorKeyboardShortcuts } from "@/components/editor/useEditorKeyboardShortcuts";
 import { useEditorTimelineMedia } from "@/components/editor/useEditorTimelineMedia";
@@ -336,6 +337,21 @@ function EditorScreenContent({
     setZoomScale(scale);
     setScrollLeft(Math.max(0, startTime * scale - padding));
   };
+  useEditorAssetCapture({
+    engine,
+    media: timelineMedia,
+    fitSelection: handleFitSelection,
+    subtitles: {
+      subtitleTracks,
+      subtitleOutputEnabled,
+      subtitleCuesReady,
+      subtitleLoading,
+      subtitleError,
+      clippedSubtitleCues,
+      setSubtitleEnabled,
+      handleSelectedSubtitleTrackChange,
+    },
+  });
   const zoomControl = useTimelineZoomControl({
     min: 0,
     max: EDITOR_MAX_ZOOM_SCALE,

apps/frontend/src/components/editor/useEditorAssetCapture.ts

@@ -0,0 +1,137 @@
+import { useEffect, useRef } from "react";
+import {
+  fromSeconds,
+  toSeconds,
+  type TimelineEngine,
+} from "@techsquidtv/canvas-timeline";
+import type { AssetCaptureController } from "@cliparr/shared/asset-capture";
+import type { useEditorTimelineMedia } from "@/components/editor/useEditorTimelineMedia";
+import type { useEditorSubtitles } from "@/components/editor/useEditorSubtitles";
+import { subtitleTrackKey } from "@/lib/selectPreferredSubtitleTrack";
+import { zoomEditorTimeline } from "@/components/editor/editorTimelineZoom";
+
+type CaptureSubtitles = Pick<
+  ReturnType<typeof useEditorSubtitles>,
+  | "subtitleTracks"
+  | "subtitleOutputEnabled"
+  | "subtitleCuesReady"
+  | "subtitleLoading"
+  | "subtitleError"
+  | "clippedSubtitleCues"
+  | "setSubtitleEnabled"
+  | "handleSelectedSubtitleTrackChange"
+>;
+
+interface CaptureBindings {
+  engine: TimelineEngine;
+  media: ReturnType<typeof useEditorTimelineMedia>;
+  subtitles: CaptureSubtitles;
+  fitSelection: () => void;
+}
+
+export function useEditorAssetCapture(bindings: CaptureBindings) {
+  const current = useRef(bindings);
+  useEffect(() => {
+    current.current = bindings;
+  });
+  useEffect(() => {
+    // Vite removes this bridge from normal builds. Never enable it in deployment.
+    if (import.meta.env.VITE_CLIPARR_ASSET_CAPTURE !== "true") {
+      return;
+    }
+    const controller: AssetCaptureController = {
+      inspect() {
+        const { engine, media, subtitles } = current.current;
+        const state = engine.getState();
+        return {
+          mediaReady: media.metadataReady,
+          previewReady:
+            !media.loadingPreview &&
+            !media.loadingPreviewFrame &&
+            media.renderedFrameTime !== null,
+          duration: media.duration,
+          inSeconds: state.inPoint ? toSeconds(state.inPoint) : 0,
+          outSeconds: state.outPoint
+            ? toSeconds(state.outPoint)
+            : media.duration,
+          currentSeconds: media.getPlaybackTime(),
+          renderedSeconds: media.renderedFrameTime,
+          frameStepSeconds: media.frameStepSeconds,
+          playing: media.playing,
+          subtitlesReady:
+            subtitles.subtitleOutputEnabled &&
+            subtitles.subtitleCuesReady &&
+            !subtitles.subtitleLoading,
+          subtitleCueCount: subtitles.clippedSubtitleCues.length,
+          subtitleTracks: subtitles.subtitleTracks.map((track) => ({
+            key: subtitleTrackKey(track),
+            title: track.title ?? track.languageCode ?? "Subtitle",
+          })),
+          error: media.error || subtitles.subtitleError || "",
+        };
+      },
+      configure({ inSeconds, outSeconds, subtitleTrackKey: trackKey }) {
+        const { engine, media, subtitles } = current.current;
+        if (
+          !media.metadataReady ||
+          !Number.isFinite(inSeconds) ||
+          !Number.isFinite(outSeconds) ||
+          inSeconds < 0 ||
+          outSeconds <= inSeconds ||
+          outSeconds > media.duration
+        ) {
+          throw new Error(
+            "Capture selection is outside the loaded media duration.",
+          );
+        }
+        if (
+          trackKey &&
+          !subtitles.subtitleTracks.some(
+            (track) => subtitleTrackKey(track) === trackKey,
+          )
+        ) {
+          throw new Error("Requested capture subtitle track is unavailable.");
+        }
+        if (subtitles.subtitleTracks.length === 0) {
+          throw new Error("Capture requires a supported text subtitle track.");
+        }
+        media.pausePlayback();
+        const result = engine.setInOutRange(
+          fromSeconds(inSeconds),
+          fromSeconds(outSeconds),
+        );
+        if (!result.ok) {
+          throw new Error("Timeline rejected the capture selection.");
+        }
+        if (trackKey) {
+          subtitles.handleSelectedSubtitleTrackChange(trackKey);
+        }
+        subtitles.setSubtitleEnabled(true);
+        media.seekToTime(inSeconds);
+      },
+      fitSelection() {
+        const { engine, fitSelection } = current.current;
+        fitSelection();
+        // Show source context on both sides so the selection reads as a clip.
+        zoomEditorTimeline(
+          engine,
+          engine.zoomScale * 0.65,
+          (engine.getState().viewportWidth ?? 0) / 2,
+        );
+      },
+      async play() {
+        const { media } = current.current;
+        if (!media.playing) {
+          await media.togglePlay();
+        }
+      },
+      pause: () => current.current.media.pausePlayback(),
+    };
+    window.cliparrAssetCapture = controller;
+    return () => {
+      if (window.cliparrAssetCapture === controller) {
+        delete window.cliparrAssetCapture;
+      }
+    };
+  }, []);
+}

apps/frontend/src/providers/types.ts

@@ -1,9 +1,5 @@
-import type {
-  SourcePlaybackError,
-  ViewerPlaybackGroup,
-} from "@cliparr/shared/providers";
-
 export type {
+  CurrentlyPlayingResponse,
   CurrentlyPlayingItem,
   MediaExportMetadata,
   PlaybackAudioSelection,
@@ -57,8 +53,3 @@ export interface MediaSourceCheckResult {
     message: string;
   };
 }
-
-export interface CurrentlyPlayingResponse {
-  viewers: ViewerPlaybackGroup[];
-  sourceErrors: SourcePlaybackError[];
-}