Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 12 additions & 6 deletions .agents/skills/cliparr-www-assets/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,19 @@ controls the editor through its capture-only bridge; no mouse interaction is nee

Expect **two original media assets**, one for the hero and one for the mobile
workspace. Both must retain their original timelines through the out point. Do not
pre-trim the sources to the selected clips. Supply embedded text subtitles or
Jellyfin-compatible sidecars alongside the media, such as `hero.en.srt`.
pre-trim the sources to the selected clips. Prefer an existing non-empty,
language-tagged Jellyfin sidecar beside each source, such as `<media>.en.srt`.
Only transcribe when the required sidecar is missing; capture-window-only SRTs are
valid as long as their cues retain the original media timestamps. The disposable
Jellyfin session prefers English text subtitles and falls back to the first
supported text track only when no English track is available.

The canonical configuration is `tools/www-assets/src/scenes.ts`:

| Scene | Cliparr in/out | Selected duration | Browser/video size | Default recording duration |
| ------ | --------------- | ----------------- | ------------------ | -------------------------- |
| Hero | 8:16.07–8:19.01 | 2.94 seconds | 1600×886 | 82/30 seconds (~2.733) |
| Mobile | 23:22–23:32 | 10 seconds | 402×874 | 3 seconds |
| Scene | Cliparr in/out | Selected duration | Browser/video size | Caption size | Default recording duration |
| ------ | --------------- | ----------------- | ------------------ | ------------ | -------------------------- |
| Hero | 8:16.07–8:19.01 | 2.94 seconds | 1600×886 | 72 px | 82/30 seconds (~2.733) |
| Mobile | 23:22–23:32 | 10 seconds | 402×874 | 150 px | 3 seconds |

The fractional hero timecodes are **decimal seconds, not frame numbers**.
Cliparr selection and website recording length are separate settings. Never infer
Expand Down Expand Up @@ -74,6 +78,8 @@ supported text track. Failure diagnostics include the available track keys.
- Subtitles must be enabled, loaded, and overlap the selected range. Do not silently
capture without them. Image-only subtitles are not a substitute for supported
text subtitles.
- Apply each scene's configured caption size through the capture bridge before
recording. Keep hero at 72 px and mobile at Cliparr's 150 px maximum.
- Do not show a mouse cursor, pointer annotations, or Playwright overlays.
- Use `window.cliparrAssetCapture` to configure selection, seek, fit the timeline,
inspect readiness, and start/pause playback. It wraps existing Cliparr hooks and
Expand Down
Binary file modified .github/img/screenshot.webp
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions apps/frontend/src/components/editor/EditorScreen.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -354,6 +354,7 @@ function EditorScreenContent({
subtitleError,
clippedSubtitleCues,
setSubtitleEnabled,
setSubtitleStyleSettings,
handleSelectedSubtitleTrackChange,
},
});
Expand Down
22 changes: 21 additions & 1 deletion apps/frontend/src/components/editor/useEditorAssetCapture.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ type CaptureSubtitles = Pick<
| "subtitleError"
| "clippedSubtitleCues"
| "setSubtitleEnabled"
| "setSubtitleStyleSettings"
| "handleSelectedSubtitleTrackChange"
>;

Expand Down Expand Up @@ -70,7 +71,12 @@ export function useEditorAssetCapture(bindings: CaptureBindings) {
error: media.error || subtitles.subtitleError || "",
};
},
configure({ inSeconds, outSeconds, subtitleTrackKey: trackKey }) {
configure({
inSeconds,
outSeconds,
subtitleTrackKey: trackKey,
subtitleFontSize,
}) {
const { engine, media, subtitles } = current.current;
if (
!media.metadataReady ||
Expand All @@ -84,6 +90,14 @@ export function useEditorAssetCapture(bindings: CaptureBindings) {
"Capture selection is outside the loaded media duration.",
);
}
if (
subtitleFontSize !== undefined &&
(!Number.isFinite(subtitleFontSize) ||
subtitleFontSize < 16 ||
subtitleFontSize > 150)
) {
throw new Error("Capture subtitle font size must be from 16 to 150.");
}
if (
trackKey &&
!subtitles.subtitleTracks.some(
Expand All @@ -106,6 +120,12 @@ export function useEditorAssetCapture(bindings: CaptureBindings) {
if (trackKey) {
subtitles.handleSelectedSubtitleTrackChange(trackKey);
}
if (subtitleFontSize !== undefined) {
subtitles.setSubtitleStyleSettings((settings) => ({
...settings,
fontSize: subtitleFontSize,
}));
}
subtitles.setSubtitleEnabled(true);
media.seekToTime(inSeconds);
},
Expand Down
Binary file modified apps/www/public/mobile-pwa-preview.mp4
Binary file not shown.
Binary file modified apps/www/public/mobile-pwa-preview.webm
Binary file not shown.
Binary file modified apps/www/public/preview.mp4
Binary file not shown.
Binary file modified apps/www/public/preview.webm
Binary file not shown.
Binary file modified apps/www/src/assets/mobile-pwa-preview.webp
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified apps/www/src/assets/screenshot.webp
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions packages/shared/src/assetCapture.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ export interface AssetCaptureSelection {
inSeconds: number;
outSeconds: number;
subtitleTrackKey?: string;
subtitleFontSize?: number;
}

export interface AssetCaptureState {
Expand Down
12 changes: 9 additions & 3 deletions tools/www-assets/src/jellyfin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -55,9 +55,15 @@ export async function startJellyfinScene(
`${scene.name}: source must include the original timeline through ${scene.selection.outSeconds} seconds.`,
);
}
const subtitle = mediaSource.MediaStreams?.find(
(stream) => stream.Type === "Subtitle" && stream.IsTextSubtitleStream,
);
const textSubtitles =
mediaSource.MediaStreams?.filter(
(stream) => stream.Type === "Subtitle" && stream.IsTextSubtitleStream,
) ?? [];
const subtitle =
textSubtitles.find((stream) => {
const language = stream.Language?.trim().toLowerCase();
return language === "en" || language === "eng";
}) ?? textSubtitles[0];
if (subtitle?.Index === undefined || subtitle.Index === null) {
throw new Error(
`${scene.name}: provide an embedded text subtitle track or matching subtitle sidecar.`,
Expand Down
12 changes: 10 additions & 2 deletions tools/www-assets/src/scenes.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,16 @@ void test("keeps source selection separate from website recording length", () =>
mobile: "mobile.mkv",
});
assert.ok(hero && mobile);
assert.deepEqual(hero.selection, { inSeconds: 496.07, outSeconds: 499.01 });
assert.deepEqual(mobile.selection, { inSeconds: 1402, outSeconds: 1412 });
assert.deepEqual(hero.selection, {
inSeconds: 496.07,
outSeconds: 499.01,
subtitleFontSize: 72,
});
assert.deepEqual(mobile.selection, {
inSeconds: 1402,
outSeconds: 1412,
subtitleFontSize: 150,
});
assert.equal(hero.recordingSeconds, 82 / 30);
assert.equal(mobile.recordingSeconds, 3);
const changed = buildScenes({
Expand Down
2 changes: 2 additions & 0 deletions tools/www-assets/src/scenes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ export function buildScenes(options: {
selection: {
inSeconds: 496.07,
outSeconds: 499.01,
subtitleFontSize: 72,
...(options.heroSubtitle
? { subtitleTrackKey: options.heroSubtitle }
: {}),
Expand All @@ -65,6 +66,7 @@ export function buildScenes(options: {
selection: {
inSeconds: 1402,
outSeconds: 1412,
subtitleFontSize: 150,
...(options.mobileSubtitle
? { subtitleTrackKey: options.mobileSubtitle }
: {}),
Expand Down