feat(adapter-hyperframes): deterministic render via @hyperframes/producer - #87
Conversation
…ucer When @hyperframes/producer is installed and the template carries Hyperframes composition markers (data-composition-id, data-duration, window.__timelines, ...), render through the real upstream pipeline: paused master timeline, per-frame seek, BeginFrame capture, parallel workers, deterministic font localization, and native audio muxing. Falls back to the existing realtime Playwright capture unchanged when the producer is not installed or the template is a plain CSS-keyframe single file, so nothing changes for current users. The producer is an optional peer dependency, mirroring the existing optional `hyperframes` peer; the dynamic import uses a variable specifier so tsc builds without the package present. Verified locally on Windows: - frame-glitch-title (no markers) -> realtime path, output unchanged - frame-kinetic-type -> deterministic path: exact 15s/900 frames at 1080p60 from data-duration, AAC audio muxed, A-Roll video frames extracted and injected (768 frames) where the realtime path records a CORS-stalled blank lead-in and no audio at all
lefarcen
added
size/M
|
Some context on why I went with HyperFrames compositions are built around a single paused master timeline that is seek-safe by contract: the engine ( The realtime capture in this adapter necessarily fights that model — the freeze/unfreeze + A few things this unlocks beyond the PR's scope, if there's interest in follow-ups:
Happy to iterate on the open questions in the description — in particular whether |
nettee
left a comment
nettee
left a comment
There was a problem hiding this comment.
I found two correctness issues in the new deterministic branch that should be fixed before merge. One is a silent fallback that masks real producer/render failures and returns a different render path than the caller requested; the other returns incorrect RenderOutput.meta values whenever the deterministic path renders an auto-duration composition.
| try { | ||
| const deterministic = await tryDeterministicRender(input, ctx); | ||
| if (deterministic) return deterministic; | ||
| } catch (err) { | ||
| if (ctx.signal?.aborted) throw err; | ||
| ctx.onProgress?.( | ||
| 5, | ||
| `deterministic render failed (${err instanceof Error ? err.message : err}) — falling back to realtime capture`, | ||
| ); | ||
| } |
There was a problem hiding this comment.
tryDeterministicRender() already returns null for the two non-applicable cases (@hyperframes/producer missing or the template not being a Hyperframes composition), and its docstring says it throws only for real render failures once the deterministic path has been entered. Catching every non-abort error here converts those real failures into a successful realtime export instead. That means a broken producer install, a render job error, or a deterministic pipeline regression will quietly produce a different video with different duration/audio semantics while still reporting success, which violates the fail-fast rule in AGENTS.md and makes this path very hard to trust in production. Please only fall back on the explicit null return, and let actual deterministic render failures propagate as render-failed errors instead of suppressing them.
| const durationSec = | ||
| input.config.duration === 'auto' ? undefined : Number(input.config.duration); | ||
| ctx.onProgress?.(100, 'done'); | ||
| return { | ||
| outputPath: input.config.outputPath, | ||
| meta: { | ||
| // The composition's own data-duration governs length in this path; fall | ||
| // back to the requested duration for reporting when it was explicit. | ||
| durationSec: durationSec ?? 0, | ||
| fileSizeBytes: st.size, | ||
| actualResolution: input.config.resolution, | ||
| fps, | ||
| renderedFrames: durationSec ? Math.round(durationSec * fps) : 0, |
There was a problem hiding this comment.
This returns incorrect metadata for every successful deterministic render with config.duration === 'auto': durationSec becomes 0 and renderedFrames becomes 0, even though the producer just emitted a non-empty file. The changed lines make that explicit by deriving both fields only from the request config, not from the produced composition duration. RenderOutput.meta is the adapter’s contract back to callers, and the existing Playwright path reports actual values here, so this creates a silent contract break for any consumer that shows render duration, frame count, or uses those values for later processing. Please compute these fields from the deterministic output instead of defaulting to zero, for example by reading the produced file with ffprobe or by pulling the resolved duration/frame count from the producer job result if that API exposes it.
3 participants
What
Adds a deterministic render path to
adapter-hyperframesthat drives the real upstream Hyperframes pipeline (@hyperframes/producer) — paused master timeline, per-frame seek, BeginFrame capture, parallel workers — instead of realtimerecordVideocapture, whenever:@hyperframes/produceris installed (new optional peer dependency, mirroring the existing optionalhyperframespeer), anddata-composition-id,data-composition-src,data-duration,window.__hf,window.__timelines).Everything else falls through to the existing realtime Playwright capture, unchanged. Without the producer installed, behavior is byte-for-byte identical to today.
Why
The current path records the page in wall-clock time, which the file itself acknowledges (
render.ts: "Upstream Hyperframes was never required at runtime",renderToHtml: "Real upstream Hyperframes integration will replace the inject"). That forces a stack of heuristics — the global animation freeze/unfreeze, the font-load dance, the lead-in-sstrim, the duration probe — and structurally cannot deliver several capabilities this adapter already advertises:tpadheuristicsdata-durationhonored exactlyaudio: multi)<video>elements@font-facelocalization at compileTest plan
Verified locally (Windows 11, Node 26):
frame-glitch-title(plain CSS keyframes, no markers) → realtime path, output unchanged — regression guard.frame-kinetic-type(multi-composition) → deterministic path: exact 15 s / 900 frames @ 1080p60, AAC audio track muxed, 768 A-Roll video frames extracted and injected. The same template on the realtime path records no audio and a CORS-stalled opening.tsc -p tsconfig.jsonbuilds with and without@hyperframes/producerinstalled (dynamic import uses a variable specifier).9 of the 23 bundled templates carry markers and benefit immediately: decision-tree, kinetic-type, nyt-graph, play-mode, product-promo, product-promo-30s, swiss-grid, vignelli, warm-grain.
Notes / open questions (draft)
meta.durationSec/renderedFramesare reported from the request config, not read back from the produced file — happy to wireffprobereadback if you'd prefer.durationMode: "explicit"currently defers to the composition's owndata-durationon the deterministic path (the Hyperframes model); realtime path semantics are unchanged. Open to trimming/padding to the explicit length instead.webm/movtrue alpha andpng-sequenceoutput — left out of this PR to keep it reviewable, but it's a one-lineformatpass-through if wanted.🤖 Generated with Claude Code