Skip to content

feat(integrations): add Integration.Video.ExtractAudio (ffmpeg audio extraction) - #907

Draft
stepandra wants to merge 2 commits into
neohaskell:mainfrom
stepandra:misc/video-transcription
Draft

stepandra wants to merge 2 commits into
neohaskell:mainfrom
stepandra:misc/video-transcription

Conversation

@stepandra

@stepandra stepandra commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Apps can extract an uploaded video's audio as mono 16 kHz WAV, then pass that audio to the existing transcription integration. Integration.Video.ExtractAudio runs local ffmpeg; chaining extraction, upload and transcription remains the application's job.

Review follow-up

  • The documentation explains that the dispatcher's 30-second default cancels work before the extraction's 180-second timeout. A typechecked example raises eventProcessingTimeoutMs through Application.withDispatcherConfig. The extraction default remains unchanged.
  • Four bounded tests run the real in-process dispatcher with a fake process runner. They distinguish a slow success and an extraction timeout reaching onError from the worker being cancelled by a shorter dispatcher deadline.
  • The Quick Start imports Integration.Video.ExtractAudio.Internal () and carries audio as Base64 text, not unserializable Bytes or lossy UTF-8 text.
  • Added Bytes.fromBase64 beside Core's existing encoder. It rejects malformed input and requires no new dependency or raw Haskell imports in application code. Tests cover known answers, every byte value, padding, empty and invalid input.
  • The ffmpeg fixture reads and checks the exact binary input bytes. Binary output also survives the JSON command/Base64 round trip.

Verification

Local pinned-toolchain results:

  • Integration.Video: 15 examples, 0 failures; repeated runs pass.
  • Bytes selection: 14 examples, 0 failures. Full Core suite: 1100 examples, 0 failures, 3 pending.
  • Doctests pass; decoder and dispatcher documentation snippets were separately typechecked.
  • Lint, typecheck, spec/criterion/signature checks, regenerated codemap and release-fragment checks pass.
  • Negative controls catch an empty input file and the old raw-text audio encoding.

The full integrations suite reports 621 examples and 2 failures in existing Oura SyncAll network tests; the same two fail on the original PR head. This is not a full-suite-green claim. No real ffmpeg, PostgreSQL/Hurl acceptance run or PR-ready security-review receipt is claimed here. Still a draft pending maintainer review.

Migration: existing apps need no change. Apps using this new integration need ffmpeg and a dispatcher deadline longer than extraction plus file-access overhead. Spec 018.

…extraction)

Adds an outbound integration that extracts an uploaded video's audio track as mono 16 kHz WAV (audio/wav) with ffmpeg, ready for Integration.Audio.Transcribe. The process boundary is an injectable Runner record, so the tests use a fake runner and never need ffmpeg. Registers the nhintegrations-test hspec suite in test-surfaces.json.
@coderabbitai

coderabbitai Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@NickSeagull

NickSeagull commented Oct 8, 2026 •

Copy link
Copy Markdown
Member

Hey, thanks a lot for this PR! A few things to fix:

  1. Align the 180-second extraction timeout with the dispatcher's 30-second default. The worker is cancelled first, so a slower video cannot finish or reach onError; document the required dispatcher override as Audio.Transcribe does, and cover that interaction.
  2. Fix the Quick Start's imports and success command: import Integration.Video.ExtractAudio.Internal () so ToAction is in scope, and encode result.audio losslessly before placing it in a JSON-serialized command. Bytes has no JSON instance, so the example cannot be used as shown.
  3. Strengthen the C3 input-file test to check the video bytes passed to ffmpeg, not just that an input path exists; writing an empty file would currently satisfy the assertion.

Thanks!

@NickSeagull

Copy link
Copy Markdown
Member

I found one more contract mismatch to fix:

  1. The output docs promise a 25 MB cap and usable first-part audio, but the extraction path relies on ffmpeg -fs and returns the result without checking size or WAV validity. FFmpeg documents -fs as allowing a file slightly larger than the requested size; the fake-runner test would also accept an oversized or unusable truncated file. I suggest either narrowing the promise to what -fs guarantees or enforcing the exact bound and checking that the returned audio is usable. If a different contract is intended, spell it out in the spec and tests.

Thanks!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants