A no-build, personal Chrome extension that translates Korean, Japanese, or auto-detected tab audio into spoken English. Mix the original and English audio independently, use continuous translation for speed, or choose one of four AI voices.
Beta: Chrome 116+ and your own billed OpenAI API project are required. Live translation quality and latency have not yet been listening-tested. This is an unpacked extension, not a Chrome Web Store listing.
- Download and unzip the latest package from GitHub Releases, or clone this repository. Open
chrome://extensionsin Chrome and enable Developer mode. - Click Load unpacked and select the extracted
live-dub-extensionfolder (or the cloned repository) containingmanifest.json. - Open a Korean, Japanese, or other foreign-language video and pin Live Dub from Chrome's extensions menu.
- Open Live Dub and enter your own OpenAI API key under API connection. The API project needs billing and access to the selected model. A ChatGPT subscription does not supply this key.
- Leave YouTube's player unmuted, select Korean, Japanese, or Auto-detect, choose a translation style, and click Start translating. Play the video once the indicator says Live translation.
- Adjust Original audio and English translation independently. Close the popup; translation continues. Reopen it to mix or stop.
No server, build step, dependencies, microphone permission, or caption scraping required.
- Continuous (default):
gpt-realtime-translateover WebRTC. Dedicated streaming interpretation; translates while source speech continues. There is no voice picker: the English voice follows each speaker's tone, pitch, and speaking style, and changes when a new speaker talks. Best starting point when speed matters. - Choose a voice:
gpt-realtime-2.1over WebRTC with Marin or Coral (female) or Cedar or Ash (male). One voice is used for every speaker; it cannot change mid-session. A constrained interpreter prompt and 300 ms silence detection. Translation follows short speech phrases. New source speech does not interrupt English playback. Sustained speech can delay output; the dedicated mode is preferable for continuous dialogue. Stop and restart to change source language, voice, or mode.
Both voices and translations are generated by AI. Naturalness and accuracy need listening verification. No measured latency guarantee: network, server load, source-language sentence structure, music and overlapping speakers all affect delay. This is live interpretation, not a frame-synchronized dubbed soundtrack. There is no speaker diarization or per-character voice assignment.
The source-language selector defaults to Korean and remembers your choice; pick Auto-detect for any other language. Continuous mode always detects the incoming speech automatically and requests English output; the selector only changes the label on that API. Choose a voice mode uses Korean or Japanese interpreter instructions, or asks the model to identify the language when Auto-detect is selected. Output is always English.
Keep the video player's own audio on, using the extension's original slider to turn down the original audio. Original volume is applied AFTER the capture branches to the translator, so the model still hears the full captured signal even at 0% original volume. Translation plays in an offscreen extension document to avoid capturing itself.
Click Stop before seeking, changing videos, or taking a break, then start again at the new position. Seeking is not automatically detected. Pausing the video silences new input but a pending translation may finish. Merely closing the popup does not stop the session or API usage. Closing the source tab or navigating to another URL stops capture. Stop/error cleanup restores normal tab audio. Sessions end if the network or service disconnects; reconnect manually.
If Continuous returns an access error, try Choose a voice. If both fail, check the API project's permissions, billing, and usage limits. The extension never silently switches to a different paid model.
This is a personal bring-your-own-key extension, not a public website or shared-key deployment. Your key is saved in chrome.storage.local on this computer (not Chrome Sync), so you only need to enter it once. Only trusted extension contexts can read it, but local extension storage is not encrypted. It is sent directly to OpenAI by the extension service worker to mint a short-lived session token. The audio document receives only that token. The key is never injected into websites or included in the package. Forget key stops translation and removes it; uninstalling the extension also deletes it.
Only audio from the explicitly selected tab goes to OpenAI after Start. All audible material in that tab, including ads, can be transmitted. No microphone capture. No audio recordings or transcript files are written locally. A rolling 1,600-character transcript is held in memory and cleared on Stop. Volume/mode preferences persist locally. OpenAI's API data handling applies to transmitted audio.
For distributing a service with a shared provider key, replace personal key entry with an authenticated backend that mints short-lived client tokens. Never bundle an API key in this extension.
See PRIVACY.md for data handling and an explanation of every Chrome permission.
| Symptom | What to do |
|---|---|
| Key rejected | Enter a valid OpenAI API key for your project. |
| Model unavailable | Check project access, or explicitly try the other translation mode. |
| Quota / rate limit | Check API billing, credit balance, and project limits. |
| No English audio | Unmute the video player, raise English volume, check that speech is playing, then restart. |
| Long translation delay | Try Continuous, reduce competing sounds, and stop/restart after seeking. |
| Capture cannot start | Use a normal HTTP/HTTPS video tab; Chrome internal pages cannot be captured this way. Another extension may already be capturing it. |
| Session ended | Start again; the extension does not reconnect automatically or silently restart paid usage. |
Stop translation before updating. If the folder stays at the same path, replace its files and click Reload on chrome://extensions. If the folder moves, remove the old unpacked entry and use Load unpacked on the new folder; re-enter your key and preferences. Do not delete the source folder while Chrome is using it.
Use Node 20+ for tests and Python 3 for packaging. There are no npm dependencies to install.
npm test
npm run packageThe packager writes dist/live-dub-extension-0.2.0.zip using an explicit file allowlist. GitHub Actions runs the tests and packaging on pushes and pull requests.
Tests use local mocks; they do not send audio or make paid requests. See tests/ for configuration, authentication/capture failure, cancellation and mixer lifecycle checks. Live Chrome capture and paid translation require a user-supplied API key and listening verification.
- https://developers.openai.com/api/docs/guides/realtime-translation
- https://developers.openai.com/api/docs/guides/voice-webrtc
- https://developers.openai.com/api/docs/guides/realtime-conversations
- https://developers.openai.com/api/docs/guides/realtime-vad
- https://developer.chrome.com/docs/extensions/how-to/web-platform/screen-capture
- Start a Korean clip in Continuous mode, then repeat with a Japanese clip. Confirm English output and compare meaning against a reliable translation.
- Test both languages with each of Marin, Cedar, Coral, and Ash in Choose a voice mode.
- Set original volume to 0%; confirm English continues. Set English to 0%; confirm original audio remains independently controllable.
- Close/reopen the popup and verify controls retain the correct language and session status.
- Stop, seek, then restart. Close the source tab and verify capture ends.
- Check behavior after a network interruption, invalid key, and API quota error.
- Measure speech-to-English delay on real material before making latency claims.
See CHANGELOG.md for release history.