Repository navigation
Add Cybernetic Commander voice, timing, and mastering workflows - #4
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b9b2562ab5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if mode == "seq": | ||
| return f"{seq:04d}" | ||
| return djb2(line["who"] + "|" + line["text"]) | ||
| return djb2(line["who"] + "|" + line.get("spokenText", line["text"])) |
There was a problem hiding this comment.
When a hash-named line supplies spokenText, this now writes the artifact under djb2(who|spokenText), but all shipped runtime consumers still accept only the display text and compute djb2(who|text) (for example, runtime/voforge.js lines 44 and 64, with the same behavior in the Unity and Godot players). Consequently, any caller preserving the subtitle text as intended will request a filename that was never generated and fall back instead of playing the rendered audio; the runtime API or a display-to-artifact mapping must expose the new lookup key.
Useful? React with 👍 / 👎.
| fade = round(sample_rate * fade_ms / 1000) | ||
| fade_in = min(fade, leading_padding) | ||
| fade_out = min(fade, len(extracted)) |
There was a problem hiding this comment.
When an F5 phrase ends with less than 20 ms of quiet material after the last active frame, fade_out is still set to the full 20 ms because it is bounded by the entire extracted phrase rather than trailing_padding. The loop therefore attenuates active phrase-final audio—potentially consonants—even though this helper promises to fade only within retained padding; bound the fade by the measured trailing padding unless modifying voiced samples is explicitly intended.
Useful? React with 👍 / 👎.
| requested_break = round(sample_rate * phrase.break_after_ms / 1000) | ||
| next_leading_padding = processed[index + 1][5] if index + 1 < len(processed) else 0 | ||
| inserted_silence = max( | ||
| 0, requested_break - trailing_padding - next_leading_padding | ||
| ) |
There was a problem hiding this comment.
For an accepted short break such as <break time="100ms"/>, the two phrases can retain 80 ms of trailing and leading padding each; this expression clamps inserted silence to zero but leaves roughly 160 ms between detected speech boundaries. Thus valid 0–150 ms breaks are silently lengthened instead of realizing their declared duration. The renderer should trim the retained padding as needed or reject break values it cannot honor.
Useful? React with 👍 / 👎.
| "revision": "84e5a410d9cead4de2f847e7c9369a6440bdfaca", | ||
| "source": "https://huggingface.co/SWivid/F5-TTS", | ||
| "sha256": "670900fd14e6c458b95da6e9ed317cdb20dbaf7a1c02ac06a05475a9d32b6a38", |
There was a problem hiding this comment.
Rendering this profile does not actually use or verify the declared revision and SHA-256: ProjectHerHarness._make_backend() passes only model.options (device here), so F5Backend constructs the default F5TTS_v1_Base checkpoint without a pinned ckpt_file or hash check. If the upstream/default cached checkpoint differs, the generated audio changes while the run summary still reports this revision and hash, invalidating the profile's reproducibility and provenance claims.
Useful? React with 👍 / 👎.
| @@ -0,0 +1,8 @@ | |||
| [ | |||
| { | |||
| "id": "commander-systems-ready-v2-260803", | |||
There was a problem hiding this comment.
With output.name set to id and the output directory set to out, this ID resolves to the exact out/commander-systems-ready-v2-260803.wav path that profile-spec.json labels as the immutable historical baseline whose inference settings are unrecovered. On a fresh clone the documented render command creates a non-equivalent file under the baseline identity, and on a machine retaining the historical file project-her render --force overwrites it; use a distinct candidate ID/path or add a hash-enforced overwrite guard.
Useful? React with 👍 / 👎.
| include examples/voice-profiles/cybernetic-commander/sources/generated-directive.mp3 | ||
| include examples/voice-profiles/cybernetic-commander/sources/generated-night-walk.mp3 | ||
| include examples/voice-profiles/cybernetic-commander/sources/source-clothes-request.mp3 | ||
| include examples/voice-profiles/cybernetic-commander/sources/source-night-walk.mp3 |
There was a problem hiding this comment.
These explicit includes place the user-supplied source MP3s and cloned reference WAV into the source distribution that the workflow builds and uploads, even though docs/consents/cybernetic-commander.md limits the recorded authorization to adding the recordings to this repository and requires a separate rights review for redistribution outside it. Publishing or sharing the built archive therefore redistributes the raw recordings without the required recorded review; keep them out of distributions or document that additional authorization before packaging them.
Useful? React with 👍 / 👎.
| seed = 261000 + rows.index(row) * 100 + index | ||
| text_hash = hashlib.sha256(phrase.text.encode("utf-8")).hexdigest()[:8] | ||
| part = part_dir / f"{index:02d}-{seed}-{text_hash}.wav" | ||
| if force_parts or not part.is_file(): |
There was a problem hiding this comment.
The resumable filename depends only on phrase index, seed, and an eight-character text hash. If the reference artifact, reference transcript, F5 checkpoint, or inference/mastering settings are revised, existing WAVs still satisfy part.is_file() and are reused, while the newly written manifest attributes them to the current reference and configuration. This can silently mix stale audio into a new experiment; key or validate cached parts against the complete synthesis provenance rather than text and seed alone.
Useful? React with 👍 / 👎.
| if args.dialogue_id == "all": | ||
| document = json.loads((ROOT / "timed-dialogue.json").read_text(encoding="utf-8")) | ||
| shared_backend = F5Backend(device="cpu") | ||
| for row in document["dialogues"]: | ||
| print(await render( | ||
| row["id"], force_parts=args.force_parts, backend=shared_backend | ||
| )) | ||
| elif len(selected_ids) > 1: | ||
| shared_backend = F5Backend(device="cpu") |
There was a problem hiding this comment.
For all and comma-separated selections, the CLI constructs the heavyweight CPU F5Backend before checking whether any phrase needs synthesis. A fully cached resumable run therefore still loads or downloads the F5 model and can fail for missing weights or memory even though assembling the existing WAV parts requires no backend. Preserve a lazily initialized shared backend so cached-only reassembly remains lightweight and offline.
Useful? React with 👍 / 👎.
Protonmatter
changed the title
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c9e07b50b3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| output = output.resolve() | ||
| manifest = manifest.resolve() | ||
| if not source.is_file(): | ||
| raise FileNotFoundError("input WAV does not exist: {}".format(source)) | ||
| if source == output: |
There was a problem hiding this comment.
When --manifest resolves to the output path, the WAV is successfully installed at line 505 and then silently replaced with JSON by _write_json_atomic; with --force, choosing the input path similarly destroys the source audio. Since the CLI accepts an arbitrary manifest path, validate that it differs from both source and output before invoking FFmpeg.
Useful? React with 👍 / 👎.
| or output_format["bitsPerSample"] != 24 | ||
| ): | ||
| raise RuntimeError("mastered output does not satisfy 24 kHz mono PCM24") | ||
| os.replace(str(candidate), str(output)) |
There was a problem hiding this comment.
When the manifest directory is unwritable, its creation fails, or its write runs out of space, this replacement has already published the new WAV and discarded any prior output even though master_voice then raises. With --force, the old manifest can remain beside the new, mismatched audio, defeating the recorded provenance and the function's atomic-publication contract; stage or validate the manifest before replacing the destination artifacts, and avoid leaving a partially published pair.
Useful? React with 👍 / 👎.
| ]: | ||
| raise ValueError("display and spoken SSML must have identical break structure") | ||
| part_dir = ROOT / "out" / "timed-parts" / dialogue_id | ||
| part_dir.mkdir(parents=True, exist_ok=True) |
There was a problem hiding this comment.
On every uncached CLI render, this constructs F5Backend with its default MasteringSettings, whose delivery rate is 22,050 Hz (voforge/audio.py line 31); F5Backend.synth then resamples the model waveform to that rate before the timed assembler reads it (voforge/backends.py lines 162 and 190). Consequently the assembled candidate is 22.05 kHz rather than the profile/reference's 24 kHz, and the renderer's stated no-resampling contract is false; supply a 24 kHz mastering setting to the backend or explicitly change the profile contract.
Useful? React with 👍 / 👎.
| "profile": "cybernetic-commander", | ||
| "input": { | ||
| "path": _manifest_path(source, manifest.parent), | ||
| "sha256": sha256_file(source), |
There was a problem hiding this comment.
If the dry input is atomically replaced or rewritten while the potentially long FFmpeg and loudness-calibration passes are running, FFmpeg can finish processing the original open file while this late hash reads the new path contents. The resulting manifest then attributes the master to bytes that did not produce it, undermining the evidence record; hash the input before processing and verify that it is unchanged before publication.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 746d5779ff
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| MANIFEST.write_text( | ||
| json.dumps(manifest, indent=2, sort_keys=True) + "\n", | ||
| encoding="utf-8", | ||
| newline="\n", |
There was a problem hiding this comment.
When the documented rebuild is run on Python 3.9, which the package and CI matrix explicitly support, Path.write_text() does not accept the newline argument, so this raises TypeError. Because the reference WAV has already been written at line 124, the command can leave a newly rebuilt reference beside the old manifest; write through open(..., newline="\n") or omit this Python 3.10-only argument.
Useful? React with 👍 / 👎.
Summary
spokenTextpronunciation support while preserving display text and invalidating stale content-addressed output namesWhy
VOForge needed a reproducible way to carry accent-specific pronunciation and explicit dramatic timing without changing subtitles or treating local generated output as source truth. The profile also needed a runnable fresh-clone contract.
The mastering experiments showed that format, loudness, and true-peak compliance are not sufficient promotion gates: formant-preserving pitch/time processing can reduce intelligibility. This PR makes that limitation explicit and requires dry/mastered comparison, ASR comparison, and human listening approval.
Checked-in name-free WAV samples
The separately licensed
examples/voice-profiles/cybernetic-commander/samples/directory includes:Every file is hash-bound with intended text, cached Whisper tiny.en transcription, PCM measurements, known or unavailable seed status, and QA disposition. Intended scripts and offline transcriptions contain no personal, speaker, or character names. Human listening has not been completed, so this is a bounded automated screen rather than a general identity or privacy guarantee.
The mastered air-conditioner clip is deliberately retained as rejected A/B evidence: it passes PCM24, loudness, true-peak, and clipping gates but materially regresses transcription accuracy.
Mastering workflow
master_voice.py:--forceis explicitThe local ignored acceptance artifact measured -16.46 LUFS and -1.10 dBTP.
Licensing and safety boundary
examples/audio/clone-f5-min-kang-*.wavfiles are not includedValidation
python -m compileall -q voforge tests examples/voice-profiles/cybernetic-commanderpython -m ruff check examples/voice-profiles/cybernetic-commander/master_voice.py tests/test_master_voice.pypython -m unittest discover -s tests- 61 tests passed on Python 3.14.3project-her validate --config examples/voice-profiles/cybernetic-commander/project-her.jsonproject-her plan --config examples/voice-profiles/cybernetic-commander/project-her.jsonpython -m build- source distribution and wheel built successfully1f6e88b2961e1a8ec1221b663d26a1f1c1d88b6d78ee5be215fce48e0a41792egit diff --checkNamed dialogue evaluation
The separately licensed
examples/voice-profiles/cybernetic-commander/dialogue/directory now includes dry and husky-EQ A/B files for a user-supplied long-form narration. This content intentionally contains declared fictional character names and is isolated from the name-freesamples/contract.Skynet; human listening and rights review remain requirednot approvedfor promotion and evaluation/non-commercial onlyCurrent local validation: manifest JSON, SHA-256, byte counts, PCM metadata, documentation targets, compileall, and 62 offline unit tests passed.