[#141] Pre-provision alignment models so no model downloads mid-transcription

[julien731]

Closes #141

Summary

WhisperX HuggingFace wav2vec2 alignment models were downloading lazily at the align stage (50%) the first time a language was used, into the bundled service's isolated HF_HOME. During that download the job sat frozen at "Aligning timestamps… 50%" — indistinguishable from a hang, with no download messaging. Only first-run provisioning had a real download-progress UI.

This pre-downloads the alignment models for a configured language set during provisioning, so the align step never downloads mid-transcription. The default set is Thai (th) — the one HuggingFace-backed language this deployment commonly uses. English and the other torch-native languages (French, German, Spanish, Italian) live in the shared ~/.cache/torch and never needed an HF download. The ALIGNMENT_TIMEOUT_SEC watchdog remains the safety net for any language left unprovisioned (auto-detect landing on an HF language, or a language outside the configured set) — surfacing that residual case with its own download indicator is tracked in #145.

Approach

Option 2 from the issue (prefetch a configured common set during provisioning), chosen over per-selection download (option 1 — large cross-surface diff across the web and native Swift upload forms) and prefetch-all (option 3 — ~15 GB, too heavy). It reuses the existing provisioning download + version-gate + progress-UI mechanism end to end, so no frontend or Swift changes are needed.

• New import-light backend/services/align_models.py holds HF_ALIGN_REPOS (a copy of WhisperX's DEFAULT_ALIGN_MODELS_HF plus this app's th override) as the single source of truth, so provisioning pre-fetches exactly the repo the transcriber loads. Kept free of whisperx/torch/huggingface_hub imports so provisioning and its tests stay lightweight (those deps are absent in CI).
• ServiceConfig.align_languages (default ["th"]) makes the prefetch set configurable via the persisted config.
• provisioning.required_repos() appends the resolved align repos (deduped); PROVISIONING_VERSION is bumped 2 → 3 so already-provisioned installs re-provision and fetch the addition instead of downloading it lazily — the same mechanism used for the earlier wespeaker fix.
• transcriber.py now resolves the align model name from the shared mapping (removing the local CUSTOM_ALIGN_MODELS); behavior-preserving because the values equal WhisperX's own defaults and th is unchanged.

Full design in docs/plans/141-pre-provision-alignment-models.md.

Verification

• ruff check and ruff format --check: clean.
• pytest tests/unit: 337 passed (plus the new/updated test_align_models.py and test_provisioning.py).
• New unit tests cover align_repos_for (skips torch-native/unknown, dedup, order-stable), the default-set provisioning list, and drift guards asserting HF_ALIGN_REPOS/TORCH_ALIGN_LANGUAGES stay in sync with WhisperX where it is importable (the guard ran and passed locally).

[julien731]

QA Confidence Verdict — #141 (PR #147)

Backend transcription-pipeline change; no UI surface, so Playwright is N/A. Verified by diff review + unit suite (WhisperX present in the local venv, so the drift-guard tests actually executed rather than skipping).

What was verified

• AC1 — Pre-download configured align set via provisioning: MET. provisioning.required_repos() appends align_models.align_repos_for(cfg.align_languages) (token-independent), deduped (provisioning.py:76-79). PROVISIONING_VERSION 2→3 (provisioning.py:27). Default align_languages=["th"] (schemas.py). Test: test_downloads_configured_align_model asserts the Thai repo passes through _snapshot_downloader, reusing the existing progress-UI path.
• AC2 — Transcriber loads exactly what was prefetched: MET. Single-language + multilingual align paths now resolve via HF_ALIGN_REPOS.get(...) (transcriber.py:413,530); local CUSTOM_ALIGN_MODELS removed (no stale refs). Single source of truth in align_models.py. Drift guard test_matches_whisperx_defaults_on_shared_keys ran green against installed WhisperX → the copied map matches DEFAULT_ALIGN_MODELS_HF, and th confirmed as app-only override.
• AC3 — Configurable: MET. ServiceConfig.align_languages (schemas.py) drives the prefetch set from persisted config. test_empty_align_languages_only_whisper confirms opting out.
• AC4 — Already-provisioned installs re-provision: MET. models_ready() version-gate (provisioning.completed AND provisioning_version >= PROVISIONING_VERSION) pre-existed from macOS app transcription hangs at 70% (and 50%) on lazy model download #142; the 2→3 bump re-triggers provisioning for the align addition. status() reports via the gated models_ready(), so wizard/health/upload-gate agree.
• AC5 — Torch-native unaffected, unprovisioned HF degrades via watchdog: MET. TORCH_ALIGN_LANGUAGES={en,fr,de,es,it} contribute no repos (test_torch_native_languages_contribute_nothing); test_torch_native_languages_match_whisperx ran green, so classification can't diverge from WhisperX's torchaudio set. ALIGNMENT_TIMEOUT_SEC watchdog on both align paths is preserved from macOS app transcription hangs at 70% (and 50%) on lazy model download #142 — unlisted/unknown HF codes resolve to None and degrade to segment-level timestamps.

Unit suite: 338 passed (tests/unit), incl. 9 in test_align_models.py (6 logic + 3 drift guards, all executed) and updated test_provisioning.py.

What needs human eyes

• End-to-end provisioning against real HuggingFace (actual th wav2vec2 download + per-repo progress rendering) — all download I/O here is mocked (_snapshot_downloader monkeypatched).
• One real mid-transcription run on a fresh, previously-provisioned install to confirm the align step no longer stalls at 50% and re-provisioning triggers on first launch.

Risk areas

• _snapshot_downloader / HF network fetch is fully mocked in tests — pass ≠ proof the Thai repo id resolves or downloads.
• Drift guard is a dev-only gate (skips where WhisperX absent, i.e. CI). It ran locally this time, but CI does not enforce it; a future WhisperX bump could drift the copied map undetected in CI. Bounded: provisioning and transcriber read the same map, so worst case is a stale-but-valid repo, still watchdogged.
• ms/ta are in ALIGNMENT_LANGUAGES but in neither map → resolve to None. Pre-existing behavior (old CUSTOM_ALIGN_MODELS.get() was also None); no regression, watchdog covers them. Out of Pre-provision alignment models so no model downloads mid-transcription #141 scope.

Suggested QA focus

Quick: one clean-install provisioning run watching for the Thai model in the progress list. Thorough: an upgrade-path run (already-provisioned install) to confirm re-provisioning fires and a Thai transcription aligns without the 50% stall. ~15 min total.

Verdict: high confidence, all 5 AC met. No code issues found. Residual risk is entirely in un-mocked network/E2E behavior, appropriately flagged for human validation. #145 correctly tracks the remaining lazy-download UI case for un-provisioned languages.