diff --git a/CONFIGURATION.md b/CONFIGURATION.md index b0ee1e3..34ab9e0 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -47,18 +47,13 @@ Only `input_path`, `output_path`, and `model_name` are required. Everything else Default: `2.0` seconds -This decides how long a silence must be before it counts as a break. It affects both the speech recognition step and the final transcript segments. +This is the pause threshold used for ordinary grouping of recognized pieces into transcript segments. -- Use a **lower** value to create more breaks. -- Use a **higher** value to keep speech together across longer pauses. +- Use a **lower** value to keep more recognized pieces separate. +- Use a **higher** value to allow more joining across pauses. -Good starting points: - -| Recording | Try | -|-----------|-----| -| Subtitles or fast speech | `0.5`-`1.0` | -| Conversation or interview | `2.0` | -| Slow speech with thinking pauses | `3.0`-`4.0` | +This setting does not change Silero VAD, the audio sent to Whisper, decoder text, or raw timestamps. Speech detection is +automatic. ### `similarity_threshold` @@ -95,41 +90,7 @@ max_segment_words: 100 Set both minimums to `1` if short replies such as “Yes” or “No” should stay on their own. -## Useful presets - -Add one of these blocks to your config, replacing the same settings if they are already there. - -### Subtitles - -```yaml -pause_threshold: 1.0 -similarity_threshold: 1.0 -min_segment_words: 1 -min_segment_chars: 1 -max_segment_words: 40 -``` - -### Interview or podcast - -```yaml -pause_threshold: 3.0 -similarity_threshold: 0.75 -min_segment_words: 3 -min_segment_chars: 15 -max_segment_words: 150 -``` - -### Meeting notes - -```yaml -pause_threshold: 2.0 -similarity_threshold: 0.8 -min_segment_words: 3 -min_segment_chars: 15 -max_segment_words: 80 -``` - -### Keep Whisper's segments mostly unchanged +## Keep Whisper's segments mostly unchanged ```yaml similarity_threshold: 1.0 @@ -194,7 +155,7 @@ files and logs. The web interface lets you change: - device and language -- pause and similarity thresholds +- transcript-grouping pause and similarity thresholds - minimum words, minimum characters, and maximum words It chooses the input and output paths for you and uses the default Whisper and embedding models. Performance and logging settings are available only in a CLI YAML file. @@ -209,14 +170,16 @@ In the usual case, two neighbouring segments are joined only when: Short fragments get an extra cleanup pass. They may be joined without passing the similarity check, but a merge is never allowed to create a segment longer than `max_segment_words`. If no suitable neighbour is available, the fragment is left as it is. +Speech detection is automatic. Changing transcript grouping does not change the audio sent to Whisper. + ## Troubleshooting | If you see this | Try this | |-----------------|----------| | Too many tiny segments | Raise `min_segment_words` or `min_segment_chars` | | Segments are too long | Lower `max_segment_words` | -| Not enough breaks | Lower `pause_threshold` or raise `similarity_threshold` | -| Too many breaks | Raise `pause_threshold` or lower `similarity_threshold` | +| Not enough transcript breaks | Lower `pause_threshold` or raise `similarity_threshold` | +| Too many transcript breaks | Raise `pause_threshold` or lower `similarity_threshold` | | Unrelated sentences are joined | Raise `similarity_threshold` to about `0.85` | | Short replies disappear into nearby text | Set both minimums to `1` | | Whisper runs out of memory | Lower `performance.whisper_batch_size` | diff --git a/README.md b/README.md index e6f9455..2be8d1e 100644 --- a/README.md +++ b/README.md @@ -44,9 +44,9 @@ uv run --locked --no-dev hf download openai/whisper-large-v3 uv run --locked --no-dev hf download sentence-transformers/all-MiniLM-L6-v2 ``` -After that, `textplease` loads models only from local files. Transcription fails instead of downloading when a configured -model is missing. Prefetch any custom Hugging Face model ID with the same `hf download ` command. A local model -directory also works. +After preparation, `textplease` loads model weights from the cache or a local directory and does not download missing +weights during a transcription job. Hugging Face may still receive non-inference model metadata requests. Prefetch any +custom Hugging Face model ID with the same `hf download ` command. A local model directory also works. ### Web interface @@ -77,9 +77,10 @@ The transcript is written to the `output_path` set in the config. For the exampl ## Local privacy and retained files -Audio and transcript content are processed locally. During transcription, supported model loaders are restricted to -local files. The Gradio UI binds to `127.0.0.1`, cannot create a share tunnel, and has analytics and monitoring disabled. -Use an operating-system firewall or disconnect the network when an external guarantee is required. +Audio and transcript content are processed on-device and are not uploaded to Hugging Face or another cloud inference +API. Model acquisition and non-inference Hugging Face metadata may use the network. The Gradio UI binds to `127.0.0.1`, +cannot create a share tunnel, and has analytics and monitoring disabled. After preparing models, use an operating-system +firewall or disconnect the network if an externally enforced zero-network guarantee is required. Temporary decoded PCM is removed after each job. Gradio checks hourly for uploaded cache files older than 24 hours and clears its cache when the server restarts. Hugging Face model caches persist for reuse. The web interface also keeps each @@ -178,12 +179,13 @@ textplease runs a modular pipeline: 1. Audio processing – extracts and normalizes audio from the input file. 2. ASR transcription – converts speech to text with multilingual Whisper models. - Language is selectable from the languages supported by the configured Whisper checkpoint. The seed evaluation currently validates English only. - - Silero-VAD-negative audio succeeds with an empty transcript and never loads Whisper. Detector-positive non-speech remains a tracked release blocker. + - Silero VAD uses one internal automatic endpointing policy. Transcript grouping settings cannot change Whisper input. + - VAD-negative audio succeeds with an empty transcript and never loads Whisper. Detector-positive non-speech remains a tracked release blocker. - Whisper batches VAD chunks on CUDA while retaining the same generation and timestamp settings. - Whisper runs via `model.generate()` with temperature fallback and compression-ratio quality gating. - A post-transcription filter removes known Whisper hallucination phrases. - Deduplication removes residual word overlap at chunk boundaries. -3. Segmentation – groups text into coherent segments using pause detection (aligned with VAD boundaries) and semantic similarity from sentence embeddings. +3. Segmentation – groups recognized spans using measured gaps and semantic similarity. Its pause setting changes layout only. 4. Post-processing – enforces length constraints, merges short segments, splits long ones, and writes the CSV. ```mermaid diff --git a/examples/config_example.yaml b/examples/config_example.yaml index badfada..4834408 100644 --- a/examples/config_example.yaml +++ b/examples/config_example.yaml @@ -9,7 +9,7 @@ language: "en" # language code (97+ languages supported) # Segmentation similarity_threshold: 0.75 # [0.0–1.0] cosine similarity required to merge two segments -pause_threshold: 2.0 # silence (seconds) that forces a split; also sets Silero-VAD boundary +pause_threshold: 2.0 # pause threshold for ordinary transcript grouping min_segment_words: 3 # segments below this are merged with a neighbour min_segment_chars: 15 # segments below this are merged with a neighbour max_segment_words: 100 # segments above this are split regardless of other settings diff --git a/tests/test_audio_utils.py b/tests/test_audio_utils.py index 276d545..1f96f5d 100644 --- a/tests/test_audio_utils.py +++ b/tests/test_audio_utils.py @@ -1,6 +1,5 @@ import wave import shutil -import socket import subprocess from pathlib import Path from unittest.mock import Mock @@ -173,36 +172,6 @@ def test_empty_pcm_is_rejected_before_model_loading(monkeypatch, tmp_path): model_loader.assert_not_called() -def test_missing_whisper_model_fails_without_network(monkeypatch, tmp_path): - audio_path = tmp_path / "audio.wav" - with wave.open(str(audio_path), "wb") as audio_file: - audio_file.setnchannels(1) - audio_file.setsampwidth(2) - audio_file.setframerate(16000) - audio_file.writeframes(b"\0\0") - - connect = Mock(side_effect=AssertionError("Model loading attempted a network connection")) - monkeypatch.setattr(socket.socket, "connect", connect) - monkeypatch.setattr(transformers_pipeline, "load_silero_vad", lambda: object()) - monkeypatch.setattr( - transformers_pipeline, - "get_speech_timestamps", - lambda *args, **kwargs: [{"start": 0, "end": 1}], - ) - transformers_pipeline._load_model_and_processor.cache_clear() - try: - with pytest.raises(OSError): - transformers_pipeline.transcribe( - str(audio_path), - "textplease/model-that-is-not-cached", - "cpu", - ) - finally: - transformers_pipeline._load_model_and_processor.cache_clear() - - connect.assert_not_called() - - def test_missing_file_raises(tmp_path): with pytest.raises(FileNotFoundError): normalize_audio(str(tmp_path / "nope.ogg"), tmp_path / "temporary") diff --git a/tests/test_gradio_ui.py b/tests/test_gradio_ui.py index 3c9f4ed..a41da12 100644 --- a/tests/test_gradio_ui.py +++ b/tests/test_gradio_ui.py @@ -21,7 +21,7 @@ def test_start_transcription_uses_gradio_cached_file(tmp_path): output_dir, str(upload_path), 0.75, - 2.0, + 4.5, 100, 3, 15, @@ -40,6 +40,7 @@ def test_start_transcription_uses_gradio_cached_file(tmp_path): assert run["output_path"] == workspace_path / "recording_transcript.csv" assert run["config_path"] == workspace_path / "config.yaml" assert yaml.safe_load(run["config_path"].read_text()) == config + assert config["pause_threshold"] == 4.5 if os.name != "nt": assert workspace_path.stat().st_mode & 0o777 == 0o700 assert run["config_path"].stat().st_mode & 0o777 == 0o600 @@ -232,7 +233,7 @@ def test_show_audio_info_returns_uploaded_file_for_preview(monkeypatch): assert details == "🕒 Duration: 12.34s\n📊 Sample rate: 16000 Hz\n🔊 Channels: 1" -def test_launch_gradio_disables_network_features(monkeypatch, tmp_path): +def test_launch_gradio_disables_remote_access_and_analytics(monkeypatch, tmp_path): blocks_type = gradio_ui.gr.Blocks blocks_init = blocks_type.__init__ blocks_options = {} diff --git a/tests/test_pipeline.py b/tests/test_pipeline.py index 449514d..ddcacf4 100644 --- a/tests/test_pipeline.py +++ b/tests/test_pipeline.py @@ -69,6 +69,33 @@ def test_no_speech_writes_header_only_transcript(monkeypatch, tmp_path, caplog): sentence_transformer.assert_not_called() +def test_pause_threshold_only_controls_transcript_grouping(monkeypatch, tmp_path): + input_path = tmp_path / "input.wav" + input_path.touch() + output_path = tmp_path / "output.csv" + recognized_segments = [ + {"start_time": "00:00:00.000", "end_time": "00:00:01.000", "text": "First complete segment."}, + {"start_time": "00:00:02.000", "end_time": "00:00:03.000", "text": "Second complete segment."}, + ] + transcribe_audio = Mock(return_value=recognized_segments) + segment_transcript = Mock(return_value=recognized_segments) + monkeypatch.setattr(pipeline, "transcribe_audio", transcribe_audio) + monkeypatch.setattr(pipeline, "segment_transcript", segment_transcript) + + pipeline.run_transcription_pipeline( + { + "input_path": str(input_path), + "output_path": str(output_path), + "model_name": "test-model", + "pause_threshold": 4.5, + "similarity_threshold": 1.0, + } + ) + + assert "pause_threshold" not in transcribe_audio.call_args.kwargs + assert segment_transcript.call_args.kwargs["pause_threshold"] == 4.5 + + def test_pipeline_rejects_input_as_output_before_transcription(monkeypatch, tmp_path): input_path = tmp_path / "input.wav" input_path.write_bytes(b"original audio") diff --git a/tests/test_whisper_batching.py b/tests/test_whisper_batching.py index fa330b0..28b4fca 100644 --- a/tests/test_whisper_batching.py +++ b/tests/test_whisper_batching.py @@ -123,6 +123,7 @@ def test_no_speech_skips_whisper(monkeypatch): assert transformers_pipeline.transcribe("input.wav", "test-model", "cpu") == [] model_loader.assert_not_called() + assert detector.call_args.kwargs["min_silence_duration_ms"] == 2000 assert detector.call_args.kwargs["return_seconds"] is False diff --git a/textplease/backends/transformers_pipeline.py b/textplease/backends/transformers_pipeline.py index 059e824..7fb85e3 100644 --- a/textplease/backends/transformers_pipeline.py +++ b/textplease/backends/transformers_pipeline.py @@ -49,7 +49,6 @@ def _load_model_and_processor( def _get_speech_segments( audio_array: np.ndarray, - pause_threshold: float, ) -> list[dict[str, int]]: """Run Silero VAD and return bounded speech intervals in source samples.""" vad_model = load_silero_vad() @@ -61,7 +60,7 @@ def _get_speech_segments( threshold=0.5, sampling_rate=TARGET_SAMPLE_RATE, min_speech_duration_ms=250, - min_silence_duration_ms=int(pause_threshold * 1000), + min_silence_duration_ms=2000, speech_pad_ms=100, return_seconds=False, ) @@ -215,7 +214,7 @@ def transcribe( audio_path: str, model_name: str, device: str, - pause_threshold: float = 2.0, + *, language: str = "en", batch_size: int = 1, ) -> list[dict[str, str]]: @@ -224,7 +223,7 @@ def transcribe( raise ValueError("Whisper batch size must be positive") audio_array = load_pcm_wav(audio_path) - speech_segments = _get_speech_segments(audio_array, pause_threshold) + speech_segments = _get_speech_segments(audio_array) if not speech_segments: logger.info("No speech detected") return [] diff --git a/textplease/gradio_ui.py b/textplease/gradio_ui.py index 4440c72..837b449 100644 --- a/textplease/gradio_ui.py +++ b/textplease/gradio_ui.py @@ -428,8 +428,8 @@ def launch_gradio( 10.0, step=0.1, value=2.0, - label="Pause Threshold (seconds)", - info="Silence that splits segments (also the Silero-VAD boundary)", + label="Transcript Grouping Pause (seconds)", + info="Used only when grouping recognized segments. Speech detection is automatic.", ) with gr.Row(): max_segment_words = gr.Slider( diff --git a/textplease/pipeline.py b/textplease/pipeline.py index d6ef5f4..037744a 100644 --- a/textplease/pipeline.py +++ b/textplease/pipeline.py @@ -165,7 +165,6 @@ def _execute_transcription_stage(params: dict, temporary_directory: str | Path) params["model_name"], params["device"], temporary_directory=temporary_directory, - pause_threshold=params["pause_threshold"], language=params["language"], batch_size=params["whisper_batch_size"], ) @@ -225,9 +224,10 @@ def run_transcription_pipeline(config: dict) -> None: params["whisper_batch_size"] = 4 if params["device"] == "cuda" else 1 logger.info(f"Input: {params['input_path']} → Output: {params['output_path']}") - logger.info(f"ASR: {params['model_name']} | Device: {params['device']} | Pause: {params['pause_threshold']}s") + logger.info(f"ASR: {params['model_name']} | Device: {params['device']}") logger.info( - f"Embedding: {params['embedding_model_name']} | Similarity threshold: {params['similarity_threshold']}" + f"Segmentation: {params['embedding_model_name']} | " + f"Similarity threshold: {params['similarity_threshold']} | Grouping pause: {params['pause_threshold']}s" ) output_parent = Path(params["output_path"]).resolve().parent diff --git a/textplease/transcriber.py b/textplease/transcriber.py index 29f629b..99882b3 100644 --- a/textplease/transcriber.py +++ b/textplease/transcriber.py @@ -12,26 +12,18 @@ def transcribe_audio( audio_path: str, model_name: str, device: str, - pause_threshold: float = 2.0, + *, language: str | None = None, batch_size: int = 1, - *, temporary_directory: str | Path, ) -> list[dict]: """Normalize local media for the built-in Whisper runtime and transcribe it.""" - audio_file = Path(audio_path) - if not audio_file.exists(): - raise FileNotFoundError(f"Audio file not found: {audio_path}") - if not audio_file.is_file(): - raise ValueError(f"Audio path is not a file: {audio_path}") - logger.info(f"Transcribing with model: {model_name}") normalized_audio_path = normalize_audio(audio_path, temporary_directory) return whisper_transcribe( audio_path=normalized_audio_path, model_name=model_name, device=device, - pause_threshold=pause_threshold, language=language or "en", batch_size=batch_size, )