voice-ai

Private, local voice dictation for Fedora, GNOME, Wayland, PipeWire, and NVIDIA GPUs.

Press F8 to start recording. Press F8 again to stop, transcribe locally, and paste the recognized text into the focused application.

The production engine is written in Rust and runs the unquantized Whisper medium.en FP16 model through whisper.cpp and Vulkan. The model stays warm while the engine is on, then shuts down after 15 idle minutes. No speech or transcript data is sent to an external service.

Features

• Local English speech-to-text with no cloud dependency.
• F8 press-to-start and press-to-stop workflow.
• Persistent warm model for low stop-to-paste latency.
• Explicit force-off that releases host memory and GPU memory.
• Automatic shutdown after 15 idle minutes.
• NVIDIA GPU offload with no silent CPU fallback.
• Optional PipeWire microphone selection.
• Bluetooth headset profile switching and restoration.
• Silence detection to avoid hallucinated transcripts.
• Side-by-side validation against the preserved legacy implementation.
• Reproducible user-level installation with checksummed model weights.

Requirements

This implementation has been built and verified on Fedora 44 with GNOME, Wayland, PipeWire, and an NVIDIA RTX 3060 Laptop GPU.

Rust 1.96 or newer is required to build the project. Install the Fedora dependencies with:

sudo dnf install cmake glslc vulkan-loader-devel gcc-c++ pkgconf-pkg-config \
    pipewire-utils wl-clipboard ydotool libnotify curl

The NVIDIA driver, nvidia-smi, and a working Vulkan device are required. The dictation engine refuses to continue if it cannot verify its own NVIDIA GPU allocation.

Quick start

git clone https://github.com/TheRealShek/voice-ai.git
cd voice-ai
scripts/install.sh

If models/ggml-medium.en.bin already exists and has the expected checksum, the installer reuses it through a symlink. Otherwise, download the model:

scripts/download-model.sh

Activate the F8 shortcut and start the engine:

scripts/activate-f8.sh
voice-ai on

The engine also starts automatically when voice-ai toggle-record or F8 is used while it is off.

Using F8

1. Focus the application where the transcript should appear.
2. Press F8 to start a dictation.
3. Speak normally.
4. Press F8 again to stop recording.
5. The transcript is copied and pasted into the focused application.

F8 runs:

~/.local/bin/voice-ai toggle-record

Command reference

Engine lifecycle

voice-ai on

Starts the user service, loads the model, performs GPU warm-up, and waits until the engine is ready.

voice-ai off

Force-cancels active recording or transcription, pastes nothing from canceled work, restores temporary audio changes, stops the service, and releases its host-memory and GPU-memory allocations. Use this before gaming.

voice-ai status

Reports one of these states:

• off: no dictation engine process or model allocation.
• starting: model loading or warm-up is in progress.
• warm: ready for a dictation.
• recording: audio capture is active.
• transcribing: a completed dictation is being processed.
• error: the engine could not become available.

voice-ai toggle-power

Starts and warms the engine when it is off, or forces it off when it is on.

Dictation

voice-ai toggle-record

Starts a new dictation or stops the current dictation and pastes its transcript. This is the command bound to F8.

Microphones

voice-ai microphones

Lists PipeWire microphone source names that can be selected for dictation.

voice-ai set-microphone SOURCE

Persists a selected microphone. Example:

voice-ai set-microphone bluez_input.1C:1F:00:00:03:C0

voice-ai clear-microphone

Clears the selected microphone and follows the current PipeWire input.

File transcription and validation

voice-ai transcribe recording.wav

Loads the configured model and transcribes one 16 kHz WAV file without copying or pasting the result.

voice-ai transcribe --model /path/to/model.bin recording.wav

Transcribes one file with an explicit model path.

voice-ai validate sample-1.wav sample-2.wav

Loads and warms the configured model once, then transcribes multiple WAV samples without copying or pasting their text.

voice-ai validate --model /path/to/model.bin sample.wav

Runs validation with an explicit model path.

Help and version

voice-ai --help
voice-ai COMMAND --help
voice-ai --version

Configuration

The installed configuration is:

~/.config/voice-ai/config.toml

All defaults and available fields are documented in config/voice-ai.toml.example.

Important settings include:

• model_path: external Whisper model file.
• vulkan_device: Vulkan device filter used for GPU offload.
• inference_threads: CPU threads used for supporting inference work.
• idle_timeout_seconds: time before an unused warm engine exits.
• max_recording_seconds: maximum duration of one dictation.
• microphone: optional selected PipeWire input.
• bluetooth_profile_switch: whether Bluetooth profile switching is enabled.
• silence_rms_threshold and no_speech_threshold: no-speech filtering controls.

The checked-in vulkan_device = "1" setting selects the RTX 3060 on the verified hybrid-GPU machine. Verify the Vulkan device order before using this value on another system. CPU fallback is not allowed.

Model weights

Model weights are intentionally excluded from Git. The production model is ggml-medium.en.bin from the official whisper.cpp model repository.

Default installed location:

~/.local/share/voice-ai/models/ggml-medium.en.bin

Required SHA-256:

cc37e93478338ec7700281a7ac30a10128929eb8f427dda2e865faa8f6da4356

scripts/download-model.sh supports resumed downloads and refuses to install a model with a different checksum. scripts/install.sh can reuse a verified model from models/ggml-medium.en.bin without copying the 1.5 GB file.

Repository scripts

Installation and desktop integration

scripts/install.sh

Builds the locked release, installs the user-level binary and systemd service, creates the configuration when absent, and reuses a verified local model when available.

scripts/download-model.sh

Downloads and checksum-verifies the production model.

scripts/activate-f8.sh

Changes the existing GNOME Toggle Dictation binding to voice-ai toggle-record. It refuses to replace an unexpected F8 command.

scripts/uninstall.sh

Stops the service and removes the installed binary and service definition. Configuration and model weights are preserved.

Validation and diagnostics

scripts/check-lifecycle.sh

Runs the real model and GPU lifecycle regression. It verifies singleton enforcement, warm-up, zero-exit shutdown, and control-socket removal.

scripts/compare-transcripts.sh sample-1.wav sample-2.wav

Compares the preserved legacy small.en INT8 profile with the production medium.en FP16 profile. It reports transcripts, process time, PSS, RSS, observed available-memory change, and process-specific GPU memory without pasting text.

scripts/measure-profile.py LABEL COMMAND [ARGUMENTS...]

Internal measurement helper used by transcript comparison. It runs one process and reports its time and resource use without storing the transcript on disk.

Preserved legacy commands

scripts/toggle_dictation.sh

The original Python and faster-whisper F8 implementation. It remains available as a fallback but is no longer the production shortcut.

./bin/python scripts/dictate.py

Legacy Python transcription helper retained with the original implementation.

scripts/toggle_dictation.sh.bak is an archived copy, not a production command.

Manual installation

The installer performs these user-level steps:

cargo build --locked --release
install -Dm755 target/release/voice-ai ~/.local/bin/voice-ai
install -Dm644 systemd/voice-ai.service ~/.config/systemd/user/voice-ai.service
mkdir -p ~/.config/voice-ai
test -e ~/.config/voice-ai/config.toml || \
    install -m644 config/voice-ai.toml.example ~/.config/voice-ai/config.toml
systemctl --user daemon-reload

Installed files:

• ~/.local/bin/voice-ai
• ~/.config/systemd/user/voice-ai.service
• ~/.config/voice-ai/config.toml
• ~/.local/share/voice-ai/models/ggml-medium.en.bin, or a symlink to a verified model

The service is started on demand. It is not enabled at login.

Development checks

cargo fmt --all -- --check
cargo test --all-targets
cargo clippy --all-targets -- -D warnings
cargo build --locked --release
bash -n scripts/*.sh
python3 -m py_compile scripts/measure-profile.py

Run the real GPU lifecycle check when the model exists at models/ggml-medium.en.bin:

scripts/check-lifecycle.sh

Performance

Measurements from the verified RTX 3060 laptop:

Check	Result
Warm inference for a 14.27-second recording	0.31 s
Installed warm-engine PSS	93.7 MiB
Installed warm-engine RSS	154.0 MiB
NVIDIA GPU allocation	1,876 MiB
Production force-off	332 ms

The same validation recording measured approximately 1,299 MiB peak PSS with the legacy Python and CTranslate2 profile and 176 MiB peak PSS with the native production process.

Privacy and failure handling

• Audio and transcripts remain local.
• Transcripts and successful recordings are not logged.
• Successful and silent recordings are deleted immediately.
• The latest failed recording is retained for diagnosis.
• Force-off deletes canceled audio and pastes nothing from canceled work.
• Validation commands print transcripts but never copy or paste them.
• Runtime files use a private per-user directory.

Troubleshooting

Check the engine and user service:

voice-ai status
systemctl --user status voice-ai.service

Inspect recent service logs:

journalctl --user -u voice-ai.service -n 100 --no-pager

Completely release the engine before retrying:

voice-ai off
voice-ai on

List microphone source names and select one explicitly when the current PipeWire input is incorrect:

voice-ai microphones
voice-ai set-microphone SOURCE

Internal service commands

The binary also contains hidden commands used by systemd and lifecycle tests:

• voice-ai daemon
• voice-ai shutdown-daemon
• voice-ai restore-audio-profile

These commands are implementation details and should not normally be run manually.

Project documentation

• CONTEXT.md: agreed terminology, resource limits, and lifecycle semantics.
• docs/WORKLOG.md: implementation history and measured checkpoints.
• docs/QUALITY_VALIDATION.md: transcript comparison and production acceptance record.
• docs/adr/0001-use-native-persistent-whisper-engine.md: architecture decision and trade-offs.

Legacy fallback

The legacy runtime uses Python 3.12 and the exact packages recorded in legacy/requirements.lock.txt. Its generated virtual environment remains excluded from Git. The legacy path is preserved for rollback and comparison, while the Rust and Vulkan engine is the production implementation.