NOVA

NOVA (Neural Operations Voice Assistant) is an AI voice assistant for macOS

Capabilities

Capable of the following:

1. Listening for wakeword
2. Continuous audio streaming
3. STT
4. Silence detection
5. LLM processing user command
6. Tool call returning and execution

Tech Stack

1. openWakeWord - Wake Word detection
2. sounddevice - audio streaming
3. faster-whisper - STT
4. Piper - TTS
5. numpy - Silence detection and audio chunks concatenation
6. Ollama qwen3:8b model - LLM for command processing and tool calling
7. soundfile - Reading audio files for TTS
8. pyautogui - Automating mouse and keyboard actions
9. pathlib - Making files tool
10. applescript - controlling macOS application through system commands
11. Python built-in modules
    1. os - For paths
    2. time - For testing
    3. wave - Opening audio files for TTS
    4. subprocess - Running system commands

Pipeline

User says "Hey NOVA"

↓

Wakeword detects and starts listening for command

↓

User completes command and silence is detected; transcription begins

↓

Command is transcribed and sent to LLM for processing

↓

LLM determines what tools to call and returns tool name and params

↓

Tools are executed and NOVA starts listening for Wakeword again

What I learned

The inspiration behind NOVA was Iron Man's JARVIS. Although it isn't as intelligent and capable as JARVIS, NOVA has taught me a lot about the concepts behind voice agents, such as TTS, STT and training voice recognition models. It also taught me a lot about automation, as it can open and close apps and control volume and brightness to name a few. I will continue working on this project and make it more advanced and capable than it is now.

Future update plans

I would like to add memory to NOVA to make it feel more personal. I am also planning to add OCR so it can "see" my screen and help me with problems. MCP servers would also be a great addition to NOVA, as it would allow it to connect to external tools and increase its capabilities.