diff --git a/README.md b/README.md index 9b76472..76e2b2b 100644 --- a/README.md +++ b/README.md @@ -1,121 +1,174 @@ # GPT-VoiceCoding -Voice-controlling terminal coding agents through a realtime voice call: the system -speaks agent progress to you, and carries your spoken instructions back to the -agents. - -> **Status: early build.** Every part is now built and tested — Bridge Core, the -> control plane, the composition root, the Codex and Claude Agent adapters with -> all three Relays, the bridge-owned Live Call, the Companion Channel, the -> menu-bar shell, and the signed app bundle. Starting and closing sessions are -> deliberately not among them: v1.0 is a bridge over the sessions you start -> yourself, and the Session Launcher is parked for a later release. It has not -> been through a release. If you want software with users on it, the -> first-generation implementation lives at -> [GPT-VoiceCoding-legacy](https://github.com/okqixiaobao727-design/GPT-VoiceCoding-legacy). +**for Claude Code & Codex** + +![macOS](https://img.shields.io/badge/macOS-only-111111) +![Apple Silicon](https://img.shields.io/badge/Apple%20Silicon-required-111111) +![Licence](https://img.shields.io/badge/licence-MIT-111111) + + + +One voice call covers every Claude Code and Codex session on your Mac: it tells you which one +stopped and what it stopped on, and carries your spoken answer back into it. The call rides +`codex app-server`'s realtime route, so it runs on the ChatGPT account you already signed +`codex` into, and there is no API key to supply. + +## How it works + +- A session stops and it calls you. The bridge watches the sessions you started yourself; it + does not launch them and does not wrap them. +- You answer by voice. You hear what the session stopped on, say what to do, and your words go + back in as an instruction that session can act on. +- Away from the desk, you answer on Telegram. Every stop arrives there too, and a reply to it + reaches the same session. + +## Screenshots + +| | | +|---|---| +| ![The Duty Lamp](docs/images/duty-lamp.png)
The Duty Lamp sits above everything and counts what is waiting on you. Hovering it offers Hang up, Control and Settings, and shows the session that moved last. | ![The Control Panel](docs/images/control-panel.png)
The Control Panel during a call: the three switches, the Call Agent's model and context use, the counts, and every live session. | +| ![Settings](docs/images/settings.png)
Settings. Nine voices to pick the Live Call's from, and the realtime model behind it. | ![Telegram](docs/images/telegram.png)
Telegram. A stop notice you can reply to, `/sessions` drawn as buttons, and the bot's own command menu. | ## Install -On an Apple Silicon Mac, run this in your terminal: +On an Apple Silicon Mac: ```sh curl -fsSL https://raw.githubusercontent.com/okqixiaobao727-design/GPT-VoiceCoding/main/scripts/install.sh | sh ``` -You need Apple's Command Line Tools (`xcode-select --install`), Python **3.12 or -newer** available as `python3` on your PATH, and HTTPS access to GitHub and PyPI. -The script checks these in order and stops with a fix if one is missing; it -does not install Homebrew or Python. It builds the full ad-hoc-signed app from -source under `~/Library/Application Support/GPT-VoiceCoding/source`, quits a -running copy, places the new bundle in `/Applications`, and opens it. There is -no notarized download. - -**Upgrade by running the same command again.** Your existing `config.toml` and -Telegram credential file stay in place. First launch creates a configuration -only if none exists, then walks through Codex login, the installed helpers, -Call Agent, optional Telegram, and a test call. Have Codex installed and log in -with `codex login` when the guide asks. Later launches are silent. Settings that -need an engine restart show one explicit **Restart now** action. - -Telegram setup (in the guide or Settings) first checks the bot token. The token -identifies the bot, not the receiving chat. Search for the displayed `@username` -in whichever Telegram client you already use and send `/start` after the token -check. Return to the app, click **I've sent /start — check**, and save after you -see the confirmation message in that chat. The app does not open a Telegram -client for you or check for that message until you click. - -The Duty Card starts at the top right of the available screen on a fresh -installation; subsequent launches restore its saved position. - -## What it is - -You are away from the keyboard. A Claude Code or Codex session stops and needs -you. GPT-VoiceCoding tells you — out loud, in a voice call it holds open — and -carries your spoken answer back into the session. It watches the sessions you -start, relays your words into them, and answers their permission prompts with -your verdict. It does not start them — you do, with your own `claude` or `codex` -command, and the bridge covers whatever is running. - -The voice call is owned by this system directly, over `codex app-server`'s realtime -route. There is no API key to supply. - -## Shape - -A thin Swift menu-bar app spawns a single Python asyncio engine from inside its own -bundle. That engine is **Bridge Core**: it owns every policy and holds the single -source of truth, and reaches everything else — the call, the coding agents, the -Companion Channel — through seams with swappable adapters. +It builds an ad-hoc-signed app from source, puts it in `/Applications` and opens it. There is no +notarized download. Run the same command again to upgrade; your configuration and Telegram +credentials stay where they are. + +You need: + +- An Apple Silicon Mac, with Apple's Command Line Tools (`xcode-select --install`). +- Python 3.12 or newer, as `python3` on your PATH. The installer checks for it and stops with a + fix rather than installing one. +- Claude Code or Codex, whichever you use. Both work, together or alone. +- Codex signed in with a ChatGPT account (`codex login`). The Live Call needs it. +- Telegram, if you want the second route. It is optional, and first launch walks you through it. + +## Architecture + +A thin Swift menu-bar app spawns one Python asyncio engine from inside its own bundle. That +engine is **Bridge Core**: it owns every policy and holds the single source of truth, and +reaches everything else through seams with swappable adapters. + +The two agents are reached differently because they publish themselves differently. Claude Code +is read from its own roster and two hooks, and written to through the inbox socket every session +already binds. Codex is read and written through the shared `codex app-server` this product +starts and your own terminals join, which is the same process the Live Call's realtime route +rides. + +```mermaid +flowchart LR + subgraph terminals["Your own terminals"] + cc["Claude Code Session"] + cx["Codex Session"] + end + + appserver["codex app-server
started by this product,
joined by your Sessions"] + + subgraph engine["The engine"] + core["Bridge Core
every policy, one source of truth"] + agents["Agent adapters"] + callad["Call adapter"] + chan["Companion Channel adapter"] + plane["Control plane
JSON over a Unix socket"] + core --- agents + core --- callad + core --- chan + core --- plane + end + + live["Live Call
Voice speaks, Call Agent acts"] + tg["Telegram"] + shell["Menu-bar app
Duty Lamp, Control Panel"] + ctl["bridgectl"] + + cc <-->|"hooks, roster, inbox socket"| agents + cx --- appserver + appserver --- agents + appserver -->|"realtime route"| callad + callad --- live + chan --- tg + plane --- shell + plane --- ctl +``` + +What happens when a session stops: + +```mermaid +sequenceDiagram + participant S as Your Session + participant C as Bridge Core + participant T as Telegram + participant K as Call Keeper + participant V as Voice + participant A as Call Agent + participant U as You + + S->>C: stops, waiting on a decision + C->>C: writes the reading to the roster + C->>T: Stop Notice, as an Anchor you can reply to + C->>K: wake + K->>V: dials, briefed from the roster as it stands + V->>U: speaks the Session Brief + U->>V: "use email login, and give me the steps" + V->>A: hands the job over + A->>C: relay + C->>S: Answer Relay +``` + +Or, without the call: + +```mermaid +sequenceDiagram + participant T as Telegram + participant U as You + participant C as Bridge Core + participant S as Your Session + + T->>U: Stop Notice + U->>T: replies to it + T->>C: inbound text, naming the Anchor's Session + C->>S: Answer Relay +``` + +The Voice has no tools. It speaks from what the engine hands it and passes anything that reads +as a job to the Call Agent, which is the only half on the call that can run a control-plane verb. ``` src/gpt_voicecoding/ ├── core/ Bridge Core — the hub. All policy, all state. -├── seams/ The interfaces Bridge Core calls through, and the control plane's vocabulary. +├── seams/ The interfaces Bridge Core calls through. ├── adapters/ The implementations behind them. Protocol libraries live only here. ├── control_plane/ The JSON-over-UDS surface: framing, sockets, translation. ├── engine/ The composition root — config in, one running engine out. ├── installation/ What this product puts in files the user owns, and takes back. └── cli/ bridgectl — a control-plane surface. -shell/ The Swift menu-bar shell (see ADR 0005). -app_bundle/ The build pipeline. Builds the product; is not part of it. -``` - -One command turns a clean checkout into a signed `.app`: - -```bash -scripts/build-app.sh +shell/ The Swift menu-bar shell. ``` -See [`docs/app-bundle.md`](docs/app-bundle.md). - -The engine runs standalone, without the menu-bar shell: -`python -m gpt_voicecoding.engine --config `. The Live Call's audio path is -an optional extra — `pip install 'gpt-voicecoding[voice]'` — because everything -else runs without a compiled media stack. See -[`docs/control-plane.md`](docs/control-plane.md) for the interface, the command -set and the configuration file. - -Start with [`docs/adr/0001`](docs/adr/0001-hub-and-spoke-bridge-core-with-seams.md). - -## Platform - -macOS only. The microphone grant, the menu-bar shell and the push path are all -macOS-shaped; cross-platform support is not planned. - -## Reading order - -1. [`CONTEXT.md`](CONTEXT.md) — the vocabulary. Every term in this repo means what - it says there. -2. [`docs/adr/`](docs/adr/README.md) — the decisions, and where each one came from. -3. [`docs/control-plane.md`](docs/control-plane.md) — the interface every surface - speaks, and what the engine is configured with. -4. [`docs/app-bundle.md`](docs/app-bundle.md) — how the `.app` is built and - signed, and every decision that pipeline holds. +The engine runs without the menu-bar app: `python -m gpt_voicecoding.engine --config `. +The Live Call's audio path is an optional extra, `pip install 'gpt-voicecoding[voice]'`, because +everything else runs without a compiled media stack. -## Contributing +Start with [`CONTEXT.md`](CONTEXT.md) for what each term means, then +[`docs/adr/0001`](docs/adr/0001-hub-and-spoke-bridge-core-with-seams.md) for why the shape is +this one. [`docs/control-plane.md`](docs/control-plane.md) has the interface, the command set and +the configuration file; [`docs/app-bundle.md`](docs/app-bundle.md) has the build. -v0 targets developers: build from source, no signed release, no notarization -(ADR 0005). Issues and pull requests are welcome once there is something to build -on — the build issues land next. +macOS only. The microphone grant, the menu-bar app and the push path are all macOS-shaped, and +cross-platform support is not planned. ## Licence diff --git a/docs/images/control-panel.png b/docs/images/control-panel.png new file mode 100644 index 0000000..4aada0f Binary files /dev/null and b/docs/images/control-panel.png differ diff --git a/docs/images/duty-lamp.png b/docs/images/duty-lamp.png new file mode 100644 index 0000000..1bccd84 Binary files /dev/null and b/docs/images/duty-lamp.png differ diff --git a/docs/images/settings.png b/docs/images/settings.png new file mode 100644 index 0000000..f6bb035 Binary files /dev/null and b/docs/images/settings.png differ diff --git a/docs/images/telegram.png b/docs/images/telegram.png new file mode 100644 index 0000000..dae9f86 Binary files /dev/null and b/docs/images/telegram.png differ