Add deliberate gesture activation and mute for Realtime conversations

[janitor-manager]

User-approved next-work order: #51 caption mute privacy, #48 clean model shutdown, then this gesture-controlled Realtime work before live overlay-tool acceptance and #50 multilingual captions.

Franci explicitly approved a held victory gesture (two fingers) to activate the agent and a held closed fist to mute/stop it. Provide visible on/off/listening/responding state and a manual fallback. Treat the Realtime settings switch as configured/available, not as proof the agent is listening. Conversation must work while another app uses AI Camera; preserve camera/microphone client demand and the meeting audio path. Stop/mute must close AI input, cancel responses/tools/captions, stop agent playback, and reject late work. Repeated held gestures must not retrigger; use confidence, dwell, release/rearm, cooldown, and stale-frame checks.

Current root cause of no automatic response: canStartRealtimeConversation disallows externalClientIsUsingMedia, only the local one-shot Talk action starts Realtime, and PipelineCoordinator gesture handling explicitly excludes realtimeEnabled. Implement a coherent deliberate activation lifecycle rather than silently listening whenever Realtime is configured.

Franci is away and authorizes progress. Use synthetic speech/gesture sequences and fake Realtime sessions for lifecycle, interruption, multi-turn, client takeover, and privacy tests; do not depend on his presence or prompt for credentials. Spoken/physical-gesture user acceptance remains separate.

[janitor-manager]

Implementation and unattended validation delivered in 891182b; signed build 42 is installed. Held victory or Start agent deliberately activates Realtime during existing camera/call demand. Held fist engages the persistent AI Camera privacy mute; victory cannot unmute it. Manual Unmute is explicit. Configuration alone never starts agent egress.

The same server session now survives successive turns. Input is closed during replies and reopens after call and local reply output drain. A separate bounded local monitor makes replies audible when the primary graph serves AI Camera Microphone. Agent demand is separate from call/test demand; Stop releases only agent ownership. Changing virtual-microphone routing stops the agent while the graph rebuilds, then requires an explicit restart. Queued gesture timestamps, late connection completion, tool execution, audio buffers, and caption generations are guarded across stop/mute/restart.

Validation: full local validation passes 168 Swift tests, unsigned four-target builds, script/metadata checks, and the HAL harness. Tests cover call/agent ownership, camera-only call demand, privacy mute with video retained, permissions, and routing changes. The real WebSocket session passes a fake-socket multi-turn/VAD/PCM/tool/cancellation fixture. The actual reply monitor passes offline 44.1/48 kHz PCM/gain/mute/reset tests and a 100 ms all-zero physical-output playback drain. Native caption privacy and existing controlled caption regressions pass. Installed UI confirms Start agent, disabled Agent muted, explicit Unmute recovery, and gesture help. Source/installed strict signatures and both versions/protected marker are 42. The app is idle, agent off, with the prior unmuted preference restored. No system component changes or captured-media files.

One optional public Codex Realtime-to-overlay probe used synthetic instructions with all Keychain UI forbidden. Its single credential read returned OSStatus -25293; no prompt, refresh, network session, or microphone capture occurred. Keep the issue open for the separate live gesture/provider/call acceptance when Franci returns. Automatic receiving-app mute integration remains tracked by #51; multilingual captions remain #50.

[janitor-manager]

Build 42 manual acceptance: Franci reported a held victory gesture did not start Realtime in QuickTime Movie Recording. Investigate the actual outgoing-camera path, add clear state/hold feedback with a native animated status blob adapted from RaccoonOps, and inset the top overlays below QuickTime’s floating title bar. Realtime is already routed from the published capture lane in 42; do not assume the failed activation was a popup-only restriction. Keep muted audio boundaries and synthetic validation.

[janitor-manager]

Delivered 786292b on main and installed protected signed Release build 43.

The published camera now includes a native RaccoonOps-style animated status orb: off, connecting, listening, thinking, speaking, muted, or unavailable, with fresh gesture hold/clarity feedback. Both top corners are inset below floating client chrome. Gesture confidence aggregates the 14 usable landmarks instead of using the least-confident folded joint; deliberate confidence/dwell/stale/rearm checks remain. Hand inference has its own bounded queue.

Validation: all 171 Swift tests, full unsigned build/script/metadata/HAL validation, native coordinator victory→fist with Realtime enabled, caption privacy across delayed completions, and synthetic pixel checks for all seven states, top margin, visibility toggle, and animation pass. Source/installed signatures verify; both versions and install marker are 43.

Installed acceptance used QuickTime New Movie Recording with AI Camera and AI Camera Microphone selected (no recording). Start agent reached Listening under external camera+microphone demand; Stop preserved both call lanes; Mute retained camera and disabled microphone/agent/captions; explicit Unmute restored availability. Closed the temporary preview and left the app idle and unmuted. No extension/driver change or saved capture.

Physical held-victory recognition and user-heard speech still need Franci’s retry; the connection test used the installed manual action. Keep this issue open for that separate acceptance.

[janitor-manager]

Franci confirms physical held-victory activation now works in build 43. Closed-fist mute is unreliable because the visible gesture is usually classified as pinch. The classifier checks thumb/index proximity before compact all-finger flexion. Fix compact fist precedence, test tucked-thumb variants and transformations through classifier → control gate → privacy callback, and retain genuine pinch classification. Deliver a signed host update without system-component changes.

[janitor-manager]

Delivered 9e163ef and installed signed host build 44. Compact four-finger flexion now takes precedence over thumb–index proximity, so a tucked thumb cannot turn the closed-fist mute pose into pinch. Existing confidence/dwell/freshness/rearm rules are unchanged; non-fist pinches remain recognized.

Regression evidence: the old classifier failed all 72 thumb-offset/rotation/scale/mirror fist variants and never emitted mute in a held victory→fist sequence. The corrected classifier passes those cases, genuine-pinch controls, all 175 Swift tests, and full build/script/metadata/HAL validation. The native privacy harness now derives the fist from actual synthetic landmarks and delivers exactly one start and mute through the Realtime-enabled coordinator.

Protected signed installation/relaunch completed. Strict nested signatures pass for the source and installed app; both versions and the install marker are 44. No system component changed or user media was saved. Repository clean and main pushed.

Franci's physical victory activation is confirmed; keep this issue open for his closed-fist retry on build 44.