Skip to content

Codex: complete Mac assistant and Desktop integration - #315

Open
shawn-t516 wants to merge 21 commits into
Louis-CFM:mainfrom
shawn-t516:codex/native-integration
Open

shawn-t516 wants to merge 21 commits into
Louis-CFM:mainfrom
shawn-t516:codex/native-integration

Conversation

@shawn-t516

@shawn-t516 shawn-t516 commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Add Codex account support to the existing Mac assistant features: Flow cleanup, prompt shaping and translation; an explicit Notetaker summary/correction engine; and natural-language voice actions and web answers. Selecting Codex uses its connected ChatGPT account without an Anthropic API key or provider fallback. Failed requests preserve the original dictation and notes.

Show managed Codex plan text and step progress separately from permission decisions. Reconcile completed file changes and cumulative turn snapshots in Mac diff cards and recaps, retaining stable file IDs and the existing bounded history. Later edit chunks stay visible until the next aggregate corrects net counts. A file menu exposes up to 50 changed files, including empty operations and renames.

This extends the native Codex integration already in this PR: project-scoped streaming chat, model discovery, attachments, read-only defaults with explicit write opt-in, Connected/Disconnect OAuth status, quota/token usage, trusted-hook diagnostics, correct Desktop/IDE return actions, and identity-checked Mac/iPhone question and approval routes. External conversations retain their original transport and owner.

For an external Codex Desktop question, install/enable the explicitly invoked $coucou-question plugin from Settings → Agents → Codex → Desktop plugin. Its answer returns to the original waiting MCP call. Existing PermissionRequest hooks handle supported external approvals. Arbitrary built-in questions from an unrelated Desktop conversation still require returning to Codex; this PR does not intercept them, infer their ownership from a project path or inject an answer into another connection.

The branch incorporates upstream 47d1c2da12bf1dce8fb28b825757d497c1e1d88d (Mac 0.5.3). Traditional Chinese from #314 is already upstream via #821; this adds the remaining Codex labels on that baseline. Background mode defaults to menu-bar only while preserving saved Dock choices; closing a window keeps the service alive and explicit Quit terminates it.

Architecture / flow

flowchart TD
    UI[Flow / Notes / Voice] --> Enable[Explicit background setup]
    Enable --> Text[CodexTextService]
    Text --> RPC[Isolated CodexConnection]
    Chat[Project chat] --> Own[Owned CodexConnection]
    Own --> Events[CodexEventAdapter]
    Desktop[Desktop hooks / question plugin] --> Events
    Events --> Mac[Questions / plans / diffs]
    Events --> Phone[iPhone identity checked routes]
    style Text fill:#dff0d8,stroke:#3c763d
    style Events fill:#dff0d8,stroke:#3c763d
    style Mac fill:#dff0d8,stroke:#3c763d
Loading

See plan.md for the approved scope. No separate issue is referenced.

Isolation and behavior

  • Text requests use a separate ephemeral Codex process/thread and empty workspace, with local tools, MCP, hooks, plugins, skills and agents disabled. They do not reuse or reset the project chat.
  • Background text features are opt-in after ChatGPT connects. Explicit setup checks only the account and lets the user authorize the native CLI to read one dedicated Keychain item; it sends no text to a model. The stable private home retains that item’s ACL, while each request has its own process, workspace, log and database paths. The item contains access/id/account data, never the native refresh token or an API key; idle/shutdown attempts to clear its access data and Disable attempts to remove the exact owned item. Cleanup is best-effort if Keychain is locked or unavailable; remaining data is still access-only and has no refresh authority. Original credential files/items and native configuration remain unchanged. Credentials are never copied to a plaintext temporary file. Cleanup follows subprocess exit on success, error, timeout and cancellation.
  • Native credential-store selection is respected; unsupported or inaccessible formats fail closed. Codex CLI 0.163.0+ and macOS Keychain/sandbox support are required. Expired access requires refreshing the normal Codex login.
  • Voice proposals pass a bounded schema/argument allowlist before the existing Coucou action dispatcher. Duplicate actions are not replayed. Cancellation, provider/account changes and stale utterances suppress old actions and UI updates. Mail opens a draft; Send remains a user action.
  • Plan notifications never grant permissions or invoke Claude ExitPlanMode. Questions and approvals retain thread/turn/request/socket identity and explicit decisions.

Verification

  • Passed — 33 focused regression commands: native auth/IPC/queue/ownership/Phone routes, plugin/export/socket lifetime, source/status/hooks, background mode, Flow/Notetaker data preservation, strict voice arguments/actions/controller lifetime, plan and diff reconciliation, existing voice runner/query/intent, geometry and translations. test-voice-runner.sh passes 111 assertions.
  • Passed — Xcode 27.0 (27A266a), signing disabled: GitHub NotchBuddy Release arm64+x86_64, CoucouAppStore Debug arm64 and CoucouPhone Debug all passed. Warnings match the earlier baseline; this does not claim the repository is warning-free.
  • Passed — 1,113 fully translated Traditional Chinese catalog entries passed format/plural/literal checks. All previous 1,088 entries/locales/metadata are preserved; 25 English/Hant entries are new.
  • Bundle checks preserve GitHub 0.5.3 (20), App Store 1.1 (6), bundle IDs and LSUIElement. GitHub packages the same six Desktop plugin files; App Store excludes the plugin and native credential bridge.
  • Archify source evidence and automated validation are included under docs/architecture/; no browser/perceptual acceptance is claimed.

The text fixtures additionally cover stable Keychain-account reuse, blocked-source deadlines, cancellation, shutdown rejecting new work, independent project-chat survival and native credentials remaining unchanged. Diff fixtures cover rename preservation, empty files, spaced/quoted paths and late completions after an authoritative undo/empty snapshot. A controlled native 0.163.0 loopback-model probe verified the effective tool profile without a live model or real credentials.

The fixtures use fake credentials, temporary processes/files/sockets and in-memory service boundaries. They do not authenticate a real account, invoke a real model, access real Contacts/Music or install a plugin.

For a fresh checkout, clone https://github.com/shawn-t516/coucou.git at branch codex/native-integration and work from its repository root. Use Xcode 27 and XcodeGen; normal dependency resolution needs network access. These focused fixtures need no account or API key. Each exits zero on success, with temporary fixture data cleaned by its script.

Reproduce:

export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer
xcodegen generate --spec NotchBuddy/project.yml
python3 scripts/test-localization.py
python3 scripts/test-codex-plugin.py
bash scripts/test-codex-text.sh
bash scripts/test-codex-flow-notes.sh
bash scripts/test-codex-voice.sh
bash scripts/test-codex-voice-actions.sh
bash scripts/test-codex-voice-controller.sh
bash scripts/test-codex-turn-state.sh
bash scripts/test-codex-parity.sh
bash scripts/test-codex-services.sh
bash scripts/test-codex-session-queue.sh
bash scripts/test-codex-plugin-mac.sh
bash scripts/test-codex-plugin-export.sh
bash scripts/test-phone-codex.sh
bash scripts/test-background-mode.sh
bash scripts/test-flow-text.sh
bash scripts/test-notetaker.sh
bash scripts/test-voice-runner.sh
xcodebuild -project NotchBuddy/NotchBuddy.xcodeproj -scheme NotchBuddy -configuration Release -destination platform=macOS CODE_SIGNING_ALLOWED=NO CODE_SIGNING_REQUIRED=NO CODE_SIGN_IDENTITY= build
xcodebuild -project NotchBuddy/NotchBuddy.xcodeproj -scheme CoucouAppStore -configuration Debug -destination platform=macOS CODE_SIGNING_ALLOWED=NO CODE_SIGNING_REQUIRED=NO CODE_SIGN_IDENTITY= build
xcodebuild -project NotchBuddy/NotchBuddy.xcodeproj -scheme CoucouPhone -configuration Debug -destination generic/platform=iOS CODE_SIGNING_ALLOWED=NO CODE_SIGNING_REQUIRED=NO CODE_SIGN_IDENTITY= build

See CONTRIBUTING.md for the remaining regression commands. Local results and GitHub Actions status are separate.

Manual acceptance and limits

Not run for this increment: a real ChatGPT/model session, Keychain authorization in the distributed signed app, microphone/Contacts/Music/mail behavior, linguistic/layout review, Desktop plugin installation/trust, VS Code entry paths, iPhone/CloudKit/notification Reply/Siri or extended soak testing.

  1. Sign in with ChatGPT and choose Codex in Chat. Click Enable background features; complete the macOS prompt for Codex Auth or Coucou temporary Codex access, choosing Always Allow. Check the separate Connected and Enabled statuses, retry/disable and quit. With no Anthropic key, try Flow cleanup/translation, Codex Notetaker summary/correction and voice actions. Check original text/notes on failure, quota errors and cancellation.
  2. Use a Coucou-managed project conversation for plan progress, changes, questions and approvals. Switch provider/account while work is pending; an old reply must not affect the next command or conversation.
  3. Export/install/enable the Desktop plugin with its hidden .agents files, invoke $coucou-question in the original local conversation, and answer once. Cancel, disconnect and try another conversation; stale answers must not cross requests. See the packaged plugin README.
  4. Disable Show Coucou in the Dock if it was explicitly saved on. Close the window, reopen from the menu bar and confirm hooks/questions continue; Quit must stop the app.

Manual cleanup: choose Disable background features in Chat after testing; revoke plugin trust/remove its installed files through Codex if no longer wanted. Existing notes and the native Codex login are retained. Roll back this increment by reverting its feature commits, or select the prior provider; do not erase native auth or note files.

No new CI wiring or Windows/Android runtime work, dependencies, version/bundle-ID changes, real credentials, private logs or binary artifacts are included.

Authorship and review

Core change: credential handling, subprocess isolation, async voice actions, event identity and Phone consumers require module-owner and second-reviewer inspection. In particular review dedicated Keychain scope, explicit authorization and cleanup, native credential compatibility, denied tools and stale request/utterance guards.

Reviewer guide: read CodexTextService/CodexAccess credential and process boundaries, CodexVoiceTurn/ClaudeVoiceBrain action validation and lifetime, then CodexTurnState/CodexEventAdapter identity and reconciliation. Spot-check the generated Xcode project and the additive English/Hant catalog entries.

AI disclosure: all task-authored changes in this PR were written with OpenAI Codex; project/Info output was generated by XcodeGen and the diagram was generated with Archify. The requester confirmed no other AI tools and no human line-by-line code review. Automated fixtures and compilation are not independent human review. Upstream changes retain their original authors.

Merge upstream c946650 while preserving terminal return, permission prompts, and Codex managed session identity. Use the upstream sessionBundleId field, retain Desktop originator precedence, and regenerate the Xcode project.
@shawn-t516
shawn-t516 marked this pull request as ready for review October 9, 2026 02:05
Merge upstream 0dffbaf, keep its plan layout, and share usage reads. Integrate native requests with the bounded approval queue and per-session registry, bind Mac and phone actions to captured request IDs, and remove old managed ownership on cancellation or replacement. Add regressions for mixed requests, stale actions, refresh throttling, expired quotas, and Demo routing.
Merge current upstream main while retaining Portfolio and the new chat providers. Verify external thread identity and project before reading tokens, keep custom quota durations and reset expiry accurate, report installed IDE extensions, and reject late previous-turn events.

Add bounded IDE and thread-usage fixture checks and preserve separate Traditional Chinese localization scope.
Reuse the existing local question card and return each answer to its original MCP call. Preserve session ownership during cancellation and cleanup, and export a complete local marketplace from GitHub Mac settings.

Show Codex source and return actions instead of the shared terminal label. Add bounded protocol, socket lifecycle, source routing and export fixtures.
@shawn-t516 shawn-t516 changed the title Codex: native ChatGPT chat and managed session controls Codex: native chat, session controls and Desktop questions Oct 15, 2026
Merge upstream Mac vitals and capture changes while preserving Codex session ownership and plugin behavior. Use Codex labels for capture actions and verify the existing image route.
Merge the latest Flow, Notetaker and error-card updates. Preserve exact native session identity, show the actual Codex failure, and return to its Desktop, IDE or managed chat without collapsing a continuation.
Preserve native approval denial and cleanup alongside upstream wait logging. Keep one provider-aware error helper with the upstream API-error fallback, and use accurate phone expiry copy for all agent transports.
Merge upstream WhatsApp and release changes while preserving the Codex setup documentation and runtime behavior. Regenerate the project for the new Mac sources and keep upstream versions, metadata and locale values.
Route Flow, notes and voice through opt-in isolated ChatGPT requests without changing native refresh credentials.

Show owned plan progress and reconcile cumulative file changes without stale or duplicate entries.

Add regression fixtures and English/Traditional Chinese setup guidance.
@shawn-t516 shawn-t516 changed the title Codex: native chat, session controls and Desktop questions Codex: complete Mac assistant and Desktop integration Oct 27, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant