Built with the arc-cua driver, which reads and controls apps in the background for Third Hand.
Third Hand was a proof of concept to show that we can use Jev for computer use, but now we are building a full-fledged assistant.
A macOS assistant you chat with. Tag an app with @ in a thread, like @Spotify play something chill, and Third Hand does it: it reads the app's accessible controls, clicks and types, and checks the result.
- Threads work like channels: each keeps its own history, so follow-ups ("now play the second one") work. A message without an @ uses the thread's last app.
- @mentions list running and installed apps; tagging one that isn't open launches it.
- Control–Space in any app opens a new thread with that app already tagged.
- Press ⇧↩ to run a task in the background: the app stays where it is, even hidden or minimized, and your pointer and front app are left alone. ↩ runs it on screen. Click Stop on a running task to cancel it. Tasks run one at a time.
Download the latest release for Apple Silicon Macs running macOS 14 or newer. Unzip the archive, move Third Hand.app to Applications, and open it. Release builds are Developer ID-signed and notarized by Apple. A ChatGPT plan that allows third-party apps and a TypeSafe API key are required.
You’ll need Xcode 15 or newer, an Apple Development or Developer ID signing certificate, uv (the build uses it to bundle arc-cua's Python runtime into the app), and a TypeSafe API key.
git clone git@github.com:shhivv/third-hand.git
cd third-hand
./rebuild.sh
open "Third Hand.app"In Settings (the gear at the bottom of the sidebar, or ⌘,):
- Enable Accessibility so Third Hand can read and control apps.
- Enable Screen Recording for local text recognition when an app’s controls aren’t accessible.
- Add your TypeSafe API key. It’s saved in macOS Keychain.
- Click Sign in with ChatGPT and finish signing in in your browser. Tokens are saved in macOS Keychain.
Open Third Hand from the Dock, start a thread, and try a specific task, such as “@Spotify search for Adele.”
The app runs on macOS 14+. Apple Intelligence is not required.
- arc-cua's driver reads the app's window through Accessibility and acts on it in the background, bringing minimized or hidden windows up out of sight and putting them back afterwards. It checks each action against the app as it is when the action runs, and reports when the app has finished reacting. It ships inside the app and runs locally.
- Apple Vision reads screen text locally when needed. Screenshots aren’t uploaded.
- ChatGPT plans the whole task in one go as a list of simple steps (click, type, press, scroll, wait), naming each control by its on-screen label, and writes any text to enter: search terms, messages, commands. It's asked again only if a step fails. Your request, app name, screen labels and values, and step results are sent to OpenAI through your ChatGPT account. Planning uses
gpt-6-solwith no reasoning effort, and retries with low effort after a failed step; pin either withdefaults write com.thirdhand.app CodexModel <model>orCodexEffort <effort>. - Jev picks the control when a step's label is ambiguous or doesn't match exactly, choosing only among controls that fit the step. Third Hand then executes and verifies each step. The step, app name, and candidate labels and values are sent to TypeSafe.
- Third Hand is not offline.
No bundled model weights, and nothing extra to install: the app carries arc-cua and its Python runtime. Third Hand never restarts the apps it controls without asking.
./rebuild.sh # Build, sign, and update Third Hand.app
swift test # Run tests without calling the live APIAlways run the repository-root Third Hand.app. The build script keeps the same signing identity to preserve macOS permissions and retains the previous app in .build/install.*. Keep .thirdhand-signing-identity on your machine; it is excluded from Git. If no certificate is available, create an Apple Development certificate in Xcode before building.
Settings shows current permission status. If macOS asks you to quit and reopen after granting access, reopen this same copy.
To test a local arc-cua checkout instead of the bundled one, run defaults write com.thirdhand.app ArcPackage -string 'arc-cua[macos] @ file:///path/to/arc-cua' (uses uv); defaults delete com.thirdhand.app ArcPackage goes back to the bundled runtime.
Diagnostic logs are written to ~/Desktop/thirdhand.log, and arc-cua's to ~/Library/Logs/Third Hand arc-cua.log. They include action status, timing, and bounded API rejection messages. Review logs before sharing: service error messages can contain request details. API keys are redacted from those messages.
In terminals, the planner can write and run shell commands. Third Hand types each command once and won't enter another until the pending one is submitted. Stay nearby: commands run with your user's permissions.
An early, experimental project. Some apps expose incomplete controls; icon-only interfaces, custom editors, and complex gestures may not work. A task can stop without completing, and reported completion still needs your judgment. Stay nearby while it works.
Issues and pull requests are welcome. Please include your macOS version, the app involved, and the steps to reproduce. Don’t include API keys or private screen content.
MIT — Shiv Shanmugam · shiv@tryisle.com