feat(gui): native GUI tools with builtin tool call observability - #3

Open

s-JoL wants to merge 17 commits into

mainfrom

codex/gui-native-tools-v2

Commits on Apr 2, 2026

feat(protocol): add BuiltinToolCallItem and BuiltinToolCallStatus types

s-JoL
and
claude
committed
3460ac9
feat(gui): add native GUI tool schema definitions

s-JoL
and
claude
committed
86e3e61
feat(gui): add native platform runtime and Swift helpers

s-JoL
and
claude
committed
79fbb6c
feat(gui): add semantic grounding engine

s-JoL
and
claude
committed
334584c
feat(gui): add GUI tool handler with observe-act-verify loop

s-JoL
and
claude
committed
0b8b7ab
feat(gui): add system prompt instructions and config options

s-JoL
and
claude
committed
3b6478f
feat(gui): promote GUI tool calls to first-class BuiltinToolCall turn items

s-JoL
and
claude
committed
55346d2
feat(gui): add TUI and CLI rendering for builtin tool calls

s-JoL
and
claude
committed
6f87ce5
test(gui): add GUI tool tests, grounding benchmark, and config tests

s-JoL
and
claude
committed
c9287d0
fix(gui): do not re-activate app during post-action evidence and wait probes

s-JoL
and
claude
committed
5137794
fix(gui): address code review findings for async safety, caching, and cleanup

s-JoL
and
claude
committed
088329d
fix(gui): fix screenshot redaction crash, TOCTOU race, and edge grounding

s-JoL
and
claude
committed
4c63914
fix(gui): improve grounding coordinate accuracy and async robustness

s-JoL
and
claude
committed
d8157b5

Commits on Apr 3, 2026

fix(gui): use modern activate API on macOS 14+ and guard mouse release

s-JoL
and
claude
committed
5b5a8a5
feat(gui): hide other apps during actions, fix mouse cleanup, and add demo

s-JoL
and
claude
committed
acd1895
feat(gui): add gui_batch tool with parallel/unified grounding and drag support

s-JoL
and
claude
committed
619a0e1

Commits on Apr 4, 2026

feat(gui): persistent Swift helper, JPEG screenshots, Escape safety, and batch config

s-JoL
and
claude
committed
82f5a63