Skip to content

Probe skill lifecycle, resume and interactive mode across harnesses #991

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
alexeyzimarev merged 21 commits into from
Sep 17, 2026
+9,814 −2,084
Merged

Probe skill lifecycle, resume and interactive mode across harnesses #991

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
e386d74
Design and plan pass 2 of the skills discovery probes (#961)
alexeyzimarev Sep 17, 2026
cd69b7e
Correct the pass 2 plan's app-server session test launch (#961)
alexeyzimarev Sep 17, 2026
e051bc6
Launch vendors from the sandbox's cwd and add linked worktrees (#961)
alexeyzimarev Sep 17, 2026
3d46705
Add lifecycle verdicts, the interactive prompt and their report colum…
alexeyzimarev Sep 17, 2026
ab20e97
Keep the live catalogue column mode-aware when only the tui ran (#961)
alexeyzimarev Sep 17, 2026
448cc04
Hold a vendor session across prompts in the daemon drivers (#961)
alexeyzimarev Sep 17, 2026
7a22df8
Drive a vendor's interactive UI through a pseudo-terminal (#961)
alexeyzimarev Sep 17, 2026
d63480a
Release the terminal on a failed spawn and window each turn's raw str…
alexeyzimarev Sep 17, 2026
ab72a01
Give every adapter a resume launch, a live session and a TUI launch (…
alexeyzimarev Sep 17, 2026
677256e
Add the lifecycle scenarios and the interactive mode to the orchestra…
alexeyzimarev Sep 17, 2026
80366e7
Pin each adapter's resume flags, session id reader and TUI launch (#961)
alexeyzimarev Sep 17, 2026
9a32eae
Read the interactive reply off an emulated screen, not the byte strea…
alexeyzimarev Sep 17, 2026
cc32a29
Answer a vendor's dialog only once its screen has settled (#961)
alexeyzimarev Sep 17, 2026
979c3e7
Record a run that produced no answer as untested, not as a negative (…
alexeyzimarev Sep 17, 2026
f5bb6e1
Give an interactive turn its own budget and skip a reload nobody has …
alexeyzimarev Sep 17, 2026
fb2ebde
Mark a later interactive turn so a redrawn transcript cannot answer i…
alexeyzimarev Sep 17, 2026
b4ac91d
Keep the startup hook off an interactive vendor's terminal (#961)
alexeyzimarev Sep 17, 2026
ca1a320
Stop OpenCode's background service before opening a session (#961)
alexeyzimarev Sep 17, 2026
4e80f16
Record the pass 2 lifecycle and interactive matrix (#961)
alexeyzimarev Sep 17, 2026
1ee3bc5
Hold back a split control sequence and stop a failed start leaking (#…
alexeyzimarev Sep 17, 2026
72cfd08
Measure Cursor's registration route, which named the wrong target (#961)
alexeyzimarev Sep 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/probes/2026-09-16-skills-discovery/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
out/
*.stderr.log
*-tui.log
36 changes: 18 additions & 18 deletions docs/probes/2026-09-16-skills-discovery/capability-matrix.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,23 +3,23 @@
Generated by `report.py` from `matrix.json`; do not edit by hand. A `—` cell means the entry
could not run (see its status line below and `findings.md`).

| Entry | Version tested | Modes | Native root | Roots consumed | Startup mechanism proven | Exclusion preserving load | Vendor-isolated destination | Reload path | GUI status | Minimum version |
| -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- |
| agy | 1.2.4, not-installed | print | .agents/skills | — | — | — | — | — | Antigravity IDE: untested (manual procedure in findings.md) | — |
| agy-clidir | 1.2.4 | print | .agents/skills | .agent/skills, .agents/skills | n/a (not run) | gitignore, info-exclude | none | n/a (not run) | Antigravity IDE: untested (manual procedure in findings.md) | 1.2.4 |
| agy-dirlayout | 1.2.4 | print | .agents/skills | .agent/skills, .agents/skills | none | gitignore, info-exclude | none | none | Antigravity IDE: untested (manual procedure in findings.md) | 1.2.4 |
| claude | 2.1.273 (Claude Code) | print | .claude/skills | .claude/skills | none | gitignore, info-exclude | none | none | n/a | 2.1.273 (Claude Code) |
| codex | codex-cli 0.154.0 | daemon, print | .agents/skills | .agents/skills, .codex/skills | none | gitignore, info-exclude | none | none | n/a | codex-cli 0.154.0 |
| copilot | GitHub Copilot CLI 1.0.85. | daemon, print | .github/skills | .agents/skills, .claude/skills, .github/skills | none | gitignore, info-exclude | .github/skills | none | n/a | GitHub Copilot CLI 1.0.85. |
| cursor | 2026.09.15-d2fe57e | daemon, print | .cursor/skills | .agents/skills, .claude/skills, .codex/skills, .cursor/skills | none | gitignore, info-exclude | .cursor/skills | none | Cursor desktop: untested (manual procedure in findings.md) | 2026.09.15-d2fe57e |
| cursor-userhooks | 2026.09.15-d2fe57e | daemon, print | .cursor/skills | n/a (not run) | none | n/a (not run) | n/a (not run) | none | n/a | 2026.09.15-d2fe57e |
| gemini | 0.60.0 | daemon, print | .gemini/skills | — | — | — | — | — | n/a | — |
| kiro | kiro-cli 2.21.4 | daemon, print | .kiro/skills | .kiro/skills | agent hooks.agentSpawn (cli 2.x) (daemon); agent hooks.agentSpawn (cli 2.x) (print) | gitignore, info-exclude | .kiro/skills | none | Kiro IDE: untested (manual procedure in findings.md) | kiro-cli 2.21.4 |
| kiro-agent-bare | kiro-cli 2.21.4 | daemon, print | .kiro/skills | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a | kiro-cli 2.21.4 |
| kiro-agent-skills | kiro-cli 2.21.4 | daemon, print | .kiro/skills | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a | kiro-cli 2.21.4 |
| opencode-v1 | 1.18.31 | daemon, print | .opencode/skills | .agents/skills, .claude/skills, .opencode/skills | plugin event session.created (print) | gitignore, info-exclude | none | none | n/a | 1.18.31 |
| opencode-v2 | opencode v2.0.4 | daemon, print | .opencode/skills | .agents/skills, .claude/skills, .opencode/skills | n/a (not run) | gitignore, info-exclude | none | n/a (not run) | n/a | opencode v2.0.4 |
| pi | 0.85.1 | daemon, print | .pi/skills | .agents/skills, .pi/skills | extension session_start + resources_discover skillPaths (daemon); extension session_start + resources_discover skillPaths (print) | gitignore, info-exclude | .pi/skills | extension session_start + resources_discover skillPaths | n/a | 0.85.1 |
| Entry | Version tested | Modes | Native root | Roots consumed | Startup mechanism proven | Exclusion preserving load | Vendor-isolated destination | Reload path | Live catalogue | Startup rewrite | Resume | Nested cwd | Worktree | Peer hook | Interactive | GUI status | Minimum version |
| -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- |
| agy | 1.2.4, 1.2.5 | print | .agents/skills | — | — | — | — | — | — | — | — | — | — | — | — | Antigravity IDE: untested (manual procedure in findings.md) | — |
| agy-clidir | 1.2.4 | print | .agents/skills | .agent/skills, .agents/skills | n/a (not run) | gitignore, info-exclude | .agent/skills | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | Antigravity IDE: untested (manual procedure in findings.md) | 1.2.4 |
| agy-dirlayout | 1.2.4, 1.2.5 | print, tui | .agents/skills | .agent/skills, .agents/skills | none | gitignore, info-exclude | .agent/skills | none | n/a (not run) | delete=stale; update=visible_first_turn | add=visible_first_turn; update=visible_first_turn | ancestor=visible_first_turn; local=visible_first_turn | linked-other=not_visible; linked-own=visible_first_turn | n/a (not run) | S1=visible_first_turn; hook-adds-skill=not_visible; add=visible_live | Antigravity IDE: untested (manual procedure in findings.md) | 1.2.4 |
| claude | 2.1.273 (Claude Code), 2.1.274 (Claude Code) | print, tui | .claude/skills | .claude/skills | --settings file over the real config root hooks.SessionStart (tui) | gitignore, info-exclude | none | none | n/a (not run) | delete=revoked; update=stale | add=visible_first_turn | ancestor=visible_first_turn; local=visible_first_turn | linked-other=not_visible; linked-own=visible_first_turn | n/a (not run) | S1=visible_first_turn; hook-adds-skill=visible_first_turn; add=not_visible | n/a | 2.1.273 (Claude Code) |
| codex | codex-cli 0.154.0 | daemon, print, tui | .agents/skills | .agents/skills, .codex/skills | none | gitignore, info-exclude | none | none | add=visible_live; delete=revoked; update=visible_live | delete=not_visible/stale; update=visible_first_turn | add=visible_first_turn; update=visible_first_turn | ancestor=visible_first_turn; local=visible_first_turn | linked-other=not_visible; linked-own=visible_first_turn | hook-from-peer=visible_live | S1=visible_first_turn; hook-adds-skill=not_visible; add=visible_live | n/a | codex-cli 0.154.0 |
| copilot | GitHub Copilot CLI 1.0.85. | daemon, print, tui | .github/skills | .agents/skills, .claude/skills, .github/skills | none | gitignore, info-exclude | .github/skills | /skills reload | add=not_visible; delete=stale; update=stale | delete=revoked; update=visible_first_turn | add=visible_first_turn; update=stale | ancestor=visible_first_turn; local=visible_first_turn | linked-other=not_visible; linked-own=visible_first_turn | hook-from-peer=not_visible | S1=visible_first_turn; hook-adds-skill=not_visible; add=not_visible; reload=visible_after_reload | n/a | GitHub Copilot CLI 1.0.85. |
| cursor | 2026.09.15-d2fe57e | daemon, print, tui | .cursor/skills | .agents/skills, .claude/skills, .codex/skills, .cursor/skills | project .cursor/hooks.json workspaceOpen pluginPaths (print) | gitignore, info-exclude | .cursor/skills | project .cursor/hooks.json workspaceOpen pluginPaths | add=not_visible; delete=revoked; update=visible_live | delete=revoked; update=visible_first_turn | add=not_visible; update=visible_first_turn | ancestor=not_visible; local=visible_first_turn | linked-other=not_visible; linked-own=visible_first_turn | hook-from-peer=not_visible | S1=visible_first_turn; hook-adds-skill=not_visible | Cursor desktop: untested (manual procedure in findings.md) | 2026.09.15-d2fe57e |
| cursor-userhooks | 2026.09.15-d2fe57e | daemon, print | .cursor/skills | n/a (not run) | user ~/.cursor/hooks.json workspaceOpen (print) | n/a (not run) | n/a (not run) | user ~/.cursor/hooks.json workspaceOpen | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a | 2026.09.15-d2fe57e |
| gemini | 0.60.0 | daemon, print, tui | .gemini/skills | — | — | — | — | — | — | — | — | — | — | — | — | n/a | — |
| kiro | kiro-cli 2.21.4 | daemon, print, tui | .kiro/skills | .kiro/skills | agent hooks.agentSpawn (cli 2.x) (daemon); agent hooks.agentSpawn (cli 2.x) (print); agent hooks.agentSpawn (cli 2.x) (tui) | gitignore, info-exclude | .kiro/skills | none | add=visible_live; delete=revoked; update=visible_live | delete=revoked; update=visible_first_turn | add=visible_first_turn; update=visible_first_turn | ancestor=not_visible; local=visible_first_turn | linked-other=not_visible; linked-own=visible_first_turn | hook-from-peer=visible_live | S1=visible_first_turn; hook-adds-skill=visible_first_turn; add=visible_live | Kiro IDE: untested (manual procedure in findings.md) | kiro-cli 2.21.4 |
| kiro-agent-bare | kiro-cli 2.21.4 | daemon, print | .kiro/skills | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a | kiro-cli 2.21.4 |
| kiro-agent-skills | kiro-cli 2.21.4 | daemon, print | .kiro/skills | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a (not run) | n/a | kiro-cli 2.21.4 |
| opencode-v1 | 1.18.31 | daemon, print, tui | .opencode/skills | .agents/skills, .claude/skills, .opencode/skills | plugin event session.created (print) | gitignore, info-exclude | .opencode/skills | none | add=not_visible; delete=stale; update=stale | delete=revoked; update=stale/visible_first_turn | add=visible_first_turn; update=visible_first_turn | ancestor=visible_first_turn; local=visible_first_turn | linked-other=not_visible; linked-own=visible_first_turn | hook-from-peer=not_visible | S1=visible_first_turn; add=not_visible | n/a | 1.18.31 |
| opencode-v2 | opencode v2.0.4 | daemon, print, tui | .opencode/skills | .agents/skills, .claude/skills, .opencode/skills | n/a (not run) | gitignore, info-exclude | .opencode/skills | n/a (not run) | add=not_visible; delete=revoked; update=stale | n/a (not run) | add=visible_first_turn; update=visible_first_turn | ancestor=visible_first_turn; local=visible_first_turn | linked-other=not_visible; linked-own=visible_first_turn | n/a (not run) | S1=visible_first_turn; add=visible_live | n/a | opencode v2.0.4 |
| pi | 0.85.1 | daemon, print, tui | .pi/skills | .agents/skills, .pi/skills | extension session_start + resources_discover skillPaths (daemon); extension session_start + resources_discover skillPaths (print) | gitignore, info-exclude | .pi/skills | extension session_start + resources_discover skillPaths | add=not_visible; delete=not_visible; update=visible_live | delete=revoked; update=visible_first_turn | add=visible_first_turn; update=visible_first_turn | ancestor=not_visible; local=visible_first_turn | linked-other=not_visible; linked-own=visible_first_turn | hook-from-peer=not_visible | S1=visible_first_turn; hook-adds-skill=not_visible; add=not_visible | n/a | 0.85.1 |

## Status per entry

Expand All @@ -31,7 +31,7 @@ could not run (see its status line below and `findings.md`).
- `copilot`: measured
- `cursor`: measured
- `cursor-userhooks`: measured
- `gemini`: untested: exception=RuntimeError("vendor exit 1 with no reply; extract failed: JSONDecodeE; exception=RuntimeError('vendor exit -9 with no reply; session/new failed: {"json
- `gemini`: untested: exception=RuntimeError("vendor exit 1 with no reply; extract failed: JSONDecodeE; exception=RuntimeError('no reply read from the screen; exit None; timeout no PRO; exception=RuntimeError('vendor exit -9 with no reply; session/new failed: {"json
- `kiro`: measured
- `kiro-agent-bare`: measured
- `kiro-agent-skills`: measured
Expand Down
Loading