Spike: run the plugin in Codex, Cursor and Kiro as well as Claude Code #61

Activity

1. sabind

   added

   triageNot yet classified by a maintainerNot yet classified by a maintainer

   on Oct 22, 2026
2. sabind

   added the

   Spike

   issue type on Oct 22, 2026
3. sabind commented on Oct 22, 2026

   

   sabind

   on Oct 22, 2026

   ContributorAuthor

   More actions

   Spike result: the freethinking plugin in Codex, Cursor and Kiro

   Short answer. The plugin goes ahead in every agent, with no tool ruled out.

   • Codex and Cursor already load the plugin as it is today: its .claude-plugin manifest, its skills/ folder and its hooks/hooks.json with CLAUDE_PLUGIN_ROOT. Kiro needs its own layout.
   • In all three, four Claude-only features fail: skill arguments, shell prefetch, ${CLAUDE_SKILL_DIR} and the blocking fixed-option question.
   • The decision: converge where a tool reads the plugin's own structure, and generate per-tool glue only for the gaps. Claude Code is the benchmark, and Cursor and Codex come first beside it, then Kiro.
   • The plan is a tech strategy, .freethinking/strategies/multi-tool-plugin.md in PR [#61] Add the multi-tool plugin strategy #62, with five streams of work filed as Compile every agent's plugin from one canonical source #63 to Deliver the plugin's skills through an MCP server any agent can connect #67.

   How this was tested. Each tool loaded a small test plugin built in the same shape as freethinking:

   • the same manifest, marketplace and hooks/hooks.json
   • our real load-rules.sh and load-rules-subagent.sh
   • a rule file with a codeword
   • an agent in our frontmatter format
   • a skill using $ARGUMENTS, named arguments, a ! prefetch line, ${CLAUDE_SKILL_DIR} and a file in its folder
   • a pair of skills where one hands off to the other

   A marker hook logged which events fired and which variables they saw.

   Each tool then ran three headless prompts in a throwaway workspace:

   1. What did the session load, and can it launch the agent by name?
   2. Call the argument skill with alpha beta.
   3. Run the hand-off, then start a sub-agent and ask whether it has the rules.

   The tools and versions:

   • Codex CLI 0.163.0, plugin installed from a local marketplace
   • Cursor CLI 2026.10.22, loaded with --plugin-dir
   • kiro-cli 2.33.1, V3 engine, with the parts placed in .kiro/ because Kiro has no package for them

   Claude Code ran the same three prompts as a control, and every part worked there.

   Each cell is tagged with its source: run (one of these runs), doc (the tool's documentation), source (the tool's source code) or inferred. Links are listed under the table.

   What the plugin uses, tool by tool

   The issue named eight parts; reading the plugin turned up five more (marked added).

   Part	Claude Code	Codex	Cursor	Kiro
   Skills	As is (doc, run)	As is: listed as probe:probe-args, called with $probe:probe-args (run)	As is, from our .claude-plugin layout (run). Cursor also picked up freethinking's skills and agents from the Claude Code install on the same machine (run)	As is from .kiro/skills/ (run). A custom agent sees skills only when its resources lists them (doc)
   Always-on rules in every session	Another way: SessionStart hook prints rules/ (doc, run)	As is: our SessionStart hook ran unchanged and the rules reached the session (run). The user trusts plugin hooks once in /hooks (doc)	Another way. Neither our hook's plain-text output nor plain .md plugin rules reached the session; a plugin rule with alwaysApply: true did not load in the CLI either (run). A sessionStart hook that prints {"additional_context": …} works (run)	Another way: inclusion: always steering in ~/.kiro/steering/ (run), or a SessionStart hook whose output is added (run on 2.33.1). A Kiro package cannot carry either (doc)
   Rules in every sub-agent	Another way: SubagentStart hook (doc, run)	As is: our SubagentStart hook fired and the sub-agent had the rules (run)	Another way: SubagentStart never fired, in Claude or Cursor format, but context from the JSON sessionStart hook reached the sub-agent (run)	Steering: as is (run). Hook output: does not reach sub-agents, and there is no sub-agent start event (run, doc)
   Hooks	As is (doc, run)	As is: reads hooks/hooks.json; sets CLAUDE_PLUGIN_ROOT, PLUGIN_ROOT and CLAUDE_PLUGIN_DATA (run)	As is for SessionStart: our Claude-format hooks.json ran with CLAUDE_PLUGIN_ROOT and CURSOR_PLUGIN_ROOT set, from the plugin's folder (run). Plain stdout is not added; JSON is (run)	Another way: .kiro/hooks/*.json; SessionStart only; no plugin-root variable (doc, run)
   Named sub-agents with their own tools, model and effort	As is (doc, run)	Not from a plugin: the agent was not offered (run). Another way: TOML files in ~/.codex/agents/, with no per-agent tool list (doc)	Loaded from our agents/ and launched by name, but tools: Read, Grep was ignored: the agent had Shell and Write (run). Effort goes inside the model ID (doc)	As is except effort: a Markdown agent in .kiro/agents/ was launched by name with only read tools (run). No per-agent effort (doc)
   Fixed-option question to the user	As is: AskUserQuestion (doc)	Plan mode only (source). Elsewhere, the prose fallback the collaboration rule already has	Another way: a question tool that does not block; the agent "continues reading files, making edits, or running commands" while waiting (doc)	None in the main agent (doc, run). Prose fallback
   Skill-to-skill hand-off	As is (run)	Works: the model read the other skill's SKILL.md by path (run)	Works: the model read the other skill's SKILL.md (run)	As is: Load skill: probe-target (run)
   Shell prefetch at load	As is (run)	No: the line arrives as literal text (run)	No: literal text (run)	No: literal text (run)
   added Skill arguments	As is (run)	No: placeholders stay literal; the model still sees the typed text (run)	No: same (run)	No in V3: same (run on 2.33.1). The V2 engine substitutes $ARGUMENTS and ${N} (run on 2.25)
   added Skill's own files (${CLAUDE_SKILL_DIR})	As is (run)	No: the variable stayed literal and the model never read the file, so it quoted the wrong codeword (run). Relative paths are the documented way (source)	The variable stayed literal, but the model found the file in the skill's folder (run)	The variable stayed literal, but the model read the file (run)
   added allowed-tools pre-approval	As is: grants permission for the turn (doc)	No: the user's own .rules files, or a hook that approves (doc)	No: the user's or project's permissions.json (doc)	No: "A cloned repo cannot inject permission rules" (doc)
   added Packaging and install	As is (doc)	As is: codex plugin marketplace add and codex plugin add installed from our .claude-plugin marketplace (run). The IDE extension loads no plugins (doc)	Loads our .claude-plugin layout with --plugin-dir (run). Marketplace install not tested; .cursor-plugin/plugin.json is the documented manifest (doc)	No package holds steering, agents and hooks; a "power" carries skills and MCP only (doc)
   added Headless runs for evals	claude plugin eval, claude -p (doc, run)	codex exec --json (run). Standard input must be closed, or it waits forever	agent -p --output-format stream-json (run)	kiro-cli chat --agent-engine v3 --no-interactive (run)

   Prior art: AWS's agent-toolkit-for-aws

   AWS's toolkit ships five plugins to Claude Code, Codex, Cursor, Devin and Kiro from one repository (read at commit ece684d, 2026-10-22).

   • Manifests: each plugin has one folder with a manifest per tool, plus a root plugin.json in the open Agent Plugins format. A CI script checks the manifests agree.
   • Skills: shared skills are copied from one canonical skills/ folder by a sync script, with a CI check.
   • Skill text: it still uses AskUserQuestion, $ARGUMENTS and ${CLAUDE_PLUGIN_ROOT}, worded so it still reads sensibly when a tool leaves them literal, as in "the developer's prompt and $ARGUMENTS".
   • Hooks and agents: both are Claude-only. Claude's hooks sit outside the default hooks/ folder so Codex never runs them, and Cursor gets its own hooks.json.
   • Rules: shipped as files users copy into their own project, not by any plugin.
   • Kiro: gets a skills-only install through npx skills add.
   • Testing: no tests run in any tool other than Claude Code.

   We follow its layout and its sync-and-check pattern. We go further on the parts it leaves Claude-only, because this plugin depends on its rules reaching every session and on its agents. The runs above show Codex runs our rule hooks unchanged, Cursor works with a JSON hook, and Kiro carries rules in steering.

   Recommendation

   Every agent is a target. None is ruled out for what it cannot do today, because their support changes month to month. Each agent's gaps are recorded and checked every week.

   Claude Code is the benchmark. One set of eval cases is shared by every tool, and each tool gets its own runner, scored against Claude Code's results. Each tool can add cases of its own for behaviour only it has. Evals stay a manual, per-target token cost, never run in CI.

   Converge first, compile at the edges. Where a tool reads the plugin's own structure, it reads the source as it is. Codex and Cursor already read our .claude-plugin tree. Where a tool cannot, glue generated at build time closes the gap. That covers the four skill-text features above, Cursor's rules hook output and Kiro's layout. Generated output is checked in, and a free check in CI fails when it is out of date.

   Order: Cursor and Codex first, beside Claude Code, then Kiro, then other agents as they gain a plugin or skills format, one destination at a time. Codex lacks plugin agents and its IDE extension loads no plugins. Cursor ignores agents' tool lists and its question tool does not block, which weakens the plugin's approval gates. These are recorded as gaps in each tool's evals, not hidden by glue.

   A weekly format watch. The test plugin from this spike is rebuilt in the repository from the description under "How this was tested". A weekly automation re-runs its three prompts in every installed agent and reports what changed. That way a gap that closes retires its glue, and a break is seen before an operator meets it.

   An MCP server as a shared delivery channel. MCP, the Model Context Protocol, is a standard way for an agent to connect to tools served by another program, and every tool here accepts MCP servers. A server could deliver skill text, templates and possibly rules to any agent, whether run locally or hosted on AWS. Four questions come first:

   • Do other agents add MCP server instructions to the session, as Claude Code does?
   • Do sub-agents see those instructions?
   • Which command-line tools show elicitation, MCP's way for a server to ask the user a question?
   • Does the model pick skills as reliably through a fetch tool as through descriptions?

   Cost per destination

   Destination	To support	To test
   Shared	Glue for the four Claude-only skill features across the eight skills; the build step and its free check	The Claude evals before and after, as the benchmark
   Cursor	A JSON branch for the rules hook, or a generated Cursor hook; a .cursor-plugin manifest	A runner for agent -p --plugin-dir, and a Cursor account
   Codex	An install step for the three agents, as TOML files or as skills that start a general sub-agent; a note on trusting hooks once	A runner for codex exec --json with its own CODEX_HOME, and an OpenAI account
   Kiro	Generated .kiro/ output (steering, agents, skills) and an installer into ~/.kiro/	A runner for kiro-cli chat --no-interactive against the generated output

   Each probe run in this spike took under a minute. claude plugin eval does not carry over, so each tool needs its own runner that grades its JSON output.

   Follow-up issues

   One issue per stream of work in the strategy, in order:

   1. Compile every agent's plugin from one canonical source #63 Core and glue. Separate the tool-neutral text of skills, rules and agents from per-tool glue, and add the build step and its free check.
      • Owner: one source with no hand-kept copies.
      • Operator: in any tool, pr 61 finds ticket 61 and each skill finds its templates.
   2. Add a target per agent: Cursor and Codex, then Kiro and Devin #64 Tool packaging and install, Cursor and Codex first.
      • Operator: in that tool, one command installs the plugin, and its rules reach every session and sub-agent.
   3. Run shared eval scenarios against every agent, checking each step along the way #65 Cross-tool evals.
      • Owner: the shared cases run in each tool and are scored against Claude Code, so a change is checked everywhere it ships.
   4. A weekly report of what each agent supports #66 Format watch.
      • Owner: a weekly report says what each installed agent supports and what changed since the week before.
   5. Deliver the plugin's skills through an MCP server any agent can connect #67 MCP delivery channel.
      • Operator: one install line, in any agent, delivers the parts MCP can carry.
      • It first answers the four questions above, and weighs a local server against an AWS-hosted one.

   Still open

   • Kiro: whether an interactive V3 session substitutes $ARGUMENTS. Moot if the text goes portable.
   • Cursor: marketplace install, and rules in IDE sub-agents. The IDE was not installed.
   • Every tool's IDE: none was run.

   Sources

   • Claude Code: skills, hooks, sub-agents, tools, plugin manifest, plugin evals, headless
   • Codex:
     • Docs: skills, plugins, building plugins, hooks, sub-agents, rules, non-interactive mode
     • Source: openai/codex at 7ace9c6. codex-rs/ext/skills/src/catalog_prompt.rs, codex-rs/protocol/src/config_types.rs
   • Cursor: skills, plugin reference, rules, sub-agents, hooks, agent overview, CLI permissions
   • Kiro: skills, steering, sub-agents, hook types, permissions, tools, powers, headless CLI, effort
   • Docs fetched and runs made on 2026-10-22.
4. sabind

   added a commit that references this issue on Oct 22, 2026

   [#61] Add the multi-tool plugin strategy (#62)

   ...

   197f737