Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "freethinking",
"version": "0.10.0",
"version": "0.11.0",
"description": "Shared engineering skills: standard workflows and prompts for day-to-day development.",
"author": {
"name": "Developer Handbook contributors"
Expand Down
5 changes: 2 additions & 3 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,9 +131,8 @@ far:
Claude-only tool only where no command does the job.
3. The Claude-only pieces are listed so a `compatibility` issue can start from
the list: `$ARGUMENTS`, `${CLAUDE_PLUGIN_ROOT}`, `${CLAUDE_SKILL_DIR}` and
the `arguments:` frontmatter list in skills, the `subagent_type: fork` that
`implement-spec` relies on, the hooks that load the rules, and the agents'
frontmatter.
the `arguments:` frontmatter list in skills, the hooks that load the rules,
and the agents' frontmatter.

The list grows from issues labelled `compatibility`.

Expand Down
10 changes: 4 additions & 6 deletions evals/skills/implement-spec/works-all-waves/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,10 @@ xcrun cache the sandbox forbids; the Makefile's implement-spec target puts
`bin` on the PATH and grants Bash, Write and Edit.

The prompt gives the start answers up front (spec cadence, no optional tests,
already on a branch), since nothing can answer mid-run. It also says the Agent
tool has no fork in this run, for the reason works-one-wave gives: in the
headless eval sandbox the Agent tool answers "Agent type 'fork' not found", so
implement-spec takes its inline fallback and invokes the Skill tool with
`freethinking:report` and `handback` itself for each report. That fallback is
the path this case grades; the fork path is not exercised.
already on a branch), since nothing can answer mid-run. implement-spec writes
each wave report and the final report by invoking the Skill tool with
`freethinking:report` and `handback` in the session itself, which is what the
`invokes-report` and `problems-first` graders read.

## Graders

Expand Down
4 changes: 2 additions & 2 deletions evals/skills/implement-spec/works-all-waves/prompt.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
description: On a scaffolded project with a committed two-wave spec, implement-spec works the whole spec at spec cadence. It works wave 0 and wave 1 without waiting between them, each with its executor agent, its own test run, its wave-reviewer agent and its full suite, writes a report after each wave, has the reviewer agent read the whole run once more in a final review, and hands back once at the end. The Agent tool has no fork in a headless run, so each wave report and the final report are written inline by the report skill, the fallback of implement-spec's step 6, and that is the path graded. Needs the Makefile's implement-spec target, which grants Bash, Write and Edit, runs the scaffold, and puts the wrappers in bin on the PATH.
description: On a scaffolded project with a committed two-wave spec, implement-spec works the whole spec at spec cadence. It works wave 0 and wave 1 without waiting between them, each with its executor agent, its own test run, its wave-reviewer agent and its full suite, writes a report after each wave, has the reviewer agent read the whole run once more in a final review, and hands back once at the end. Each wave report and the final report are written by the report skill, invoked inline by implement-spec's step 6. Needs the Makefile's implement-spec target, which grants Bash, Write and Edit, runs the scaffold, and puts the wrappers in bin on the PATH.
tags: [skill, implement-spec]
max_turns: 70
allowed_tools: [Read, Glob, Grep, Agent, Skill]
expected_outcome: Claude validates the graph, launches the task-executor agent for task 1, runs the tests itself, launches the wave-reviewer agent for wave 0, writes the wave's report and starts wave 1 without waiting, does the same for task 2, then launches the wave-reviewer agent once more over the whole run and hands back with a final report carrying a Wave review line for each wave and the `Final review of the run:` line. It writes each wave report and the final report inline with the report skill, each opening with `## Problems` or `No problems.` before `## Done`. Its attempts to tick the tasks are refused, because the sandbox denies edits under .claude/, and it says so rather than working around it.
---
/freethinking:implement-spec calc-subtract spec

I will not be able to answer questions during this run. Cadence: spec, run every wave and hand back only at the end. There are no optional tests. The branch is already calc-subtract. This sandbox refuses edits under .claude/, so if ticking a task in tasks.md is refused, say so in the report and carry on to the next wave as if it were ticked. The Agent tool has no fork in this run; write every hand-back inline.
I will not be able to answer questions during this run. Cadence: spec, run every wave and hand back only at the end. There are no optional tests. The branch is already calc-subtract. This sandbox refuses edits under .claude/, so if ticking a task in tasks.md is refused, say so in the report and carry on to the next wave as if it were ticked.
10 changes: 4 additions & 6 deletions evals/skills/implement-spec/works-one-wave/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,10 @@ need an xcrun cache the sandbox forbids; the Makefile's implement-spec target
puts `bin` on the PATH and grants Bash, Write and Edit.

The prompt gives the start answers up front (wave cadence, no optional tests,
already on a branch), since nothing can answer mid-run. It also says the Agent
tool has no fork in this run: in the headless eval sandbox the Agent tool
answers "Agent type 'fork' not found", so implement-spec's step 6 takes its
inline fallback and invokes the Skill tool with `freethinking:report` and
`handback` itself. That fallback is the path this case grades; the fork path
is not exercised.
already on a branch), since nothing can answer mid-run. implement-spec's step 6
writes the hand-back by invoking the Skill tool with `freethinking:report` and
`handback` in the session itself, which is what the `invokes-report` and
`problems-first` graders read.

## Graders

Expand Down
4 changes: 2 additions & 2 deletions evals/skills/implement-spec/works-one-wave/prompt.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
description: On a scaffolded project with a committed two-wave spec, implement-spec works the first wave at wave cadence. It launches the executor agent for the task, runs the tests itself, has the reviewer agent read the wave, ticks the task, runs the full suite, and hands back without starting wave 1. The Agent tool has no fork in a headless run, so the hand-back is written inline by the report skill, the fallback of implement-spec's step 6, and that is the path graded. Needs the Makefile's implement-spec target, which grants Bash, Write and Edit, runs the scaffold, and puts the wrappers in bin on the PATH.
description: On a scaffolded project with a committed two-wave spec, implement-spec works the first wave at wave cadence. It launches the executor agent for the task, runs the tests itself, has the reviewer agent read the wave, ticks the task, runs the full suite, and hands back without starting wave 1. The hand-back is written by the report skill, invoked inline by implement-spec's step 6. Needs the Makefile's implement-spec target, which grants Bash, Write and Edit, runs the scaffold, and puts the wrappers in bin on the PATH.
tags: [skill, implement-spec]
max_turns: 40
allowed_tools: [Read, Glob, Grep, Agent, Skill]
expected_outcome: Claude validates the graph, launches the task-executor agent for task 1, runs the tests itself, launches the wave-reviewer agent once for the wave and reports its verdict, and leaves wave 1 alone (no test_subtract written). It writes the hand-back inline with the report skill, and the Message opens with `## Problems` or `No problems.` before `## Done`, and keeps the `Wave review of wave n:` line under Checks. It also tries to tick task 1 and is refused, because the sandbox denies edits under .claude/, and says so rather than working around it.
---
/freethinking:implement-spec calc-subtract wave

I will not be able to answer questions during this run. Cadence: one wave. There are no optional tests. The branch is already calc-subtract. Work the first wave and hand back. The Agent tool has no fork in this run; write the hand-back inline.
I will not be able to answer questions during this run. Cadence: one wave. There are no optional tests. The branch is already calc-subtract. Work the first wave and hand back.
Loading
Loading