diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json index b697ef9..af8f18d 100644 --- a/.claude-plugin/plugin.json +++ b/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "freethinking", - "version": "0.10.0", + "version": "0.11.0", "description": "Shared engineering skills: standard workflows and prompts for day-to-day development.", "author": { "name": "Developer Handbook contributors" diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index 6f6ecc6..768399f 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -131,9 +131,8 @@ far: Claude-only tool only where no command does the job. 3. The Claude-only pieces are listed so a `compatibility` issue can start from the list: `$ARGUMENTS`, `${CLAUDE_PLUGIN_ROOT}`, `${CLAUDE_SKILL_DIR}` and - the `arguments:` frontmatter list in skills, the `subagent_type: fork` that - `implement-spec` relies on, the hooks that load the rules, and the agents' - frontmatter. + the `arguments:` frontmatter list in skills, the hooks that load the rules, + and the agents' frontmatter. The list grows from issues labelled `compatibility`. diff --git a/evals/skills/implement-spec/works-all-waves/README.md b/evals/skills/implement-spec/works-all-waves/README.md index 943df47..9278116 100644 --- a/evals/skills/implement-spec/works-all-waves/README.md +++ b/evals/skills/implement-spec/works-all-waves/README.md @@ -21,12 +21,10 @@ xcrun cache the sandbox forbids; the Makefile's implement-spec target puts `bin` on the PATH and grants Bash, Write and Edit. The prompt gives the start answers up front (spec cadence, no optional tests, -already on a branch), since nothing can answer mid-run. It also says the Agent -tool has no fork in this run, for the reason works-one-wave gives: in the -headless eval sandbox the Agent tool answers "Agent type 'fork' not found", so -implement-spec takes its inline fallback and invokes the Skill tool with -`freethinking:report` and `handback` itself for each report. That fallback is -the path this case grades; the fork path is not exercised. +already on a branch), since nothing can answer mid-run. implement-spec writes +each wave report and the final report by invoking the Skill tool with +`freethinking:report` and `handback` in the session itself, which is what the +`invokes-report` and `problems-first` graders read. ## Graders diff --git a/evals/skills/implement-spec/works-all-waves/prompt.md b/evals/skills/implement-spec/works-all-waves/prompt.md index 89ec627..3d37992 100644 --- a/evals/skills/implement-spec/works-all-waves/prompt.md +++ b/evals/skills/implement-spec/works-all-waves/prompt.md @@ -1,5 +1,5 @@ --- -description: On a scaffolded project with a committed two-wave spec, implement-spec works the whole spec at spec cadence. It works wave 0 and wave 1 without waiting between them, each with its executor agent, its own test run, its wave-reviewer agent and its full suite, writes a report after each wave, has the reviewer agent read the whole run once more in a final review, and hands back once at the end. The Agent tool has no fork in a headless run, so each wave report and the final report are written inline by the report skill, the fallback of implement-spec's step 6, and that is the path graded. Needs the Makefile's implement-spec target, which grants Bash, Write and Edit, runs the scaffold, and puts the wrappers in bin on the PATH. +description: On a scaffolded project with a committed two-wave spec, implement-spec works the whole spec at spec cadence. It works wave 0 and wave 1 without waiting between them, each with its executor agent, its own test run, its wave-reviewer agent and its full suite, writes a report after each wave, has the reviewer agent read the whole run once more in a final review, and hands back once at the end. Each wave report and the final report are written by the report skill, invoked inline by implement-spec's step 6. Needs the Makefile's implement-spec target, which grants Bash, Write and Edit, runs the scaffold, and puts the wrappers in bin on the PATH. tags: [skill, implement-spec] max_turns: 70 allowed_tools: [Read, Glob, Grep, Agent, Skill] @@ -7,4 +7,4 @@ expected_outcome: Claude validates the graph, launches the task-executor agent f --- /freethinking:implement-spec calc-subtract spec -I will not be able to answer questions during this run. Cadence: spec, run every wave and hand back only at the end. There are no optional tests. The branch is already calc-subtract. This sandbox refuses edits under .claude/, so if ticking a task in tasks.md is refused, say so in the report and carry on to the next wave as if it were ticked. The Agent tool has no fork in this run; write every hand-back inline. +I will not be able to answer questions during this run. Cadence: spec, run every wave and hand back only at the end. There are no optional tests. The branch is already calc-subtract. This sandbox refuses edits under .claude/, so if ticking a task in tasks.md is refused, say so in the report and carry on to the next wave as if it were ticked. diff --git a/evals/skills/implement-spec/works-one-wave/README.md b/evals/skills/implement-spec/works-one-wave/README.md index 931b2c5..f80db81 100644 --- a/evals/skills/implement-spec/works-one-wave/README.md +++ b/evals/skills/implement-spec/works-one-wave/README.md @@ -18,12 +18,10 @@ need an xcrun cache the sandbox forbids; the Makefile's implement-spec target puts `bin` on the PATH and grants Bash, Write and Edit. The prompt gives the start answers up front (wave cadence, no optional tests, -already on a branch), since nothing can answer mid-run. It also says the Agent -tool has no fork in this run: in the headless eval sandbox the Agent tool -answers "Agent type 'fork' not found", so implement-spec's step 6 takes its -inline fallback and invokes the Skill tool with `freethinking:report` and -`handback` itself. That fallback is the path this case grades; the fork path -is not exercised. +already on a branch), since nothing can answer mid-run. implement-spec's step 6 +writes the hand-back by invoking the Skill tool with `freethinking:report` and +`handback` in the session itself, which is what the `invokes-report` and +`problems-first` graders read. ## Graders diff --git a/evals/skills/implement-spec/works-one-wave/prompt.md b/evals/skills/implement-spec/works-one-wave/prompt.md index 7f9e51e..b3eb5eb 100644 --- a/evals/skills/implement-spec/works-one-wave/prompt.md +++ b/evals/skills/implement-spec/works-one-wave/prompt.md @@ -1,5 +1,5 @@ --- -description: On a scaffolded project with a committed two-wave spec, implement-spec works the first wave at wave cadence. It launches the executor agent for the task, runs the tests itself, has the reviewer agent read the wave, ticks the task, runs the full suite, and hands back without starting wave 1. The Agent tool has no fork in a headless run, so the hand-back is written inline by the report skill, the fallback of implement-spec's step 6, and that is the path graded. Needs the Makefile's implement-spec target, which grants Bash, Write and Edit, runs the scaffold, and puts the wrappers in bin on the PATH. +description: On a scaffolded project with a committed two-wave spec, implement-spec works the first wave at wave cadence. It launches the executor agent for the task, runs the tests itself, has the reviewer agent read the wave, ticks the task, runs the full suite, and hands back without starting wave 1. The hand-back is written by the report skill, invoked inline by implement-spec's step 6. Needs the Makefile's implement-spec target, which grants Bash, Write and Edit, runs the scaffold, and puts the wrappers in bin on the PATH. tags: [skill, implement-spec] max_turns: 40 allowed_tools: [Read, Glob, Grep, Agent, Skill] @@ -7,4 +7,4 @@ expected_outcome: Claude validates the graph, launches the task-executor agent f --- /freethinking:implement-spec calc-subtract wave -I will not be able to answer questions during this run. Cadence: one wave. There are no optional tests. The branch is already calc-subtract. Work the first wave and hand back. The Agent tool has no fork in this run; write the hand-back inline. +I will not be able to answer questions during this run. Cadence: one wave. There are no optional tests. The branch is already calc-subtract. Work the first wave and hand back. diff --git a/skills/implement-spec/SKILL.md b/skills/implement-spec/SKILL.md index 580afed..39316fa 100644 --- a/skills/implement-spec/SKILL.md +++ b/skills/implement-spec/SKILL.md @@ -18,7 +18,7 @@ Terms used below: - A **leaf task** is a checkbox task in `tasks.md` with no sub-tasks: a top-level task with no sub-tasks is a leaf; a parent with sub-tasks is not. A leaf task marked `*` is an **optional test**. - An **ephemeral check** is a scratch script or call that exercises new code to prove a task works, is never added to the codebase, and is deleted before the task is done. It is how a code task is verified when its tests are a task of their own. - A **wave** is one entry of the Task Dependency Graph at the end of `tasks.md`. Waves run in order. Task numbers only identify tasks; the graph decides the order of work. -- The **cadence** is the author's choice of when a run hands back: after each leaf task (task cadence), after each wave (wave cadence), or only at the end of the run (spec cadence). At spec cadence every wave still ends with its review, its full suite and its report, but the report is written into the conversation without waiting for the author, and the next wave starts at once, so the session holds one short report per wave and never the diff; once the last wave is done, the reviewer reads the whole run once more, in a final review, and the run hands back with the final report. Before the end it hands back only for a failure, an unfixed finding or a proposal. +- The **cadence** is the author's choice of when a run hands back: after each leaf task (task cadence), after each wave (wave cadence), or only at the end of the run (spec cadence). At spec cadence every wave still ends with its review, its full suite and its report, but the report is written into the conversation without waiting for the author, and the next wave starts at once, in the same turn, so the session holds one short report per wave and never the diff; once the last wave is done, the reviewer reads the whole run once more, in a final review, and the run hands back with the final report. Before the end it hands back only for a failure, an unfixed finding or a proposal. - The **optional-tests choice** is the author's choice of what to do with optional tests: ask about them (at the start of each wave that has one, or at spec cadence once at the end of the run), always do them, or never do them. - A **skipped test** is an optional test the author chose not to do in this run. It stays unticked. - An **executor** is the plugin's `task-executor` agent, launched as `freethinking:task-executor`, one per leaf task. Its own definition holds its procedure, so the launch carries only the task: it gathers the context the task needs, implements exactly what the task describes, proves it, and returns a summary with a `status` of `done` or `stopped`, the files it changed, the checks it ran, its ephemeral check's code, what it is unsure of and, when stopped, why. Tests are tasks of their own in a spec, so an executor writes a test only when its task is a test task. @@ -33,7 +33,7 @@ Terms used below: - Only when something is wrong: the approval of a graph fix in step 2; the question in step 6 of how to proceed after a failing suite, a task that will not pass or an unfixed finding; the proposal in step 6 when the design cannot be followed or a task needs a new dependency. - Only when the author asks for it: the question in step 3 whether to do the remaining optional tests of an otherwise complete spec; the optional-tests question at the start of each wave that has one, in step 5, when the author chose to be asked, or at spec cadence the one offer of every skipped optional test at the end of the run, in step 8; the confirmation in step 7 before skipping a non-optional task or working out of wave order; the request in step 5 for a task done by hand, when the spec has one. - **How it asks.** With the AskUserQuestion tool, fixed options plus free text. The set-up round of step 4 is one call holding the cadence (after each task; after each wave; run every wave and hand back only at the end), the optional tests (ask about them, at each wave that has one or at spec cadence once at the end; always do them; never do them) and the branch (create it under the proposed name). The tool also asks the per-wave optional-tests question (do them in this wave, skip them), the end-of-run optional-tests offer at spec cadence (do them now, leave them), the choice of spec in step 1, the confirmations of steps 3 and 7, and the approval of a graph fix or spec edit once the edit is shown in full. The question of how to proceed after a hand-back, and the request for a task done by hand, are prose, since the answer is the author's own words. -- **Hand-backs.** The two tables of step 2; the report of step 6, written by the plugin's `report` skill in a fork of the conversation, with the failing output verbatim or the proposed edit quoted as it would read when the hand-back is for one; at spec cadence, the report of step 6 for each wave, written without waiting, and the report of step 8, covering the run with a line for the final review. +- **Hand-backs.** The two tables of step 2; the report of step 6, written by the plugin's `report` skill, with the failing output verbatim or the proposed edit quoted as it would read when the hand-back is for one; at spec cadence, the report of step 6 for each wave, written without waiting, and the report of step 8, covering the run with a line for the final review. - **Hand-offs.** `start-task` when there is no spec or no `tasks.md`; `pr` when every non-optional task is ticked, at step 3 or step 8. ## 1. Find the spec @@ -147,19 +147,17 @@ Tick only after that: every task of the wave whose checks passed, any that appea ### Ending a wave -With the review done and every task in the wave ticked, other than a skipped test or a task the author chose to skip under step 7, run the project's full test suite, the way the project runs it. If it passes, go to step 6 and report, with the suite's result in the report; at task cadence that is the report for the wave's last task. At spec cadence a pass goes to step 6 too, for the wave's report, but does not wait: write the report in full, as step 6 writes it, as a message of its own, then start the next wave at once. A line saying the wave is done is not the report; the report is what the final review reads back. If it fails, go to step 6 with the failing output verbatim and ask the author how to proceed before starting the next wave. Never start a task from a later wave while a task in this wave, other than a skipped test, is unticked. +With the review done and every task in the wave ticked, other than a skipped test or a task the author chose to skip under step 7, run the project's full test suite, the way the project runs it. If it passes, go to step 6 and report, with the suite's result in the report; at task cadence that is the report for the wave's last task. At spec cadence a pass goes to step 6 too, for the wave's report, but does not wait: write the report in full, as step 6 writes it, then launch the next wave's executors in the same turn, straight after it, with nothing written between. The report ends the message, not the turn. A line saying the wave is done is not the report; the report is what the final review reads back. If it fails, go to step 6 with the failing output verbatim and ask the author how to proceed before starting the next wave. Never start a task from a later wave while a task in this wave, other than a skipped test, is unticked. ## 6. Hand back and report -Hand back here at the end of a cadence unit (after a task at task cadence, after a wave at wave or spec cadence), after a failed full suite, after a task whose tests still fail after two fix attempts, after a review finding that is still not fixed after two attempts, or when an executor reports that the design cannot be followed or needs a new dependency. For the final wave, when its full suite passes, step 8 takes this step's place; at spec cadence, write that wave's report first, without waiting, as for any other wave, so that every wave's report is in the conversation for step 8. Write the report as the last thing in the message with nothing after it, then wait for the author; at spec cadence, after a wave whose full suite passed, the report is still a message of its own, written without waiting, and the next wave starts at once. +Hand back here at the end of a cadence unit (after a task at task cadence, after a wave at wave or spec cadence), after a failed full suite, after a task whose tests still fail after two fix attempts, after a review finding that is still not fixed after two attempts, or when an executor reports that the design cannot be followed or needs a new dependency. For the final wave, when its full suite passes, step 8 takes this step's place; at spec cadence, write that wave's report first, without waiting, as for any other wave, so that every wave's report is in the conversation for step 8. The report ends the message, with nothing written after it in that message. A hand-back that waits for the author is the only place the turn ends: at task or wave cadence that is every report, and at spec cadence it is the final report of step 8 or a stop for a failure, an unfixed finding or a proposal. At spec cadence, after a wave whose full suite passed, the report is written without waiting, and the next wave's executors are launched in the same turn, straight after it, as the next tool calls. -To write the report, launch the Agent tool with `subagent_type: fork`, in the foreground, with the prompt "Invoke the Skill tool with `skill: freethinking:report` and `args: handback`, and return the Message as your whole result". Paste the result verbatim as the Message to the author, with nothing before or after it. - -If the Agent tool offers no fork, the fork fails, or it returns anything other than one Message (one that drops the failing output, the finding or the proposed edit this hand-back is for counts as that), paste nothing from it: invoke the Skill tool inline with the same `skill` and `args`, once, and let the Message end with one line saying the report was written inline. If the Skill tool cannot find `freethinking:report`, say the plugin's `report` skill is missing and write the hand-back by the communication rule, Problems first. +To write the report, invoke the Skill tool with `skill: freethinking:report` and `args: handback`, once, in this session; its Message is the report, with nothing written before or after it in that message. If the Skill tool cannot find `freethinking:report`, say the plugin's `report` skill is missing and write the hand-back by the communication rule, Problems first. At spec cadence each report covers its own wave, and the reports stack up in the conversation, one per wave, for the final review and the final report of step 8 to read back. A hand-back that waits, for a failure, an unfixed finding or a proposal, covers the wave it stops in, with the earlier waves' reports already above it. -The fork inherits this conversation, so the failing output, the finding, the proposed edit and the question are already in its view. Each goes under the report's problems or its next step (the Problems and Next sections in the plugin's template), by the cause of the hand-back: +The failing output, the finding, the proposed edit and the question are already in the conversation the report skill reads. Each goes under the report's problems or its next step (the Problems and Next sections in the plugin's template), by the cause of the hand-back: - **A failing full suite or a task that will not pass**: the failing output verbatim, then which causes are confirmed and which are open; next, the question of how to proceed. Do not start the next wave. - **A review finding that is still not fixed**: the finding quoted, then what each of the two attempts did; next, the question of how to proceed. @@ -179,7 +177,7 @@ When the author asks to skip a non-optional task or to work out of wave order, s When every leaf task is ticked, apart from skipped tests and any non-optional task the author chose to skip under step 7, and the final wave's full suite passes, finish. -At task or wave cadence, write the report for the final wave with its full suite result, by the same fork as step 6, in place of step 6. The Message's next step (its Next section in the plugin's template) names any skipped tests as left unticked and recommends `/freethinking:pr` to open the pull request, which the fork writes because this step's text is in its view; add nothing after the paste. Then stop. +At task or wave cadence, write the report for the final wave with its full suite result, by the same invocation as step 6, in place of step 6. The Message's next step (its Next section in the plugin's template) names any skipped tests as left unticked and recommends `/freethinking:pr` to open the pull request, which the report skill writes because this step's text is in the conversation; add nothing after it. Then stop. At spec cadence, finish in three parts: @@ -192,4 +190,4 @@ At spec cadence, finish in three parts: - the files that cite the changed ones, found as for a wave review. Nothing more, and never the diff, which the reviewer gathers itself. Its job is the whole: consistency across waves, nits and other items carried from wave to wave that have piled up, and anything one wave's review could not see. Handle its Findings and `unsure` items by the rules of "Reviewing the wave" in step 5: a `blocker`, a `should-fix` or an `unsure` item that is a defect is fixed by an executor within two attempts, with the owning task's checks re-run here; a `nit` is fixed or carried; a finding in the spec goes to step 6 as a proposal; and a finding still not fixed after two attempts goes to step 6. If anything was fixed, run the full suite once more; a failure goes to step 6. There is no second final review. -3. **Final report.** Write the report by the same fork as step 6, covering the whole run: every task ticked in it, the full suite result and the wave-review verdict of each wave, the final review's verdict and whether anything was carried, and the files the run changed. The plugin's template writes the verdicts as the lines `Wave review of wave : ` and `Final review of the run: `, each followed by `carried: see Problems` or `nothing carried`; with a repository's own template, the final review reads the wave verdicts, and the final report writes the run's verdict, wherever that template puts them. The Message's next step names any skipped tests as left unticked and recommends `/freethinking:pr` to open the pull request, which the fork writes because this step's text is in its view; add nothing after the paste. Then stop. +3. **Final report.** Write the report by the same invocation as step 6, covering the whole run: every task ticked in it, the full suite result and the wave-review verdict of each wave, the final review's verdict and whether anything was carried, and the files the run changed. The plugin's template writes the verdicts as the lines `Wave review of wave : ` and `Final review of the run: `, each followed by `carried: see Problems` or `nothing carried`; with a repository's own template, the final review reads the wave verdicts, and the final report writes the run's verdict, wherever that template puts them. The Message's next step names any skipped tests as left unticked and recommends `/freethinking:pr` to open the pull request, which the report skill writes because this step's text is in the conversation; add nothing after it. Then stop.