Skip to content

[#16] implement-spec: CX block, spec cadence and final review - #25

Merged
sabind merged 1 commit into
mainfrom
group-dev-tools-issues-ae4cad73
Oct 15, 2026
Merged

sabind merged 1 commit into
mainfrom
group-dev-tools-issues-ae4cad73

Conversation

@sabind

@sabind sabind commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Why

Closes #16

Until now implement-spec asked its cadence question in prose, and the author could not see its stops up front without reading the whole procedure. The stops are important for how the skill engages the author, so that the collaboration leans on user agency and user accountability.

What

implement-spec opens with a Working with the author section that lists every stop, how each asks, what each hand-back holds and what it hands off to, and its set-up questions become one AskUserQuestion call. It gains a third cadence, spec, which runs every wave end to end, writes a report after each wave and has the wave-reviewer read the whole run once more before the one hand-back. The reviewer now diffs from the branch's base so a mid-run commit hides nothing, and the workflow rule and start-task hand-off stop describing implement-spec's internals.

Changes

  • Add a Working with the author section to skills/implement-spec/SKILL.md, between the Terms and step 1: the stops in four tiers (every run, only when something was left out, only when something is wrong, only when the author asks for it), how it asks, the hand-backs and the hand-offs. The stops are important for how the skill engages the author, so that the collaboration leans on user agency and user accountability. It covers what to do and how, with limited instruction on why, so it may be hard to guide the prose without that input. It has no defence instructions against a request to skip or work another way, and it is unclear whether authors should be forced down a path or only guided; since the author is accountable for their actions they can break out, and only they pay the cost, or they may make other users pay it if they skipped reviewing their code while writing it.
  • Three cadences, task, wave and spec, and an optional-tests choice (ask, always, never), asked together in one AskUserQuestion call at set-up and held through every resume; step 1 accepts spec or "run all the waves", and names the spec and proceeds when exactly one qualifies instead of confirming it. The new cadence was found at dev time: run all the waves and collect the reviews as you go, then review all the reviews, or run task level or wave level with stops. This covers the logic around the stops; it does not go deeper into what testing theory might need to change when the waves run together, especially with stacking changes in wave 1 carried into wave 2 on assumptions. We are assuming the LLM can handle it.
  • Spec cadence itself: each wave's report is written in full inline without waiting, the run stops early only for a failed suite, an unfixed finding or a design proposal, and step 8 ends with one offer of the skipped optional tests, a final review of the run with the reviewer launched once over every file it changed, and a final report carrying a Final review line. This is the end-to-end automatic implementation without wave-level actions, which the author can trigger.
  • agents/wave-reviewer.md finds the branch's base (the merge base with the default branch, falling back to the local default, then HEAD), diffs every tracked file from it, reports the base in its Findings and can review every file a whole run changed; its README row says so. This was found mid-run: the reviewer must not rely on uncommitted files alone in case there are mid-run commits, and needs a good way to find all the work in the branch. It is small and the evals need it. It could break, but no differently from the previous option, which was broken by default. It may have more context than it needs when looking at diffs, which is better than less, or incorrect mid-run half-committed work.
  • Trim the spec bullet of rules/workflow.md and the hand-off of skills/start-task/SKILL.md to name implement-spec and point at the first wave, without reciting its cadences, ticking or sub-agents. An optimisation found mid-run: the rules were getting fat and did not need so much skill context, given the 10,000-character ceiling on rules. The frontmatter was updated to carry the triggers instead, to avoid rule bloat.
  • Add checks/test_descriptions.py, which fails make validate when any skill or agent description is missing, empty or over 1,024 characters, and shorten the implement-spec description from 978 to 772 characters. There are limits to frontmatter descriptions, so we added a test and shortened it.
  • Add the works-all-waves eval case under evals/skills/implement-spec/, reusing the calc-subtract scaffold at spec cadence, with seven graders including a trace regex that proves wave 0's report lands before wave 1's executor launches; evals/README.md notes that the implement-spec target needs the cost ceiling raised. Added for the new features.
  • Add the Propose a spike issue form (.github/ISSUE_TEMPLATE/spike.yml, type Spike, label triage) and list Spike as a fourth issue type in .github/CONTRIBUTING.md. We found something that was neither a bug nor a feature, created a spike and put the work there. Spikes are a well documented issue tracker feature for time-boxed research work.
  • Bump the plugin to 0.8.0 and update the README row for implement-spec with the third cadence. The bump follows semver; it is needed and the level matters.

Testing

make validate: 16 tests OK.

make eval-skill-implement-spec, the three existing cases, run before the last wording changes to the skill:

CASE SCORE PASS% RUNS NOTES
invalid-graph-stops 1.00 100% 3
no-spec-recommends-start-task 1.00 100% 3
works-one-wave 1.00 100% 2 third run stopped by the $2 cost ceiling

The new case on the final wording, --runs 2:

CASE SCORE PASS% RUNS NOTES
works-all-waves 1.00 100% 2 7 of 7 graders each run, $1.92 for both

An earlier kept run of works-all-waves scored 1.00 while its transcript showed the per-wave reports were not written inline, only a one-line "wave done" per wave. The skill was tightened and the reports-wave-0-inline grader added, then the two runs above.

The skill was also tested by hand and confirmed working.

The workflow, collaboration and start-task evals were not re-run on this branch after the rule and start-task trims and the shortened description.

Risk and rollout

Minor: no new cross-skill dependencies were added, so the worst case is that the spec skill is a little more pedantic. The rule and start-task trims carry a risk of rules not triggering skills as they did before, and the shorter description may make the skill less likely to trigger, with less context to anchor to.

Notes for reviewers

  • Read every prompt.md, SKILL.md and rules file changed here, and check the eval cases cover every angle and vector for testing.
  • Judge whether the waves at spec cadence are aware enough to stop, or whether this is a lot of risk: cascading wrong assumptions can increase work in complex cases compared with going wave by wave and answering follow-ups before the waves stack. The author is given the control and is accountable for the outcome.
  • For the reviewer's base-finding, the thing to weigh is whether the core concept is good; the rest is small.
  • Check the rule and start-task content is correctly segregated: the rule routes and the skill carries its own procedure.
  • Check what was removed from the implement-spec description did not drop a key trigger.
  • Known gaps: checks/test_descriptions.py measures a description written on one line, so a quoted or multi-line YAML description would be mis-measured; the works-all-waves spec files are copies of works-one-wave's, so a change to the calc-subtract spec has to be made in both.
  • Follow-up: Spike: should implement-spec own per-wave commits at spec cadence? #24, a spike on whether implement-spec should own per-wave commits at spec cadence.

🤖 Generated with Claude Code

Add a Working with the author section to implement-spec that lists every
stop in four tiers, how each asks, the hand-backs and the hand-offs, so
the collaboration leans on user agency and user accountability. Add a
third cadence, spec, found at dev time: run all the waves, collect the
reviews as you go, then review all the reviews, with the wave-reviewer
launched once more over the whole run. Ask cadence and optional tests in
one AskUserQuestion call at set-up.

The wave-reviewer now diffs from the branch's base so mid-run commits do
not hide earlier work. The workflow rule and start-task hand-off name
implement-spec without reciting its internals, to avoid rule bloat. A
free check holds every description under 1,024 characters. A new eval
case, works-all-waves, proves spec cadence. Add the Propose a spike
issue form. Bump the plugin to 0.8.0.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@sabind
sabind merged commit 753c223 into main Oct 15, 2026
1 check passed
@sabind
sabind deleted the group-dev-tools-issues-ae4cad73 branch October 15, 2026 21:18
sabind pushed a commit that referenced this pull request Oct 16, 2026
…eport skill

Takes PR #31 (report skill, communication rule) and PR #25 into the branch.
Step 7 of the bugfix skill now invokes freethinking:report in handback mode
and puts its Message to the author as the hand-off question's text; the three
exits stay. The collaboration rule keeps the new bullet where "Bad news first"
was. Version 0.10.0 on top of 0.9.0. The hands-off-to-pr case gains graders
on the report call and its Message.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@sabind

sabind commented Oct 16, 2026

Copy link
Copy Markdown
Contributor Author

The two known gaps in this description are now issues, filed 2026-10-16: #34 (test_descriptions.py measures only the first line of a description) and #35 (the two implement-spec cases carry identical copies of the calc-subtract spec). #24, the per-wave commit spike, was decided no and closed the same day.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

implement-spec skill: add a CX block

2 participants