Skip to content

[#11] Add ticket argument, offers and review prompts to the pr skill - #23

Merged
dansabin merged 2 commits into
mainfrom
11-pr-skill-ticket
Oct 14, 2026
Merged

dansabin merged 2 commits into
mainfrom
11-pr-skill-ticket

Conversation

@sabind

@sabind sabind commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

Why

Learnings from opening #9, #10 and the PR for #2 with /freethinking:pr on 2026-10-13: the skill's only argument was a base branch, so a ticket passed to it was read as a base override; the walkthrough asked for a reason and moved on; and the Why bullet named the ticket when the template already asked for a Closes line. The ticket is now the argument and the issue feeds the Why, the author answers a short set of review prompts for every change, and the description is assembled from those answers rather than written fresh at the end. The two recoverable starting checks, on the default branch and with uncommitted changes, become offers instead of stops, and the skill gets the CX block that records its stops and the prose walkthrough as a deliberate departure.

Closes #11
Closes #20
Closes #18

What

/freethinking:pr [ticket] [base-branch] reads the ticket with gh issue view before the walkthrough, offers to create the branch when on the default branch and to commit uncommitted work after the walkthrough, puts five numbered review prompts to the author for each change in prose, and drafts the description from their answers, naming the ticket exactly once.

Changes

  • Reshape skills/pr/SKILL.md: two named arguments ticket and base read by shape and by whether origin/<base> exists, the starting facts and the issue read arriving as injected commands, a "Working with the author" block, the branch and commit offers, a ticket step, five review prompts per change with the ticket's words quoted under "why", the name-the-ticket-once rule, and the commit made after the walkthrough with git commit -F. During this PR's own walkthrough, step 0 gained a fallback: when $ticket fails its test, the first word of $ARGUMENTS is tested, because Claude Code parses the typed text as a shell command line and drops a #n left on its own line as a comment. The ticket's description still holds; there were tweaks with lessons learned about the injected commands and the named args. It does not cover user testing: the PR flow gets a little heavier and will need more fine tuning. External issue systems could behave in confusing ways, but it should be recoverable even if friction is a little high.
  • Wire the skill in: the start-task hand-off recommends /freethinking:pr <ticket> and says pr offers to branch and commit, the README row and CONTRIBUTING note describe the new arguments and the branch offer, and the plugin version moves to 0.7.0. The start-task skill was changed to make the transition more smooth; the rest of the edits are just glue. The git logic moves to pr, where it is expected, instead of floating in between.
  • Add the spec under .claude/specs/pr-skill-ticket/, revised during implementation from a shape grammar to Claude Code's named arguments and injected commands. These are optimisations learned from implementation to help reduce LLM churn. Task 7.1, a real ticket in this repository, is ticked by this PR's own walkthrough.
  • Evals: the fake gh learns issue view, the happy-path prompt answers all five review prompts in advance, stops-at-starting-checks drops the default-branch wording, and two new cases, opens-pr-with-ticket and offers-branch-on-default, cover the ticket path and the branch offer. These mock what is needed and test the injected commands and the pr skill entirely. They do not cover a real use case; manual testing covers that.
  • Add .claude/settings.json, enabling the GitHub plugin locally for better GitHub interactions, since this is a GitHub project.

Testing

make validate: 11 checks OK, both claude plugin validate --strict runs passed.

This PR was opened with the branch's own /freethinking:pr #11, which is task 7.1 of the spec: the first message stated the argument reading, the issue and its two comments were read with gh, each change arrived with five numbered prompts, open prompts were asked for by number, the Closes question and the approval rendered in the tool, and the draft names each ticket once.

make eval-skill-start-task:

CASE SCORE PASS% RUNS
bug-offers-bugfix 1.00 100% 3
contained-change-offers-executor 1.00 100% 3
names-task-and-asks 1.00 100% 3
wrong-repo-asks-fit 1.00 100% 3

make eval-skill-pr, best complete run per case across today's invocations:

CASE SCORE PASS% RUNS NOTES
offers-branch-on-default 1.00 100% 3
opens-pr-with-stub-gh 1.00 100% 3
opens-pr-with-ticket 1.00 100% 3 after the step 0 fallback; see notes
stops-at-starting-checks 0.83 67% 3 one run timed out after 300s at $0.01

Four runs across the earlier invocations timed out after 300s. Each trace ends in a single api_retry with error unknown and nothing after it, so the API stalled in the sandbox; the skill was not working for five minutes. Three of those were opens-pr-with-ticket runs before the fallback; its rerun after the fallback passed three of three in 92s.

Risk and rollout

None. The named arguments and injected commands are Claude Code features and are not one-to-one transferable to other coding agents, which locks the plugin to Claude for now; they will come out later as compatibility work.

Notes for reviewers

  • Look at the review prompts and the skill changes, and consider whether the PR description they produce is helpful or whether too much is asked of the author.
  • The step 0 fallback exists because Claude Code parses the typed text as bash: /freethinking:pr #7 followed by a newline and prose gave $ticket as I and $base as am in one eval trace. The eval case's three earlier passes came from the model recovering the ticket from the raw text, not from $ticket.
  • The spec's Claude-specific parts should be read at face value for function and the specific edits and requirements.
  • Because gh is mocked, its signature can change and the mocks go stale with nothing to show it; real gh and git cannot run in the eval sandbox, so there is no other option unless something is missing. Look at the prompts and coverage for other cases worth evaluating. The canned issue answers every gh issue view the same, including --json title.
  • Task 7.2 of the spec, both offers from the default branch, is still unticked.
  • evals/README.md carries one over-long line where the note on the copied fake was spliced into a wrapped paragraph.

🤖 Generated with Claude Code

dansabin and others added 2 commits October 13, 2026 14:40
… block

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The ticket is now the first argument and the issue feeds the Why; the
default-branch and uncommitted-changes checks become offers; each change
gets five review prompts whose answers assemble the description; the
ticket is named once. The start-task hand-off is smoothed to recommend
`/freethinking:pr <ticket>`, the rest of the wiring is glue, and the spec
is revised with the lessons learned about injected commands and named
arguments. Step 0 falls back to the first word of $ARGUMENTS because
Claude Code drops a `#n` on its own line as a shell comment. Evals mock
`gh issue view` and add the ticket and branch-offer cases.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@dansabin
dansabin merged commit c36f34a into main Oct 14, 2026
1 check passed
@dansabin
dansabin deleted the 11-pr-skill-ticket branch October 14, 2026 21:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants