Skip to content

fix(pi): end the run on a skill's [ABORT] marker - #1488

Closed
Silthus wants to merge 2 commits into
PostHog:mainfrom
Silthus:fix/pi-skill-abort
Closed

Silthus wants to merge 2 commits into
PostHog:mainfrom
Silthus:fix/pi-skill-abort

Conversation

@Silthus

@Silthus Silthus commented Oct 22, 2026 •

Copy link
Copy Markdown

Problem

The Pi harness, the Wizard's default agent harness, stops a run only on the AI gateway's refusal marker ([posthog:abort]). When a skill hits a stop condition and writes [ABORT] <reason>, Pi carries on. The run can end on the success outro and tell the user setup worked when it did not. This hits every Pi program that relies on [ABORT]: revenue analytics, MCP analytics, self-driving, replay vision, wizard skill <name> and the transactional email program in #1487.

Impact

Pays into Grow Workflows through activation, measured in the team's internal Workflows growth review. In scope: the Wizard's transactional email setup (#1487), which reports a stop such as "the API is not enabled for this project" through [ABORT]. Expected move: small, up. A failed setup now says it failed and names the next step, so users don't walk away from a false "set up". Run cost: n/a.

Merge Danger

Note

🔵 Merge risk: low · two-way door (merge-risk rubric)

  • Changes when every Pi run ends: a reply with [ABORT] <reason> now stops the run with the error outro. A run whose agent writes the marker by mistake stops too. No shared prompt mentions the marker; only skills and the wizard_ask fallback do.
  • Limited by harness tests in both linear and orchestrator task modes, including the remark and post-complete_task exemptions. A revert restores the old behavior with nothing left behind.

Changes

  • skillAbortReason in src/agent/signals.ts parses [ABORT] <reason>. Both harnesses use it, so Pi and Anthropic read the marker the same way.
  • Pi reads the marker from every text block of a work turn, in linear runs (index.ts) and orchestrator task runs (task.ts). The first marker, a skill's [ABORT] or the gateway's refusal, is latched with its source and ends the run through the existing cancellation. The harness skips the continue nudges and the remark, and returns abort with the reason.
  • The linear sequence then shows the program's matched abort case, or its generic "aborted" fallback with the reason. The orchestrator ends the run as failed with the reason.
  • The remark reply never ends a run, and in task runs neither does a reply after complete_task reported.
  • agent aborted tags only a gateway refusal with source: 'gateway_policy'.

Test plan

  • Test seam: piBackend.run and runPiTask in src/agent/runner/harness/pi/__tests__/structured-run.test.ts, the harness's public entries, with the Pi session faked at the SDK boundary. Each case runs in both linear and task mode:
    • A skill reply with [ABORT] Could not find a Stripe integration returns abort with that reason, sends no nudge, never stops the spinner on the success message, captures no agent completed, and tags no gateway_policy source. The program declares no abort cases. Before the fix, the run hung waiting for a stop that never came.
    • The marker in a later text block of the same turn ends the run too.
    • The marker in the remark reply leaves a successful run successful.
    • In task mode, a skill or policy marker in the reply after complete_task leaves the task successful.
  • The sequence half, an abort result turning into the matched error outro, is covered by src/agent/__tests__/run-agent-standalone.test.ts.
  • pnpm typecheck passes, lint has no errors, and pnpm test passes (3809 tests). An earlier full run had a 5-second timeout flake in src/cli/__tests__/cli.test.ts (--region), a file this PR does not touch.
  • Live run: gap, accepted. This devbox has no gateway service key or personal API key for a headless --ci run, so a real Pi run against a skill that aborts did not run. What stays unknown is the real Pi SDK resolving prompt() after the cancellation. The gateway refusal path already relies on that same cancellation.

Decisions

Question Answer Who Where
Where does the fix live? In the Pi harness, the same way as the refusal marker, so every program gets it with none special-cased 👤 handoff brief
Accept the live-run gap? Yes, the cancellation path is shared with the gateway refusal 👍 comment
Does [ABORT] end runs for programs without abort cases? Yes, like the refusal marker; the sequence picks matched copy or its generic fallback. Replaced a first cut that copied the Anthropic gate on declared abort cases 🤖 comment
Do orchestrator task runs honor [ABORT]? Yes, with the refusal marker's remark and post-complete_task exemptions 🤖 comment
Does the Anthropic harness drop its abort-case gate too? No, left as a follow-up: it is the legacy fallback harness 🤖 comment

Involvement: 0 grilling rounds, 1 design pass (GPT-6.1 Sol), 1 review round and 1 final read (Codex adversarial-review, GPT-6.1 Sol), live-run proof not captured (gap accepted).

LLM context

Side fix from the Resend parity map, built test-first with Claude Code (Opus 5.5).

🤖 Generated with Claude Code

Silthus and others added 2 commits October 22, 2026 06:22
The Pi harness stopped only on the gateway's refusal marker, so a skill
that emitted [ABORT] <reason> ran on and could end on the success outro.
It now ends the run with the skill's reason for programs that declare
abort cases, like the Anthropic harness, and both share one parser.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every Pi run now ends on a skill's [ABORT] marker, whether or not the
program declares abort cases; the sequence picks the matched copy or its
generic fallback. Orchestrator task runs end on it too, with the same
remark and complete_task boundaries as the gateway refusal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Silthus

Silthus commented Oct 22, 2026

Copy link
Copy Markdown
Author

Note

🤖 Claude Opus 5.5 responding on behalf of Michael

Decisions made during the build and review of this PR:

  • 🤖 Does [ABORT] end runs for programs without abort cases? Yes, like the refusal marker; the sequence picks matched copy or its generic fallback. Replaced a first cut that copied the Anthropic gate on declared abort cases.
  • 🤖 Do orchestrator task runs honor [ABORT]? Yes, with the refusal marker's remark and post-complete_task exemptions.
  • 🤖 Does the Anthropic harness drop its abort-case gate too? No, left as a follow-up: it is the legacy fallback harness.
  • 👤 Accept the live-run gap? Yes: no gateway credentials on the build box, and the cancellation path is shared with the gateway refusal.

@Silthus Silthus closed this Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant