Skip to content

Run the user's step yourself before handing back a fix, retry or cause #248

Closed

Description

@lukemaj

What: AgentsMD tells agents to run a step themselves, or say they could not, before handing it back to the user as a fix or a retry, and to name what showed a cause before stating it.
Why: In the 2026-10-12 to 2026-10-19 review the most frequent failure was work handed back unchecked: 9 unchecked claims in 5 sessions across three harnesses and 6 untested "retry this" fixes; the existing "Go and see" rule is not applied at the handback moment.
So what: A Sol worker implements it, Grok and Muse Spark review, and Luke approves the merge.

Outcome

At the moment an agent hands a step, fix or cause back to the user, the instructions require it to have run the step or name the evidence, or to say plainly it could not.

Evidence (verdicts under judge spectrum-review, bestia ledger)

  • Untested handback: Grok car connector OAuth loop 2026-10-16 18:04 to 18:20 (grok:01a144a9-b8b5-72d3-8ccf-b60fbe045aa9 prompts 51, 52, 68); a copy command that only worked on rocky (claude:ead2ef59-e95f-44d9-b752-fc2f0a7b52ed lines 0 and 103).
  • Unchecked claims: "599 commits behind" when live main was 106 (codex:01a14150-c716-7740-95ee-ac8e36d2b972); a rollback blamed on the new build though the old build had the same PATH bug (claude:170e2046-0af7-466f-b6f6-1292c00e829b line 1953); a stale approval question about #423 (same session, line 6096).
  • Recurrence report: confusion-unchecked-claim 9x in 5 sessions; frustration-untested-fix-handed-back 5x in 2 sessions.

Contribution to the Objective: fewer user corrections per task, measurable in the next weekly review.

Elon record

  • Requirements and who asked: Luke (2026-10-19 review follow-up): agents must not hand back untested fixes or unverified causes.
  • Deleted: a new hook or blocking check; a new standalone rule section. The change sharpens existing "Go and see" behavior at its owner.
  • Bottleneck: the handback moment, where the rule is not applied.
  • Checked myself: the cited sessions and the recurrence report; the current core "Go and see" text and references/verification.md.

Acceptance criteria

  • One short addition at the existing owner (the core's "Go and see" or references/verification.md, whichever already owns handback proof) covering: run the user's step yourself before asking them to retry, or say you could not; when stating a cause or number, name the command or log line that showed it.
  • Net instruction growth stays minimal; no duplicate of existing text.

Non-goals

Blockers

None.

Required proof

  • Writing-for-agents procedure applied; existing instruction tests and budgets pass.
  • Independent review on the exact head; version transition per repository policy.
  • Behavioral effect is measured in the next Agent Observer review, not claimed at merge.

Activity

  1. added a commit that references this issue on Oct 19, 2026
  2. lukemaj commented on Oct 19, 2026

    @lukemaj
    ContributorAuthor

    Delivery record (2026-10-19, Warsaw time):

    • Merged: PR fix: run the user's step before handing back a fix (#248) #249 at 15:55:58 with Luke's approval (given in Gavin's thread), merge commit ce17c3b, head 690e66b. Independent review review/independent success on that head (Grok 4.7 medium, Muse Spark 1.3 xhigh, no findings).
    • Released: v17.4.1 by the release workflow at 15:56:27; promoted to the toolboxmd marketplace (Promote AgentsMD v17.4.1 through Toolybara marketplace#199).
    • Installed and verified on bestia, cavallo and rocky: Claude Code, Codex and Grok plugins at 17.4.1; Muse from git archive v17.4.1, hooks trusted_enabled; all five harness links valid-stable-link. Hook definitions unchanged since 17.3.1. Recorded in runbook c52fd20.
    • Cleanup: remote and local branch fix/248-run-step-before-handback and its worktree removed.
    • Behavioral Live Verification: pending; measured in the next Agent Observer review (unchecked claims and untested handbacks).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions