Valid handoff note rejected when the note is committed more than once #213

Activity

1. lbacik

   added

   bugSomething isn't workingSomething isn't working

   needs-triage

   on Oct 11, 2026
2. lbacik commented on Oct 11, 2026

   

   lbacik

   on Oct 11, 2026

   OwnerAuthor

   More actions

   Second occurrence: the proposed fix doesn't cover it

   Same rejection on simple-rpg-engine issue #34, attempt 2026-10-11T17:03:20.578527Z, agent 0.4.3, same model behind LiteLLM. This time, though, the "skip note-only commits" rule proposed above would not fix it, because the first note commit also carries a work change.

   Branch agent/issue-34 after the attempt (newest first):

   3cd85bb Seal attempt work for #34 (model_limit_reached)
   832d60a Handoff note: issue #34      <- note_commit
   755aa72 Handoff note: issue #34      <- note + `rename instrumentation.ts => src/instrumentation.ts (100%)`
   d40b4a0 Merge pull request #52 ...   <- base_revision; the note says last_work_commit: d40b4a0…
   

   Timeline from agent_output.json:

   UTC	event
   17:26:37	token_soft_threshold_crossed (6 308 118 / soft 6 300 000)
   17:26:38	token_soft_threshold_handoff_context_injected. At this point HEAD is d40b4a0 (= base) and all the work is uncommitted
   17:27:17	The model writes the note by hand and runs git add .agent/handoff/34.md && git commit. A git mv it had staged earlier gets swept into the same commit, 755aa72 (2 files changed)
   17:27:22	code_review_nudge, then skill_call code-review
   17:28:24	token_soft_threshold_handoff_followup. The model rewrites the note (same header) and commits it again as 832d60a (note only)
   17:28:50	model_execution_finished: status=model_limit_reached, then the seal commit 3cd85bb with the uncommitted work
   17:28:50	handoff_note_rejected, then completion_decided: outcome=incomplete

   The validator's expected value is commits[note_index + 1] = 755aa72. The proposed fix would keep 755aa72, because that commit touches more than the note, so the note's d40b4a0 would still be rejected.

   Suggested amendment

   When deriving expected_last_work_commit, skip the consecutive commits beneath note_commit that touch handoff_note_relpath(issue_number), whatever other files they also change. Use the first commit below that run, or base_revision if there is none. Checked against both cases:

   • Implement setup/check execution and completion evaluation #33: skip f70753a, 2fca1c4; expected 67b744f, accepted.
   • Implement branch publication, pull requests and attempt result comments #34: skip 832d60a, 755aa72; expected d40b4a0 (base), accepted.

   Trade-off: a change swept into a note commit (here the rename) doesn't count as "last work". It is still preserved on the branch, and the continuation reads the branch, so nothing is lost.

   For the regression tests, add a second case next to the one proposed above: a work commit, then a note commit that also changes another file, then a note-only commit, with last_work_commit set to the work commit (or to base_revision when there is no work commit). Assert outcome=handoff.
3. lbacik commented on Oct 12, 2026

   

   lbacik

   on Oct 12, 2026

   OwnerAuthor

   More actions

   This was generated by AI during triage.

   Agent Brief

   Category: bug
   Summary: Accept a valid handoff note when this attempt committed the note more than once

   Current behavior:
   _validate_handoff_note finds the newest commit that touches the issue's handoff note (handoff_note_relpath(issue_number)). It then takes the attempt commit directly beneath it (newest-first) as the expected last_work_commit, or base_revision when there is none. If the model commits the note more than once in the same attempt, the commit beneath the newest note commit is the previous note commit. The note then fails with handoff_note_rejected: Handoff note's last_work_commit does not match the last preserved work commit., and the attempt is downgraded from handoff to incomplete. Triage reproduced both shapes from this issue on main (0.4.3):

   1. Implement setup/check execution and completion evaluation #33 shape: work commit W, note commit, note commit again. The note names W and is rejected.
   2. Implement branch publication, pull requests and attempt result comments #34 shape: the first note commit also includes another staged file, then a note-only commit. The note names base_revision and is rejected.

   Desired behavior:
   When deriving the expected last_work_commit, skip the unbroken run of commits beneath the newest note commit that both:

   • touch the handoff note path, whatever other files they change, and
   • belong to this attempt, using the same rule handoff_note_commit already applies: author date at or after the attempt start (attempt_id / checkpoint started_at), with no time bound when attempt_id can't be parsed.

   The expected value is the first commit below that run, or base_revision if the run reaches the bottom of the attempt commits.

   The attempt-only limit matters. On a continued branch, an earlier round's note commit can sit directly under this round's notes. When this round has made no commit before the note, the pre-filled note header and the runner's emergency note both set last_work_commit to HEAD. HEAD is then that earlier-round note commit, and it must stay accepted, as it is today. Do not skip earlier rounds' note commits.

   Trade-off, accepted on purpose: a non-note change swept into one of this attempt's note commits (the #34 shape) doesn't count as "last work". It is still preserved on the branch, which is what a continuation reads.

   All other checks in the validator stay as they are: title, issue, reason, started_at as a warning only, and the minimum prefix length of last_work_commit. A seal commit on top of the note stays handled, as it is today.

   Key interfaces:

   • AttemptCommit only carries revision and subject, so the validator can't currently tell which commits beneath the note touched the note path. The AttemptWorkspace protocol, and its git-backed implementation GitWorkspace, need a way to report this attempt's note-touching commits, not just the newest one. One option is a method returning all of this attempt's commits that touch the note path, newest first; the newest is what handoff_note_commit returns today. The agent decides the exact shape. Keep handoff_note_commit's contract unchanged, or derive it from the new method. Update every in-memory/fake workspace used in tests, plus the workspace conformance tests, so they implement the new capability.
   • _validate_handoff_note(...): receives the extra information it needs (for example, the set of this attempt's note-commit revisions) and applies the skip rule above. Every call site in the lifecycle passes it consistently. There are several: the normal completion path, the MODEL_LIMIT_REACHED path, and the recovery/resume paths.

   Acceptance criteria:

   • End-to-end (alongside test_handoff_note_with_stale_last_work_commit_publishes_as_ordinary_incomplete): a work commit W, then two note-only commits, both with last_work_commit = W. The attempt ends with outcome=handoff.
   • End-to-end: a note commit that also changes another file, then a note-only commit, both with last_work_commit = base_revision. The attempt ends with outcome=handoff. A variant with a work commit W before them, naming W, also ends in handoff.
   • Continuation case: the branch carries an earlier round's work commit and note commit. This round commits a new note (once, and also a case with twice) whose last_work_commit is the earlier round's note commit (HEAD at injection). The attempt ends with outcome=handoff, not incomplete.
   • The existing stale-commit test (last_work_commit = "0" * 40) still ends incomplete with the last_work_commit rejection in the comment.
   • A note naming a commit inside the skipped run (for example, the earlier note commit of this attempt) is still rejected, as is any other mismatch.
   • Workspace conformance tests cover the new note-commit reporting, including that earlier rounds' note commits are excluded.
   • The full test suite passes.

   Out of scope:

   • Getting the model to invoke the handoff skill instead of writing the note by hand. That is a separate behavior; a model-written note is already accepted on this path when it validates.
   • Changing the note format, its required fields, or _ALLOWED_HANDOFF_REASONS.
   • Relaxing other validator checks (title/issue/reason), or turning the last_work_commit check into a warning.
   • Rewriting or squashing note commits on the branch.
4. lbacik

   added

   ready-for-agentFully specified implementation issue ready for an unattended agentFully specified implementation issue ready for an unattended agent

   and removed

   needs-triage

   on Oct 12, 2026
5. lbacik

   self-assigned this

   on Oct 12, 2026
6. lbacik commented on Oct 12, 2026

   

   lbacik

   on Oct 12, 2026

   OwnerAuthor

   More actions

   Agent Attempt Result: complete

   Branch: agent/issue-213
   PR: #214

   Evidence

   • Check command: UV_PROJECT_ENVIRONMENT=/home/agent/venv-py311 uv run --python 3.11 --extra dev ruff check && UV_PROJECT_ENVIRONMENT=/home/agent/venv-py311 uv run --python 3.11 --extra dev pytest && UV_PROJECT_ENVIRONMENT=/home/agent/venv-py313 uv run --python 3.13 --extra dev pytest → exit code 0
   • Reviews performed: 1
   • Review repair cycles: 0/2
   • Review findings: not collected

   Details

   Claude SDK completed the implementation workflow.
7. lbacik

   removed

   ready-for-agentFully specified implementation issue ready for an unattended agentFully specified implementation issue ready for an unattended agent

   on Oct 12, 2026
8. lbacik

   removed their assignment

   on Oct 12, 2026
9. lbacik

   closed this as completedin #214on Oct 12, 2026