Repository navigation
fix(agent_result): declare blocking-human checkpoints, latest outcome wins - #224
Conversation
Replace the reason-prose heuristic from e3f9ddd with an explicit `checkpoint` field on the DEVFLOW_RESULT marker (49-REVIEW CR-01). The heuristic auto-resumed ordinary failures whose reason mentioned a gate, including through "investigate". - AgentResult gains a leniently parsed `checkpoint` field and `declares_blocking_human_checkpoint()` (failed + "blocking-human"). - The GateReview arm confirms a checkpoint from that field on the stage's evaluated final result, or from a `Gate:` declaration line as before. - The completion protocol shows agents the checkpoint form. - The line matcher is restored to its pre-heuristic form. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0182ocSM2E4fb2zu52jxGLzi
Act on the external review of the declared-checkpoint fix: - A marker's checkpoint is dropped unless the marker itself reports failure, so the exit-code reconcile can no longer turn a success that named a checkpoint into a declaring failure. - The pipeline reads the declared field only for the Code stage, the one stage that runs plans. - The declaration sentence moves from the protocol every stage receives to the Code-stage policy. The auto-decide resume prompt no longer re-offers it, and its example sits mid-sentence, so an echoed prompt cannot parse as a declaring marker. It now names a subagent's halt. - An absent checkpoint is not serialized. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0182ocSM2E4fb2zu52jxGLzi
The stream checkpoint check scanned every top-level result with `.any`, so a gate declared in an earlier turn outranked a later, ordinary failure: the stage was resumed and told to decide the human's gate. Read results newest first instead. The first that declares the gate or carries a DEVFLOW_RESULT marker decides. Silent wake-up turns are still skipped, so a declaration followed only by them is still reported. Operator decision 2026-10-10, from the external review of the CR-01 fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0182ocSM2E4fb2zu52jxGLzi
The latest stated outcome decides a checkpoint in every capture form: a DEVFLOW_RESULT marker or a Gate line, read newest first, line by line, over the whole text. The declaration is offered and accepted only in the Code stage, a success marker's checkpoint is dropped, and an absent checkpoint is not serialized. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0182ocSM2E4fb2zu52jxGLzi
A JSON line in a capture not classified as a Claude stream matched only Gate lines inside its strings, so a later ordinary-failure marker was skipped and an earlier gate won. Each string is now read with the latest stated outcome, and an ordinary outcome wins within a document. Adds a Code pipeline test where only the capture scan confirms. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0182ocSM2E4fb2zu52jxGLzi
An array is ordered, so its elements now follow the latest stated outcome like lines of text; only an object's unordered fields keep the ordinary-outcome-wins rule. Tests now assert the noisy fixture takes the raw-capture path, cover both field orders and both block orders, and check the gate-line relaunch spawns a monitor. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0182ocSM2E4fb2zu52jxGLzi
There was a problem hiding this comment.
Graphify reviewed this change.
Code graph out of date: Health delta baseline: last indexed commit 71a5192 (diverged from this PR's base — delta is approximate). Health delta baseline was built by an older engine whose node ids do not match this review's — delta is approximate. Impact and hotspot numbers below may be wrong.
No verdict: the code graph is out of date. Static checks found no blocking issues, but they compared against an outdated graph. The next push after the repository re-indexes gets a full review.
Not checked: tests were not run; formal verification proved 0 of 26 changed function(s) (26 not verified).
Formal verification. PR-changed functions: 0/26 verified (0 proven, 0 may-equivalent, 0 distinguished) · 26 not verified (25 unsupported, 1 over cap).
Not verified on this run: advance\_with (unsupported), advance\_with\_declared\_checkpoint\_and\_reported\_gate\_relaunches\_and\_records (unsupported), write\_confirmed\_checkpoint\_capture (unsupported), arm\_a\_ambiguous\_outcome\_gates\_on\_cycle\_one (unsupported), classifier\_fixture (unsupported), external\_verify\_agreement\_advances\_to\_ship (unsupported), external\_verify\_disagreement\_gates\_immediately (unsupported), external\_verify\_no\_verdict\_gates\_immediately (unsupported), non\_success\_status\_never\_classifies\_as\_passed\_even\_with\_verdict\_pass (unsupported), antigravity\_stream\_envelope\_failure (unsupported), and 15 more; 1 changed function(s) beyond the cap and 0 skipped when the time budget ran out.
Graphify review — findings
Tightens unattended checkpoint auto-resume in advance_with so it fires only for Claude at the Code stage. The agent must also explicitly report a blocking-human checkpoint. That means "checkpoint": "blocking-human" on the final DEVFLOW_RESULT marker via declares_blocking_human_checkpoint, or a trailing Gate: line with no later stated outcome. A failure reason that merely describes a blocking-human halt in prose now takes the ordinary review gate instead of relaunching the session.
Review partial — this diff was larger than one review pass covers, so later files were not reviewed; some findings may be missing.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 818 functions depend on the 631 functions this change touches.
Health — this change adds coupling hotspots:
- new:
handle_validate_outcome()— 31 callers, 8 callees - new:
evaluate_layer1()— 21 callers, 11 callees - new:
run_pipe_owning_monitor()— 11 callers, 13 callees - new:
evaluate_agent_result_inner()— 20 callers, 6 callees - new:
advance_with()— 6 callers, 19 callees - new:
resume()— 15 callers, 7 callees - new:
parse_claude_event_result()— 11 callers, 9 callees - new:
parse_opencode_event_result()— 13 callers, 6 callees - …and 67 more — each is listed as a finding
Verification — 818 functions in the blast radius were not formally verified this run (proofs are advisory here).
Health delta baseline: last indexed commit 71a5192 (diverged from this PR's base — delta is approximate).
Health delta baseline was built by an older engine whose node ids do not match this review's — delta is approximate.
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 791 function(s) in the blast radius were not formally verified this run
Formal verification
Could not verify: Could not verify advance\_with.
The verifier did not have enough to check advance\_with, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unsupported return type: Result<(),CliError>
Could not verify: Could not verify advance\_with\_declared\_checkpoint\_and\_reported\_gate\_relaunches\_and\_records.
The verifier did not have enough to check advance\_with\_declared\_checkpoint\_and\_reported\_gate\_relaunches\_and\_records, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unit-returning functions have no value to compare
Could not verify: Could not verify write\_confirmed\_checkpoint\_capture.
The verifier did not have enough to check write\_confirmed\_checkpoint\_capture, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unit-returning functions have no value to compare
Could not verify: Could not verify arm\_a\_ambiguous\_outcome\_gates\_on\_cycle\_one.
The verifier did not have enough to check arm\_a\_ambiguous\_outcome\_gates\_on\_cycle\_one, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unit-returning functions have no value to compare
Could not verify: Could not verify classifier\_fixture.
The verifier did not have enough to check classifier\_fixture, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unsupported return type: agent_result::AgentResult
Could not verify: Could not verify external\_verify\_agreement\_advances\_to\_ship.
The verifier did not have enough to check external\_verify\_agreement\_advances\_to\_ship, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unit-returning functions have no value to compare
Could not verify: Could not verify external\_verify\_disagreement\_gates\_immediately.
The verifier did not have enough to check external\_verify\_disagreement\_gates\_immediately, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unit-returning functions have no value to compare
Could not verify: Could not verify external\_verify\_no\_verdict\_gates\_immediately.
The verifier did not have enough to check external\_verify\_no\_verdict\_gates\_immediately, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unit-returning functions have no value to compare
Could not verify: Could not verify non\_success\_status\_never\_classifies\_as\_passed\_even\_with\_verdict\_pass.
The verifier did not have enough to check non\_success\_status\_never\_classifies\_as\_passed\_even\_with\_verdict\_pass, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unit-returning functions have no value to compare
Could not verify: Could not verify antigravity\_stream\_envelope\_failure.
The verifier did not have enough to check antigravity\_stream\_envelope\_failure, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unsupported return type: Option<AgentResult>
Could not verify: Could not verify blocking\_human\_checkpoint\_reported.
The verifier did not have enough to check blocking\_human\_checkpoint\_reported, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: rustc failed: error[E0425]: cannot find value `text_reports_human_gate` in this scope
Could not verify: Could not verify blocking\_human\_checkpoint\_reported\_true\_when\_only\_first\_result\_declares\_gate.
The verifier did not have enough to check blocking\_human\_checkpoint\_reported\_true\_when\_only\_first\_result\_declares\_gate, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unit-returning functions have no value to compare
Could not verify: Could not verify capture\_reports\_human\_gate.
The verifier did not have enough to check capture\_reports\_human\_gate, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: rustc failed: error[E0433]: failed to resolve: use of undeclared crate or module `serde_json`
Could not verify: Could not verify claude\_stream\_envelope\_failure.
The verifier did not have enough to check claude\_stream\_envelope\_failure, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unsupported return type: Option<AgentResult>
Could not verify: Could not verify claude\_stream\_reports\_human\_gate.
The verifier did not have enough to check claude\_stream\_reports\_human\_gate, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unsupported parameter type(s)
Could not verify: Could not verify detect\_claude\_envelope\_failure.
The verifier did not have enough to check detect\_claude\_envelope\_failure, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unsupported return type: Option<AgentResult>
Could not verify: Could not verify evaluate\_layer0.
The verifier did not have enough to check evaluate\_layer0, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unsupported return type: Option<AgentResult>
Could not verify: Could not verify evaluate\_layer2.
The verifier did not have enough to check evaluate\_layer2, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unsupported return type: Result<Option<AgentResult>,ResultError>
Could not verify: Could not verify evaluate\_layer3.
The verifier did not have enough to check evaluate\_layer3, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unsupported return type: Result<AgentResult,ResultError>
Could not verify: Could not verify idle\_timeout\_result.
The verifier did not have enough to check idle\_timeout\_result, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unsupported return type: AgentResult
Could not verify: Could not verify indeterminate\_capture\_failure.
The verifier did not have enough to check indeterminate\_capture\_failure, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unsupported return type: AgentResult
Could not verify: Could not verify parse\_codex\_event\_result.
The verifier did not have enough to check parse\_codex\_event\_result, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unsupported return type: Option<AgentResult>
Could not verify: Could not verify parse\_marker\_lines.
The verifier did not have enough to check parse\_marker\_lines, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unsupported return type: Option<AgentResult>
Could not verify: Could not verify parse\_opencode\_event\_result.
The verifier did not have enough to check parse\_opencode\_event\_result, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unsupported return type: Option<AgentResult>
Could not verify: Could not verify rate\_limited\_result.
The verifier did not have enough to check rate\_limited\_result, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unsupported return type: AgentResult
· 1 grounded finding(s) anchored inline below; 74 more finding(s) on lines outside this diff (see the check run).
| /// The declared marker field alone confirms the checkpoint: same route and | ||
| /// audit trail as a reported `Gate:` line (49-REVIEW CR-01). | ||
| #[test] | ||
| fn advance_with_declared_checkpoint_and_marker_field_relaunches_and_records() { |
There was a problem hiding this comment.
advance_with_declared_checkpoint_and_marker_field_relaunches_and_records()
fans out to 6 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
Problem
DevFlow decided whether a failed Code stage had stopped at a
blocking-humancheckpoint by substring-matching the free-textreasonof the agent'sDEVFLOW_RESULTmarker (49-REVIEW CR-01). A confirmed checkpoint resumes the agent's session and tells it to decide the human's gate itself, so ordinary failures that merely mentioned a gate in prose authorized agent decisions meant for a human.Fix
"status": "failed", "checkpoint": "blocking-human". The prose heuristic is removed. The field is dropped unless the marker's own status isfailed, so the exit-code reconcile cannot promote a success marker into a declaration. An absent field is not serialized.CODE_STAGE_POLICYoffers the declaration (mid-sentence, so an echoed prompt never parses as one), andcheckpoint_confirmedrequiresStage::Codefor both the field and theGate:line route.DEVFLOW_RESULTmarker or aGate: blocking-humanline states an outcome; reading newest first, the first one found decides. This applies to Claude streamresultevents, envelope text, plain captures, and JSON lines inside a plain capture (arrays newest first; an object's unordered fields resolve to the ordinary outcome when they disagree). The scan reads whole texts, so a long tail after a later marker cannot let an earlier gate win.AgentResultgains a publiccheckpointfield, which breaks downstream struct literals. Accepted: 3.0.0 is a major release.Verification
scripts/check.sh allgreen at the head commit: 1,495 tests, 0 failed, clippy, fmt, gitleaks.None, an order-blind scan, gate-only JSON matching, checkpoint outranking an ordinary outcome in one document, folding arrays like objects, last-field-wins, dropping the capture route, and a relaunch that records but never spawns.What this does not establish: that no other capture shape is misread. The tests cover the shapes the reviews constructed and their variants.
Known, pre-existing, not addressed here
develop.Gate:line inside its reason can win.developis no safer here (any gate line won there); the protocol requires a one-line marker.- DEVFLOW_RESULT: ...) is not recognised as a marker, while aGate:line with markup is.developis no safer here (any gate line won there).The first skips a human gate rather than auto-deciding one; it is not bounded by the resume ceiling and needs its own fix. The other two are false checkpoints, bounded by the static plan declaration, the resume ceiling, and the
checkpoint_auto_decidedaudit event.🤖 Generated with Claude Code
https://claude.ai/code/session_0182ocSM2E4fb2zu52jxGLzi