Skip to content

fix(agent_result): declare blocking-human checkpoints, latest outcome wins - #224

Merged
denniyahh merged 6 commits into
developfrom
bugfix/declared-checkpoint-detection
Oct 10, 2026
Merged

denniyahh merged 6 commits into
developfrom
bugfix/declared-checkpoint-detection

Conversation

@denniyahh

Copy link
Copy Markdown
Owner

Problem

DevFlow decided whether a failed Code stage had stopped at a blocking-human checkpoint by substring-matching the free-text reason of the agent's DEVFLOW_RESULT marker (49-REVIEW CR-01). A confirmed checkpoint resumes the agent's session and tells it to decide the human's gate itself, so ordinary failures that merely mentioned a gate in prose authorized agent decisions meant for a human.

Fix

  • Declared, not guessed. A checkpoint is declared with a structured field on the final marker: "status": "failed", "checkpoint": "blocking-human". The prose heuristic is removed. The field is dropped unless the marker's own status is failed, so the exit-code reconcile cannot promote a success marker into a declaration. An absent field is not serialized.
  • Code stage only. Only CODE_STAGE_POLICY offers the declaration (mid-sentence, so an echoed prompt never parses as one), and checkpoint_confirmed requires Stage::Code for both the field and the Gate: line route.
  • The latest stated outcome wins (operator decision). A DEVFLOW_RESULT marker or a Gate: blocking-human line states an outcome; reading newest first, the first one found decides. This applies to Claude stream result events, envelope text, plain captures, and JSON lines inside a plain capture (arrays newest first; an object's unordered fields resolve to the ordinary outcome when they disagree). The scan reads whole texts, so a long tail after a later marker cannot let an earlier gate win.

AgentResult gains a public checkpoint field, which breaks downstream struct literals. Accepted: 3.0.0 is a major release.

Verification

  • scripts/check.sh all green at the head commit: 1,495 tests, 0 failed, clippy, fmt, gitleaks.
  • Each fix has a deliberate-break control that fails with an assertion (not a hang): reverting the stream rule, removing the Code scope, serializing None, an order-blind scan, gate-only JSON matching, checkpoint outranking an ordinary outcome in one document, folding arrays like objects, last-field-wins, dropping the capture route, and a relaunch that records but never spawns.
  • Four rounds of external adversarial review (Codex, DeepSeek via Pi, Gemini). Round 4: two lanes "safe to merge"; the third lane's critical input did not reproduce as written (checked with a probe); its valid points (array order, test gaps) are fixed in the last commit.

What this does not establish: that no other capture shape is misread. The tests cover the shapes the reviews constructed and their variants.

Known, pre-existing, not addressed here

  • If the agent halts at a gate and its final result event carries no marker, Layer 1 returns nothing and a clean exit with commits advances past Code without consulting the capture scan. Same on develop.
  • A marker torn across lines (a real line break inside it) cannot be parsed, so a Gate: line inside its reason can win. develop is no safer here (any gate line won there); the protocol requires a one-line marker.
  • A marker with list markup in front (- DEVFLOW_RESULT: ...) is not recognised as a marker, while a Gate: line with markup is. develop is no safer here (any gate line won there).

The first skips a human gate rather than auto-deciding one; it is not bounded by the resume ceiling and needs its own fix. The other two are false checkpoints, bounded by the static plan declaration, the resume ceiling, and the checkpoint_auto_decided audit event.

🤖 Generated with Claude Code

https://claude.ai/code/session_0182ocSM2E4fb2zu52jxGLzi

denniyahh and others added 6 commits October 9, 2026 21:30
Replace the reason-prose heuristic from e3f9ddd with an explicit
`checkpoint` field on the DEVFLOW_RESULT marker (49-REVIEW CR-01). The
heuristic auto-resumed ordinary failures whose reason mentioned a gate,
including through "investigate".

- AgentResult gains a leniently parsed `checkpoint` field and
  `declares_blocking_human_checkpoint()` (failed + "blocking-human").
- The GateReview arm confirms a checkpoint from that field on the stage's
  evaluated final result, or from a `Gate:` declaration line as before.
- The completion protocol shows agents the checkpoint form.
- The line matcher is restored to its pre-heuristic form.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182ocSM2E4fb2zu52jxGLzi
Act on the external review of the declared-checkpoint fix:

- A marker's checkpoint is dropped unless the marker itself reports
  failure, so the exit-code reconcile can no longer turn a success that
  named a checkpoint into a declaring failure.
- The pipeline reads the declared field only for the Code stage, the one
  stage that runs plans.
- The declaration sentence moves from the protocol every stage receives to
  the Code-stage policy. The auto-decide resume prompt no longer re-offers
  it, and its example sits mid-sentence, so an echoed prompt cannot parse as
  a declaring marker. It now names a subagent's halt.
- An absent checkpoint is not serialized.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182ocSM2E4fb2zu52jxGLzi
The stream checkpoint check scanned every top-level result with `.any`,
so a gate declared in an earlier turn outranked a later, ordinary failure:
the stage was resumed and told to decide the human's gate. Read results
newest first instead. The first that declares the gate or carries a
DEVFLOW_RESULT marker decides. Silent wake-up turns are still skipped, so
a declaration followed only by them is still reported.

Operator decision 2026-10-10, from the external review of the CR-01 fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182ocSM2E4fb2zu52jxGLzi
The latest stated outcome decides a checkpoint in every capture form:
a DEVFLOW_RESULT marker or a Gate line, read newest first, line by
line, over the whole text. The declaration is offered and accepted
only in the Code stage, a success marker's checkpoint is dropped, and
an absent checkpoint is not serialized.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182ocSM2E4fb2zu52jxGLzi
A JSON line in a capture not classified as a Claude stream matched only
Gate lines inside its strings, so a later ordinary-failure marker was
skipped and an earlier gate won. Each string is now read with the
latest stated outcome, and an ordinary outcome wins within a document.
Adds a Code pipeline test where only the capture scan confirms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182ocSM2E4fb2zu52jxGLzi
An array is ordered, so its elements now follow the latest stated
outcome like lines of text; only an object's unordered fields keep the
ordinary-outcome-wins rule. Tests now assert the noisy fixture takes
the raw-capture path, cover both field orders and both block orders,
and check the gate-line relaunch spawns a monitor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182ocSM2E4fb2zu52jxGLzi

@graphify-labs graphify-labs Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Code graph out of date: Health delta baseline: last indexed commit 71a5192 (diverged from this PR's base — delta is approximate). Health delta baseline was built by an older engine whose node ids do not match this review's — delta is approximate. Impact and hotspot numbers below may be wrong.

No verdict: the code graph is out of date. Static checks found no blocking issues, but they compared against an outdated graph. The next push after the repository re-indexes gets a full review.

Not checked: tests were not run; formal verification proved 0 of 26 changed function(s) (26 not verified).

Formal verification. PR-changed functions: 0/26 verified (0 proven, 0 may-equivalent, 0 distinguished) · 26 not verified (25 unsupported, 1 over cap).

Not verified on this run: advance\_with (unsupported), advance\_with\_declared\_checkpoint\_and\_reported\_gate\_relaunches\_and\_records (unsupported), write\_confirmed\_checkpoint\_capture (unsupported), arm\_a\_ambiguous\_outcome\_gates\_on\_cycle\_one (unsupported), classifier\_fixture (unsupported), external\_verify\_agreement\_advances\_to\_ship (unsupported), external\_verify\_disagreement\_gates\_immediately (unsupported), external\_verify\_no\_verdict\_gates\_immediately (unsupported), non\_success\_status\_never\_classifies\_as\_passed\_even\_with\_verdict\_pass (unsupported), antigravity\_stream\_envelope\_failure (unsupported), and 15 more; 1 changed function(s) beyond the cap and 0 skipped when the time budget ran out.


Graphify review — findings

Tightens unattended checkpoint auto-resume in advance_with so it fires only for Claude at the Code stage. The agent must also explicitly report a blocking-human checkpoint. That means "checkpoint": "blocking-human" on the final DEVFLOW_RESULT marker via declares_blocking_human_checkpoint, or a trailing Gate: line with no later stated outcome. A failure reason that merely describes a blocking-human halt in prose now takes the ordinary review gate instead of relaunching the session.

Review partial — this diff was larger than one review pass covers, so later files were not reviewed; some findings may be missing.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 818 functions depend on the 631 functions this change touches.

Health — this change adds coupling hotspots:

  • new: handle_validate_outcome() — 31 callers, 8 callees
  • new: evaluate_layer1() — 21 callers, 11 callees
  • new: run_pipe_owning_monitor() — 11 callers, 13 callees
  • new: evaluate_agent_result_inner() — 20 callers, 6 callees
  • new: advance_with() — 6 callers, 19 callees
  • new: resume() — 15 callers, 7 callees
  • new: parse_claude_event_result() — 11 callers, 9 callees
  • new: parse_opencode_event_result() — 13 callers, 6 callees
  • …and 67 more — each is listed as a finding

Verification — 818 functions in the blast radius were not formally verified this run (proofs are advisory here).

Health delta baseline: last indexed commit 71a5192 (diverged from this PR's base — delta is approximate).

Health delta baseline was built by an older engine whose node ids do not match this review's — delta is approximate.

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 791 function(s) in the blast radius were not formally verified this run

Formal verification

Could not verify: Could not verify advance\_with.

The verifier did not have enough to check advance\_with, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unsupported return type: Result<(),CliError>

Could not verify: Could not verify advance\_with\_declared\_checkpoint\_and\_reported\_gate\_relaunches\_and\_records.

The verifier did not have enough to check advance\_with\_declared\_checkpoint\_and\_reported\_gate\_relaunches\_and\_records, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unit-returning functions have no value to compare

Could not verify: Could not verify write\_confirmed\_checkpoint\_capture.

The verifier did not have enough to check write\_confirmed\_checkpoint\_capture, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unit-returning functions have no value to compare

Could not verify: Could not verify arm\_a\_ambiguous\_outcome\_gates\_on\_cycle\_one.

The verifier did not have enough to check arm\_a\_ambiguous\_outcome\_gates\_on\_cycle\_one, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unit-returning functions have no value to compare

Could not verify: Could not verify classifier\_fixture.

The verifier did not have enough to check classifier\_fixture, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unsupported return type: agent_result::AgentResult

Could not verify: Could not verify external\_verify\_agreement\_advances\_to\_ship.

The verifier did not have enough to check external\_verify\_agreement\_advances\_to\_ship, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unit-returning functions have no value to compare

Could not verify: Could not verify external\_verify\_disagreement\_gates\_immediately.

The verifier did not have enough to check external\_verify\_disagreement\_gates\_immediately, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unit-returning functions have no value to compare

Could not verify: Could not verify external\_verify\_no\_verdict\_gates\_immediately.

The verifier did not have enough to check external\_verify\_no\_verdict\_gates\_immediately, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unit-returning functions have no value to compare

Could not verify: Could not verify non\_success\_status\_never\_classifies\_as\_passed\_even\_with\_verdict\_pass.

The verifier did not have enough to check non\_success\_status\_never\_classifies\_as\_passed\_even\_with\_verdict\_pass, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unit-returning functions have no value to compare

Could not verify: Could not verify antigravity\_stream\_envelope\_failure.

The verifier did not have enough to check antigravity\_stream\_envelope\_failure, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unsupported return type: Option<AgentResult>

Could not verify: Could not verify blocking\_human\_checkpoint\_reported.

The verifier did not have enough to check blocking\_human\_checkpoint\_reported, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: rustc failed: error[E0425]: cannot find value `text_reports_human_gate` in this scope

Could not verify: Could not verify blocking\_human\_checkpoint\_reported\_true\_when\_only\_first\_result\_declares\_gate.

The verifier did not have enough to check blocking\_human\_checkpoint\_reported\_true\_when\_only\_first\_result\_declares\_gate, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unit-returning functions have no value to compare

Could not verify: Could not verify capture\_reports\_human\_gate.

The verifier did not have enough to check capture\_reports\_human\_gate, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: rustc failed: error[E0433]: failed to resolve: use of undeclared crate or module `serde_json`

Could not verify: Could not verify claude\_stream\_envelope\_failure.

The verifier did not have enough to check claude\_stream\_envelope\_failure, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unsupported return type: Option<AgentResult>

Could not verify: Could not verify claude\_stream\_reports\_human\_gate.

The verifier did not have enough to check claude\_stream\_reports\_human\_gate, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unsupported parameter type(s)

Could not verify: Could not verify detect\_claude\_envelope\_failure.

The verifier did not have enough to check detect\_claude\_envelope\_failure, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unsupported return type: Option<AgentResult>

Could not verify: Could not verify evaluate\_layer0.

The verifier did not have enough to check evaluate\_layer0, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unsupported return type: Option<AgentResult>

Could not verify: Could not verify evaluate\_layer2.

The verifier did not have enough to check evaluate\_layer2, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unsupported return type: Result<Option<AgentResult>,ResultError>

Could not verify: Could not verify evaluate\_layer3.

The verifier did not have enough to check evaluate\_layer3, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unsupported return type: Result<AgentResult,ResultError>

Could not verify: Could not verify idle\_timeout\_result.

The verifier did not have enough to check idle\_timeout\_result, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unsupported return type: AgentResult

Could not verify: Could not verify indeterminate\_capture\_failure.

The verifier did not have enough to check indeterminate\_capture\_failure, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unsupported return type: AgentResult

Could not verify: Could not verify parse\_codex\_event\_result.

The verifier did not have enough to check parse\_codex\_event\_result, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unsupported return type: Option<AgentResult>

Could not verify: Could not verify parse\_marker\_lines.

The verifier did not have enough to check parse\_marker\_lines, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unsupported return type: Option<AgentResult>

Could not verify: Could not verify parse\_opencode\_event\_result.

The verifier did not have enough to check parse\_opencode\_event\_result, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unsupported return type: Option<AgentResult>

Could not verify: Could not verify rate\_limited\_result.

The verifier did not have enough to check rate\_limited\_result, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unsupported return type: AgentResult

· 1 grounded finding(s) anchored inline below; 74 more finding(s) on lines outside this diff (see the check run).

/// The declared marker field alone confirms the checkpoint: same route and
/// audit trail as a reported `Gate:` line (49-REVIEW CR-01).
#[test]
fn advance_with_declared_checkpoint_and_marker_field_relaunches_and_records() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression — advance_with_declared_checkpoint_and_marker_field_relaunches_and_records()

fans out to 6 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

@denniyahh
denniyahh merged commit 358274c into develop Oct 10, 2026
14 checks passed
@denniyahh
denniyahh deleted the bugfix/declared-checkpoint-detection branch October 10, 2026 13:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant