Skip to content

Latest commit

 

History

1 Commit

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 

Repository files navigation

agent-delegate-review

A loop for handing real tickets to cheap coding agents — and catching what they get wrong before it merges.

You run an expensive, careful model as the reviewer. Cheap, fast agents do the implementation, isolated from each other, in parallel. Nothing merges until the reviewer has reproduced every claim from a primary source.

The one rule everything else hangs off:

Never grade an agent on its own report. Agents rarely lie. They misread their own evidence — and a confident summary launders a wrong reading into a fact.


Does it actually work? Here is the measurement.

One real session, a large Android/Kotlin-Multiplatform monorepo, three tickets from the same epic. Full method, timings and confounds: docs/06-measurements.md.

Throughput

Baseline (same epic, prior single-agent sessions, recorded before this run) 60–84 min per ticket
This run — 3 tickets, first brief → last merge 91 min total
Two parallel tickets, briefs → both merged 45 min wall clock
Implied speed-up vs the sequential baseline ~2.0×–2.8×

⚠️ This is one session, n=3, not a controlled experiment — and the tickets were selected for being atomic and compile-verifiable. That selection is part of the method, but it means this is not a like-for-like comparison against a random backlog item. Read the confounds before quoting the number.

The part that matters more: what review caught

Review rounds across 3 tickets 7
Blocking findings 4
Findings that appeared in the agent's own report as complete and passing 4 of 4
Findings that would have shipped real damage if merged 2

Every single blocking finding was inside work an agent had already declared finished, with green gates attached. The unreviewed version of this loop ships all four.

What the four were, in shape (details in docs/04-review-checklist.md):

  1. Scope creep that weakened security. A typography-tokenisation ticket also stripped internal from a tap-jacking guard function — widening a module's public API, which on Kotlin/Native reaches the exported Swift surface. It was item 5 of 5 in the agent's own diff summary.
  2. A green gate built on a pre-existing red build. That same change existed only to make a gate pass; the build had been broken on the base branch for weeks. Silently fixing it converted visible breakage into an invisible API change.
  3. Tests that certified the bug. A ticket correctly introduced a two-value enum to separate two error conditions, then populated it from which function received the error rather than from the error itself — so one backend code meant opposite things on two paths. The new tests asserted the conflation as intended behaviour.
  4. A team-documentation entry with inverted attribution, which erased the very asymmetry the document existed to teach.

Safety

Concurrent agents on one machine 2
Gradle builds run 20
OOM / resource-contention failures 0

The prior recorded incident on this machine — two unlocked concurrent builds — cost a 26-minute build and produced a 6.4 GB heap dump. Worktrees isolate files; a machine-wide build lock isolates RAM. You need both. See docs/02-isolation.md.


The loop

1. Pick tickets that are actually delegable        docs/01-choosing-tickets.md
2. Verify the ticket's claims YOURSELF first       docs/01-choosing-tickets.md#ground-truth
3. One git worktree + pre-made branch per agent    docs/02-isolation.md
4. Write the brief                                 docs/03-briefing.md
5. Agent implements, stops before merge
6. Review adversarially from primary sources       docs/04-review-checklist.md
7. Passed → the AGENT merges. Then clean up.       docs/05-merge-and-cleanup.md

Start here: docs/00-quickstart.md — 10 minutes to a first delegated ticket.


Why the reviewer, not the agent, is the product

Most "AI agent" setups optimise the wrong half. They spend on making the implementer smarter and accept its self-report. This loop assumes the implementer will be wrong in specific, recurring ways, and spends the expensive model on detecting that.

That buys you three things a smarter implementer alone does not:

  • Cheap parallelism. If the reviewer is the quality gate, the implementers can be small, fast models — and you can run several at once.
  • Failures that are findable. The checklist targets a fixed set of shapes (fake-green gates, scope creep, unfalsifiable guards, tests that lock in the defect). These recur across models and codebases.
  • An honest record. The verdict lands as a PR comment with evidence, so the reasoning survives the session.

It is not free. Budget roughly one reviewer-round per ticket beyond the first pass — in this run, 3 tickets needed 7 rounds total.


Requirements

  • git ≥ 2.5 (worktrees), tmux
  • Any agent CLI that runs in a terminal — this was validated with Gemini/Antigravity CLI as the implementer and Claude Opus as the reviewer; the loop is agent-agnostic
  • A repo with a runbook the agent can follow end-to-end (branch → implement → gates → PR). See docs/03-briefing.md.

Not a fit when

  • The ticket needs a human decision first (product/design), or carries an unresolved blocker
  • The change spans dozens of files — blast radius too large to review credibly
  • You will not actually do the review. Without step 6 this is worse than not delegating: you get confident, green-looking, unverified changes at higher volume.

License

MIT — see LICENSE.

About

Hand real tickets to cheap coding agents, then catch what they get wrong before it merges. Worktree isolation, adversarial review checklist, and the measurements from a real run.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages