Skip to content

fix: retire orphaned reports after a landing, not before - #16

Open
kim-em wants to merge 4 commits into
mainfrom
fix/sweep-orphans-after-landing
Open

kim-em wants to merge 4 commits into
mainfrom
fix/sweep-orphans-after-landing

Conversation

@kim-em

@kim-em kim-em commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

This PR retires an area's unmergeable reports from the merge workflow, once the compare-and-swap has already chosen a winner, and stops the publishing path closing anything at all.

An area can hold several open reports and only one can land. The gate requires a byte-exact append at the cursor recorded in PROGRESS.md, so the moment one lands and the cursor moves, every other open report for that area is unmergeable for good. Nothing retired them, so they accumulated: one per area per round, never shed.

The rule is now the one that needs no prediction: a report is retired only when committed PROGRESS.md on main shows the cursor its window starts at already appended at and moved past. The branch name's seven-character prefix only nominates candidates; the decision compares full SHAs, taking the report's starting SHA from the section its own head appends, and keeps the report whenever that cannot be read. No window comparison, no pull-request metadata, no ordering. Nothing is retired when the area exists under both TauCetiRoadmap/ and Completed/, or when a failed lookup leaves that unknown, since report branches do not record their parent.

Retiring beforehand, on the grounds that another open report covers more of the window, is wrong in three ways that each cost real work. The favoured report may fail its build, leaving nothing landed and the other closed-unmerged, which apply treats as permanently refused and no later run repairs. The close races this very workflow, which reads a pull request's state when it collects and trusts that snapshot until it writes. And "covers more" has to be read from a pull request body, which anyone may edit, so a stale or forged one retires a report that was alive. Waiting for the ref update removes all three at once, because there is then nothing left to predict.

Ownership is deliberately not consulted, unlike in apply. There the question is whose work may be superseded and the answer has to be "only our own", or a stranger can be vetoed. Here the report is unmergeable for its author as much as for anyone, and leaving it open marks the area in flight against them.

The merge job gains a checkout of the validator at the ref it was already pinned to, since it is the first thing in that job to run code from this repository rather than call gh. The retiring rules are therefore reviewed alongside the gate, not separately.

This also re-reads the pull request immediately before the compare-and-swap, which narrows the window in which a pull request closed mid-run still lands. It does not eliminate it: a close between that read and the ref update is not seen. It reads to refuse and never to authorise: the tree and the parent still come from the two pinned SHAs, so nothing read there can cause a landing, only prevent one. Three attempts, because the distinction that matters is between an answer saying the pull request moved, which is a considered non-landing and exits clean, and no answer at all, which is infrastructure failing and exits non-zero. Conflating those would let a single timeout report a green run that landed nothing, after which the planner finds the pull request still open and in flight and never retriggers it.

Supersedes #15.

🤖 Prepared with Claude Code

Kim Morrison added 3 commits September 21, 2026 10:24
An area can hold several open reports and only one can land: the gate requires a byte-exact append
at the cursor in `PROGRESS.md`, so once `main` moves, every other open report for that area is
unmergeable for good. Nothing retired them, so they accumulated one per area per round.

Retire them from the merge workflow, after the compare-and-swap has already chosen a winner, on the
one rule that needs no prediction: the window does not start at the live cursor. That is decidable
from the branch name and the cursor alone.

Doing it beforehand, on the grounds that another open report covers more of the window, is wrong in
three ways that each cost real work. The favoured report may fail its build, leaving nothing landed
and the other closed-unmerged, which `apply` treats as permanently refused. The close races the
merge workflow, which reads a pull request's state when it collects and trusts that snapshot until
it writes. And "covers more" has to be read from a pull request body, which anyone may edit, so a
stale or forged one retires a live report. Waiting for the ref update removes all three.

The publishing path therefore closes nothing at all, and the merge workflow gains a checkout of the
validator at the ref it was already pinned to, so the retiring rules are reviewed alongside the gate
rather than separately.

Also re-read the pull request immediately before the compare-and-swap, so that closing one stops it
rather than usually stopping it. Three attempts: an answer that says the pull request moved is a
considered non-landing and exits clean, while no answer at all is infrastructure failing and exits
non-zero. Conflating those would let one timeout report a green run that landed nothing, after
which the planner sees the pull request still open and in flight and never retriggers it.

🤖 Prepared with Claude Code
`TauCetiRoadmap/<area>` and `Completed/<area>` are different roadmaps whose cursors are different
values in different files, which is why the collector derives the parent from the changed paths
rather than probing in a fixed order. The sweep hard-coded `TauCetiRoadmap/`, so a report landing
under `Completed/` read the active roadmap's cursor, or none at all when no active roadmap of that
name exists.

Carry the validated parent through the bundle and the workflow alongside the area, and build the
path from it.

Report branches are `progress/<from7>-<to7>/<Area>` and record no parent, so when a name exists
under both, the open reports cannot be attributed to one roadmap or the other. Retire nothing in
that case and say so: leaving a few orphans for a human beats discarding another roadmap's live
work on the cursor we happen to be holding.

🤖 Prepared with Claude Code
…cleanup

Retiring on "its cursor is not the current one" reads a mutable snapshot and treats every
disagreement alike, so a stale contents response retires a report that starts *ahead* of it -- the
live one. Retire instead on positive evidence from committed history: the log shows that cursor
already appended at and moved past. Reading less history shrinks the evidence, which can only retire
fewer reports, so staleness fails in the safe direction.

Seven hex characters are not a commit, so a prefix matching both a spent cursor and the live one
proves nothing and the report is kept.

Match the gate's exact branch grammar and require the base branch a report must target. The looser
parser would accept `progress/nothex-whatever/Area`, which is somebody's ordinary pull request;
failing an automated gate is not a reason to close a human's work.

Thread the repository through to the close, rather than reading one repository's pull request
numbers and closing another's by the same number.

List open pull requests by pagination rather than a capped fetch that filters afterwards. The cap
was a starvation lever: enough newer pull requests of any kind, needing neither to merge nor to be
plausible, push an area's stranded reports out of the window indefinitely.

Move the cleanup into its own job. It does not need the bypass credential and must not hold it --
retiring a report is an ordinary `pull-requests: write` operation, while the App token exists to
write to a protected branch. The separate job also stops a failing checkout reddening a merge whose
content is already on `main`, which no `|| true` on a later step could catch, and replaces a
step-level `if:` whose implicit `success()` would have skipped cleanup after any earlier hiccup.

🤖 Prepared with Claude Code
@roed-math

Copy link
Copy Markdown
Contributor

I did a fresh pass over the current head (fe07e572). The overall redesign looks substantially safer than the pre-landing superseding logic, but I think there are two correctness issues worth fixing before merge, plus one wording/TOCTOU point.

  1. Sibling-parent lookup should fail closed on API errors.
    sweep_area correctly avoids retiring anything if the same area exists under both TauCetiRoadmap/ and Completed/, since the branch name does not encode the parent. But gh.file_on_default_branch currently returns None both for “file does not exist” and for any GhError after retries. So if the sibling really exists but that read times out / rate-limits / 5xx's, the code treats it as absent and proceeds to retire based on the other parent’s history. That defeats the safety check and could close a live report from the sibling roadmap.

    I would distinguish 404/nonexistent from “could not determine”; the latter should abort this cleanup without closing anything. Please add a regression test where the sibling lookup raises GhError and verify that close_orphans is never called.

  2. Stale history + a 7-hex collision can still retire a live report.
    The new test for a prefix matching both a spent cursor and the snapshot’s live cursor is good, but there is another case. Suppose the stale log is A -> B, while actual history has advanced B -> C, and C[:7] == A[:7]. A genuinely live report starting at C is evaluated against the stale snapshot: it does not match stale-live B, but it does match spent A, so it gets retired. Thus the “staleness can only retire fewer reports” argument is not strictly true after projecting full SHAs to 7-character prefixes.

    This is low probability, but destructive cleanup should not depend on a 28-bit prefix being globally unique. I’d add a regression test combining staleness with an unseen live-prefix collision, and make the destructive decision use/verify the full starting SHA rather than only from7.

  3. Minor wording point: the late PR re-read reduces but cannot eliminate the close-vs-land race. There is still a TOCTOU window between the final gh api .../pulls/$PR read and the ref PATCH; a human can close/retarget in that interval and the CAS can still land if main has not moved. The automatic-retirement race is much safer now because retirement happens only after another landing has advanced main, but comments saying “closing it stopped it” is now simply true are a little stronger than the implementation guarantees.

Aside from those, I like the direction here: retiring only after the CAS, removing mutable PR-body metadata from the destructive decision, separating cleanup from the bypass token, carrying the parent through validation, and removing the capped PR listing are all improvements. CI is green on the current head.

Prepared with GPT 6.

`file_on_default_branch` now returns None only for a 404 and raises on
any other failure, so the sweep retires nothing when it cannot tell
whether the area also exists under the other parent.

A report is retired only when the full `from_sha` of the section its
own head appends is a spent cursor; the seven-character branch prefix
merely nominates candidates. This keeps a live report whose start shares
a prefix with a spent cursor that a stale log still shows.

The late pull-request re-read is described as narrowing the close-vs-land
window, not closing it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@kim-em
kim-em requested a review from a team as a code owner September 25, 2026 06:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants