Skip to content

refactor!: simplify automation for v4 - #62

Merged
edbfi merged 11 commits into
mainfrom
refactor/v4-simplify
Sep 25, 2026
Merged

edbfi merged 11 commits into
mainfrom
refactor/v4-simplify

Conversation

@edbfi

@edbfi edbfi commented Sep 24, 2026 •

Copy link
Copy Markdown
Owner

Implementation of automation v4: one complete, breaking release that removes the repair dispatch path and the machinery built around it, moves automerge to GitHub auto-merge, and adds a completion audit for the fleet. Nothing here changes consumers until the v4 tag is published and their migration PRs merge.

What changes

  • Repair no longer dispatches CI (0c0b3d3). App-token pushes already trigger PR CI and policy. On real repairs in setun, zondarr and poyo-studio, every App-era repair got a normal pull_request run and a duplicate dispatched run. ci-workflow and actions: write are gone.
  • Removed (c558380): actions/ci-recovery (it ran about 2,200 times in 7 days and dispatched twice), actions/dispatch-guard, repair-recovery.yml, .github/repair-policy.json, and the go/rust/content workflows (0 consumers).
  • automerge.json (c0695ef): platformAutomerge: true, rebase, and no wildcard rule, so renovate-config updates stay manual and opt-outs win. default.json no longer sets the merge mechanism.
  • Fleet audit (fe50cbe): scripts/fleet_audit.py + scripts/fleet.json + scripts/renovate_effective.mjs. Read-only. Unknown evidence never counts as compliant, and the audit exits 0 only when every declared repo is compliant or excepted. replex is deliberately excluded.
  • Docs (4c9ce84): README is now a short reference with the v4 migration checklist; the merge model, eligibility rule and pilot evidence are in docs/renovate-migration.md; AGENTS.md is the real file and CLAUDE.md imports it.
  • Policy refreshes its own stale verdicts (d303390). GitHub keeps one policy verdict per triggering event, so a withdrawn objection could leave an old failure blocking. After a pass, policy re-runs the other event's older failed run for the same head. It is bounded and can't loop (failing runs never re-run anything, newer failures are kept, attempts are capped at 3), and it was verified live on test(ci): pilot the v4 stale-verdict refresh guides#68. Callers grant actions: write. Goal: no human step anywhere in the normal flow.
  • Test fix (01450fc): the prek hook test failed locally, on main too, because its fixture commit relied on git guessing an identity.

Evidence: guides pilot (edbfi/guides#58)

With GitHub auto-merge armed and no admin bypass, these all blocked the merge: failed, skipped, cancelled, pending and missing required checks; a head changed after green; a branch behind main; a hold added while CI was still running. On a real Renovate PR (edbfi/guides#67), Renovate armed auto-merge itself, a changes-requested review and a review request blocked it, and it merged 76 s after going green. The previous median wait from green to merge was 219 min.

Finding: policy verdicts are kept per triggering event, so a later success from one event doesn't replace an earlier failure from the other. d303390 fixes this automatically.

Before release

  • Two more real Renovate PRs auto-merge in guides (currently 1/3)
  • Release commit: point pr-policy.yml, biome-repair.yml, the tests/workflows.mjs assertion and renovate.json at v4.0.0
  • Merge, then tag v4.0.0 and publish the release notes with the migration checklist

Local validation

bun run test (86 Python tests, renovate.mjs, workflows.mjs), bun run validate:renovate (strict, 4 files), actionlint: all pass.

@edbfi

edbfi commented Sep 24, 2026

Copy link
Copy Markdown
Owner Author

Live test: stale-verdict refresh (edbfi/guides#68, 2026-09-24)

A throwaway guides PR called this branch's policy action, pinned to d303390, with actions: write. It never merged.

Step Event Policy
Hold added pull_request failure
Comment review while held pull_request_review failure
Hold removed pull_request success. Log: Re-ran stale failed policy run 36018364827 for the same head
The re-run pull_request_review, attempt 2 success
60 s later none review run still at attempt 2 and passing run at attempt 1, so no further re-runs

GitHub then reported mergeStateStatus: CLEAN: the stale failure no longer blocks, and no human step was needed. GITHUB_TOKEN with actions: write can re-run the run. The reverse direction (an objection withdrawn by a review event, as on guides#67) runs the same code path; it gets confirmed on a real Renovate PR during the canary migration.

@edbfi
edbfi force-pushed the refactor/v4-simplify branch from d303390 to 050244f Compare September 25, 2026 08:53
@edbfi

edbfi commented Sep 25, 2026

Copy link
Copy Markdown
Owner Author

Canary evidence: edbfi/v4-canary (2026-09-25)

This is a disposable consumer that started on the exact v3 contract (guard, recovery, inline automerge keys). It was migrated with the v4 checklist and ran against this branch's code, via the throwaway ref test/v4-canary (#62's head plus internal refs pinned to it).

# Scenario Result
S1 Migration PR: protection minus guard (other checks and app 15368 kept, strict), Allow auto-merge, pinned auto-merge edbfi/v4-canary#1 merged, no bypass
S2 Untouched Renovate PR edbfi/v4-canary#4 auto-merged 24 s after green
S3 Human PR through gh pr merge --auto #1
S4 Biome repair: publish, then the new head gets one pull_request ci run and one PR policy run, 0 workflow_dispatch runs; after a strict rebase, repair re-published and the PR auto-merged edbfi/v4-canary#3, 28 s after green; a second repair run on the repaired head found no diff
S5 Reverse self-refresh: changes requested, label add/remove (pull_request failures), review dismissed; the pull_request_review pass re-ran the latest pull_request failure (attempt 2 success) edbfi/v4-canary#2 auto-merged 32 s after dismissal, with no manual re-run
S6 Fail-closed: Renovate armed auto-merge, ci / required failed edbfi/v4-canary#5 stays open

Push CI on main passed after every merge (5 of 5). The repo has 0 workflow_dispatch runs in total.

@edbfi
edbfi force-pushed the refactor/v4-simplify branch from b691c1b to 2ef36e1 Compare September 25, 2026 16:50
@edbfi
edbfi marked this pull request as ready for review September 25, 2026 16:50
@edbfi
edbfi enabled auto-merge (rebase) September 25, 2026 16:50
The App-authenticated branch update already triggers the PR's normal CI and
policy runs, so the explicit workflow dispatch only produced a duplicate full
CI run for every repaired head. Publication now ends with the non-force ref
update. Remove the post-publication handoff retry, the dispatch helper, the
ci-workflow input and the publish job's actions: write permission.

BREAKING CHANGE: the biome-repair action no longer accepts ci-workflow and no
longer dispatches ci.yml; callers drop both and their actions: write grant.

Signed-off-by: edbfi <326875205+edbfi@users.noreply.github.com>
…flows

Recovery reconciled lost repair dispatches, which no longer exist, and
dispatch-guard validated those dispatches. The go, rust and content workflows
had no consumers. This repository's CI drops the guard steps and dispatch
inputs, and the workflow tests assert the retired pieces stay removed.

BREAKING CHANGE: actions/ci-recovery, actions/dispatch-guard and the go, rust
and content reusable workflows are removed.

Signed-off-by: edbfi <326875205+edbfi@users.noreply.github.com>
automerge.json now enables automerge with platformAutomerge, rebase merges and
no wildcard package rule, so the base preset's manual renovate-config rule and
repository opt-outs still win. default.json no longer sets the merge
mechanism. The guides pilot (edbfi/guides#58, #67) verified that every
required-check and policy condition blocks GitHub auto-merge, and the median
wait from green to merge dropped from 219 minutes to seconds.

Add scripts/renovate_effective.mjs, a conservative resolver that reports
automerge as potentially enabled whenever any automerge: true remains and
treats an omitted platformAutomerge as Renovate's default, true.

BREAKING CHANGE: automerge.json no longer opts renovate-config updates in and
now uses platform automerge; consumers need Allow auto-merge enabled.

Signed-off-by: edbfi <326875205+edbfi@users.noreply.github.com>
scripts/fleet_audit.py checks every repository declared in scripts/fleet.json
against the v4 contract: repository settings, retired files and inputs,
release references, enforced and actually emitted required checks, post-merge
CI and automerge consistency. Inaccessible evidence is unknown and never
compliant; the audit exits non-zero until every check passes or carries a
declared exception.

Signed-off-by: edbfi <326875205+edbfi@users.noreply.github.com>
Rewrite the README as a short reference with the v4 migration checklist,
restate the merge model and automerge eligibility with the pilot evidence,
drop recovery and dispatch from the repair and policy docs, record that
policy verdicts are kept per triggering event, and move the dated fleet
review to docs/archive/. AGENTS.md is now the agent guidance file and
CLAUDE.md imports it.

Signed-off-by: edbfi <326875205+edbfi@users.noreply.github.com>
The provider repository runs with an empty global config, so git had to guess
an identity and failed where no valid email can be derived from the host name.

Signed-off-by: edbfi <326875205+edbfi@users.noreply.github.com>
GitHub keeps one policy verdict per triggering event, so a pass from a review
event did not replace an earlier failure from a pull_request event, and the head
stayed blocked until someone re-ran the failed run (seen in edbfi/guides#67).
After a pass, policy now re-runs the other event's latest run for the same head
when it failed and started earlier. Failing runs never re-run anything, newer
failures are kept, a passing re-run finds its sibling green, and attempts are
capped at three, so it cannot loop. The re-run endpoint is the action's only
write.

BREAKING CHANGE: callers of pr-policy.yml must grant actions: write.

Signed-off-by: edbfi <326875205+edbfi@users.noreply.github.com>
automation runs ./actions/pr-policy from its own policy.yml instead of the
reusable workflow, so its self-audit could never pass. Any workflow that
runs policy, through the reusable workflow or the action, must grant
actions: write.

Signed-off-by: edbfi <326875205+edbfi@users.noreply.github.com>
Deleting a merged branch updates its PR, so the Phase 0 sweep made old
PRs look newest. The audit then sampled a PR from before a job existed
and reported that job as required but not emitted (mover-status,
wings-vpn, zondarr, otpravkarr). Pick the latest merged_at in the window.

Signed-off-by: edbfi <326875205+edbfi@users.noreply.github.com>
wings-vpn takes upstream pelican/wings syncs as merge commits. The other
forks' upstream updates arrive as ordinary commits, and merge commits are
now off there.

Signed-off-by: edbfi <326875205+edbfi@users.noreply.github.com>
Signed-off-by: edbfi <326875205+edbfi@users.noreply.github.com>
@edbfi
edbfi force-pushed the refactor/v4-simplify branch from 2ef36e1 to 5fd4997 Compare September 25, 2026 17:00
@edbfi
edbfi merged commit b0027bf into main Sep 25, 2026
4 checks passed
@edbfi
edbfi deleted the refactor/v4-simplify branch September 25, 2026 17:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant