From eebd642c85a3cbe974d8177a58c72b6c0d0e1801 Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Mon, 21 Sep 2026 10:26:01 +0200 Subject: [PATCH 1/2] =?UTF-8?q?feat(release):=20batching=20is=20the=20defa?= =?UTF-8?q?ult=20merge=20path=20=E2=80=94=20scripts/batch=5Ffold.sh=20+=20?= =?UTF-8?q?APR-RELEASE-001=20=C2=A713=20(#3673)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Refs #3673 #3669 #3650 paiml-implement#317 Operator 2026-09-21: "we are arbitrarily going slow because each PR adds time, but in most cases 80% of PR can be batched." Code handed off by the cop (aprender-04) as an uncommitted draft; rewritten here. scripts/batch_fold.sh -- fold receipted PR branches into release/-batch, --no-ff, in order. It resolves ONLY the generated set: roadmap.yaml, census.json, contracts.nt, shapes.ttl (pv extract writes it), and the README CONTRACT_COUNT blocks. Every other conflict aborts that branch: SKIP + paths. - README is merged three-way with the counts normalised out (git merge-file), so a prose conflict is a SKIP. The draft took README wholesale with --theirs, which silently dropped the batch side's prose. - A CLEAN fold that touches the generated set marks it stale: two PRs that each add a contract merge textually and the census is then off. Stale without --regen ends `REGEN REQUIRED`, exit 3, never silent. - --regen regenerates once with the tree-built pv (pv_bin.sh), asserts the three fixed points (make roadmap-aggregate-check, pv extract --check, readme_sync --check), refuses writes outside the set, commits. - refuses a dirty batch worktree; exit 0/1/2/3 documented. scripts/check_batch_fold.sh -- the case table, 21 rows over throwaway repos with stub pv/make/readme_sync, plus --self-test with 4 mutants, each RED: README wholly generated (the draft's rule), every conflict taken, a clean fold never stale, a failed fixed point ignored. It is a check_*.sh on purpose: the draft's own `--self-test)` case made check_guards_are_wired.sh RED ("NEW: batch_fold.sh", measured), and guard_tree.sh --no-cargo dispatches this file with no workflow edit. The draft, judged by this table: 13 of 21 rows RED (3 of them only on output wording). APR-RELEASE-001: - §13 (new): roles (author stops at the receipt; never arms, never merges main to stay current), assembly, the generated set, left-out rules, what needs a new quorum, push/disarm, body/arm/close, a red batch, what it costs (a SQUASH queue lands a batch as ONE commit: bisect on main stops at the batch, so the batch branch is kept until the next tag), falsifiers FX-B1..3. - hard rule 9 AMENDED. It read "No hand-squash of batches ... each PR lands as its own commit ... never hand-bisected", which the new default contradicts. The old text is kept verbatim as superseded, with the reason: the queue it described (group size 8) was not the queue we had (SQUASH, maximumEntriesToMerge=1, measured on #3658). - §8 stop "a hand-squashed batch" -> "a batch assembled by hand instead of scripts/batch_fold.sh (§13)". - revision note 2026-09-21. Gates: check_batch_fold 21/21 + self-test 4/4; bashrs 0 errors on both files; bashrs-gate PASS; guards-are-wired PASS; guard_tree dispatches it; guard_tree_test 23/0; pipe-into-grep-q at ceiling; hardcoded-paths +0; apr pinned; no-timing-in-required; check_ci_unwedge and check_ont_ratchet (read the spec) green; roadmap aggregate + sorted/unique/additive; fmt, deny, aprender-contracts --lib 1666. Co-Authored-By: Claude Opus 5 (1M context) --- docs/roadmaps/entries/PMAT-3673.yaml | 17 ++ docs/roadmaps/roadmap.yaml | 17 ++ .../APR-RELEASE-001-train-and-build-kaizen.md | 121 +++++++++++- scripts/batch_fold.sh | 183 ++++++++++++++++++ scripts/check_batch_fold.sh | 169 ++++++++++++++++ 5 files changed, 503 insertions(+), 4 deletions(-) create mode 100644 docs/roadmaps/entries/PMAT-3673.yaml create mode 100755 scripts/batch_fold.sh create mode 100644 scripts/check_batch_fold.sh diff --git a/docs/roadmaps/entries/PMAT-3673.yaml b/docs/roadmaps/entries/PMAT-3673.yaml new file mode 100644 index 0000000000..4e0e102b9e --- /dev/null +++ b/docs/roadmaps/entries/PMAT-3673.yaml @@ -0,0 +1,17 @@ +- id: PMAT-3673 + github_issue: 3673 + item_type: task + title: 'Batching is the default merge path: scripts/batch_fold.sh + APR-RELEASE-001 §13 (operator 2026-09-21)' + status: planned + priority: critical + assigned_to: null + created: 2026-09-21T08:23:21Z + updated: 2026-09-21T08:23:21Z + spec: docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md + acceptance_criteria: [] + phases: [] + subtasks: [] + estimated_effort: null + labels: + - kind:code + notes: 'ACCEPTANCE (quoted from issue #3673 "What lands"): 1. scripts/batch_fold.sh — fold receipted PR branches into one release/-batch branch, git merge --no-ff in the given order: a conflict ONLY in the generated set {docs/roadmaps/roadmap.yaml, contracts/census.json, contracts/contracts.nt, contracts/shapes.ttl, the CONTRACT_COUNT blocks of README.md} is taken and marked for regeneration; a conflict ANYWHERE else — including README prose outside a count block — aborts that branch''s merge and reports SKIP with the paths, never resolved by picking a side; --regen regenerates the generated set ONCE with the tree-built pv (scripts/pv_bin.sh), asserts the fixed points (aggregate --check, pv extract --check, readme_sync.sh --check), and commits the regeneration; exit 0 all folded; 1 any SKIP; 2 usage/git/regeneration failure; a folded-but-unregenerated generated set is never silent; --self-test case table, wired into CI. 2. APR-RELEASE-001 §13 — the batching procedure as the default path (cop prompt §7): assembly, generated-set regeneration, left-out rules, what needs a new quorum (only content new to the batch), constituent disarm/close, a red batch, and the review-lane VOID rule while paiml-implement#317 is open. AUTHOR NOTES (not criteria): the case table lives in scripts/check_batch_fold.sh (guard-tree runs it) because a --self-test case inside batch_fold.sh made check_guards_are_wired.sh RED; §13 required amending hard rule 9 ("No hand-squash of batches") and the matching §8 stop, which the new default contradicts.' diff --git a/docs/roadmaps/roadmap.yaml b/docs/roadmaps/roadmap.yaml index 84e48c530f..174da9efbf 100644 --- a/docs/roadmaps/roadmap.yaml +++ b/docs/roadmaps/roadmap.yaml @@ -19007,3 +19007,20 @@ roadmap: labels: - kind:code notes: 'ACCEPTANCE (hand-entered from issue #3629 done_when; minted on the PR branch so the AD-04 quorum can run -- pmat work status refuses a missing item). Done when: scripts/fleet_utilization.sh --selftest has no pipe into grep -q; a mutation that restores the pipe AND forces EPIPE (out padded to >64 KiB so the write cannot complete before grep exits) turns the self-test RED deterministically — the positive control for the race, in the PR body the 55-site count is in the PR body from the command that produced it, and a guard refuses a new site (check_guards_are_wired.sh names it; ratchet baseline committed) main''s guard-tree green on the merge commit without rerun ' +- id: PMAT-3673 + github_issue: 3673 + item_type: task + title: 'Batching is the default merge path: scripts/batch_fold.sh + APR-RELEASE-001 §13 (operator 2026-09-21)' + status: planned + priority: critical + assigned_to: null + created: 2026-09-21T08:23:21Z + updated: 2026-09-21T08:23:21Z + spec: docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md + acceptance_criteria: [] + phases: [] + subtasks: [] + estimated_effort: null + labels: + - kind:code + notes: 'ACCEPTANCE (quoted from issue #3673 "What lands"): 1. scripts/batch_fold.sh — fold receipted PR branches into one release/-batch branch, git merge --no-ff in the given order: a conflict ONLY in the generated set {docs/roadmaps/roadmap.yaml, contracts/census.json, contracts/contracts.nt, contracts/shapes.ttl, the CONTRACT_COUNT blocks of README.md} is taken and marked for regeneration; a conflict ANYWHERE else — including README prose outside a count block — aborts that branch''s merge and reports SKIP with the paths, never resolved by picking a side; --regen regenerates the generated set ONCE with the tree-built pv (scripts/pv_bin.sh), asserts the fixed points (aggregate --check, pv extract --check, readme_sync.sh --check), and commits the regeneration; exit 0 all folded; 1 any SKIP; 2 usage/git/regeneration failure; a folded-but-unregenerated generated set is never silent; --self-test case table, wired into CI. 2. APR-RELEASE-001 §13 — the batching procedure as the default path (cop prompt §7): assembly, generated-set regeneration, left-out rules, what needs a new quorum (only content new to the batch), constituent disarm/close, a red batch, and the review-lane VOID rule while paiml-implement#317 is open. AUTHOR NOTES (not criteria): the case table lives in scripts/check_batch_fold.sh (guard-tree runs it) because a --self-test case inside batch_fold.sh made check_guards_are_wired.sh RED; §13 required amending hard rule 9 ("No hand-squash of batches") and the matching §8 stop, which the new default contradicts.' diff --git a/docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md b/docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md index b96f30d350..814b114df8 100644 --- a/docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md +++ b/docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md @@ -13,6 +13,10 @@ tag inheritance from the release commit, B2 split by capability (cpu shards + gp derived from `cargo metadata` (TIERS deleted), unattended cascade under standing authorization with receipts, v0.68.0 = GitHub-only / 0.68.1 = crates.io, P0·Fan-out as the 0.69 train's first PR. Earlier revisions kept where still true. *(Merged 2026-09-18 onto the `main` lineage — #3268 §4.1–4.3, §5.1, §6.1–6.4, §10–§12 and #3455 `check_milestone_cut.sh` — which the 09-15/16/17 working copy had been cut before; where the two disagreed the later operator ruling wins and the older text is kept as history.)* +**Revision 2026-09-21** — §13: integration batches are the DEFAULT merge path (operator: "we are +arbitrarily going slow because each PR adds time, but in most cases 80% of PR can be batched"), +assembled by `scripts/batch_fold.sh`. Hard rule 9 and the §8 "hand-squashed batch" stop are amended +to match (§13.9 says what that costs). **Revision 2026-09-16** — adds §1.6 critical path for 0.68, milestone hygiene (§6), the CI-wedge mechanism and its two sanctioned rules (P0·Unwedge), clean-room-on-tag (T-3), reaper budget as a manifest fact, and the capacity-source rule (no runners API, ever). @@ -178,9 +182,15 @@ that decides a gate. 2026-09-13 stands: "T4 is never mine … releases are automated by release train … for ALL releases".)* 8. `pmat work add` from the driver session only. Stop the line on RED — no reruns-as-passes, no `--skip`, no waivers. -9. **No hand-squash of batches.** Batch verification is the merge queue's job (merge_group, - group size 8); each PR lands as its own commit. A red batch fans out one job per PR, never - hand-bisected. +9. **A batch is an integration PR assembled by `scripts/batch_fold.sh`, never by hand (§13, + operator 2026-09-21).** Receipted PRs fold into `release/-batch`; only the generated set is + ever resolved by the tool, every other conflict leaves that PR out, and a red batch is traced to + its constituent with `git log --merges` on the batch branch (§13.7). + *(Superseded 2026-09-21. Until then this rule read: "No hand-squash of batches. Batch + verification is the merge queue's job (merge_group, group size 8); each PR lands as its own + commit. A red batch fans out one job per PR, never hand-bisected." The queue it described was + not the queue we had: mergeMethod SQUASH, maximumEntriesToMerge=1, measured on #3658 — so every + PR paid its own CI run, queue slot and roadmap re-merge.)* 10. **WIP is enforced at the producer.** paiml-implement's pre-push hook refuses to mint a PR when open aprender PRs ≥ 13 (basis: measured drain 8.8/day × 1.5 day, 2026-09-15; ratchet after 20 records) and emits an issue instead. Sweeps emit issues, not PRs. @@ -654,7 +664,7 @@ next: train eligible at - Any queue entry ejected twice on the same cause → stop, five-whys, no third re-queue. - Any measurement whose tree line shows `behind≠0`, or taken outside a per-session worktree → discard and re-measure from `origin/main`. -- Any step would need an invented threshold, a hand-squashed batch, a hand-cancelled run with started +- Any step would need an invented threshold, a batch assembled by hand instead of `scripts/batch_fold.sh` (§13), a hand-cancelled run with started jobs, a label edit to dodge a runner, an env bypass of a prejob gate, `--allow-dirty`, or a host CONFIG change made over SSH instead of through forjar (§3.5 — SSH for measurement and for unclogging is expected) → stop. - The upstream ontology spec, at the version this spec cites, is not reachable at a committed sha @@ -987,3 +997,106 @@ them are report lines. | FX-3 | §12.1.3 | every counter on a §7 line has a named ratchet target and moves one way | move one backward by hand → RED | | FX-4 | §12.2.C | `beats:` says `measured-on published` only when the dogfood ran on a `cargo install aprender` binary (G13: no embedded SHA, `DOGFOOD_ALLOW_UNPINNED=1`) | report `published` from a dev build → RED | | FX-5 | §12.2.C | a beat's status in `docs/BEATS.md` matches its contract | change the status without the contract → `readme_contract` RED | + +## §13 Integration batches — the default merge path (operator 2026-09-21) + +> "we are arbitrarily going slow because each PR adds time, but in most cases 80% of PR can be batched." + +The per-PR queue is the slow path. Every PR paid one CI run (~32 min median wall, ~68 runner-min), +one queue slot (~26 min per merge, measured), and, while `roadmap.yaml` is committed (#3650), about +one re-merge of the generated files per landing. The default is now ONE integration batch PR that +carries every receipted PR that is ready. First run: #3669, 18 PRs plus an andon fix. + +### §13.1 Who does what + +- **An author** stops at the receipt: branch, roadmap fragment, AD-04 quorum, + `docs/audits/quorum-.json` committed on the branch, lanes path to the batch driver. **Never + arms, and never merges main just to stay current.** +- **The batch driver** (the traffic cop) folds, regenerates, pushes and arms the batch. +- **A folded PR** is never pushed to again. Fixes go to the batch branch, or to the author's + branch followed by a re-fold. + +### §13.2 Assembly + +``` +git worktree add -b release/-batch origin/main +cd && bash scripts/batch_fold.sh --regen ... # priority order, cut-blockers first +``` + +Run it under **bash**. zsh does not word-split `$var`, which breaks a hand-rolled loop over branch +names. `batch_fold.sh` merges each branch `--no-ff`. It resolves ONLY the **generated set**: +`docs/roadmaps/roadmap.yaml`, `contracts/census.json`, `contracts/contracts.nt`, +`contracts/shapes.ttl`, and the `CONTRACT_COUNT` blocks of `README.md`. README is merged three-way +with the counts normalised out, so a README conflict outside a count block is a real conflict. Every +other conflict aborts that branch's merge (`SKIP : conflict in `). A clean fold that +touches the generated set still marks it stale, because two PRs that each add a contract merge +textually and the census is then wrong. + +`--regen` regenerates the set ONCE with the pv built from the batch tree (`scripts/pv_bin.sh`): +roadmap aggregate, `pv census`, `pv extract`, `readme_sync.sh --write`. It then asserts the fixed +points (`make roadmap-aggregate-check`, `pv extract --check`, `readme_sync.sh --check`) and commits +the regeneration. Exit codes: + +| Exit | Meaning | +|---|---| +| 0 | all folded, set consistent | +| 1 | a SKIP | +| 2 | a usage, git or regeneration failure, including a failed fixed point | +| 3 | folded, but the generated set is stale (`REGEN REQUIRED`) | + +### §13.3 Left out, and said so + +A PR is left out of the batch for exactly two reasons, and the batch body names which: + +- a real conflict (`SKIP`, e.g. #3606 against #3634); +- a tree-wide gate it fails on its own (e.g. `pv census` refused #3642's YAML). + +Nothing else keeps a receipted PR out. + +### §13.4 What needs a new quorum + +Each constituent's receipt stands, because its diff is unchanged inside the batch. Only content +NEW to the batch needs a quorum: fixes made in the batch branch, run with `--base `. The +regeneration needs none; its fixed points are asserted by `--regen`. While paiml-implement#317 is +open, a review lane whose agy conversation store (`~/.gemini/antigravity-cli/conversations/.db`) +never references the judged worktree is **VOID**. Stale shared scratch clones have produced +`measured` PASSes from another branch with rewritten baselines (#3664). A NON-author's measured +verification in the judged tree, posted on the PR with commands and outputs, fills the seat. + +### §13.5 Push, then disarm + +1. Push the batch first so CI starts. +2. Run `cargo check --workspace --all-targets` locally in parallel. +3. Comment "folded into #N" on each constituent and disarm it. +4. Cancel in-flight PR CI outside the batch; it goes stale when the batch moves main. +5. Dequeue anything the batch supersedes. + +### §13.6 The batch body, arming, closing + +The body has a table of constituents: PR, receipt path, head sha. It carries every constituent's +closing keywords, plus a `keep-open:` line naming the constituent PR numbers, or +`check_pr_closes_issue.sh` fails. Arm the batch. After it merges, close each constituent as "landed +in #N". + +### §13.7 A red batch + +1. Run `git log --merges` on the batch branch. +2. Find the constituent whose merge introduced the red, and reproduce it on that merge. +3. Either fix it in the batch (new content, so §13.4 applies) or drop it: rebuild the batch without + it, then `--regen`. +4. Push once. + +### §13.8 What this costs, stated rather than found + +The queue is SQUASH, so a batch lands on main as ONE commit. `git bisect` on main therefore stops at +the batch, not the constituent. The constituent history survives on the batch branch (the `--no-ff` +merges) and in the body's head-sha table, so the batch branch is kept until the train's next tag. +Deleting it early turns "which PR broke main" into a rebuild. + +### §13.9 Falsifiers + +| # | Assertion | Mutation | +|---|---|---| +| FX-B1 | `batch_fold.sh` resolves only the generated set: a code conflict and a README prose conflict are SKIP, a count-block conflict folds (`check_batch_fold.sh`, run by guard-tree on every PR) | treat README as wholly generated, or take every conflict → `check_batch_fold.sh --self-test` RED | +| FX-B2 | a stale generated set is never silent: a clean fold touching the census exits 3 | never mark a clean fold stale → RED | +| FX-B3 | a failed fixed point stops the batch (exit 2, the check named) | ignore `pv extract --check` → RED | diff --git a/scripts/batch_fold.sh b/scripts/batch_fold.sh new file mode 100755 index 0000000000..c3b9994b97 --- /dev/null +++ b/scripts/batch_fold.sh @@ -0,0 +1,183 @@ +#!/usr/bin/env bash +# batch_fold.sh -- fold receipted PR branches into ONE integration batch branch +# (APR-RELEASE-001 §13; operator 2026-09-21: "we are arbitrarily going slow +# because each PR adds time, but in most cases 80% of PR can be batched"). +# +# Run INSIDE the batch worktree (release/-batch, cut from origin/main). +# Each branch is merged --no-ff, in the order given (cut-blockers first). +# +# THE GENERATED SET is the only thing this script ever resolves by itself: +# docs/roadmaps/roadmap.yaml aggregate of docs/roadmaps/entries/ +# contracts/census.json pv census +# contracts/contracts.nt pv extract +# contracts/shapes.ttl pv extract (every `shape:` block) +# README.md CONTRACT_COUNT scripts/readme_sync.sh -- the COUNT BLOCKS only +# A conflict in those is taken and marked for regeneration. README.md is merged +# three-way with the counts normalised out, so a README conflict OUTSIDE a count +# block is a real conflict, not a generated one. Any real conflict aborts that +# branch's merge and reports SKIP with the paths -- never resolved by picking a +# side. The branch is left out of the batch and named. +# +# A merge that folds CLEANLY can still leave the generated set wrong: two PRs +# that each add a contract merge textually and the census count is then off by +# one. So every fold that touches a generated file marks the set STALE, and a +# stale set is never silent: --regen regenerates it ONCE, with the pv built from +# THIS tree (scripts/pv_bin.sh), asserts the three fixed points, and commits the +# regeneration; without --regen the run ends `REGEN REQUIRED` and exits 3. +# +# bash scripts/batch_fold.sh [--regen] BRANCH... +# bash scripts/batch_fold.sh --regen regenerate only +# +# Output, one line per branch: +# folded generated=[] merged; were taken for regeneration +# SKIP : conflict in merge aborted, branch left out +# ERROR : git could not merge it at all +# Exit: 0 every branch folded and the generated set is consistent; 1 any SKIP; +# 2 a usage, git or regeneration failure (including a failed fixed-point +# check); 3 folded, but the generated set is stale (run --regen). +# The case table is scripts/check_batch_fold.sh (guard-tree runs it). +# +# Test seam: BATCH_FOLD_PV= skips building pv (scripts/pv_bin.sh) and uses it. +set -uo pipefail + +GENERATED_WHOLE="docs/roadmaps/roadmap.yaml contracts/census.json contracts/contracts.nt contracts/shapes.ttl" +COUNT_RE='()[0-9]+()' + +die() { printf 'ERROR %s\n' "$*" >&2; exit 2; } +rmtree() { case "${1:-}" in ''|/) return 0 ;; *) [ -d "$1" ] && rm -rf -- "$1" ;; esac; return 0; } + +is_generated_whole() { # PATH -> 0 iff the whole file is derived + local g + for g in $GENERATED_WHOLE; do [ "$1" = "$g" ] && return 0; done + return 1 +} + +touches_generated() { # REV_A REV_B -> prints the generated paths that differ + local p + while IFS= read -r p; do + [ -n "$p" ] || continue + if is_generated_whole "$p" || [ "$p" = README.md ]; then printf '%s\n' "$p"; fi + done < <(git diff --no-renames --name-only "$1" "$2") +} + +# readme_counts_only -> 0 iff README.md's conflict is ONLY in CONTRACT_COUNT blocks. +# On 0 the file is written merged (counts left for readme_sync.sh) and staged. +readme_counts_only() { + local t rc=0 + t=$(mktemp -d "${TMPDIR:-/tmp}/batch-fold-readme.XXXXXX") || return 1 + if ! { git show :1:README.md > "$t/base" && git show :2:README.md > "$t/ours" && git show :3:README.md > "$t/theirs"; } 2>/dev/null; then + rmtree "$t"; return 1 # an add/add or delete conflict is not a count conflict + fi + local n + n=$(grep -m1 -oE "$COUNT_RE" "$t/ours" | sed -E 's/[^0-9]//g') + sed -E -i "s/$COUNT_RE/\\1N\\2/g" "$t/base" "$t/ours" "$t/theirs" + git merge-file -p "$t/ours" "$t/base" "$t/theirs" > "$t/merged" 2>/dev/null || rc=$? + if [ "$rc" = 0 ]; then + # put OUR count back so the file stays well-formed; --regen rewrites every block + sed -E -i "s/()N()/\\1${n:-0}\\2/g" "$t/merged" + cp -- "$t/merged" README.md && git add -- README.md || rc=1 + fi + rmtree "$t" + return "$rc" +} + +fold_one() { # BRANCH -> one verdict line; 0 folded, 1 skipped, 2 git failure + local br=$1 f real="" gen="" conf before + before=$(git rev-parse HEAD) || return 2 + if git merge --no-ff --no-edit "$br" >/dev/null 2>&1; then + gen=$(touches_generated "$before" HEAD | tr '\n' ' ') + [ -z "$gen" ] || STALE=1 + printf 'folded %s generated=[%s]\n' "$br" "${gen% }"; return 0 + fi + if ! git rev-parse -q --verify MERGE_HEAD >/dev/null 2>&1; then + printf 'ERROR %s: git merge failed without a conflict state\n' "$br"; return 2 + fi + conf=$(git diff --name-only --diff-filter=U) + while IFS= read -r f; do + [ -n "$f" ] || continue + if is_generated_whole "$f"; then + gen="$gen $f" + elif [ "$f" = README.md ] && readme_counts_only; then + gen="$gen README.md" + else + real="$real $f" + fi + done <<< "$conf" + if [ -n "$real" ]; then + git merge --abort >/dev/null 2>&1 + printf 'SKIP %s: conflict in%s\n' "$br" "$real"; return 1 + fi + for f in $gen; do + [ "$f" = README.md ] && continue # already merged and staged above + { git checkout --theirs -- "$f" && git add -- "$f"; } >/dev/null 2>&1 \ + || { git merge --abort >/dev/null 2>&1; printf 'ERROR %s: cannot take %s\n' "$br" "$f"; return 2; } + done + git -c commit.gpgsign=false commit -q --no-edit >/dev/null 2>&1 \ + || { git merge --abort >/dev/null 2>&1; printf 'ERROR %s: commit failed\n' "$br"; return 2; } + STALE=1 + printf 'folded %s generated=[%s]\n' "$br" "${gen# }"; return 0 +} + +regen() { # regenerate the generated set once, assert the fixed points, commit it + local PV extra + if [ -n "${BATCH_FOLD_PV:-}" ]; then + PV=$BATCH_FOLD_PV + else + # shellcheck source=scripts/pv_bin.sh + . scripts/pv_bin.sh || die "regen: scripts/pv_bin.sh could not build/resolve pv from this tree" + fi + make roadmap-aggregate >/dev/null 2>&1 || die "regen: make roadmap-aggregate failed" + "$PV" census contracts --format json > contracts/census.json 2>/dev/null || die "regen: pv census failed" + "$PV" extract contracts >/dev/null 2>&1 || die "regen: pv extract failed" + bash scripts/readme_sync.sh --write >/dev/null 2>&1 || die "regen: readme_sync.sh --write failed" + # the fixed points: each regenerated artifact must equal a second regeneration + make roadmap-aggregate-check >/dev/null 2>&1 || die "regen: fixed point FAILED -- make roadmap-aggregate-check" + "$PV" extract contracts --check >/dev/null 2>&1 || die "regen: fixed point FAILED -- pv extract contracts --check" + bash scripts/readme_sync.sh --check >/dev/null 2>&1 || die "regen: fixed point FAILED -- readme_sync.sh --check" + # only the generated set may have moved; anything else is a tool writing where it should not + extra=$(git status --porcelain --untracked-files=no | awk '{print $2}' \ + | grep -vxE 'docs/roadmaps/roadmap\.yaml|contracts/census\.json|contracts/contracts\.nt|contracts/shapes\.ttl|README\.md' || true) + [ -z "$extra" ] || die "regen: wrote outside the generated set: $(printf '%s ' $extra)" + if [ -n "$(git status --porcelain --untracked-files=no)" ]; then + git add -- docs/roadmaps/roadmap.yaml contracts/census.json contracts/contracts.nt contracts/shapes.ttl README.md 2>/dev/null + git -c commit.gpgsign=false commit -q -m "batch: regenerate the generated set once (batch_fold.sh --regen) + +roadmap aggregate, pv census, pv extract (contracts.nt + shapes.ttl) and the +README CONTRACT_COUNT blocks, regenerated with the pv built from this tree; +fixed points asserted: make roadmap-aggregate-check, pv extract --check, +readme_sync.sh --check." >/dev/null 2>&1 || die "regen: commit failed" + printf 'regenerated the generated set and committed it (fixed points asserted)\n' + else + printf 'regenerated the generated set: already consistent (fixed points asserted)\n' + fi +} + +case "${1:-}" in -h|--help) sed -n '2,42p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;; esac + +REGEN=0; branches=() +while [ $# -gt 0 ]; do + case "$1" in + --regen) REGEN=1; shift ;; + -*) printf 'batch_fold.sh: unknown flag %s\n' "$1" >&2; exit 2 ;; + *) branches+=("$1"); shift ;; + esac +done +[ "${#branches[@]}" -gt 0 ] || [ "$REGEN" = 1 ] \ + || { printf 'usage: bash scripts/batch_fold.sh [--regen] BRANCH... | --regen\n' >&2; exit 2; } +git rev-parse --is-inside-work-tree >/dev/null 2>&1 || die "not inside a git work tree" +[ -z "$(git status --porcelain --untracked-files=no)" ] \ + || die "the batch worktree has uncommitted changes; fold into a clean tree only" + +STALE=0; status=0 +for br in "${branches[@]}"; do + fold_one "$br"; rc=$? + [ "$rc" = 2 ] && exit 2 + [ "$rc" = 1 ] && status=1 +done +if [ "$REGEN" = 1 ]; then + regen +elif [ "$STALE" = 1 ]; then + printf 'REGEN REQUIRED: a folded branch touched the generated set; run: bash scripts/batch_fold.sh --regen\n' + [ "$status" = 0 ] && status=3 +fi +exit "$status" diff --git a/scripts/check_batch_fold.sh b/scripts/check_batch_fold.sh new file mode 100644 index 0000000000..37d980b225 --- /dev/null +++ b/scripts/check_batch_fold.sh @@ -0,0 +1,169 @@ +#!/usr/bin/env bash +# check_batch_fold.sh -- the case table for scripts/batch_fold.sh, the integration-batch +# folder (#3673, APR-RELEASE-001 §13). Batching is the DEFAULT merge path now (operator +# 2026-09-21), so the folder decides what lands with a batch: it may resolve ONLY the +# generated set, and must leave out -- never silently resolve -- every real conflict. +# +# The table runs the real batch_fold.sh against throwaway git repos. Regeneration uses +# stub tools (a stub pv via BATCH_FOLD_PV, a stub Makefile and readme_sync.sh in the +# fixture), so no build and no network are needed. +# +# WHY THIS FILE AND NOT `batch_fold.sh --self-test`: a script that carries its own +# --self-test case claims to be a guard, and check_guards_are_wired.sh requires a +# workflow to run it -- measured: the handed-off draft did exactly that and turned +# the meta-guard RED ("NEW: batch_fold.sh"). A check_*.sh is dispatched by +# guard_tree.sh --no-cargo, so the table runs on every PR with no workflow edit. +# +# check_batch_fold.sh the case table +# check_batch_fold.sh --self-test the mutants: four wrong folders, each must turn a row RED +set -uo pipefail +ROOT="$(cd "$(dirname "$0")/.." && pwd)" || exit 2 +SUBJECT="${BATCH_FOLD_SUBJECT:-$ROOT/scripts/batch_fold.sh}" + +rmtree() { case "${1:-}" in ''|/) return 0 ;; *) [ -d "$1" ] && rm -rf -- "$1" ;; esac; return 0; } + +# mkfixture DIR -- a repo with a generated set, README count blocks, code, and branches: +# plain (no generated path) gen1/gen2 (roadmap-only conflict) code1/code2 (real conflict) +# rc1/rc2 (README count-only conflict) rp1/rp2 (README prose conflict) +# cen (clean census edit) plus stub tools for --regen +mkfixture() { + local d=$1 + git init -q -b main "$d" || return 2 + ( + cd "$d" || exit 2 + export GIT_AUTHOR_NAME=t GIT_AUTHOR_EMAIL=t@t GIT_COMMITTER_NAME=t GIT_COMMITTER_EMAIL=t@t + export GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=core.hooksPath GIT_CONFIG_VALUE_0=/dev/null + mkdir -p docs/roadmaps contracts src scripts + printf 'a\n' > docs/roadmaps/roadmap.yaml + printf '{"n_files": 5}\n' > contracts/census.json + printf 't0\n' > contracts/contracts.nt + printf 's0\n' > contracts/shapes.ttl + printf 'x\n' > src/lib.rs + printf '# T\nprose line one\ncount: 5\nprose line two\n' > README.md + printf 'roadmap-aggregate:\n\t@printf %s > docs/roadmaps/roadmap.yaml\nroadmap-aggregate-check:\n\t@test "$$(cat docs/roadmaps/roadmap.yaml)" = aggregated\n' "'aggregated\\n'" > Makefile + cat > scripts/readme_sync.sh <<'SH' +#!/usr/bin/env bash +case "$1" in + --write) sed -E -i 's/()[0-9]+()/\17\2/g' README.md ;; + --check) grep -q 'CONTRACT_COUNT_START -->7510511[0-9]+7