diff --git a/docs/roadmaps/entries/PMAT-3646.yaml b/docs/roadmaps/entries/PMAT-3646.yaml new file mode 100644 index 0000000000..b978739a78 --- /dev/null +++ b/docs/roadmaps/entries/PMAT-3646.yaml @@ -0,0 +1,23 @@ +- id: PMAT-3646 + github_issue: 3646 + item_type: task + title: check_pr_review_counts.sh is RED on main (6 rows disagree) and check_pr_review_receipt.sh has no caller — both dark since PRQ-013 moved the receipt job, classified out of --no-cargo by a comment + status: planned + priority: high + assigned_to: null + created: 2026-09-21T06:00:51Z + updated: 2026-09-21T06:00:51Z + spec: null + acceptance_criteria: + - 'Issue #3646 "What lands" item 1, verbatim: "Fix the 6 disagreeing rows in PR-REVIEW-SKILL-002''s files (or the derivation), then reword the one comment so the guard leaves the cargo population and `guard-tree` runs it; delete its ledger line in the same commit (shrink-only)."' + - 'Issue #3646 "What lands" item 2, verbatim: "Decide `check_pr_review_receipt.sh`''s caller: either `pr-review-quorum.yml` invokes it on the PR''s receipt (a workflow edit — operator''s call), or it is folded into `check_pr_review_arm4.sh`, which is already wired with args. Delete its ledger line with whichever lands."' + phases: [] + subtasks: [] + estimated_effort: null + labels: + - kind:code + notes: 'Handed over by aprender-b3 (2026-09-21); claimed on the issue. Measured on origin/main @ 237fbc32f per the issue: `bash scripts/check_pr_review_counts.sh` rc=1 "--- 6 row(s) disagree with the tree ---"; `bash scripts/check_pr_review_receipt.sh` rc=1 "A run over zero receipts is not a pass (S6.3: a missing receipt is RED)." The ledger line for each guard is added by #3647 (scripts/unwired_guards_baseline.txt, shrink-only); it is deleted in the commit that wires that guard, after #3647 merges. Item 1 lands first; item 2 lands here only if it folds into check_pr_review_arm4.sh with no workflow edit, otherwise the ticket stays open for the operator''s call and the PR says Refs, not Closes. + + What the item-1 PR does. (a) The 6 RED rows were ci.yml sites whose sentences PRQ-013 deleted along with the receipt job (roadmap: "STATED LOSS"), so the rows go. The bats_tests derivation goes too, because no file states that count any more; ci.yml''s fixture_rows site goes from 2 to 1; and scripts/check_pr_review_wiring.sh''s "43-row" joins the table with a self-test row of its own. (b) The :196 comment is reworded so CARGO_RE stops matching: `guard_tree.sh --dry-run --no-cargo` prints `run: scripts/check_pr_review_counts.sh`, and restoring the old wording moves it back to --cargo-only (mutation-verified). (c) The --list mode of mutate-guard.sh and mutate_quorum_arm.sh requires only awk. Before this it demanded bats/jq/minisign/check-jsonschema, so wiring the counts guard into guard-tree, which runs on clean-room HOSTS including the gx10/yoga docker runners, would have turned a dark guard into an ENV-red job (reproduced: rc=2 under a PATH without those four). Sweeps still refuse without them (rc=1). + + Item 2, measured: check_pr_review_receipt.sh HAS a caller with a receipt. check_pr_review_arm4.sh runs `bash "$guard" "$scratch"` (A3) and `bash "$guard" "$best_dir"` (A4) with guard defaulting to it, and pr-review-quorum.yml:141 runs arm4. So there is nothing to fold. What remains is whether guard_tree.sh and the unwired ledger should treat a guard reachable only through another wired guard as wired. That is recorded on the issue and not changed here.' diff --git a/docs/roadmaps/roadmap.yaml b/docs/roadmaps/roadmap.yaml index 84e48c530f..4b996853c8 100644 --- a/docs/roadmaps/roadmap.yaml +++ b/docs/roadmaps/roadmap.yaml @@ -19007,3 +19007,26 @@ roadmap: labels: - kind:code notes: 'ACCEPTANCE (hand-entered from issue #3629 done_when; minted on the PR branch so the AD-04 quorum can run -- pmat work status refuses a missing item). Done when: scripts/fleet_utilization.sh --selftest has no pipe into grep -q; a mutation that restores the pipe AND forces EPIPE (out padded to >64 KiB so the write cannot complete before grep exits) turns the self-test RED deterministically — the positive control for the race, in the PR body the 55-site count is in the PR body from the command that produced it, and a guard refuses a new site (check_guards_are_wired.sh names it; ratchet baseline committed) main''s guard-tree green on the merge commit without rerun ' +- id: PMAT-3646 + github_issue: 3646 + item_type: task + title: check_pr_review_counts.sh is RED on main (6 rows disagree) and check_pr_review_receipt.sh has no caller — both dark since PRQ-013 moved the receipt job, classified out of --no-cargo by a comment + status: planned + priority: high + assigned_to: null + created: 2026-09-21T06:00:51Z + updated: 2026-09-21T06:00:51Z + spec: null + acceptance_criteria: + - 'Issue #3646 "What lands" item 1, verbatim: "Fix the 6 disagreeing rows in PR-REVIEW-SKILL-002''s files (or the derivation), then reword the one comment so the guard leaves the cargo population and `guard-tree` runs it; delete its ledger line in the same commit (shrink-only)."' + - 'Issue #3646 "What lands" item 2, verbatim: "Decide `check_pr_review_receipt.sh`''s caller: either `pr-review-quorum.yml` invokes it on the PR''s receipt (a workflow edit — operator''s call), or it is folded into `check_pr_review_arm4.sh`, which is already wired with args. Delete its ledger line with whichever lands."' + phases: [] + subtasks: [] + estimated_effort: null + labels: + - kind:code + notes: 'Handed over by aprender-b3 (2026-09-21); claimed on the issue. Measured on origin/main @ 237fbc32f per the issue: `bash scripts/check_pr_review_counts.sh` rc=1 "--- 6 row(s) disagree with the tree ---"; `bash scripts/check_pr_review_receipt.sh` rc=1 "A run over zero receipts is not a pass (S6.3: a missing receipt is RED)." The ledger line for each guard is added by #3647 (scripts/unwired_guards_baseline.txt, shrink-only); it is deleted in the commit that wires that guard, after #3647 merges. Item 1 lands first; item 2 lands here only if it folds into check_pr_review_arm4.sh with no workflow edit, otherwise the ticket stays open for the operator''s call and the PR says Refs, not Closes. + + What the item-1 PR does. (a) The 6 RED rows were ci.yml sites whose sentences PRQ-013 deleted along with the receipt job (roadmap: "STATED LOSS"), so the rows go. The bats_tests derivation goes too, because no file states that count any more; ci.yml''s fixture_rows site goes from 2 to 1; and scripts/check_pr_review_wiring.sh''s "43-row" joins the table with a self-test row of its own. (b) The :196 comment is reworded so CARGO_RE stops matching: `guard_tree.sh --dry-run --no-cargo` prints `run: scripts/check_pr_review_counts.sh`, and restoring the old wording moves it back to --cargo-only (mutation-verified). (c) The --list mode of mutate-guard.sh and mutate_quorum_arm.sh requires only awk. Before this it demanded bats/jq/minisign/check-jsonschema, so wiring the counts guard into guard-tree, which runs on clean-room HOSTS including the gx10/yoga docker runners, would have turned a dark guard into an ENV-red job (reproduced: rc=2 under a PATH without those four). Sweeps still refuse without them (rc=1). + + Item 2, measured: check_pr_review_receipt.sh HAS a caller with a receipt. check_pr_review_arm4.sh runs `bash "$guard" "$scratch"` (A3) and `bash "$guard" "$best_dir"` (A4) with guard defaulting to it, and pr-review-quorum.yml:141 runs arm4. So there is nothing to fold. What remains is whether guard_tree.sh and the unwired ledger should treat a guard reachable only through another wired guard as wired. That is recorded on the issue and not changed here.' diff --git a/scripts/check_pr_review_counts.sh b/scripts/check_pr_review_counts.sh index d2ba36f814..9955aa41ca 100755 --- a/scripts/check_pr_review_counts.sh +++ b/scripts/check_pr_review_counts.sh @@ -101,15 +101,6 @@ derive_fixture_rows() { printf '%s\n' "$n" } -# bats tests: `@test` at column 0 in the file the CI step actually runs. -derive_bats_tests() { - local root=$1 n - [ -f "$root/tests/pr-review.bats" ] || return 1 - n=$(grep -c '^@test ' "$root/tests/pr-review.bats") - [ "$n" -gt 0 ] || return 1 - printf '%s\n' "$n" -} - # S13 (PRREV-015) adds a SECOND table, a SECOND bats file and a SECOND mutation set. # They are derived here rather than left uncounted, because the universe this guard # iterates is what decides whether it can fail at all: a table it has never heard of is @@ -193,9 +184,11 @@ derive_publish_rows() { # Falsification tests: the contract's own list, counted from its `- id:` labels rather # than from the prose that states the total. It went stale unseen - `pass_criteria` said # 11 over 15 entries at PRREV-013 and over 17 at PRREV-015 - and the only thing that -# noticed was `cargo test -p aprender-contracts --test validate_contracts` printing +# noticed was the aprender-contracts `validate_contracts` integration test printing # `pr-review-skill-v2: pass_criteria says 11 tests, actual 17` into a corpus-wide failure -# nobody reads per-contract. Counting labels and not lines, the same reason S7's rows +# nobody reads per-contract. (Name that test runner's binary in prose here and +# guard_tree.sh's CARGO_RE classifies this whole guard as build-tool-using, which +# drops it from guard-tree's --no-cargo run. That is how it went dark: #3646.) Counting labels and not lines, the same reason S7's rows # carry ids B1..B6. derive_falsification_tests() { local root=$1 n @@ -218,32 +211,43 @@ derive_falsification_tests() { # --------------------------------------------------------------------------- # The site table. id | file | occurrences | template (@N@ -> derived value) +# +# SIX ci.yml ROWS LEFT WITH THE JOB THAT STATED THEM (#3646). PRQ-013 (#2982, #2985) +# deleted ci.yml's receipt job, and with it the step names that carried "233/233", +# "165 tests", "134-mutant" x2, "83 rows" and "Arm 4 case table: 10 rows", plus one +# of the two "43-row" sites. That is the "sentence was deleted" case, and here it was +# deliberate: the roadmap records the sweeps and the fixture table as a STATED LOSS. +# So the rows go, not the sentences. Nobody noticed for weeks because a comment +# naming the build tool had classified this guard out of guard-tree's --no-cargo +# population, so it went RED on main and ran nowhere. The ci.yml rows that remain are +# for step names that still exist. +# +# The bats_tests count went with them. No file states the size of tests/pr-review.bats +# any more, and a derivation with no site checks nothing, so it was removed rather +# than left to read as coverage. If a file states that count again, derive it again +# and add the row. # --------------------------------------------------------------------------- SITES=' mutants|.claude/skills/pr-review/SKILL.md|2|@N@/@N@ mutants|contracts/binding.yaml|1|@N@/@N@ -mutants|.github/workflows/ci.yml|1|@N@/@N@ mutants|docs/specifications/PR-REVIEW-SKILL-002-v2.md|1|@N@/@N@ -fixture_rows|.github/workflows/ci.yml|2|@N@-row +fixture_rows|.github/workflows/ci.yml|1|@N@-row +fixture_rows|scripts/check_pr_review_wiring.sh|1|@N@-row fixture_rows|tests/pr-review.bats|1|@N@ row fixture_rows|tests/pr-review.bats|1|-eq @N@ ] -bats_tests|.github/workflows/ci.yml|2|@N@ tests quorum_mutants|docs/specifications/PR-REVIEW-SKILL-002-v2.md|3|@N@/@N@ quorum_mutants|.claude/skills/pr-review/SKILL.md|1|@N@/@N@ quorum_mutants|contracts/pr-review-skill-v2.yaml|1|@N@/@N@ quorum_mutants|.claude/skills/pr-review/SKILL.md|1|@N@-mutant -quorum_mutants|.github/workflows/ci.yml|2|@N@-mutant quorum_mutants|docs/specifications/PR-REVIEW-SKILL-002-v2.md|2|@N@ mutants quorum_bats_tests|docs/specifications/PR-REVIEW-SKILL-002-v2.md|1|@N@ rows quorum_bats_tests|docs/specifications/PR-REVIEW-SKILL-002-v2.md|1|@N@-row quorum_bats_tests|.claude/skills/pr-review/SKILL.md|2|@N@ rows -quorum_bats_tests|.github/workflows/ci.yml|1|@N@ rows quorum_rows|tests/pr-review-quorum.bats|1|-eq @N@ ] quorum_rows|tests/pr-review-quorum.bats|1|expected @N@ q-* falsification_tests|contracts/pr-review-skill-v2.yaml|1|All @N@ falsification tests shadow_rows|.github/workflows/ci.yml|1|case table: @N@ rows publish_rows|.github/workflows/ci.yml|1|Publisher case table: @N@ rows -arm4_rows|.github/workflows/ci.yml|1|Arm 4 case table: @N@ rows arm4_rows|.github/workflows/pr-review-quorum.yml|1|Arm 4 case table: @N@ rows signer_rows|.github/workflows/ci.yml|1|Signer case table: @N@ rows ' @@ -259,8 +263,6 @@ check() { derived[mutants]=$v v=$(derive_fixture_rows "$root") || die_env "cannot derive the fixture-row count" derived[fixture_rows]=$v - v=$(derive_bats_tests "$root") || die_env "cannot derive the bats test count" - derived[bats_tests]=$v v=$(derive_quorum_mutants "$root") || die_env "cannot derive the S13 mutation-set size (mutate_quorum_arm.sh --list)" derived[quorum_mutants]=$v v=$(derive_quorum_rows "$root") || die_env "cannot derive the S13 q-* fixture-row count" @@ -280,7 +282,7 @@ check() { [ -n "$quiet" ] || { echo "=== the counts these files state as measured, against the tree ($PROG) ===" - echo "derived: mutants=${derived[mutants]} fixture_rows=${derived[fixture_rows]} bats_tests=${derived[bats_tests]}" + echo "derived: mutants=${derived[mutants]} fixture_rows=${derived[fixture_rows]}" echo " quorum_mutants=${derived[quorum_mutants]} quorum_rows=${derived[quorum_rows]} quorum_bats_tests=${derived[quorum_bats_tests]}" echo " falsification_tests=${derived[falsification_tests]} shadow_rows=${derived[shadow_rows]} publish_rows=${derived[publish_rows]} arm4_rows=${derived[arm4_rows]} signer_rows=${derived[signer_rows]}" } @@ -381,7 +383,8 @@ self_test() { scripts/pr_review_quorum_arm.sh \ scripts/pr_review_shadow_record.sh \ scripts/pr_review_shadow_publish.sh \ - scripts/check_pr_review_arm4.sh scripts/pr_review_sign_receipt.sh ) \ + scripts/check_pr_review_arm4.sh scripts/pr_review_sign_receipt.sh \ + scripts/check_pr_review_wiring.sh ) \ | ( cd "$base" && tar -xf - ) || die_env "could not stage a copy of the tree" local nrows=0 @@ -428,8 +431,9 @@ self_test() { sed -i "s#$n/$n#$((n - 1))/$((n - 1))#" "$1/.claude/skills/pr-review/SKILL.md"; } stale_rows() { local n; n=$(derive_fixture_rows "$1") sed -i "s#$n-row#$((n - 1))-row#" "$1/.github/workflows/ci.yml"; } - stale_tests() { local n; n=$(derive_bats_tests "$1") - sed -i "s#$n tests#$((n - 1)) tests#" "$1/.github/workflows/ci.yml"; } + # The site #3646 added: check_pr_review_wiring.sh states the fixture-row count too. + stale_wrows() { local n; n=$(derive_fixture_rows "$1") + sed -i "s#$n-row#$((n - 1))-row#" "$1/scripts/check_pr_review_wiring.sh"; } delete_claim() { local n; n=$(derive_mutants "$1") sed -i "0,\#$n/$n#{\#$n/$n#d}" "$1/contracts/binding.yaml"; } stale_ftests() { local n; n=$(derive_falsification_tests "$1") @@ -440,7 +444,6 @@ self_test() { # let exactly this edit through green. grow_ftests() { sed -i 's#^- id: F-PRREV-002$#- id: F-INVENTED-999\n rule: a falsification test the contract does not count\n\n&#' \ "$1/contracts/pr-review-skill-v2.yaml"; } - grow_bats() { printf '\n@test "a new test the docs do not count" {\n true\n}\n' >> "$1/tests/pr-review.bats"; } grow_rows() { mkdir -p "$1/tests/fixtures/pr-review/row-27-invented"; } grow_mutants() { sed -i 's#^ \[ -n "$head" \] || reject B1 "predicate.head_sha is absent" || return 1#&\n [ -n "$head" ] || reject B1 "an invented rule nothing documents" || return 1#' \ "$1/scripts/check_pr_review_receipt.sh"; } @@ -470,9 +473,10 @@ self_test() { row baseline 0 "the tree as committed" noop row stale-mutation-score 1 "the mutation score written back by one" stale_mutants row stale-fixture-rows 1 "the fixture-row count written back by one" stale_rows - row stale-bats-count 1 "the bats test count written back by one" stale_tests + row stale-wiring-rows 1 "the wiring guard's fixture-row count written back by one" stale_wrows + # stale-bats-count and tree-grew-a-test left with the bats_tests derivation (#3646): + # no file states that count, so neither edit has a claim to falsify. row claim-deleted 1 "the sentence stating the count is deleted" delete_claim - row tree-grew-a-test 1 "a bats test is added and no file says so" grow_bats row tree-grew-a-fixture-row 1 "a row-* fixture is added and no file says so" grow_rows row tree-grew-a-mutant 1 "a reject site is added and no file says so" grow_mutants row stale-quorum-mutation 1 "the S13 mutation score written back by 18" stale_qmutants @@ -505,7 +509,6 @@ case "${1:-}" in --show) printf 'mutants %s\n' "$(derive_mutants "$REPO_ROOT")" printf 'fixture_rows %s\n' "$(derive_fixture_rows "$REPO_ROOT")" - printf 'bats_tests %s\n' "$(derive_bats_tests "$REPO_ROOT")" printf 'quorum_mutants %s\n' "$(derive_quorum_mutants "$REPO_ROOT")" printf 'quorum_rows %s\n' "$(derive_quorum_rows "$REPO_ROOT")" printf 'quorum_bats_tests %s\n' "$(derive_quorum_bats_tests "$REPO_ROOT")" diff --git a/scripts/mutate-guard.sh b/scripts/mutate-guard.sh index 450fc4fced..ad466564b9 100755 --- a/scripts/mutate-guard.sh +++ b/scripts/mutate-guard.sh @@ -104,7 +104,12 @@ while [ "$#" -gt 0 ]; do esac done -for t in bats jq awk minisign check-jsonschema git; do +# --list prints the catalogue and runs no mutant, so it needs only what catalogue() +# reads with. Demanding the sweep's tools there made every count derived from the +# catalogue (check_pr_review_counts.sh) an ENV failure on a runner that cannot run the +# sweep, and guard-tree runs on clean-room hosts, not in a sweep image (#3646). +if [ "$LIST_ONLY" = 1 ]; then need='awk'; else need='bats jq awk minisign check-jsonschema git'; fi +for t in $need; do command -v "$t" >/dev/null 2>&1 || { echo "$PROG: FAIL - $t is not on PATH. A mutation score that could not be measured" >&2 echo " must not be reported as one (S8: guard_mutation_score is a one, not a floor)." >&2 diff --git a/scripts/mutate_quorum_arm.sh b/scripts/mutate_quorum_arm.sh index 995ca8b6da..0139c4de2d 100755 --- a/scripts/mutate_quorum_arm.sh +++ b/scripts/mutate_quorum_arm.sh @@ -72,7 +72,10 @@ while [ "$#" -gt 0 ]; do esac done -for t in bats jq awk minisign check-jsonschema git; do +# --list runs no mutant; it needs only what catalogue() reads with (see +# mutate-guard.sh, #3646). +if [ "$LIST_ONLY" = 1 ]; then need='awk'; else need='bats jq awk minisign check-jsonschema git'; fi +for t in $need; do command -v "$t" >/dev/null 2>&1 || { echo "$PROG: FAIL - $t is not on PATH. A mutation score that could not be measured" >&2 echo " must not be reported as one (S13.10: this number is a one, not a floor)." >&2