Repository navigation
ci(mutants): ratchet mode — report-only gate + shrink-only debt baseline (PMAT-4646) - #4647
Conversation
…t baseline (PMAT-4646) Operator rulings C187(a)/C188(a), 2026-10-01: a new gate is adopted as a ratchet, never as an instant absolute bar. No ruleset change. - ci / gate: mutation results (the mutants section and, on PRs that add it, the survivor table) are printed and no longer fail the gate. Every check before it is unchanged: workspace-test, guard-tree, guard-cargo, sov.gate, determinism-compare. - ci/mutants-debt.tsv: 590 known survivors (MissedMutant + Timeout), with file, mutant, sha and run id, from completed shard artifacts of #4587 run 36725778557 and #4588 run 36703131059. - scripts/check_mutants_debt_ratchet.sh (auto-run by guard-tree): the debt file is shrink-only against the merge base. A grown file, a swapped row, a missing file, a malformed or duplicate row, or an unresolvable base is RED. Self-test: 10 cases; 7/7 planted mutants RED. The RESTORE PR deletes the MUTANTS-RATCHET block after crates.io 0.70. Agent: aprender-ca Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he quorum) Refs #4646 Agent: aprender-ca Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
§13.11 rung 1 — quorum shadow verdict Shadow mode: this records a verdict and merges nothing. A refusal |
…ut); gate echo names only defined vars check_mutants_debt_ratchet.sh [run] was RED in run 36829781758: merge-base(HEAD, origin/main) is unresolvable on a depth-1 checkout. It now uses resolve_base, the base every differential guard uses, which refuses rather than judge HEAD against HEAD (new case R11, plus R12 for the default origin/main path; 12/12; a fail-open mutant of the new branch is RED). check_workflow_env_defined: the gate echo interpolated IN_SCOPE/SCOPE/TABLE, which main's gate job does not define; dropped. Refs #4646 Agent: aprender-ca Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nts in ratchet mode (C188) - scripts/tree_reader_tests.txt: regenerated (check_tree_reader_tests.sh --update); its drift was the ws-test 1/2/3 + guard-cargo red. - gate: the MUTANTS-RATCHET block of #4647 (#4646), byte-identical and at the same spot, so this PR's own run does not wait for #4647, and the merge with #4647 is a clean identical change (merge-tree: 0 conflicts). Every check above the block is unchanged; the RESTORE PR #4648 deletes it after crates.io. Refs #4646 Agent: aprender-ca Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
quorum-review (AD-04): NOT agreed (auto_merge: checked=true was_armed=true disarmed=true) {
"ticket": "PMAT-4646",
"head": "6874cbb2a480ba02a448f15ee357f87012fc2728",
"width": 3,
"executor": "agy",
"agreed": false,
"auto_merge": {
"checked": true,
"was_armed": true,
"disarmed": true,
"note": "auto-merge was armed from an earlier round; disarmed before lanes launched"
},
"lanes": [
{
"lane": 1,
"verdict": "PASS",
"findings": 1
},
{
"lane": 2,
"verdict": "FAIL",
"findings": 1
},
{
"lane": 3,
"verdict": "PASS",
"findings": 0
}
]
} |
… per q-4647c finding Pmat-Ticket: PMAT-4646 Agent: aprender-ca Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nd that the mutants rules are off on purpose until #4648 Quorum lane finding (q-4587c, sonnet): the comment named ci/mutants-debt.tsv and scripts/check_mutants_debt_ratchet.sh as the blocking rule, but on a branch without #4647 neither exists. Same block text in #4647/#4587/#4588 so the three still merge without conflict. No check changed. Pmat-Ticket: PMAT-4646 Agent: aprender-ca Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sonnet seat (agent:claude-sonnet-5-5/pr-4647-review), base 00052c0. Verdict FINDINGS (2 warnings, 5 notes; none blocking): PRREV-RATCHET-001 (new-survivor detection from CI results is follow-up, report-only by C188(a)); bashrs 0 errors. Pmat-Ticket: PMAT-4646 Agent: aprender-ca Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nd that the mutants rules are off on purpose until #4648 Quorum lane finding (q-4587c, sonnet): the comment named ci/mutants-debt.tsv and scripts/check_mutants_debt_ratchet.sh as the blocking rule, but on a branch without #4647 neither exists. Same block text in #4647/#4587/#4588 so the three still merge without conflict. No check changed. Pmat-Ticket: PMAT-4646 Agent: aprender-ca Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nd that the mutants rules are off on purpose until #4648 Quorum lane finding (q-4587c, sonnet): the comment named ci/mutants-debt.tsv and scripts/check_mutants_debt_ratchet.sh as the blocking rule, but on a branch without #4647 neither exists. Same block text in #4647/#4587/#4588 so the three still merge without conflict. No check changed. Pmat-Ticket: PMAT-4646 Agent: aprender-ca Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
quorum-review (AD-04): NOT agreed (auto_merge: checked=true was_armed=false disarmed=false) {
"ticket": "PMAT-4646",
"head": "3dd28ae0e20f716f9a876e3da3f468b0321e18d3",
"width": 3,
"executor": "agy",
"agreed": false,
"auto_merge": {
"checked": true,
"was_armed": false,
"disarmed": false,
"note": "auto-merge not armed"
},
"lanes": [
{
"lane": 1,
"verdict": "FAIL",
"findings": 3
},
{
"lane": 2,
"verdict": "FAIL",
"findings": 3
},
{
"lane": 3,
"verdict": "FAIL",
"findings": 2
}
]
} |
…vivor detection is #4649 q-4647d (3 lanes) findings, all fixed: - scripts/check_mutants_debt_ratchet.sh: bashrs lint 0 warnings (quoted assignments, EXIT trap beside RETURN, false positives tagged); 12/12 self-test. - AC3 named a survivor-table section the gate does not have: now 'the mutants section result'. - AC4 now lists the PR's own review artifacts (evidence/, quorum json). - New-survivor RED (C187) is out of scope here, filed as #4649. Pmat-Ticket: PMAT-4646 Agent: aprender-ca Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
quorum-review (AD-04): NOT agreed (auto_merge: checked=true was_armed=false disarmed=false) {
"ticket": "PMAT-4646",
"head": "9c8d426b8a6a8793cbe6cbdfaa1deecbcc3126de",
"width": 3,
"executor": "agy",
"agreed": false,
"auto_merge": {
"checked": true,
"was_armed": false,
"disarmed": false,
"note": "auto-merge not armed"
},
"lanes": [
{
"lane": 1,
"verdict": "PASS",
"findings": 1
},
{
"lane": 2,
"verdict": "FAIL",
"findings": 2
},
{
"lane": 3,
"verdict": "FAIL",
"findings": 3
}
]
} |
…wed diff from review outputs q-4647f (gemini, haiku): the in-tree receipt for 368f05f still carried the pre-fix bashrs finding, so lanes judged the current script by it; and AC4 listed roadmap.yaml and the quorum json, which the quorum brief does not contain. The receipt for this head is committed next; the stale one binds nothing. Pmat-Ticket: PMAT-4646 Agent: aprender-ca Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
quorum-review (AD-04): three PASS — agreed (auto_merge: checked=true was_armed=false disarmed=false) {
"ticket": "PMAT-4646",
"head": "1f2e3598eab98240fc12ccd3bcffd201348b470f",
"width": 3,
"executor": "agy",
"agreed": true,
"auto_merge": {
"checked": true,
"was_armed": false,
"disarmed": false,
"note": "auto-merge not armed"
},
"lanes": [
{
"lane": 1,
"verdict": "PASS",
"findings": 0
},
{
"lane": 2,
"verdict": "PASS",
"findings": 0
},
{
"lane": 3,
"verdict": "PASS",
"findings": 0
}
]
} |
Sonnet seat (agent:claude-sonnet-5-5/pr-4647-review), base 00052c0, verdict FINDINGS, none blocking. Quorum q-4647g AGREED 3/3 (committed after the signer). Pmat-Ticket: PMAT-4646 Agent: aprender-ca Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… — operator C197 None of the five was discharged by its 2026-09-30 expiry, and the D6 andon (scripts/lib/spec_conformance.py) turned every PR and main RED on 2026-10-01. §12 allows exactly one way to move an expiry: an amendment recording who moved it and why. This is that amendment: - Moved by the operator (Noah), ruling C197. Reason: "not delivered for 0.70; carried to 0.71". - Rows 13, 15, 19 and 21 carry the typed date. Row 18 derives it from row 15. - An Appendix D row records the move. - derived_expiries.json was regenerated with --write; exactly the five rows changed. The andon is unchanged. It passes today, and with SPEC_CONFORMANCE_TODAY=2026-10-16 it is RED again (D6). The release notes carry §12's consequence: "NO SPEED DELIVERED: PP-LLAMA-001 rows 13, 15, 18, 19, 21 (re-dated to 2026-10-15)". Refs #4651. Advance warning before expiry: #4652 (post-0.70). Agent: aprender-ca Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
quorum-review (AD-04): NOT agreed (auto_merge: checked=true was_armed=false disarmed=false) {
"ticket": "PMAT-4646,PMAT-4651",
"head": "65d3fb620266e1bf7f80e026201fde16a7e2752f",
"width": 3,
"executor": "agy",
"agreed": false,
"auto_merge": {
"checked": true,
"was_armed": false,
"disarmed": false,
"note": "auto-merge not armed"
},
"lanes": [
{
"lane": 1,
"verdict": "PASS",
"findings": 1
},
{
"lane": 2,
"verdict": "FAIL",
"findings": 1
},
{
"lane": 3,
"verdict": "PASS",
"findings": 0
}
]
} |
… ratchet could never reach clean Quorum on 65d3fb6 (lanes claude-sonnet-5 + gemini-3.1-pro, independently): check_rows fed awk `<<<""` when the debt file had only its header, so the single empty line hit `NF != 5` and the guard went RED on the final shrink. PMAT-4646 requires GREEN on a shrink. - awk skips NF == 0 (rows() already strips blank lines, so no real row is empty) - self-test R13: header-only file vs a 2-row base is GREEN "2 -> 0". RED before the fix (rc=1), GREEN after; deleting the NF==0 line reds R13 only. - PMAT-4646 AC: 13/13 (R1-R13). - Non-author receipt for 65d3fb6 (verdict FINDINGS) committed as evidence. Refs #4646 Agent: aprender-ca Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
quorum-review (AD-04): NOT agreed (auto_merge: checked=true was_armed=false disarmed=false) {
"ticket": "PMAT-4646,PMAT-4651",
"head": "a683b4a5283ed971fd6de1836783ab4246310e21",
"width": 3,
"executor": "agy",
"agreed": false,
"auto_merge": {
"checked": true,
"was_armed": false,
"disarmed": false,
"note": "auto-merge not armed"
},
"lanes": [
{
"lane": 1,
"verdict": "PASS",
"findings": 0
},
{
"lane": 2,
"verdict": "PASS",
"findings": 0
},
{
"lane": 3,
"verdict": "FAIL",
"findings": 5
}
]
} |
…omment states what actually blocks Review of a683b4a (haiku lane FAIL, non-author seat FINDINGS): - 3 sed mutants survived the self-test (PRREV-MUT-002): empty run id accepted, file-name check unanchored, NF<5 instead of NF!=5. New cases R14 (empty run id), R15 (file named mid-string), R16 (6 fields): each mutant now reds exactly its own case; 16/16. - ci.yml comment said "the blocking rule is the shrink-only survivor baseline". It blocks only GROWING the file; new survivors are #4649 and line-shifted rows are #4653 (new issue). Comment and script header say so. Comment-only; no gate changed. - PMAT-4651 AC: the "NO SPEED DELIVERED" line belongs to the v0.70.0 release notes at tag time, not to this PR. - Not changed: the lane's "R13 is RED" claim. Measured at this head: R13 rc=0, self-test 13/13 then 16/16. - Non-author receipt for a683b4a committed as evidence. Refs #4646 #4651 #4653 Agent: aprender-ca Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…AML map 0711f26 broke roadmap-valid (sov.security + sov.gate, CI 36873128678): "roadmap[1098].acceptance_criteria[1]: invalid type: map". The ": " inside the unquoted AC made it a mapping. Quoted it; pmat work validate rc 0, pmat work status PMAT-4646 rc 0, roadmap sorted rc 0. The cop committed without checking validate's rc; it is checked explicitly now. Refs #4646 Agent: aprender-ca Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
quorum-review (AD-04): three PASS — agreed (auto_merge: checked=true was_armed=false disarmed=false) {
"ticket": "PMAT-4646,PMAT-4651",
"head": "bd851543c6d65fb6b09d46425669afa944fddc3c",
"width": 3,
"executor": "agy",
"agreed": true,
"auto_merge": {
"checked": true,
"was_armed": false,
"disarmed": false,
"note": "auto-merge not armed"
},
"lanes": [
{
"lane": 1,
"verdict": "PASS",
"findings": 0
},
{
"lane": 2,
"verdict": "PASS",
"findings": 0
},
{
"lane": 3,
"verdict": "PASS",
"findings": 0
}
]
} |
…S) at bd85154 Agent: aprender-ca Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… (Arm 4 A2 binds on-disk) Agent: aprender-ca Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…read the CI event, Arm 4 read the squash 1. check_mutants_debt_ratchet.sh self-test R11/R12 ran the script with the job's GITHUB_EVENT_NAME. Under merge_group, resolve_base accepts a single parent as the base, so "no nameable base" returned rc=0 and R11 failed in the queue only (runs 36888741624, 36890959980). The temp repo is not a CI checkout: the subprocess now runs with -u GITHUB_EVENT_NAME. Proof: GITHUB_EVENT_NAME=merge_group --self-test rc 1 before, rc 0 after; pull_request and unset rc 0 both. 2. pr-review-quorum.yml passed the merge_group head (the queue squash) as ARM4_PR_HEAD_SHA. The signer posts pr-review-signature on the PR head, so every queued PR read UNSIGNED (#4632 run 36602609576; #4647 runs 36888741583, 36890960012). ARM4_PR_HEAD_SHA is now the pull_request event's head (empty on merge_group), and Arm 4 reads the PR head from the pulls API; the job gains pull-requests: read. Proof on queue sha 79594c8 against the live API: old wiring rc 1 UNSIGNED; new wiring rc 0, signature on 163c660 verifies, binds pr=4647 pid=41b11c0a, A3 positive control still rejects a corrupt sig. No gate loosened: same key, same trusted-comment binding, same A2 diff bind. Pmat-Ticket: PMAT-4646 Agent: aprender-27 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
quorum-review (AD-04): NOT agreed (auto_merge: checked=true was_armed=false disarmed=false) {
"ticket": "PMAT-4646,PMAT-4651",
"head": "b6ccc0354fa399aaddc8afbc73682f12485bf948",
"width": 3,
"executor": "agy",
"agreed": false,
"auto_merge": {
"checked": true,
"was_armed": false,
"disarmed": false,
"note": "auto-merge not armed"
},
"lanes": [
{
"lane": 1,
"verdict": "FAIL",
"findings": 1
},
{
"lane": 2,
"verdict": "FAIL",
"findings": 1
},
{
"lane": 3,
"verdict": "PASS",
"findings": 0
}
]
} |
… fix (round-5 quorum failed it as out-of-ticket) Operator 2026-09-28 14:52Z pre-approves gate-keeping workflow edits with red/green proof; cop decision c213 (option a). Cites runs 36888741583/36890960012 and queue sha 79594c8. Pmat-Ticket: PMAT-4646 Agent: aprender-r5 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
quorum-review (AD-04): three PASS — agreed (auto_merge: checked=true was_armed=false disarmed=false) {
"ticket": "PMAT-4646,PMAT-4651",
"head": "ce8036ebc14c1d54f90507c8b2686f993897a699",
"width": 3,
"executor": "agy",
"agreed": true,
"auto_merge": {
"checked": true,
"was_armed": false,
"disarmed": false,
"note": "auto-merge not armed"
},
"lanes": [
{
"lane": 1,
"verdict": "PASS",
"findings": 0
},
{
"lane": 2,
"verdict": "PASS",
"findings": 0
},
{
"lane": 3,
"verdict": "PASS",
"findings": 0
}
]
} |
…judged by the #4647 ratchet section The intel mutants-shard runs hit the 720m timeout on every past run, so the A' survivor table cannot finish. With no head refs listed, the mutants section is report-only and ci/mutants-debt.tsv stays shrink-only (#4647). Prepared LOCAL ONLY, pending the operator's choice. Pmat-Ticket: PMAT-4502 Agent: aprender-07 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…judged by the #4647 ratchet section The intel mutants-shard runs hit the 720m timeout on every past run, so the A' survivor table cannot finish. With no head refs listed, the mutants section is report-only and ci/mutants-debt.tsv stays shrink-only (#4647). Prepared LOCAL ONLY, pending the operator's choice. Also: scripts/tree_reader_tests.txt gains the derived row for aprender-review-experiment (build_corpus), the crate this PR adds -- CI run 36927004090 workspace-test-shard (3) drift; regenerated with check_tree_reader_tests.sh --update. Pmat-Ticket: PMAT-4502 Agent: aprender-07 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fa7a1e6 said 65d3fb6 was on origin/fold/0.70 and not yet on main. That checked ancestry, not content. Main already carries the re-date: row 15 "Expires 2026-10-15 (was 2026-09-30)" and Appendix D row 3.1 of 2026-10-01 (operator C197), through 430be65 (#4647). Agent: la-0.71 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Closes #4646 (PMAT-4646). Closes #4651 (PMAT-4651, folded in by operator ruling C197).
keep-open: #4652 is the post-0.70 follow-up (a 7-day warning before an expiry), not done here. #4587 #4588 #4603 #4614 #4620 #4630 #4640 are the release-batch PRs this unblocks, cited for merge compatibility; they are pull requests, not issues this PR resolves. Operator rulings C187(a)/C188(a), 2026-10-01: the mutants gate moves to ratchet mode. No ruleset change.
What changes
ci / gate: aMUTANTS-RATCHETblock prints the mutation results and exits 0 before the mutants rules. Every check above it is unchanged: workspace-test, guard-tree, guard-cargo, sov.gate and determinism-compare. The old rule blocks stay in place byte for byte, so the RESTORE PR is a pure deletion.ci/mutants-debt.tsv: the baseline, with 590 known survivors (558 MissedMutant + 32 Timeout; sources in the file header): file, mutant, sha and run id. They come from theoutcomes.jsonartifacts of split(#4502 PR-1): contracts layer — PVL rows PMAT-4199/4200/4347, GH-4122, ONT-4c4 (#4069) #4587 run 36725778557 and split(#4502 PR-2): ONT-10 rest — everything in #4502 outside the contracts layer (stacked on #4587) #4588 run 36703131059. Shards without outcomes are not measured and are not listed.scripts/check_mutants_debt_ratchet.sh: auto-discovered by guard-tree. The blocking rule is that the debt file is shrink-only against the merge base.Falsifiers (self-test, 10 cases, each pinned to the rule that decides)
R1 grow → RED · R2 missing → RED · R3 shrink → GREEN · R4 4-field row → RED · R5 swap → RED · R6 unchanged → GREEN · R7 duplicate → RED · R8 unresolvable base → RED · R9 Caught outcome → RED · R10 introduction → GREEN.
Mutation check: 7/7 planted mutants of the guard turn the self-test RED.
Merge compatibility
A 3-way
git merge-fileofci.ymlwith #4587, #4588, #4620 and #4603 gives 0 conflicts. #4630, #4614 and #4640 do not touch ci.yml. A simulated merged gate with mutants/table red gives rc 0; with guard-tree, workspace-test or determinism red it gives rc 1.Restore: the follow-up PR deletes the block after crates.io 0.70 and re-arms new-survivor = RED.
Agent: aprender-ca
Folded in: PMAT-4651, re-date PP-LLAMA-001 §12 (operator C197, commit 65d3fb6)
On 2026-10-01 the D6 date andon turned every PR and main RED. The cause: rows 13, 15, 19 and 21 expired 2026-09-30 undelivered, and row 18's date is derived from row 15. The rows are re-dated to 2026-10-15 using the spec's own procedure (an Appendix D amendment). Moved by: the operator (Noah). Reason: "not delivered for 0.70; carried to 0.71". The check (
scripts/lib/spec_conformance.py) is unchanged.derived_expiries.jsonwas regenerated with--write; exactly 5 rows changed.bash scripts/spec_conformance.shexits 0.SPEC_CONFORMANCE_TODAY=2026-10-16it is RED (D6).This rides in #4647's push so the release costs one CI cycle, not two.
🤖 Generated with Claude Code