From 532c92ae5ab408ba3bd123c865f6fb2d3914ed92 Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Sat, 19 Sep 2026 20:37:02 +0200 Subject: [PATCH 1/4] =?UTF-8?q?=C2=A711.1=20+=20check=5Fpr=5Font=5Fdelta:?= =?UTF-8?q?=20the=20docs/specifications=20sink=20is=20the=20release=20spec?= =?UTF-8?q?,=20not=20every=20design=20spec;=20the=20predicate=20gets=20its?= =?UTF-8?q?=20self-test=20(Refs=20PMAT-3535)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Measured 2026-09-19 (aprender-ba): #3499, #3420 and #3424 — a guard-regex fix and two design specs — RED on guard-tree with `FAIL no ont-delta: line`, each touching one file under docs/specifications/ and nothing else §11.1 names. The guard was FAITHFUL to the sentence, which named `docs/specifications/**` whole as a prose sink. A design spec is the finding's home, not a sink a sweep writes into; the sink under that directory is the train's own spec, where sweep findings are recorded. Spec and guard move together: §11.1 names `docs/specifications/APR-RELEASE-001-*.md` and says a `*-MASTER.md` design spec is not a sweep; ONT_SWEEP_SINKS follows (the file, not the directory). The predicate half had no self-test (all 15 rows fed check_body_text). predicate_self_test — aprender-ba's leg, 10 rows: crate source, a contract, ci.yml are not sweeps; night.yml, README, CLAUDE, the registry, a derived baseline, the release spec are — chains into --self-test. The falsifying row "a DESIGN spec is not a sweep" is want=1 and goes RED (want=1 got=0) with the directory-wide sink restored, so the leg discriminates. --self-test 15 + 10 passed; bashrs lint 0 errors. Pmat-Ticket: PMAT-3535 Co-Authored-By: Claude Fable 5.1 --- docs/roadmaps/entries/PMAT-3535.yaml | 17 +++++++ docs/roadmaps/roadmap.yaml | 17 +++++++ .../APR-RELEASE-001-train-and-build-kaizen.md | 5 +- scripts/check_pr_ont_delta.sh | 50 +++++++++++++++++-- 4 files changed, 85 insertions(+), 4 deletions(-) create mode 100644 docs/roadmaps/entries/PMAT-3535.yaml diff --git a/docs/roadmaps/entries/PMAT-3535.yaml b/docs/roadmaps/entries/PMAT-3535.yaml new file mode 100644 index 0000000000..3f203e1802 --- /dev/null +++ b/docs/roadmaps/entries/PMAT-3535.yaml @@ -0,0 +1,17 @@ +- id: PMAT-3535 + github_issue: 3535 + item_type: task + title: '§11.1 + check_pr_ont_delta: the docs/specifications sink is the release spec, not every design spec — three maintainer PRs red for a delta that does not exist; predicate self-test with the falsifying design-spec row (Refs aprender#3535)' + status: planned + priority: medium + assigned_to: null + created: 2026-09-19T18:35:25Z + updated: 2026-09-19T18:35:25Z + spec: null + acceptance_criteria: + - 'aprender-ba measured #3499/#3420/#3424 red on ont-delta; the guard is faithful to a sentence that names the whole directory. Sentence and ONT_SWEEP_SINKS move together; predicate_self_test lands with ''a DESIGN spec is not a sweep'' at want=1.' + phases: [] + subtasks: [] + estimated_effort: null + labels: [] + notes: null diff --git a/docs/roadmaps/roadmap.yaml b/docs/roadmaps/roadmap.yaml index 9454eacbd3..e54d1854c3 100644 --- a/docs/roadmaps/roadmap.yaml +++ b/docs/roadmaps/roadmap.yaml @@ -18647,3 +18647,20 @@ roadmap: - orch:fable - orch-basis:state notes: null +- id: PMAT-3535 + github_issue: 3535 + item_type: task + title: '§11.1 + check_pr_ont_delta: the docs/specifications sink is the release spec, not every design spec — three maintainer PRs red for a delta that does not exist; predicate self-test with the falsifying design-spec row (Refs aprender#3535)' + status: planned + priority: medium + assigned_to: null + created: 2026-09-19T18:35:25Z + updated: 2026-09-19T18:35:25Z + spec: null + acceptance_criteria: + - 'aprender-ba measured #3499/#3420/#3424 red on ont-delta; the guard is faithful to a sentence that names the whole directory. Sentence and ONT_SWEEP_SINKS move together; predicate_self_test lands with ''a DESIGN spec is not a sweep'' at want=1.' + phases: [] + subtasks: [] + estimated_effort: null + labels: [] + notes: null diff --git a/docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md b/docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md index 339a2f3c9e..67ad7c397b 100644 --- a/docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md +++ b/docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md @@ -767,7 +767,10 @@ A **surface** is anything the train already sweeps: the `(crate, feature)` matri examples (§4.2), the `apr` command registry, `contracts/`, the published docs, `.github/workflows/`, model files, and the three triage surfaces (§6.2). A **sweep PR** is one that writes a finding into a prose sink — operationally, one touching any of `.github/workflows/night.yml`, -`docs/specifications/**`, `contracts/apr-cli-commands-v1.yaml`, `README.md`, `CLAUDE.md`, or a +`docs/specifications/APR-RELEASE-001-*.md` (this spec — the sink the train writes its findings into; a +*design* spec such as a `*-MASTER.md` is the finding's home, not a sink, and touching one is not a +sweep — aprender#3535, three maintainer PRs red for a delta that did not exist), +`contracts/apr-cli-commands-v1.yaml`, `README.md`, `CLAUDE.md`, or a known-red list anywhere. A **delta** is one of exactly four things, any one of which closes the sweep: diff --git a/scripts/check_pr_ont_delta.sh b/scripts/check_pr_ont_delta.sh index bffd96f94b..ca2f0e62e8 100755 --- a/scripts/check_pr_ont_delta.sh +++ b/scripts/check_pr_ont_delta.sh @@ -39,10 +39,12 @@ usage() { exit 2 } -# The sinks §11.1 names, verbatim. A `docs/specifications/` entry is a PREFIX. +# The sinks §11.1 names, verbatim. Under docs/specifications/ the sink is the release spec ITSELF — +# the train writes its findings there — never a design spec (aprender#3535: three maintainer PRs, +# each touching one design document, were red for a delta that did not exist). ONT_SWEEP_SINKS=( ".github/workflows/night.yml" - "docs/specifications/" + "docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md" "contracts/apr-cli-commands-v1.yaml" "README.md" "CLAUDE.md" @@ -166,10 +168,52 @@ trailer' [ "$fail" -eq 0 ] } +# The PREDICATE half had no self-test: every row above feeds check_body_text, so +# sweep_reason could have classified every PR as a sweep (or none) and the +# self-test would stay green. Measured 2026-09-19: three PRs by a maintainer +# (#3499 a guard-regex fix, #3420 and #3424 design specs) were RED on this +# guard, every one of them because `docs/specifications/**` is a prefix sink +# and §11.1 says so verbatim. A design spec that IS the finding's home is not +# "writing a finding into a prose sink"; whether the sentence or the row moves +# is a §11.1 amendment, and the row below is the falsifying case either way. +predicate_row() { # predicate_row NAME WANT CHANGED_PATH (want: 0 = sweep, 1 = not a sweep) + local name="$1" want="$2" path="$3" tmp rc + tmp="$(mktemp -d)" + case "$tmp" in /tmp/*|/var/tmp/*) : ;; *) printf 'NO-GO: mktemp -d gave %s\n' "$tmp" >&2; return 2 ;; esac + printf '%s\n' "$path" > "$tmp/c.txt" + set +e; sweep_reason "$tmp/c.txt" >/dev/null 2>&1; rc=$?; set -e + [ -n "$tmp" ] && [ -d "$tmp" ] && rm -rf "$tmp" + if [ "$rc" -eq "$want" ]; then printf ' ok %-46s want=%s\n' "$name" "$want"; return 0 + else printf ' FAIL %-46s want=%s got=%s\n' "$name" "$want" "$rc"; return 1; fi +} + +predicate_self_test() { + local pass=0 fail=0 + printf 'check_pr_ont_delta predicate self-test (sweep_reason)\n' + prow() { if predicate_row "$@"; then pass=$((pass+1)); else fail=$((fail+1)); fi; } + prow "crate source is not a sweep" 1 'crates/aprender-core/src/lib.rs' + prow "a contract yaml is not a sweep" 1 'contracts/tensor-layout-v1.yaml' + prow "a workflow other than night is not" 1 '.github/workflows/ci.yml' + prow "night.yml is a prose sink" 0 '.github/workflows/night.yml' + prow "README.md is a prose sink" 0 'README.md' + prow "CLAUDE.md is a prose sink" 0 'CLAUDE.md' + prow "apr-cli-commands registry is a sink" 0 'contracts/apr-cli-commands-v1.yaml' + prow "a known-red baseline is DERIVED" 0 'scripts/cb200_baseline.txt' + prow "the release spec itself is a sink" 0 'docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md' + # THE FALSIFYING ROW. A design spec (PP-QUANT-001, #3420) carries no finding + # from a sweep; under the §11.1 sentence as first written it WAS a sweep (want=0) + # and the guard redded a maintainer's spec PR for a line about a delta that + # did not exist. The amendment (aprender#3535) makes this row want=1; restoring the + # directory-wide sink turns it RED again, which is the proof it discriminates. + prow "a DESIGN spec is not a sweep (§11.1 amended)" 1 'docs/specifications/PP-QUANT-001-MASTER.md' + printf 'predicate self-test: %s passed, %s failed\n' "$pass" "$fail" + [ "$fail" -eq 0 ] +} + main() { local body="" changed="" reason if [ $# -eq 0 ] || [ "${1:-}" = "--self-test" ]; then - self_test; return $? + self_test && predicate_self_test; return $? fi while [ $# -gt 0 ]; do case "$1" in From db63b816e64e87d8eae2f70f17fe8329c83b5546 Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Sat, 19 Sep 2026 20:47:26 +0200 Subject: [PATCH 2/4] =?UTF-8?q?=C2=A711.1=20and=20the=20guard=20name=20the?= =?UTF-8?q?=20same=20bytes;=20the=20design-spec=20row=20is=20named=20as=20?= =?UTF-8?q?the=20criterion=20names=20it=20(Refs=20PMAT-3535)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Quorum round 1, lane 1: the spec said APR-RELEASE-001-*.md while ONT_SWEEP_SINKS said the exact file — 'verbatim' means the same string in both; and the row name carried a suffix the criterion did not. Criteria rewritten from the diff. Pmat-Ticket: PMAT-3535 Co-Authored-By: Claude Fable 5.1 --- docs/roadmaps/entries/PMAT-3535.yaml | 3 ++- docs/roadmaps/roadmap.yaml | 3 ++- docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md | 2 +- scripts/check_pr_ont_delta.sh | 2 +- 4 files changed, 6 insertions(+), 4 deletions(-) diff --git a/docs/roadmaps/entries/PMAT-3535.yaml b/docs/roadmaps/entries/PMAT-3535.yaml index 3f203e1802..058332018c 100644 --- a/docs/roadmaps/entries/PMAT-3535.yaml +++ b/docs/roadmaps/entries/PMAT-3535.yaml @@ -9,7 +9,8 @@ updated: 2026-09-19T18:35:25Z spec: null acceptance_criteria: - - 'aprender-ba measured #3499/#3420/#3424 red on ont-delta; the guard is faithful to a sentence that names the whole directory. Sentence and ONT_SWEEP_SINKS move together; predicate_self_test lands with ''a DESIGN spec is not a sweep'' at want=1.' + - 'APR-RELEASE-001 §11.1 names docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md (the release spec, where the train writes its findings) as the docs/specifications sink and says a *-MASTER.md design spec is the finding''s home, not a sink; scripts/check_pr_ont_delta.sh ONT_SWEEP_SINKS carries that exact path in place of the docs/specifications/ prefix — the spec sentence and the guard list name the same bytes.' + - 'predicate_self_test (10 rows) chains into --self-test: crate source, a contract yaml and ci.yml are not sweeps; night.yml, README.md, CLAUDE.md, the apr-cli-commands registry, a derived baseline and the release spec are; the row "a DESIGN spec is not a sweep" (docs/specifications/PP-QUANT-001-MASTER.md) is want=1 and goes RED (got=0) when the directory-wide sink is restored. --self-test 15 + 10 passed; bashrs lint 0 errors; this PR''s own body passes check_pr_ont_delta.sh (ont-delta: none) and check_pr_closes_issue.sh.' phases: [] subtasks: [] estimated_effort: null diff --git a/docs/roadmaps/roadmap.yaml b/docs/roadmaps/roadmap.yaml index e54d1854c3..fcf273278f 100644 --- a/docs/roadmaps/roadmap.yaml +++ b/docs/roadmaps/roadmap.yaml @@ -18658,7 +18658,8 @@ roadmap: updated: 2026-09-19T18:35:25Z spec: null acceptance_criteria: - - 'aprender-ba measured #3499/#3420/#3424 red on ont-delta; the guard is faithful to a sentence that names the whole directory. Sentence and ONT_SWEEP_SINKS move together; predicate_self_test lands with ''a DESIGN spec is not a sweep'' at want=1.' + - 'APR-RELEASE-001 §11.1 names docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md (the release spec, where the train writes its findings) as the docs/specifications sink and says a *-MASTER.md design spec is the finding''s home, not a sink; scripts/check_pr_ont_delta.sh ONT_SWEEP_SINKS carries that exact path in place of the docs/specifications/ prefix — the spec sentence and the guard list name the same bytes.' + - 'predicate_self_test (10 rows) chains into --self-test: crate source, a contract yaml and ci.yml are not sweeps; night.yml, README.md, CLAUDE.md, the apr-cli-commands registry, a derived baseline and the release spec are; the row "a DESIGN spec is not a sweep" (docs/specifications/PP-QUANT-001-MASTER.md) is want=1 and goes RED (got=0) when the directory-wide sink is restored. --self-test 15 + 10 passed; bashrs lint 0 errors; this PR''s own body passes check_pr_ont_delta.sh (ont-delta: none) and check_pr_closes_issue.sh.' phases: [] subtasks: [] estimated_effort: null diff --git a/docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md b/docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md index 67ad7c397b..b96f30d350 100644 --- a/docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md +++ b/docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md @@ -767,7 +767,7 @@ A **surface** is anything the train already sweeps: the `(crate, feature)` matri examples (§4.2), the `apr` command registry, `contracts/`, the published docs, `.github/workflows/`, model files, and the three triage surfaces (§6.2). A **sweep PR** is one that writes a finding into a prose sink — operationally, one touching any of `.github/workflows/night.yml`, -`docs/specifications/APR-RELEASE-001-*.md` (this spec — the sink the train writes its findings into; a +`docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md` (this spec — the sink the train writes its findings into; a *design* spec such as a `*-MASTER.md` is the finding's home, not a sink, and touching one is not a sweep — aprender#3535, three maintainer PRs red for a delta that did not exist), `contracts/apr-cli-commands-v1.yaml`, `README.md`, `CLAUDE.md`, or a diff --git a/scripts/check_pr_ont_delta.sh b/scripts/check_pr_ont_delta.sh index ca2f0e62e8..73c4c243d1 100755 --- a/scripts/check_pr_ont_delta.sh +++ b/scripts/check_pr_ont_delta.sh @@ -205,7 +205,7 @@ predicate_self_test() { # and the guard redded a maintainer's spec PR for a line about a delta that # did not exist. The amendment (aprender#3535) makes this row want=1; restoring the # directory-wide sink turns it RED again, which is the proof it discriminates. - prow "a DESIGN spec is not a sweep (§11.1 amended)" 1 'docs/specifications/PP-QUANT-001-MASTER.md' + prow "a DESIGN spec is not a sweep" 1 'docs/specifications/PP-QUANT-001-MASTER.md' printf 'predicate self-test: %s passed, %s failed\n' "$pass" "$fail" [ "$fail" -eq 0 ] } From ad99389f066c1e77f33e54c2591f8f2cd96e40ef Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Sat, 19 Sep 2026 20:57:43 +0200 Subject: [PATCH 3/4] check_pr_ont_delta: say in predicate_row that sweep_reason takes a FILE of paths (Refs PMAT-3535) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Quorum round 2, lane 3 (flash) claimed, "measured", that predicate_row hands sweep_reason the temp path itself so six rows fail with got=1. It does not: sweep_reason reads the changed paths OUT of the file it is given (`while IFS= read -r f … done < "$changed"`), which is the shape main() hands it from --changed; predicate_row writes one path into a temp file and passes the file. Measured on this head, `bash scripts/check_pr_ont_delta.sh --self-test`: self-test 15 passed, 0 failed; predicate self-test 10 passed, 0 failed — and with the directory-wide sink restored exactly one row (a DESIGN spec is not a sweep) goes want=1 got=0. The claim was a misread of the function's contract; the comment now states that contract where the misread happened, so the next reader does not need to trace it. Pmat-Ticket: PMAT-3535 Co-Authored-By: Claude Fable 5.1 --- scripts/check_pr_ont_delta.sh | 3 +++ 1 file changed, 3 insertions(+) diff --git a/scripts/check_pr_ont_delta.sh b/scripts/check_pr_ont_delta.sh index 73c4c243d1..6a2f462a77 100755 --- a/scripts/check_pr_ont_delta.sh +++ b/scripts/check_pr_ont_delta.sh @@ -176,6 +176,9 @@ trailer' # and §11.1 says so verbatim. A design spec that IS the finding's home is not # "writing a finding into a prose sink"; whether the sentence or the row moves # is a §11.1 amendment, and the row below is the falsifying case either way. +# sweep_reason takes a FILE of changed paths, one per line (the same shape main() hands it from +# --changed), and reads the paths out of it — so each row writes its one path into a temp file and +# passes the FILE. Quorum round 2 misread this as passing the temp path itself; --self-test is the measurement. predicate_row() { # predicate_row NAME WANT CHANGED_PATH (want: 0 = sweep, 1 = not a sweep) local name="$1" want="$2" path="$3" tmp rc tmp="$(mktemp -d)" From 6206d8ebca65a553451bb6581b368233249ca881 Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Sat, 19 Sep 2026 21:08:49 +0200 Subject: [PATCH 4/4] audit: quorum-PMAT-3535 round 3 AGREED 3/3 on ad99389f0 (Refs PMAT-3535) Round 1: two exactness findings (the row name, the sink spelled the same in spec and guard), both taken. Round 2: one lane misread sweep_reason's file argument; refuted by the measured self-test and the contract stated in a comment. Pmat-Ticket: PMAT-3535 Co-Authored-By: Claude Fable 5.1 --- docs/audits/quorum-PMAT-3535.json | 84 +++++++++++++++++++++++++++++++ 1 file changed, 84 insertions(+) create mode 100644 docs/audits/quorum-PMAT-3535.json diff --git a/docs/audits/quorum-PMAT-3535.json b/docs/audits/quorum-PMAT-3535.json new file mode 100644 index 0000000000..e2c624e8ad --- /dev/null +++ b/docs/audits/quorum-PMAT-3535.json @@ -0,0 +1,84 @@ +{ + "ticket": "PMAT-3535", + "base": "origin/main", + "base_resolved": "origin/main", + "base_note": "no origin/origin/main exists; judged against the local ref", + "head": "ad99389f066c1e77f33e54c2591f8f2cd96e40ef", + "diff_sha256": "be6fea3c60ba4472b8771de77a229826d95ce1eac01f98ee1a1d193729994829", + "width": 3, + "executor": "agy", + "prompt_mode": "inline", + "prompt_bytes": 10095, + "author": { + "model": "claude-fable-5-1", + "family": "claude", + "source": "flag" + }, + "agreed": true, + "lanes": [ + { + "lane": 1, + "status": "SUCCESS", + "verdict": "PASS", + "summary": "The diff perfectly aligns with PMAT-3535. It updates the spec and the script to narrow the `docs/specifications/` sink to exactly the release spec (`APR-RELEASE-001-train-and-build-kaizen.md`), preventing design specs from incorrectly triggering as sweeps. It also adds the requested 10-row `predicate_self_test` to `check_pr_ont_delta.sh` which explicitly checks the falsifying design-spec row and correctly chains into the `--self-test` execution.", + "findings": [], + "raw_bytes": 2145, + "err_bytes": 0, + "envelope_status": "SUCCESS", + "verdict_source": "structured_output", + "grounding_check": "parity", + "model": "gemini-3.1-pro-high", + "model_measured": "gemini-3.1-pro-high", + "model_source": "measured", + "family": "gemini" + }, + { + "lane": 2, + "status": "SUCCESS", + "verdict": "PASS", + "summary": "I have reviewed the diff against PMAT-3535. The diff perfectly implements the requested changes:\n1. `APR-RELEASE-001-train-and-build-kaizen.md` is updated to clarify that the release spec itself is the sink, not any design spec.\n2. `check_pr_ont_delta.sh` replaces the `docs/specifications/` prefix with the exact path `docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md`.\n3. `predicate_self_test` is implemented with exactly the 10 requested rows (verifying crate source, contract yaml, ci.yml, night.yml, README.md, CLAUDE.md, registry, derived baseline, release spec, and the falsifying design spec `PP-QUANT-001-MASTER.md` as `want=1`), and is chained correctly from `--self-test`.\nThe bash code appears correct and well-formed. No grounds for refutation found.", + "findings": [], + "raw_bytes": 2828, + "err_bytes": 0, + "envelope_status": "SUCCESS", + "verdict_source": "structured_output", + "grounding_check": "parity", + "model": "gemini-3.1-pro-low", + "model_measured": "gemini-3.1-pro-low", + "model_source": "measured", + "family": "gemini" + }, + { + "lane": 3, + "status": "SUCCESS", + "verdict": "PASS", + "summary": "Reviewed diff for PMAT-3535 (Lane 3 of 3). The diff fully implements all ticket acceptance criteria: APR-RELEASE-001 §11.1 and ONT_SWEEP_SINKS in check_pr_ont_delta.sh have been updated to target the specific release spec docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md rather than the directory-wide prefix; predicate_self_test (10 rows) is added and chained into --self-test in main(); and roadmap entries have been added. No gates were weakened, no extra unrequested changes were made, and all claims are backed by the diff. Verdict: PASS.", + "findings": [], + "raw_bytes": 2379, + "err_bytes": 0, + "envelope_status": "SUCCESS", + "verdict_source": "structured_output", + "grounding_check": "parity", + "model": "gemini-3.6-flash-low", + "model_measured": "gemini-3.6-flash-low", + "model_source": "measured", + "family": "gemini" + } + ], + "dissent": [], + "dedup": [], + "uncovered": [], + "coverage_source": "lanes", + "partial": false, + "partial_reasons": [], + "auto_merge": { + "checked": true, + "was_armed": false, + "disarmed": false, + "note": "auto-merge not armed" + }, + "lint": { + "ok": true, + "output": "receipt complete: kind=artifact lanes=3 author=claude-fable-5-1/claude" + } +}