Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions docs/audits/quorum-PMAT-3535.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
{
"ticket": "PMAT-3535",
"base": "origin/main",
"base_resolved": "origin/main",
"base_note": "no origin/origin/main exists; judged against the local ref",
"head": "ad99389f066c1e77f33e54c2591f8f2cd96e40ef",
"diff_sha256": "be6fea3c60ba4472b8771de77a229826d95ce1eac01f98ee1a1d193729994829",
"width": 3,
"executor": "agy",
"prompt_mode": "inline",
"prompt_bytes": 10095,
"author": {
"model": "claude-fable-5-1",
"family": "claude",
"source": "flag"
},
"agreed": true,
"lanes": [
{
"lane": 1,
"status": "SUCCESS",
"verdict": "PASS",
"summary": "The diff perfectly aligns with PMAT-3535. It updates the spec and the script to narrow the `docs/specifications/` sink to exactly the release spec (`APR-RELEASE-001-train-and-build-kaizen.md`), preventing design specs from incorrectly triggering as sweeps. It also adds the requested 10-row `predicate_self_test` to `check_pr_ont_delta.sh` which explicitly checks the falsifying design-spec row and correctly chains into the `--self-test` execution.",
"findings": [],
"raw_bytes": 2145,
"err_bytes": 0,
"envelope_status": "SUCCESS",
"verdict_source": "structured_output",
"grounding_check": "parity",
"model": "gemini-3.1-pro-high",
"model_measured": "gemini-3.1-pro-high",
"model_source": "measured",
"family": "gemini"
},
{
"lane": 2,
"status": "SUCCESS",
"verdict": "PASS",
"summary": "I have reviewed the diff against PMAT-3535. The diff perfectly implements the requested changes:\n1. `APR-RELEASE-001-train-and-build-kaizen.md` is updated to clarify that the release spec itself is the sink, not any design spec.\n2. `check_pr_ont_delta.sh` replaces the `docs/specifications/` prefix with the exact path `docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md`.\n3. `predicate_self_test` is implemented with exactly the 10 requested rows (verifying crate source, contract yaml, ci.yml, night.yml, README.md, CLAUDE.md, registry, derived baseline, release spec, and the falsifying design spec `PP-QUANT-001-MASTER.md` as `want=1`), and is chained correctly from `--self-test`.\nThe bash code appears correct and well-formed. No grounds for refutation found.",
"findings": [],
"raw_bytes": 2828,
"err_bytes": 0,
"envelope_status": "SUCCESS",
"verdict_source": "structured_output",
"grounding_check": "parity",
"model": "gemini-3.1-pro-low",
"model_measured": "gemini-3.1-pro-low",
"model_source": "measured",
"family": "gemini"
},
{
"lane": 3,
"status": "SUCCESS",
"verdict": "PASS",
"summary": "Reviewed diff for PMAT-3535 (Lane 3 of 3). The diff fully implements all ticket acceptance criteria: APR-RELEASE-001 §11.1 and ONT_SWEEP_SINKS in check_pr_ont_delta.sh have been updated to target the specific release spec docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md rather than the directory-wide prefix; predicate_self_test (10 rows) is added and chained into --self-test in main(); and roadmap entries have been added. No gates were weakened, no extra unrequested changes were made, and all claims are backed by the diff. Verdict: PASS.",
"findings": [],
"raw_bytes": 2379,
"err_bytes": 0,
"envelope_status": "SUCCESS",
"verdict_source": "structured_output",
"grounding_check": "parity",
"model": "gemini-3.6-flash-low",
"model_measured": "gemini-3.6-flash-low",
"model_source": "measured",
"family": "gemini"
}
],
"dissent": [],
"dedup": [],
"uncovered": [],
"coverage_source": "lanes",
"partial": false,
"partial_reasons": [],
"auto_merge": {
"checked": true,
"was_armed": false,
"disarmed": false,
"note": "auto-merge not armed"
},
"lint": {
"ok": true,
"output": "receipt complete: kind=artifact lanes=3 author=claude-fable-5-1/claude"
}
}
18 changes: 18 additions & 0 deletions docs/roadmaps/entries/PMAT-3535.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
- id: PMAT-3535
github_issue: 3535
item_type: task
title: '§11.1 + check_pr_ont_delta: the docs/specifications sink is the release spec, not every design spec — three maintainer PRs red for a delta that does not exist; predicate self-test with the falsifying design-spec row (Refs aprender#3535)'
status: planned
priority: medium
assigned_to: null
created: 2026-09-19T18:35:25Z
updated: 2026-09-19T18:35:25Z
spec: null
acceptance_criteria:
- 'APR-RELEASE-001 §11.1 names docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md (the release spec, where the train writes its findings) as the docs/specifications sink and says a *-MASTER.md design spec is the finding''s home, not a sink; scripts/check_pr_ont_delta.sh ONT_SWEEP_SINKS carries that exact path in place of the docs/specifications/ prefix — the spec sentence and the guard list name the same bytes.'
- 'predicate_self_test (10 rows) chains into --self-test: crate source, a contract yaml and ci.yml are not sweeps; night.yml, README.md, CLAUDE.md, the apr-cli-commands registry, a derived baseline and the release spec are; the row "a DESIGN spec is not a sweep" (docs/specifications/PP-QUANT-001-MASTER.md) is want=1 and goes RED (got=0) when the directory-wide sink is restored. --self-test 15 + 10 passed; bashrs lint 0 errors; this PR''s own body passes check_pr_ont_delta.sh (ont-delta: none) and check_pr_closes_issue.sh.'
phases: []
subtasks: []
estimated_effort: null
labels: []
notes: null
18 changes: 18 additions & 0 deletions docs/roadmaps/roadmap.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18647,3 +18647,21 @@ roadmap:
- orch:fable
- orch-basis:state
notes: null
- id: PMAT-3535
github_issue: 3535
item_type: task
title: '§11.1 + check_pr_ont_delta: the docs/specifications sink is the release spec, not every design spec — three maintainer PRs red for a delta that does not exist; predicate self-test with the falsifying design-spec row (Refs aprender#3535)'
status: planned
priority: medium
assigned_to: null
created: 2026-09-19T18:35:25Z
updated: 2026-09-19T18:35:25Z
spec: null
acceptance_criteria:
- 'APR-RELEASE-001 §11.1 names docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md (the release spec, where the train writes its findings) as the docs/specifications sink and says a *-MASTER.md design spec is the finding''s home, not a sink; scripts/check_pr_ont_delta.sh ONT_SWEEP_SINKS carries that exact path in place of the docs/specifications/ prefix — the spec sentence and the guard list name the same bytes.'
- 'predicate_self_test (10 rows) chains into --self-test: crate source, a contract yaml and ci.yml are not sweeps; night.yml, README.md, CLAUDE.md, the apr-cli-commands registry, a derived baseline and the release spec are; the row "a DESIGN spec is not a sweep" (docs/specifications/PP-QUANT-001-MASTER.md) is want=1 and goes RED (got=0) when the directory-wide sink is restored. --self-test 15 + 10 passed; bashrs lint 0 errors; this PR''s own body passes check_pr_ont_delta.sh (ont-delta: none) and check_pr_closes_issue.sh.'
phases: []
subtasks: []
estimated_effort: null
labels: []
notes: null
Original file line number Diff line number Diff line change
Expand Up @@ -767,7 +767,10 @@ A **surface** is anything the train already sweeps: the `(crate, feature)` matri
examples (§4.2), the `apr` command registry, `contracts/`, the published docs, `.github/workflows/`,
model files, and the three triage surfaces (§6.2). A **sweep PR** is one that writes a finding into
a prose sink — operationally, one touching any of `.github/workflows/night.yml`,
`docs/specifications/**`, `contracts/apr-cli-commands-v1.yaml`, `README.md`, `CLAUDE.md`, or a
`docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md` (this spec — the sink the train writes its findings into; a
*design* spec such as a `*-MASTER.md` is the finding's home, not a sink, and touching one is not a
sweep — aprender#3535, three maintainer PRs red for a delta that did not exist),
`contracts/apr-cli-commands-v1.yaml`, `README.md`, `CLAUDE.md`, or a
known-red list anywhere.

A **delta** is one of exactly four things, any one of which closes the sweep:
Expand Down
53 changes: 50 additions & 3 deletions scripts/check_pr_ont_delta.sh
Original file line number Diff line number Diff line change
Expand Up @@ -39,10 +39,12 @@ usage() {
exit 2
}

# The sinks §11.1 names, verbatim. A `docs/specifications/` entry is a PREFIX.
# The sinks §11.1 names, verbatim. Under docs/specifications/ the sink is the release spec ITSELF —
# the train writes its findings there — never a design spec (aprender#3535: three maintainer PRs,
# each touching one design document, were red for a delta that did not exist).
ONT_SWEEP_SINKS=(
".github/workflows/night.yml"
"docs/specifications/"
"docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md"
"contracts/apr-cli-commands-v1.yaml"
"README.md"
"CLAUDE.md"
Expand Down Expand Up @@ -166,10 +168,55 @@ trailer'
[ "$fail" -eq 0 ]
}

# The PREDICATE half had no self-test: every row above feeds check_body_text, so
# sweep_reason could have classified every PR as a sweep (or none) and the
# self-test would stay green. Measured 2026-09-19: three PRs by a maintainer
# (#3499 a guard-regex fix, #3420 and #3424 design specs) were RED on this
# guard, every one of them because `docs/specifications/**` is a prefix sink
# and §11.1 says so verbatim. A design spec that IS the finding's home is not
# "writing a finding into a prose sink"; whether the sentence or the row moves
# is a §11.1 amendment, and the row below is the falsifying case either way.
# sweep_reason takes a FILE of changed paths, one per line (the same shape main() hands it from
# --changed), and reads the paths out of it — so each row writes its one path into a temp file and
# passes the FILE. Quorum round 2 misread this as passing the temp path itself; --self-test is the measurement.
predicate_row() { # predicate_row NAME WANT CHANGED_PATH (want: 0 = sweep, 1 = not a sweep)
local name="$1" want="$2" path="$3" tmp rc
tmp="$(mktemp -d)"
case "$tmp" in /tmp/*|/var/tmp/*) : ;; *) printf 'NO-GO: mktemp -d gave %s\n' "$tmp" >&2; return 2 ;; esac
printf '%s\n' "$path" > "$tmp/c.txt"
set +e; sweep_reason "$tmp/c.txt" >/dev/null 2>&1; rc=$?; set -e
[ -n "$tmp" ] && [ -d "$tmp" ] && rm -rf "$tmp"
if [ "$rc" -eq "$want" ]; then printf ' ok %-46s want=%s\n' "$name" "$want"; return 0
else printf ' FAIL %-46s want=%s got=%s\n' "$name" "$want" "$rc"; return 1; fi
}

predicate_self_test() {
local pass=0 fail=0
printf 'check_pr_ont_delta predicate self-test (sweep_reason)\n'
prow() { if predicate_row "$@"; then pass=$((pass+1)); else fail=$((fail+1)); fi; }
prow "crate source is not a sweep" 1 'crates/aprender-core/src/lib.rs'
prow "a contract yaml is not a sweep" 1 'contracts/tensor-layout-v1.yaml'
prow "a workflow other than night is not" 1 '.github/workflows/ci.yml'
prow "night.yml is a prose sink" 0 '.github/workflows/night.yml'
prow "README.md is a prose sink" 0 'README.md'
prow "CLAUDE.md is a prose sink" 0 'CLAUDE.md'
prow "apr-cli-commands registry is a sink" 0 'contracts/apr-cli-commands-v1.yaml'
prow "a known-red baseline is DERIVED" 0 'scripts/cb200_baseline.txt'
prow "the release spec itself is a sink" 0 'docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md'
# THE FALSIFYING ROW. A design spec (PP-QUANT-001, #3420) carries no finding
# from a sweep; under the §11.1 sentence as first written it WAS a sweep (want=0)
# and the guard redded a maintainer's spec PR for a line about a delta that
# did not exist. The amendment (aprender#3535) makes this row want=1; restoring the
# directory-wide sink turns it RED again, which is the proof it discriminates.
prow "a DESIGN spec is not a sweep" 1 'docs/specifications/PP-QUANT-001-MASTER.md'
printf 'predicate self-test: %s passed, %s failed\n' "$pass" "$fail"
[ "$fail" -eq 0 ]
}

main() {
local body="" changed="" reason
if [ $# -eq 0 ] || [ "${1:-}" = "--self-test" ]; then
self_test; return $?
self_test && predicate_self_test; return $?
fi
while [ $# -gt 0 ]; do
case "$1" in
Expand Down
Loading