From 427f002c03c59eeed641da1c68e04ab96c7165cc Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Tue, 20 Oct 2026 14:28:56 +0200 Subject: [PATCH 01/22] docs(contract): advisory-gate-diff-v1 DRAFT + PMAT-5792 ticket (Refs #5792, #5791) PR gate fails only on advisories the PR introduces; main-resident advisory = andon + P0 <=4h, no block; one scanner of record; auto re-arm. Contract and ticket only: no gate, workflow or script change (gate-accept needs solo + non-author sign-off + operator yes first). Co-Authored-By: Claude Sonnet 5.5 --- README.md | 4 +- contracts/advisory-gate-diff-v1.yaml | 122 +++++++++++++++++++++++++++ contracts/contracts.nt | 5 ++ docs/roadmaps/entries/PMAT-5792.yaml | 19 +++++ 4 files changed, 148 insertions(+), 2 deletions(-) create mode 100644 contracts/advisory-gate-diff-v1.yaml create mode 100644 docs/roadmaps/entries/PMAT-5792.yaml diff --git a/README.md b/README.md index 68106cff29..01c71588c7 100644 --- a/README.md +++ b/README.md @@ -114,7 +114,7 @@ publishing — all backed by YAML provable contracts that fail CI on drift. | Metric | Count | Source of truth | |-------:|------:|---| | Workspace crates | **81** workspace crates | `cargo metadata --no-deps` (NOT `ls crates/` — 4 are `exclude`d, 1 has no Cargo.toml) | -| Provable contracts | **1996** provable contracts | `contracts/census.json` `.n_files` — the set `pv lint` walks (`pv census`, ONT-001 ONT-1; regenerated by `make contracts`, written by `make readme-sync`, guarded by `scripts/check_readme_claims.sh`) | +| Provable contracts | **1997** provable contracts | `contracts/census.json` `.n_files` — the set `pv lint` walks (`pv census`, ONT-001 ONT-1; regenerated by `make contracts`, written by `make readme-sync`, guarded by `scripts/check_readme_claims.sh`) | | CLI commands | **113** CLI commands | `contracts/apr-cli-commands-v1.yaml` §`commands` (parse the list; `apr --help` also lists `help` itself, and `grep -c '^ - name:'` gives 119 — other same-indent `name:` keys exist in the file) | | Book CLI chapters | **114** chapters | `ls book/src/cli/*.md` | | Book lib chapters | **73** chapters | `ls book/src/lib/*.md` (parity with `pub mod`) | @@ -403,7 +403,7 @@ falsification_tests: prediction: apr validate bad-model.apr exits non-zero ``` -The tree carries 1996 contracts across inference, training, quantization, attention, FFN, +The tree carries 1997 contracts across inference, training, quantization, attention, FFN, tokenization, model formats, CLI safety — and this README itself. ## Migration from old crates diff --git a/contracts/advisory-gate-diff-v1.yaml b/contracts/advisory-gate-diff-v1.yaml new file mode 100644 index 0000000000..73c5350aad --- /dev/null +++ b/contracts/advisory-gate-diff-v1.yaml @@ -0,0 +1,122 @@ +metadata: + version: 1.0.0 + created: '2026-10-20' + author: PAIML Engineering + description: "Advisory gate: a PR fails only on advisories it introduces; an advisory already on main is an andon with a P0 4 h clock and does not block PRs; one scanner of record. Goal: 0 h total stop on a RustSec publication. DRAFT; gate-accept (one-way door), solo + non-author sign-off + operator yes before arming; any .github/workflows edit goes to a quorum first." + kind: pattern + references: + - "paiml/aprender#5792 (PMAT-5792) -- this ticket" + - "paiml/aprender#5791 -- cargo deny advisories is blind to RUSTSEC-2026-0378/0379 on main's lock; only cargo audit sees them" + - "paiml/aprender#5789 -- lock-only wasmtime bump that unblocked the queue" + - "MG run 39413691864 (#5781) -- P0 red on a publication, not a PR change" + +five_whys: + symptom: "Every PR in the merge queue went red when RUSTSEC-2026-0378/0379 were published against wasmtime 48.0.6 already on main." + why_1: "The security step fails on ANY advisory in the lock, not on advisories the PR adds." + why_2: "The gate compares the lock to the advisory DB, never to main's lock, so a new DB entry and a new dependency look identical." + why_3: "Two scanners (cargo deny, cargo audit) read different graphs (deny: all-features=false) and disagree; neither is the scanner of record, so the queue's red came from whichever saw it last." + why_4: "Nothing routes a main-resident advisory to a fix ticket with a clock; the only response available is to stop merging." + why_5: "Gate design treated 'the tree is clean' and 'this PR is safe to merge' as one question." + root_cause: "The merge decision is coupled to the global advisory state of main instead of the delta of the PR." + +equations: + pr_gate_verdict: + formula: "fail(PR) iff A(lock_PR, db) \\ A(lock_main, db) != {} ; A(lock, db) = advisories in db that hit a crate@version in lock, minus ignored ids" + domain: "lock_PR, lock_main from the PR merge base and head; db is ONE snapshot used for both sides" + codomain: "pass | fail" + invariants: + - "Both sides use the same advisory-db snapshot (otherwise a DB refresh between the runs reads as a PR-introduced advisory)" + - "A PR that does not touch Cargo.lock has an empty delta and passes" + - "A PR that removes an advisory-hit crate passes" + main_resident_advisory: + formula: "A(lock_main, db) != {} => andon raised AND P0 ticket with due <= 4h AND PR gate unaffected" + domain: "scheduled scan of main plus the pr_gate_verdict baseline" + codomain: "ticket id + due time, or none" + invariants: + - "The andon fires within one scan period of the DB publishing the advisory" + - "The ticket carries advisory ids, affected crate@version, and the clock start" + - "A main-resident advisory never turns the PR gate or the merge queue red" + one_scanner_of_record: + formula: "exactly one scanner decides A(); the other is retired or run as a LAB cross-check that cannot block" + domain: "cargo deny check advisories, cargo audit" + codomain: "the scanner name recorded in deny.toml / the gate script header" + invariants: + - "The scanner of record sees the whole Cargo.lock (all features), proven on the 48.0.6 lock as the positive control" + - "Exemptions live in one file" + auto_rearm: + formula: "when a gate fix lands, every PR that was ready (green, approved) before the stop is re-armed without a human" + domain: "PRs with auto-merge armed or ready at the time of the stop" + codomain: "armed set restored" + invariants: + - "Re-arm never arms a PR whose head changed after its approval" + +proof_obligations: + - id: ADG-INV-001 + type: invariant + property: "A PR that adds no advisory passes while main holds one" + formal: "A(lock_PR) = A(lock_main) != {} => verdict = pass" + applies_to: pr_gate_verdict + - id: ADG-INV-002 + type: invariant + property: "A PR that introduces an advisory fails" + formal: "A(lock_PR) \\ A(lock_main) != {} => verdict = fail" + applies_to: pr_gate_verdict + - id: ADG-INV-003 + type: invariant + property: "A main-resident advisory raises an andon and a <=4h P0 ticket" + formal: "A(lock_main) != {} => ticket(P0, due<=4h) exists" + applies_to: main_resident_advisory + - id: ADG-INV-004 + type: invariant + property: "The scanner of record is blind to nothing the other scanner sees" + formal: "A_record(lock_main_48.0.6) contains RUSTSEC-2026-0378 and RUSTSEC-2026-0379" + applies_to: one_scanner_of_record + - id: ADG-INV-005 + type: invariant + property: "Ready PRs are re-armed after a gate fix" + formal: "armed_after >= ready_before for PRs whose head is unchanged" + applies_to: auto_rearm + +falsification_tests: + - id: FALSIFY-ADG-001 + rule: "Main-resident advisory does not block a PR" + prediction: "with lock_PR == lock_main (wasmtime 48.0.6) and a DB holding 0378/0379, the PR verdict is pass and a main-advisory andon row is emitted" + test: "planted-fixture case table of the diff gate script (to be written on the branch): same lock both sides" + if_fails: "the gate still compares lock to DB, not PR to main" + mutation: "compare lock_PR to the empty set; the case must turn RED" + - id: FALSIFY-ADG-002 + rule: "A PR-introduced advisory blocks" + prediction: "with lock_main = 48.0.7 and lock_PR = 48.0.6, the verdict is fail naming 0378 and 0379" + test: "same case table: lock_PR adds the vulnerable version" + if_fails: "the gate cannot turn RED on a real regression (a gate that cannot fail)" + mutation: "make the delta symmetric difference's right side empty; the case must turn RED" + - id: FALSIFY-ADG-003 + rule: "Scanner of record sees the optional-feature crate" + prediction: "on the 48.0.6 lock the scanner of record exits non-zero and names 0378 and 0379" + test: "run the scanner of record on the 48.0.6 lock fixture" + if_fails: "#5791 is not resolved: the record scanner is as blind as deny with all-features=false" + mutation: "flip all-features to false in the scanner config; the case must turn RED" + - id: FALSIFY-ADG-004 + rule: "Same DB snapshot on both sides" + prediction: "a DB entry published between the main run and the PR run does not appear in the delta" + test: "case table: two DB snapshots, delta computed against one" + if_fails: "DB refreshes read as PR-introduced advisories and re-create the stop" + mutation: "read the DB twice; the case must turn RED" + - id: FALSIFY-ADG-005 + rule: "Andon clock" + prediction: "the P0 ticket's due time is <= 4 h after the advisory first appears on main" + test: "case table over the ticket emitter" + if_fails: "main-resident advisories sit unowned" + mutation: "set the due time to 8 h; the case must turn RED" + +qa_gate: + id: F-ADG-001 + name: Advisory gate diff contract + description: "PR delta gate, main andon, one scanner of record, auto re-arm" + checks: + - pr_gate_verdict + - main_resident_advisory + - one_scanner_of_record + - auto_rearm + pass_criteria: "All 5 falsification tests pass; the scanner script has a must-fail/must-pass case table" + falsification: "Revert the gate to lock-vs-DB; FALSIFY-ADG-001 must turn RED" diff --git a/contracts/contracts.nt b/contracts/contracts.nt index 87447a2049..bb93209bb0 100644 --- a/contracts/contracts.nt +++ b/contracts/contracts.nt @@ -4691,6 +4691,11 @@ . "contracts/adamw-kernel-v1.yaml"^^ . "adamw-kernel-v1"^^ . + . + . + "contracts/advisory-gate-diff-v1.yaml"^^ . + "advisory-gate-diff-v1"^^ . + "pattern"^^ . . . . diff --git a/docs/roadmaps/entries/PMAT-5792.yaml b/docs/roadmaps/entries/PMAT-5792.yaml new file mode 100644 index 0000000000..fd2d76887a --- /dev/null +++ b/docs/roadmaps/entries/PMAT-5792.yaml @@ -0,0 +1,19 @@ +- id: PMAT-5792 + github_issue: 5792 + item_type: task + title: 'Advisory gate: PR fails only on advisories it introduces; main-resident advisory = andon, not a block (0h stop)' + status: planned + priority: critical + assigned_to: null + created: 2026-10-20T12:27:03Z + updated: 2026-10-20T12:27:03Z + spec: null + acceptance_criteria: + - 'Contract contracts/advisory-gate-diff-v1.yaml. Refs #5791. Gate-accept: solo + non-author sign-off + Noah''s yes; workflows edits go to quorum first.' + phases: [] + subtasks: [] + estimated_effort: null + labels: + - kind:code + notes: null + epic: 3863 From e54ee6becaf4457039ff096c4f01154a7a01110d Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Tue, 20 Oct 2026 14:31:37 +0200 Subject: [PATCH 02/22] chore(contracts): regenerate witness for advisory-gate-diff-v1 (Refs #5792) Co-Authored-By: Claude Sonnet 5.5 --- ...eb2609244c59534f015f0be2f90923120a3027c176ad56723d0d850.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contracts/witness/811c78184eb2609244c59534f015f0be2f90923120a3027c176ad56723d0d850.json b/contracts/witness/811c78184eb2609244c59534f015f0be2f90923120a3027c176ad56723d0d850.json index 5ca9abb1de..daee4d60ee 100644 --- a/contracts/witness/811c78184eb2609244c59534f015f0be2f90923120a3027c176ad56723d0d850.json +++ b/contracts/witness/811c78184eb2609244c59534f015f0be2f90923120a3027c176ad56723d0d850.json @@ -1,5 +1,5 @@ { - "census_id_set_sha256": "db20faa02774eeffec52d05d3e579c0ec7047e2f0bed655de26b7e3eeff34151", + "census_id_set_sha256": "08a3fa4fb2b2814e04d0f19f101025396cf31cd9b20b3291402539b7edf94921", "relations_sha256": "811c78184eb2609244c59534f015f0be2f90923120a3027c176ad56723d0d850", "checkable_n": 87, "result": { From 95891b3e9d69c9af38261a570232f4ef23dee9d0 Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Tue, 20 Oct 2026 14:36:27 +0200 Subject: [PATCH 03/22] fix(contract): advisory-gate-diff formals carry declared symbols (Refs #5792) Co-Authored-By: Claude Sonnet 5.5 --- contracts/advisory-gate-diff-v1.yaml | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/contracts/advisory-gate-diff-v1.yaml b/contracts/advisory-gate-diff-v1.yaml index 73c5350aad..b50ba2e860 100644 --- a/contracts/advisory-gate-diff-v1.yaml +++ b/contracts/advisory-gate-diff-v1.yaml @@ -54,27 +54,27 @@ proof_obligations: - id: ADG-INV-001 type: invariant property: "A PR that adds no advisory passes while main holds one" - formal: "A(lock_PR) = A(lock_main) != {} => verdict = pass" + formal: "∀ pr: A(lock_PR) = A(lock_main) ∧ A(lock_main) ≠ ∅ ⇒ verdict = pass" applies_to: pr_gate_verdict - id: ADG-INV-002 type: invariant property: "A PR that introduces an advisory fails" - formal: "A(lock_PR) \\ A(lock_main) != {} => verdict = fail" + formal: "∀ pr: A(lock_PR) \\ A(lock_main) ≠ ∅ ⇒ verdict = fail" applies_to: pr_gate_verdict - id: ADG-INV-003 type: invariant property: "A main-resident advisory raises an andon and a <=4h P0 ticket" - formal: "A(lock_main) != {} => ticket(P0, due<=4h) exists" + formal: "∀ main: A(lock_main) ≠ ∅ ⇒ ticket(P0) ∧ due ≤ 4h" applies_to: main_resident_advisory - id: ADG-INV-004 type: invariant property: "The scanner of record is blind to nothing the other scanner sees" - formal: "A_record(lock_main_48.0.6) contains RUSTSEC-2026-0378 and RUSTSEC-2026-0379" + formal: "∀ lock_48_0_6: A_record(lock) ∋ RUSTSEC-2026-0378 ∧ A_record(lock) ∋ RUSTSEC-2026-0379" applies_to: one_scanner_of_record - id: ADG-INV-005 type: invariant property: "Ready PRs are re-armed after a gate fix" - formal: "armed_after >= ready_before for PRs whose head is unchanged" + formal: "∀ pr: head_unchanged ⇒ armed_after ≥ armed_before" applies_to: auto_rearm falsification_tests: From 289bba53292d07f207c0f91aba2b7294c5551e78 Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Tue, 20 Oct 2026 14:48:32 +0200 Subject: [PATCH 04/22] feat(ci): advisory_diff.sh fail-closed diff verdict + case table; contract domain per quorum CHANGES (Refs #5792) Co-Authored-By: Claude Sonnet 5.5 --- contracts/advisory-gate-diff-v1.yaml | 8 +-- scripts/ci/advisory_diff.sh | 92 ++++++++++++++++++++++++++++ 2 files changed, 96 insertions(+), 4 deletions(-) create mode 100755 scripts/ci/advisory_diff.sh diff --git a/contracts/advisory-gate-diff-v1.yaml b/contracts/advisory-gate-diff-v1.yaml index b50ba2e860..61bbfe11a4 100644 --- a/contracts/advisory-gate-diff-v1.yaml +++ b/contracts/advisory-gate-diff-v1.yaml @@ -21,8 +21,8 @@ five_whys: equations: pr_gate_verdict: - formula: "fail(PR) iff A(lock_PR, db) \\ A(lock_main, db) != {} ; A(lock, db) = advisories in db that hit a crate@version in lock, minus ignored ids" - domain: "lock_PR, lock_main from the PR merge base and head; db is ONE snapshot used for both sides" + formula: "fail(PR) iff A(lock_PR, db) \\ A(lock_main, db) != {} ; A(lock, db) = findings keyed code|id|crate@version (yanked included) in db that hit a crate@version in lock, minus ignored ids" + domain: "lock_PR at head, lock_main at the PR merge base (merge_group base_sha), scanned from a base-commit worktree with the BASE deny.toml and scripts; db is ONE pinned snapshot for both sides" codomain: "pass | fail" invariants: - "Both sides use the same advisory-db snapshot (otherwise a DB refresh between the runs reads as a PR-introduced advisory)" @@ -30,7 +30,7 @@ equations: - "A PR that removes an advisory-hit crate passes" main_resident_advisory: formula: "A(lock_main, db) != {} => andon raised AND P0 ticket with due <= 4h AND PR gate unaffected" - domain: "scheduled scan of main plus the pr_gate_verdict baseline" + domain: "scheduled or push-to-main scan only (never the PR path: C324 rule 6, read-only token, GH-1); due is a label plus created_at" codomain: "ticket id + due time, or none" invariants: - "The andon fires within one scan period of the DB publishing the advisory" @@ -81,7 +81,7 @@ falsification_tests: - id: FALSIFY-ADG-001 rule: "Main-resident advisory does not block a PR" prediction: "with lock_PR == lock_main (wasmtime 48.0.6) and a DB holding 0378/0379, the PR verdict is pass and a main-advisory andon row is emitted" - test: "planted-fixture case table of the diff gate script (to be written on the branch): same lock both sides" + test: "planted-fixture case table of the diff gate script scripts/ci/advisory_diff.sh --self-test, 11 cases: same lock both sides" if_fails: "the gate still compares lock to DB, not PR to main" mutation: "compare lock_PR to the empty set; the case must turn RED" - id: FALSIFY-ADG-002 diff --git a/scripts/ci/advisory_diff.sh b/scripts/ci/advisory_diff.sh new file mode 100755 index 0000000000..ccf5e25fa6 --- /dev/null +++ b/scripts/ci/advisory_diff.sh @@ -0,0 +1,92 @@ +#!/usr/bin/env bash +# advisory_diff.sh -- PR advisory verdict = findings the PR introduces (PMAT-5792, contract advisory-gate-diff-v1). +# +# advisory_diff.sh keys FILE print sorted finding keys "code|id|crate@version" from cargo-deny NDJSON +# advisory_diff.sh verdict PR_JSON PR_RC MAIN_JSON MAIN_RC +# exit 0: nothing new; 1: PR introduces findings; 75: could not check +# advisory_diff.sh --self-test case table +# +# Fail closed (exit 75, never a pass): rc 0 with findings, rc 1 with none, rc other than 0/1, unparsable JSON, +# a missing file. The key is code|advisory-id|crate@version so a new crate under an id main already carries, +# and a yanked finding (no id), are both seen. cargo-deny writes NDJSON diagnostics to STDERR; the caller +# redirects it to FILE. +set -euo pipefail + +CANT=75 + +keys() { # FILE -> keys on stdout; nonzero when the file does not parse + jq -R -r 'fromjson? | select(.type == "diagnostic" or has("fields")) + | .fields | select(.severity == "error" or .code == "yanked") + | "\(.code // "?")|\(.advisory.id // "-")|\(.graphs[0].Krate.name // "?")@\(.graphs[0].Krate.version // "?")"' "$1" | LC_ALL=C sort -u +} + +parses() { # every non-blank line is JSON + jq -R -n -e '[inputs | select(length > 0) | fromjson] | length >= 0' "$1" >/dev/null 2>&1 +} + +side() { # JSON RC -> echoes keys; returns CANT when the side cannot be trusted + local f=$1 rc=$2 k + [ -f "$f" ] || { echo "COULD-NOT-CHECK: $f missing" >&2; return "$CANT"; } + case "$rc" in 0 | 1) ;; *) echo "COULD-NOT-CHECK: cargo-deny rc=$rc" >&2; return "$CANT" ;; esac + parses "$f" || { echo "COULD-NOT-CHECK: $f is not NDJSON" >&2; return "$CANT"; } + k=$(keys "$f") + local e; e=$(jq -R -r 'fromjson? | .fields? | select(.severity == "error") | .code' "$f" | head -c1) + if [ "$rc" -eq 1 ] && [ -z "$e" ]; then echo "COULD-NOT-CHECK: rc=1 but no findings parsed" >&2; return "$CANT"; fi + if [ "$rc" -eq 0 ] && [ -n "$e" ]; then echo "COULD-NOT-CHECK: rc=0 but error findings parsed" >&2; return "$CANT"; fi + printf '%s\n' "$k" +} + +verdict() { + local pr main new + pr=$(side "$1" "$2") || return "$CANT" + main=$(side "$3" "$4") || return "$CANT" + new=$(comm -23 <(printf '%s\n' "$pr" | sed '/^$/d') <(printf '%s\n' "$main" | sed '/^$/d')) + if [ -n "$new" ]; then + echo "PR introduces advisories:" >&2 + printf '%s\n' "$new" >&2 + return 1 + fi + if [ -n "$(printf '%s' "$main" | tr -d '\n')" ]; then + echo "ANDON: advisories already on main (not blocking this PR): $(printf '%s' "$main" | tr '\n' ' ')" >&2 + fi + return 0 +} + +self_test() { + local d bad=0 + d=$(mktemp -d) + # shellcheck disable=SC2064 + trap "rm -rf -- '${d:?}'" RETURN + row() { printf '{"type":"diagnostic","fields":{"code":"%s","severity":"%s","advisory":{"id":"%s"},"graphs":[{"Krate":{"name":"%s","version":"%s"}}]}}\n' "$@"; } + row vulnerability error RUSTSEC-2026-0378 wasmtime 48.0.6 >"$d/a.json" + row vulnerability error RUSTSEC-2026-0378 wasmtime 48.0.6 >"$d/a2.json"; row vulnerability error RUSTSEC-2026-0379 wasmtime-x 48.0.6 >>"$d/a2.json" + row vulnerability error RUSTSEC-2026-0378 wasmtime 48.0.6 >"$d/a3.json"; row vulnerability error RUSTSEC-2026-0378 wasmtime-y 1.0.0 >>"$d/a3.json" + : >"$d/empty.json" + printf 'garbage\n' >"$d/bad.json" + printf '{"type":"diagnostic","fields":{"code":"yanked","severity":"warning","graphs":[{"Krate":{"name":"y","version":"1"}}]}}\n' >"$d/yank.json" + t() { # name want pr-file pr-rc main-file main-rc + local rc=0 + verdict "$d/$3" "$4" "$d/$5" "$6" >/dev/null 2>&1 || rc=$? + [ "$rc" -eq "$2" ] || { echo "SELF-TEST FAIL: $1 wanted rc=$2 got $rc" >&2; bad=1; } + } + t 'same advisory on both sides: pass' 0 a.json 1 a.json 1 + t 'both clean: pass' 0 empty.json 0 empty.json 0 + t 'PR adds a new id: block' 1 a2.json 1 a.json 1 + t 'PR adds a crate under an id main has: block' 1 a3.json 1 a.json 1 + t 'PR adds a yanked crate: block' 1 yank.json 0 empty.json 0 + t 'main has it, PR fixed it: pass' 0 empty.json 0 a.json 1 + t 'rc 0 but findings parsed: cannot check' 75 a.json 0 empty.json 0 + t 'rc 1 but nothing parsed: cannot check' 75 empty.json 1 empty.json 0 + t 'tool error rc 2: cannot check' 75 empty.json 2 empty.json 0 + t 'garbage JSON: cannot check' 75 bad.json 1 empty.json 0 + t 'main side missing: cannot check' 75 a.json 1 nope.json 1 + [ "$bad" -eq 0 ] && echo "advisory_diff self-test: 11 cases ok" + return "$bad" +} + +case "${1:-}" in + --self-test) self_test ;; + keys) keys "$2" ;; + verdict) verdict "$2" "$3" "$4" "$5" ;; + *) echo "usage: $0 keys F | verdict PR PRRC MAIN MAINRC | --self-test" >&2; exit 2 ;; +esac From 602ee69bbcd3a6246601e3442948aa291741e85d Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Tue, 20 Oct 2026 15:12:47 +0200 Subject: [PATCH 05/22] fix(ci): advisory gate per agy 3/3 REJECT + quorum r2: ADG-002 mutation, key id+crate+ver, BASE ignores, pinned merge base, FALSIFY-ADG-006 re-arm (Refs #5792) Co-Authored-By: Claude Sonnet 5.5 --- contracts/advisory-gate-diff-v1.yaml | 13 ++++++++++-- scripts/ci/advisory_diff.sh | 27 ++++++++++++++++++++++-- scripts/ci/rearm_ready.sh | 31 ++++++++++++++++++++++++++++ 3 files changed, 67 insertions(+), 4 deletions(-) create mode 100755 scripts/ci/rearm_ready.sh diff --git a/contracts/advisory-gate-diff-v1.yaml b/contracts/advisory-gate-diff-v1.yaml index 61bbfe11a4..14ff297981 100644 --- a/contracts/advisory-gate-diff-v1.yaml +++ b/contracts/advisory-gate-diff-v1.yaml @@ -43,6 +43,9 @@ equations: invariants: - "The scanner of record sees the whole Cargo.lock (all features), proven on the 48.0.6 lock as the positive control" - "Exemptions live in one file" + - "Findings are keyed code|advisory-id|crate@version over every graph, so a new crate under an id main already carries, and a yanked crate with no id, are both PR-introduced" + - "Both sides are scanned with the BASE deny.toml; a PR that adds an advisories ignore id fails (ignores-added), so a PR cannot hide its own finding" + - "The base is pinned: merge_group uses merge_group.base_sha, pull_request uses pull_request.base.sha, anything else (or a non-sha) is COULD-NOT-CHECK rc 75, never a pass" auto_rearm: formula: "when a gate fix lands, every PR that was ready (green, approved) before the stop is re-armed without a human" domain: "PRs with auto-merge armed or ready at the time of the stop" @@ -89,7 +92,7 @@ falsification_tests: prediction: "with lock_main = 48.0.7 and lock_PR = 48.0.6, the verdict is fail naming 0378 and 0379" test: "same case table: lock_PR adds the vulnerable version" if_fails: "the gate cannot turn RED on a real regression (a gate that cannot fail)" - mutation: "make the delta symmetric difference's right side empty; the case must turn RED" + mutation: "in scripts/ci/advisory_diff.sh verdict() subtract the PR keys from themselves instead of main keys (comm -23 PR PR); self-test cases PR adds a new id, PR adds a crate under an id main has, and PR adds a yanked crate must turn RED (measured 2026-10-20)" - id: FALSIFY-ADG-003 rule: "Scanner of record sees the optional-feature crate" prediction: "on the 48.0.6 lock the scanner of record exits non-zero and names 0378 and 0379" @@ -108,6 +111,12 @@ falsification_tests: test: "case table over the ticket emitter" if_fails: "main-resident advisories sit unowned" mutation: "set the due time to 8 h; the case must turn RED" + - id: FALSIFY-ADG-006 + rule: "Re-arm never arms a changed head (ADG-INV-005)" + prediction: "a ready PR whose head sha moved after approval is not re-armed; one whose head is unchanged is" + test: "scripts/ci/rearm_ready.sh --self-test, 2 cases (head unchanged arms, moved or unapproved does not, empty arms nothing)" + if_fails: "a gate fix arms an unreviewed head into the queue" + mutation: "drop the approved_sha == current_sha test in select_; the moved-head case must turn RED" qa_gate: id: F-ADG-001 @@ -118,5 +127,5 @@ qa_gate: - main_resident_advisory - one_scanner_of_record - auto_rearm - pass_criteria: "All 5 falsification tests pass; the scanner script has a must-fail/must-pass case table" + pass_criteria: "All 6 falsification tests pass; the scanner script has a must-fail/must-pass case table" falsification: "Revert the gate to lock-vs-DB; FALSIFY-ADG-001 must turn RED" diff --git a/scripts/ci/advisory_diff.sh b/scripts/ci/advisory_diff.sh index ccf5e25fa6..a883eabc3b 100755 --- a/scripts/ci/advisory_diff.sh +++ b/scripts/ci/advisory_diff.sh @@ -17,7 +17,7 @@ CANT=75 keys() { # FILE -> keys on stdout; nonzero when the file does not parse jq -R -r 'fromjson? | select(.type == "diagnostic" or has("fields")) | .fields | select(.severity == "error" or .code == "yanked") - | "\(.code // "?")|\(.advisory.id // "-")|\(.graphs[0].Krate.name // "?")@\(.graphs[0].Krate.version // "?")"' "$1" | LC_ALL=C sort -u + | (.code // "?") as $c | (.advisory.id // "-") as $i | ((.graphs // [{}])[] | .Krate? // {}) | "\($c)|\($i)|\(.name // "?")@\(.version // "?")"' "$1" | LC_ALL=C sort -u } parses() { # every non-blank line is JSON @@ -36,6 +36,21 @@ side() { # JSON RC -> echoes keys; returns CANT when the side cannot be trusted printf '%s\n' "$k" } +base_ref() { # EVENT MG_BASE_SHA PR_BASE_SHA -> the pinned base sha; 75 when it cannot be pinned + local ev=$1 mg=${2:-} pr=${3:-} + case "$ev" in + merge_group) [[ "$mg" =~ ^[0-9a-f]{40}$ ]] || { echo "COULD-NOT-CHECK: merge_group base_sha missing" >&2; return "$CANT"; }; echo "$mg" ;; + pull_request) [[ "$pr" =~ ^[0-9a-f]{40}$ ]] || { echo "COULD-NOT-CHECK: pull_request base sha missing" >&2; return "$CANT"; }; echo "$pr" ;; + *) echo "COULD-NOT-CHECK: event $ev has no merge base (push-to-main runs the full check)" >&2; return "$CANT" ;; + esac +} + +ignores_added() { # BASE_deny.toml PR_deny.toml -> ids the PR adds to [advisories] ignore; exit 1 when any + local add + add=$(comm -13 <(grep -o "RUSTSEC-[0-9]*-[0-9]*" "$1" | sort -u) <(grep -o "RUSTSEC-[0-9]*-[0-9]*" "$2" | sort -u)) + [ -z "$add" ] || { echo "PR widens exemptions:" >&2; printf "%s\n" "$add" >&2; return 1; } +} + verdict() { local pr main new pr=$(side "$1" "$2") || return "$CANT" @@ -80,13 +95,21 @@ self_test() { t 'tool error rc 2: cannot check' 75 empty.json 2 empty.json 0 t 'garbage JSON: cannot check' 75 bad.json 1 empty.json 0 t 'main side missing: cannot check' 75 a.json 1 nope.json 1 - [ "$bad" -eq 0 ] && echo "advisory_diff self-test: 11 cases ok" + sha=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + r() { local w=$1 rc=0; shift; base_ref "$@" >/dev/null 2>&1 || rc=$?; [ "$rc" -eq "$w" ] || { echo "SELF-TEST FAIL: base_ref $* wanted $w got $rc" >&2; bad=1; }; } + r 0 merge_group "$sha" ""; r 75 merge_group "" "$sha"; r 75 merge_group "main" ""; r 0 pull_request "" "$sha"; r 75 push "$sha" "$sha" + printf "ignore = [\"RUSTSEC-2026-0001\"]\n" >"$d/b.toml"; cp "$d/b.toml" "$d/same.toml"; printf "ignore = [\"RUSTSEC-2026-0001\", \"RUSTSEC-2026-0378\"]\n" >"$d/wide.toml" + ignores_added "$d/b.toml" "$d/same.toml" 2>/dev/null || { echo "SELF-TEST FAIL: same ignores must pass" >&2; bad=1; } + ! ignores_added "$d/b.toml" "$d/wide.toml" 2>/dev/null || { echo "SELF-TEST FAIL: added ignore must block" >&2; bad=1; } + [ "$bad" -eq 0 ] && echo "advisory_diff self-test: (11 verdict + 5 base_ref + 2 ignores) ok" return "$bad" } case "${1:-}" in --self-test) self_test ;; keys) keys "$2" ;; + base-ref) base_ref "$2" "${3:-}" "${4:-}" ;; + ignores-added) ignores_added "$2" "$3" ;; verdict) verdict "$2" "$3" "$4" "$5" ;; *) echo "usage: $0 keys F | verdict PR PRRC MAIN MAINRC | --self-test" >&2; exit 2 ;; esac diff --git a/scripts/ci/rearm_ready.sh b/scripts/ci/rearm_ready.sh new file mode 100755 index 0000000000..3d15f99863 --- /dev/null +++ b/scripts/ci/rearm_ready.sh @@ -0,0 +1,31 @@ +#!/usr/bin/env bash +# rearm_ready.sh -- which ready PRs a gate fix may re-arm (contract advisory-gate-diff-v1, ADG-INV-005). +# +# rearm_ready.sh select FILE FILE = TSV "pr approved_sha current_sha"; prints PRs to arm +# rearm_ready.sh --self-test case table +# A PR whose head moved after its approval is NOT re-armed (a changed head needs a fresh review). +set -euo pipefail + +select_() { + awk -F'\t' 'NF >= 3 && $1 != "" && $2 ~ /^[0-9a-f]{40}$/ && $2 == $3 { print $1 }' "$1" +} + +self_test() { + local d bad=0 a b + d=$(mktemp -d) + # shellcheck disable=SC2064 + trap "rm -rf -- '${d:?}'" RETURN + a=$(printf 'a%.0s' {1..40}); b=$(printf 'b%.0s' {1..40}) + printf '1\t%s\t%s\n2\t%s\t%s\n3\t\t%s\n' "$a" "$a" "$a" "$b" "$a" >"$d/t.tsv" + [ "$(select_ "$d/t.tsv" | tr '\n' ' ')" = "1 " ] || { echo "SELF-TEST FAIL: only PR 1 (head unchanged) may be re-armed" >&2; bad=1; } + printf '' >"$d/e.tsv" + [ -z "$(select_ "$d/e.tsv")" ] || { echo "SELF-TEST FAIL: empty input arms nothing" >&2; bad=1; } + [ "$bad" -eq 0 ] && echo "rearm_ready self-test: 2 cases ok" + return "$bad" +} + +case "${1:-}" in + --self-test) self_test ;; + select) select_ "$2" ;; + *) echo "usage: $0 select FILE | --self-test" >&2; exit 2 ;; +esac From fd5e2851cdc9bc0688cb5b546437dfbe6c31a715 Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Tue, 20 Oct 2026 15:18:32 +0200 Subject: [PATCH 06/22] ci: advisory diff gate wired into guard-cargo; docker deny scoped; andon emitter (PMAT-5792, Refs #5792) Co-Authored-By: Claude Sonnet 5.5 --- ci/sections.yml | 4 +-- scripts/check_deny_exemptions_live.sh | 5 ++- scripts/ci/advisory_andon.sh | 34 ++++++++++++++++++ scripts/ci/advisory_gate.sh | 51 +++++++++++++++++++++++++++ 4 files changed, 91 insertions(+), 3 deletions(-) create mode 100644 scripts/ci/advisory_andon.sh create mode 100755 scripts/ci/advisory_gate.sh diff --git a/ci/sections.yml b/ci/sections.yml index cdb4bea410..390522aedb 100644 --- a/ci/sections.yml +++ b/ci/sections.yml @@ -2039,7 +2039,7 @@ jobs: -e GIT_CONFIG_KEY_0=http.https://github.com/.extraheader \ -e "GIT_CONFIG_VALUE_0=AUTHORIZATION: basic $B64" \ "$IMAGE" \ - cargo deny check + cargo deny check licenses bans sources # check_apr_bin_pinned asserts CALLERS pin the binary; it says nothing # about which binary the resolver hands them. apr_bin.sh returned # target/release/apr whenever it existed, so a stale release build beside @@ -2676,7 +2676,7 @@ jobs: export GIT_TERMINAL_PROMPT=0 GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=http.https://github.com/.extraheader GIT_CONFIG_VALUE_0="AUTHORIZATION: basic $(printf 'x-access-token:%s' "$GITHUB_TOKEN" | base64 | tr -d '\n')" export GIT_CONFIG_VALUE_0 - cargo deny check advisories + bash scripts/ci/advisory_gate.sh # Separate from the check above on purpose: a newly-FIXED upstream must # never fail anyone's build. It fails only this guard, whose remedy is # deleting a line. diff --git a/scripts/check_deny_exemptions_live.sh b/scripts/check_deny_exemptions_live.sh index 71fcd21c04..911b83e3e9 100755 --- a/scripts/check_deny_exemptions_live.sh +++ b/scripts/check_deny_exemptions_live.sh @@ -64,7 +64,10 @@ if [ "$dead" -gt 0 ]; then exit 1 fi -if [ "$rc" -ne 0 ]; then +if [ "$rc" -ne 0 ] && grep -qE '^error\[(vulnerability|unmaintained|unsound|yanked|notice)\]' "$LOG"; then + # PMAT-5792: findings are advisory-diff territory (scripts/ci/advisory_gate.sh), not this guard's. + printf '\nNOTE: advisory findings present; judged by advisory_gate.sh, not here.\n' +elif [ "$rc" -ne 0 ]; then printf '\nFAIL: `cargo deny check advisories` exited %s. Its last lines:\n' "$rc" tail -n 8 "$LOG" | sed 's/^/ /' tail -20 "$LOG" | sed 's|^| |' diff --git a/scripts/ci/advisory_andon.sh b/scripts/ci/advisory_andon.sh new file mode 100644 index 0000000000..d3365c2547 --- /dev/null +++ b/scripts/ci/advisory_andon.sh @@ -0,0 +1,34 @@ +#!/usr/bin/env bash +# advisory_andon.sh -- main-resident advisories are an andon plus a P0 ticket, never a PR block +# (PMAT-5792, contracts/advisory-gate-diff-v1.yaml main_resident_advisory). Runs ONLY on a +# schedule or push to main, never on a PR or merge_group path (C324: no third state there). +# +# advisory_andon.sh keys FILE finding keys of a cargo-deny NDJSON file (advisory_diff.sh keys) +# advisory_andon.sh due CREATED NOW exit 0 if the ticket is older than 4h (label p0-overdue), else 1 +# advisory_andon.sh --self-test +set -euo pipefail +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +DUE_SECS=$((4 * 3600)) + +due() { # ISO created, ISO now + local c n + c=$(date -u -d "$1" +%s) n=$(date -u -d "$2" +%s) + [ $((n - c)) -gt "$DUE_SECS" ] +} + +self_test() { + local bad=0 rc + chk() { rc=0; due "$2" "$3" || rc=$?; [ "$rc" -eq "$4" ] || { echo "SELF-TEST FAIL: $1 want $4 got $rc" >&2; bad=1; }; } + chk 'under 4h is not due' 2026-10-20T10:00:00Z 2026-10-20T13:59:00Z 1 + chk 'exactly 4h is not due' 2026-10-20T10:00:00Z 2026-10-20T14:00:00Z 1 + chk 'over 4h is due' 2026-10-20T10:00:00Z 2026-10-20T14:00:01Z 0 + [ "$bad" -eq 0 ] && echo "advisory_andon self-test: 3 cases ok" + return "$bad" +} + +case "${1:-}" in + keys) bash "$HERE/advisory_diff.sh" keys "$2" ;; + due) due "$2" "$3" ;; + --self-test) self_test ;; + *) echo "usage: $0 keys FILE | due CREATED NOW | --self-test" >&2; exit 2 ;; +esac diff --git a/scripts/ci/advisory_gate.sh b/scripts/ci/advisory_gate.sh new file mode 100755 index 0000000000..a67631cd88 --- /dev/null +++ b/scripts/ci/advisory_gate.sh @@ -0,0 +1,51 @@ +#!/usr/bin/env bash +# advisory_gate.sh -- the advisories step of guard-cargo (PMAT-5792, contract advisory-gate-diff-v1). +# +# pull_request / merge_group fail only on advisories the PR INTRODUCES against the pinned build base +# (refs/remotes/origin/main, pinned by pin_build_base.sh); an advisory +# already on the base is reported, never a block. +# any other event (push) the full `cargo deny check advisories`, as before: a red there is the +# main-resident andon, not a PR block. +# Fails closed: anything that cannot be checked exits 75 and fails the step. Both sides are scanned with the +# BASE deny.toml, from the BASE worktree's script, over ONE advisory-db fetched once by this job. +set -euo pipefail + +HERE="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" +EV="${GITHUB_EVENT_NAME:-}" +case "$EV" in + pull_request | merge_group) ;; + *) exec cargo deny check advisories ;; +esac + +base_sha() { # the build base the pin step named + git rev-parse --verify -q refs/remotes/origin/main^{commit} || { echo "COULD-NOT-CHECK: origin/main is not pinned" >&2; return 75; } +} +BASE=$(base_sha) || exit 75 +# merge_group: the pin must equal merge_group.base_sha; a drifted pin is refused, not guessed around. +if [ "$EV" = merge_group ] && [ -n "${MG_BASE_SHA:-}" ] && [ "$BASE" != "$MG_BASE_SHA" ]; then + echo "COULD-NOT-CHECK: pinned base $BASE != merge_group base_sha $MG_BASE_SHA" >&2 + exit 75 +fi +bash "$HERE/advisory_diff.sh" base-ref "$EV" "${MG_BASE_SHA:-}" "$BASE" >/dev/null || exit 75 + +cargo deny fetch db || { echo "COULD-NOT-CHECK: advisory-db fetch failed" >&2; exit 75; } + +W="${RUNNER_TEMP:-${TMPDIR:-/tmp}}/advgate-base-$$" +trap 'git worktree remove --force "${W:?}" >/dev/null 2>&1 || true' EXIT +git worktree add --detach "$W" "$BASE" >/dev/null 2>&1 || { echo "COULD-NOT-CHECK: no worktree for base $BASE (shallow fetch?)" >&2; exit 75; } + +# The BASE script when it has one; the PR's copy only to bootstrap the merge that introduces the script. +GATE="$W/scripts/ci/advisory_diff.sh" +[ -f "$GATE" ] || GATE="$HERE/advisory_diff.sh" + +bash "$GATE" ignores-added "$W/deny.toml" deny.toml || exit 1 + +scan() { # TREE OUT.ndjson -> prints the cargo-deny rc + local rc=0 + (cd "$1" && cargo deny --locked --all-features --format json check --config "$W/deny.toml" --disable-fetch advisories) \ + >/dev/null 2>"$2" || rc=$? + echo "$rc" +} +PRRC=$(scan "$PWD" "$W/../advgate-pr-$$.ndjson") +BASERC=$(scan "$W" "$W/../advgate-base-$$.ndjson") +bash "$GATE" verdict "$W/../advgate-pr-$$.ndjson" "$PRRC" "$W/../advgate-base-$$.ndjson" "$BASERC" From e62e52815fc49cf2b7a0872db9bd058c342d5ad7 Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Tue, 20 Oct 2026 15:29:06 +0200 Subject: [PATCH 07/22] ci: advisory gate round fixes -- MG_BASE_SHA to the step, one DB fetched with base config + HEAD logged, trap cleans temp, self-test in CI, x86-main-advisories uses the gate (PMAT-5792, Refs #5792) Co-Authored-By: Claude Sonnet 5.5 --- .github/workflows/ci.yml | 10 +++++++++- ci/sections.yml | 4 +++- scripts/ci/advisory_gate.sh | 10 +++++++--- scripts/ci_mg_reuse.sh | 2 +- 4 files changed, 20 insertions(+), 6 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d0f3850566..20372f9cd2 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -320,6 +320,13 @@ jobs: contents: read steps: - uses: actions/checkout@v7 + with: + fetch-depth: 0 + - name: Pin origin/main to this run's build base (PMAT-5792) + env: + PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }} + MG_BASE_SHA: ${{ github.event.merge_group.base_sha }} + run: bash scripts/ci/pin_build_base.sh - name: Install cargo-deny (if absent) env: CARGO_INSTALL_ROOT: ${{ runner.temp }}/cargo-deny @@ -333,11 +340,12 @@ jobs: - name: Advisories must pass, with deny.toml exemptions honoured env: GITHUB_TOKEN: ${{ github.token }} + MG_BASE_SHA: ${{ github.event.merge_group.base_sha }} run: | export GIT_TERMINAL_PROMPT=0 GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=http.https://github.com/.extraheader GIT_CONFIG_VALUE_0="AUTHORIZATION: basic $(printf 'x-access-token:%s' "$GITHUB_TOKEN" | base64 | tr -d '\n')" export GIT_CONFIG_VALUE_0 - cargo deny check advisories + bash scripts/ci/advisory_gate.sh - name: Every deny.toml exemption must still be live env: GITHUB_TOKEN: ${{ github.token }} diff --git a/ci/sections.yml b/ci/sections.yml index 390522aedb..de51996a84 100644 --- a/ci/sections.yml +++ b/ci/sections.yml @@ -2019,7 +2019,7 @@ jobs: # `ci / security` from the reusable workflow runs `cargo audit`, which does # NOT read deny.toml -- so this covers licences, bans and sources, which # nothing else does. - - name: cargo-deny (licences, bans, sources, advisories) + - name: cargo-deny (licences, bans, sources) env: GITHUB_TOKEN: ${{ github.token }} run: | @@ -2672,10 +2672,12 @@ jobs: - name: Advisories must pass, with deny.toml exemptions honoured env: GITHUB_TOKEN: ${{ github.token }} + MG_BASE_SHA: ${{ github.event.merge_group.base_sha }} run: | export GIT_TERMINAL_PROMPT=0 GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=http.https://github.com/.extraheader GIT_CONFIG_VALUE_0="AUTHORIZATION: basic $(printf 'x-access-token:%s' "$GITHUB_TOKEN" | base64 | tr -d '\n')" export GIT_CONFIG_VALUE_0 + bash scripts/ci/advisory_diff.sh --self-test bash scripts/ci/advisory_gate.sh # Separate from the check above on purpose: a newly-FIXED upstream must # never fail anyone's build. It fails only this guard, whose remedy is diff --git a/scripts/ci/advisory_gate.sh b/scripts/ci/advisory_gate.sh index a67631cd88..928439e3f9 100755 --- a/scripts/ci/advisory_gate.sh +++ b/scripts/ci/advisory_gate.sh @@ -28,12 +28,16 @@ if [ "$EV" = merge_group ] && [ -n "${MG_BASE_SHA:-}" ] && [ "$BASE" != "$MG_BAS fi bash "$HERE/advisory_diff.sh" base-ref "$EV" "${MG_BASE_SHA:-}" "$BASE" >/dev/null || exit 75 -cargo deny fetch db || { echo "COULD-NOT-CHECK: advisory-db fetch failed" >&2; exit 75; } - W="${RUNNER_TEMP:-${TMPDIR:-/tmp}}/advgate-base-$$" -trap 'git worktree remove --force "${W:?}" >/dev/null 2>&1 || true' EXIT +trap 'git worktree remove --force "${W:?}" >/dev/null 2>&1 || true; rm -f "${W:?}"/../advgate-pr-$$.ndjson "${W:?}"/../advgate-base-$$.ndjson' EXIT git worktree add --detach "$W" "$BASE" >/dev/null 2>&1 || { echo "COULD-NOT-CHECK: no worktree for base $BASE (shallow fetch?)" >&2; exit 75; } +# One DB for both scans: fetched with the BASE config (the db-path both scans read), HEAD logged. +cargo deny fetch --config "$W/deny.toml" db || { echo "COULD-NOT-CHECK: advisory-db fetch failed" >&2; exit 75; } +for d in "$HOME"/.cargo/advisory-dbs/*; do + [ -d "$d/.git" ] && echo "advisory-db $(basename "$d") HEAD $(git -C "$d" rev-parse --short HEAD 2>/dev/null) $(git -C "$d" log -1 --format=%cI 2>/dev/null)" +done + # The BASE script when it has one; the PR's copy only to bootstrap the merge that introduces the script. GATE="$W/scripts/ci/advisory_diff.sh" [ -f "$GATE" ] || GATE="$HERE/advisory_diff.sh" diff --git a/scripts/ci_mg_reuse.sh b/scripts/ci_mg_reuse.sh index 86573f5aff..30ab17f798 100755 --- a/scripts/ci_mg_reuse.sh +++ b/scripts/ci_mg_reuse.sh @@ -515,7 +515,7 @@ STUB if grep -qF 'outputs.mac' <<<"$(job mg-reuse)"; then fail=$((fail + 1)); echo "FAIL wiring: mg-reuse still exports mac, which no ci.yml job reads"; else pass=$((pass + 1)); fi wire x86-main-advisories "if: \${{ !cancelled() && needs.mg-reuse.outputs.x86 == '1' }}" - wire x86-main-advisories 'cargo deny check advisories' + wire x86-main-advisories 'bash scripts/ci/advisory_gate.sh' wire x86-main-advisories 'bash scripts/check_deny_exemptions_live.sh' wire ci-gate 'MG-REUSE-VERDICT-BEGIN' wire ci-gate 'needs: [x86-main, mg-reuse, x86-main-advisories]' From b8f5485111e4d8c2110bb2b07557177d6a77ea00 Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Tue, 20 Oct 2026 15:30:18 +0200 Subject: [PATCH 08/22] ci: advisory gate runs from the BASE ref; yanked follows deny.toml severity; scheduled main-advisory andon (PMAT-5792, Refs #5792) Co-Authored-By: Claude Sonnet 5.5 --- .github/workflows/advisory-andon.yml | 41 +++++++++++++++++++++++++ .github/workflows/ci.yml | 13 +++++++- ci/sections.yml | 14 +++++++-- scripts/ci/advisory_andon.sh | 46 +++++++++++++++++++++++++++- scripts/ci/advisory_diff.sh | 10 +++--- scripts/ci_mg_reuse.sh | 2 +- 6 files changed, 117 insertions(+), 9 deletions(-) create mode 100644 .github/workflows/advisory-andon.yml diff --git a/.github/workflows/advisory-andon.yml b/.github/workflows/advisory-andon.yml new file mode 100644 index 0000000000..1985debba3 --- /dev/null +++ b/.github/workflows/advisory-andon.yml @@ -0,0 +1,41 @@ +name: advisory-andon +# PMAT-5792: an advisory already on main is an andon and a P0 issue within 4h; it never blocks a PR. +# Scheduled and push-to-main only -- never a PR or merge_group job (C324: no third state on that path). +on: + schedule: + - cron: '17 */2 * * *' + push: + branches: [main] + workflow_dispatch: +permissions: + contents: read + issues: write +jobs: + andon: + runs-on: [self-hosted, Linux, X64, clean-room] + timeout-minutes: 30 + steps: + - uses: actions/checkout@v7 + - name: Install cargo-deny (if absent) + env: + CARGO_INSTALL_ROOT: ${{ runner.temp }}/cargo-deny + run: | + if ! command -v cargo-deny > /dev/null 2>&1; then + cargo install cargo-deny --locked + echo "$CARGO_INSTALL_ROOT/bin" >> "$GITHUB_PATH" + export PATH="$CARGO_INSTALL_ROOT/bin:$PATH" + fi + cargo deny --version + - name: Scan main and open or age the P0 issues + env: + GITHUB_TOKEN: ${{ github.token }} + GH_TOKEN: ${{ github.token }} + run: | + export GIT_TERMINAL_PROMPT=0 GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=http.https://github.com/.extraheader + GIT_CONFIG_VALUE_0="AUTHORIZATION: basic $(printf 'x-access-token:%s' "$GITHUB_TOKEN" | base64 | tr -d '\n')" + export GIT_CONFIG_VALUE_0 + bash scripts/ci/advisory_andon.sh --self-test + out="${RUNNER_TEMP:?}/andon.ndjson" + rc=0 + cargo deny --locked --all-features --format json check advisories >/dev/null 2>"$out" || rc=$? + bash scripts/ci/advisory_andon.sh emit "$out" "$rc" diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 20372f9cd2..244461861d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -345,7 +345,18 @@ jobs: export GIT_TERMINAL_PROMPT=0 GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=http.https://github.com/.extraheader GIT_CONFIG_VALUE_0="AUTHORIZATION: basic $(printf 'x-access-token:%s' "$GITHUB_TOKEN" | base64 | tr -d '\n')" export GIT_CONFIG_VALUE_0 - bash scripts/ci/advisory_gate.sh + # The gate runs from the BASE ref, never the PR tree (PMAT-5792): a PR cannot edit its own judge. + # Bootstrap only: the merge that introduces the scripts has no base copy, so it uses its own. + S="${RUNNER_TEMP:-${TMPDIR:-/tmp}}/advgate-scripts-$$" + if git cat-file -e refs/remotes/origin/main:scripts/ci/advisory_gate.sh 2>/dev/null; then + mkdir -p "$S" + git archive refs/remotes/origin/main scripts/ci | tar -x -C "$S" + G="$S/scripts/ci" + else + G=scripts/ci + fi + bash "$G/advisory_diff.sh" --self-test + bash "$G/advisory_gate.sh" - name: Every deny.toml exemption must still be live env: GITHUB_TOKEN: ${{ github.token }} diff --git a/ci/sections.yml b/ci/sections.yml index de51996a84..b1daa85d59 100644 --- a/ci/sections.yml +++ b/ci/sections.yml @@ -2677,8 +2677,18 @@ jobs: export GIT_TERMINAL_PROMPT=0 GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=http.https://github.com/.extraheader GIT_CONFIG_VALUE_0="AUTHORIZATION: basic $(printf 'x-access-token:%s' "$GITHUB_TOKEN" | base64 | tr -d '\n')" export GIT_CONFIG_VALUE_0 - bash scripts/ci/advisory_diff.sh --self-test - bash scripts/ci/advisory_gate.sh + # The gate runs from the BASE ref, never the PR tree (PMAT-5792): a PR cannot edit its own judge. + # Bootstrap only: the merge that introduces the scripts has no base copy, so it uses its own. + S="${RUNNER_TEMP:-${TMPDIR:-/tmp}}/advgate-scripts-$$" + if git cat-file -e refs/remotes/origin/main:scripts/ci/advisory_gate.sh 2>/dev/null; then + mkdir -p "$S" + git archive refs/remotes/origin/main scripts/ci | tar -x -C "$S" + G="$S/scripts/ci" + else + G=scripts/ci + fi + bash "$G/advisory_diff.sh" --self-test + bash "$G/advisory_gate.sh" # Separate from the check above on purpose: a newly-FIXED upstream must # never fail anyone's build. It fails only this guard, whose remedy is # deleting a line. diff --git a/scripts/ci/advisory_andon.sh b/scripts/ci/advisory_andon.sh index d3365c2547..be42ad9e03 100644 --- a/scripts/ci/advisory_andon.sh +++ b/scripts/ci/advisory_andon.sh @@ -5,6 +5,7 @@ # # advisory_andon.sh keys FILE finding keys of a cargo-deny NDJSON file (advisory_diff.sh keys) # advisory_andon.sh due CREATED NOW exit 0 if the ticket is older than 4h (label p0-overdue), else 1 +# advisory_andon.sh emit FILE RC open one P0 issue per main-resident finding; label p0-overdue past 4h # advisory_andon.sh --self-test set -euo pipefail HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" @@ -16,19 +17,62 @@ due() { # ISO created, ISO now [ $((n - c)) -gt "$DUE_SECS" ] } +GH="${GH:-gh}" +emit() { # NDJSON-FILE cargo-deny-rc + case "$2" in 0 | 1) ;; *) echo "COULD-NOT-CHECK: cargo deny rc $2" >&2; return 75 ;; esac + local keys k found now + keys=$(bash "$HERE/advisory_diff.sh" keys "$1") || return 75 + now=$(date -u +%Y-%m-%dT%H:%M:%SZ) + "$GH" label create advisory-andon --color B60205 2>/dev/null || true + "$GH" label create P0 --color B60205 2>/dev/null || true + "$GH" label create p0-overdue --color B60205 2>/dev/null || true + while IFS= read -r k; do + [ -n "$k" ] || continue + found=$("$GH" issue list --label advisory-andon --state open --search "in:title \"$k\"" --json number,createdAt,labels \ + --jq '.[0] | if . == null then "" else "\(.number) \(.createdAt) \([.labels[].name] | index("p0-overdue") != null)" end') || return 75 + if [ -z "$found" ]; then + "$GH" issue create --label advisory-andon --label P0 --title "ADVISORY ANDON: $k" \ + --body "Advisory $k is on main. It does not block PRs (PMAT-5792); fix within 4h of this issue's creation." || return 75 + else + set -- $found + if due "$2" "$now" && [ "$3" = false ]; then + "$GH" issue edit "$1" --add-label p0-overdue || return 75 + "$GH" issue comment "$1" --body "Open past the 4h P0 window." || return 75 + fi + fi + done <<<"$keys" +} + self_test() { local bad=0 rc chk() { rc=0; due "$2" "$3" || rc=$?; [ "$rc" -eq "$4" ] || { echo "SELF-TEST FAIL: $1 want $4 got $rc" >&2; bad=1; }; } chk 'under 4h is not due' 2026-10-20T10:00:00Z 2026-10-20T13:59:00Z 1 chk 'exactly 4h is not due' 2026-10-20T10:00:00Z 2026-10-20T14:00:00Z 1 chk 'over 4h is due' 2026-10-20T10:00:00Z 2026-10-20T14:00:01Z 0 - [ "$bad" -eq 0 ] && echo "advisory_andon self-test: 3 cases ok" + local d; d=$(mktemp -d) + # stub gh: logs argv; `issue list` answers from $STUB_LIST + printf '#!/usr/bin/env bash\necho "$*" >>"%s/calls"\n[ "$1 $2" = "issue list" ] && printf "%%s" "$STUB_LIST"\nexit 0\n' "$d" >"$d/gh"; chmod +x "$d/gh" + printf '{"type":"diagnostic","fields":{"code":"vulnerability","severity":"error","advisory":{"id":"RUSTSEC-1"},"graphs":[{"Krate":{"name":"c","version":"1"}}]}}\n' >"$d/f.json" + ec() { # name want-rc want-grep(or -) rc list + : >"$d/calls"; rc=0; GH="$d/gh" STUB_LIST="$5" emit "$d/f.json" "$4" >/dev/null 2>&1 || rc=$? + [ "$rc" -eq "$2" ] || { echo "SELF-TEST FAIL: $1 rc want $2 got $rc" >&2; bad=1; } + if [ "$3" != - ]; then grep -q -- "$3" "$d/calls" || { echo "SELF-TEST FAIL: $1 missing call $3" >&2; bad=1; }; fi + } + ec 'new finding opens a P0 issue' 0 'issue create' 1 '' + ec 'tool error is cannot-check' 75 - 2 '' + ec 'old open issue gets p0-overdue' 0 'add-label p0-overdue' 1 '7 2020-01-01T00:00:00Z false' + ec 'fresh open issue is left alone' 0 - 1 "7 $(date -u +%Y-%m-%dT%H:%M:%SZ) false" + : >"$d/calls"; GH="$d/gh" STUB_LIST="7 $(date -u +%Y-%m-%dT%H:%M:%SZ) false" emit "$d/f.json" 1 >/dev/null 2>&1 || true + if grep -q 'issue create\|add-label' "$d/calls"; then echo "SELF-TEST FAIL: fresh issue was touched" >&2; bad=1; fi + rm -rf -- "${d:?}" + [ "$bad" -eq 0 ] && echo "advisory_andon self-test: 7 cases ok" return "$bad" } case "${1:-}" in keys) bash "$HERE/advisory_diff.sh" keys "$2" ;; due) due "$2" "$3" ;; + emit) emit "$2" "$3" ;; --self-test) self_test ;; *) echo "usage: $0 keys FILE | due CREATED NOW | --self-test" >&2; exit 2 ;; esac diff --git a/scripts/ci/advisory_diff.sh b/scripts/ci/advisory_diff.sh index a883eabc3b..2579246530 100755 --- a/scripts/ci/advisory_diff.sh +++ b/scripts/ci/advisory_diff.sh @@ -8,7 +8,7 @@ # # Fail closed (exit 75, never a pass): rc 0 with findings, rc 1 with none, rc other than 0/1, unparsable JSON, # a missing file. The key is code|advisory-id|crate@version so a new crate under an id main already carries, -# and a yanked finding (no id), are both seen. cargo-deny writes NDJSON diagnostics to STDERR; the caller +# and a no-id finding (yanked at error severity, as deny.toml sets it), are both seen. cargo-deny writes NDJSON diagnostics to STDERR; the caller # redirects it to FILE. set -euo pipefail @@ -16,7 +16,7 @@ CANT=75 keys() { # FILE -> keys on stdout; nonzero when the file does not parse jq -R -r 'fromjson? | select(.type == "diagnostic" or has("fields")) - | .fields | select(.severity == "error" or .code == "yanked") + | .fields | select(.severity == "error") | (.code // "?") as $c | (.advisory.id // "-") as $i | ((.graphs // [{}])[] | .Krate? // {}) | "\($c)|\($i)|\(.name // "?")@\(.version // "?")"' "$1" | LC_ALL=C sort -u } @@ -88,7 +88,9 @@ self_test() { t 'both clean: pass' 0 empty.json 0 empty.json 0 t 'PR adds a new id: block' 1 a2.json 1 a.json 1 t 'PR adds a crate under an id main has: block' 1 a3.json 1 a.json 1 - t 'PR adds a yanked crate: block' 1 yank.json 0 empty.json 0 + t 'PR adds a yanked crate at warn (deny.toml default): pass' 0 yank.json 0 empty.json 0 + printf '{"type":"diagnostic","fields":{"code":"yanked","severity":"error","graphs":[{"Krate":{"name":"y","version":"1"}}]}}\n' >"$d/yanke.json" + t 'PR adds a yanked crate at error: block' 1 yanke.json 1 empty.json 0 t 'main has it, PR fixed it: pass' 0 empty.json 0 a.json 1 t 'rc 0 but findings parsed: cannot check' 75 a.json 0 empty.json 0 t 'rc 1 but nothing parsed: cannot check' 75 empty.json 1 empty.json 0 @@ -101,7 +103,7 @@ self_test() { printf "ignore = [\"RUSTSEC-2026-0001\"]\n" >"$d/b.toml"; cp "$d/b.toml" "$d/same.toml"; printf "ignore = [\"RUSTSEC-2026-0001\", \"RUSTSEC-2026-0378\"]\n" >"$d/wide.toml" ignores_added "$d/b.toml" "$d/same.toml" 2>/dev/null || { echo "SELF-TEST FAIL: same ignores must pass" >&2; bad=1; } ! ignores_added "$d/b.toml" "$d/wide.toml" 2>/dev/null || { echo "SELF-TEST FAIL: added ignore must block" >&2; bad=1; } - [ "$bad" -eq 0 ] && echo "advisory_diff self-test: (11 verdict + 5 base_ref + 2 ignores) ok" + [ "$bad" -eq 0 ] && echo "advisory_diff self-test: (12 verdict + 5 base_ref + 2 ignores) ok" return "$bad" } diff --git a/scripts/ci_mg_reuse.sh b/scripts/ci_mg_reuse.sh index 30ab17f798..22de715059 100755 --- a/scripts/ci_mg_reuse.sh +++ b/scripts/ci_mg_reuse.sh @@ -515,7 +515,7 @@ STUB if grep -qF 'outputs.mac' <<<"$(job mg-reuse)"; then fail=$((fail + 1)); echo "FAIL wiring: mg-reuse still exports mac, which no ci.yml job reads"; else pass=$((pass + 1)); fi wire x86-main-advisories "if: \${{ !cancelled() && needs.mg-reuse.outputs.x86 == '1' }}" - wire x86-main-advisories 'bash scripts/ci/advisory_gate.sh' + wire x86-main-advisories 'advisory_gate.sh"' wire x86-main-advisories 'bash scripts/check_deny_exemptions_live.sh' wire ci-gate 'MG-REUSE-VERDICT-BEGIN' wire ci-gate 'needs: [x86-main, mg-reuse, x86-main-advisories]' From b7cd43ece523c4eca91da7ddd33addf917619691 Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Tue, 20 Oct 2026 15:45:23 +0200 Subject: [PATCH 09/22] ci: advisory gate -- MG_BASE_SHA via outer guard step, andon fails closed + exact-title dedupe + concurrency, temp cleanup (PMAT-5792, Refs #5792) Co-Authored-By: Claude Sonnet 5.5 --- .github/workflows/advisory-andon.yml | 3 +++ .github/workflows/ci.yml | 5 ++++- ci/sections.yml | 8 ++++++-- contracts/advisory-gate-diff-v1.yaml | 2 +- scripts/ci/advisory_andon.sh | 22 +++++++++++++--------- scripts/ci/advisory_diff.sh | 3 ++- scripts/ci/advisory_gate.sh | 8 ++++---- 7 files changed, 33 insertions(+), 18 deletions(-) diff --git a/.github/workflows/advisory-andon.yml b/.github/workflows/advisory-andon.yml index 1985debba3..172a251d06 100644 --- a/.github/workflows/advisory-andon.yml +++ b/.github/workflows/advisory-andon.yml @@ -10,6 +10,9 @@ on: permissions: contents: read issues: write +concurrency: + group: advisory-andon + cancel-in-progress: false jobs: andon: runs-on: [self-hosted, Linux, X64, clean-room] diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 244461861d..610f700574 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -356,7 +356,10 @@ jobs: G=scripts/ci fi bash "$G/advisory_diff.sh" --self-test - bash "$G/advisory_gate.sh" + rc=0 + bash "$G/advisory_gate.sh" || rc=$? + rm -rf -- "${S:?}" + exit "$rc" - name: Every deny.toml exemption must still be live env: GITHUB_TOKEN: ${{ github.token }} diff --git a/ci/sections.yml b/ci/sections.yml index b1daa85d59..26bfde5f11 100644 --- a/ci/sections.yml +++ b/ci/sections.yml @@ -1886,6 +1886,7 @@ jobs: env: GITHUB_TOKEN: ${{ github.token }} GH_TOKEN: ${{ github.token }} + MG_BASE_SHA: ${{ github.event.merge_group.base_sha }} run: bash scripts/ci_guards.sh guard-cargo guard-cargo-steps: @@ -2672,11 +2673,11 @@ jobs: - name: Advisories must pass, with deny.toml exemptions honoured env: GITHUB_TOKEN: ${{ github.token }} - MG_BASE_SHA: ${{ github.event.merge_group.base_sha }} run: | export GIT_TERMINAL_PROMPT=0 GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=http.https://github.com/.extraheader GIT_CONFIG_VALUE_0="AUTHORIZATION: basic $(printf 'x-access-token:%s' "$GITHUB_TOKEN" | base64 | tr -d '\n')" export GIT_CONFIG_VALUE_0 + # MG_BASE_SHA comes from the process env of the outer guard-cargo step (the manifest runner resolves no merge_group context). # The gate runs from the BASE ref, never the PR tree (PMAT-5792): a PR cannot edit its own judge. # Bootstrap only: the merge that introduces the scripts has no base copy, so it uses its own. S="${RUNNER_TEMP:-${TMPDIR:-/tmp}}/advgate-scripts-$$" @@ -2688,7 +2689,10 @@ jobs: G=scripts/ci fi bash "$G/advisory_diff.sh" --self-test - bash "$G/advisory_gate.sh" + rc=0 + bash "$G/advisory_gate.sh" || rc=$? + rm -rf -- "${S:?}" + exit "$rc" # Separate from the check above on purpose: a newly-FIXED upstream must # never fail anyone's build. It fails only this guard, whose remedy is # deleting a line. diff --git a/contracts/advisory-gate-diff-v1.yaml b/contracts/advisory-gate-diff-v1.yaml index 14ff297981..64ced0dfdb 100644 --- a/contracts/advisory-gate-diff-v1.yaml +++ b/contracts/advisory-gate-diff-v1.yaml @@ -84,7 +84,7 @@ falsification_tests: - id: FALSIFY-ADG-001 rule: "Main-resident advisory does not block a PR" prediction: "with lock_PR == lock_main (wasmtime 48.0.6) and a DB holding 0378/0379, the PR verdict is pass and a main-advisory andon row is emitted" - test: "planted-fixture case table of the diff gate script scripts/ci/advisory_diff.sh --self-test, 11 cases: same lock both sides" + test: "planted-fixture case table of the diff gate script scripts/ci/advisory_diff.sh --self-test, 12 cases: same lock both sides" if_fails: "the gate still compares lock to DB, not PR to main" mutation: "compare lock_PR to the empty set; the case must turn RED" - id: FALSIFY-ADG-002 diff --git a/scripts/ci/advisory_andon.sh b/scripts/ci/advisory_andon.sh index be42ad9e03..eb4022ffe8 100644 --- a/scripts/ci/advisory_andon.sh +++ b/scripts/ci/advisory_andon.sh @@ -21,15 +21,16 @@ GH="${GH:-gh}" emit() { # NDJSON-FILE cargo-deny-rc case "$2" in 0 | 1) ;; *) echo "COULD-NOT-CHECK: cargo deny rc $2" >&2; return 75 ;; esac local keys k found now - keys=$(bash "$HERE/advisory_diff.sh" keys "$1") || return 75 + keys=$(bash "$HERE/advisory_diff.sh" checked-keys "$1" "$2") || return 75 now=$(date -u +%Y-%m-%dT%H:%M:%SZ) "$GH" label create advisory-andon --color B60205 2>/dev/null || true "$GH" label create P0 --color B60205 2>/dev/null || true "$GH" label create p0-overdue --color B60205 2>/dev/null || true while IFS= read -r k; do [ -n "$k" ] || continue - found=$("$GH" issue list --label advisory-andon --state open --search "in:title \"$k\"" --json number,createdAt,labels \ - --jq '.[0] | if . == null then "" else "\(.number) \(.createdAt) \([.labels[].name] | index("p0-overdue") != null)" end') || return 75 + found=$("$GH" issue list --label advisory-andon --state open --limit 200 --json number,createdAt,labels,title \ + | jq -r --arg t "ADVISORY ANDON: $k" 'map(select(.title == $t)) | .[0] + | if . == null then "" else "\(.number) \(.createdAt) \([.labels[].name] | index("p0-overdue") != null)" end') || return 75 if [ -z "$found" ]; then "$GH" issue create --label advisory-andon --label P0 --title "ADVISORY ANDON: $k" \ --body "Advisory $k is on main. It does not block PRs (PMAT-5792); fix within 4h of this issue's creation." || return 75 @@ -51,7 +52,7 @@ self_test() { chk 'over 4h is due' 2026-10-20T10:00:00Z 2026-10-20T14:00:01Z 0 local d; d=$(mktemp -d) # stub gh: logs argv; `issue list` answers from $STUB_LIST - printf '#!/usr/bin/env bash\necho "$*" >>"%s/calls"\n[ "$1 $2" = "issue list" ] && printf "%%s" "$STUB_LIST"\nexit 0\n' "$d" >"$d/gh"; chmod +x "$d/gh" + printf '#!/usr/bin/env bash\necho "$*" >>"%s/calls"\n[ "$1 $2" = "issue list" ] && printf "%%s" "${STUB_LIST:-[]}"\nexit 0\n' "$d" >"$d/gh"; chmod +x "$d/gh" printf '{"type":"diagnostic","fields":{"code":"vulnerability","severity":"error","advisory":{"id":"RUSTSEC-1"},"graphs":[{"Krate":{"name":"c","version":"1"}}]}}\n' >"$d/f.json" ec() { # name want-rc want-grep(or -) rc list : >"$d/calls"; rc=0; GH="$d/gh" STUB_LIST="$5" emit "$d/f.json" "$4" >/dev/null 2>&1 || rc=$? @@ -59,13 +60,16 @@ self_test() { if [ "$3" != - ]; then grep -q -- "$3" "$d/calls" || { echo "SELF-TEST FAIL: $1 missing call $3" >&2; bad=1; }; fi } ec 'new finding opens a P0 issue' 0 'issue create' 1 '' + ec 'a different open issue does not match' 0 'issue create' 1 '[{"number":9,"createdAt":"2020-01-01T00:00:00Z","labels":[],"title":"ADVISORY ANDON: other"}]' + : >"$d/empty.json"; rc=0; GH="$d/gh" emit "$d/empty.json" 1 >/dev/null 2>&1 || rc=$? + [ "$rc" -eq 75 ] || { echo "SELF-TEST FAIL: rc1 with no findings must be 75, got $rc" >&2; bad=1; } ec 'tool error is cannot-check' 75 - 2 '' - ec 'old open issue gets p0-overdue' 0 'add-label p0-overdue' 1 '7 2020-01-01T00:00:00Z false' - ec 'fresh open issue is left alone' 0 - 1 "7 $(date -u +%Y-%m-%dT%H:%M:%SZ) false" - : >"$d/calls"; GH="$d/gh" STUB_LIST="7 $(date -u +%Y-%m-%dT%H:%M:%SZ) false" emit "$d/f.json" 1 >/dev/null 2>&1 || true + ec 'old open issue gets p0-overdue' 0 'add-label p0-overdue' 1 '[{"number":7,"createdAt":"2020-01-01T00:00:00Z","labels":[],"title":"ADVISORY ANDON: vulnerability|RUSTSEC-1|c@1"}]' + ec 'fresh open issue is left alone' 0 - 1 "[{\"number\":7,\"createdAt\":\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\",\"labels\":[],\"title\":\"ADVISORY ANDON: vulnerability|RUSTSEC-1|c@1\"}]" + : >"$d/calls"; GH="$d/gh" STUB_LIST="[{\"number\":7,\"createdAt\":\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\",\"labels\":[],\"title\":\"ADVISORY ANDON: vulnerability|RUSTSEC-1|c@1\"}]" emit "$d/f.json" 1 >/dev/null 2>&1 || true if grep -q 'issue create\|add-label' "$d/calls"; then echo "SELF-TEST FAIL: fresh issue was touched" >&2; bad=1; fi rm -rf -- "${d:?}" - [ "$bad" -eq 0 ] && echo "advisory_andon self-test: 7 cases ok" + [ "$bad" -eq 0 ] && echo "advisory_andon self-test: 9 cases ok" return "$bad" } @@ -74,5 +78,5 @@ case "${1:-}" in due) due "$2" "$3" ;; emit) emit "$2" "$3" ;; --self-test) self_test ;; - *) echo "usage: $0 keys FILE | due CREATED NOW | --self-test" >&2; exit 2 ;; + *) echo "usage: $0 keys FILE | due CREATED NOW | emit FILE RC | --self-test" >&2; exit 2 ;; esac diff --git a/scripts/ci/advisory_diff.sh b/scripts/ci/advisory_diff.sh index 2579246530..aca21e54dd 100755 --- a/scripts/ci/advisory_diff.sh +++ b/scripts/ci/advisory_diff.sh @@ -14,7 +14,7 @@ set -euo pipefail CANT=75 -keys() { # FILE -> keys on stdout; nonzero when the file does not parse +keys() { # FILE -> keys on stdout (use side/checked-keys to fail closed on unparsable input) jq -R -r 'fromjson? | select(.type == "diagnostic" or has("fields")) | .fields | select(.severity == "error") | (.code // "?") as $c | (.advisory.id // "-") as $i | ((.graphs // [{}])[] | .Krate? // {}) | "\($c)|\($i)|\(.name // "?")@\(.version // "?")"' "$1" | LC_ALL=C sort -u @@ -110,6 +110,7 @@ self_test() { case "${1:-}" in --self-test) self_test ;; keys) keys "$2" ;; + checked-keys) side "$2" "$3" ;; base-ref) base_ref "$2" "${3:-}" "${4:-}" ;; ignores-added) ignores_added "$2" "$3" ;; verdict) verdict "$2" "$3" "$4" "$5" ;; diff --git a/scripts/ci/advisory_gate.sh b/scripts/ci/advisory_gate.sh index 928439e3f9..f4f7fad19a 100755 --- a/scripts/ci/advisory_gate.sh +++ b/scripts/ci/advisory_gate.sh @@ -29,7 +29,7 @@ fi bash "$HERE/advisory_diff.sh" base-ref "$EV" "${MG_BASE_SHA:-}" "$BASE" >/dev/null || exit 75 W="${RUNNER_TEMP:-${TMPDIR:-/tmp}}/advgate-base-$$" -trap 'git worktree remove --force "${W:?}" >/dev/null 2>&1 || true; rm -f "${W:?}"/../advgate-pr-$$.ndjson "${W:?}"/../advgate-base-$$.ndjson' EXIT +trap 'rm -f "${W:?}.pr.ndjson" "${W:?}.base.ndjson"; git worktree remove --force "${W:?}" >/dev/null 2>&1 || true' EXIT git worktree add --detach "$W" "$BASE" >/dev/null 2>&1 || { echo "COULD-NOT-CHECK: no worktree for base $BASE (shallow fetch?)" >&2; exit 75; } # One DB for both scans: fetched with the BASE config (the db-path both scans read), HEAD logged. @@ -50,6 +50,6 @@ scan() { # TREE OUT.ndjson -> prints the cargo-deny rc >/dev/null 2>"$2" || rc=$? echo "$rc" } -PRRC=$(scan "$PWD" "$W/../advgate-pr-$$.ndjson") -BASERC=$(scan "$W" "$W/../advgate-base-$$.ndjson") -bash "$GATE" verdict "$W/../advgate-pr-$$.ndjson" "$PRRC" "$W/../advgate-base-$$.ndjson" "$BASERC" +PRRC=$(scan "$PWD" "$W.pr.ndjson") +BASERC=$(scan "$W" "$W.base.ndjson") +bash "$GATE" verdict "$W.pr.ndjson" "$PRRC" "$W.base.ndjson" "$BASERC" From e942ac651b98cbb2542897111086a48bb58c92d4 Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Tue, 20 Oct 2026 15:53:39 +0200 Subject: [PATCH 10/22] ci: advisory andon passes the bashrs DET gate; step cleans scripts dir on self-test failure (PMAT-5792, Refs #5792) Co-Authored-By: Claude Sonnet 5.5 --- .github/workflows/ci.yml | 3 +-- ci/sections.yml | 3 +-- scripts/ci/advisory_andon.sh | 8 ++++---- 3 files changed, 6 insertions(+), 8 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 610f700574..6c4999dbc8 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -355,9 +355,8 @@ jobs: else G=scripts/ci fi - bash "$G/advisory_diff.sh" --self-test rc=0 - bash "$G/advisory_gate.sh" || rc=$? + bash "$G/advisory_diff.sh" --self-test && bash "$G/advisory_gate.sh" || rc=$? rm -rf -- "${S:?}" exit "$rc" - name: Every deny.toml exemption must still be live diff --git a/ci/sections.yml b/ci/sections.yml index 26bfde5f11..e6ffa7bfed 100644 --- a/ci/sections.yml +++ b/ci/sections.yml @@ -2688,9 +2688,8 @@ jobs: else G=scripts/ci fi - bash "$G/advisory_diff.sh" --self-test rc=0 - bash "$G/advisory_gate.sh" || rc=$? + bash "$G/advisory_diff.sh" --self-test && bash "$G/advisory_gate.sh" || rc=$? rm -rf -- "${S:?}" exit "$rc" # Separate from the check above on purpose: a newly-FIXED upstream must diff --git a/scripts/ci/advisory_andon.sh b/scripts/ci/advisory_andon.sh index eb4022ffe8..08dc7837fa 100644 --- a/scripts/ci/advisory_andon.sh +++ b/scripts/ci/advisory_andon.sh @@ -13,7 +13,7 @@ DUE_SECS=$((4 * 3600)) due() { # ISO created, ISO now local c n - c=$(date -u -d "$1" +%s) n=$(date -u -d "$2" +%s) + c=$(date -u -d "$1" +%s) n=$(date -u -d "$2" +%s) # bashrs disable-line=DET002 (elapsed-time arithmetic, not a stored timestamp) [ $((n - c)) -gt "$DUE_SECS" ] } @@ -65,11 +65,11 @@ self_test() { [ "$rc" -eq 75 ] || { echo "SELF-TEST FAIL: rc1 with no findings must be 75, got $rc" >&2; bad=1; } ec 'tool error is cannot-check' 75 - 2 '' ec 'old open issue gets p0-overdue' 0 'add-label p0-overdue' 1 '[{"number":7,"createdAt":"2020-01-01T00:00:00Z","labels":[],"title":"ADVISORY ANDON: vulnerability|RUSTSEC-1|c@1"}]' - ec 'fresh open issue is left alone' 0 - 1 "[{\"number\":7,\"createdAt\":\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\",\"labels\":[],\"title\":\"ADVISORY ANDON: vulnerability|RUSTSEC-1|c@1\"}]" - : >"$d/calls"; GH="$d/gh" STUB_LIST="[{\"number\":7,\"createdAt\":\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\",\"labels\":[],\"title\":\"ADVISORY ANDON: vulnerability|RUSTSEC-1|c@1\"}]" emit "$d/f.json" 1 >/dev/null 2>&1 || true + ec 'fresh open issue is left alone' 0 - 1 "[{\"number\":7,\"createdAt\":\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\",\"labels\":[],\"title\":\"ADVISORY ANDON: vulnerability|RUSTSEC-1|c@1\"}]" # bashrs disable-line=DET002 + : >"$d/calls"; GH="$d/gh" STUB_LIST="[{\"number\":7,\"createdAt\":\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\",\"labels\":[],\"title\":\"ADVISORY ANDON: vulnerability|RUSTSEC-1|c@1\"}]" emit "$d/f.json" 1 >/dev/null 2>&1 || true # bashrs disable-line=DET002 if grep -q 'issue create\|add-label' "$d/calls"; then echo "SELF-TEST FAIL: fresh issue was touched" >&2; bad=1; fi rm -rf -- "${d:?}" - [ "$bad" -eq 0 ] && echo "advisory_andon self-test: 9 cases ok" + [ "$bad" -eq 0 ] && echo "advisory_andon self-test: ok" return "$bad" } From e9ee8d680e4af378ef23d4418501b01acdf6c2c5 Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Tue, 20 Oct 2026 16:33:26 +0200 Subject: [PATCH 11/22] ci: ADG-003/004 gate self-test and sov-advisories-off check, each with RED mutants; re-vendor sovereign-ci (PMAT-5792, Refs #5792) Re-vendors sovereign-ci.yml from paiml/.github#86 head 9078879912 (advisories_diff_only input); move the pin to the merge commit when #86 lands. Co-Authored-By: Claude Sonnet 5.5 --- .github/workflows/ci.yml | 2 +- ci/sections.yml | 6 ++- ci/vendor/sovereign-ci.yml | 66 ++++++++++++++++++++---- contracts/advisory-gate-diff-v1.yaml | 14 ++++-- scripts/ci/advisory_gate_selftest.sh | 70 ++++++++++++++++++++++++++ scripts/ci/check_sov_advisories_off.sh | 47 +++++++++++++++++ 6 files changed, 187 insertions(+), 18 deletions(-) create mode 100644 scripts/ci/advisory_gate_selftest.sh create mode 100644 scripts/ci/check_sov_advisories_off.sh diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 6c4999dbc8..b13526a4ff 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -356,7 +356,7 @@ jobs: G=scripts/ci fi rc=0 - bash "$G/advisory_diff.sh" --self-test && bash "$G/advisory_gate.sh" || rc=$? + bash "$G/advisory_diff.sh" --self-test && bash scripts/ci/check_sov_advisories_off.sh --self-test && bash scripts/ci/check_sov_advisories_off.sh && bash scripts/ci/advisory_gate_selftest.sh "$G/advisory_gate.sh" && bash "$G/advisory_gate.sh" || rc=$? rm -rf -- "${S:?}" exit "$rc" - name: Every deny.toml exemption must still be live diff --git a/ci/sections.yml b/ci/sections.yml index e6ffa7bfed..d7b3d66f25 100644 --- a/ci/sections.yml +++ b/ci/sections.yml @@ -18,12 +18,14 @@ # The old `ci:` job, verbatim: `uses:` is the pin ci/vendor/sovereign-ci.yml was # copied from (scripts/check_ci_reusable_workflow_pinned.sh checks both agree). sovereign-ci: - uses: paiml/.github/.github/workflows/sovereign-ci.yml@f713290c86fcd70d6f26a0faab14e67e6713586f + uses: paiml/.github/.github/workflows/sovereign-ci.yml@9078879912d106237a38137dee06be6ac6c04ac7 with: repo: ${{ github.event.repository.name }} enable_sccache: true use_nextest: true coverage_on: 'tag' + # PMAT-5792: the guard-cargo diff gate is the scanner of record (check_sov_advisories_off.sh) + advisories_diff_only: true # A matrix job becomes one section per pinned combination. workspace-test-shard # runs as THREE shards, each its own ci.yml job on its own runner: as one shard @@ -2689,7 +2691,7 @@ jobs: G=scripts/ci fi rc=0 - bash "$G/advisory_diff.sh" --self-test && bash "$G/advisory_gate.sh" || rc=$? + bash "$G/advisory_diff.sh" --self-test && bash scripts/ci/check_sov_advisories_off.sh --self-test && bash scripts/ci/check_sov_advisories_off.sh && bash scripts/ci/advisory_gate_selftest.sh "$G/advisory_gate.sh" && bash "$G/advisory_gate.sh" || rc=$? rm -rf -- "${S:?}" exit "$rc" # Separate from the check above on purpose: a newly-FIXED upstream must diff --git a/ci/vendor/sovereign-ci.yml b/ci/vendor/sovereign-ci.yml index a5647865d8..2522882439 100644 --- a/ci/vendor/sovereign-ci.yml +++ b/ci/vendor/sovereign-ci.yml @@ -1,5 +1,5 @@ -# VENDORED (#4433): paiml/.github .github/workflows/sovereign-ci.yml @ f713290c86fcd70d6f26a0faab14e67e6713586f -# sha256 of the upstream file: 7d93385d5f4327862e765302e9ea692e80388609563e83d0aa80e33f4497e502 +# VENDORED (#4433): paiml/.github .github/workflows/sovereign-ci.yml @ 9078879912d106237a38137dee06be6ac6c04ac7 +# sha256 of the upstream file: 40ab5f65bfe48630b475d124b1a492a7fcbaa4a9181b4af151889579ceb19c30 # Not a workflow here: scripts/ci/fat_driver.py runs its jobs as sections sov. of # the x86-main fat job. Do not edit by hand -- re-vendor from a new pin, and move the # pin in ci/sections.yml (sovereign-ci.uses) in the same commit. @@ -75,6 +75,14 @@ on: required: false default: 'always' type: string + advisories_diff_only: + description: >- + true: the caller judges advisories itself, as a diff against its base (aprender + PMAT-5792), so the plain `cargo deny check advisories` in `lint` and `cargo audit` + in `security` are skipped; licenses and sources still run. Default false: unchanged. + required: false + default: false + type: boolean run_benchmarks: description: 'Run criterion benchmarks (slow, use for nightly only)' required: false @@ -220,7 +228,7 @@ jobs: # uid:gid because the runner user has no entry in the image's passwd. chown -R 1000:1000 "$WORK_ROOT" 2>/dev/null || true echo "pre-checkout ownership restored under $WORK_ROOT" - - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - name: Install extra packages @@ -360,16 +368,29 @@ jobs: # workspace-member lib tests are exercised. Default stays `--lib` (root only) for # back-compat: many repos have workspace members that don't build in the sovereign-ci # container. Opt-in callers pair this with `test_args` exclusions as needed. + # BIN-ONLY FALLBACK (2026-07-30) — the last link in each chain drops + # `--lib`. Every branch above passes it, and `cargo test --lib` does not + # merely fail on a crate without a [lib]: it ERRORS with "no library + # targets found in package " (exit 101). A bin-only repo could + # therefore never pass this job. Dropping `--lib` covers bins, + # integration tests and doctests, with or without a [lib]. + # + # Purely additive: a crate that HAS a lib target satisfies an earlier + # branch and never reaches these lines, so nothing changes for the repos + # that work today. if [ "$USE_NEXTEST" = "true" ]; then cargo nextest run $TEST_SCOPE $TEST_ARGS 2>&1 || \ cargo nextest run --lib -p "$REPO_NAME" $TEST_ARGS 2>&1 || \ + cargo nextest run $TEST_ARGS 2>&1 || \ { echo "::warning::nextest failed — falling back to cargo test"; \ cargo test $TEST_SCOPE $TEST_ARGS 2>&1 || \ cargo test --lib -p "$REPO_NAME" $TEST_ARGS 2>&1 || \ + cargo test $TEST_ARGS 2>&1 || \ { echo "::error::Tests failed — check workspace path dependencies"; exit 1; }; } else cargo test $TEST_SCOPE $TEST_ARGS 2>&1 || \ cargo test --lib -p "$REPO_NAME" $TEST_ARGS 2>&1 || \ + cargo test $TEST_ARGS 2>&1 || \ { echo "::error::Tests failed — check workspace path dependencies"; exit 1; } fi - name: Record sccache stats @@ -467,7 +488,7 @@ jobs: # uid:gid because the runner user has no entry in the image's passwd. chown -R 1000:1000 "$WORK_ROOT" 2>/dev/null || true echo "pre-checkout ownership restored under $WORK_ROOT" - - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - name: Install extra packages @@ -614,6 +635,7 @@ jobs: # the stricter check. env: GITHUB_TOKEN: ${{ github.token }} + SOV_ADVISORIES_DIFF_ONLY: ${{ inputs.advisories_diff_only }} run: | # forjar#422: the advisory-db clone is an anonymous git-over-HTTPS # fetch of a public repo, and GitHub answers the fleet's IP with 401 @@ -631,7 +653,13 @@ jobs: GIT_CONFIG_VALUE_0="AUTHORIZATION: basic $(printf 'x-access-token:%s' "$GITHUB_TOKEN" | base64 | tr -d '\n')" export GIT_CONFIG_COUNT GIT_CONFIG_KEY_0 GIT_CONFIG_VALUE_0 GIT_TERMINAL_PROMPT=0 if [ -f deny.toml ]; then - cargo deny check advisories licenses sources + # advisories_diff_only: true: the caller judges advisories as a diff against its + # base (aprender PMAT-5792), so a main-resident advisory must not stop its PRs. + if [ "${SOV_ADVISORIES_DIFF_ONLY:-}" = true ]; then + cargo deny check licenses sources + else + cargo deny check advisories licenses sources + fi else echo "::warning::No deny.toml — skipping supply chain audit" fi @@ -722,7 +750,7 @@ jobs: # uid:gid because the runner user has no entry in the image's passwd. chown -R 1000:1000 "$WORK_ROOT" 2>/dev/null || true echo "pre-checkout ownership restored under $WORK_ROOT" - - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - name: Install extra packages @@ -854,6 +882,17 @@ jobs: # test job comment for back-compat rationale. cargo llvm-cov test $TEST_SCOPE --no-cfg-coverage --no-cfg-coverage-nightly --lcov --output-path lcov.info $TEST_ARGS 2>&1 || \ cargo llvm-cov test --lib --no-cfg-coverage --no-cfg-coverage-nightly -p "$REPO_NAME" --lcov --output-path lcov.info 2>&1 || \ + # BIN-ONLY FALLBACK (2026-07-30). Every branch above passes `--lib`, and + # `cargo test --lib` does not merely fail on a crate without a [lib] — it + # ERRORS: "no library targets found in package " (exit 101). So a + # bin-only repo could never produce coverage; cohete's job went red for + # this and nothing else. Dropping `--lib` covers bins, integration tests + # and doctests, and works with or without a [lib]. + # + # Purely additive: a crate that HAS a lib target satisfies one of the + # branches above and never reaches this line, so behaviour is unchanged + # for every repo that works today. + cargo llvm-cov test --no-cfg-coverage --no-cfg-coverage-nightly --lcov --output-path lcov.info 2>&1 || \ { echo "::error::Coverage failed — check workspace path dependencies"; exit 1; } - name: Enforce coverage floor (OPT-IN ratchet — PMAT build-system audit gap #1) # ZERO BEHAVIOR CHANGE GUARANTEE: @@ -1014,7 +1053,7 @@ jobs: # uid:gid because the runner user has no entry in the image's passwd. chown -R 1000:1000 "$WORK_ROOT" 2>/dev/null || true echo "pre-checkout ownership restored under $WORK_ROOT" - - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - name: Install extra packages @@ -1152,7 +1191,7 @@ jobs: 2>/dev/null || echo "::warning::sccache stats unavailable" - name: Upload benchmark results if: always() - uses: actions/upload-artifact@v4 + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 with: name: bench-${{ inputs.repo }} path: bench-results.txt @@ -1241,7 +1280,7 @@ jobs: else echo "pre-checkout ownership restored under $WORK_ROOT" fi - - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - name: roadmap-valid @@ -1472,6 +1511,7 @@ jobs: - name: Audit env: GITHUB_TOKEN: ${{ github.token }} + SOV_ADVISORIES_DIFF_ONLY: ${{ inputs.advisories_diff_only }} run: | # forjar#422: same throttle as the cargo-deny step — authenticate the # advisory-db fetch with the job's own token via a url rewrite. @@ -1512,7 +1552,11 @@ jobs: fi fi - cargo audit $IGNORE_FLAGS + if [ "${SOV_ADVISORIES_DIFF_ONLY:-}" = true ]; then + echo "cargo audit skipped: the caller's diff gate is the scanner of record (advisories_diff_only: true)" + else + cargo audit $IGNORE_FLAGS + fi # Symmetry with the container jobs (paiml/.github#42): every clean-room # job both tolerates poison on entry and refuses to leave any on exit, # so the invariant is uniform and greppable rather than per-job folklore. @@ -1568,7 +1612,7 @@ jobs: attestations: write contents: read steps: - - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - name: Generate SLSA provenance diff --git a/contracts/advisory-gate-diff-v1.yaml b/contracts/advisory-gate-diff-v1.yaml index 64ced0dfdb..a7f8d85688 100644 --- a/contracts/advisory-gate-diff-v1.yaml +++ b/contracts/advisory-gate-diff-v1.yaml @@ -96,15 +96,21 @@ falsification_tests: - id: FALSIFY-ADG-003 rule: "Scanner of record sees the optional-feature crate" prediction: "on the 48.0.6 lock the scanner of record exits non-zero and names 0378 and 0379" - test: "run the scanner of record on the 48.0.6 lock fixture" + test: "scripts/ci/advisory_gate_selftest.sh runs advisory_gate.sh against a stub cargo; every advisories scan must carry --all-features (the 48.0.6 lock fixture is covered by ADG-001 verdicts)" if_fails: "#5791 is not resolved: the record scanner is as blind as deny with all-features=false" - mutation: "flip all-features to false in the scanner config; the case must turn RED" + mutation: "sed out --all-features from advisory_gate.sh scan(); the 'ADG-003' mutant case in advisory_gate_selftest.sh must turn RED (measured 2026-10-20)" - id: FALSIFY-ADG-004 rule: "Same DB snapshot on both sides" prediction: "a DB entry published between the main run and the PR run does not appear in the delta" - test: "case table: two DB snapshots, delta computed against one" + test: "advisory_gate_selftest.sh: exactly one `cargo deny fetch`, both scans --disable-fetch with one --config" if_fails: "DB refreshes read as PR-introduced advisories and re-create the stop" - mutation: "read the DB twice; the case must turn RED" + mutation: "sed out --disable-fetch, or add a second fetch before the base scan; the two 'ADG-004' mutant cases must turn RED (measured 2026-10-20)" + - id: FALSIFY-ADG-007 + rule: "No second scanner blocks a PR on main's advisories" + prediction: "sov.lint and sov.security skip plain cargo deny advisories / cargo audit (advisories_diff_only: true in the vendored sovereign-ci)" + test: "scripts/ci/check_sov_advisories_off.sh --self-test, 4 cases, then the real tree" + if_fails: "the vendored scans still stop every PR on an advisory already on main (the #5793 REJECT)" + mutation: "set advisories_diff_only: false in ci/sections.yml, or change the vendored '= true' test; the matching case must turn RED (measured 2026-10-20)" - id: FALSIFY-ADG-005 rule: "Andon clock" prediction: "the P0 ticket's due time is <= 4 h after the advisory first appears on main" diff --git a/scripts/ci/advisory_gate_selftest.sh b/scripts/ci/advisory_gate_selftest.sh new file mode 100644 index 0000000000..de6d0fd1b8 --- /dev/null +++ b/scripts/ci/advisory_gate_selftest.sh @@ -0,0 +1,70 @@ +#!/usr/bin/env bash +# advisory_gate_selftest.sh -- FALSIFY-ADG-003/004 (contracts/advisory-gate-diff-v1.yaml): run +# advisory_gate.sh against a stub `cargo` and read what it asked cargo to do. +# ADG-003 every advisories scan carries --all-features (the optional-feature crate is in scope) +# ADG-004 ONE advisory-db fetch; both scans read it with --disable-fetch and the same --config +# usage: advisory_gate_selftest.sh [GATE_SCRIPT] (default: the sibling advisory_gate.sh) +set -euo pipefail + +HERE="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" +GATE_SRC="${1:-$HERE/advisory_gate.sh}" + +run_gate() { # prints the stub's argv log + local d repo log + d=$(mktemp -d) + # shellcheck disable=SC2064 + trap "rm -rf -- '${d:?}'" RETURN + repo="$d/repo"; log="$d/cargo.log"; mkdir -p "$repo" "$d/bin" "$d/home" "$d/scripts" + cp "$GATE_SRC" "$d/scripts/advisory_gate.sh"; cp "$HERE/advisory_diff.sh" "$d/scripts/advisory_diff.sh" + printf '#!/usr/bin/env bash\necho "$*" >>"%s"\nexit 0\n' "$log" >"$d/bin/cargo"; chmod +x "$d/bin/cargo" + ( + cd "$repo" + git init -q . && printf '[advisories]\n' >deny.toml && git add deny.toml + # plumbing, not `git commit`: a throwaway repo must not run the global commit hooks + git update-ref refs/remotes/origin/main "$(git -c user.name=t -c user.email=t@t commit-tree -m base "$(git write-tree)")" + git update-ref HEAD refs/remotes/origin/main + PATH="$d/bin:$PATH" HOME="$d/home" TMPDIR="$d" GITHUB_EVENT_NAME=pull_request \ + bash "$d/scripts/advisory_gate.sh" >/dev/null 2>&1 + ) || { echo "gate-did-not-pass"; return 0; } + cat "$log" +} + +verdict_of() { # stdin: stub log -> "ok" or the first defect + local log fetch checks nall nfd ncfg + log=$(cat) + [ "$log" != gate-did-not-pass ] || { echo "gate did not pass against the stub"; return; } + fetch=$(grep -c '^deny fetch' <<<"$log" || true) + checks=$(grep -c 'check' <<<"$(grep -v '^deny fetch' <<<"$log")" || true) + nall=$(grep -c -- '--all-features' <<<"$(grep -v '^deny fetch' <<<"$log")" || true) + nfd=$(grep -c -- '--disable-fetch' <<<"$(grep -v '^deny fetch' <<<"$log")" || true) + ncfg=$(grep -v '^deny fetch' <<<"$log" | grep -o -- '--config [^ ]*' | sort -u | wc -l) + [ "$checks" -eq 2 ] || { echo "expected 2 scans, saw $checks"; return; } + [ "$nall" -eq 2 ] || { echo "ADG-003: $nall of 2 scans carry --all-features"; return; } + [ "$fetch" -eq 1 ] || { echo "ADG-004: $fetch advisory-db fetches, want 1"; return; } + [ "$nfd" -eq 2 ] || { echo "ADG-004: $nfd of 2 scans carry --disable-fetch"; return; } + [ "$ncfg" -eq 1 ] || { echo "ADG-004: scans use $ncfg different --config files"; return; } + echo ok +} + +main() { + local bad=0 d m got + d=$(mktemp -d) + # shellcheck disable=SC2064 + trap "rm -rf -- '${d:?}'" EXIT + got=$(run_gate | verdict_of) + [ "$got" = ok ] || { echo "SELF-TEST FAIL: real gate: $got" >&2; bad=1; } + mutant() { # name sed-expr want-substring + sed -e "$2" "$GATE_SRC" >"$d/m.sh" + cmp -s "$d/m.sh" "$GATE_SRC" && { echo "SELF-TEST FAIL: mutant '$1' changed nothing" >&2; bad=1; return; } + m=$(run_gate_with "$d/m.sh" | verdict_of) + [[ "$m" == *"$3"* ]] || { echo "SELF-TEST FAIL: mutant '$1' stayed green: $m" >&2; bad=1; } + } + mutant 'scan without --all-features' 's/ --all-features//' 'ADG-003' + mutant 'scan reads the net (no --disable-fetch)' 's/ --disable-fetch//' 'ADG-004' + mutant 'second fetch before the base scan' 's|^BASERC=|cargo deny fetch --config "$W/deny.toml" db; BASERC=|' 'ADG-004' + [ "$bad" -eq 0 ] && echo "advisory_gate self-test: 1 real + 3 mutants ok" + return "$bad" +} + +run_gate_with() { GATE_SRC="$1"; run_gate; } +main diff --git a/scripts/ci/check_sov_advisories_off.sh b/scripts/ci/check_sov_advisories_off.sh new file mode 100644 index 0000000000..9333c84682 --- /dev/null +++ b/scripts/ci/check_sov_advisories_off.sh @@ -0,0 +1,47 @@ +#!/usr/bin/env bash +# check_sov_advisories_off.sh -- PMAT-5792: the vendored sov.lint `cargo deny check advisories` and +# sov.security `cargo audit` stop PRs on advisories already on main. They are off only when BOTH halves hold: +# 1. ci/sections.yml sovereign-ci.with sets advisories_diff_only: true +# 2. the vendored body reads that input in both steps and skips the scan on it +# usage: check_sov_advisories_off.sh [SECTIONS] [VENDORED] | --self-test +set -euo pipefail + +HERE="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" +ROOT="$(cd -- "$HERE/../.." && pwd)" + +check() { # SECTIONS VENDORED -> 0 ok, 1 gap + local sec=$1 ven=$2 bad=0 + grep -Eq '^ advisories_diff_only: true$' "$sec" || { echo "GAP: $sec does not set advisories_diff_only: true under sovereign-ci.with" >&2; bad=1; } + [ "$(grep -c 'SOV_ADVISORIES_DIFF_ONLY: \${{ inputs.advisories_diff_only }}' "$ven")" -ge 2 ] \ + || { echo "GAP: $ven does not pass the input to both the deny and the audit step" >&2; bad=1; } + [ "$(grep -c '"${SOV_ADVISORIES_DIFF_ONLY:-}" = true' "$ven")" -ge 2 ] \ + || { echo "GAP: $ven does not skip both scans on the input" >&2; bad=1; } + return "$bad" +} + +self_test() { + local d bad=0 rc + d=$(mktemp -d) + # shellcheck disable=SC2064 + trap "rm -rf -- '${d:?}'" RETURN + case_() { # name want sections vendored + rc=0; check "$3" "$4" >/dev/null 2>&1 || rc=$? + [ "$rc" -eq "$2" ] || { echo "SELF-TEST FAIL: $1 wanted rc=$2 got $rc" >&2; bad=1; } + } + S="$ROOT/ci/sections.yml"; V="$ROOT/ci/vendor/sovereign-ci.yml" + case_ 'real tree' 0 "$S" "$V" + sed 's/^ advisories_diff_only: true$/ advisories_diff_only: false/' "$S" >"$d/s.yml" + case_ 'MUTANT input off in sections' 1 "$d/s.yml" "$V" + sed 's/"${SOV_ADVISORIES_DIFF_ONLY:-}" = true/"${SOV_ADVISORIES_DIFF_ONLY:-}" = never/' "$V" >"$d/v.yml" + case_ 'MUTANT vendored body does not honour it' 1 "$S" "$d/v.yml" + sed '0,/SOV_ADVISORIES_DIFF_ONLY: ${{ inputs.advisories_diff_only }}/{//d}' "$V" >"$d/v2.yml" + case_ 'MUTANT one step loses the env' 1 "$S" "$d/v2.yml" + [ "$bad" -eq 0 ] && echo "check_sov_advisories_off self-test: 4 cases ok" + return "$bad" +} + +case "${1:-}" in + --self-test) self_test ;; + "") check "$ROOT/ci/sections.yml" "$ROOT/ci/vendor/sovereign-ci.yml" ;; + *) check "$1" "$2" ;; +esac From 0c08174520e28c8069984fa927c8204371f43ce0 Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Tue, 20 Oct 2026 17:05:48 +0200 Subject: [PATCH 12/22] =?UTF-8?q?ci:=20advisory=20gate=20=E2=80=94=20ADG?= =?UTF-8?q?=20ids=20in=20order,=20rearm=20self-test=20wired,=20--all-featu?= =?UTF-8?q?res=20on=20every=20scan=20site=20(Refs=20#5792)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Sonnet 5.5 --- .github/workflows/advisory-andon.yml | 1 + .github/workflows/ci.yml | 2 +- ci/sections.yml | 2 +- contracts/advisory-gate-diff-v1.yaml | 13 +++++++------ scripts/check_deny_exemptions_live.sh | 2 +- scripts/ci/advisory_gate.sh | 2 +- scripts/ci/advisory_gate_selftest.sh | 9 ++++++++- 7 files changed, 20 insertions(+), 11 deletions(-) diff --git a/.github/workflows/advisory-andon.yml b/.github/workflows/advisory-andon.yml index 172a251d06..4b5479ee55 100644 --- a/.github/workflows/advisory-andon.yml +++ b/.github/workflows/advisory-andon.yml @@ -38,6 +38,7 @@ jobs: GIT_CONFIG_VALUE_0="AUTHORIZATION: basic $(printf 'x-access-token:%s' "$GITHUB_TOKEN" | base64 | tr -d '\n')" export GIT_CONFIG_VALUE_0 bash scripts/ci/advisory_andon.sh --self-test + bash scripts/ci/rearm_ready.sh --self-test out="${RUNNER_TEMP:?}/andon.ndjson" rc=0 cargo deny --locked --all-features --format json check advisories >/dev/null 2>"$out" || rc=$? diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b13526a4ff..0ff363d6df 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -356,7 +356,7 @@ jobs: G=scripts/ci fi rc=0 - bash "$G/advisory_diff.sh" --self-test && bash scripts/ci/check_sov_advisories_off.sh --self-test && bash scripts/ci/check_sov_advisories_off.sh && bash scripts/ci/advisory_gate_selftest.sh "$G/advisory_gate.sh" && bash "$G/advisory_gate.sh" || rc=$? + bash "$G/advisory_diff.sh" --self-test && bash scripts/ci/check_sov_advisories_off.sh --self-test && bash scripts/ci/check_sov_advisories_off.sh && bash scripts/ci/advisory_gate_selftest.sh "$G/advisory_gate.sh" && bash scripts/ci/rearm_ready.sh --self-test && bash "$G/advisory_gate.sh" || rc=$? rm -rf -- "${S:?}" exit "$rc" - name: Every deny.toml exemption must still be live diff --git a/ci/sections.yml b/ci/sections.yml index d7b3d66f25..651cfe05e0 100644 --- a/ci/sections.yml +++ b/ci/sections.yml @@ -2691,7 +2691,7 @@ jobs: G=scripts/ci fi rc=0 - bash "$G/advisory_diff.sh" --self-test && bash scripts/ci/check_sov_advisories_off.sh --self-test && bash scripts/ci/check_sov_advisories_off.sh && bash scripts/ci/advisory_gate_selftest.sh "$G/advisory_gate.sh" && bash "$G/advisory_gate.sh" || rc=$? + bash "$G/advisory_diff.sh" --self-test && bash scripts/ci/check_sov_advisories_off.sh --self-test && bash scripts/ci/check_sov_advisories_off.sh && bash scripts/ci/advisory_gate_selftest.sh "$G/advisory_gate.sh" && bash scripts/ci/rearm_ready.sh --self-test && bash "$G/advisory_gate.sh" || rc=$? rm -rf -- "${S:?}" exit "$rc" # Separate from the check above on purpose: a newly-FIXED upstream must diff --git a/contracts/advisory-gate-diff-v1.yaml b/contracts/advisory-gate-diff-v1.yaml index a7f8d85688..ab1aa6dcce 100644 --- a/contracts/advisory-gate-diff-v1.yaml +++ b/contracts/advisory-gate-diff-v1.yaml @@ -45,6 +45,7 @@ equations: - "Exemptions live in one file" - "Findings are keyed code|advisory-id|crate@version over every graph, so a new crate under an id main already carries, and a yanked crate with no id, are both PR-introduced" - "Both sides are scanned with the BASE deny.toml; a PR that adds an advisories ignore id fails (ignores-added), so a PR cannot hide its own finding" + - "deny.toml advisories ignores are id-wide: cargo-deny has no per-crate ignore. They stay bounded, not scoped: ignores-added fails any PR that adds one, and check_deny_exemptions_live.sh (now --all-features, same as the gate) fails a dead one. Per-crate scoping needs an upstream cargo-deny feature and is out of this PR" - "The base is pinned: merge_group uses merge_group.base_sha, pull_request uses pull_request.base.sha, anything else (or a non-sha) is COULD-NOT-CHECK rc 75, never a pass" auto_rearm: formula: "when a gate fix lands, every PR that was ready (green, approved) before the stop is re-armed without a human" @@ -105,12 +106,6 @@ falsification_tests: test: "advisory_gate_selftest.sh: exactly one `cargo deny fetch`, both scans --disable-fetch with one --config" if_fails: "DB refreshes read as PR-introduced advisories and re-create the stop" mutation: "sed out --disable-fetch, or add a second fetch before the base scan; the two 'ADG-004' mutant cases must turn RED (measured 2026-10-20)" - - id: FALSIFY-ADG-007 - rule: "No second scanner blocks a PR on main's advisories" - prediction: "sov.lint and sov.security skip plain cargo deny advisories / cargo audit (advisories_diff_only: true in the vendored sovereign-ci)" - test: "scripts/ci/check_sov_advisories_off.sh --self-test, 4 cases, then the real tree" - if_fails: "the vendored scans still stop every PR on an advisory already on main (the #5793 REJECT)" - mutation: "set advisories_diff_only: false in ci/sections.yml, or change the vendored '= true' test; the matching case must turn RED (measured 2026-10-20)" - id: FALSIFY-ADG-005 rule: "Andon clock" prediction: "the P0 ticket's due time is <= 4 h after the advisory first appears on main" @@ -123,6 +118,12 @@ falsification_tests: test: "scripts/ci/rearm_ready.sh --self-test, 2 cases (head unchanged arms, moved or unapproved does not, empty arms nothing)" if_fails: "a gate fix arms an unreviewed head into the queue" mutation: "drop the approved_sha == current_sha test in select_; the moved-head case must turn RED" + - id: FALSIFY-ADG-007 + rule: "No second scanner blocks a PR on main's advisories" + prediction: "sov.lint and sov.security skip plain cargo deny advisories / cargo audit (advisories_diff_only: true in the vendored sovereign-ci)" + test: "scripts/ci/check_sov_advisories_off.sh --self-test, 4 cases, then the real tree" + if_fails: "the vendored scans still stop every PR on an advisory already on main (the #5793 REJECT)" + mutation: "set advisories_diff_only: false in ci/sections.yml, or change the vendored '= true' test; the matching case must turn RED (measured 2026-10-20)" qa_gate: id: F-ADG-001 diff --git a/scripts/check_deny_exemptions_live.sh b/scripts/check_deny_exemptions_live.sh index 911b83e3e9..adf7177427 100755 --- a/scripts/check_deny_exemptions_live.sh +++ b/scripts/check_deny_exemptions_live.sh @@ -35,7 +35,7 @@ trap 'rm -f "${LOG:?}"' EXIT # Redirect, never pipe: reading this through a pipe would report the exit status # of the last stage instead of cargo-deny's. -cargo deny check advisories > "$LOG" 2>&1 +cargo deny --all-features check advisories > "$LOG" 2>&1 rc=$? declared="$(grep -c 'id = "RUSTSEC' deny.toml || true)" diff --git a/scripts/ci/advisory_gate.sh b/scripts/ci/advisory_gate.sh index f4f7fad19a..3c1687ebd0 100755 --- a/scripts/ci/advisory_gate.sh +++ b/scripts/ci/advisory_gate.sh @@ -14,7 +14,7 @@ HERE="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" EV="${GITHUB_EVENT_NAME:-}" case "$EV" in pull_request | merge_group) ;; - *) exec cargo deny check advisories ;; + *) exec cargo deny --all-features check advisories ;; esac base_sha() { # the build base the pin step named diff --git a/scripts/ci/advisory_gate_selftest.sh b/scripts/ci/advisory_gate_selftest.sh index de6d0fd1b8..20b1a67997 100644 --- a/scripts/ci/advisory_gate_selftest.sh +++ b/scripts/ci/advisory_gate_selftest.sh @@ -62,7 +62,14 @@ main() { mutant 'scan without --all-features' 's/ --all-features//' 'ADG-003' mutant 'scan reads the net (no --disable-fetch)' 's/ --disable-fetch//' 'ADG-004' mutant 'second fetch before the base scan' 's|^BASERC=|cargo deny fetch --config "$W/deny.toml" db; BASERC=|' 'ADG-004' - [ "$bad" -eq 0 ] && echo "advisory_gate self-test: 1 real + 3 mutants ok" + # ADG-003 on the other two scan sites: the push path and the exemption-liveness guard. + local f + for f in "$GATE_SRC" "${EXEMPT_SRC:-scripts/check_deny_exemptions_live.sh}"; do + if grep -v printf "$f" | grep -E '^[^#]*cargo deny .*check .*advisories' | grep -qv -- '--all-features'; then + echo "SELF-TEST FAIL: ADG-003: $f scans advisories without --all-features" >&2; bad=1 + fi + done + [ "$bad" -eq 0 ] && echo "advisory_gate self-test: 1 real + 3 mutants + 2 sites ok" return "$bad" } From ed33417585db1c738fbe41267c3161c62ca71fe3 Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Tue, 20 Oct 2026 17:53:32 +0200 Subject: [PATCH 13/22] ci: wire rearm_ready select into a real caller (rearm-ready.yml) with a stub-gh RED row (Refs #5792) Co-Authored-By: Claude Sonnet 5.5 --- .github/workflows/rearm-ready.yml | 28 +++++++++++++++++++ contracts/advisory-gate-diff-v1.yaml | 4 +-- scripts/ci/rearm_ready.sh | 40 +++++++++++++++++++++++++--- 3 files changed, 67 insertions(+), 5 deletions(-) create mode 100644 .github/workflows/rearm-ready.yml diff --git a/.github/workflows/rearm-ready.yml b/.github/workflows/rearm-ready.yml new file mode 100644 index 0000000000..cc97410173 --- /dev/null +++ b/.github/workflows/rearm-ready.yml @@ -0,0 +1,28 @@ +name: rearm-ready +# PMAT-5792 (ADG-INV-005): after main moves or on a timer, re-arm ready PRs whose head is the one that was approved. +# Never a PR or merge_group job. A PR whose head moved after approval is left alone (needs a fresh review). +on: + schedule: + - cron: '41 * * * *' + push: + branches: [main] + workflow_dispatch: +permissions: + contents: read + pull-requests: write +concurrency: + group: rearm-ready + cancel-in-progress: false +jobs: + rearm: + runs-on: [self-hosted, Linux, X64, clean-room] + timeout-minutes: 10 + steps: + - uses: actions/checkout@v7 + - name: Self-test, then re-arm approved un-armed PRs + env: + GH_TOKEN: ${{ github.token }} + run: | + set -euo pipefail + bash scripts/ci/rearm_ready.sh --self-test + bash scripts/ci/rearm_ready.sh run diff --git a/contracts/advisory-gate-diff-v1.yaml b/contracts/advisory-gate-diff-v1.yaml index ab1aa6dcce..6748b93a8c 100644 --- a/contracts/advisory-gate-diff-v1.yaml +++ b/contracts/advisory-gate-diff-v1.yaml @@ -115,9 +115,9 @@ falsification_tests: - id: FALSIFY-ADG-006 rule: "Re-arm never arms a changed head (ADG-INV-005)" prediction: "a ready PR whose head sha moved after approval is not re-armed; one whose head is unchanged is" - test: "scripts/ci/rearm_ready.sh --self-test, 2 cases (head unchanged arms, moved or unapproved does not, empty arms nothing)" + test: "scripts/ci/rearm_ready.sh --self-test, 3 cases (head unchanged arms, moved or unapproved does not, empty arms nothing, and `run` against a stub gh arms exactly the approved-unchanged PR); .github/workflows/rearm-ready.yml calls `rearm_ready.sh run`" if_fails: "a gate fix arms an unreviewed head into the queue" - mutation: "drop the approved_sha == current_sha test in select_; the moved-head case must turn RED" + mutation: "drop the approved_sha == current_sha test in select_; the moved-head case must turn RED; replace select_ in run_ with cat|cut and the stub-gh case must turn RED (measured 2026-10-20)" - id: FALSIFY-ADG-007 rule: "No second scanner blocks a PR on main's advisories" prediction: "sov.lint and sov.security skip plain cargo deny advisories / cargo audit (advisories_diff_only: true in the vendored sovereign-ci)" diff --git a/scripts/ci/rearm_ready.sh b/scripts/ci/rearm_ready.sh index 3d15f99863..b62f37f33e 100755 --- a/scripts/ci/rearm_ready.sh +++ b/scripts/ci/rearm_ready.sh @@ -2,7 +2,8 @@ # rearm_ready.sh -- which ready PRs a gate fix may re-arm (contract advisory-gate-diff-v1, ADG-INV-005). # # rearm_ready.sh select FILE FILE = TSV "pr approved_sha current_sha"; prints PRs to arm -# rearm_ready.sh --self-test case table +# rearm_ready.sh run list open un-armed PRs (one gh call), select, arm each with plain `gh pr merge` +# rearm_ready.sh --self-test case table, plus `run` against a stub gh (RED when select_ is not called) # A PR whose head moved after its approval is NOT re-armed (a changed head needs a fresh review). set -euo pipefail @@ -10,6 +11,25 @@ select_() { awk -F'\t' 'NF >= 3 && $1 != "" && $2 ~ /^[0-9a-f]{40}$/ && $2 == $3 { print $1 }' "$1" } +# stdin: `gh pr list --json number,headRefOid,autoMergeRequest,reviews`; stdout: TSV "pr approved_sha current_sha" +# approved_sha = the full sha named in the latest review whose body says APPROVE and not HOLD/CHANGES. +plan_() { + jq -r '.[] | select(.autoMergeRequest == null) | . as $p + | ([.reviews[] | select(.body | test("APPROVE") and (test("HOLD|CHANGES-NEEDED|REQUEST_CHANGES") | not))] | last // {body: ""}) as $r + | [$p.number, ($r.body | capture("(?[0-9a-f]{40})").s // ""), $p.headRefOid] | @tsv' +} + +run_() { + local t n + t=$(mktemp) + # shellcheck disable=SC2064 + trap "rm -f -- '${t:?}'" RETURN + gh pr list --state open --limit 100 --json number,headRefOid,autoMergeRequest,reviews | plan_ >"$t" + for n in $(select_ "$t"); do + gh pr merge "$n" || echo "rearm: PR $n not armed (see above)" >&2 + done +} + self_test() { local d bad=0 a b d=$(mktemp -d) @@ -20,12 +40,26 @@ self_test() { [ "$(select_ "$d/t.tsv" | tr '\n' ' ')" = "1 " ] || { echo "SELF-TEST FAIL: only PR 1 (head unchanged) may be re-armed" >&2; bad=1; } printf '' >"$d/e.tsv" [ -z "$(select_ "$d/e.tsv")" ] || { echo "SELF-TEST FAIL: empty input arms nothing" >&2; bad=1; } - [ "$bad" -eq 0 ] && echo "rearm_ready self-test: 2 cases ok" + # run_ end to end: stub gh lists PR 1 (approved, head unchanged), 2 (head moved), 3 (already armed); only 1 is armed. + mkdir -p "$d/bin" + cat >"$d/bin/gh" <>"$d/armed" +fi +STUB + chmod +x "$d/bin/gh" + PATH="$d/bin:$PATH" run_ >/dev/null 2>&1 || true + [ "$(cat "$d/armed" 2>/dev/null)" = "pr merge 1" ] || { echo "SELF-TEST FAIL: run must arm exactly PR 1 via select_, armed: $(tr '\n' ',' <"$d/armed" 2>/dev/null)" >&2; bad=1; } + [ "$bad" -eq 0 ] && echo "rearm_ready self-test: 3 cases ok" return "$bad" } case "${1:-}" in --self-test) self_test ;; select) select_ "$2" ;; - *) echo "usage: $0 select FILE | --self-test" >&2; exit 2 ;; + run) run_ ;; + *) echo "usage: $0 select FILE | run | --self-test" >&2; exit 2 ;; esac From 6e5e9e0e4c16044392db6c274016ad5f65abcfbe Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Tue, 20 Oct 2026 18:20:03 +0200 Subject: [PATCH 14/22] =?UTF-8?q?ci:=20rearm=5Fready=20=E2=80=94=20a=20lat?= =?UTF-8?q?er=20HOLD/CHANGES-NEEDED=20cancels=20an=20APPROVE;=20arm=20with?= =?UTF-8?q?=20expectedHeadOid=20(Refs=20#5792)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Sonnet 5.5 --- contracts/advisory-gate-diff-v1.yaml | 2 +- scripts/ci/rearm_ready.sh | 53 +++++++++++++++++++--------- 2 files changed, 38 insertions(+), 17 deletions(-) diff --git a/contracts/advisory-gate-diff-v1.yaml b/contracts/advisory-gate-diff-v1.yaml index 6748b93a8c..9c9a9c6a61 100644 --- a/contracts/advisory-gate-diff-v1.yaml +++ b/contracts/advisory-gate-diff-v1.yaml @@ -115,7 +115,7 @@ falsification_tests: - id: FALSIFY-ADG-006 rule: "Re-arm never arms a changed head (ADG-INV-005)" prediction: "a ready PR whose head sha moved after approval is not re-armed; one whose head is unchanged is" - test: "scripts/ci/rearm_ready.sh --self-test, 3 cases (head unchanged arms, moved or unapproved does not, empty arms nothing, and `run` against a stub gh arms exactly the approved-unchanged PR); .github/workflows/rearm-ready.yml calls `rearm_ready.sh run`" + test: "scripts/ci/rearm_ready.sh --self-test, 3 cases (head unchanged arms, moved or unapproved does not, empty arms nothing, and `run` against a stub gh enqueues exactly the PRs whose LATEST review is an APPROVE, each with expectedHeadOid = the approved sha; a later HOLD/CHANGES-NEEDED cancels an earlier APPROVE, and a mutant without expectedHeadOid or without the cancel turns it RED); .github/workflows/rearm-ready.yml calls `rearm_ready.sh run`" if_fails: "a gate fix arms an unreviewed head into the queue" mutation: "drop the approved_sha == current_sha test in select_; the moved-head case must turn RED; replace select_ in run_ with cat|cut and the stub-gh case must turn RED (measured 2026-10-20)" - id: FALSIFY-ADG-007 diff --git a/scripts/ci/rearm_ready.sh b/scripts/ci/rearm_ready.sh index b62f37f33e..3f7819a8d5 100755 --- a/scripts/ci/rearm_ready.sh +++ b/scripts/ci/rearm_ready.sh @@ -8,26 +8,45 @@ set -euo pipefail select_() { - awk -F'\t' 'NF >= 3 && $1 != "" && $2 ~ /^[0-9a-f]{40}$/ && $2 == $3 { print $1 }' "$1" + awk -F'\t' 'NF >= 3 && $1 != "" && $2 ~ /^[0-9a-f]{40}$/ && $2 == $3 { print $1 "\t" $3 "\t" $4 }' "$1" } -# stdin: `gh pr list --json number,headRefOid,autoMergeRequest,reviews`; stdout: TSV "pr approved_sha current_sha" -# approved_sha = the full sha named in the latest review whose body says APPROVE and not HOLD/CHANGES. +# stdin: `gh pr list --json id,number,headRefOid,autoMergeRequest,reviews`; stdout: TSV "pr approved_sha current_sha node_id" +# approved_sha = the full sha named in the latest APPROVE review, unless a LATER review is negative +# (HOLD, CHANGES-NEEDED, REQUEST_CHANGES, state CHANGES_REQUESTED): a later negative review cancels the approval. plan_() { jq -r '.[] | select(.autoMergeRequest == null) | . as $p - | ([.reviews[] | select(.body | test("APPROVE") and (test("HOLD|CHANGES-NEEDED|REQUEST_CHANGES") | not))] | last // {body: ""}) as $r - | [$p.number, ($r.body | capture("(?[0-9a-f]{40})").s // ""), $p.headRefOid] | @tsv' + | def neg: (.state == "CHANGES_REQUESTED") or (.body | test("HOLD|CHANGES-NEEDED|REQUEST_CHANGES")); + def pos: ((neg | not) and (.body | test("APPROVE"))); + [.reviews | to_entries[]] as $r + | ([$r[] | select(.value | pos) | .key] | last) as $ai + | (if $ai == null or any($r[]; .key > $ai and (.value | neg)) then "" + else ($r[] | select(.key == $ai) | .value.body | capture("(?[0-9a-f]{40})").s // "") end) as $sha + | [$p.number, $sha, $p.headRefOid, $p.id] | @tsv' } +# Arms atomically against the reviewed head: enqueuePullRequest(expectedHeadOid) is refused by GitHub if the head moved +# after the select (the --match-head-commit guard). The queue owns the merge method, so none is passed. If the enqueue is +# refused for a non-head reason (checks still pending), fall back to the plain `gh pr merge ` that #5794 mandates, +# but only after re-reading the head and finding it still equals the approved sha. run_() { - local t n + local t n sha id cur t=$(mktemp) # shellcheck disable=SC2064 trap "rm -f -- '${t:?}'" RETURN - gh pr list --state open --limit 100 --json number,headRefOid,autoMergeRequest,reviews | plan_ >"$t" - for n in $(select_ "$t"); do - gh pr merge "$n" || echo "rearm: PR $n not armed (see above)" >&2 - done + gh pr list --state open --limit 100 --json id,number,headRefOid,autoMergeRequest,reviews | plan_ >"$t" + while IFS=$'\t' read -r n sha id; do + if gh api graphql -f query='mutation($id:ID!,$h:GitObjectID!){enqueuePullRequest(input:{pullRequestId:$id,expectedHeadOid:$h}){clientMutationId}}' \ + -f id="$id" -f h="$sha" >/dev/null 2>&1; then + continue + fi + cur=$(gh pr view "$n" --json headRefOid --jq .headRefOid 2>/dev/null || true) + if [ "$cur" = "$sha" ]; then + gh pr merge "$n" || echo "rearm: PR $n not armed" >&2 + else + echo "rearm: PR $n head moved since review; not armed" >&2 + fi + done < <(select_ "$t") } self_test() { @@ -37,22 +56,24 @@ self_test() { trap "rm -rf -- '${d:?}'" RETURN a=$(printf 'a%.0s' {1..40}); b=$(printf 'b%.0s' {1..40}) printf '1\t%s\t%s\n2\t%s\t%s\n3\t\t%s\n' "$a" "$a" "$a" "$b" "$a" >"$d/t.tsv" - [ "$(select_ "$d/t.tsv" | tr '\n' ' ')" = "1 " ] || { echo "SELF-TEST FAIL: only PR 1 (head unchanged) may be re-armed" >&2; bad=1; } + [ "$(select_ "$d/t.tsv" | cut -f1 | tr '\n' ' ')" = "1 " ] || { echo "SELF-TEST FAIL: only PR 1 (head unchanged) may be re-armed" >&2; bad=1; } printf '' >"$d/e.tsv" [ -z "$(select_ "$d/e.tsv")" ] || { echo "SELF-TEST FAIL: empty input arms nothing" >&2; bad=1; } - # run_ end to end: stub gh lists PR 1 (approved, head unchanged), 2 (head moved), 3 (already armed); only 1 is armed. + # run_ end to end with a stub gh. PRs: 1 approved+unchanged; 2 head moved; 3 already armed; + # 4 APPROVE then a later CHANGES-NEEDED; 5 APPROVE then a later HOLD; 6 CHANGES-NEEDED then a fresh APPROVE. mkdir -p "$d/bin" cat >"$d/bin/gh" <>"$d/armed" fi STUB chmod +x "$d/bin/gh" PATH="$d/bin:$PATH" run_ >/dev/null 2>&1 || true - [ "$(cat "$d/armed" 2>/dev/null)" = "pr merge 1" ] || { echo "SELF-TEST FAIL: run must arm exactly PR 1 via select_, armed: $(tr '\n' ',' <"$d/armed" 2>/dev/null)" >&2; bad=1; } + got=$(grep 'expectedHeadOid:\$h' "$d/armed" 2>/dev/null | grep -o 'id=N[0-9]* -f h=[0-9a-f]*' | tr '\n' ',') + [ "$got" = "id=N1 -f h=$a,id=N6 -f h=$a," ] || { echo "SELF-TEST FAIL: run must enqueue exactly PR 1 and 6, each with expectedHeadOid=its approved sha (a later HOLD/CHANGES-NEEDED cancels), got: $got" >&2; bad=1; } [ "$bad" -eq 0 ] && echo "rearm_ready self-test: 3 cases ok" return "$bad" } From e0fde5b87396ff32e6ca56ae4cdcbb77e888876d Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Tue, 20 Oct 2026 19:17:08 +0200 Subject: [PATCH 15/22] ci: advisory gate works on cargo-deny 0.20 (global --config, --offline), pass_criteria count, roadmap aggregate (Refs #5792) CI's cargo-deny 0.20.2 rejects 'fetch --config' and 'check --disable-fetch', so the gate exited 75 on every run. Probe the flag style once; self-test covers both. Co-Authored-By: Claude Sonnet 5.5 --- contracts/advisory-gate-diff-v1.yaml | 2 +- docs/roadmaps/roadmap.yaml | 19 +++++++++++++++++++ scripts/ci/advisory_gate.sh | 16 +++++++++++++--- scripts/ci/advisory_gate_selftest.sh | 21 ++++++++++++--------- 4 files changed, 45 insertions(+), 13 deletions(-) diff --git a/contracts/advisory-gate-diff-v1.yaml b/contracts/advisory-gate-diff-v1.yaml index 9c9a9c6a61..02237036f1 100644 --- a/contracts/advisory-gate-diff-v1.yaml +++ b/contracts/advisory-gate-diff-v1.yaml @@ -134,5 +134,5 @@ qa_gate: - main_resident_advisory - one_scanner_of_record - auto_rearm - pass_criteria: "All 6 falsification tests pass; the scanner script has a must-fail/must-pass case table" + pass_criteria: "All 7 falsification tests pass; the scanner script has a must-fail/must-pass case table" falsification: "Revert the gate to lock-vs-DB; FALSIFY-ADG-001 must turn RED" diff --git a/docs/roadmaps/roadmap.yaml b/docs/roadmaps/roadmap.yaml index 3bce667faa..1590e10d3a 100644 --- a/docs/roadmaps/roadmap.yaml +++ b/docs/roadmaps/roadmap.yaml @@ -25340,3 +25340,22 @@ Not done, pending the ruling: moving the 31 fixture files to an obviously synthe - kind:code notes: null epic: 5510 +- id: PMAT-5792 + github_issue: 5792 + item_type: task + title: 'Advisory gate: PR fails only on advisories it introduces; main-resident advisory = andon, not a block (0h stop)' + status: planned + priority: critical + assigned_to: null + created: 2026-10-20T12:27:03Z + updated: 2026-10-20T12:27:03Z + spec: null + acceptance_criteria: + - 'Contract contracts/advisory-gate-diff-v1.yaml. Refs #5791. Gate-accept: solo + non-author sign-off + Noah''s yes; workflows edits go to quorum first.' + phases: [] + subtasks: [] + estimated_effort: null + labels: + - kind:code + notes: null + epic: 3863 diff --git a/scripts/ci/advisory_gate.sh b/scripts/ci/advisory_gate.sh index 3c1687ebd0..6cdc552801 100755 --- a/scripts/ci/advisory_gate.sh +++ b/scripts/ci/advisory_gate.sh @@ -32,8 +32,13 @@ W="${RUNNER_TEMP:-${TMPDIR:-/tmp}}/advgate-base-$$" trap 'rm -f "${W:?}.pr.ndjson" "${W:?}.base.ndjson"; git worktree remove --force "${W:?}" >/dev/null 2>&1 || true' EXIT git worktree add --detach "$W" "$BASE" >/dev/null 2>&1 || { echo "COULD-NOT-CHECK: no worktree for base $BASE (shallow fetch?)" >&2; exit 75; } +# cargo-deny <= 0.19 takes --config on `fetch`/`check` and --disable-fetch on `check`; 0.20 made --config global +# (before the subcommand) and replaced --disable-fetch with the global --offline. Probe once, never assume the runner's version. +if cargo deny check --help 2>&1 | grep -q -- '--config'; then DENY_OLD=1; else DENY_OLD=0; fi + # One DB for both scans: fetched with the BASE config (the db-path both scans read), HEAD logged. -cargo deny fetch --config "$W/deny.toml" db || { echo "COULD-NOT-CHECK: advisory-db fetch failed" >&2; exit 75; } +if [ "$DENY_OLD" = 1 ]; then cargo deny fetch --config "$W/deny.toml" db; else cargo deny --config "$W/deny.toml" fetch db; fi \ + || { echo "COULD-NOT-CHECK: advisory-db fetch failed" >&2; exit 75; } for d in "$HOME"/.cargo/advisory-dbs/*; do [ -d "$d/.git" ] && echo "advisory-db $(basename "$d") HEAD $(git -C "$d" rev-parse --short HEAD 2>/dev/null) $(git -C "$d" log -1 --format=%cI 2>/dev/null)" done @@ -46,8 +51,13 @@ bash "$GATE" ignores-added "$W/deny.toml" deny.toml || exit 1 scan() { # TREE OUT.ndjson -> prints the cargo-deny rc local rc=0 - (cd "$1" && cargo deny --locked --all-features --format json check --config "$W/deny.toml" --disable-fetch advisories) \ - >/dev/null 2>"$2" || rc=$? + if [ "$DENY_OLD" = 1 ]; then + (cd "$1" && cargo deny --locked --all-features --format json check --config "$W/deny.toml" --disable-fetch advisories) \ + >/dev/null 2>"$2" || rc=$? + else + (cd "$1" && cargo deny --config "$W/deny.toml" --locked --offline --all-features --format json check advisories) \ + >/dev/null 2>"$2" || rc=$? + fi echo "$rc" } PRRC=$(scan "$PWD" "$W.pr.ndjson") diff --git a/scripts/ci/advisory_gate_selftest.sh b/scripts/ci/advisory_gate_selftest.sh index 20b1a67997..2701ef7d1d 100644 --- a/scripts/ci/advisory_gate_selftest.sh +++ b/scripts/ci/advisory_gate_selftest.sh @@ -16,7 +16,8 @@ run_gate() { # prints the stub's argv log trap "rm -rf -- '${d:?}'" RETURN repo="$d/repo"; log="$d/cargo.log"; mkdir -p "$repo" "$d/bin" "$d/home" "$d/scripts" cp "$GATE_SRC" "$d/scripts/advisory_gate.sh"; cp "$HERE/advisory_diff.sh" "$d/scripts/advisory_diff.sh" - printf '#!/usr/bin/env bash\necho "$*" >>"%s"\nexit 0\n' "$log" >"$d/bin/cargo"; chmod +x "$d/bin/cargo" + # `cargo deny check --help` is the version probe: it is answered, never logged (STUB_OLD=1 plays cargo-deny <= 0.19) + printf '#!/usr/bin/env bash\ncase "$*" in *--help*) [ "${STUB_OLD:-0}" = 1 ] && echo -- --config; exit 0;; esac\necho "$*" >>"%s"\nexit 0\n' "$log" >"$d/bin/cargo"; chmod +x "$d/bin/cargo" ( cd "$repo" git init -q . && printf '[advisories]\n' >deny.toml && git add deny.toml @@ -33,15 +34,15 @@ verdict_of() { # stdin: stub log -> "ok" or the first defect local log fetch checks nall nfd ncfg log=$(cat) [ "$log" != gate-did-not-pass ] || { echo "gate did not pass against the stub"; return; } - fetch=$(grep -c '^deny fetch' <<<"$log" || true) - checks=$(grep -c 'check' <<<"$(grep -v '^deny fetch' <<<"$log")" || true) - nall=$(grep -c -- '--all-features' <<<"$(grep -v '^deny fetch' <<<"$log")" || true) - nfd=$(grep -c -- '--disable-fetch' <<<"$(grep -v '^deny fetch' <<<"$log")" || true) - ncfg=$(grep -v '^deny fetch' <<<"$log" | grep -o -- '--config [^ ]*' | sort -u | wc -l) + fetch=$(grep -cE '^deny (--config [^ ]+ )?fetch' <<<"$log" || true) + checks=$(grep -c 'check' <<<"$(grep -vE '^deny (--config [^ ]+ )?fetch' <<<"$log")" || true) + nall=$(grep -c -- '--all-features' <<<"$(grep -vE '^deny (--config [^ ]+ )?fetch' <<<"$log")" || true) + nfd=$(grep -cE -- '--disable-fetch|--offline' <<<"$(grep -vE '^deny (--config [^ ]+ )?fetch' <<<"$log")" || true) + ncfg=$(grep -vE '^deny (--config [^ ]+ )?fetch' <<<"$log" | grep -o -- '--config [^ ]*' | sort -u | wc -l) [ "$checks" -eq 2 ] || { echo "expected 2 scans, saw $checks"; return; } [ "$nall" -eq 2 ] || { echo "ADG-003: $nall of 2 scans carry --all-features"; return; } [ "$fetch" -eq 1 ] || { echo "ADG-004: $fetch advisory-db fetches, want 1"; return; } - [ "$nfd" -eq 2 ] || { echo "ADG-004: $nfd of 2 scans carry --disable-fetch"; return; } + [ "$nfd" -eq 2 ] || { echo "ADG-004: $nfd of 2 scans carry --disable-fetch/--offline"; return; } [ "$ncfg" -eq 1 ] || { echo "ADG-004: scans use $ncfg different --config files"; return; } echo ok } @@ -53,6 +54,8 @@ main() { trap "rm -rf -- '${d:?}'" EXIT got=$(run_gate | verdict_of) [ "$got" = ok ] || { echo "SELF-TEST FAIL: real gate: $got" >&2; bad=1; } + got=$(STUB_OLD=1 run_gate | verdict_of) + [ "$got" = ok ] || { echo "SELF-TEST FAIL: real gate vs cargo-deny <= 0.19 flags: $got" >&2; bad=1; } mutant() { # name sed-expr want-substring sed -e "$2" "$GATE_SRC" >"$d/m.sh" cmp -s "$d/m.sh" "$GATE_SRC" && { echo "SELF-TEST FAIL: mutant '$1' changed nothing" >&2; bad=1; return; } @@ -60,7 +63,7 @@ main() { [[ "$m" == *"$3"* ]] || { echo "SELF-TEST FAIL: mutant '$1' stayed green: $m" >&2; bad=1; } } mutant 'scan without --all-features' 's/ --all-features//' 'ADG-003' - mutant 'scan reads the net (no --disable-fetch)' 's/ --disable-fetch//' 'ADG-004' + mutant 'scan reads the net (no --disable-fetch)' 's/ --disable-fetch//;s/ --offline//' 'ADG-004' mutant 'second fetch before the base scan' 's|^BASERC=|cargo deny fetch --config "$W/deny.toml" db; BASERC=|' 'ADG-004' # ADG-003 on the other two scan sites: the push path and the exemption-liveness guard. local f @@ -69,7 +72,7 @@ main() { echo "SELF-TEST FAIL: ADG-003: $f scans advisories without --all-features" >&2; bad=1 fi done - [ "$bad" -eq 0 ] && echo "advisory_gate self-test: 1 real + 3 mutants + 2 sites ok" + [ "$bad" -eq 0 ] && echo "advisory_gate self-test: 2 real (cargo-deny 0.19 and 0.20 flags) + 3 mutants + 2 sites ok" return "$bad" } From 1fc66fca2e392cf9c8c43677506a41f8866d927e Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Tue, 20 Oct 2026 19:21:38 +0200 Subject: [PATCH 16/22] ci: rearm anchors APPROVE, reads commit.oid, allowlists reviewers, --match-head-commit fallback; empty graphs[] fails closed (Refs #5792) Co-Authored-By: Claude Sonnet 5.5 --- .github/workflows/rearm-ready.yml | 1 + contracts/advisory-gate-diff-v1.yaml | 4 +- scripts/ci/advisory_diff.sh | 7 +++- scripts/ci/rearm_ready.sh | 59 ++++++++++++++++++++-------- 4 files changed, 51 insertions(+), 20 deletions(-) diff --git a/.github/workflows/rearm-ready.yml b/.github/workflows/rearm-ready.yml index cc97410173..3b4fd19469 100644 --- a/.github/workflows/rearm-ready.yml +++ b/.github/workflows/rearm-ready.yml @@ -22,6 +22,7 @@ jobs: - name: Self-test, then re-arm approved un-armed PRs env: GH_TOKEN: ${{ github.token }} + REARM_REVIEWERS: ${{ vars.REARM_REVIEWERS }} run: | set -euo pipefail bash scripts/ci/rearm_ready.sh --self-test diff --git a/contracts/advisory-gate-diff-v1.yaml b/contracts/advisory-gate-diff-v1.yaml index 02237036f1..2a2396b777 100644 --- a/contracts/advisory-gate-diff-v1.yaml +++ b/contracts/advisory-gate-diff-v1.yaml @@ -93,7 +93,7 @@ falsification_tests: prediction: "with lock_main = 48.0.7 and lock_PR = 48.0.6, the verdict is fail naming 0378 and 0379" test: "same case table: lock_PR adds the vulnerable version" if_fails: "the gate cannot turn RED on a real regression (a gate that cannot fail)" - mutation: "in scripts/ci/advisory_diff.sh verdict() subtract the PR keys from themselves instead of main keys (comm -23 PR PR); self-test cases PR adds a new id, PR adds a crate under an id main has, and PR adds a yanked crate must turn RED (measured 2026-10-20)" + mutation: "in scripts/ci/advisory_diff.sh verdict() subtract the PR keys from themselves instead of main keys (comm -23 PR PR); self-test cases (incl. an error finding with graphs:[] must still block; a mutant yielding no key for it turns RED) PR adds a new id, PR adds a crate under an id main has, and PR adds a yanked crate must turn RED (measured 2026-10-20)" - id: FALSIFY-ADG-003 rule: "Scanner of record sees the optional-feature crate" prediction: "on the 48.0.6 lock the scanner of record exits non-zero and names 0378 and 0379" @@ -115,7 +115,7 @@ falsification_tests: - id: FALSIFY-ADG-006 rule: "Re-arm never arms a changed head (ADG-INV-005)" prediction: "a ready PR whose head sha moved after approval is not re-armed; one whose head is unchanged is" - test: "scripts/ci/rearm_ready.sh --self-test, 3 cases (head unchanged arms, moved or unapproved does not, empty arms nothing, and `run` against a stub gh enqueues exactly the PRs whose LATEST review is an APPROVE, each with expectedHeadOid = the approved sha; a later HOLD/CHANGES-NEEDED cancels an earlier APPROVE, and a mutant without expectedHeadOid or without the cancel turns it RED); .github/workflows/rearm-ready.yml calls `rearm_ready.sh run`" + test: "scripts/ci/rearm_ready.sh --self-test, 4 cases (head unchanged arms, moved or unapproved does not, empty arms nothing, and `run` against a stub gh enqueues exactly the PRs whose LATEST review is an anchored APPROVE (not `NOT APPROVED`) by an allowlisted reviewer other than the PR author, each with expectedHeadOid = the review's commit.oid; any later non-approval review cancels it; the fallback gh pr merge carries --match-head-commit ; a mutant without expectedHeadOid, the anchor, the allowlist, the author exclusion, the cancel, commit.oid or --match-head-commit turns it RED); .github/workflows/rearm-ready.yml calls `rearm_ready.sh run`" if_fails: "a gate fix arms an unreviewed head into the queue" mutation: "drop the approved_sha == current_sha test in select_; the moved-head case must turn RED; replace select_ in run_ with cat|cut and the stub-gh case must turn RED (measured 2026-10-20)" - id: FALSIFY-ADG-007 diff --git a/scripts/ci/advisory_diff.sh b/scripts/ci/advisory_diff.sh index aca21e54dd..445b2a3399 100755 --- a/scripts/ci/advisory_diff.sh +++ b/scripts/ci/advisory_diff.sh @@ -17,7 +17,7 @@ CANT=75 keys() { # FILE -> keys on stdout (use side/checked-keys to fail closed on unparsable input) jq -R -r 'fromjson? | select(.type == "diagnostic" or has("fields")) | .fields | select(.severity == "error") - | (.code // "?") as $c | (.advisory.id // "-") as $i | ((.graphs // [{}])[] | .Krate? // {}) | "\($c)|\($i)|\(.name // "?")@\(.version // "?")"' "$1" | LC_ALL=C sort -u + | (.code // "?") as $c | (.advisory.id // "-") as $i | (((.graphs // []) | if length == 0 then [{}] else . end)[] | .Krate? // {}) | "\($c)|\($i)|\(.name // "?")@\(.version // "?")"' "$1" | LC_ALL=C sort -u } parses() { # every non-blank line is JSON @@ -97,13 +97,16 @@ self_test() { t 'tool error rc 2: cannot check' 75 empty.json 2 empty.json 0 t 'garbage JSON: cannot check' 75 bad.json 1 empty.json 0 t 'main side missing: cannot check' 75 a.json 1 nope.json 1 + printf '{"type":"diagnostic","fields":{"code":"vulnerability","severity":"error","advisory":{"id":"RUSTSEC-2026-0999"},"graphs":[]}}\n' >"$d/nog.json" + t 'error finding with empty graphs[] is still a finding: block' 1 nog.json 1 empty.json 0 + t 'same empty-graphs finding on both sides: pass' 0 nog.json 1 nog.json 1 sha=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa r() { local w=$1 rc=0; shift; base_ref "$@" >/dev/null 2>&1 || rc=$?; [ "$rc" -eq "$w" ] || { echo "SELF-TEST FAIL: base_ref $* wanted $w got $rc" >&2; bad=1; }; } r 0 merge_group "$sha" ""; r 75 merge_group "" "$sha"; r 75 merge_group "main" ""; r 0 pull_request "" "$sha"; r 75 push "$sha" "$sha" printf "ignore = [\"RUSTSEC-2026-0001\"]\n" >"$d/b.toml"; cp "$d/b.toml" "$d/same.toml"; printf "ignore = [\"RUSTSEC-2026-0001\", \"RUSTSEC-2026-0378\"]\n" >"$d/wide.toml" ignores_added "$d/b.toml" "$d/same.toml" 2>/dev/null || { echo "SELF-TEST FAIL: same ignores must pass" >&2; bad=1; } ! ignores_added "$d/b.toml" "$d/wide.toml" 2>/dev/null || { echo "SELF-TEST FAIL: added ignore must block" >&2; bad=1; } - [ "$bad" -eq 0 ] && echo "advisory_diff self-test: (12 verdict + 5 base_ref + 2 ignores) ok" + [ "$bad" -eq 0 ] && echo "advisory_diff self-test: (14 verdict + 5 base_ref + 2 ignores) ok" return "$bad" } diff --git a/scripts/ci/rearm_ready.sh b/scripts/ci/rearm_ready.sh index 3f7819a8d5..41d25372cb 100755 --- a/scripts/ci/rearm_ready.sh +++ b/scripts/ci/rearm_ready.sh @@ -11,17 +11,19 @@ select_() { awk -F'\t' 'NF >= 3 && $1 != "" && $2 ~ /^[0-9a-f]{40}$/ && $2 == $3 { print $1 "\t" $3 "\t" $4 }' "$1" } -# stdin: `gh pr list --json id,number,headRefOid,autoMergeRequest,reviews`; stdout: TSV "pr approved_sha current_sha node_id" -# approved_sha = the full sha named in the latest APPROVE review, unless a LATER review is negative -# (HOLD, CHANGES-NEEDED, REQUEST_CHANGES, state CHANGES_REQUESTED): a later negative review cancels the approval. +# stdin: `gh pr list --json id,number,author,headRefOid,autoMergeRequest,reviews`; stdout: TSV "pr approved_sha current_sha node_id" +# approved_sha = commit.oid of the latest anchored APPROVE review by an allowlisted reviewer (REARM_REVIEWERS, never the PR author); +# ANY later review that is not such an approval (negative or a plain comment) cancels it. plan_() { - jq -r '.[] | select(.autoMergeRequest == null) | . as $p + jq -r --arg ok "${REARM_REVIEWERS:-}" '($ok | split(" ") | map(select(. != ""))) as $allow + | .[] | select(.autoMergeRequest == null) | . as $p | def neg: (.state == "CHANGES_REQUESTED") or (.body | test("HOLD|CHANGES-NEEDED|REQUEST_CHANGES")); - def pos: ((neg | not) and (.body | test("APPROVE"))); + def tok: (.body | test("(^|\\n|: |- )APPROVE([^A-Za-z]|$)")); + def pos: (neg | not) and tok and (.author.login as $l | ($allow | index($l)) != null and $l != $p.author.login); [.reviews | to_entries[]] as $r | ([$r[] | select(.value | pos) | .key] | last) as $ai - | (if $ai == null or any($r[]; .key > $ai and (.value | neg)) then "" - else ($r[] | select(.key == $ai) | .value.body | capture("(?[0-9a-f]{40})").s // "") end) as $sha + | (if $ai == null or any($r[]; .key > $ai and (.value | pos | not)) then "" + else ($r[] | select(.key == $ai) | .value.commit.oid // "") end) as $sha | [$p.number, $sha, $p.headRefOid, $p.id] | @tsv' } @@ -34,7 +36,7 @@ run_() { t=$(mktemp) # shellcheck disable=SC2064 trap "rm -f -- '${t:?}'" RETURN - gh pr list --state open --limit 100 --json id,number,headRefOid,autoMergeRequest,reviews | plan_ >"$t" + gh pr list --state open --limit 100 --json id,number,author,headRefOid,autoMergeRequest,reviews | plan_ >"$t" while IFS=$'\t' read -r n sha id; do if gh api graphql -f query='mutation($id:ID!,$h:GitObjectID!){enqueuePullRequest(input:{pullRequestId:$id,expectedHeadOid:$h}){clientMutationId}}' \ -f id="$id" -f h="$sha" >/dev/null 2>&1; then @@ -42,7 +44,7 @@ run_() { fi cur=$(gh pr view "$n" --json headRefOid --jq .headRefOid 2>/dev/null || true) if [ "$cur" = "$sha" ]; then - gh pr merge "$n" || echo "rearm: PR $n not armed" >&2 + gh pr merge "$n" --match-head-commit "$sha" || echo "rearm: PR $n not armed" >&2 else echo "rearm: PR $n head moved since review; not armed" >&2 fi @@ -62,19 +64,44 @@ self_test() { # run_ end to end with a stub gh. PRs: 1 approved+unchanged; 2 head moved; 3 already armed; # 4 APPROVE then a later CHANGES-NEEDED; 5 APPROVE then a later HOLD; 6 CHANGES-NEEDED then a fresh APPROVE. mkdir -p "$d/bin" + # rv LOGIN BODY OID -> one review object; pr N HEAD AUTOMERGE REVIEWS -> one PR object + rv() { printf '{"author":{"login":"%s"},"state":"COMMENTED","body":"%s","commit":{"oid":"%s"}}' "$1" "$2" "$3"; } + pr() { printf '{"id":"N%s","number":%s,"author":{"login":"au"},"headRefOid":"%s","autoMergeRequest":%s,"reviews":[%s]}' "$1" "$1" "$2" "$3" "$4"; } + { + printf '[' + pr 1 "$a" null "$(rv s1 'seat 1: APPROVE' "$a")"; printf , + pr 2 "$b" null "$(rv s1 'seat 1: APPROVE' "$a")"; printf , + pr 3 "$a" '{"x":1}' "$(rv s1 'seat 1: APPROVE' "$a")"; printf , + pr 4 "$a" null "$(rv s1 'seat 1: APPROVE' "$a"),$(rv s2 'seat 2: CHANGES-NEEDED' "$a")"; printf , + pr 5 "$a" null "$(rv s1 'seat 1: APPROVE' "$a"),$(rv s2 'HOLD' "$a")"; printf , + pr 6 "$a" null "$(rv s2 'CHANGES-NEEDED' "$a"),$(rv s1 'seat 1: APPROVE' "$a")"; printf , + pr 7 "$a" null "$(rv s1 'seat 1: NOT APPROVED' "$a")"; printf , + pr 8 "$a" null "$(rv s1 'seat 1: APPROVE' "$a"),$(rv s2 'just a comment' "$a")"; printf , + pr 9 "$a" null "$(rv rando 'seat 1: APPROVE' "$a")"; printf , + pr 10 "$a" null "$(rv au 'seat 1: APPROVE' "$a")"; printf , + pr 11 "$a" null "$(rv s1 'seat 1: APPROVE' "$b")" + printf ']' + } >"$d/prs.json" cat >"$d/bin/gh" <>"$d/armed" +if [ "\$1 \$2" = "pr list" ]; then cat "$d/prs.json" +elif [ "\$1 \$2" = "pr view" ]; then echo "$a" +elif [ "\$1" = api ]; then echo "\$*" >>"$d/armed"; [ -z "\${STUB_REFUSE:-}" ] +elif [ "\$1 \$2" = "pr merge" ]; then echo "merge \$*" >>"$d/merged" fi STUB chmod +x "$d/bin/gh" - PATH="$d/bin:$PATH" run_ >/dev/null 2>&1 || true + REARM_REVIEWERS="s1 s2 au" PATH="$d/bin:$PATH" run_ >/dev/null 2>&1 || true got=$(grep 'expectedHeadOid:\$h' "$d/armed" 2>/dev/null | grep -o 'id=N[0-9]* -f h=[0-9a-f]*' | tr '\n' ',') - [ "$got" = "id=N1 -f h=$a,id=N6 -f h=$a," ] || { echo "SELF-TEST FAIL: run must enqueue exactly PR 1 and 6, each with expectedHeadOid=its approved sha (a later HOLD/CHANGES-NEEDED cancels), got: $got" >&2; bad=1; } - [ "$bad" -eq 0 ] && echo "rearm_ready self-test: 3 cases ok" + # 1 ok; 2 head moved; 3 armed; 4/5 later negative; 6 fresh approve after negative -> arms; 7 "NOT APPROVED" is no approval; + # 8 any later non-approval cancels; 9 reviewer off the allowlist; 10 author self-approval; 11 commit.oid (b) != head (a), so not armed. + [ "$got" = "id=N1 -f h=$a,id=N6 -f h=$a," ] || { echo "SELF-TEST FAIL: run must enqueue exactly PR 1 and 6 with expectedHeadOid=commit.oid, got: $got" >&2; bad=1; } + rm -f "${d:?}/armed" "${d:?}/merged" + REARM_REVIEWERS="s1 s2 au" STUB_REFUSE=1 PATH="$d/bin:$PATH" run_ >/dev/null 2>&1 || true + if [ "$(grep -c -- "--match-head-commit $a" "$d/merged" 2>/dev/null)" -ne 2 ] || [ "$(wc -l <"$d/merged")" -ne 2 ]; then + echo "SELF-TEST FAIL: the fallback gh pr merge must pass --match-head-commit (PR 1 and 6 only)" >&2; bad=1 + fi + [ "$bad" -eq 0 ] && echo "rearm_ready self-test: 4 cases ok" return "$bad" } From 12d16b87ea24d9255b3ac9cb5600a1ec5d955829 Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Tue, 20 Oct 2026 20:01:59 +0200 Subject: [PATCH 17/22] ci: advisory step is one command per line under -e and a self-test proves it goes RED on the gate's status (Refs #5792) Co-Authored-By: Claude Sonnet 5.5 --- .github/workflows/ci.yml | 14 +++-- ci/sections.yml | 14 +++-- contracts/advisory-gate-diff-v1.yaml | 9 +++- scripts/ci/advisory_step_selftest.sh | 78 ++++++++++++++++++++++++++++ 4 files changed, 106 insertions(+), 9 deletions(-) create mode 100755 scripts/ci/advisory_step_selftest.sh diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 0ff363d6df..a1b7eac977 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -355,10 +355,16 @@ jobs: else G=scripts/ci fi - rc=0 - bash "$G/advisory_diff.sh" --self-test && bash scripts/ci/check_sov_advisories_off.sh --self-test && bash scripts/ci/check_sov_advisories_off.sh && bash scripts/ci/advisory_gate_selftest.sh "$G/advisory_gate.sh" && bash scripts/ci/rearm_ready.sh --self-test && bash "$G/advisory_gate.sh" || rc=$? - rm -rf -- "${S:?}" - exit "$rc" + # One command per line under -e: the first non-zero exit ends the step with that status (no && chain, no saved rc). + trap 'rm -rf -- "${S:?}"' EXIT + set -e + bash "$G/advisory_diff.sh" --self-test + bash scripts/ci/check_sov_advisories_off.sh --self-test + bash scripts/ci/check_sov_advisories_off.sh + bash scripts/ci/advisory_gate_selftest.sh "$G/advisory_gate.sh" + bash scripts/ci/advisory_step_selftest.sh + bash scripts/ci/rearm_ready.sh --self-test + bash "$G/advisory_gate.sh" - name: Every deny.toml exemption must still be live env: GITHUB_TOKEN: ${{ github.token }} diff --git a/ci/sections.yml b/ci/sections.yml index 651cfe05e0..db6cd3bc89 100644 --- a/ci/sections.yml +++ b/ci/sections.yml @@ -2690,10 +2690,16 @@ jobs: else G=scripts/ci fi - rc=0 - bash "$G/advisory_diff.sh" --self-test && bash scripts/ci/check_sov_advisories_off.sh --self-test && bash scripts/ci/check_sov_advisories_off.sh && bash scripts/ci/advisory_gate_selftest.sh "$G/advisory_gate.sh" && bash scripts/ci/rearm_ready.sh --self-test && bash "$G/advisory_gate.sh" || rc=$? - rm -rf -- "${S:?}" - exit "$rc" + # One command per line under -e: the first non-zero exit ends the step with that status (no && chain, no saved rc). + trap 'rm -rf -- "${S:?}"' EXIT + set -e + bash "$G/advisory_diff.sh" --self-test + bash scripts/ci/check_sov_advisories_off.sh --self-test + bash scripts/ci/check_sov_advisories_off.sh + bash scripts/ci/advisory_gate_selftest.sh "$G/advisory_gate.sh" + bash scripts/ci/advisory_step_selftest.sh + bash scripts/ci/rearm_ready.sh --self-test + bash "$G/advisory_gate.sh" # Separate from the check above on purpose: a newly-FIXED upstream must # never fail anyone's build. It fails only this guard, whose remedy is # deleting a line. diff --git a/contracts/advisory-gate-diff-v1.yaml b/contracts/advisory-gate-diff-v1.yaml index 2a2396b777..92ca7f7c11 100644 --- a/contracts/advisory-gate-diff-v1.yaml +++ b/contracts/advisory-gate-diff-v1.yaml @@ -125,6 +125,13 @@ falsification_tests: if_fails: "the vendored scans still stop every PR on an advisory already on main (the #5793 REJECT)" mutation: "set advisories_diff_only: false in ci/sections.yml, or change the vendored '= true' test; the matching case must turn RED (measured 2026-10-20)" + - id: FALSIFY-ADG-008 + rule: "The CI step fails closed on the gate's exit status" + prediction: "the 'Advisories must pass' step of ci.yml exits with the gate's (or any earlier self-test's) non-zero status; a step without the gate line does not" + test: "scripts/ci/advisory_step_selftest.sh, 4 cases + 1 mutant: the step's run block is extracted from ci.yml and run against stub scripts on a throwaway origin/main; gate rc 75 gives 75, gate rc 1 gives 1, a failing self-test gives its rc, all green gives 0" + if_fails: "a red or could-not-check gate leaves the step green and a PR merges on an unjudged advisory" + mutation: "delete the advisory_gate.sh invocation from the step; the mutant case must stay green on a failing gate, which the cases above forbid (measured 2026-10-20)" + qa_gate: id: F-ADG-001 name: Advisory gate diff contract @@ -134,5 +141,5 @@ qa_gate: - main_resident_advisory - one_scanner_of_record - auto_rearm - pass_criteria: "All 7 falsification tests pass; the scanner script has a must-fail/must-pass case table" + pass_criteria: "All 8 falsification tests pass; the scanner script has a must-fail/must-pass case table" falsification: "Revert the gate to lock-vs-DB; FALSIFY-ADG-001 must turn RED" diff --git a/scripts/ci/advisory_step_selftest.sh b/scripts/ci/advisory_step_selftest.sh new file mode 100755 index 0000000000..5dd10521cc --- /dev/null +++ b/scripts/ci/advisory_step_selftest.sh @@ -0,0 +1,78 @@ +#!/usr/bin/env bash +# advisory_step_selftest.sh -- the CI step "Advisories must pass" must go RED when the gate (or any +# self-test before it) exits non-zero (PMAT-5792, ADG-INV-004). Extracts the step's run block from +# .github/workflows/ci.yml, runs it in a throwaway repo whose origin/main holds stub scripts, and +# requires the step's exit status to equal the stub's. A mutant (gate line removed) must NOT be red. +# +# advisory_step_selftest.sh [WORKFLOW] WORKFLOW defaults to .github/workflows/ci.yml +set -euo pipefail + +wf=${1:-.github/workflows/ci.yml} +d=$(mktemp -d) +# shellcheck disable=SC2064 +trap "rm -rf -- '${d:?}'" EXIT + +extract() { # WORKFLOW -> the run block, de-indented + awk '/^ - name: Advisories must pass/ {on=1; next} + on && /^ - name:/ {exit} + on && /^ run: \|/ {run=1; next} + on && run {sub(/^ /, ""); print}' "$1" +} + +# stubs: each script exits with the status in .rc (default 0) +make_repo() { # DIR -> repo with scripts/ci stubs committed on refs/remotes/origin/main + local r=$1 f + mkdir -p "$r/scripts/ci" + for f in advisory_diff advisory_gate advisory_gate_selftest advisory_step_selftest check_sov_advisories_off rearm_ready; do + printf '#!/usr/bin/env bash\nexit "$(cat "$(dirname "$0")/%s.rc" 2>/dev/null || echo 0)"\n' "$f" >"$r/scripts/ci/$f.sh" + done + git -C "$r" init -q + git -C "$r" add -A + git -C "$r" -c user.name=t -c user.email=t@t write-tree >/dev/null + local tree c + tree=$(git -C "$r" write-tree) + c=$(git -C "$r" -c user.name=t -c user.email=t@t commit-tree "$tree" -m stub) + git -C "$r" update-ref refs/remotes/origin/main "$c" +} + +run_step() { # SCRIPTFILE REPO -> step exit status + local rc=0 + (cd "$2" && env -u RUNNER_TEMP TMPDIR="$d" GITHUB_TOKEN=x bash -c "$(cat "$1")" >/dev/null 2>&1) || rc=$? + echo "$rc" +} + +# $1: gate rc, $2: first self-test rc -- set in the BASE copy (what the step actually runs) and in the tree +case_() { # name want gate_rc selftest_rc script + local name=$1 want=$2 got r + r="$d/repo-$RANDOM" + make_repo "$r" + printf '%s' "$3" >"$r/scripts/ci/advisory_gate.rc" + printf '%s' "$4" >"$r/scripts/ci/advisory_diff.rc" + git -C "$r" add -A + local tree c + tree=$(git -C "$r" write-tree) + c=$(git -C "$r" -c user.name=t -c user.email=t@t commit-tree "$tree" -m stub2) + git -C "$r" update-ref refs/remotes/origin/main "$c" + got=$(run_step "$5" "$r") + if [ "$got" -ne "$want" ]; then + echo "SELF-TEST FAIL: $name wanted step rc=$want, got $got" >&2 + bad=1 + fi +} + +bad=0 +extract "$wf" >"$d/step.sh" +[ -s "$d/step.sh" ] || { echo "SELF-TEST FAIL: could not extract the 'Advisories must pass' step from $wf" >&2; exit 1; } +case_ 'gate rc 75 -> step RED with 75' 75 75 0 "$d/step.sh" +case_ 'gate rc 1 -> step RED with 1' 1 1 0 "$d/step.sh" +case_ 'advisory_diff self-test rc 3 -> step RED with 3' 3 0 3 "$d/step.sh" +case_ 'all green -> step green' 0 0 0 "$d/step.sh" +# mutant: the gate invocation removed from the step; a failing gate must then (wrongly) pass -- proving the cases above bind to it +grep -v 'advisory_gate.sh"$' "$d/step.sh" >"$d/mut.sh" +cmp -s "$d/step.sh" "$d/mut.sh" && { echo "SELF-TEST FAIL: mutant did not change the step (gate line not found)" >&2; bad=1; } +case_ 'MUTANT (gate line removed) must NOT be red on a failing gate' 0 75 0 "$d/mut.sh" +# mutant 2: the old `|| rc=$?` shape with the gate dropped from the chain is caught the same way (covered by the case above) +if [ "$bad" -eq 0 ]; then + echo "advisory_step self-test: 4 cases + 1 mutant ok" +fi +exit "$bad" From 78695b9136f8eedf6d47247eceb7ceffd6eb3625 Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Wed, 21 Oct 2026 00:55:45 +0200 Subject: [PATCH 18/22] ci(rearm): drop the gh pr merge --match-head-commit fallback; the enqueue mutation is the head pin (Refs #5793) main's check_queue_merge_plain.sh (#5794) bans every flag on `gh pr merge` in a merge-queue repo, so the fallback in rearm_ready.sh (L47/90/102) turned the merged tree RED. Allow-listing the file would edit the scanner (gate content) and keep a check-then-act path: re-read the head, then a plain merge that pins nothing. Dropping the fallback keeps the pin safe: enqueuePullRequest(expectedHeadOid) is refused by GitHub if the head moved, and nothing else can arm a PR. A refused enqueue (checks pending) is left for the next run. The self-test now fails if a refused enqueue triggers any other gh call. Both rearm_ready.sh --self-test and check_queue_merge_plain.sh --self-test (real-tree scan) pass. Co-Authored-By: Claude Sonnet 5.5 --- scripts/ci/rearm_ready.sh | 24 +++++++++--------------- 1 file changed, 9 insertions(+), 15 deletions(-) diff --git a/scripts/ci/rearm_ready.sh b/scripts/ci/rearm_ready.sh index 41d25372cb..e886848a76 100755 --- a/scripts/ci/rearm_ready.sh +++ b/scripts/ci/rearm_ready.sh @@ -28,11 +28,11 @@ plan_() { } # Arms atomically against the reviewed head: enqueuePullRequest(expectedHeadOid) is refused by GitHub if the head moved -# after the select (the --match-head-commit guard). The queue owns the merge method, so none is passed. If the enqueue is -# refused for a non-head reason (checks still pending), fall back to the plain `gh pr merge ` that #5794 mandates, -# but only after re-reading the head and finding it still equals the approved sha. +# after the select. The queue owns the merge method, so none is passed. There is NO `gh pr merge` fallback: #5794 bans +# every flag on it (so no --match-head-commit), and a plain merge after a re-read of the head is a check-then-act race that +# could arm a head nobody reviewed. A refused enqueue (head moved, or checks still pending) is left for the next run. run_() { - local t n sha id cur + local t n sha id t=$(mktemp) # shellcheck disable=SC2064 trap "rm -f -- '${t:?}'" RETURN @@ -42,12 +42,7 @@ run_() { -f id="$id" -f h="$sha" >/dev/null 2>&1; then continue fi - cur=$(gh pr view "$n" --json headRefOid --jq .headRefOid 2>/dev/null || true) - if [ "$cur" = "$sha" ]; then - gh pr merge "$n" --match-head-commit "$sha" || echo "rearm: PR $n not armed" >&2 - else - echo "rearm: PR $n head moved since review; not armed" >&2 - fi + echo "rearm: PR $n enqueue refused at the reviewed head; not armed" >&2 done < <(select_ "$t") } @@ -85,9 +80,8 @@ self_test() { cat >"$d/bin/gh" <>"$d/armed"; [ -z "\${STUB_REFUSE:-}" ] -elif [ "\$1 \$2" = "pr merge" ]; then echo "merge \$*" >>"$d/merged" +else echo "\$*" >>"$d/other"; exit 1 fi STUB chmod +x "$d/bin/gh" @@ -96,10 +90,10 @@ STUB # 1 ok; 2 head moved; 3 armed; 4/5 later negative; 6 fresh approve after negative -> arms; 7 "NOT APPROVED" is no approval; # 8 any later non-approval cancels; 9 reviewer off the allowlist; 10 author self-approval; 11 commit.oid (b) != head (a), so not armed. [ "$got" = "id=N1 -f h=$a,id=N6 -f h=$a," ] || { echo "SELF-TEST FAIL: run must enqueue exactly PR 1 and 6 with expectedHeadOid=commit.oid, got: $got" >&2; bad=1; } - rm -f "${d:?}/armed" "${d:?}/merged" + rm -f "${d:?}/armed" "${d:?}/other" REARM_REVIEWERS="s1 s2 au" STUB_REFUSE=1 PATH="$d/bin:$PATH" run_ >/dev/null 2>&1 || true - if [ "$(grep -c -- "--match-head-commit $a" "$d/merged" 2>/dev/null)" -ne 2 ] || [ "$(wc -l <"$d/merged")" -ne 2 ]; then - echo "SELF-TEST FAIL: the fallback gh pr merge must pass --match-head-commit (PR 1 and 6 only)" >&2; bad=1 + if [ -e "$d/other" ]; then + echo "SELF-TEST FAIL: a refused enqueue must not fall back to any other gh call (the head pin is the mutation only)" >&2; bad=1 fi [ "$bad" -eq 0 ] && echo "rearm_ready self-test: 4 cases ok" return "$bad" From 0477d9f4ee2c36c031f7fe156dbdc819c662ba70 Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Wed, 21 Oct 2026 00:56:23 +0200 Subject: [PATCH 19/22] ci(advisory): step self-test makes its repo dirs with mktemp, not $RANDOM (bashrs DET001) (Refs #5793) Co-Authored-By: Claude Sonnet 5.5 --- scripts/ci/advisory_step_selftest.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/advisory_step_selftest.sh b/scripts/ci/advisory_step_selftest.sh index 5dd10521cc..7409c91d50 100755 --- a/scripts/ci/advisory_step_selftest.sh +++ b/scripts/ci/advisory_step_selftest.sh @@ -44,7 +44,7 @@ run_step() { # SCRIPTFILE REPO -> step exit status # $1: gate rc, $2: first self-test rc -- set in the BASE copy (what the step actually runs) and in the tree case_() { # name want gate_rc selftest_rc script local name=$1 want=$2 got r - r="$d/repo-$RANDOM" + r=$(mktemp -d "$d/repo-XXXXXX") make_repo "$r" printf '%s' "$3" >"$r/scripts/ci/advisory_gate.rc" printf '%s' "$4" >"$r/scripts/ci/advisory_diff.rc" From bdda40ae664d42f5b6c4a0cb9e65e17627540e05 Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Wed, 21 Oct 2026 05:17:39 +0200 Subject: [PATCH 20/22] advisory_diff: print cargo-deny stderr head on 'rc=1 but no findings parsed' (Refs #5793) m70 failed on CI with that message and could not be reproduced with the same flags on cargo-deny 0.20.2 (both sides rc=0); the next red now names its cause. Co-Authored-By: Claude Sonnet 5.5 --- scripts/ci/advisory_diff.sh | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/scripts/ci/advisory_diff.sh b/scripts/ci/advisory_diff.sh index 445b2a3399..20a7643bd6 100755 --- a/scripts/ci/advisory_diff.sh +++ b/scripts/ci/advisory_diff.sh @@ -31,7 +31,11 @@ side() { # JSON RC -> echoes keys; returns CANT when the side cannot be trusted parses "$f" || { echo "COULD-NOT-CHECK: $f is not NDJSON" >&2; return "$CANT"; } k=$(keys "$f") local e; e=$(jq -R -r 'fromjson? | .fields? | select(.severity == "error") | .code' "$f" | head -c1) - if [ "$rc" -eq 1 ] && [ -z "$e" ]; then echo "COULD-NOT-CHECK: rc=1 but no findings parsed" >&2; return "$CANT"; fi + if [ "$rc" -eq 1 ] && [ -z "$e" ]; then + echo "COULD-NOT-CHECK: rc=1 but no findings parsed; cargo-deny stderr (first 20 lines of $f):" >&2 + head -n 20 "$f" | cut -c1-300 >&2 + return "$CANT" + fi if [ "$rc" -eq 0 ] && [ -n "$e" ]; then echo "COULD-NOT-CHECK: rc=0 but error findings parsed" >&2; return "$CANT"; fi printf '%s\n' "$k" } From 0c50ee2a6eb5be4434ab59a85b0b8038e6ceca76 Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Wed, 21 Oct 2026 06:24:58 +0200 Subject: [PATCH 21/22] fix(ci): advisory gate drops --offline on cargo-deny 0.20 (m70 rc=1, cold registry cache) (Refs #5793) cargo-deny 0.20.2 --offline makes cargo metadata need every crate in the registry cache; a cold runner exits rc=1 with log lines only, no diagnostics (reproduced with an empty CARGO_HOME: 'no matching package named arrow-array'). Without --offline the same scan is rc=0. Self-test: ADG-005 forbids --offline, with a planted mutant. Co-Authored-By: Claude Sonnet 5.5 --- scripts/ci/advisory_gate.sh | 6 ++++-- scripts/ci/advisory_gate_selftest.sh | 17 ++++++++++------- 2 files changed, 14 insertions(+), 9 deletions(-) diff --git a/scripts/ci/advisory_gate.sh b/scripts/ci/advisory_gate.sh index 6cdc552801..8fcdaf4505 100755 --- a/scripts/ci/advisory_gate.sh +++ b/scripts/ci/advisory_gate.sh @@ -33,7 +33,9 @@ trap 'rm -f "${W:?}.pr.ndjson" "${W:?}.base.ndjson"; git worktree remove --force git worktree add --detach "$W" "$BASE" >/dev/null 2>&1 || { echo "COULD-NOT-CHECK: no worktree for base $BASE (shallow fetch?)" >&2; exit 75; } # cargo-deny <= 0.19 takes --config on `fetch`/`check` and --disable-fetch on `check`; 0.20 made --config global -# (before the subcommand) and replaced --disable-fetch with the global --offline. Probe once, never assume the runner's version. +# (before the subcommand); 0.20 has no --disable-fetch and `check` never fetches the db. No --offline on 0.20: its cargo metadata then +# needs every crate in the registry cache, and a cold runner (or a base tree with other deps) fails "no matching package" with rc=1 and no +# diagnostics (measured 2026-10-21, cargo-deny 0.20.2, empty CARGO_HOME). Probe once, never assume the runner's version. if cargo deny check --help 2>&1 | grep -q -- '--config'; then DENY_OLD=1; else DENY_OLD=0; fi # One DB for both scans: fetched with the BASE config (the db-path both scans read), HEAD logged. @@ -55,7 +57,7 @@ scan() { # TREE OUT.ndjson -> prints the cargo-deny rc (cd "$1" && cargo deny --locked --all-features --format json check --config "$W/deny.toml" --disable-fetch advisories) \ >/dev/null 2>"$2" || rc=$? else - (cd "$1" && cargo deny --config "$W/deny.toml" --locked --offline --all-features --format json check advisories) \ + (cd "$1" && cargo deny --config "$W/deny.toml" --locked --all-features --format json check advisories) \ >/dev/null 2>"$2" || rc=$? fi echo "$rc" diff --git a/scripts/ci/advisory_gate_selftest.sh b/scripts/ci/advisory_gate_selftest.sh index 2701ef7d1d..9b696ac410 100644 --- a/scripts/ci/advisory_gate_selftest.sh +++ b/scripts/ci/advisory_gate_selftest.sh @@ -30,19 +30,21 @@ run_gate() { # prints the stub's argv log cat "$log" } -verdict_of() { # stdin: stub log -> "ok" or the first defect +verdict_of() { # [old|new] (cargo-deny flavour the stub played); stdin: stub log -> "ok" or the first defect local log fetch checks nall nfd ncfg log=$(cat) [ "$log" != gate-did-not-pass ] || { echo "gate did not pass against the stub"; return; } fetch=$(grep -cE '^deny (--config [^ ]+ )?fetch' <<<"$log" || true) checks=$(grep -c 'check' <<<"$(grep -vE '^deny (--config [^ ]+ )?fetch' <<<"$log")" || true) nall=$(grep -c -- '--all-features' <<<"$(grep -vE '^deny (--config [^ ]+ )?fetch' <<<"$log")" || true) - nfd=$(grep -cE -- '--disable-fetch|--offline' <<<"$(grep -vE '^deny (--config [^ ]+ )?fetch' <<<"$log")" || true) + nfd=$(grep -cE -- '--disable-fetch' <<<"$(grep -vE '^deny (--config [^ ]+ )?fetch' <<<"$log")" || true) ncfg=$(grep -vE '^deny (--config [^ ]+ )?fetch' <<<"$log" | grep -o -- '--config [^ ]*' | sort -u | wc -l) [ "$checks" -eq 2 ] || { echo "expected 2 scans, saw $checks"; return; } [ "$nall" -eq 2 ] || { echo "ADG-003: $nall of 2 scans carry --all-features"; return; } [ "$fetch" -eq 1 ] || { echo "ADG-004: $fetch advisory-db fetches, want 1"; return; } - [ "$nfd" -eq 2 ] || { echo "ADG-004: $nfd of 2 scans carry --disable-fetch/--offline"; return; } + if [ "${1:-new}" = old ]; then [ "$nfd" -eq 2 ] || { echo "ADG-004: $nfd of 2 scans carry --disable-fetch"; return; }; fi + # 0.20: --offline makes cargo metadata need a warm registry cache; a cold runner then exits rc=1 with no diagnostics (m70, #5793) + [ "$(grep -c -- '--offline' <<<"$log")" -eq 0 ] || { echo "ADG-005: a scan carries --offline"; return; } [ "$ncfg" -eq 1 ] || { echo "ADG-004: scans use $ncfg different --config files"; return; } echo ok } @@ -54,16 +56,17 @@ main() { trap "rm -rf -- '${d:?}'" EXIT got=$(run_gate | verdict_of) [ "$got" = ok ] || { echo "SELF-TEST FAIL: real gate: $got" >&2; bad=1; } - got=$(STUB_OLD=1 run_gate | verdict_of) + got=$(STUB_OLD=1 run_gate | verdict_of old) [ "$got" = ok ] || { echo "SELF-TEST FAIL: real gate vs cargo-deny <= 0.19 flags: $got" >&2; bad=1; } mutant() { # name sed-expr want-substring sed -e "$2" "$GATE_SRC" >"$d/m.sh" cmp -s "$d/m.sh" "$GATE_SRC" && { echo "SELF-TEST FAIL: mutant '$1' changed nothing" >&2; bad=1; return; } - m=$(run_gate_with "$d/m.sh" | verdict_of) + m=$(STUB_OLD="${5:-0}" run_gate_with "$d/m.sh" | verdict_of "${4:-new}") [[ "$m" == *"$3"* ]] || { echo "SELF-TEST FAIL: mutant '$1' stayed green: $m" >&2; bad=1; } } mutant 'scan without --all-features' 's/ --all-features//' 'ADG-003' - mutant 'scan reads the net (no --disable-fetch)' 's/ --disable-fetch//;s/ --offline//' 'ADG-004' + mutant 'scan reads the net (no --disable-fetch)' 's/ --disable-fetch//' 'ADG-004' old 1 + mutant 'scan runs --offline (0.20)' 's/--locked --all-features/--locked --offline --all-features/' 'ADG-005' mutant 'second fetch before the base scan' 's|^BASERC=|cargo deny fetch --config "$W/deny.toml" db; BASERC=|' 'ADG-004' # ADG-003 on the other two scan sites: the push path and the exemption-liveness guard. local f @@ -72,7 +75,7 @@ main() { echo "SELF-TEST FAIL: ADG-003: $f scans advisories without --all-features" >&2; bad=1 fi done - [ "$bad" -eq 0 ] && echo "advisory_gate self-test: 2 real (cargo-deny 0.19 and 0.20 flags) + 3 mutants + 2 sites ok" + [ "$bad" -eq 0 ] && echo "advisory_gate self-test: 2 real (cargo-deny 0.19 and 0.20 flags) + 4 mutants + 2 sites ok" return "$bad" } From 10c946a4d7f2287825a2306e33270f52597d194e Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Wed, 21 Oct 2026 06:30:04 +0200 Subject: [PATCH 22/22] fix(ci): advisory gate keeps --offline, warms the cargo cache with 'cargo fetch --locked' on both trees first (Refs #5793) Dropping --offline let 0.20 check re-fetch the advisory db, so the PR and base scans could read different db heads. Both scans stay --offline over the one db; cargo fetch --locked runs on both trees before either scan (measured: cold CARGO_HOME, both scans rc=0). ADG-005 asserts the flag and the order; two planted mutants. Comment corrected. Co-Authored-By: Claude Sonnet 5.5 --- scripts/ci/advisory_gate.sh | 12 +++++++----- scripts/ci/advisory_gate_selftest.sh | 14 ++++++++++---- 2 files changed, 17 insertions(+), 9 deletions(-) diff --git a/scripts/ci/advisory_gate.sh b/scripts/ci/advisory_gate.sh index 8fcdaf4505..ed1a127ed9 100755 --- a/scripts/ci/advisory_gate.sh +++ b/scripts/ci/advisory_gate.sh @@ -32,10 +32,10 @@ W="${RUNNER_TEMP:-${TMPDIR:-/tmp}}/advgate-base-$$" trap 'rm -f "${W:?}.pr.ndjson" "${W:?}.base.ndjson"; git worktree remove --force "${W:?}" >/dev/null 2>&1 || true' EXIT git worktree add --detach "$W" "$BASE" >/dev/null 2>&1 || { echo "COULD-NOT-CHECK: no worktree for base $BASE (shallow fetch?)" >&2; exit 75; } -# cargo-deny <= 0.19 takes --config on `fetch`/`check` and --disable-fetch on `check`; 0.20 made --config global -# (before the subcommand); 0.20 has no --disable-fetch and `check` never fetches the db. No --offline on 0.20: its cargo metadata then -# needs every crate in the registry cache, and a cold runner (or a base tree with other deps) fails "no matching package" with rc=1 and no -# diagnostics (measured 2026-10-21, cargo-deny 0.20.2, empty CARGO_HOME). Probe once, never assume the runner's version. +# (before the subcommand) and replaced --disable-fetch with the global --offline, which `check` needs so both scans read the ONE advisory-db +# fetched above (without it 0.20 `check` may re-fetch the db and the two scans see different heads). --offline also makes cargo metadata need +# every crate in the registry cache (a cold runner exits rc=1 with log lines only, m70 #5793, measured 2026-10-21 on 0.20.2 with an empty +# CARGO_HOME), so `cargo fetch --locked` warms the cache for BOTH trees before either scan. Probe once, never assume the runner's version. if cargo deny check --help 2>&1 | grep -q -- '--config'; then DENY_OLD=1; else DENY_OLD=0; fi # One DB for both scans: fetched with the BASE config (the db-path both scans read), HEAD logged. @@ -57,11 +57,13 @@ scan() { # TREE OUT.ndjson -> prints the cargo-deny rc (cd "$1" && cargo deny --locked --all-features --format json check --config "$W/deny.toml" --disable-fetch advisories) \ >/dev/null 2>"$2" || rc=$? else - (cd "$1" && cargo deny --config "$W/deny.toml" --locked --all-features --format json check advisories) \ + (cd "$1" && cargo deny --config "$W/deny.toml" --locked --offline --all-features --format json check advisories) \ >/dev/null 2>"$2" || rc=$? fi echo "$rc" } +# warm the registry cache for both trees (network allowed here, nothing else reads it); the scans then run offline +for t in "$PWD" "$W"; do (cd "$t" && cargo fetch --locked) >/dev/null 2>&1 || { echo "COULD-NOT-CHECK: cargo fetch --locked failed in $t" >&2; exit 75; }; done PRRC=$(scan "$PWD" "$W.pr.ndjson") BASERC=$(scan "$W" "$W.base.ndjson") bash "$GATE" verdict "$W.pr.ndjson" "$PRRC" "$W.base.ndjson" "$BASERC" diff --git a/scripts/ci/advisory_gate_selftest.sh b/scripts/ci/advisory_gate_selftest.sh index 9b696ac410..ba64bf0e5d 100644 --- a/scripts/ci/advisory_gate_selftest.sh +++ b/scripts/ci/advisory_gate_selftest.sh @@ -43,8 +43,13 @@ verdict_of() { # [old|new] (cargo-deny flavour the stub played); stdin: stub log [ "$nall" -eq 2 ] || { echo "ADG-003: $nall of 2 scans carry --all-features"; return; } [ "$fetch" -eq 1 ] || { echo "ADG-004: $fetch advisory-db fetches, want 1"; return; } if [ "${1:-new}" = old ]; then [ "$nfd" -eq 2 ] || { echo "ADG-004: $nfd of 2 scans carry --disable-fetch"; return; }; fi - # 0.20: --offline makes cargo metadata need a warm registry cache; a cold runner then exits rc=1 with no diagnostics (m70, #5793) - [ "$(grep -c -- '--offline' <<<"$log")" -eq 0 ] || { echo "ADG-005: a scan carries --offline"; return; } + # 0.20 (m70, #5793): both scans run --offline (one advisory-db head), and `cargo fetch --locked` runs on BOTH trees BEFORE the first scan, + # else --offline on a cold registry cache exits rc=1 with no diagnostics. The stub log is in call order. + if [ "${1:-new}" != old ]; then + [ "$(grep -c -- '--offline' <<<"$log")" -eq 2 ] || { echo "ADG-005: not both scans carry --offline"; return; } + [ "$(grep -c '^fetch --locked$' <<<"$log")" -eq 2 ] || { echo "ADG-005: cargo fetch --locked is not run once per tree"; return; } + [ "$(grep -n '^fetch --locked$' <<<"$log" | tail -1 | cut -d: -f1)" -lt "$(grep -n -- '--offline' <<<"$log" | head -1 | cut -d: -f1)" ] || { echo "ADG-005: cargo fetch --locked runs after a scan"; return; } + fi [ "$ncfg" -eq 1 ] || { echo "ADG-004: scans use $ncfg different --config files"; return; } echo ok } @@ -66,7 +71,8 @@ main() { } mutant 'scan without --all-features' 's/ --all-features//' 'ADG-003' mutant 'scan reads the net (no --disable-fetch)' 's/ --disable-fetch//' 'ADG-004' old 1 - mutant 'scan runs --offline (0.20)' 's/--locked --all-features/--locked --offline --all-features/' 'ADG-005' + mutant 'scan not offline (0.20)' 's/--locked --offline --all-features/--locked --all-features/' 'ADG-005' + mutant 'no cargo fetch --locked before the scans (0.20)' 's/cargo fetch --locked)/true)/' 'ADG-005' mutant 'second fetch before the base scan' 's|^BASERC=|cargo deny fetch --config "$W/deny.toml" db; BASERC=|' 'ADG-004' # ADG-003 on the other two scan sites: the push path and the exemption-liveness guard. local f @@ -75,7 +81,7 @@ main() { echo "SELF-TEST FAIL: ADG-003: $f scans advisories without --all-features" >&2; bad=1 fi done - [ "$bad" -eq 0 ] && echo "advisory_gate self-test: 2 real (cargo-deny 0.19 and 0.20 flags) + 4 mutants + 2 sites ok" + [ "$bad" -eq 0 ] && echo "advisory_gate self-test: 2 real (cargo-deny 0.19 and 0.20 flags) + 5 mutants + 2 sites ok" return "$bad" }