From d18529e56a5d2199e69c21a3c7af04155ec22efa Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Mon, 21 Sep 2026 03:41:14 +0200 Subject: [PATCH 1/3] fix(guard): a step NAME read as a bare guard_tree.sh dispatch hid four dark guards; the #3305 guard runs nowhere and dies on intel's python 3.10 (#3644) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Refs #3644 #3646 #3305 #3626 THE FINDING, proven by mutation before it was argued. check_guards_are_wired.sh counts a guard wired-by-dispatch when a workflow runs guard_tree.sh in a mode whose --dry-run RUN set contains it. Its invocation regex accepts `guard_tree.sh` followed by `'` (for a quoted run: string). ci.yml has two step NAMES, `"guard_tree.sh's own case table …"` and `"guard_tree.sh's parallel dispatcher …"`; the apostrophe matched, neither line says --no-cargo, so each was read as a bare dispatch in mode "all" -- 55 cargo-classified guards counted wired that `guard-tree` (--no-cargo) never runs and `guard-cargo` never names. Rewriting those two names (nothing else) turns the meta-guard RED: 3 -> 7 unwired -- check_model_ladder.sh, check_pathonly_devdeps_unused_in_src.sh, check_pr_review_counts.sh, check_pr_review_receipt.sh. Three of the four are cargo-classified by a COMMENT that says the build tool's name; none of those three invokes it. check_pathonly_devdeps_unused_in_src.sh (#3305: src/ may not use a dev-dep that publishing deletes; clean-room red 8/8 across two releases) has therefore run nowhere since it was written on 2026-09-15. And it could not have run on the intel hosts: `import os, re, sys, tomllib` at module level, python 3.10.12 there with neither tomllib nor tomli (measured on mac-server tonight; the sovereign-ci container has no python3 at all). Reproduced with python3.10 -S: the traceback is swallowed by `out=$(scan … || true)` and the case table prints "FAIL: missed hit_use" ×4 -- the regex reported broken by an interpreter that never ran it. Same class as #3626's guard on intel-clean-room-6. WHAT CHANGES scripts/check_guards_are_wired.sh - `_not_a_name_line` drops `name:` lines before the invocation match, in dispatcher_wired() and the per-guard scan. A step name is documentation whatever it contains. - rows 8-10: the exact ci.yml fixture (`- name: "guard_tree.sh's own case table …"`, no run: line) must leave the dispatched guard unwired; a QUOTED run: line still dispatches (the control the `'` exists for); a step named after a guard wires nothing. Mutant (drop removed): rows 8 and 10 RED. - the ledger's ratchet is now set-aperture, owned by this file. scripts/lib_baseline_ratchet.sh, scripts/check_baseline_ratchets.sh - set-aperture gains a NAME-entry admission for sets of FILES: an added entry with no `:` is admitted iff the comparand carries that file (at the entry's path, or beside the owning guard -- the ledger names siblings by basename and guard_tree.sh reads it so) AND the owning guard changed in the diff. A file this branch created is refused (PERF-028's shape). Six rows: predates -> admitted; branch WROTE it -> refused; no guard edit -> refused; escaping path -> refused; beside the OWNER -> admitted; beside a different owner -> refused. Lib mutants (always admit / branch removed / sibling resolution removed) each turn rows RED; a separate absolute-path check survived its mutant because `git cat-file -e` already refuses those, so it is not there. - classify: unwired_guards_baseline.txt -> set-aperture, owner named. scripts/unwired_guards_baseline.txt 3 -> 6, as APERTURE REVEALS with a reason per line (each may only leave): check_model_ladder.sh (T-2 release gate, the multiplatform_dogfood class); check_pr_review_counts.sh (RED on main today, "6 row(s) disagree" -- #3646); check_pr_review_receipt.sh (takes a receipt path; nothing passes one -- #3646). check_pathonly_devdeps_unused_in_src.sh is NOT ledgered: see below. scripts/check_pathonly_devdeps_unused_in_src.sh - readers tomllib -> tomli -> ENV rc=2 naming python version and $RUNNER_NAME. No purpose-built manifest reader: a second TOML implementation over 78 manifests, where a wrong "has source" is a silent PASS on exactly the #3305 class, is not a fallback worth shipping. UNMEASURED-with-a-name is the honest state on a runner with no TOML library. - the scanner ends with `SCAN-DONE manifests=N reader=…`; the shell REQUIRES it. Output without it (no reader, /bin/false, no python3) is ENV rc=2 -- never rc=1, never "no violations". The selftest and the tree scan propagate 2 instead of swallowing it. - five death rows: both readers absent (sys.modules nulled so the imports fail for real); interpreter exits 1 silently; no interpreter (127); the working interpreter as control; and THE WHOLE GUARD under the intel shape (nested, recursion-guarded) -- the call site is where `|| true` hid it. Mutants: trailer not required (3 rows RED); trailer printed on the no-reader path (1 RED); `|| true` restored (the whole-guard row RED). - three comments and one FAIL string reworded so the file no longer says the build tool's name with a space after it: it invokes none, and that substring is what CARGO_RE classifies on. guard_tree.sh --dry-run --no-cargo now lists it as `run:`; bare run on this tree rc=0 in 3 s. Passes: check_guards_are_wired --self-test 10/10; check_baseline_ratchets --self-test 61 rows; pathonly --selftest under python 3.13.1 and 3.10.12 (tomli present there; the forced-no-reader run gives ENV rc=2 with both); guard_tree_test.sh 23/0; bashrs-gate, pipe-into-grep-q, hardcoded-paths +0, apr pinned, no-timing-in-required, roadmap sorted/unique/additive/aggregate. No workflow file is edited; wiring #3646's pair is that ticket's. Co-Authored-By: Claude Opus 5 (1M context) --- docs/roadmaps/entries/PMAT-3644.yaml | 17 +++ docs/roadmaps/roadmap.yaml | 17 +++ scripts/check_baseline_ratchets.sh | 30 +++- scripts/check_guards_are_wired.sh | 57 +++++++- .../check_pathonly_devdeps_unused_in_src.sh | 128 ++++++++++++++++-- scripts/lib_baseline_ratchet.sh | 39 ++++-- scripts/unwired_guards_baseline.txt | 19 +++ 7 files changed, 278 insertions(+), 29 deletions(-) create mode 100644 docs/roadmaps/entries/PMAT-3644.yaml diff --git a/docs/roadmaps/entries/PMAT-3644.yaml b/docs/roadmaps/entries/PMAT-3644.yaml new file mode 100644 index 0000000000..f368071f82 --- /dev/null +++ b/docs/roadmaps/entries/PMAT-3644.yaml @@ -0,0 +1,17 @@ +- id: PMAT-3644 + github_issue: 3644 + item_type: task + title: 'check_guards_are_wired.sh reads a step NAME as a bare guard_tree.sh dispatch: 4 guards dark, incl. the #3305 publish-strip guard (which also dies on intel''s python 3.10)' + status: planned + priority: critical + assigned_to: null + created: 2026-09-21T01:39:50Z + updated: 2026-09-21T01:39:50Z + spec: null + acceptance_criteria: [] + phases: [] + subtasks: [] + estimated_effort: null + labels: + - kind:code + notes: 'ACCEPTANCE (hand-entered from issue #3644 "What lands"; minted on the PR branch so the AD-04 quorum can run). 1. check_guards_are_wired.sh drops name: lines before the invocation match, in dispatcher_wired() and the per-guard scan; rows 8-10 (the exact ci.yml step-name fixture must NOT count as a dispatch; a quoted run: line still does; a step named after a guard wires nothing); the mutation (remove the drop) turns rows 8 and 10 RED. 2. check_pathonly_devdeps_unused_in_src.sh: tomllib -> tomli -> ENV rc=2 naming interpreter and runner; the scanner ends with a SCAN-DONE trailer the shell REQUIRES; five death rows incl. the whole guard under the intel shape; comments reworded so CARGO_RE no longer classifies it out of --no-cargo, so guard-tree runs it (bare run on the tree: rc 0, 3 s). 3. unwired_guards_baseline.txt records the other three dark guards as APERTURE REVEALS (set-aperture gains a NAME-entry admission: a file the comparand carries, in a diff that changes the owning guard; six case rows, each admission beside its refusal) with the reason each may only leave; #3646 owns the pr_review pair. No workflow file is edited.' diff --git a/docs/roadmaps/roadmap.yaml b/docs/roadmaps/roadmap.yaml index c4d7dc5532..43ea64664e 100644 --- a/docs/roadmaps/roadmap.yaml +++ b/docs/roadmaps/roadmap.yaml @@ -18869,3 +18869,20 @@ roadmap: labels: - kind:code notes: 'ACCEPTANCE (hand-entered; pmat work add derived neither spec: nor acceptance_criteria: — defect paiml-mcp-agent-toolkit#1414). Spec: docs/specifications/ruling-receipts-under-contract.md (operator ruling 2026-09-20). Done when: contracts/parity-receipt-v1.yaml with its shape on origin/main; extract:parity-receipt implemented; pv lint --gate shapes --path evidence/ in the required check; PR body shows 7 older receipts RED before back-fill and 0 after, plant violation = 1, mutation RED, pc_shape fired; check_parity_receipt.sh folded into the shape or its remainder listed under not_expressible; ONT-4c3 bound with the parity receipt as focus node; verdict rendered on a fleet host once the pv pin lands, else checkout-only: true. Sigma parent bound: json. Back-fill denominator: 7. STOP: subset-insufficient; shared-file-touched without the guard label; any threshold typed into the shape instead of resolved from thresholds.yaml.' +- id: PMAT-3644 + github_issue: 3644 + item_type: task + title: 'check_guards_are_wired.sh reads a step NAME as a bare guard_tree.sh dispatch: 4 guards dark, incl. the #3305 publish-strip guard (which also dies on intel''s python 3.10)' + status: planned + priority: critical + assigned_to: null + created: 2026-09-21T01:39:50Z + updated: 2026-09-21T01:39:50Z + spec: null + acceptance_criteria: [] + phases: [] + subtasks: [] + estimated_effort: null + labels: + - kind:code + notes: 'ACCEPTANCE (hand-entered from issue #3644 "What lands"; minted on the PR branch so the AD-04 quorum can run). 1. check_guards_are_wired.sh drops name: lines before the invocation match, in dispatcher_wired() and the per-guard scan; rows 8-10 (the exact ci.yml step-name fixture must NOT count as a dispatch; a quoted run: line still does; a step named after a guard wires nothing); the mutation (remove the drop) turns rows 8 and 10 RED. 2. check_pathonly_devdeps_unused_in_src.sh: tomllib -> tomli -> ENV rc=2 naming interpreter and runner; the scanner ends with a SCAN-DONE trailer the shell REQUIRES; five death rows incl. the whole guard under the intel shape; comments reworded so CARGO_RE no longer classifies it out of --no-cargo, so guard-tree runs it (bare run on the tree: rc 0, 3 s). 3. unwired_guards_baseline.txt records the other three dark guards as APERTURE REVEALS (set-aperture gains a NAME-entry admission: a file the comparand carries, in a diff that changes the owning guard; six case rows, each admission beside its refusal) with the reason each may only leave; #3646 owns the pr_review pair. No workflow file is edited.' diff --git a/scripts/check_baseline_ratchets.sh b/scripts/check_baseline_ratchets.sh index ddb9162967..40c27e21f2 100644 --- a/scripts/check_baseline_ratchets.sh +++ b/scripts/check_baseline_ratchets.sh @@ -93,7 +93,7 @@ classify() { # classify -> "[reason]", rc 1 if unclassifie cb200_baseline.txt) printf 'count\n' ;; # mirrors .pmat-gates.toml [tdg] baseline (PMAT-937) test_fixture_path_baseline.txt) printf 'count\n' ;; tracked_ignored_baseline.txt) printf 'count\n' ;; - unwired_guards_baseline.txt) printf 'set\n' ;; + unwired_guards_baseline.txt) printf 'set-aperture\tscripts/check_guards_are_wired.sh\n' ;; # NAME entries: a guard file that predates the comparand may be ledgered when the meta-guard itself widens (#3644) # NOT a ratchet either, and for the same reason one level along: this # registry is DERIVED from the test sources on every run # (scripts/check_tree_reader_tests.sh) and must equal that derivation @@ -323,6 +323,12 @@ if [ "${1:-}" = "--self-test" ] || [ "${1:-}" = "--selftest" ]; then # PREDATES; `fresh.md` is written by the working tree only. printf 'the old claim, 2.93 times faster\n' > "$SR/pre.md" printf 'GUARD v1\n' > "$SR/guard.sh" + # for the NAME-entry rows (#3644): an owner in a subdirectory with a + # sibling file, the shape of scripts/check_guards_are_wired.sh and + # its ledger of basenames + mkdir -p "$SR/sub" + printf 'OWNER v1\n' > "$SR/sub/owner.sh" + printf 'a sibling that predates\n' > "$SR/sub/sib.sh" git -C "$SR" add -A >/dev/null 2>&1 git -C "$SR" -c commit.gpgsign=false commit -qm 'aperture base' >/dev/null 2>&1 SR_APER=$(git -C "$SR" rev-parse HEAD) @@ -386,6 +392,28 @@ if [ "${1:-}" = "--self-test" ] || [ "${1:-}" = "--selftest" ]; then # guard edit at all -- otherwise the new kind would be strictly # WORSE than `set` on the ordinary path. ap_row 'aperture unchanged is green' 0 '# header\n' 'GUARD v1\n' 'the old claim, 2.93 times faster\n' + # -- NAME entries (#3644). unwired_guards_baseline.txt is a set of + # FILES, not of lines: the meta-guard's name:-line blindness hid + # four dark guards, and widening it reveals files already in the + # tree. (a) for a name is "the comparand carries the file" -- at the + # entry's path, or beside the owning guard, since the ledger names + # siblings by basename. Each admission has its refusal beside it. + ap_row 'aperture NAME entry that predates is admitted' 0 '# header\npre.md\n' 'GUARD v2\n' 'the old claim, 2.93 times faster\n' + ap_row 'aperture NAME entry this branch WROTE refuses' 1 '# header\nfresh.md\n' 'GUARD v2\n' 'the old claim, 2.93 times faster\n' + ap_row 'aperture NAME entry without a guard edit refuses' 1 '# header\npre.md\n' 'GUARD v1\n' 'the old claim, 2.93 times faster\n' + ap_row 'aperture NAME entry escaping the repo refuses' 1 '# header\n../pre.md\n' 'GUARD v2\n' 'the old claim, 2.93 times faster\n' + # beside the owner: `sib.sh` is not at the root, it is next to sub/owner.sh + printf '# header\nsib.sh\n' > "$SR/$P" + printf 'OWNER v2\n' > "$SR/sub/owner.sh" + ( BASELINE_RATCHET_BASE_REF="$SR_APER" \ + baseline_ratchet_check "$SR" "$P" set-aperture sub/owner.sh ) >/dev/null 2>&1 + say_row 'aperture NAME entry beside the OWNER is admitted' 0 $? + printf '# header\nsib.sh\n' > "$SR/$P" + printf 'GUARD v2\n' > "$SR/guard.sh" + ( BASELINE_RATCHET_BASE_REF="$SR_APER" \ + baseline_ratchet_check "$SR" "$P" set-aperture guard.sh ) >/dev/null 2>&1 + say_row 'aperture NAME entry beside a DIFFERENT owner refuses' 1 $? + printf 'OWNER v1\n' > "$SR/sub/owner.sh" # A missing owner argument must fail CLOSED. Called with no guard # path, "could not check" must never read as "no growth". printf '%b' "$AP_BASE" > "$SR/$P" diff --git a/scripts/check_guards_are_wired.sh b/scripts/check_guards_are_wired.sh index 2d581ed8f1..309dddedc0 100755 --- a/scripts/check_guards_are_wired.sh +++ b/scripts/check_guards_are_wired.sh @@ -66,11 +66,25 @@ BASELINE="${REPO_ROOT}/scripts/unwired_guards_baseline.txt" # committed by the very guard that exists to catch it. An argument-wired guard # is still named by its own workflow and is found by the scan below on that # account; a release-time guard is deliberately unwired and stays reported. +# A STEP NAME IS NOT AN INVOCATION (#3644). ci.yml carries +# - name: "guard_tree.sh's own case table (BSE-01, wired BSE-02)" +# - name: "guard_tree.sh's parallel dispatcher case table (PMAT-1098)" +# The invocation regex below accepts `guard_tree.sh` followed by `'` -- meant +# for a quoted `run: "..."` -- so the apostrophe in `guard_tree.sh's` matched, +# neither line says --no-cargo, and each was read as a BARE dispatch in mode +# "all": the full --dry-run RUN set, 55 cargo-classified guards that +# `--no-cargo` never runs, all counted wired. Four were dark behind it, +# including the #3305 publish-strip guard, and this meta-guard said PASS. +# Proven by rewriting those two names (nothing else): 3 -> 7 unwired. +# Mention-vs-execution, one level below the trailing-comment case above. +# A `name:` line is documentation whatever it contains; drop it before matching. +_not_a_name_line() { grep -vE '^[[:space:]]*-?[[:space:]]*name:' || true; } + dispatcher_wired() { local root="$1" lines modes mode flag out [ -f "$root/scripts/guard_tree.sh" ] || return 0 lines=$(grep -rh --include='*.yml' --include='*.yaml' -- 'guard_tree.sh' \ - "$root"/.github/workflows/ 2>/dev/null | sed 's/#.*$//') || lines='' + "$root"/.github/workflows/ 2>/dev/null | sed 's/#.*$//' | _not_a_name_line) || lines='' # Invocation, not mention -- same test as the scan below. lines=$(grep -E "(^|[[:space:];&|(])((ba)?sh[[:space:]]+|\\./)?[^[:space:]]*guard_tree\\.sh([[:space:]]|$|['\"])" \ <<< "$lines") || lines='' @@ -154,7 +168,7 @@ unwired_in() { local mentions mentions=$(grep -rh --include='*.yml' --include='*.yaml' -- "$base" \ "$root"/.github/workflows/ 2>/dev/null \ - | sed 's/#.*$//') || mentions='' + | sed 's/#.*$//' | _not_a_name_line) || mentions='' if ! grep -qE "(^|[[:space:];&|(])((ba)?sh[[:space:]]+|\\./)?[^[:space:]]*${base}([[:space:]]|$|['\"])" <<< "$mentions" ; then printf '%s\n' "$base" fi @@ -272,8 +286,39 @@ if [ "${1:-}" = "--self-test" ]; then printf 'FAIL row 7 got [%s], expected [check_nightly_only.sh check_nowhere.sh ]\n' "$got7"; fails=1 fi + # ── Rows 8-10: A STEP NAME IS NOT AN INVOCATION (#3644) ───────────────── + # + # Row 8 is the exact ci.yml shape that hid four guards: a step NAMED + # `guard_tree.sh's ...`, no dispatcher run: line. The dispatched fixture + # guard must come back as unwired. Row 9 is the control the `'` in the + # regex exists for: a QUOTED run: line still dispatches. Row 10 is the same + # blindness in the per-guard scan: a step named after a guard wires nothing. + printf 'jobs:\n gate:\n steps:\n - name: "guard_tree.sh'"'"'s own case table (BSE-01, wired BSE-02)"\n run: echo a name is not a dispatch\n' \ + > "$TD2/.github/workflows/ci.yml" + got8=$(unwired_in "$TD2" | tr '\n' ' ') + if [ "$got8" = "check_dispatched.sh " ]; then + printf 'ok row 8 a step NAMED guard_tree.sh'"'"'s ... is not a dispatch; the guard stays unwired\n' + else + printf 'FAIL row 8 got [%s], expected [check_dispatched.sh ] -- a step name read as a bare dispatch\n' "$got8"; fails=1 + fi + printf 'jobs:\n gate:\n steps:\n - run: "bash scripts/guard_tree.sh --no-cargo"\n' \ + > "$TD2/.github/workflows/ci.yml" + if [ -z "$(unwired_in "$TD2")" ]; then + printf 'ok row 9 a QUOTED run: line still dispatches (the control for row 8)\n' + else + printf 'FAIL row 9 quoted dispatch not honoured: [%s]\n' "$(unwired_in "$TD2" | tr '\n' ' ')"; fails=1 + fi + printf 'jobs:\n x:\n steps:\n - name: "check_dark.sh'"'"'s job"\n run: bash scripts/check_wired.sh\n' \ + > "$TD/.github/workflows/ci.yml" + got10=$(unwired_in "$TD" | tr '\n' ' ') + if [ "$got10" = "check_dark.sh " ]; then + printf 'ok row 10 a step named after a guard wires nothing\n' + else + printf 'FAIL row 10 got [%s], expected [check_dark.sh ]\n' "$got10"; fails=1 + fi + [ "$fails" -eq 0 ] || { printf '\nSELF-TEST FAILED\n'; exit 1; } - printf '\nSELF-TEST PASSED (7/7)\n' + printf '\nSELF-TEST PASSED (10/10)\n' exit 0 fi @@ -332,7 +377,11 @@ fi # branch cannot rewrite, and never the branch against itself. # shellcheck source=scripts/lib_baseline_ratchet.sh . "${REPO_ROOT}/scripts/lib_baseline_ratchet.sh" || exit 1 -baseline_ratchet_check "${REPO_ROOT}" scripts/unwired_guards_baseline.txt set || exit 1 +# set-aperture, owned by this file (#3644): when THIS guard widens and reveals +# guards that were already dark, the ledger may record them -- only guards the +# comparand already carries, only in a diff that changes this file, every +# admission printed. On every other PR it is `set`: shrink-only. +baseline_ratchet_check "${REPO_ROOT}" scripts/unwired_guards_baseline.txt set-aperture scripts/check_guards_are_wired.sh || exit 1 if [ ! -f "$BASELINE" ]; then printf 'FAIL: %s missing. Run --update once to establish it.\n' "$BASELINE" diff --git a/scripts/check_pathonly_devdeps_unused_in_src.sh b/scripts/check_pathonly_devdeps_unused_in_src.sh index c6ae4812a8..092016d44f 100755 --- a/scripts/check_pathonly_devdeps_unused_in_src.sh +++ b/scripts/check_pathonly_devdeps_unused_in_src.sh @@ -1,9 +1,9 @@ #!/usr/bin/env bash # check_pathonly_devdeps_unused_in_src.sh - src/ may not reference a dev-dep -# that `cargo publish` deletes. +# that publishing to crates.io deletes. # # THE CLASS. A dev-dependency written as `{ path = "..." }` with no version, -# no git and no workspace inheritance carries NO SOURCE. `cargo publish` omits +# no git and no workspace inheritance carries NO SOURCE. The publish step omits # it from the published manifest entirely -- that omission is deliberate and is # what lets a dev-dep cycle (aprender-compute -> aprender-core) be legal. But # the crate's own `#[cfg(test)]` code is published with it. If that code names @@ -29,8 +29,30 @@ # baseline fails, and a baseline pair with no remaining violation also fails, so # a fix must delete its row in the same commit. Draining it to empty is #3306. # +# WHO RUNS IT, AND WHY THIS FILE NEVER WRITES THE BUILD TOOL'S NAME WITH A SPACE +# AFTER IT (#3644). guard_tree.sh classifies a guard as cargo-using by the +# substring CARGO_RE, comments included; `guard-tree` runs only the cargo-free +# population (--no-cargo) and `guard-cargo` names its guards by hand. This file +# said the tool's name in three comments and one FAIL string, was classified +# cargo-using, was named by nothing, and RAN NOWHERE from the day it was +# written (2026-09-15) -- while check_guards_are_wired.sh reported PASS, blinded +# by a step name. It invokes no build tool: python over the manifests, then +# grep. Now it is in the --no-cargo population and guard-tree runs it. +# +# ENV IS NEVER A CODE VERDICT (#3644, the same shape as #3626's guard). The +# scanner needs a TOML reader: tomllib (python 3.11+) or tomli. The intel +# runner host has python 3.10.12 and neither; the old module-level import died +# with a traceback, `out=$(scan ... || true)` swallowed it, and the case table +# said "FAIL: missed hit_use" -- the regex reported broken by an interpreter +# that never ran it. Now the scanner ends with a `SCAN-DONE manifests=N` +# trailer that the shell REQUIRES; anything without it (no reader, a dead +# interpreter, no python3) is ENV rc=2 naming the interpreter and the runner -- +# never rc=1, never "no violations". +# # Runs bare, no arguments, from anywhere in the repo. `--selftest` runs the case # table only. A bare run does BOTH: the case table first, then the tree. +# PATHONLY_GUARD_PYTHON= test seam: a dead one reproduces the intel shape +# PATHONLY_GUARD_FORCE_NO_TOML=1 test seam: both readers fail to import, for real set -euo pipefail REPO="$(git -C "$(dirname "$0")" rev-parse --show-toplevel)" @@ -47,13 +69,33 @@ cleanup() { } trap cleanup EXIT -# scan -> prints "manifest|alias|src-file:line:text" per violation +# scan -> prints "manifest|alias|src-file:line:text" per violation. +# rc 0 with the trailer consumed; rc 2 (ENV) when the scanner did not finish. scan() { - python3 - "$1" <<'PY' -import os, re, sys, tomllib + local out rc=0 interp="${PATHONLY_GUARD_PYTHON:-python3}" + out=$(PATHONLY_GUARD_RUNNER="${RUNNER_NAME:-unknown}" "$interp" - "$1" 2>&1 <<'PY' +import os, re, sys + +runner = os.environ.get("PATHONLY_GUARD_RUNNER", "unknown") +if os.environ.get("PATHONLY_GUARD_FORCE_NO_TOML") == "1": + sys.modules["tomllib"] = None # the imports below then raise for real + sys.modules["tomli"] = None + +toml = None +for name in ("tomllib", "tomli"): + try: + toml = __import__(name) + break + except ImportError: + continue +if toml is None: + print("ENV: no TOML reader on this interpreter (python %d.%d, need tomllib >= 3.11 or tomli) runner=%s -- " + "the manifests were not read; this is not 'no violations'" % (sys.version_info[0], sys.version_info[1], runner)) + sys.exit(2) root = sys.argv[1] KINDS = ("dev-dependencies",) +manifests = 0 def sourceless(spec): """True when the dep carries a path and nothing that survives publish.""" @@ -73,9 +115,10 @@ for dirpath, dirnames, filenames in os.walk(root): continue try: with open(manifest, "rb") as fh: - data = tomllib.load(fh) - except (tomllib.TOMLDecodeError, OSError): + data = toml.load(fh) + except (toml.TOMLDecodeError, OSError): continue + manifests += 1 aliases = [a for k in KINDS for a, s in (data.get(k) or {}).items() if sourceless(s)] if not aliases: continue @@ -100,7 +143,21 @@ for dirpath, dirnames, filenames in os.walk(root): rel_m = os.path.relpath(manifest, root) rel_s = os.path.relpath(path, root) print(f"{rel_m}|{alias}|{rel_s}:{n}:{stripped[:100]}") +# positive evidence that the scan RAN TO THE END; the shell refuses any output without it +print("SCAN-DONE manifests=%d reader=%s" % (manifests, toml.__name__)) PY + ) || rc=$? + case "$out" in + *"SCAN-DONE manifests="*) ;; + *) + printf 'ENV: the scanner did not finish (interpreter=%s rc=%s runner=%s) -- not a pass, not a code verdict\n' \ + "$interp" "$rc" "${RUNNER_NAME:-unknown}" >&2 + [ -z "$out" ] || printf '%s\n' "$out" | sed 's/^/ /' >&2 + return 2 ;; + esac + # the trailer is evidence, not a row + printf '%s\n' "$out" | grep -v '^SCAN-DONE ' || true + return 0 } # ── case table: every row is a fixture the regex/selector must classify ── @@ -125,7 +182,7 @@ selftest() { mk hit_underscore 'two-words = { path = "../tw" }' 'use two_words::x;' mk hit_extern 'sib = { path = "../sib" }' 'extern crate sib;' - # MUST NOT MATCH -- these all survive `cargo publish`, or are not a reference + # MUST NOT MATCH -- these all survive publishing, or are not a reference mk ok_versioned 'sib = { path = "../sib", version = "0.1.0" }' 'use sib::thing;' mk ok_workspace 'sib = { workspace = true }' 'use sib::thing;' mk ok_git 'sib = { git = "https://x/y", path = "../sib" }' 'use sib::thing;' @@ -134,7 +191,14 @@ selftest() { mk ok_comment 'sib = { path = "../sib" }' '// use sib::thing;' mk ok_substring 'sib = { path = "../sib" }' 'use sibling::thing;' - out="$(scan "$tmp" || true)" + # ENV from the scanner is ENV here: rc 2, never "missed hit_use" (#3644) + local env_rc=0 + out="$(scan "$tmp")" || env_rc=$? + if [ "$env_rc" -ne 0 ]; then + printf 'ENV: the case table could not be measured (scan rc=%s)\n' "$env_rc" + cleanup; FIXTURES="" + return 2 + fi # Piping a producer into a quiet grep is banned here # (check_no_pipe_into_grep_q.sh): grep exits on its first match and the @@ -156,18 +220,54 @@ selftest() { for row in ok_versioned ok_workspace ok_git ok_registry ok_unused ok_comment ok_substring; do if row_present "$row"; then printf 'FAIL: false positive on %s\n' "$row"; rc=1; fi done + + # THE DEATH ROWS (#3644): every way the scanner can fail to run must come + # back as ENV rc=2, never as a row verdict. A must-match fixture is scanned + # each time, so a wrong classification would have to invent "missed hit_use" + # (rc 1) or "no violations" (rc 0) out of an interpreter that never ran. + death() { # death