From b31a2c2a96ad62bf69b2d9fc63723cc6a7ffec1e Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Mon, 14 Sep 2026 21:38:00 +0200 Subject: [PATCH 1/2] =?UTF-8?q?fix(nightly):=20guards-nightly=20has=20been?= =?UTF-8?q?=20RED=20naming=20a=20step=20that=20runs=20every=20night=20?= =?UTF-8?q?=E2=80=94=20the=20matcher=20cannot=20see=20a=20YAML-quoted=20na?= =?UTF-8?q?me?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Five scheduled lanes are red. This is the one whose cause is entirely inside the guard rather than in the thing it guards. FAIL: manifest names a step this workflow does not run: Test-tier decision case table (BSE-17): quick/full/reuse, drift is ENV The step is there. guards-nightly.yml line 158: - name: "Test-tier decision case table (BSE-17): quick/full/reuse, drift is ENV" The inline check matched the BARE spelling: grep -Fq -- "- name: $name" .github/workflows/guards-nightly.yml A step name containing a colon-space MUST be quoted in YAML, and this one contains `(BSE-17): quick`. So the matcher looked for `- name: Test-tier...` while the file said `- name: "Test-tier...`, and the lane has been red ever since that name was quoted — reporting a step that runs on every nightly as one the workflow does not run. A guard that names a real step as missing is worse than no guard: it teaches the reader to skip the lane. EXTRACTED rather than patched in place. Logic inside a `run:` block cannot be unit-tested and cannot carry a case table, which is why a one-character class of defect survived in it. scripts/check_guards_nightly_manifest.sh now matches on the PARSED name, so all three YAML quotings of one string are one step. Case table, 7 rows, both directions: bare name parsed ok DOUBLE-quoted name unquoted (the defect) ok single-quoted name unquoted ok trailing whitespace stripped ok manifest fully covered -> exit 0 a genuinely absent step -> exit 1 <- still fails; not made permissive vacuity floor, <5 entries parsed -> exit 1 Against the real tree: `ok 7 manifest entries, every one a step in this workflow`. Mutation: appending one bogus entry turns it RED again, so the fix widened the matcher without disarming it. The self-test found a defect in its OWN HARNESS first: invoking `step_names` through `bash -c 'source ...'` re-ran this file's dispatch and returned nothing, so four rows failed while the parser was correct. A broken harness reporting a working parser as broken is the same error as the one being fixed, in the other direction. Wiring: `guard_tree.sh --dry-run` reports it `wired-elsewhere guards-nightly.yml` and check_guards_are_wired.sh exits 0. bashrs 7.4.1: 0 errors. check_roadmap_sorted.sh caught the roadmap entry inserted at a stale slot — main gained PMAT-3228/3229 while this branch was being written, so "after 3222" was no longer sorted. Re-inserted among its numeric peers. ont-delta: none — a CI guard fix; it adds no entity, shape, verdict reason or resolves target of its own. Pmat-Ticket: PMAT-3234 Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/guards-nightly.yml | 32 ++++--- docs/roadmaps/roadmap.yaml | 20 +++++ scripts/check_guards_nightly_manifest.sh | 110 +++++++++++++++++++++++ 3 files changed, 145 insertions(+), 17 deletions(-) create mode 100755 scripts/check_guards_nightly_manifest.sh diff --git a/.github/workflows/guards-nightly.yml b/.github/workflows/guards-nightly.yml index 17da37b29d..8a0ea9d37b 100644 --- a/.github/workflows/guards-nightly.yml +++ b/.github/workflows/guards-nightly.yml @@ -98,24 +98,22 @@ jobs: # names a step; each must appear VERBATIM as a `- name:` in this file. # Fail-closed: an empty manifest, a missing file or an unreadable line is # a failure, never a pass. + # EXTRACTED to scripts/check_guards_nightly_manifest.sh (PMAT-3234). The + # inline version matched `grep -Fq -- "- name: $name"` — the BARE spelling — + # while a step name containing a colon-space must be QUOTED in YAML, and one + # of them is. So the matcher looked for `- name: Test-tier...` and the file + # said `- name: "Test-tier...`, and this lane went RED naming a step that + # runs on every nightly as one the workflow does not run. + # + # Logic that lives in a `run:` block cannot be unit-tested and cannot carry a + # case table; that is why the defect survived. The script matches on the + # PARSED name, so all three YAML quotings of one string are one step, and it + # ships 7 rows in both directions (a genuinely absent step still fails, and a + # manifest that parsed almost nothing is refused rather than passed). + - name: Manifest-vs-workflow case table (PMAT-3234, both directions) + run: bash scripts/check_guards_nightly_manifest.sh --self-test - name: Every step the manifest says moved here must exist here - run: | - set -euo pipefail - manifest=scripts/guards_nightly_manifest.txt - [ -s "$manifest" ] || { echo "FAIL: $manifest missing or empty"; exit 1; } - n=0; bad=0 - while IFS=$'\t' read -r secs name; do - case "${secs:-}" in ''|'#'*) continue ;; esac - [ -n "${name:-}" ] || { echo "FAIL: manifest line has no step name: $secs"; bad=1; continue; } - n=$((n + 1)) - if ! grep -Fq -- "- name: $name" .github/workflows/guards-nightly.yml; then - echo "FAIL: manifest names a step this workflow does not run: $name" - bad=1 - fi - done < <(grep -vE '^[[:space:]]*(#|$)' "$manifest") - [ "$n" -ge 5 ] || { echo "FAIL (vacuity): only $n manifest entries parsed"; exit 1; } - [ "$bad" -eq 0 ] || exit 1 - echo "ok $n manifest entries, every one a step in this workflow" + run: bash scripts/check_guards_nightly_manifest.sh # THE COMPARAND, FETCHED ONCE, BEFORE ANY GUARD THAT NEEDS IT -- the # SATD step below measures its ceiling on origin/main with the HEAD-built # suite, and a depth-1 checkout has no such ref until this runs. diff --git a/docs/roadmaps/roadmap.yaml b/docs/roadmaps/roadmap.yaml index 216704f752..8dadba9a8f 100644 --- a/docs/roadmaps/roadmap.yaml +++ b/docs/roadmaps/roadmap.yaml @@ -17738,3 +17738,23 @@ roadmap: labels: - kind:code notes: null + +- id: PMAT-3234 + github_issue: null + item_type: task + title: 'guards-nightly has been RED on a matcher that cannot see a YAML-quoted step name' + status: inprogress + priority: high + assigned_to: null + created: 2026-09-14T19:45:00Z + updated: 2026-09-14T19:45:00Z + spec: docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md + acceptance_criteria: [] + phases: [] + subtasks: [] + estimated_effort: null + labels: + - kind:code + notes: 'a step name containing a colon-space must be quoted in YAML; the inline + grep -F matched only the bare spelling, so the lane named a step that runs every + night as one the workflow does not run' diff --git a/scripts/check_guards_nightly_manifest.sh b/scripts/check_guards_nightly_manifest.sh new file mode 100755 index 0000000000..111676c245 --- /dev/null +++ b/scripts/check_guards_nightly_manifest.sh @@ -0,0 +1,110 @@ +#!/usr/bin/env bash +# check_guards_nightly_manifest.sh — every step scripts/guards_nightly_manifest.txt +# says moved to guards-nightly.yml must actually be a step in it. +# +# THE DEFECT THIS FIXES. The check lived inline in the workflow and matched: +# +# grep -Fq -- "- name: $name" .github/workflows/guards-nightly.yml +# +# A fixed-string match against the BARE spelling. But a step name containing a +# colon-space must be QUOTED in YAML, and one of them does: +# +# manifest: Test-tier decision case table (BSE-17): quick/full/reuse, drift is ENV +# workflow: - name: "Test-tier decision case table (BSE-17): quick/full/reuse, drift is ENV" +# +# so the matcher looked for `- name: Test-tier...` and the file said +# `- name: "Test-tier...`. guards-nightly has been RED on this since the name was +# quoted — reporting a step that runs on every nightly as one the workflow does +# not run. A guard that names a real step as missing is worse than no guard: it +# trains the reader to ignore it. +# +# THE MATCH IS NOW ON THE PARSED NAME, not on a spelling. All three YAML forms of +# the same string are the same step, so all three match, and a name that is +# genuinely absent still fails. The case table below pins both directions. +# +# bash scripts/check_guards_nightly_manifest.sh +# bash scripts/check_guards_nightly_manifest.sh --self-test +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +WORKFLOW="${GUARDS_NIGHTLY_WORKFLOW:-$REPO_ROOT/.github/workflows/guards-nightly.yml}" +MANIFEST="${GUARDS_NIGHTLY_MANIFEST:-$REPO_ROOT/scripts/guards_nightly_manifest.txt}" + +# Every `- name:` in a workflow, with YAML quoting removed. One per line. +step_names() { # step_names WORKFLOW + sed -nE 's/^[[:space:]]*-[[:space:]]+name:[[:space:]]*(.*)$/\1/p' "$1" \ + | sed -E 's/[[:space:]]+$//' \ + | sed -E 's/^"(.*)"$/\1/; s/^'"'"'(.*)'"'"'$/\1/' +} + +check() { + local n=0 bad=0 secs name names + [ -s "$MANIFEST" ] || { printf 'FAIL: %s missing or empty\n' "$MANIFEST" >&2; return 1; } + [ -s "$WORKFLOW" ] || { printf 'FAIL: %s missing or empty\n' "$WORKFLOW" >&2; return 1; } + names="$(step_names "$WORKFLOW")" + while IFS=$'\t' read -r secs name; do + case "${secs:-}" in ''|'#'*) continue ;; esac + [ -n "${name:-}" ] || { printf 'FAIL: manifest line has no step name: %s\n' "$secs"; bad=1; continue; } + n=$((n + 1)) + if ! grep -Fqx -- "$name" <<<"$names"; then + printf 'FAIL: manifest names a step this workflow does not run: %s\n' "$name" + bad=1 + fi + done < <(grep -vE '^[[:space:]]*(#|$)' "$MANIFEST") + # Vacuity: a manifest that parsed almost nothing would pass by reading nothing. + [ "$n" -ge 5 ] || { printf 'FAIL (vacuity): only %s manifest entries parsed\n' "$n" >&2; return 1; } + [ "$bad" -eq 0 ] || return 1 + printf 'ok %s manifest entries, every one a step in this workflow\n' "$n" +} + +self_test() { + local t pass=0 fail=0 rc + t="$(mktemp -d)" + case "$t" in /tmp/*|/var/tmp/*) : ;; *) printf 'NO-GO: odd mktemp path %s\n' "$t" >&2; return 2 ;; esac + row() { if [ "$2" = "$3" ]; then pass=$((pass+1)); printf ' ok %-48s %s\n' "$1" "$3" + else fail=$((fail+1)); printf ' FAIL %-48s want=%s got=%s\n' "$1" "$3" "$2"; fi; } + + cat > "$t/wf.yml" <<'YML' +jobs: + x: + steps: + - name: Plain step + - name: "Quoted step (BSE-17): has a colon" + - name: 'Single quoted step' + - name: Trailing space step +YML + # THE PARSER, four rows: the defect was here and nowhere else. + # step_names is in scope here; invoking it through `bash -c 'source ...'` + # re-ran this file's own dispatch and returned nothing, which is a broken + # HARNESS reporting a working parser as broken — the row that matters most + # is the one most easily faked, in either direction. + local got + got="$(step_names "$t/wf.yml")" + grep -Fqx 'Plain step' <<<"$got" && row "bare name parsed" ok ok || row "bare name parsed" no ok + grep -Fqx 'Quoted step (BSE-17): has a colon' <<<"$got" && row "DOUBLE-quoted name unquoted (the defect)" ok ok || row "DOUBLE-quoted name unquoted (the defect)" no ok + grep -Fqx 'Single quoted step' <<<"$got" && row "single-quoted name unquoted" ok ok || row "single-quoted name unquoted" no ok + grep -Fqx 'Trailing space step' <<<"$got" && row "trailing whitespace stripped" ok ok || row "trailing whitespace stripped" no ok + + # END TO END, both directions. + printf '1\tPlain step\n2\tQuoted step (BSE-17): has a colon\n3\tSingle quoted step\n4\tTrailing space step\n5\tPlain step\n' > "$t/ok.txt" + set +e; GUARDS_NIGHTLY_WORKFLOW="$t/wf.yml" GUARDS_NIGHTLY_MANIFEST="$t/ok.txt" bash "${BASH_SOURCE[0]}" >/dev/null 2>&1; rc=$?; set -e + row "manifest fully covered -> pass" "$rc" "0" + + printf '1\tPlain step\n2\tQuoted step (BSE-17): has a colon\n3\tSingle quoted step\n4\tTrailing space step\n5\tA step that is not there\n' > "$t/bad.txt" + set +e; GUARDS_NIGHTLY_WORKFLOW="$t/wf.yml" GUARDS_NIGHTLY_MANIFEST="$t/bad.txt" bash "${BASH_SOURCE[0]}" >/dev/null 2>&1; rc=$?; set -e + row "a genuinely absent step -> STILL FAILS" "$rc" "1" + + printf '1\tPlain step\n' > "$t/thin.txt" + set +e; GUARDS_NIGHTLY_WORKFLOW="$t/wf.yml" GUARDS_NIGHTLY_MANIFEST="$t/thin.txt" bash "${BASH_SOURCE[0]}" >/dev/null 2>&1; rc=$?; set -e + row "vacuity floor: <5 entries -> refuse" "$rc" "1" + + printf 'self-test: %s passed, %s failed\n' "$pass" "$fail" + [ -n "$t" ] && [ -d "$t" ] && rm -rf "$t" + [ "$fail" -eq 0 ] +} + +case "${1:-}" in + --self-test) self_test ;; + "") check ;; + *) printf 'usage: %s [--self-test]\n' "$(basename "$0")" >&2; exit 2 ;; +esac From 5b313d65feca13df4d76b7ca1fa9451d6c663412 Mon Sep 17 00:00:00 2001 From: Noah Gift Date: Mon, 14 Sep 2026 22:44:14 +0200 Subject: [PATCH 2/2] fix(roadmap): collapse my insertion's stray blank line (roadmap_trim.py) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Third time tonight. My insertion helper rejoins entry blocks with an added newline, re-serialising the entry ABOVE the insertion without any field changing — PMAT-980 (#2874). The remedy is the one check_roadmap_diff_additive.sh names. Pmat-Ticket: PMAT-3234 Co-Authored-By: Claude Opus 5 (1M context) --- docs/roadmaps/roadmap.yaml | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/roadmaps/roadmap.yaml b/docs/roadmaps/roadmap.yaml index 8dadba9a8f..9f5d86960a 100644 --- a/docs/roadmaps/roadmap.yaml +++ b/docs/roadmaps/roadmap.yaml @@ -17738,7 +17738,6 @@ roadmap: labels: - kind:code notes: null - - id: PMAT-3234 github_issue: null item_type: task