From 6e7d02de77f2332bdab3bcc9dcf364f2db41b7fb Mon Sep 17 00:00:00 2001 From: Fanboynz Date: Tue, 29 Sep 2026 13:49:45 +1300 Subject: [PATCH] ci: run the parity repository-acceptance suite on the unfiltered leg `Tools/tests/test_parity_governance_acceptance.py` does not test the scanner. It asserts that the CHECKED-IN authority and baseline still reproduce from the CURRENT product source, so a product change is what invalidates it. It ran only in parity-governance.yml, whose filter names `Tools/parity_*`. So a change could move the declaration and call-site graph the ledger measures without the gate that measures it ever running. That is not hypothetical. Earlier today #2557 took the only production call to `primarySessionRestingHR` on both platforms, turning it test-only, and the two resulting findings went unrecorded. Seventeen green checks, none of them this one. `main` carried a red ratchet until #2581, and it surfaced on #2577, an outside contributor's PR that had caused none of it. parity-governance.yml's own header notes that shape had already happened four times, three of them on an outside contributor. Here, and not behind a filter naming product paths, for the reason the guard test below already gives about a different suite: the list would be whatever the assertions happen to read today, so a filter only moves the trap. #1691 is the precedent, where a `Tools/**` filter silenced a suite that reads `TodayScreen.kt` and `AppModel.swift`, and main carried it red through nine merges. Its own step and its own floor, so the core floor of 114 stays independent, which is the reasoning already written beside that floor. Cost, measured on this runner rather than estimated: 51 s for 49 tests, against roughly 25 s for the rest of the leg. I first assumed this suite cost 4 to 6 minutes and designed a separate parallel workflow around that; the 4 to 6 minutes is `parity_ratchet.py`, which checks out the base tree and derives twice. This suite derives once. The guard is extended to assert the unfiltered leg actually invokes the module. The failure mode is not someone arguing against this, it is someone dropping the step or hanging a filter off it later, which is exactly what #1691 was. Mutation-verified: pointing the step at a different module makes the guard fail. The assertion reads the dotted module path, which appears only in the invocation and not in the prose above it, so a comment cannot satisfy it. Verified: `tests.test_parity_governance_acceptance` plus `tests.test_parity_disposition_kinds`, 51 tests, pass. The workflow parses. Refs #2567, #2557, #1691 --- .github/workflows/tools-python.yml | 36 +++++++++++++++++++ .../test_parity_governance_acceptance.py | 12 +++++++ 2 files changed, 48 insertions(+) diff --git a/.github/workflows/tools-python.yml b/.github/workflows/tools-python.yml index 2b4868379f..2223e05327 100644 --- a/.github/workflows/tools-python.yml +++ b/.github/workflows/tools-python.yml @@ -81,6 +81,42 @@ jobs: fi working-directory: Tools/linux-capture + # The ONE parity suite that belongs on an unfiltered leg (#2567). + # + # `Tools/tests/test_parity_governance_acceptance.py` does not test the scanner. It asserts that the + # CHECKED-IN authority and baseline still reproduce from the CURRENT product source, so product + # changes are what invalidate it. It used to run only in parity-governance.yml, whose filter names + # `Tools/parity_*` — a filter that does not describe this suite's inputs, which is the same shape as + # the `test_home_i18n.py` trap in #1691 and is called out in that workflow's own header. + # + # It cost main a red ratchet on 2026-09-29: a change took the only production call to + # `primarySessionRestingHR` on both platforms, turning it test-only, and the two new findings went + # unrecorded because the change touched no `Tools/parity_*` path. Seventeen green checks, and the + # gate that governs call-site structure never ran. It surfaced on an outside contributor's PR that + # had caused none of it, which that header notes had already happened four times. + # + # Here rather than in a filter naming product paths, because that list is whatever the assertions + # happen to read today and would only move the trap. Its own step and its own floor so the core + # floor above stays independent, matching the reasoning already written there. + # + # COST, measured on this runner and not on a workstation: about 134 s for 49 tests, against about + # 6 s for the core step beside it. It derives the semantic authority once over both trees, which is + # the whole expense. Locally the same suite takes about 51 s, so do not size this from a laptop. + # It runs in parallel with the Swift and Android legs, which are minutes long on any product PR, so + # the wall-clock cost there is roughly nothing; a docs-only PR pays the full 134 s. The scanner's own + # multi-minute unit tests stay path-filtered in parity-governance.yml where they belong. + - name: Run the parity repository-acceptance suite + run: | + set -o pipefail + python3 -m unittest -v tests.test_parity_governance_acceptance 2>&1 | tee "$RUNNER_TEMP/acc.txt" + ran=$(grep -oE '^Ran [0-9]+ test' "$RUNNER_TEMP/acc.txt" | grep -oE '[0-9]+') + echo "collected ${ran:-0} tests" + if [ "${ran:-0}" -lt 49 ]; then + echo "::error::expected at least 49 acceptance tests, collected ${ran:-0} — discovery is broken, not the suite" + exit 1 + fi + working-directory: Tools + # Parity governance has its own path-filtered workflow because that suite scans the repository and # is intentionally much slower. Keep this core floor independent so growth in parity cannot hide # accidental deletion here; these six source-reference checks raise it to 114. diff --git a/Tools/tests/test_parity_governance_acceptance.py b/Tools/tests/test_parity_governance_acceptance.py index cc4d6a4abd..4108b9e098 100644 --- a/Tools/tests/test_parity_governance_acceptance.py +++ b/Tools/tests/test_parity_governance_acceptance.py @@ -130,6 +130,18 @@ def test_core_tools_filter_covers_every_governance_tool_path(self) -> None: self.assertEqual([], core_paths) self.assertIn("working-directory: Tools\n", core) self.assertNotIn("unittest discover -s tests", core) + # THIS suite runs on the unfiltered leg, and that is the point of #2567. + # + # It does not test the scanner. It asserts the checked-in authority and baseline still reproduce + # from the CURRENT product source, so a product change is what invalidates it. While it ran only + # in parity-governance.yml, whose filter names `Tools/parity_*`, a change could move the + # declaration and call-site graph the ledger measures without the gate running at all. That cost + # main a red ratchet on 2026-09-29 and surfaced on an outside contributor's PR. + # + # Asserted here rather than left to the workflow, because the failure mode is someone quietly + # dropping the step or hanging a filter off it later. That already happened once to the leg this + # test guards (#1691), which is why the leg is unfiltered in the first place. + self.assertIn("tests.test_parity_governance_acceptance", core) # The Windows leg keeps a filter, because it runs ONLY the Tools/linux-capture tests and those # read nothing outside their own package. That is the whole reason it could be split off: the # runner costs twice a Linux minute, and the cost argument is true for this job alone.