Repository navigation
ci: run the parity repository-acceptance suite on the unfiltered leg - #2582
Merged
Merged
Conversation
`Tools/tests/test_parity_governance_acceptance.py` does not test the scanner. It asserts that the CHECKED-IN authority and baseline still reproduce from the CURRENT product source, so a product change is what invalidates it. It ran only in parity-governance.yml, whose filter names `Tools/parity_*`. So a change could move the declaration and call-site graph the ledger measures without the gate that measures it ever running. That is not hypothetical. Earlier today #2557 took the only production call to `primarySessionRestingHR` on both platforms, turning it test-only, and the two resulting findings went unrecorded. Seventeen green checks, none of them this one. `main` carried a red ratchet until #2581, and it surfaced on #2577, an outside contributor's PR that had caused none of it. parity-governance.yml's own header notes that shape had already happened four times, three of them on an outside contributor. Here, and not behind a filter naming product paths, for the reason the guard test below already gives about a different suite: the list would be whatever the assertions happen to read today, so a filter only moves the trap. #1691 is the precedent, where a `Tools/**` filter silenced a suite that reads `TodayScreen.kt` and `AppModel.swift`, and main carried it red through nine merges. Its own step and its own floor, so the core floor of 114 stays independent, which is the reasoning already written beside that floor. Cost, measured on this runner rather than estimated: 51 s for 49 tests, against roughly 25 s for the rest of the leg. I first assumed this suite cost 4 to 6 minutes and designed a separate parallel workflow around that; the 4 to 6 minutes is `parity_ratchet.py`, which checks out the base tree and derives twice. This suite derives once. The guard is extended to assert the unfiltered leg actually invokes the module. The failure mode is not someone arguing against this, it is someone dropping the step or hanging a filter off it later, which is exactly what #1691 was. Mutation-verified: pointing the step at a different module makes the guard fail. The assertion reads the dotted module path, which appears only in the invocation and not in the prose above it, so a comment cannot satisfy it. Verified: `tests.test_parity_governance_acceptance` plus `tests.test_parity_disposition_kinds`, 51 tests, pass. The workflow parses. Refs #2567, #2557, #1691
ryanbr
force-pushed
the
fix/2567-acceptance-runs-unfiltered
branch
from
September 29, 2026 00:54
c19e556 to
6e7d02d
Compare
This was referenced Sep 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tools/tests/test_parity_governance_acceptance.pydoes not test the scanner. It asserts that the checked-in authority and baseline still reproduce from the current product source, so a product change is what invalidates it.It ran only in
parity-governance.yml, whose filter namesTools/parity_*. So a change could move the declaration and call-site graph the ledger measures without the gate that measures it ever running.This is what happened today
#2557 took the only production call to
primarySessionRestingHRon both platforms, turning it test-only. The two resulting findings went unrecorded. It had seventeen green checks and none of them was this one.mainthen carried a red ratchet until #2581, and the failure surfaced on #2577, an outside contributor's PR that had caused none of it.parity-governance.yml's own header notes that this shape had already happened four times, three of them landing on an outside contributor.The header also names the cause exactly:
It chose a daily schedule as the mitigation, which bounds exposure to about a day. Today's exposure was about an hour and still reached a contributor.
Why on the unfiltered leg and not behind a product-path filter
Because the guard test in this very file already gives the reason, about a different suite:
#1691 is the precedent: a
Tools/**filter silenced a suite that readsTodayScreen.ktandAppModel.swift, andmaincarried it red through nine merges. The remedy this repository settled on is no filter, which is why the core leg has none. This suite reads product source for a living, so it belongs there.Its own step and its own floor, so the core floor of 114 stays independent. That is the reasoning already written beside that floor.
Cost, measured rather than estimated
About 134 s for 49 tests on the CI runner, against about 6 s for the core step beside it. Confirmed from this PR's own run, which is the first time the step executed: it collected 49 tests, matching its floor.
Two corrections worth recording, because both were mine and both were caught by measuring rather than by thinking harder.
I first assumed this suite cost 4 to 6 minutes and designed a separate parallel workflow around that. The 4 to 6 minutes is
parity_ratchet.py, which checks out the base tree and derives twice; this suite derives once. That removed the need for the more complicated design.Then I wrote 51 s into the workflow comment, which is what it takes on my workstation. On the runner it is about 134 s. In a CI file "on this runner" reads as the CI runner, so that number was misleading where it sat. It now says both, and says not to size it from a laptop.
The practical effect: on any product PR this runs in parallel with the Swift and Android legs, which are minutes long, so the wall-clock cost is roughly nothing. A docs-only PR pays the full 134 s.
The scanner's own multi-minute unit tests stay path-filtered in
parity-governance.yml, where the "repeated self-testing of the scanner" argument in its header genuinely applies.The guard
test_core_tools_filter_covers_every_governance_tool_pathnow asserts the unfiltered leg actually invokes the module.The failure mode is not someone arguing against this change. It is someone dropping the step, or hanging a filter off it, in six months. That is precisely what #1691 was.
Verification
tests.test_parity_governance_acceptanceandtests.test_parity_disposition_kinds: 51 tests, pass. The workflow parses.Refs #2567, #2557, #1691