Skip to content

ci: run the Tools suites that read product source on every change, and filter only the leg that does not - #2451

Merged
ryanbr merged 1 commit into
mainfrom
rb-pathgate
Sep 25, 2026
Merged

ryanbr merged 1 commit into
mainfrom
rb-pathgate

Conversation

@ryanbr

@ryanbr ryanbr commented Sep 25, 2026 •

Copy link
Copy Markdown
Owner

tools-python.yml carried a Tools/** path filter, so a change touching no Python never ran it.

The workflow already argued against its own filter

Its header says, verbatim:

NOT path-filtered, and that is deliberate. It used to fire only on Tools/**.py, on the premise that a suite can only break when its own source changes. That is false for test_home_i18n.py, which asserts on the CONTENTS of app and package files. A Swift-only commit broke it (#1671) and the filter meant this job never ran, so main sat red until a contributor touching Tools/ for something unrelated tripped over it in their own PR (#1691).

And the on: block directly below it re-added paths: ['Tools/**'].

The reason it was re-added was sound on its own terms: a Windows runner billed at twice a Linux minute should not run on the many pull requests that touch no Python. The remedy was not, because it silenced the Linux leg too, and the Linux leg is the one that reads product source.

It then cost the same thing a second time. On 2026-09-24 an Android-only commit changed the hero label padding test_home_i18n.py asserts on, this job did not run, and main carried a red suite through nine merges until a commit that happened to touch Tools/ finally tripped it. One of those nine was an outside contributor's.

The split

paths is workflow-level, not job-level, so one filter had to serve two jobs with different inputs. They are now two workflows:

workflow runs filter
tools-python.yml Tools/linux-capture decoders and the core Tools/ suites none: the core suites read product source, so no path list can describe its inputs. Ubuntu, ~25s, every PR and every push to main.
tools-python-windows.yml only the Tools/linux-capture tests, under legacy Windows console encodings Tools/linux-capture/**: those tests read nothing outside their own package, so the cost argument is true here, and answered here.

Job names are unchanged, linux-capture and windows-capture, so the check names are too.

The test that pinned the old filter

test_core_tools_filter_covers_every_governance_tool_path asserted the old paths exactly. Repinned to the new shape rather than dropped:

  • the core workflow must carry no path filter, and must still run the core Tools/ step
  • the Windows workflow must carry exactly the linux-capture filter, and must not run the core step

Reinstating Tools/** on the core leg now turns it red, verified by doing so. That is what both previous rounds of this lacked: the header said what should be true and nothing enforced it.

Parity governance is deliberately untouched

Its filter is intentional, a test asserts it excludes product globs, and it already carries a daily schedule on main for exactly the authority staleness a path filter cannot see. I had earlier described it as silently skipped; that was wrong, and the schedule is why.

The documentation was already describing the state this restores

Two contributor-facing docs asserted the unfiltered behaviour while the filter was in place:

  • CONTRIBUTING.md check table: linux-capture … every PR
  • docs/CONTRIBUTING.md: source-hygiene, tools-python and i18n-coverage have no path filter, so all three run on everything

Both were false under the filter. That is worse than the header contradiction, because contributors were told a gate covered their change while it was silently skipped. Both are true again now, with no edit needed.

AGENTS.md was the one that was accurate ("active, path-filtered") and is updated, plus a row for the new Windows workflow its table never carried. CONTRIBUTING.md's table gains the windows-capture check it never listed either. Yesterday a stale AGENTS.md workflow row sent three contributors chasing the wrong thing, which is why these were checked rather than left.

Testing

Core Tools suite 130 tests, no failures. Governance acceptance 49 tests, no failures. The repin mutation-tested by reinstating the filter.

…d filter only the leg that does not

`tools-python.yml` carried a `Tools/**` path filter, so a change touching no Python never ran it.
Its own header already argued against that, in detail: the core Tools/ suites assert on the CONTENTS
of app and package files, `test_home_i18n.py` reads TodayScreen.kt, AppModel.swift and the xcstrings
catalogues, and a Swift-only commit had already broken it once while the filter kept the job from
running, leaving main red until an unrelated contributor touching Tools/ tripped over it in their own
PR (#1691).

The filter was reinstated underneath that paragraph anyway, for a reason that was sound on its own
terms: a Windows runner billed at twice a Linux minute should not run on the many pull requests that
touch no Python.

It then cost the same thing a second time. On 2026-09-24 an Android-only commit changed the hero
label padding that `test_home_i18n.py` asserts on, this job did not run, and main carried a red suite
through nine merges until a commit that happened to touch Tools/ finally tripped it. One of those
nine was an outside contributor's.

So the two legs are split rather than the argument had a third time:

  tools-python.yml          UNFILTERED. Runs the Tools/linux-capture decoders AND the core Tools/
                            suites, and the core suites read product source, so no path list can
                            describe its inputs. Ubuntu, about 25 seconds, every PR and every push.

  tools-python-windows.yml  FILTERED on Tools/linux-capture/**. Runs ONLY the linux-capture tests,
                            a second time under legacy Windows console encodings, and those read
                            nothing outside their own package. The cost argument is true here and
                            is answered here.

Job names are unchanged, `linux-capture` and `windows-capture`, so the check names are too.

`test_core_tools_filter_covers_every_governance_tool_path` pinned the old filter exactly, so it is
repinned to the new shape rather than dropped: the core workflow must carry NO path filter and must
still run the core Tools/ step, and the Windows workflow must carry exactly the linux-capture filter
and must NOT run the core step. Reinstating `Tools/**` on the core leg now reds it, which is what
the last two rounds of this lacked.

Parity governance is left alone. Its filter is deliberate, there is a test asserting it excludes
product globs, and it already carries a daily schedule on main for exactly the staleness a filter
cannot see.

Three documents described this and two of them were already describing the state this restores, not
the state that shipped. CONTRIBUTING.md's check table said `linux-capture` runs on "every PR", and
docs/CONTRIBUTING.md said `source-hygiene`, `tools-python` and `i18n-coverage` "have no path filter,
so all three run on everything". Both were false under the filter: contributors were told a gate
covered them while it was silent. They are true again now.

AGENTS.md was the one that was accurate, "active, path-filtered", and it is updated, along with a
row for the new Windows workflow that its table had never carried. CONTRIBUTING.md's table gains the
`windows-capture` check it had never listed either.

Core Tools suite 130 tests, no failures. Governance acceptance 49 tests, no failures.
@ryanbr
ryanbr merged commit 974cda4 into main Sep 25, 2026
5 checks passed
@ryanbr
ryanbr deleted the rb-pathgate branch September 25, 2026 04:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant