Skip to content

Document the tag-rules evaluator in the README - #504

Open
dstrodtman wants to merge 1 commit into
mainfrom
document-tag-rules-evaluator
Open

dstrodtman wants to merge 1 commit into
mainfrom
document-tag-rules-evaluator

Conversation

@dstrodtman

@dstrodtman dstrodtman commented Aug 24, 2026 •

Copy link
Copy Markdown

Summary

Adds a Tag Rules and Test Selection section to the README documenting how rayci maps a PR's changed files to tags and selects pipeline steps. Until now this behavior lived only in the source (raycicmd/tag_rule*.go, tag_rule_run.go, test_rules_cmd.go), so a contributor reasoning about which steps a change triggers had to read the Go to find out.

The section covers:

  • The algorithm — PR-only selection, the master / release-branch / non-PR short-circuit to ["*"], per-file first-match within each independent ruleset, and the union across rulesets and files.
  • Rule file grammar — tag declarations, directory-prefix / exact-file / glob matching (including the regex-escaping behavior that makes [abc] a literal rather than a character class), skip rules, and ; termination.
  • The test-rules conformance check — companion-file discovery and the exact-set-equality assertion semantics, and that it runs the same matcher as live per-PR selection.

Provenance

Doc-only change. Every claim is grounded in the raycicmd/ source. The tag-rule evaluator is byte-identical from v0.46.0 through v0.48.0 (the release Ray currently pins via .rayciversion) and up to current main, so the section describes the same algorithm on main and at the pinned version. No behavior change.

I'm a docs-team contributor writing this up as present-state reference; happy to adjust framing or depth to match how REEf wants the evaluator described. Defer to the maintainers on anything technical.

Add a "Tag Rules and Test Selection" section covering how rayci maps a
PR's changed files to tags and selects steps: the PR-only selection with
the master/non-PR short-circuit to every tag, per-file first-match within
each independent ruleset, the union across rulesets, the rule-file
grammar, and the exact-set-equality semantics of the test-rules
conformance check.

The behavior lived only in the source (raycicmd/tag_rule*.go,
tag_rule_run.go, test_rules_cmd.go); this gives it a discoverable
present-state home for contributors reasoning about which steps a change
triggers.

Signed-off-by: Douglas Strodtman <douglas@anyscale.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the README.md to document the tag rules and test selection mechanism in rayci, including details on the algorithm, rule file grammar, and the test-rules conformance check. The feedback suggests clarifying the wording regarding which builds trigger the short-circuit to run all tests, specifically pointing out that any non-PR build (including direct pushes to feature branches) will run the full pipeline.

Comment thread README.md
Comment on lines +22 to +25
Tag selection runs only for pull-request builds. On the master branch, a
`releases/*` branch, any non-PR build, or when `RAYCI_RUN_ALL_TESTS=1` is set,
`RunTagAnalysis` short-circuits to `["*"]` — every tag, the full pipeline. This
is why a merge to master runs everything regardless of what changed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The current wording suggests that only specific branches (master, releases/*) or general non-PR builds trigger the short-circuit. In reality, the code checks !isPullRequest(env), meaning any non-PR build (including direct pushes to feature branches) will run all tests. Clarifying this helps developers understand why pushing a feature branch directly runs the full pipeline.

Suggested change
Tag selection runs only for pull-request builds. On the master branch, a
`releases/*` branch, any non-PR build, or when `RAYCI_RUN_ALL_TESTS=1` is set,
`RunTagAnalysis` short-circuits to `["*"]` — every tag, the full pipeline. This
is why a merge to master runs everything regardless of what changed.
Tag selection runs only for pull-request builds. For any non-PR build (including post-merge builds on `master` or `releases/*`, as well as direct pushes to feature branches), or when `RAYCI_RUN_ALL_TESTS=1` is set, `RunTagAnalysis` short-circuits to `["*"]` — running every tag and the full pipeline. This is why a merge to master runs everything regardless of what changed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant