Rulesets: merge queue in a repository ruleset; urgent changes

CI.md

@@ -14,7 +14,7 @@ The pieces in this repository:
 | [`actions/ci-mode`](actions/ci-mode/action.yml) | Decides the tier of a run (below). |
 | [`scripts/check_ci_conformance.py`](scripts/check_ci_conformance.py) | Checks a repository's workflows against this page. |
 | [`.github/workflows/ci-conformance.yml`](.github/workflows/ci-conformance.yml) | Runs that check; the organization ruleset requires it. |
-| [`rulesets/`](rulesets/) and [`scripts/apply_rulesets.py`](scripts/apply_rulesets.py) | The organization ruleset and custom property, and the script that applies or checks them. |
+| [`rulesets/`](rulesets/) and [`scripts/apply_rulesets.py`](scripts/apply_rulesets.py) | The custom property, the organization ruleset, the merge-queue repository ruleset, and the script that applies or checks them. |
 
 ## Tiers
 
@@ -68,7 +68,7 @@ required `name (value)` check from it never arrives.
 
 ## The merge queue
 
-The `ci-merge-queue` ruleset configures it: squash merges, every entry must
+The `merge-queue` repository ruleset configures it: squash merges, every entry must
 pass (`ALLGREEN`), and at most 4 builds at once. That build limit is the CI
 quota: a queued pull request waits for a free build slot.
 
@@ -107,7 +107,26 @@ The queue squashes. A pull request that must land as a merge commit, such as
 an upstream merge, bypasses the queue: repository admins may merge pull
 requests directly. Run the full tier on its branch first with
 `gh workflow run`, wait for it to pass, then merge with
-`gh pr merge --merge --admin`. Nothing else bypasses the queue.
+`gh pr merge --merge --admin`. Apart from an incident (below), nothing else
+bypasses the queue.
+
+## Urgent changes
+
+An urgent fix jumps the queue: it goes to the front, is built next on top of
+the default branch, and still lands tested. Builds already in progress
+restart, because their base changed. Repository admins can jump the queue;
+a custom repository role can grant it to others. The web UI offers it when
+adding a pull request to the queue, and so does the API:
+
+```bash
+gh api graphql -F id="$(gh pr view <number> --json id --jq .id)" \
+  -f query='mutation($id:ID!){enqueuePullRequest(input:{pullRequestId:$id, jump:true}){mergeQueueEntry{position}}}'
+```
+
+Bypassing the queue (`gh pr merge --squash --admin`) lands a change without
+its full CI run. It is only for an incident in which CI itself is broken.
+The Change Management Policy requires green CI even for a hotfix, so such a
+bypass is a deviation and goes in the incident log.
 
 ## Caches
 
@@ -118,12 +137,21 @@ runs and the nightly run, and every run can read those.
 ## Enforcement
 
 A repository opts in by setting its `ci-tier` custom property to
-`merge-queue`. The `ci-merge-queue` organization ruleset then applies to its
-default branch: the merge queue, the `ci-gate` check, and the `ci-conformance`
-workflow from this repository. Other required checks stay in the
-repository's own rulesets.
+`merge-queue`, after which an organization owner runs
+`scripts/apply_rulesets.py`. Two rulesets then apply to its default branch:
+
+- the `ci-merge-queue` organization ruleset, which requires the `ci-gate`
+  check and the `ci-conformance` workflow from this repository;
+- the `merge-queue` repository ruleset, which the script adds to the
+  repository and which turns the queue on. GitHub's organization rulesets
+  don't accept a merge-queue rule.
+
+Other required checks stay in the repository's own rulesets. Both rulesets
+let repository admins merge pull requests directly, for merge commits and
+incidents. To leave the queue, unset the property and delete the
+repository's `merge-queue` ruleset.
 
-- Apply or check the ruleset and property: `scripts/apply_rulesets.py
+- Apply or check the property and rulesets: `scripts/apply_rulesets.py
   [--check]`. It needs an organization owner with the `admin:org` scope, and
   the Change Management Policy asks that the change be announced.
 - Check a repository locally:
@@ -160,3 +188,5 @@ Checked on 2026-10-22 in a temporary private repository:
 - The commit the queue pushes to `main` is the commit its merge group tested.
 - A repository admin merged a pull request around the queue as a merge
   commit with both parents.
+- GitHub's organization rulesets reject a `merge_queue` rule; repository
+  rulesets accept it.

rulesets/org/ci-merge-queue.json

@@ -0,0 +1,58 @@
+{
+  "name": "ci-merge-queue",
+  "target": "branch",
+  "enforcement": "active",
+  "bypass_actors": [
+    {
+      "actor_id": 5,
+      "actor_type": "RepositoryRole",
+      "bypass_mode": "pull_request"
+    }
+  ],
+  "conditions": {
+    "ref_name": {
+      "include": [
+        "~DEFAULT_BRANCH"
+      ],
+      "exclude": []
+    },
+    "repository_property": {
+      "include": [
+        {
+          "name": "ci-tier",
+          "property_values": [
+            "merge-queue"
+          ],
+          "source": "custom"
+        }
+      ],
+      "exclude": []
+    }
+  },
+  "rules": [
+    {
+      "type": "required_status_checks",
+      "parameters": {
+        "strict_required_status_checks_policy": false,
+        "required_status_checks": [
+          {
+            "context": "ci-gate",
+            "integration_id": 15368
+          }
+        ]
+      }
+    },
+    {
+      "type": "workflows",
+      "parameters": {
+        "workflows": [
+          {
+            "path": ".github/workflows/ci-conformance.yml",
+            "repository_id": 941333632,
+            "ref": "refs/tags/v1"
+          }
+        ]
+      }
+    }
+  ]
+}

rulesets/repo/merge-queue.json

@@ -0,0 +1,34 @@
+{
+  "name": "merge-queue",
+  "target": "branch",
+  "enforcement": "active",
+  "bypass_actors": [
+    {
+      "actor_id": 5,
+      "actor_type": "RepositoryRole",
+      "bypass_mode": "pull_request"
+    }
+  ],
+  "conditions": {
+    "ref_name": {
+      "include": [
+        "~DEFAULT_BRANCH"
+      ],
+      "exclude": []
+    }
+  },
+  "rules": [
+    {
+      "type": "merge_queue",
+      "parameters": {
+        "merge_method": "SQUASH",
+        "grouping_strategy": "ALLGREEN",
+        "max_entries_to_build": 4,
+        "max_entries_to_merge": 4,
+        "min_entries_to_merge": 1,
+        "min_entries_to_merge_wait_minutes": 5,
+        "check_response_timeout_minutes": 90
+      }
+    }
+  ]
+}

scripts/apply_rulesets.py

@@ -4,9 +4,17 @@
     scripts/apply_rulesets.py --check   # exit 1 when GitHub differs from rulesets/
     scripts/apply_rulesets.py           # create or update to match rulesets/
 
+- rulesets/ci-tier-property.json: the `ci-tier` custom property.
+- rulesets/org/*.json: organization rulesets.
+- rulesets/repo/*.json: repository rulesets, put on every repository whose
+  `ci-tier` is `merge-queue`. GitHub doesn't accept a merge-queue rule in an
+  organization ruleset, so the queue lives here.
+
 Needs the `gh` CLI signed in as an organization owner with the admin:org scope
 (`gh auth refresh -h github.com -s admin:org`). Org-setting changes follow the
-Change Management Policy: announce them when made.
+Change Management Policy: announce them when made. A repository that leaves
+the queue (its property unset) keeps its `merge-queue` ruleset until someone
+deletes it: `gh api -X DELETE repos/<owner>/<name>/rulesets/<id>`.
 """
 
 from __future__ import annotations
@@ -20,6 +28,7 @@
 
 ROOT = pathlib.Path(__file__).resolve().parent.parent / "rulesets"
 PROPERTY_FILE = ROOT / "ci-tier-property.json"
+OPT_IN_VALUE = "merge-queue"
 
 
 def gh(*args: str, payload: dict[str, Any] | None = None) -> Any:
@@ -35,6 +44,12 @@ def gh(*args: str, payload: dict[str, Any] | None = None) -> Any:
     return json.loads(result.stdout) if result.stdout.strip() else None
 
 
+def gh_list(endpoint: str) -> list[Any]:
+    # `--slurp` wraps the pages in one array; `--paginate` alone prints one
+    # JSON array per page, which is not one JSON document.
+    return [item for page in gh(endpoint, "--paginate", "--slurp") for item in page]
+
+
 def differences(want: Any, have: Any, where: str = "") -> list[str]:
     """List where `have` differs from `want`, ignoring keys `want` doesn't set."""
     if isinstance(want, dict):
@@ -51,8 +66,16 @@ def differences(want: Any, have: Any, where: str = "") -> list[str]:
     return [] if want == have else [f"{where}: expected {json.dumps(want)}, found {json.dumps(have)}"]
 
 
-def sync_property(org: str, check: bool) -> bool:
-    want = json.loads(PROPERTY_FILE.read_text())
+def opted_in(property_values: list[dict[str, Any]], property_name: str) -> list[str]:
+    """The repositories whose `property_name` is the opt-in value."""
+    return sorted(
+        repo["repository_name"]
+        for repo in property_values
+        if any(p["property_name"] == property_name and p["value"] == OPT_IN_VALUE for p in repo["properties"])
+    )
+
+
+def sync_property(org: str, want: dict[str, Any], check: bool) -> bool:
     have = {p["property_name"]: p for p in gh(f"orgs/{org}/properties/schema")}.get(want["property_name"])
     found = differences(want, have) if have else ["missing"]
     if not found:
@@ -67,28 +90,27 @@ def sync_property(org: str, check: bool) -> bool:
     return True
 
 
-def sync_ruleset(org: str, path: pathlib.Path, check: bool) -> bool:
-    want = json.loads(path.read_text())
-    # `--slurp` wraps the pages in one array; `--paginate` alone prints one
-    # JSON array per page, which is not one JSON document.
-    pages = gh(f"orgs/{org}/rulesets", "--paginate", "--slurp")
-    existing = {r["name"]: r["id"] for page in pages for r in page}
+def sync_ruleset(scope: str, want: dict[str, Any], check: bool) -> bool:
+    """Make the ruleset named `want["name"]` under `scope` (orgs/<org> or repos/<owner>/<name>) match `want`."""
+    label = f"{scope} ruleset {want['name']}"
+    # A repository's list includes the organization's rulesets unless told not to.
+    listing = f"{scope}/rulesets" + ("?includes_parents=false" if scope.startswith("repos/") else "")
+    existing = {r["name"]: r["id"] for r in gh_list(listing)}
     if want["name"] in existing:
-        have = gh(f"orgs/{org}/rulesets/{existing[want['name']]}")
-        found = differences(want, have)
+        found = differences(want, gh(f"{scope}/rulesets/{existing[want['name']]}"))
         if not found:
-            print(f"ruleset {want['name']}: up to date")
+            print(f"{label}: up to date")
             return True
-        print(f"ruleset {want['name']}: " + "; ".join(found))
+        print(f"{label}: " + "; ".join(found))
         if check:
             return False
-        gh("-X", "PUT", f"orgs/{org}/rulesets/{existing[want['name']]}", payload=want)
+        gh("-X", "PUT", f"{scope}/rulesets/{existing[want['name']]}", payload=want)
     else:
-        print(f"ruleset {want['name']}: missing")
+        print(f"{label}: missing")
         if check:
             return False
-        gh("-X", "POST", f"orgs/{org}/rulesets", payload=want)
-    print(f"ruleset {want['name']}: applied")
+        gh("-X", "POST", f"{scope}/rulesets", payload=want)
+    print(f"{label}: applied")
     return True
 
 
@@ -97,11 +119,16 @@ def main() -> int:
     parser.add_argument("--org", default="cocoindex-io")
     parser.add_argument("--check", action="store_true", help="report drift and exit 1 instead of applying")
     args = parser.parse_args()
-    # The property comes first: a ruleset's repository_property condition names it.
-    ok = sync_property(args.org, args.check)
-    for path in sorted(ROOT.glob("*.json")):
-        if path != PROPERTY_FILE:
-            ok = sync_ruleset(args.org, path, args.check) and ok
+    prop = json.loads(PROPERTY_FILE.read_text())
+    # The property comes first: the organization ruleset's condition names it.
+    ok = sync_property(args.org, prop, args.check)
+    for path in sorted((ROOT / "org").glob("*.json")):
+        ok = sync_ruleset(f"orgs/{args.org}", json.loads(path.read_text()), args.check) and ok
+    repos = opted_in(gh_list(f"orgs/{args.org}/properties/values"), prop["property_name"])
+    print(f"repositories with {prop['property_name']}={OPT_IN_VALUE}: {', '.join(repos) or 'none'}")
+    for repo in repos:
+        for path in sorted((ROOT / "repo").glob("*.json")):
+            ok = sync_ruleset(f"repos/{args.org}/{repo}", json.loads(path.read_text()), args.check) and ok
     return 0 if ok else 1
 
 

scripts/test_apply_rulesets.py

@@ -17,3 +17,16 @@ def test_rules_compare_by_type_and_report_changes() -> None:
 
 def test_missing_rule() -> None:
     assert differences({"rules": [{"type": "a"}]}, {"rules": [{"type": "b"}]})
+
+
+def test_opted_in_selects_the_opt_in_value() -> None:
+    from apply_rulesets import opted_in
+
+    values = [
+        {"repository_name": "b", "properties": [{"property_name": "ci-tier", "value": "merge-queue"}]},
+        {"repository_name": "a", "properties": [{"property_name": "ci-tier", "value": "merge-queue"},
+                                                {"property_name": "product", "value": "true"}]},
+        {"repository_name": "c", "properties": [{"property_name": "ci-tier", "value": None}]},
+        {"repository_name": "d", "properties": [{"property_name": "product", "value": "merge-queue"}]},
+    ]
+    assert opted_in(values, "ci-tier") == ["a", "b"]