Skip to content

CI guardrails: standard as code, ci-auditor, reusable policy/rust/image/dry-run workflows (WI-3087) - #9

Merged
thedancingdeveloper merged 5 commits into
mainfrom
vogt-agent/ci-guardrails-standard
Nov 2, 2026
Merged

thedancingdeveloper merged 5 commits into
mainfrom
vogt-agent/ci-guardrails-standard

Conversation

@thedancingdeveloper

Copy link
Copy Markdown
Contributor

CI guardrails framework, part 1 of 2 (WI-3087). The ops side, which deploys the auditor and adds dashboards and alerts, is a separate PR on Forgejo indexarr/ops.

How this lands: github-policy isn't on graft, so this goes as a vogt-agent/ branch plus PR, per the WI-3087 brief. Rebase-merge it, so each commit lands as its own commit.

What's in it

  • ci-standard/rules.yaml has 24 rules, CI-01…CI-24. Each has a statement, severity, kind (static, settings or runtime), a machine check and a fix. It also holds the runner capability classes: ci-small, rust-heavy, docker, bigmem, publish, tailnet, windows, graft-dev, and hosted (public repos only, pinned images).
  • ci-standard/exemptions.yaml plus its JSON schema. Each exemption names one repo, one rule, a reason, an owner, and created/expires dates (at most 180 days apart). An expired entry suppresses nothing. The 4 runner-exceptions.json entries are carried over. The graft CI-10 exemption (squash queue) expires 2026-11-16.
  • ci-standard/README.md covers the rule table, auditor design, visibility gaps (admin:org) and the rollout plan with the per-repo migration order, with the 16 quick wins folded in.
  • auditor/ is ci-auditor (Python 3.13, mypy --strict, 53 pytest tests, ruff):
    • lint is the in-repo gate;
    • serve is the org-wide exporter;
    • validate checks rules and exemptions;
    • identity checks author, committer, paths and content, never commit bodies.
  • Reusable workflows, each versioned by tag:
    • policy.yml: actionlint (pinned, checksum-verified), ci-auditor lint and identity. Report-only until enforce: true.
    • rust-ci.yml: rust-cache with save-if main only, and nextest.
    • image.yml: BuildKit registry cache written from main only, and push by digest.
    • release-dry-run.yml: release preflight.
  • templates/ci-gate.yml (the standard CI shape, with ci as the only required context), policy-stub.yml and release-dry-run-stub.yml.
  • .github/workflows/ci-auditor.yml runs this repo's own checks on ubuntu-24.04. It builds the image in the merge queue and pushes ghcr.io/thedancingdeveloper-org/ci-auditor:<tag> on tags.
  • This repo now passes its own lint in enforce mode:
    • timeouts and concurrency on the legacy workflows;
    • no re-run on push to main;
    • three dated exemptions: the legacy gate's bare [self-hosted], self-pinning, and the image dry run.

The legacy runner-policy-reusable.yml and runner_policy.py are unchanged, apart from a timeout. Existing callers see no difference.

Verified

  • uv run ruff check, ruff format --check, mypy (strict) and pytest (53 passed) in auditor/.
  • actionlint -config-file ci-standard/actionlint.yaml is clean on every workflow here.
  • ci-auditor validate: 24 rules, 9 classes, 8 exemptions.
  • docker build -f auditor/Dockerfile . builds, and validate runs in the image.
  • Live read-only audit of the org plus fluent-gpui (serve --once): 80 repos in 391 s, 1,920 results, 608 failing.
    • Spot checks match the CI review: rustnzb and graft CI-13 (continue-on-error), vogt CI-02/CI-04/CI-19, graft CI-10 exempt (squash).

Not verified

  • policy.yml hasn't run in a caller yet; the first adopter is the test.
  • The ci-small runner may not have python3-venv. If it doesn't, the policy job's install step fails, and it should run on [self-hosted, rust-heavy] via the runs-on input until the image is updated.

After merge (oversight)

  1. Tag v1.0.0. The tag push builds and pushes ghcr.io/thedancingdeveloper-org/ci-auditor:v1.0.0.
  2. Make the GHCR package public, so the cluster can pull it without a pull secret.
  3. Merge the ops PR that deploys it.

🤖 Generated with Claude Code

thedancingdeveloper and others added 5 commits November 2, 2026 21:49
The CI guardrails standard as code: 24 rules (CI-01..CI-24) with a machine
check each, runner capability classes, and an exemptions file with a schema
(repo, rule, reason, owner, expiry; at most 180 days; expired entries
suppress nothing).

auditor/ is the one checker: `ci-auditor lint` runs the static rules in a
checkout (the policy gate), `ci-auditor serve` audits every org repository
hourly and exports Prometheus metrics and JSON finding lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…flows (WI-3087)

policy.yml runs actionlint, ci-auditor lint and the author/committer
identity check in the caller's repository, report-only until the caller
sets enforce. rust-ci.yml, image.yml and release-dry-run.yml are the
minimal shared builds the standard points at. ci-auditor.yml tests the
auditor, dogfoods the lint, and publishes the ci-auditor image on tags.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dupe findings

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…own standard (WI-3087)

ci-standard/README.md: the rule table, runner classes, exemption format,
auditor design (Python exporter, why not CronJob+pushgateway), what a
read-only token cannot see (admin:org gaps), and the report-then-enforce
rollout with the per-repo migration order and the 16 quick wins folded in.

This repository now passes its own lint in enforce mode: timeouts and
concurrency on the legacy workflows, no CI re-run on push to main, the
self-audit on ubuntu-24.04 (public repo), and three dated exemptions for
the legacy gate, self-pinning and the image dry run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…3087)

The legacy runner-policy gate (still the required check here) predates the
2026-11-02 decision allowing hosted runners in public repos, and its
line-based parser reads a reusable workflow's `runs-on` input key as a
runner selection. Dated exceptions until the legacy gate is retired;
ci-auditor CI-05 judges these files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thedancingdeveloper
thedancingdeveloper merged commit 23e8d36 into main Nov 2, 2026
4 of 5 checks passed
@thedancingdeveloper
thedancingdeveloper deleted the vogt-agent/ci-guardrails-standard branch November 2, 2026 22:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant