Repository navigation
CI has no smart segmentation for docs-only PRs — full fat job + mandatory pr-review quorum run regardless of change type #4472
Description
What
CI has no "docs-only" classification that scales down to the checks a docs-only diff
actually needs. Every PR — including one that touches only README.md and two
book/src/**.md pages, zero Rust, zero config, zero CI/workflow files — pays for
close to the full verification surface: the fat x86-main job (mutation testing, the
sov.* matrix, the determinism raster, both pr-review-shadow/pr-review-sign),
mac-check, determinism, full book chapter-compile, and the mandatory pr-review
quorum receipt (which itself requires a real pmat index build plus a mandatory
cross-vendor agy consultation, exactly as heavy as it would be for a CUDA kernel
rewrite).
This is not a request to remove verification for docs PRs — a docs change can still
ship a false claim (docs/BEATS.md, README's own count-drift class of bug this repo
has hit before) or break a contract-gated book page. The request is that the
checks a diff actually cannot affect should not run for it, the same principle
scripts/ci_test_tier.sh already states for Rust compilation but does not apply
consistently, and does not apply at all to the pr-review quorum.
Concrete evidence: PR #4467
PR #4467 (this repo) changes exactly 3 files: README.md,
book/src/introduction.md, book/src/getting-started/installation.md — adding one
curl | sh install line to each, under existing headings. No Rust file, no
Cargo.toml, no CI workflow, no contract YAML is touched.
What ran anyway (gh pr checks 4467):
| check | result | note |
|---|---|---|
pr-review-quorum / present (Arm 4) |
FAIL — blocks merge | requires the full receipt: mandatory pmat index + mandatory cross-vendor agy consultation, same weight as any other PR |
x86-main (the fat job: sov.*, mutants, determinism[X64], pr-review-shadow, pr-review-sign, guard-tree, guard-cargo, vendored-schemas, workspace-test) |
ran in full | 240-minute timeout budget available to a 3-file markdown diff |
mac-check |
ran, 3m32s | |
determinism |
ran, 3m33s | |
Chapter Examples Compile |
ran, 17m45s | compiles every book chapter's example code |
Book Integration Tests |
ran, 3m31s |
scripts/ci_test_tier.sh on this exact diff (I ran it by hand):
tier=quick
crates=
targets=<136 tree-reader test targets across ~40 crates>
reason=pull_request: no workspace crate touched -> nothing to test;
plus 136 tree-reader target(s) from scripts/tree_reader_tests.txt
So the one piece of this repository's CI that does already reason about "no crate
was touched" still resolves to 136 test targets spread across ~40 crates, because
each one is registered as reading some part of the tree and the registry doesn't
distinguish "reads README.md" from "reads src/**.rs". That's a real, if partial,
cost reduction (it is not the ~80k-test full workspace run) — but it is not what a
docs-only diff should cost, and it says nothing about the mutation sweep, the sov.*
matrix, the determinism raster, or the pr-review-sign/pr-review-shadow jobs, which
run unconditionally regardless of tier.
What already exists (so this is an extension, not new infrastructure)
scripts/ci_test_tier.shalready has atier=nonefast path for diffs where every
touched path is underdocs/roadmaps/ordocs/audits/— but that allowlist does
not includeREADME.md,book/src/**, or top-level*.mdfiles, so this PR's
actual diff doesn't qualify for it.scripts/check_pr_review_receipt.sh's own--match-*predicates already correctly
identify a docs-only diff as not-triggering CUDA/CRUX/mutation (I verified this by
hand for PR ci: publish scripts/install.sh to paiml.com/apr/ via CloudFront/S3 #4461, structurally the same shape as docs: advertise curl | sh install via paiml.com/apr (README + book) #4467) — but "not-triggered" for
those three consultations does not reduce what's mandatory:pmatand the
cross-vendoragyrun are required on every PR regardless, at full weight.- Neither mechanism talks to the other. There is no single, shared "what kind of
change is this" classifier that bothci_test_tier.shand thepr-reviewguard
consult — each has independently reinvented a narrower slice of the same idea.
Proposed direction (not a design to implement as-is — needs its own review)
- One shared classifier, not two independent ones. A diff's classification
(docs / ci-config / source / mixed) should be computed once (e.g. as a
ci_test_tier.sh-adjacent script, or an extension of it) and consumed by both the
fat job's section list and the pr-review guard, rather than each maintaining its
own path patterns that can and do drift apart. - Widen the existing
tier=none/docs allowlist to cover the paths that are
actually docs by any reasonable definition —README.md,book/src/**,
docs/**broadly (not justroadmaps/+audits/), root-level*.md— while
still running the specific checks that verify doc content itself
(readme_contract,book_contracts, the per-pagepv validatecontract checks,
link/anchor checks) rather than the unrelated 136-target tree-reader sweep. - Make the fat job's non-source sections tier-aware.
mutants(mutation
testing) and thesov.*matrix have no meaningful signal to produce over a diff
that adds two lines of Markdown; they should be skippable (reported as
not-triggered, not silently green) for a docs classification, the same honesty
discipline the pr-review skill already applies to its own consultations (§3.0:
"not-triggered" is a distinct, visible state from "consulted, found nothing" —
apply that discipline to the fat job's sections too). - Give pr-review quorum a genuinely lighter receipt tier for a docs
classification — not zero verification, but proportionate:- Keep: the diff-patch-id binding + signature (integrity/provenance still
matters regardless of diff size), thepmatduplication/trigger checks (already
fast once the index is warm), and the existing CUDA/CRUX/mutation trigger checks. - Make optional: the mandatory cross-vendor
agyconsultation, which is
genuinely expensive (a disposable git-archive tree, several minutes of wall
time, real token cost) and adds ~nothing over a diff that adds an install
instruction to three Markdown files, where the CUDA/CRUX/mutation triggers
already independently confirm nothing user-facing, GPU-related, or
comparative-claim-bearing changed.
- Keep: the diff-patch-id binding + signature (integrity/provenance still
- Whatever a docs classification skips must be recorded as
not-triggeredwith a
named reason, never silently absent — this repo's ownpr-reviewskill already
states the principle that governs this (§3.0): "not-triggered" and "could not
consult" must never read the same as "consulted, found nothing," and the same
applies one level up, to entire jobs/sections being skipped for a diff
classification rather than for a failure.
Non-goals
- Not proposing docs PRs skip verification of doc content —
readme_contract,
book_contracts, and the per-page contract checks all caught real problems before
and should keep running (in fact should probably run for MORE doc paths than they
currently do narrowly, not fewer). - Not proposing a source-touching PR (however small) get a lighter tier — the
existingrule (ii)(root manifest touched -> full) and the crate-cap rule in
gate_touched_crates.shshould stay exactly as strict as they are. - Not proposing an implementation here — this crosses
ci_test_tier.sh,
ci/sections.yml's fat-job section list, andcontracts/pr-review-skill-v2.yaml's
§3 triggers, and deserves its own design pass (particularly picking the shared
classifier's exact path rules) before anyone writes the workflow/script changes.
Activity
wip(#4472): quorum fixes — docs/ is markdown-only, the claim scan fai…
merge origin/main into fold/pr-review-ci
noahgift commentedon Oct 4, 2026
PR #4717 implements this: one
change-classjob (scripts/ci_change_class.sh). On class=docs (every path is an A/M*.mdoutside .github/ that no crateinclude_str!s or names in a build.rs), x86-main runs only the doc-content sections (guard-tree, guard-cargo, vendored-schemas, pr-review-shadow/sign), and determinism and mac-check skip. workspace-test is unchanged.gateandci / gatestill run and name sov.gate and determinism-compare asnot-triggered: docs-only. Any code change, any non-PR event, and a failed classifier all mean the full set.Proof so far: classifier table 14/14, merge-ref emulation (README+book → docs, README+src/lib.rs → full), and verdict table 14/14 over the real gate scripts. The PR's own run is the docs+code arm. The live docs-only arm can only run after merge (ci.yml triggers on PRs to main), and I'll post those run IDs here. I'm leaving this issue open until then.
Agent: aprender-w4472
ci(#4472): docs only from a SUCCEEDED change-class; the verdict rule …
ci(#4472): kill the review's five surviving mutants — the event paths…
noahgift commentedon Oct 7, 2026
slipped_from: 0.70.2 -- carried to backlog at the 0.70.2 cut by the release autopilot (not must-carry, and not listed by the 0.71.0 epic)
noahgift commentedon Oct 14, 2026
P0 to P1 (2026-10-14 P0 triage): P0 now means it blocks the next release or carries the operator's own words on it; this issue does neither, so it stays on the 0.71.0 milestone at P1.
noahgift commentedon Oct 14, 2026
Folded into #3676 as a checklist row (TRIAGE-100, 2026-10-14): the open-ticket cap is 100, so this ticket is now tracked on that master and closed as not planned. Nothing is lost: reopen it when its work resumes, and an open PR for it reopens it.
Agent: aprender-clerk