Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/book-contracts.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,11 @@ on:
- "crates/aprender-core/src/**"
- "scripts/book-gate.sh"
- ".github/workflows/book-contracts.yml"
# #3676 rule 3 (scripts/check_workflow_path_filters.sh): a path filter is a
# claim that nothing outside these paths can break this gate; the nightly run
# re-checks the claim every day. Set ~5h early for the dispatch delay (#3292).
schedule:
- cron: '45 22 * * *'

permissions:
contents: read
Expand Down
5 changes: 5 additions & 0 deletions .github/workflows/book.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,11 @@ on:
- "crates/aprender-core/src/lib.rs"
- "crates/apr-cli/src/extended_commands.rs"
- "crates/apr-cli/src/tool_commands.rs"
# #3676 rule 3 (scripts/check_workflow_path_filters.sh): a path filter is a
# claim that nothing outside these paths can break this gate; the nightly run
# re-checks the claim every day. Set ~5h early for the dispatch delay (#3292).
schedule:
- cron: '30 22 * * *'

permissions:
contents: read
Expand Down
91 changes: 73 additions & 18 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ name: CI
on:
push:
branches: [main, master]
# #3676: sovereign-ci's `coverage_on: tag` (below) runs coverage only on a push of a
# v* tag, and its gate REQUIRES it to succeed there. A push filtered to branches never
# fires for a tag, so this is the reusable's second required edit.
tags: ['v*']
pull_request:
branches: [main, master]
# Merge queue: GitHub dispatches a `merge_group` event for each batch it forms.
Expand Down Expand Up @@ -63,7 +67,7 @@ jobs:
# 23916c00d adds an OPTIONAL `runs_on` input whose default is the label set
# this repo already gets, and 70e51ec04 swaps the digest on every job. No
# input this repo passes changes meaning.
uses: paiml/.github/.github/workflows/sovereign-ci.yml@70e51ec045d95983e9a3d02342d13283bf5f9dc1
uses: paiml/.github/.github/workflows/sovereign-ci.yml@f713290c86fcd70d6f26a0faab14e67e6713586f
with:
repo: ${{ github.event.repository.name }}
# Phase 3 pilot (heavy workload) — build-performance.md §7 Phase 3.
Expand Down Expand Up @@ -112,6 +116,20 @@ jobs:
# MECHANISM is live fleet-wide via sovereign-ci (#37); a single-crate repo is
# the natural first coverage pilot. aprender's blocking-quality pilot is the
# diff-scoped mutation gate below.
#
# #3676 (operator 2026-09-21: "YES, coverage on tags release only"): so this
# job gated nothing AND measured nothing (0 tests on the facade root), at the
# cost of an instrumented rebuild on every PR, merge group and push (237 s
# on main run 35563537942). `coverage_on: tag` runs it only on a push of a v*
# tag (the trigger above) and on workflow_dispatch; everywhere else the job is
# skipped and sovereign-ci's gate prints "coverage: NOT MEASURED", never a
# green. On a v* tag the gate REQUIRES it. The first spelling,
# `skip_coverage: true`, was read by that gate as a mandatory job that did not
# succeed, so `ci / gate` went red on every PR (#3707). Every consumer of the
# number keeps a producer, asserted by scripts/check_coverage_has_producers.sh
# (R1-R5): the pre-publish dogfood's `make coverage-check`, coverage-nightly's
# schedule, and this job on each release tag.
coverage_on: tag
secrets: inherit

# APR-MONO: Workspace-wide test (all 75 crates)
Expand Down Expand Up @@ -1164,8 +1182,14 @@ jobs:
# unwiring the §4.9.1 signature arm drops it to 23 passed, 6 broken), so a
# gate that silently stops discriminating now fails here rather than
# certifying a release later.
- name: Perf gate can still discriminate (APR-PERF-GATE-001, case table)
run: bash scripts/perf_gate.sh --selftest
#
# #3676: PATH-SCOPED now. The table's verdict changes only when perf_gate.sh
# or a file it reads changes, and it cost 405 s on every PR, merge group and
# push (main run 35563537942). scripts/check_perf_gate_selftest_scoped.sh --
# dispatched by guard_tree.sh --no-cargo above, no step here -- runs it when
# the diff vs origin/main touches its DERIVED scope (every $ROOT/ path it
# reads, followed through the scripts it calls) and says so on a SUMMARY
# line when it does not; guards-nightly.yml runs the table whole.
# PERF-007, the HOST half of §4.9.1. `perf_gate.sh --selftest` already
# runs this table, but check_guards_are_wired.sh cannot see one level of
# indirection and its remedy for an unwired guard is to wire it, not to
Expand Down Expand Up @@ -1365,12 +1389,12 @@ jobs:
- name: "Receipt-wiring guard: every rule it states, mutated (15/15 kill)"
run: bash scripts/mutate_pr_review_wiring_guard.sh
# APR-RELEASE-001 §6 R-2, WIRED (#3200 shipped the guard; this is its
# caller). check_pr_closes_issue.sh is a check_*.sh, so guard_tree.sh
# auto-discovers it and runs it BARE -- and bare runs only its own
# self-test, "self-test OK: 9 case(s)", exit 0. Nine green rows about a
# regex, on every PR, judging NO PR body. The 0.67.0 reconcile found 48
# merged PRs since the previous tag of which only 9 closed anything; that
# is what this guard exists to stop, and until now it had never read one.
# caller). This step is its ONLY runner: guard_tree.sh skips it as
# wired-with-args (it needs a body), so its case table ran in NO job until
# this step called `--self-test` beside `--body` (found by aprender-f0,
# 2026-09-21; the comment here used to claim guard_tree ran it bare). The
# 0.67.0 reconcile found 48 merged PRs since the previous tag of which only
# 9 closed anything; that is what this guard exists to stop.
#
# THE BODY ARRIVES THROUGH `env:`, NEVER INTERPOLATED INTO THE SCRIPT.
# `${{ }}` is substituted textually BEFORE bash sees the line, so a body
Expand All @@ -1394,6 +1418,7 @@ jobs:
PR_BODY: ${{ github.event.pull_request.body }}
run: |
set -euo pipefail
bash scripts/check_pr_closes_issue.sh --self-test
printf '%s' "$PR_BODY" > "$RUNNER_TEMP/pr-body.txt"
bash scripts/check_pr_closes_issue.sh --body "$RUNNER_TEMP/pr-body.txt"
# APR-RELEASE-001 §11.1, WIRED. The section shipped as prose in #3268 and
Expand Down Expand Up @@ -2178,9 +2203,13 @@ jobs:
# 3. the suites, judged against the measured ceiling
DR_ENV=(-e "SATD_BASELINE=$n")
dr "$GITHUB_WORKSPACE" cargo test -p aprender-core --features model-tests \
--test falsification_spec_v10_tests \
--test falsification_stress_tests \
--test falsification_gpu_state_tests
--test falsification_spec_v10_tests
# #3676: the PR path keeps ONLY falsification_spec_v10_tests -- it holds the
# SATD ratchet (F-CHECKLIST-005) measured against origin/main above, the one
# property here a diff can change, and the reason this step was KEPT at the
# 67-E3 merge (on the nightly HEAD == origin/main, so there the ratchet is
# vacuous). falsification_stress_tests and falsification_gpu_state_tests
# judge the whole tree, not the diff; guards-nightly.yml runs all three.
# 4. the satd class of the ratchet-semantics case table (rebuilds nothing)
DR_ENV=()
dr "$GITHUB_WORKSPACE" bash scripts/tests/ratchet_semantics_test.sh --class satd
Expand Down Expand Up @@ -3094,11 +3123,22 @@ jobs:
# main-branch pushes are not blocked by a job that intentionally
# did not run. Only an explicit `failure` blocks.
MUT="${{ needs.mutants.result }}"
if [ "$MUT" = "failure" ]; then
echo "mutants (diff-scoped mutation) failed: $MUT"
exit 1
fi
echo "mutants result: $MUT (success/skipped both pass)"
EVT="${{ github.event_name }}"
# --- GATE-MUTANTS-RULE-BEGIN (#3676; scripts/check_ci_gate_mutants_rule.sh runs this block) ---
# `skipped` is a pass ONLY when the skip is the job's own filter:
# mutants declares `if: github.event_name == 'pull_request'`, so on every
# other event it is skipped by design. On a pull_request it must have RUN
# and succeeded -- a diff with no .rs still succeeds (its steps skip, the
# job passes), so a pull_request `skipped` or `cancelled` means the job
# never ran, and the old rule ("success/skipped both pass") passed that.
case "$EVT:$MUT" in
pull_request:success) echo "mutants result: success on a pull_request" ;;
pull_request:*) echo "mutants on a pull_request must succeed; got '$MUT' -- a skip here is not the job's filter"; exit 1 ;;
*:skipped) echo "mutants result: skipped on $EVT (its filter: pull_request only)" ;;
*:success) echo "mutants result: success on $EVT" ;;
*) echo "mutants (diff-scoped mutation) on $EVT: $MUT"; exit 1 ;;
esac
# --- GATE-MUTANTS-RULE-END ---
# Receipt presence is NOT read here any more (PP-066 C0-5, PRQ-013,
# #2982). This job runs under the PR head's definition of ci.yml, so a
# receipt clause here was a check the PR under review could edit. The
Expand Down Expand Up @@ -3158,6 +3198,7 @@ jobs:
# Need history + base branch to compute the PR diff for --in-diff.
fetch-depth: 0
- name: Compute PR diff for mutation scoping
id: prdiff
# cargo-mutants --in-diff takes a unified diff and mutates ONLY the
# lines it adds/changes. We diff the PR head against the merge-base with
# the target branch so the scope is exactly "what this PR introduces".
Expand All @@ -3180,7 +3221,19 @@ jobs:
if [ ! -s pr.diff ]; then
echo "Empty diff — no code to mutate"
fi
# #3676: cargo-mutants mutates Rust. A diff with no .rs path has nothing to
# mutate, but used to pull the image and `cargo install cargo-mutants`
# anyway. --no-renames: a .rs file renamed away still counts.
git diff --no-renames --name-only "$MERGE_BASE"...HEAD > pr.files
if grep -qE '\.rs$' pr.files; then
echo "has_rs=true" >> "$GITHUB_OUTPUT"
echo "Rust paths in the diff: $(grep -cE '\.rs$' pr.files)"
else
echo "has_rs=false" >> "$GITHUB_OUTPUT"
echo "No .rs path in the diff ($(grep -c . pr.files || true) path(s)) -- nothing to mutate; the mutation steps skip and the job passes"
fi
- name: Pull sovereign-ci image (with retry + local-cache fallback)
if: steps.prdiff.outputs.has_rs == 'true'
# Same two-layer resilience as workspace-test — see that job for full context.
run: |
if docker image inspect "$IMAGE" > /dev/null 2>&1; then
Expand All @@ -3203,6 +3256,7 @@ jobs:
delay=$((delay + 6))
done
- name: Install cargo-mutants
if: steps.prdiff.outputs.has_rs == 'true'
run: |
docker run --rm \
-e CI -e GITHUB_ACTIONS -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_RUN_ID -e GITHUB_EVENT_NAME -e GITHUB_WORKFLOW \
Expand All @@ -3211,6 +3265,7 @@ jobs:
"$IMAGE" \
cargo install cargo-mutants --locked
- name: Run diff-scoped mutation testing (BLOCKING)
if: steps.prdiff.outputs.has_rs == 'true'
# No continue-on-error: a missed mutant on the PR diff fails the job,
# which fails `gate`, which blocks merge. --in-diff pr.diff restricts
# mutation to PR-touched lines. Empty diff → 0 mutants → clean pass.
Expand Down Expand Up @@ -3311,7 +3366,7 @@ jobs:
exit 0
'
- name: Upload mutation results
if: always()
if: always() && steps.prdiff.outputs.has_rs == 'true'
uses: actions/upload-artifact@v7
with:
name: mutation-results
Expand Down
7 changes: 6 additions & 1 deletion .github/workflows/coverage-nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,18 @@ on:
# `gh workflow run` (#3292); until then, re-measure before trusting.
- cron: '0 22 * * *' # 22:00 UTC -> lands ~03:00 UTC
workflow_dispatch: {}
# #3676: no v* tag trigger here. The release tag's coverage is owned by ci.yml
# (`coverage_on: tag`, which sovereign-ci's gate requires on a tag push), so a tag
# is measured once. scripts/check_coverage_has_producers.sh R3 keeps it that way.

permissions:
contents: read
issues: write # open/update the single coverage tracking issue

concurrency:
group: coverage-nightly
# Per ref (#3676): a tag's run and the nightly on main must not cancel each
# other. A second nightly on the same ref still supersedes the first.
group: coverage-nightly-${{ github.ref }}
cancel-in-progress: true

jobs:
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/guards-nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,8 @@ jobs:
run: bash scripts/check_hermetic_stdin_tests.sh
- name: Publish safety (binary/oversize files in published packages)
run: bash scripts/check_publish_safety.sh
- name: Perf gate can still discriminate (APR-PERF-GATE-001, case table)
run: bash scripts/perf_gate.sh --selftest
- name: Book rust examples compile
run: bash scripts/check_book_examples_compile.sh
- name: aprender-profile lib tests (was gated by nothing)
Expand Down
5 changes: 5 additions & 0 deletions .github/workflows/install-script.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,11 @@ on:
- ".github/workflows/install-script.yml"
- ".github/workflows/binary-release.yml"
- ".github/workflows/nightly.yml"
# #3676 rule 3 (scripts/check_workflow_path_filters.sh): a path filter is a
# claim that nothing outside these paths can break this gate; the nightly run
# re-checks the claim every day. Set ~5h early for the dispatch delay (#3292).
schedule:
- cron: '0 23 * * *'

permissions:
contents: read
Expand Down
17 changes: 17 additions & 0 deletions docs/roadmaps/entries/PMAT-3676.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
- id: PMAT-3676
github_issue: 3676
item_type: task
title: 'CI diet: aprender itself burns 99 intel runner-hours/day — coverage on tag/nightly only, path-filter book jobs, perf/model suites nightly, mutants only on .rs diffs'
status: planned
priority: critical
assigned_to: null
created: 2026-09-21T09:50:38Z
updated: 2026-09-21T09:50:38Z
spec: null
acceptance_criteria: []
phases: []
subtasks: []
estimated_effort: null
labels:
- kind:code
notes: 'ACCEPTANCE (quoted from issue #3676 "Proposal" + "done_when"; operator 08:38Z: "YES, coverage on tags release only"; Noah approved all five workflow items directly to aprender-b3): 1. Coverage (ci / coverage) on the release tag and the nightly only, never on pull_request or merge_group. COV_FLOOR enforcement moves to the tag gate (T-2) and the nightly, where a regression still blocks the release. 2. Book Chapter Examples and mdBook: path-filtered. They run when book/** or the crates a chapter imports change, and nightly always. 3. The perf gate case table and the model-tests falsification suites move to nightly. The PR path keeps a smoke row. 4. mutants runs only when the diff has .rs. 5. Every moved job still runs on the tag. check_workflow_path_filters.sh and the gate aggregator are updated so a skipped job is a pass only when the skip reason is the documented filter. done_when: intel runner-minutes per 24 h drop by >= 40% at equal PR throughput (measured after merge); the tag gate still runs coverage with COV_FLOOR, and a planted coverage drop fails the tag (control); a book-only PR runs the book jobs, a crate-only PR not imported by a chapter does not (both rows); gate accepts skipped only for jobs whose filter says skip, any other skipped state fails (mutation). COP ADDENDUM (aprender-04, relaying rmedia-88): before removing coverage from ci.yml, enumerate every consumer of the coverage number and prove each still has a producer, with a guard asserting the chain. (A), THE ci / gate FIX (cop ruling 2026-09-21 ~15:00Z; batch-1 #3707 left this row out): the first spelling, `skip_coverage: true` on sovereign-ci@70e51ec, made `ci / gate` RED on every PR: that reusable''s gate counts a skipped coverage job as a mandatory failure ("coverage did not succeed (result: skipped)"). This PR''s own CI was red on it (run 35585906113, cancelled unread) and so was batch-1. The fix: pin sovereign-ci to paiml/.github f713290c86fcd70d6f26a0faab14e67e6713586f (#74 `coverage_on` + #77), pass `coverage_on: tag`, add `push: tags: [v*]` to ci.yml (the reusable''s second required edit), and remove coverage-nightly''s v* tag trigger so each tag is measured once, by ci.yml. The same pin also brings #73''s roadmap-fragment-parity, which would have refused every aprender PR (aprender still commits roadmap.yaml). #77 keys it on the BASE tree: NOT-ACTIVE while roadmap.yaml is tracked, and it arms by itself when #3709 migrates. scripts/check_coverage_has_producers.sh now asserts R3 (the nightly runs on schedule, NOT on v* tags) and R5 (ci.yml has coverage_on: tag, no live skip_coverage: true, and on: push tags [v*]), with 16/16 self-test rows including "skip_coverage: true instead of coverage_on -> RED". On f7b25a6c6 the guard reports R3 and R5 RED; after the change it PASSES. Also in this change (cop, found by aprender-f0): ci.yml runs `check_pr_closes_issue.sh --self-test` beside `--body`, because guard_tree skips it as wired-with-args and its case table ran in no job, and the ci.yml comment that claimed otherwise is corrected. PROOF REQUIRED before re-folding: this PR''s own CI green, with the gate logging `coverage: NOT MEASURED` and `roadmap-fragment-parity: NOT-ACTIVE`.'
Loading
Loading