Repository navigation
Crate fan-out five-whys: 18/20 merged PRs trigger ALL under affected-set #5809
Description
Five-whys on crate fan-out (operator 13:25Z): why 18 of the last 20 merged PRs would run the whole suite under affected-set (#5802, parked at 64602c8).
Measured (workspace 85 crates / 838 test binaries; affected_set.sh over past diffs of origin/main first-parent)
- 18/20 merged PRs give ALL. Among the last 20 PRs that touch only
crates/**/*.rs, 6/20 still give ALL (CORE crate or build.rs) and the rest have median 64 crates / 804 binaries (p90 85 / 838). - Largest reverse-dependency closures (crates that depend on it, transitively):
aprender-contracts-macros57,aprender-contracts55 (CORE),aprender-present-core53,aprender-build-sha53. Outside CORE the top 3 areaprender-contracts-macros(57),aprender-present-core(53),aprender-build-sha(53).
Why ALL (PRs hit, by path; a PR can hit several)
contracts/.yaml 10, contracts/contracts.nt 7, contracts/witness/ 6, any non-.rs file or build.rs under crates/ 6, README.md 5, scripts/ 4, ci/sections.yml 4, .claude/ 3, docs/findings 3, ci/explicit-test-commands.d 2, Cargo.lock 2, docs/audits 2, one each: docs/specifications, docs/roadmaps, docs/proposals, Cargo.toml, evidence/, ci/shard-step-costs.tsv, contracts/*.ttl.
Which tests read those paths (crates with a .rs mentioning the path; counts are files)
- contracts/: 43 crates (aprender-core 226, apr-cli 118, aprender-serve 103). witness: 8 crates (test-lib, core, contracts).
- docs/: 25 crates (aprender-core 125, aprender-orchestrate 38, aprender-serve 35).
- README.md: 13 crates (orchestrate 18, train 11, core 7).
- scripts/: 20 crates (test-lib 13, serve 10, contracts 8). ci/ and sections.yml: 1 crate (orchestrate). Cargo.lock: 5. .claude: 3.
Whys
- Why ALL? Every non-
crates/**/*.rspath is ALL, because the tool has no map from a data path to the tests that read it. - Why no map? Reader edges are not in cargo metadata; readers are found by grep and are over-approximated (50 crates in READERS).
- Why so many readers? contracts/ and docs/ are read by 43 and 25 crates, often by tests that walk the whole directory.
- Why walk the whole directory? Contract and docs tests are written as global scans, not per-crate inputs.
- Why does that matter? The closure is already 48-64 crates, and most PRs also change a global-read path, so fan-out does not fall below the whole workspace.
Next (proposal, not done)
Give each global scan a declared input glob (a manifest the test reads), so a path maps to its tests; until then ALL stays correct and affected-set is not worth wiring for the merge path.
Activity
noahgift commentedon Oct 20, 2026
Countermeasure proposal (no code until ruled)
Proposal: contracts-reading tests become one shared test binary (
aprender-contract-scans), so a diff touching onlycontracts/runs that binary plus the crates it names, instead of ALL.Baseline (measured): PR run 39467426033 (3-shard split): shards 1535 s / 1364 s / 1191 s, so about 68 CPU-min and a 25.6 min wall for the shard stage (
x86-main2422 s runs in parallel). The 12-shard split (#5794) will change both numbers; re-measure after it lands.Predicted for a batch it applies to (assumption, not measured): build the binary from a warm cache (about 4-6 min) and run it (1-2 min), so about 8 CPU-min and a 6-8 min wall. That is about -58 CPU-min and -18 min per applicable batch.
How often it applies (measured): of the 18 ALL PRs in the last 20, only 1 is ALL purely from
contracts/paths, and 4 are ALL only fromcontracts/plusdocs//README.md/evidence/. So at most 4 of 20 batches (20%). Averaged over all batches: about -12 CPU-min and -3.6 min wall. The other 14 ALL PRs also touchscripts/,ci/sections.yml,crates/**non-.rsorCargo.lock, which this does not address.Honest read: this moves the average by about a fifth at most and needs tests in 43 crates relocated. It is the only candidate whose trigger paths are concentrated (contracts/*.yaml 10, .nt 7, witness 6). Cutting build-sha's 53-crate closure helps only crate-only PRs (a minority) and I would not do it first. Success test: after it lands, re-run the 20-PR measurement; ALL share should fall below 14/20, else revert (two-way door).
noahgift commentedon Oct 20, 2026
Numbers merged from #5814 (cargo metadata --no-deps, 84 members, transitive dependents, all dep kinds): contracts-macros 57, contracts 55, present-core 53, build-sha 53 (27 direct, build-dep), present-test-macros 48 (1 direct). Normal-deps only: macros 56, present-core 52, present-terminal 45, common 44, simulate 43, gpu 42.
New datum: churn over the last 90 days: aprender-contracts 64 commits, present-core 6, build-sha 3, contracts-macros 1, present-test-macros 0. So the CORE crate with the biggest closure is also the one that changes most. Not measured: how many of the 64 touched only engine internals.
fix(affected-set): ci/sections.yml stays ALL (it holds the CI job bod…
fix(affected-set): every error path prints ALL, never nothing (Refs #…
fix(affected-set): a --diff-from with .. fails closed to ALL instead …
ci: affected-set guards - no pipe into grep -q, setsid wraps the self…
ci: wrap each affected-readers guard invocation in setsid --wait (Refs …
ci: drop duplicate 815 fragment and dead docs/findings arm in affecte…
ci: affected-set guards move from the PR path to the guards-nightly L…
ci: affected-set LAB step gets its one-ticket opener (C324 rule 5) (R…
noahgift commentedon Oct 22, 2026
Tracked in #5510 (triage 2026-10-22).
noahgift commentedon Oct 22, 2026
tracked in #5963
ci(lab): the affected-set readers night opens its own ticket label, s…
noahgift commentedon Oct 23, 2026
Reopened by aprender-cop 00:40Z: its work is in flight on #6035 (epic/n-receipt-v13) and is not on main; EMG-7 needs an open PMAT-Ticket. Closed NOT_PLANNED 10-22 ~16:55Z during REDUCE-1022. Closes when #6035 merges.