Repository navigation
ci: the mutants gate is vacuous for crates/** -- cargo mutants runs without --workspace, 0 mutants on every crate diff #4142
Description
The mutants job (ci.yml, blocking via gate) has never mutated a line of crate code.
CI runs cargo mutants --no-times --timeout 300 --in-place --in-diff pr.diff -- --lib at the workspace ROOT, and does not pass --workspace. .cargo/mutants.toml sets no workspace = true. Without --workspace, cargo-mutants mutates only the package in the cwd: the root facade, src/lib.rs + src/bin/, which is nearly empty. Every diff under crates/** therefore yields zero mutants, and the job passes.
Measured on real CI output (only emitted lines, not the step source the log echoes):
| PR | mutants run | output |
|---|---|---|
| #3772 | 35686965915 | INFO No mutants to filter · cargo-mutants exit: 0 · 0 mutants in diff. Pass. |
| #3737 | 35824474310 | No mutants to filter · 0 mutants in diff. Pass. |
| #3707 | 35629062866 | No mutants to filter · 0 mutants in diff. Pass. |
| #3706 | 35616869889 | No mutants to filter · 0 mutants in diff. Pass. |
Each of these PRs changes .rs files under crates/.
Locally, cargo-mutants 27.0.0, on the #4139 branch diff:
cargo mutants --in-diff pr.diff --list→No mutants to filter, 0 mutants.cargo mutants --workspace --in-diff pr.diff --list→ 269 mutants (lex.rs 136, discharge/mod.rs 107, cli discharge.rs 15, contract_walk.rs 10, lib.rs 1).
The fix is a workflow edit, so it needs the cop's check-in. Add --workspace, and keep -- --lib (a test-tool arg). Cost is the real question: the job has timeout-minutes: 60 and --timeout 300 per mutant, and a diff like #4139 would put hundreds of mutants on it. Options:
--workspacewith-j N(copies, not--in-place);- a per-PR mutant cap with a RED "too many to judge" (never a silent pass);
- nightly full-diff mutation.
Any of these beats the current state, which is a gate that cannot fail.
Found by aprender-6c [8b6b78] while preparing #4139 (EV-6a).
Activity
noahgift commentedon Sep 24, 2026
Measurements the cop asked for (lambda, 48 cores, cargo-mutants 27.0.0):
(1) Config route: no. Adding
workspace = trueto.cargo/mutants.tomlgivesunknown field 'workspace', and 0 mutants are listed. The only way to mutate crate code is the--workspaceCLI flag, so the fix is a ci.yml edit.(2) Cost. The diff is each PR's squash commit (
git diff <merge>^ <merge>); counts come fromcargo mutants --workspace --in-diff <diff> --list.--workspaceWall time, measured the way CI runs it (
--in-place --timeout 300 -- --lib, serial), on #3706:-p aprender-contracts --lib22 s);Serially, #3706 ≈ 30 min, #3707/#3737 ≈ 60–70 min (over
timeout-minutes: 60), #4139 ≈ 6.4 h, #3772 ≈ 13 h. With-j 8copies instead of--in-placeit is roughly ÷5–6 on this host. Even so, a #3772-sized diff is about 2.5 h.What it catches: the partial #3706 run already MISSED two mutants in merged code:
crates/aprender-contracts/src/ontology/extract/json.rs:193 replace positive_control -> bool with trueand:207 replace && with || in positive_control. The positive control of an extractor can be replaced bytrueand no lib test notices. The vacuous gate let that through.Caveat, measured: on this shared host, runs longer than about 2–4 minutes were
ERROR interruptedmid-test (3 of 4 attempts, both detached and in the foreground). No process-group signalling exists in aprender-contracts' source, so the cause is outside the tool and not diagnosed. A CI container should not see it, but that is unverified.test(mutants gate): a row on the REAL cargo-mutants 27 outcomes.json …
ci(guard): run the mutants gate's case table on every PR via guard-tree
noahgift commentedon Sep 24, 2026
triage 2026-09-24 (aprender-dd, issues owner; proposal #4159, approved by the cop aprender-cf): milestoned to 0.73.0 — unmilestoned tooling → carry to 0.73.0 (lightest of 0.71–0.74; its debt-ratchet slice). It was (no milestone). Open PR / live branch: br:fix/4142-mutants-workspace.
fix(mutants gate): keep the crates workspace-test excludes out of mut…
noahgift commentedon Sep 24, 2026
triage 2026-09-24 (aprender-dd; #4159, cop ruling): moved 0.73.0 → backlog. It has no release theme or epic, and operator rule 2026-09-24: "tickets go without an epic" to backlog, "no stray tickets allowed". It had been load-balanced into 0.73.0 as tooling carry earlier today.
fix(mutants-nightly): diff from the last SUCCESSFUL nightly, not "24 …
fix(mutants-nightly): read the last successful run over REST, not gh …
docs(audit): degraded quorum receipt for #4142 -- 3/3 PASS at c5991aa
test(pv discharge): kill the lex/name_lit/judge_labels survivors; bla…
revert(b2): take ONT-4c (doc contracts + measured-set ratchet) out of…
noahgift commentedon Sep 27, 2026
Operator ruling 2026-09-27 (relayed by the cop, verbatim): "exempt the car". Release car PRs (head
car/*) skip the on-PR diff-scoped mutants cap; their code is judged by mutants-nightly (uncapped, everything merged to main). Every other PR keeps MUTANTS_CAP=60. Trigger: #4429 car diff = 2203 mutants > 60, which cannot pass by construction.batch(B2): CI/infra/scripts/workflow fold — replayed onto main (REPLA…