Coverage owes 0.69.1: 87.82% vs floor 88, and COVERAGE_EXCLUDE_REGEX has described a different tree since April

Coverage was DEFERRED for the 0.69.1 pre-publish dogfood by operator ruling, 2026-09-22. This row is the obligation. The release did not pass coverage and does not claim to.

What was deferred

824853 / 939270 = 87.82%     enforced floor COV_FLOOR := 88   (Makefile:508)

Red in Coverage Nightly for 8 consecutive runs back to 2026-09-15 (latest 35672739017, a9502d992), independent of the 0.69.1 batch. dogfood.sh:613 runs gate coverage make coverage-check, gate sets FAILED=1, and VERDICT is GO only when FAILED -eq 0 — so without the deferral this is a hard publish stop via preflight R5.

Two pieces of work, and they are not the same piece

1. The measurement has described a different tree since April

COVERAGE_EXCLUDE_REGEX (Makefile:492) was last touched 2026-02-08. APR-MONO Phase 2c moved realizar to crates/aprender-serve, entrenar to crates/aprender-train, trueno to crates/aprender-compute on 2026-04-06. Nothing updated the patterns (measured by @aprender-d8):

pattern	intends to exclude	actually matches
`realizar/`	the inference engine	2 of 2194 `.rs` in `crates/aprender-serve`, plus 7 unrelated in `aprender-train/src/ecosystem/realizar/`
`entrenar/`	the training engine	0 files. The pattern is dead.
`trueno`	the SIMD/GPU crate	0 of 580 `.rs` in `crates/aprender-compute` — but 73 in `aprender-zram`, plus orchestrate/profile/viz/core

Wrong in both directions: ~3900 files it means to exclude are measured, 88 it never meant to touch are excluded.

And the denominator contains code the target is configured never to execute. Ten of the top 28 entries in the nightly gap report (#2307) are crates/aprender-compute/src/backends/gpu/device/... at 0.0% — the crate the dead trueno pattern is supposed to remove — while Makefile:628 passes --skip cuda --skip gpu_. That code is 0% by construction, on every host, forever, including hosts with a GPU. It is not a coverage regression; it is a measurement artifact.

2. The real gap, once the measurement is honest

Unknown until (1) is computed. The nominal gap is ~1,700 lines. Repairing the regex to its documented intent changes the denominator and must be done as its own change, with the before/after LH/LF and excluded-file counts recorded, not folded into feature work and not done during a release — it moves a release gate's number in the direction that helps whoever is moving it.

Also owed: the coupling that hides the number

make coverage runs cargo llvm-cov test --workspace --lib, which includes aprender-contracts --lib, which contains lint::tests::lint_passes_on_real_contracts — a test that reads the live contracts/ tree and the on-disk receipts. So the coverage gate is coupled to release-receipt state. During 0.69.1 it died in the instrumented test run on the ladder-green shape violation, never reached its threshold check, never wrote an lcov.info, and reported [FAIL] coverage with no percentage at all — pointing at the wrong subsystem. Nobody would predict that flipping a ladder rung to required: true breaks coverage.

It is invisible for a second reason: dogfood.sh:196 picks the row note by grepping the output for the first line matching error/fail/warning/denied/regression, so a build warning becomes the visible note for the whole row. The same picker mislabelled the contracts row in the same rehearsal.

Done when

COVERAGE_EXCLUDE_REGEX repaired to its documented intent, with before/after LH, LF, percentage and excluded-file count in the commit message
the GPU-crate contradiction resolved: either the crate is excluded (it is skipped) or the tests run on a GPU host — not both
COV_FLOOR reset to the newly measured value as a separate commit, so the floor move is reviewable apart from the measurement fix
the receipt-state coupling broken or documented: a coverage gate that cannot measure reports "did not measure", never a coverage verdict
coverage removed from PREPUBLISH_DEFERRABLE once it can pass on its merits
CLAUDE.md's 88.78% (dated 2026-07-29) re-derived — it has not been re-measured since, because the gate that would have re-measured it has been failing for a week and a half

Refs #2307, #2333, #3835

🤖 Generated with Claude Code

release: coverage is DEFERRABLE in pre-publish, recorded as OWED (#3839)

Revert "release: coverage is DEFERRABLE in pre-publish, recorded as O…

Direction changed: this is now the FIX, not the obligation

Operator, 2026-09-22: "since this is so trivial to fix, actually fix coverage." The deferral is reverted (f3fba7c1f) — PREPUBLISH_DEFERRABLE is back to its two original rows and dogfood's coverage row is a hard gate again. 0.69.1 will pass coverage or not ship.

Static analysis, which kills the option this issue originally implied

The issue above framed the work as "repair COVERAGE_EXCLUDE_REGEX to its documented intent" — realizar/ → crates/aprender-serve/, entrenar/ → crates/aprender-train/, trueno → crates/aprender-compute/. Measured over git ls-files '*.rs':

total tracked .rs                      10445
excluded by CURRENT regex               1105
excluded by "repair the paths"          4960     <-- 47% of the tree
excluded by "delete the three"          1010
excluded by current + gpu/device        1119

  trueno                    ->    88 files   (aprender-zram, orchestrate, profile, viz, core)
  realizar/                 ->     7
  entrenar/                 ->     0          DEAD
  crates/aprender-compute/  ->   586
  crates/aprender-serve/    ->  2236
  crates/aprender-train/    ->  1149
  gpu/device/               ->    14

"Repairing the paths" would exclude the inference engine, the training engine and the SIMD crate from aprender's own coverage number — 4,960 of 10,445 files. A figure that measures everything except serve, train and compute is not a coverage figure.

So the premise in the issue body is wrong, and the correction matters: those patterns were not stale references in need of repair. They were exclusions of separate repositories that had their own CI coverage. APR-MONO did not break them — it made them obsolete. @aprender-d8's reading was the right one and mine was not.

What follows, and it is uncomfortable

The honest fix makes the number worse. Deleting the three obsolete patterns newly measures 95 files (88 trueno + 7 realizar/) that are silently excluded today and were never meant to be. So the headline finding here is under-measurement, not over-measurement: the tree has been quietly not counting 88 files in aprender-zram and elsewhere because a pattern meant for a different repository happens to match their paths.

The only candidate that can close a gap and still stand on its own logic is narrow:

Makefile:628 passes --skip cuda --skip gpu_, so the 14 files under crates/aprender-compute/src/backends/gpu/device/ cannot be executed by this target on any host — including the release host, which has a GPU. A denominator cannot be held accountable for code the measurement is configured to refuse to run.

That argument does not mention the floor, which is the test of whether it is a real argument.

Open, pending one lcov

There is no lcov.info on the box, so this needs one instrumented run (with lint_passes_on_real_contracts skipped and named, per the receipt-state coupling described above). From that single file every candidate is arithmetic:

LH/LF for the 14 gpu/device/ files — if their uncovered count is ≥ ~1,705, the narrow exclusion closes the gap alone
LH/LF for the 95 files the deletion newly measures — so any commit can state that the same change both excludes never-executable code and adds previously-hidden code
whether the cuda/gpu_ skip is genuinely total, i.e. whether any non-skipped test reaches that code (if it does, "cannot execute" is false and the narrow exclusion is wrong)
priced but not recommended: dropping the skips for the release-gate invocation, since the release host has a GPU. It raises the real number instead of subtracting, but a gate whose value depends on the host reads differently in CI than in the release.

If the narrow exclusion plus the deletion does not clear 88, there is no measurement fix and the choice is tests or a floor move — with the table, not a preference.

Commits will be split so the measurement fix is reviewable apart from any floor move.

Revert "Revert "release: coverage is DEFERRABLE in pre-publish, recor…

The dominant cause is not missing tests — the gate skips 1,207 of its own tests by name substring

Measured by @aprender-d8 with cargo test -p <crate> --lib -- --list against Makefile:628's exact skip list:

aprender-serve    16044 -> 15267   removed 777
aprender-compute   3518 ->  3388   removed 130
aprender-core     14284 -> 14158   removed 126
aprender-train     7646 ->  7472   removed 174
                                   REMOVED 1207

libtest --skip is a substring match on the full test path, so:

--skip falsification eats the entire module prune::falsification_checklist — all 6 of get_checklist's tests. That function is [TOP 10] Implement K-Nearest Neighbors (kNN) Classifier #23 on this issue's own gap list at 0.0% cov, 153 uncovered lines, grade A. It is thoroughly tested code the measurement refuses to execute.
--skip gpu_ eats every wgpu_* test (35) because wgpu_ contains gpu_, plus roughly 1,100 tests whose names merely mention GPU while needing no GPU at all — including force_cpu_preloads_no_accelerator_for_any_format, which is apr chat --no-gpu still uploads the GGUF to CUDA (VRAM held outside the GPU lock), and apr chat reports no machine-readable backend #3794's own pure-CPU predicate test.

Why this has been silently steering the roadmap

The gap report cannot distinguish "has no tests" from "has tests that were skipped by name." Both present as uncovered lines in the instrumented run. So #2307 and pmat query --coverage-gaps have been ranking well-tested modules at the top of the coverage priority list, and anyone working that list top-down does invisible no-op work: adding a test to a skipped module changes nothing, and adding one in a differently-named module raises the number while covering code that was already tested.

Grade A with 0.0% coverage is the tell. Grade A code with no tests is rare; grade A code whose tests are skipped is what this is.

Two directions, one number

Combined with the stale COVERAGE_EXCLUDE_REGEX above, the measurement is wrong in both directions at once and neither direction is small:

	effect
`COVERAGE_EXCLUDE_REGEX` (2026-02-08, pre-APR-MONO)	~3900 files it means to exclude are measured; 88 files it never meant to touch are excluded
`--skip` substring patterns	1,207 tests the suite contains are never run, so the code they cover reads as uncovered

87.82% is therefore not "coverage". It is coverage-of-the-non-skipped-subset, over a denominator assembled by patterns that describe the pre-April tree.

Added to done-when

--skip patterns match intent, not substrings: anchored paths, --skip-exact, or rename the offenders. gpu_ must not eat wgpu_, and falsification must not eat a module that happens to be named for the technique.
Separate "no tests" from "tests skipped" in the gap report, so the priority list stops ranking tested code first. Until then pmat query --coverage-gaps output on this repo needs the skip list applied before it is actionable.
Any coverage figure published anywhere states the skipped-test count alongside it.

Still 0.70.0 per the operator's ruling ("minimally add more tests … and push more coverage to .7"). This finding argues for that ruling: the fix is patterns and reporting, not tests, and it is not release-night work.

docs(audit): #3839 receipt — +577 lines, and why the gap report misdi…

fix(dogfood,gate): three defects in the release protocol's own report…

test(pv): pin kaizen's two copies of the grade ladder to each other (#…

test(train): a blocked plan must never reach the apply phase (#3839)

test(present-yaml): the transform executor had no tests at all (#3839)

docs(audit): #3839 receipt — +577 lines, and why the gap report misdi…

fix(dogfood): keep the coverage log, and correct the record on the co…

0.70.0 brief — two things that cost real time to find

Written up here because both are invisible until you trip over them, and both point at the wrong remedy when you do.

1. The coverage gate refuses to run its own tests

Makefile's coverage target passes nineteen --skip patterns to the instrumented run. libtest's --skip is a substring match on the FULL test path, not on the test name and not anchored — so a pattern matches module names too, and eats every test in any module whose path contains it.

Measured with --list against the gate's exact skip list:

crate	in the binary	survive the skips	removed
aprender-serve	16044	15267	777
aprender-train	7646	7472	174
aprender-compute	3518	3388	130
aprender-core	14284	14158	126
aprender-orchestrate	6554	5955	599
aprender-simulate	1853	1751	102
			1908 over these six

Two mechanisms, both accidental:

--skip falsification eats whole modules. prune::falsification_checklist has six tests and zero survive. That is why get_checklist reads 0.0% coverage, 153 uncovered lines, grade A in Nightly coverage report #2307.
--skip gpu_ matches inside wgpu_ (35 tests), and matches ~1,100 tests whose names merely mention GPU reporting — compute_class_is_cpu_without_a_gpu_feature, force_cpu_preloads_no_accelerator_for_any_format (apr chat --no-gpu still uploads the GGUF to CUDA (VRAM held outside the GPU lock), and apr chat reports no machine-readable backend #3794's, a pure CPU predicate). None needs a GPU.

17,119 uncovered lines — 15% of all uncovered lines in the measured tree — sit in modules whose path matches a skip pattern. The gap to the 88% floor was ~1,705. Six files in the top 18 by uncovered lines are of this kind and carry 271 tests between them:

829 uncov  0%  aprender-test-lib/src/presentar/falsification.rs              15 tests
884 uncov  0%  aprender-test-lib/src/gpu_pixels/mod.rs                       82 tests
796 uncov  0%  aprender-orchestrate/src/falsification/performance_waste.rs   48 tests
771 uncov  0%  aprender-simulate/src/falsification/mod.rs                    44 tests
771 uncov  0%  aprender-orchestrate/src/falsification/numerical_reproducibility.rs  41 tests
691 uncov  0%  aprender-test-cli/src/load_testing.rs                         41 tests

Why this is worse than the number

pmat --coverage-gaps and #2307 cannot distinguish "no tests" from "tests skipped by name". Both present as uncovered lines. So the published gap report has been ranking thoroughly-tested modules at the top of the coverage priority list, and anyone working it top-down does invisible no-op work. It directed the first target assigned in the 0.69.1 coverage push.

The tell: grade A or B code at 0.0% coverage is nearly always tested code the measurement will not execute. Untested code rarely grades A. Check before writing a line:

cargo test -p <crate> --lib -- --list | grep <module>
cargo test -p <crate> --lib -- <the gate's skips> --list | grep <module>

Consequence when WRITING tests

A new test whose name or module path contains a skip substring is invisible to the gate. Forbidden anywhere in the path: gpu_, cuda, falsification, slow, heavy, chaos, disconnect, load_test, h12_, j2_, benchmark_parity, qwen2_generation, qwen2_golden, qwen2_weight, spec_checklist_w, spec_checklist_u, verify_audio, g9_roofline, prop_gbm_expected_value.

This is easy to walk into. #3826 nearly shipped a row called a_gpu_run_that_succeeded…, and #3821's whole module had to be named subject_verdict_tests rather than anything descriptive, because the natural names all contained gpu_. Verify, do not assume — count rows with and without the skip list.

The fix is not tests

Skip patterns that match intent rather than substrings: anchored names, --skip-exact, or renaming the offenders. Whatever is chosen, the re-measured floor should be set from the result, and #2307's ranking is not trustworthy until it is.

2. Rule 7 and the sigma ratchet are in tension

CLAUDE.md Rule 7 says coverage and contracts co-evolve: every coverage commit adds or strengthens contract obligations. The sigma gate says:

formal_prose rose 1464 -> 1469: a `formal:` entry carrying no symbol Σ declares
was added. The baseline in contracts/lint-baseline.json is shrink-only

So every obligation added in the natural way fails the gate, because a formal: written in prose increments a shrink-only counter. Adding three obligations to two contracts tripped it immediately.

The error message names the wrong remedy. It points at contracts/lint-baseline.json — which, being shrink-only, is the one file you must not touch. Reaching for it is exactly the "bulk-refresh the baseline so the number goes down" move the ratchet exists to prevent.

The right fix is to write formal: entries in Σ-declared glyphs (contracts/ontology.yaml §symbols): ∀ ∃ ∧ ∨ ¬ ⇒ ⟹ → ≡ ≈ ≠ ≥ ≤ ∈ ∉ ⊆ ∪ ∩ ∅ · × ‖ Σ ε plus len shape isFinite isNaN modifies preserves max min abs exp log sqrt sum mod. For example:

formal: '∀ P: fromStr(toJson(P)) ≡ P ∧ fromStr(toYaml(P)) ≡ P'
formal: 'checkCounts(P) = (a, b, c) ⟹ a + b + c ≡ len(P.preFlight)'

And the gate passed one entry by luck. Of six new formal: entries, five counted as prose and one did not — because it happened to contain len(), and len is a declared Σ function. Nothing about that entry was more formal than the others. A rule whose green depends on an incidental substring is worth looking at.

Suggested for 0.70.0

Either make the sigma failure name the remedy (glyphs and where they are declared) instead of the baseline, or give Rule 7 a sanctioned shape for a new obligation so the two rules stop pulling in opposite directions. Today the path of least resistance from a correct Rule 7 instinct is a baseline edit.

Both found while doing #3839's minimal-test push. Neither is a coverage problem, which is the point: the number was never the thing to fix.

fix(ci): three guard-tree failures — a ratchet, a stale ledger, and a…

fix(dogfood): keep every gate log and pick anchored failure notes; la…

aprender-a2 (claimed 21:49Z): state, and one scope decision needed before the regex repair

Landed vs pending. PMAT-3839-coverage-skip-substring is superseded: #4023 (fix/coverage-nightly-port-3839) carries the exact skips and --ignore-run-fail, scopes every llvm-cov report by a derived -p list, and moves COV_FLOOR 88→89 on a measured 849871/941605 = 90.26% (run 35908686532). It rides #4315 (B2, BLOCKED on red checks), and nothing from it is in batch/0.70.0 or main yet. That run uploaded no artifacts, so no lcov is available to re-slice offline.

done_when 1 is still open. The regex is unchanged on the port branch. Measured on its tree (7197 src/*.rs files, 918 excluded):

pattern	hits
`entrenar/`	0 files (dead)
`realizar/`	7 files, all in aprender-train `ecosystem/realizar/`; 0 in aprender-serve
`trueno`	54/56 aprender-zram, +2 orchestrate, 1 profile, 1 train, 1 viz; 0 in aprender-compute
`apr-cli/`	504/521 apr-cli (the rest via other patterns)

The conflict. The Makefile comment says "Coverage tracks the main aprender library only … external deps (trueno, realizar) excluded". #4023, which was quorum-approved and cop-ruled, deliberately measures the monorepo, including an aprender-serve phase. Repairing "to documented intent" would exclude aprender-serve, aprender-train and aprender-compute, reversing #4023.

Recommendation (A): monorepo scope. Delete the dead trueno|realizar/|entrenar/ alternatives, so zram is measured and nothing is excluded by accident. Rewrite the comment to say the monorepo is measured. Resolve done_when 2 by excluding crates/aprender-compute/src/backends/gpu/ from the report: no coverage runner executes it, so it stays out of the denominator until a GPU coverage lane exists. Measure with one coverage-nightly workflow_dispatch on the branch; the "before" is run 35908686532. Floor-reset stays a separate commit.
(B) is the documented-intent scope, which would exclude the three crates.

Needs a cop ruling on A/B. The dispatch is ~1 coverage run on the shared runner pool.

Folded into batch/0.70.0 @ 568c9e2 (aprender-a2, cop ruling A).

Coverage-nightly run 36065904363 on 237f10f (the regex change on top of the #4023 port):

	LH / LF	%	excluded src files
before (run 35908686532, #4023)	849871 / 941605	90.26%	918
after (run 36065904363)	849706 / 935347	90.84%	893

All 100 test binaries reported ok: no failures, no kills, no DID NOT MEASURE. Both phases ran (the workspace, and the aprender-serve module groups). The floor 89 holds. The gate suggests raising it to 90; per the release-cop ruling that waits for two consecutive nightlies at or above 90.5%, and it will be a separate commit.

The fold also brings the #4023 stack (16 commits, 14 files, no workflow files) into batch/0.70.0. The roadmap aggregate was regenerated and aggregate --check passes. guard_tree --no-cargo on the merged tree matches the untouched batch; the one FAIL on both is check_baseline_ratchets bashrs 7.4.1-vs-7.4.2 tool_version, which is on the local box and not caused by this fold.

done_when status: 1 (regex) ✅; 2 (GPU contradiction: compute gpu/ is excluded, and aprender-gpu is excluded from both run and report) ✅; 3 (floor as a separate commit) ✅ via #4023's 8dfaff5; 4 ("did not measure" is never a verdict) ✅ via #4023. Still open: 5 (drop coverage from PREPUBLISH_DEFERRABLE) and 6 (the CLAUDE.md 88.78% figure). I'll take those next.

docs(claude-md): coverage re-derived - 90.84% on the monorepo scope, …

docs(claude-md): name the coverage sample as a branch dispatch, not m…

done_when 5 and 6 (aprender-a2), folded into batch/0.70.0 @ 7e471d6

5. coverage is not in PREPUBLISH_DEFERRABLE (scripts/check_publish_preflight.sh:67), and git log -S shows it never was. The 0.69.1 deferral was an operator ruling. The preflight (lines 125-132) refuses any receipt that defers a row the list does not name, so coverage has to pass on its merits. It now does: 90.84% against the 89 floor.
6. CLAUDE.md now reads 90.84% (849706/935347). It names the sample as a workflow_dispatch run (36065904363) on the branch, and says it is not on main until the batch merges. It also cites main's own scheduled nightly 35937679357 @49fe19c28: 826969/941649 = 87.82% under the old regex. COV_FLOOR is cited by name; the old Makefile:287 citation had drifted to line 525. readme_contract passes 15/15.

Quorum 3/3 APPROVE (degraded: same-family). One provenance finding (branch dispatch vs main nightly) was fixed. With #4023's items, every done_when is met once batch/0.70.0 merges.

feat(qa): GPU-correct is one subject — reconcile the two surfaces tha…

merge origin/main into batch/b2-ci-infra — #4016 fit carried onto #4338…

fix(ci.sh): stop renaming ~/.cargo/config.toml around coverage (#4097)

Parked by APR-EPIC-001: over S3 budget (rank 8/142). Reopen = pull. Applied by aprender-d1: claude+agy both said close, apr shadow row recorded (handoff/close-quorum-2026-09-25.json).

batch(B2): CI/infra/scripts/workflow fold — replayed onto main (REPLA…

Navigation Menu

Repository navigation

Coverage owes 0.69.1: 87.82% vs floor 88, and COVERAGE_EXCLUDE_REGEX has described a different tree since April #3839

What was deferred

Two pieces of work, and they are not the same piece

1. The measurement has described a different tree since April

2. The real gap, once the measurement is honest

Also owed: the coupling that hides the number

Done when

Direction changed: this is now the FIX, not the obligation

Static analysis, which kills the option this issue originally implied

What follows, and it is uncomfortable

Open, pending one lcov

The dominant cause is not missing tests — the gate skips 1,207 of its own tests by name substring

Why this has been silently steering the roadmap

Two directions, one number

Added to done-when

0.70.0 brief — two things that cost real time to find

1. The coverage gate refuses to run its own tests

Why this is worse than the number

Consequence when WRITING tests

The fix is not tests

2. Rule 7 and the sigma ratchet are in tension

Suggested for 0.70.0

12 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Coverage owes 0.69.1: 87.82% vs floor 88, and COVERAGE_EXCLUDE_REGEX has described a different tree since April #3839

Description

What was deferred

Two pieces of work, and they are not the same piece

1. The measurement has described a different tree since April

2. The real gap, once the measurement is honest

Also owed: the coupling that hides the number

Done when

Activity

noahgift commented on Sep 22, 2026

Direction changed: this is now the FIX, not the obligation

Static analysis, which kills the option this issue originally implied

What follows, and it is uncomfortable

Open, pending one lcov

noahgift commented on Sep 22, 2026

The dominant cause is not missing tests — the gate skips 1,207 of its own tests by name substring

Why this has been silently steering the roadmap

Two directions, one number

Added to done-when

noahgift commented on Sep 22, 2026

0.70.0 brief — two things that cost real time to find

1. The coverage gate refuses to run its own tests

Why this is worse than the number

Consequence when WRITING tests

The fix is not tests

2. Rule 7 and the sigma ratchet are in tension

Suggested for 0.70.0

12 remaining items

noahgift commented on Sep 24, 2026

noahgift commented on Sep 24, 2026

noahgift commented on Sep 24, 2026

noahgift commented on Sep 25, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions