Repository navigation
F-PROVE-007 asserts 297 compile-time proofs against a tree that generates 578, and never executes #2732
Description
Assimilated from #2728 · child of #2706 (APR-PERF-GATE-001, the receipt rule)
Behaviour
f_prove_007_proof_count_is_exactly_297 asserts the tree contains exactly 297 compile-time proofs. The tree generates 578. The assertion has never fired, because the file it lives in is compiled out by default — and the only two targets that would compile it in report success unconditionally.
Two independent failures: it does not run, and it would fail if it did.
Reproduction
It does not run
$ cargo test -p aprender-core --test falsification_spec_v10_tests f_prove_007
rc=0
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
crates/aprender-core/tests/falsification_spec_v10_tests.rs:16 — #![cfg(feature = "model-tests")] gates all 138 tests in the file.
It would fail if it did
$ cargo test -p aprender-core --features model-tests --test falsification_spec_v10_tests f_prove_007
rc=101
thread 'f_prove_007_proof_count_is_exactly_297' panicked at
crates/aprender-core/tests/includes/falsification_spec_v10_contract_model.rs:259:5:
assertion `left == right` failed: F-PROVE-007: Expected 297 proofs, found 578
left: 578
right: 297
Ground truth, counted directly out of the build directory:
$ grep -c "const _: () = assert!" \
/mnt/nvme-raid0/targets/aprender/release/build/aprender-core-*/out/model_families_generated.rs
578 # identical across all generated copies
Nothing that would run it can report failure
model-tests appears nowhere in .github/. It appears twice in the Makefile, and both targets exit 0 no matter what happens:
$ make test-spec
rc=0
🔬 Running spec structural tests...
✅ Spec tests complete # ran zero tests
$ make test-model
rc=0
❌ f_ollama_001 FAILED
❌ f_ollama_002 FAILED
... (all 10)
✅ Model tests complete
Two compounding causes:
- Neither passes
-p aprender-core(Makefile:113,Makefile:121). From the workspace root,cargo test --features model-testsresolves the feature against the rootaprenderfacade, which does not define it:
error: the package 'aprender' does not contain this feature: model-tests.
No test is ever compiled, let alone run. .ONESHELL:(Makefile:20) with noset -o pipefailand noset -e. The whole recipe runs as one bash script, so the recipe's exit status is its last command — which in both targets is a decorative@echo "✅ ...". Thecargo … 2>&1 | grep "test result:"pipeline already discards cargo'src=101in favour of grep's status, and even grep's non-zero is then overwritten by the trailing echo.
make test-model additionally prints ❌ … FAILED ten times and still exits 0 — the failure is visible on stdout and invisible to the exit code.
This is the feedback_verification_discipline rule 1 and the pass-grep class, in the one lane that would have caught the stale constant.
Which number is wrong
Not established here, and the fix should not assume. CLAUDE.md and #2728's H0-064 both claim 297; build_codegen.rs emits 578. Either the generator gained proofs nobody re-baselined, or the constant was aspirational. Whichever it is, the constant must be derived or explicitly re-baselined with a dated note — not silently bumped to 578 to make the test green. Bumping it to match the current output reproduces the original defect one commit later.
RED-turning mutation to register
- Un-dark it. Fix
Makefile:113,121to pass-p aprender-core, and make both recipes fail-closed: capture${PIPESTATUS[0]}(or drop the pipe) and drop the unconditional trailingecho "✅", or addset -euo pipefailat the top of the ONESHELL recipe.
Mutation: point the recipe at a test known to fail.make test-specmust exit non-zero. It currently exits 0 — so run this mutation first, against the current Makefile, to confirm the target is presently unable to red. - Re-baseline F-PROVE-007 to the decided value, with the derivation recorded next to the constant.
- Mutation on the assertion: change the expected count by ±1.
cargo test -p aprender-core --features model-tests --test falsification_spec_v10_tests f_prove_007must go RED. Revert; confirm GREEN. - Discrimination: the corrected test must pass on an unmodified tree — a gate that flags everything is as broken as one that flags nothing.
- Wire the structural (non-model, non-GPU) subset into a lane that actually runs. Note the standing trap: a suite gated only in the Makefile is invisible to
ci.yml.
Why this belongs to #2706
F-PROVE-007 is a receipt for a claim the project publishes: "297 compile-time algebraic invariant proofs" (CLAUDE.md, #2728 H0-064, SHOWCASE-BRICK-001 §16). The receipt has never been checked, disagrees with the artifact by 281, and the lane that would check it ends in echo "✅". That is the epic's thesis in miniature.
Activity
noahgift commentedon Aug 28, 2026
Correction: the
297half was already fixed, and the replacement is also inadequateThis issue was filed against
d6c6c6f8f— the stale main checkout, which is 58 commits behindorigin/main. Onorigin/mainthe literal is gone: #2631 (51bfcb189) replaced it, and the repro command gives rc=0, not 101. My apologies; that is the third finding today traced to reading the stale tree.What survived is real: the function was still named
..._proof_count_is_exactly_297, and the file is still#![cfg(feature = "model-tests")]— dark by default. The darkness half is fixed separately (the Makefile omits-p aprender-core, so--features model-testsresolved against a facade that does not declare it and no test had ever compiled).297 vs 578: neither is wrong, and a literal is the wrong shape
The archived spec that published 297 states its own basis — "297 total across 8 families, 24 size variants" — and its per-class table sums to exactly 297 at that size. The registry has since grown to 16 families / 47 variants and the same generator emits 578. Per-variant density is unchanged: 297/24 = 12.4, 578/47 = 12.3. Nothing stopped being proved; more models arrived.
Decomposing today's file against the generator's rules predicts 578 exactly: 9 unconditional × 47 = 423, + 47 Vaswani + 46 Ainslie + 31 RoPE-even + 31 RoPE-maxpos. Both numbers are snapshots of a moving population, so any literal is wrong again at the next family YAML. The gate now enumerates the variants the generated file itself declares and requires each to carry each of the nine unconditional proofs.
#2631's replacement does not hold either
count >= KNOWN_FAMILIES.len() * 3is578 >= 48— a 12x margin. Measured: deleting one unconditional proof frombuild_codegen.rsdrops 578 → 531, and>= 48stays GREEN. It would tolerate the generator losing eight of its nine unconditional proofs.A second live defect, found only by mutation
find_generated_filescanned the shared<target>/debug/build/directory. I verified independently: that directory currently holds 401aprender-core-<hash>/out/copies, and the walk returns whichever it hits first.The first mutation run came back GREEN because the gate read a months-old OUT_DIR that still contained the deleted proof. It now resolves
env!("OUT_DIR")— the same mechanismfamily_registry.rs:464uses toinclude!the file — which also un-darkens F-PROVE-003/004/005, and cross-checks the file'sKNOWN_FAMILIESagainst the compiled-in list so a foreign artifact is named rather than tolerated.This is the shared-target-dir hazard reaching a test: worktrees do not isolate it here (
target_directory=/mnt/nvme-raid0/targets/aprenderfor every worktree).env!("OUT_DIR")families * 3gatenoahgift commentedon Sep 25, 2026
Parked by APR-EPIC-001: over S1 budget (rank 19/163). Reopen = pull. Applied by aprender-d1 under a 3-lane quorum receipt (handoff/balance-quorum-rev4.json).