feat(scripts): kernel-level axiom sweep with committed regression baseline - #127
Open
alexanderlhicks wants to merge 3 commits into
Open
feat(scripts): kernel-level axiom sweep with committed regression baseline#127alexanderlhicks wants to merge 3 commits into
alexanderlhicks wants to merge 3 commits into
Conversation
…eline Add `lake exe axiomsweep`: walks the compiled environment and computes, for every PolyFun.* declaration, its transitive axiom dependencies — the #print axioms information, library-wide, in one pass. Reads elaborated .olean data, so private and macro-generated declarations are included and no source heuristics are involved. Baseline at this commit: 8006 declarations across 224 modules, 0 sorryAx-tainted, 0 non-standard-axiom-tainted — the library is fully kernel-clean, and --check now keeps it that way (fails iff a declaration is tainted that scripts/axiom_baseline.json does not list). Wire-up: ./scripts/validate.sh --axioms (enforcing, opt-in; verified green end-to-end) and a report-only CI step in ci.yml; docs/wiki/quickstart.md documents the workflow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…, docs Collector: two-phase DFS + fixpoint repair. The one-pass DFS finalized self-referencing constants (every inductive/ctor pair) prematurely and memoized the wrong result for all later roots — reviews confirmed rows diverging from #print axioms (both directions) and constructible shapes where a new sorry passed --check. The repair pass re-derives every set in finalization order until stable: the least fixpoint = true kernel closure, strictly more accurate than #print axioms inside mutual families. Also: axiom *types* traversed (CollectAxioms parity), duplicate constNames rows deduped, native trust-axiom names normalized to their owner (ax_N_M counters are Elab.async/toolchain-volatile), --check/--update-baseline mutually exclusive, unknown --root fails gracefully, nonstandard shrinkage detected, bare Lean.ofReduceBool/Lean.trustCompiler never baselinable (floor). Verified post-fix: sorry-set unchanged; previously-wrong rows now match #print axioms ground truth; regeneration deterministic; gate exit codes intact. Known blind spots documented (structure-field defaults and examples never enter any environment walk; unimported files — paired with the repo's import-completeness gate). CI: infrastructure failures (exit != 1) now fail the step; only taint findings are report-only during the soak. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> PolyFun specifics: baseline unchanged (8006/224, 0/0 — reviewer's 14 attribution-divergent rows involved only standard axioms); --axioms documented in the AGENTS.md Building section; --root on a glob-based lib (PolyFunTest) now errors cleanly instead of crashing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
lake exe axiomsweep: kernel-level axiom /sorryaccounting for the whole library, with a committed regression baseline.The tool loads the built
.oleanenvironment (the same data the kernel checked) and computes, for every declaration under the swept root modules, the set of axioms it transitively depends on — i.e. the#print axiomsinformation, library-wide, in one pass. Because it reads elaborated data rather than source text, it sees exactly what the kernel accepted: private declarations, macro-generated declarations, and instances included; no regex heuristics.Why
PolyFun feeds VCV-io and ArkLib, and its kernel-level state turns out to be pristine — 0
sorryAx, 0 non-standard axioms across all 8,006 declarations. This PR proves that and adds the regression gate that keeps it true, mirroring the same tool on the sibling repos. Reviews of downstream repos have repeatedly found theorems that silently inheritsorryAxthrough framework stubs; this makes that class of defect mechanical to catch.How
scripts/AxiomSweep.lean(+lean_exe axiomsweepin the lakefile): memoised traversal of the compiled environment; only imports coreLean, runtime-imports the built oleans, so it needs a completedlake buildand adds no dependencies. Runs in seconds.scripts/axiom_baseline.json: the currently-knownsorryAx-tainted declarations (and any non-standard-axiom dependents).--checkfails iff a declaration is tainted that the baseline does not list; closing gaps stays green and prints a nudge to shrink the baseline via--update-baseline.::warningon regression). Flip to enforcing by deleting the|| echoonce the baseline has soaked for a few PRs.Modes
Baseline at this commit
PolyFun.*modules.#print axioms(PFunctor.Display.responderCoalgebraEquiv_postcondition→[Quot.sound]);./scripts/validate.sh --axiomsverified green end-to-end.Notes
alh/axiom-sweepbranch (multi-root variant); the copies are intentionally self-contained per repo, mirroring how other scripts are shared across the org. If it proves its keep everywhere, extracting a shared home is a possible follow-up.lean4exportwas considered and deliberately not used: it targets external kernel checking, has no source positions, and would add a per-toolchain pinned dependency; walking the environment yields the same information with line numbers and zero new deps.Adversarial review
Five independent adversarial reviewers (one per repo deployment) attacked the tool, wiring, and baselines before this PR was opened. Everything they confirmed is fixed in the follow-up commit on this branch:
#print axiomsin both directions, with constructible shapes where a newsorrypassed--check. Fixed with a fixpoint-repair phase computing the true kernel closure (now strictly more accurate than#print axioms, whose ownCollectAxiomswobbles inside mutual families). Post-fix rows spot-verified against ground truth; sorry-sets unchanged everywhere.CollectAxiomsparity); duplicateconstNamesrows deduped;native_decideaxiom names normalized to their owning declaration (theax_N_Mcounters areElab.async/toolchain-volatile);--check/--update-baselinemutually exclusive; bareLean.ofReduceBool/Lean.trustCompilerare never baselinable (floor).examples never enter any environment walk (#print axiomsincluded); unimported files are invisible — paired with each repo's import-completeness gate.What survived attack unchanged: baseline determinism (byte-identical regeneration), all headline numbers, spot-checks vs
#print axioms, gate exit-code semantics, and the CI/step placement.🤖 Generated with Claude Code