Add experimental IxBy execution backends and repair Aiur let hoisting - #626
Open
johnchandlerburnham wants to merge 30 commits into
Open
Add experimental IxBy execution backends and repair Aiur let hoisting#626johnchandlerburnham wants to merge 30 commits into
johnchandlerburnham wants to merge 30 commits into
Conversation
johnchandlerburnham
force-pushed
the
jcb/ixby
branch
from
September 11, 2026 20:36
9a7286e to
306a5d7
Compare
Add reference semantics, validation, crypto primitives, canonical codecs, commitment statements, and scalar/control/immutable-object Aiur interpreters. Include representation lemmas, adversarial and proof regressions, measurements, and tracked implementation plans. Closures, general application, and full circuit refinement remain explicit follow-up work. Based directly on origin/main without Flock-specific changes. The isolated build and 4,129 targeted runtime checks pass.
Add checked object reconstruction, immutable-store preservation, canonical declaration-table binding, and byte/u32 and zero-count parser contracts. Include 56 kernel-checked lemmas and 704 diagnostic tests; document the remaining recursive parser, execution, and AIR obligations.
Prove compiled ID equality and uniqueness, bounded declaration parsing, raw-advice loading, and the actual is_run header/constructor prefix. Preserve exact continuation state and document remaining whole-program obligations. Add 5,568 regression checks across these checkpoints; all 8,231 targeted checks pass, including existing FRI workloads. Audit 109 new public lemmas and keep generated kernels unchanged.
johnchandlerburnham
force-pushed
the
jcb/ixby
branch
from
September 12, 2026 10:54
306a5d7 to
00ca5e4
Compare
Group object proof components under Aiur/Objects and mirror the hierarchy in Tests/Ixby. Preserve suite selectors and executable names while updating namespaces, imports, runner roots, and documentation paths.
Defer runtime setup, share LSpec and Aiur fixture/proof helpers, preserve fresh-verifier construction, and split parser conformance tests into focused modules. Add merge-queue proving coverage and an exact 314-root theorem trust manifest with source-frontier and negative checks. Rename the shared host adapter to System and consolidate the current contracts and upstream compiler attribution.
Import reusable arithmetic, Boolean, byte, BLAKE3, and Merkle conformance gates with source provenance and pinned Flock parameters. Keep the trace/circuit/FFLONK workspace independent of the production Cargo graph. Retain explicit root-conditional and gadget-only labels; this does not implement generic IxBy execution or close the terminal roots. Validation: 30 Stage 3 and 149 Stage 4 ordinary tests pass; both workspaces pass formatting and all-target Clippy with warnings denied.
Add canonical Exec/public-result statements, a strict closed-CheckEnv adapter, proof-free backend setup types, and the byte/source/terminal composition contracts. Keep the private input digest bound to execution and retain explicit compiler, backend, root-closure, and hash-collision premises. Validation: IxBy codec/claim/setup regressions and the exact 322-root trust audit pass. No production Ixon wire tags or Claim.eval semantics change.
Implement current canonical key/proof/claim codecs and the full deterministic transcript, OOD/logUp, MMCS, PCS/FRI, and claim-returning source wrapper. Add independent transcript, graph, arithmetic, OOD, and MMCS row/leaf relations with an exact 105-root trust audit. Wire fast and opt-in native differentials into CI and document the remaining refinement, compiler, generic-execution, and closed-terminal-proof gates. This is partial certification, not completed M2 or Stage 3/4. Validation: warnings-as-errors builds and both theorem audits pass; 404 focused Lean checks and 47 real-proof/stand-in binding checks pass. The stand-in aggregate does not establish production Ixon validity.
Prove complete boundary-frontier/cap MMCS equivalence, positional bit reversal and bounds, ordered Lagrange row folding, and authenticated per-height quotient reduction. Refactor the pure reductions into explicit first-order state transitions with exact length checks before pairing coordinates. Add nonzero, multi-height and order-sensitive input vectors. The query-fold chain and full PCS/Stage 2 refinement remain unfinished. Validation: 140 exact audited theorem roots; 416 focused Lean checks; 47 current native proof and stand-in claim-binding checks; warnings-as-errors builds pass.
Connect canonical program/input decoding, fixed instruction and operand access, scalar dispatch, ordered control steps, terminal output encoding, and all Exec commitments under a proof-free setup API. Verify 39 executions with one setup in fresh digest-only child processes and reject recomputed decoder-advice splices. Keep native refinement and Stage 4 closure explicitly unfinished.
Bind all deferred root families to setup-owned exact table programs using the constrained replay points and claims, with only externally expected Q public. Add fail-closed streaming admission and retain the actual supported-domain rejection prefix. The complete closed relation and FFLONK proof remain ungenerated.
Preserve the legacy default identity and bind packed compression to a distinct setup/transcript. Reuse ten word tables across the full commitment chain, cover all native drivers and exact Stage 4 roots, and exercise the complete scalar proof corpus with isolated verification. Record the whole small-class closed census refusal without claiming a terminal key or proof.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Introduce IxBy as an experimental functional bytecode execution layer, with a
pure Lean reference model and three bounded Aiur/FRI interpreters. Each fixed
interpreter profile proves execution of different guest programs under the same
interpreter key. Also repair a shared Aiur let-hoisting bug discovered by object
interpreter differential tests, and regenerate the affected native kernels.
This is an experimental implementation and proof checkpoint, not certified
compilation from Ixon or a complete AIR-to-reference soundness theorem. Guest
programs are hand-authored. The ISA, profiles, wire format, and proof parameters
are not production commitments.
The branch is rebased onto
mainat72238a58, with tip306a5d75. Seven commitsseparate the shared compiler repair, initial IxBy implementation, generated
kernels, and subsequent proof checkpoints. The rebased tree is byte-for-byte
identical to the saved, tested pre-rebase checkpoint.
No Flock-specific implementation, CLI, fixtures, dependencies, Compilatrix
changes, or production claim-format changes are included. Rust changes relative
to
mainare confined to the three regenerated files incrates/ixvm-codegen/src.Implementation
cases, direct/self/tail calls, explicit continuations, and reference
PAP/application semantics. Evaluation is total and fuel-bounded, with checked
determinism, fuel-extension, and conditional composition results.
byte, and BLAKE3 operations; strict bounded profile/program/value codecs; and
domain-separated profile/program/input/output commitments. Checked byte
execution is connected to the functional reference semantics.
bytes, admit the whole image, execute it, serialize the actual terminal value,
and check the expected output commitment. Unused code is admitted too. Decoded
tables, execution traces, object pointers, ranks, and output values are not
supplied as execution advice.
without rerunning reference execution or receiving execution advice. The
adapter retains native-execution preflight and bounded proof decoding with
exact reserialization checks.
The object suite includes recursive list map and tail fold. Constructor identity
uses the full 40-byte name. Computed ranks and bounded field-list traversal
support finite-value representation; external input/output tree limits are
separate from intermediate object bounds. Each backend has its own profile/key,
reused across its guest images. Changing capacities can change the key.
Kernel-checked proof components
The proof modules now extend beyond conditional frame/heap representation to
contracts for concrete memory and actual compiled parser components:
preservation, and checked constructor-table binding.
bounded duplicate traversal, and complete bounded constructor-declaration
parsing, including field-count and duplicate rejection.
genuine bytes and preserves prior reads/I/O under explicit metadata, address,
length, and allocation bounds.
block headers, exact zero-count Nil stores, and continuation handoffs.
field guard accepts precisely canonical eight-byte Goldilocks encodings;
Boolean and field-component rejection, five/six-field layouts, u32 local-index
bounds, actual Call behavior, and loader-to-operand composition are checked.
Structural certificates bind the relevant bytecode bodies, operations, dispatch
edges, return layouts, and required same-toplevel callees. Tests cover both full
compilation and pruned, relocated function indices. The latest checkpoint adds
67 public kernel-checked lemmas: 27 for code headers and 40 for scalars/operands.
The audited proofs use only Lean's standard logical axioms, with no proof holes,
custom axioms,
native_decide, or proof-limit overrides in these new modules.These contracts concern the Lean bytecode evaluator. Their byte-range, metadata,
capacity, and frame premises are explicit; loader composition discharges the
genuine-byte premise at identified prefixes. They do not assume or establish
complete instruction/table admission, canonical whole-program binding, compiler
correctness, or native/gadget/trace/AIR soundness. Errors carry no state, so the
parser contracts do not claim rollback on failure.
Shared Aiur compiler repair
Object differential testing exposed an assertion bypass: hoisting a
continuation's shadowing binding could change which variable an earlier
assertion checked. The original IxBy entry consequently accepted trailing
program bytes in compiled execution while source execution rejected them.
The repair makes caller/callee renaming collision-free, preserves shared pattern
bindings, sequences complete strict arguments left to right, and keeps
assertions, I/O, and debug operations before their continuations. Ordered
statement lets also preserve lowering of guarded inlined branches. The object
entry uses its original shadowed names again and rejects trailing bytes without
a local naming workaround.
Focused regressions compare explicit expected results/errors across original
source, normalized source, bytecode evaluation, and native execution, with
successful cases also proved, serialized, and independently verified. This is
regression coverage, not a formal compiler-refinement proof.
All three checked-in Rust kernels (
aiur_ixvm.rs,aiur_multi_stark.rs, andaiur_ix_aggr.rs) are regenerated. The rebase's sole conflict was the generatedIxVM kernel; regenerating from the rebased Lean sources resolved it.
Rollout: rebuild affected compiled circuits and proving/verifying keys.
The source repair does not retroactively fix old bytecode or old keys, and
unchanged wire revisions do not imply artifact compatibility.
Validation
The rebased tree passes a warning-free build and all 25,383 targeted IxBy
runtime checks, including full proving for all three backends:
Parser coverage includes certificate mutations, exact codec agreement,
canonical field/Boolean rejection, local bounds, full-field pointers, padding,
truncation and malformed cells, fuel and Call boundaries, duplicate IDs,
allocation/deduplication, and loader composition. Forged non-byte memory tests
document the genuine-byte premise; they are not native/AIR soundness claims.
Additional validation of the shared repair and main integration passed:
and 37 proof roundtrips, plus existing cross-engine/hash/cost/recursive-verifier
coverage.
ixvm-codegentests.ix codegen --checkfor all three generated kernels, repeated after rebasing.The latest proof additions introduce no new FRI workloads. Historical timing and
proof-size observations remain labeled as historical; no new isolated
performance claim is made. Repeated runs and theorem counts are not added to the
runtime-test total. These are local validation results; full GitHub CI and review
remain separate merge gates.
Reproduce
lake build --wfail ix IxTests AiurHoistingTests \ Ix.Ixby.Aiur.ObjectsCodeHeaders Ix.Ixby.Aiur.ObjectsScalars \ Ix.Ixby.Aiur.ObjectsOperands RAYON_NUM_THREADS=8 .lake/build/bin/IxTests \ ixby ixby-crypto ixby-codec ixby-objects-memory ixby-objects-table ixby-objects-parser RAYON_NUM_THREADS=8 .lake/build/bin/IxTests --ignored \ ixby-aiur-prove ixby-control-prove ixby-objects-prove RAYON_NUM_THREADS=8 .lake/build/bin/AiurHoistingTests --prove --regression RAYON_NUM_THREADS=8 .lake/build/bin/IxTests --ignored ixvm .lake/build/bin/ix codegen --check cargo test --release -p ixvm-codegenReview focus and follow-up
The highest-impact review area is the shared hoisting repair: scope hygiene,
strict-argument effect ordering, and continuation/return behavior. The generated
kernel content check and rebuilt native parity are essential companion checks.
For the proof modules, review the exact certificate boundaries and remaining
premises rather than treating representation alone as execution verification.
Follow-up work remains explicitly outside this experimental checkpoint:
cross-table validation, and authenticated canonical whole-program binding.
resource-bound discharge, general codec inverse/injectivity results, and full
compiler/gadget/hash/trace/AIR refinement and malicious-witness auditing.
remaining crypto primitives. Broader reference support does not imply
constrained support; Nat/String and
caseNatremain outside the crypto profile.interpreter, terminal compression, and production activation.
Proof parameters and capacities are test-only, not deployment security
recommendations. The low-level native prover can still panic on execution
errors; negative fixtures stop at native execution, and this PR does not change
that API contract.
Further detail:
IxBy overview,
encoding,
and object backend and proof boundaries.