Skip to content

feat(rust): expression graph, typed IR, evaluator and forward-mode AD - #5956

Open
BradyPlanden wants to merge 1 commit into
mainfrom
stack/01-rust-core-graph
Open

BradyPlanden wants to merge 1 commit into
mainfrom
stack/01-rust-core-graph

Conversation

@BradyPlanden

Copy link
Copy Markdown
Member

Description

Adds pybamm-core, a Rust crate in a new Cargo workspace at packages/pybamm-rust, which compiles a PyBaMM expression into a tape and evaluates it. This PR is the compiler on its own: an arena DAG, rewrite passes, forward-mode differentiation, a typed slot-addressed IR, and scalar and lane-batched interpreters. Nothing in Python uses it yet; the bindings arrive in #5958 and Symbol.to_rust in #5959.

  • Arena is append-only and stores every child before its parent, so a graph is acyclic by construction, and deserialising one re-checks the order. Node has 51 kinds. Its checked constructors (state_vector, state_vector_dot, index, conditional, interpolant_nd, and try_new on the matrix and interpolant data) return a CoreError rather than build an invalid node. A 1D linear table may repeat an interior knot, which makes a right-continuous step as in CasADi, but not its first or last knot; cubic and N-D tables need strictly increasing breakpoints.
  • ParamRegistry is the one place an input parameter's registration index and packed slot are decided, through typed ParamIdx, ParamSlot and PackedWidth. It sits outside Arena because the rewrite passes build fresh arenas.
  • simplify_pipeline runs CasADi-style folds, shape-aware zero propagation, and CSE on a bitwise structural hash. A constant power is rewritten as x*x or 1/x only for the exponents where that takes pow's value bitwise (0, 1, 2 and −1). Folds such as 0*x and x/x return 0 and 1 where the original gives NaN (0*inf, 0/0), so the rewrites keep values where every intermediate is finite, up to the sign of zero.
  • tangent_wrt_states, tangent_wrt_params and tangent_wrt_subset build forward-mode JVP graphs. Step functions, the Heavisides and derivative-only nodes differentiate to 0, so there are no second derivatives, and Modulo ignores its jumps. tangent_wrt_subset is public here, and feat(rust): sparsity, coloured Jacobians, compiled models and the C ABI #5957 narrows it to the crate once the Jacobian builder calls it.
  • TypedIr is a tape of 24-byte instructions with a constant pool and a liveness-based slot allocator. CompiledExpr evaluates through &self into caller-owned scratch, so one compiled tape can be shared across concurrent solves. The scalar interpreter runs only the active branch of a top-level Conditional, provided privatising its branches stays within 4× the node count; over that budget it still skips the nodes only one branch owns.
  • eval_batch evaluates K lanes of values and run_tangent_batch 4 or 8 lanes of tangents, both bitwise equal to the scalar interpreter. The batched interpreter runs every branch block that some lane selects.
  • Transcendentals and minimum/maximum are evaluated through the libm crate, not the platform's maths library.
  • The crate is edition 2024 with rust-version = "1.89", has no unsafe, and depends on rustc-hash, thiserror and libm, plus serde and bincode behind a serialize feature. Clippy's pedantic and nursery lints warn and correctness denies. suboptimal_flops is allowed, because mul_add rounds once and so breaks CasADi parity; on baseline x86-64 wheels it is also a slow libm call.

CI gains rust_msrv (cargo +1.89 check --locked --all-targets --all-features) and rust_tests (cargo test --locked --workspace --all-features), both required by ci_gate, and packages/pybamm-rust/** joins the pybamm path filter. Pre-commit gains cargo fmt and cargo clippy --all-targets --all-features -- -D warnings, as checks rather than --fix, since cargo clippy --fix exits 0 even when -D warnings fires. They run through .github/scripts/run_cargo.py, which skips them with a message where cargo is not installed, and pre-commit.ci skips them. The style job therefore now runs cargo fmt and cargo clippy, on a pinned Rust 1.99.1 so that a new stable release's lints cannot break CI. The root pyproject.toml excludes packages/pybamm-rust from the uv workspace.

TypedIr::from_dag and CompiledExpr::new may panic on a graph that validate rejects, so callers validate first. validate refuses an InputParameter whose packed slot ends past u32::MAX. The left operand of a MatMul must be a constant matrix. Scalar eval does no length checks; callers size their buffers from scratch_len and ReadExtents.

Review on #5732

Type of change

New feature, under ## Features in CHANGELOG.md.

Key checklist

  • No style issues: pre-commit run, including cargo fmt and cargo clippy
  • Rust tests pass (205 with --all-features, including 2 doctests)
  • MSRV check passes: cargo +1.89 check --locked --all-targets --all-features
  • Tests added: 203 Rust tests. Unit tests cover each module: arena ordering and serde round-trips, the checked constructors' refusals, parameter registration, the folds, CSE and zero propagation, the closed-form tangent rules, branch ownership, IR validation and slot reuse, and the linear interpolant bit for bit against CasADi, including a repeated knot. Integration tests check from the executed instruction count that only the active branch runs, and that every primitive evaluates bitwise as libm on every path, with libm's outputs pinned. Proptests check that batched evaluation equals scalar evaluation bitwise, that a slot-reusing layout equals the SSA layout, that CSE is bitwise exact and the folds exact where every intermediate is finite, that split evaluation equals monolithic, and that 4- and 8-lane tangent batches equal scalar tangents, with a fixed case for a zero-width node. A compile_fail doctest proves a split expression cannot be batch-evaluated
Rust execution engine stack: proposal #5667, showcase #5732

Each PR is one commit on the one above it.

# PR Adds
0 #5954 Function.create_copy keeps the function's class
0b #5955 CasADi's RegPower and uniform-grid interpolant lookups match evaluate
1 #5956 pybamm-core: expression graph, typed IR, interpreters, forward-mode AD ← this PR
2 #5957 pybamm-core: CompiledModel, coloured Jacobians, the C ABI
3 #5958 pybamm.rust bindings, built into the pybamm wheel
4 #5959 Symbol.to_rust
5 #5960 convert_to_format = "rust" for ScipySolver, AlgebraicSolver and EIS
6 #5961 pybammsolvers: IDAKLU over the C ABI
7 #5962 IDAKLUSolver on rust models
8 #5963 variables and sensitivities read in Rust
9 #5964 pybamm-core: diffsol DAE solver
10 #5965 pybamm.DiffsolSolver
11 #5966 "rust" as the default convert_to_format

@codecov

codecov Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.24%. Comparing base (5b81c32) to head (73c0262).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5956   +/-   ##
=======================================
  Coverage   98.23%   98.24%           
=======================================
  Files         350      350           
  Lines       36104    36104           
=======================================
+ Hits        35468    35469    +1     
+ Misses        636      635    -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@BradyPlanden
BradyPlanden force-pushed the stack/01-rust-core-graph branch from 591ec68 to 96e653a Compare October 28, 2026 09:55
Base automatically changed from stack/00b-casadi-numerics-fix to main October 28, 2026 15:28
@BradyPlanden
BradyPlanden force-pushed the stack/01-rust-core-graph branch from 96e653a to 1923d05 Compare October 28, 2026 15:28
The packages/pybamm-rust cargo workspace with the first pybamm-core layer:
arena, typed IR, scalar and lane-batched interpreters, conditional branch
regions, rewrite passes and forward-mode tangents. Nothing in Python uses it
yet. CI gains rust_tests and rust_msrv; pre-commit gains cargo fmt/clippy.
@BradyPlanden
BradyPlanden force-pushed the stack/01-rust-core-graph branch from 1923d05 to 73c0262 Compare October 29, 2026 08:01
@BradyPlanden
BradyPlanden marked this pull request as ready for review October 29, 2026 08:55
@BradyPlanden
BradyPlanden requested a review from a team as a code owner October 29, 2026 08:55
@github-actions

Copy link
Copy Markdown
Contributor

🐰 Bencher Report

ProjectPyBaMM
Branchstack/01-rust-core-graph
Testbedbare-metal

⚠️ WARNING: Truncated view!

The full continuous benchmarking report exceeds the maximum length allowed on this platform.

🐰 View full continuous benchmarking report in Bencher

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant