[BradyPlanden]

Description

Reads variables, and their sensitivities, from a rust-model IDAKLUSolver solution in Rust.

• set_up binds the CompiledModel to the model's ObserverCache (refactor: read solution variables through per-model observer caches #5786).
• build_variable then evaluates each variable on a tape compiled into the model's retained graph: one Rust call per segment, cubic-Hermite between time points when the solution stores derivatives.
• Sensitivities are computed by the chain rule in Rust, one JVP sweep per parameter per segment.

Besides CompiledModel.graph, a getter it adds to the bindings with its stub, this PR is Python only; the other Rust methods it calls (n_states, ExprGraph.from_expr) are from #5962.

• ObserverCache:
  • It gains bind_rust_model, native_leaf (a variable's tape, compiled once per model), native_set (several variables on one tape, sharing subexpressions) and post_sum_leaf.
  • Rebinding the model on a new set_up drops the native leaves.
  • A leaf is one segment's compiled form of one variable, as the module docstring of observation.py says.
• NativeObserver: evaluates raw values (eval_trajectory), Hermite values (eval_trajectory_hermite) and sensitivities. Time integrals add their post-sum sensitivities through native_postfix_sensitivities. On an output_variables solve, a variable that is not an output reads its values and sensitivities from the placeholder states, as on the CasADi path.
• Solution.update([...]):
  • It deduplicates the names and reads the eligible ones together. Each segment evaluates one SharedOutputs tape and sweeps its sensitivities once per parameter, and each variable reads its rows through a SharedOutputLeaf. A segment with no time points gives empty rows.
  • Values and sensitivities are bitwise equal to reading the names one by one.
  • It builds every listed variable before storing any, so if one fails (an unknown name, or a state-dependent variable on an output_variables solve) nothing is stored. Before, on CasADi and rust alike, the names before the failing one were rebuilt and kept. update("name") with one string is unchanged.
  • QuickPlot and Solution.get_data_dict build all their variables this way.
• Time integrals: ProcessedVariableTimeIntegral.sum_over_time is public, and check_discrete_times is a method, for the native path.

When native reading applies:

• A solution reads natively when every segment's model is a rust model set up by a solver that observes natively; in this PR that solver is IDAKLUSolver.
• Other solutions read through CasADi, as do VectorField variables.
• Time integrals and VectorField variables are read one by one, not on a shared tape.
• Time-integral post-sum values are evaluated in Python.
• Asking for sensitivities without calculate_sensitivities raises ValueError, as on main.

On the CasADi path, values are unchanged. Solution.update deduplicates its names and stores all or nothing, and QuickPlot and get_data_dict build every variable up front.

Review on #5732

• "Observation" sounds like a measurement: observation.py keeps the observer naming from refactor: read solution variables through per-model observer caches #5786 on main (ObserverCache, VariableObserver), and the class under review is not in this stack.
• What a leaf is, and what backend[key] is:
  • "Leaf" is defined in the module docstring.
  • The cache is keyed by variable name.
  • SharedOutputs holds one result for as many reads as it has leaves, which its docstring states. Each call keeps its own result and a lock guards the held one, so threads reading different solutions while the GIL is released get their own rows.
• Missing return type: every function and method this PR adds has a return annotation.

Type of change

New feature, under ## Features in CHANGELOG.md.

Key checklist

• No style issues: pre-commit run
• Unit tests pass (N), as do the integration tests (N)
• Tests added: 78 Python test functions, with 3 existing tests parametrised over the format.
  • Native against CasADi reading, each from its own rust solve:
    • direct variables at rtol 1e-9;
    • time integrals, explicit integrals, discrete sums and experiments at 1e-7 to 1e-8;
    • dense t_interp;
    • 2D DFN and 3D MPM layouts at 1e-4.
  • Sensitivities: from a rust solve against a CasADi solve (rtol 1e-5).
  • Through IDAKLU:
    • off-grid reads, 1D and 2D variables, events, experiments and Hermite off;
    • sensitivities, including time integrals, and other variables' sensitivities on an output_variables solve;
    • first and last states, and a reduced solution;
    • no CasADi observe function compiled.
  • Batching: batched reads bitwise equal to per-name reads, over an empty segment, and from 8 threads at once; update storing nothing when one name fails.
  • Caching and fallback:
    • reuse across solves, rebinding and model copies;
    • joined solutions of two rust models;
    • a CasADi segment falling back to CasADi;
    • pickling.

Rust execution engine stack: proposal #5667, showcase #5732

Each PR is one commit on the one above it.

#	PR	Adds
0	#5954	Function.create_copy keeps the function's class
0b	#5955	CasADi's RegPower and uniform-grid interpolant lookups match evaluate
1	#5956	pybamm-core: expression graph, typed IR, interpreters, forward-mode AD
2	#5957	pybamm-core: CompiledModel, coloured Jacobians, the C ABI
3	#5958	pybamm.rust bindings, built into the pybamm wheel
4	#5959	Symbol.to_rust
5	#5960	convert_to_format = "rust" for ScipySolver, AlgebraicSolver and EIS
6	#5961	pybammsolvers: IDAKLU over the C ABI
7	#5962	IDAKLUSolver on rust models
8	#5963	variables and sensitivities read in Rust ← this PR
9	#5964	pybamm-core: diffsol DAE solver
10	#5965	pybamm.DiffsolSolver
11	#5966	"rust" as the default convert_to_format

[codecov]

Codecov Report

❌ Patch coverage is 97.08738% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 98.20%. Comparing base (1826172) to head (52b69b1).

Files with missing lines	Patch %	Lines
...ges/pybamm/src/pybamm/solvers/variable_observer.py	94.85%	9 Missing ⚠️

Additional details and impacted files

@@                      Coverage Diff                      @@
##           stack/07-idaklu-rust-mode    #5963      +/-   ##
=============================================================
- Coverage                      98.21%   98.20%   -0.02%     
=============================================================
  Files                            353      353              
  Lines                          36733    37029     +296     
=============================================================
+ Hits                           36076    36363     +287     
- Misses                           657      666       +9     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

Bencher Report

Project	PyBaMM
Branch	stack/08-native-observation
Testbed	bare-metal

⚠️ WARNING: Truncated view!

The full continuous benchmarking report exceeds the maximum length allowed on this platform.

🐰 View full continuous benchmarking report in Bencher