The compiler accepts a deterministic torch.export.ExportedProgram, ordered public input names, ordered output names, example inputs and a retained-tensor budget. It emits a monolith plus a bounded set of legal incremental strategies. It requires no architecture annotations, module-name allow-lists or model-specific cut hints.
The public implementation is split into five layers:
analysis.pypropagates named-input dependencies through the exported FX graph.compiler.pyenumerates candidate cut sets, builds split graphs and rejects non-equivalent candidates.contracts.pycreates canonical semantic identities for strategy-independent region interning.planning.pyvalidates manifests, prunes only globally dominated strategies and performs change-aware selection.residency.pywraps lazy callable loading with a hard estimated-residency budget and LRU eviction accounting.
Each placeholder is labelled by its public input name. Dependency sets propagate through all graph nodes to outputs. Candidate boundaries require a producer whose dependency set is a strict subset of the full input set, downstream convergence into a broader dependency set, a materialised tensor contract and enough avoided work to justify enumeration.
Mutation, unsupported effects, unsafe assertions and unrepresentable symbolic constraints are rejected. The compiler does not silently treat unknown graph behaviour as pure.
A strategy is an ordered callable recipe, not a separate cache namespace. Reusable prefixes are interned by canonical identity; strategy-local fusion tails remain distinct when their graphs or ports differ. Each candidate is executed against the monolith on the example inputs using the configured numerical contract before it is emitted.
The manifest always contains:
monolithic → (main);- ordered public input contracts;
- complete callable input and output contracts;
- canonical cache identity for every callable;
- strategy costs, retained-cache bytes and estimated resident bytes;
- explicit rejection reasons for illegal candidates.
Input versions determine the changed-input mask. A cache key combines canonical callable identity with versions for exactly its dependencies. Selection considers predicted latency, cold/warm state, current cache hits, load/specialisation cost, logical residency, retained tensor bytes and online exponentially weighted observations.
The monolith is mandatory when all inputs changed unless a strategy is explicitly forced for validation. Before execution, a preselected decision is revalidated against current versions, cache context and both hard budgets. A forged or stale decision fails.
Strategy A globally dominates B only if A is no worse for every change mask and cache/residency state across steady, cold, p95, all-changed, load, specialisation, cache, function-count and memory cost—and A's callable set is a subset of B's.
The callable-subset condition is not optional. Without it, B can be preferable when its private functions are already resident or its private result cache is hot. Correctness-safe pruning removed zero strategies from both final 17-strategy portfolios.
The native packager places canonical regions and strategy fusion tails into a multifunction .aimodel. Named functions are loaded lazily. Python handle eviction is observable and correctly budgeted; physical backing-store release is not exposed by Core AI. The final CLIP result shows that this abstraction boundary prevents active-region memory scaling.