fix(metro): drop Metro's transform cache under bundle=load too - #163
Merged
Conversation
`withStasis` dropped `cacheStores` for a capture or a frozen verify but left
them alone under `bundle=load`, on the theory that a cache hit there costs no
attestation because the transform derives from the bundle's own bytes. It costs
something else: the transform cache decides which modules METRO ITSELF loads.
`new Transformer` computes the global cache key only when the cache is enabled:
const globalCacheKey = this._cache.isDisabled
? ''
: getTransformCacheKey({...}) // metro/src/DeltaBundler/Transformer.js
and `Cache.isDisabled` is just `stores.length === 0`. So that call tree --
`require(transformerPath)`, then `Transformer.getCacheKey()` reaching
`metro-transform-plugins.getTransformPluginCacheKeyFiles()` and its
`require.resolve()` calls -- runs in load mode and never during a capture,
whose cache we had dropped. The capture cannot record edges it never resolved,
so the replay fails closed on them:
Failed to construct transformer: Error: Cannot find module 'metro-transform-worker'
imported from node_modules/metro/src/DeltaBundler/getTransformCacheKey.js
at State.getImport (stasis-core/src/state.js)
at resolve (stasis-core/src/hooks.js)
Metro swallows that into "Failed to construct transformer" and then dies in
`Bundler.end` on the half-built instance. Whether a given resolution throws
depends on which resolver it reaches -- the fail-closed getImport path, or the
Module._resolveFilename shim, which falls through to disk for an edge the bundle
lacks -- so it presents intermittently.
Dropping the stores in every active mode makes the loaded set independent of
cache state, which is the same reason a capture drops them. Measured against
real Metro 0.87.0 (4 workers, `--child-process`, withStasis):
* before: capture runs with stores=0, isDisabled=true, getTransformCacheKey
never called; the `--lock=frozen --bundle=load` replay runs with stores=1,
isDisabled=false, getTransformCacheKey CALLED -> ERR_MODULE_NOT_FOUND.
* after: both run with stores=0 and never call it; the replay exits 0 and
writes its bundle, 3 for 3. The capture's attested set is unchanged
(105 buckets, 639 files, 1243 formats, same hashes).
The bundle=load test asserted the old behaviour, with the reasoning this
disproves, so it is inverted rather than deleted: a replay must drop the stores,
and a caller's own stores being replaced must still be announced.
Cost: a load build no longer reuses transforms. For `--lock=frozen` that is a
gain -- #152 already notes a verify over a warm cache "passes vacuously, it
verifies transforms nobody performed" -- but a long-lived `--bundle=load` dev
server now re-transforms on every start. Keeping the cache there needs the
capture to run the cache-key path instead, i.e. a per-run `cacheVersion` nonce
with the stores left in place (the mechanism in #156), which is a follow-up.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TMPW6hy7z3BTnJTJjS8k7k
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The error
The file is in the bundle and on disk. Only the edge is missing — nothing ever recorded that
getTransformCacheKey.jsimportsmetro-transform-worker, so load mode fails closed on it.Why the edge can't exist
withStasisdroppedcacheStoresfor a capture or a frozen verify but left them alone underbundle=load, reasoning that a hit there costs no attestation because the transform derives from the bundle's own bytes. It costs something else: the transform cache decides which modules Metro itself loads.Cache.isDisabledis exactlystores.length === 0. SogetTransformCacheKey()— and its whole call tree:require(transformerPath), thenTransformer.getCacheKey()reachingmetro-transform-plugins.getTransformPluginCacheKeyFiles()and itsrequire.resolve()calls — runs only when the cache is enabled. A capture has it disabled, so it never executes that tree and cannot record its edges. The replay has it enabled, executes it for real, andgetImportrejects an edge no capture could have produced.Instrumented at that exact branch on the repro (real Metro 0.87.0, 4 workers,
--child-process,withStasis):isDisabledgetTransformCacheKey--lock=add --bundle=add--lock=frozen --bundle=load(before)ERR_MODULE_NOT_FOUND--lock=frozen --bundle=load(after)Metro swallows the throw into
Failed to construct transformer:and then dies inBundler.endon the half-built instance (Cannot read properties of undefined (reading 'end')).Why it's intermittent: which module in that tree throws first depends on which resolver the resolution reaches — the fail-closed
getImportpath, or theModule._resolveFilenameshim, which falls through to disk for an edge the bundle lacks. My repro first failed onrequire.resolve('./index.js')frommetro-transform-plugins/src/index.js; the report above failed one frame earlier onrequire(transformerPath). Same cause, and one run out of several passed outright.The fix
needsTransformsno longer excludesloadBundle, so the stores are dropped in every active mode. The loaded set then can't diverge between record and replay — the same reason a capture drops them.Capture behaviour is untouched: the attested set is identical before and after (105 buckets, 639 files, 1243 formats, same file/format hashes), and the replay now exits 0 and writes its output bundle.
The
bundle=loadtest asserted the old behaviour, with the reasoning this disproves, so it's inverted rather than deleted: a replay must drop the stores, and replacing a caller's own stores must still be announced.Cost, and the follow-up
A load build no longer reuses transforms. For
--lock=frozenthat's a gain — #152 already notes a verify over a warm cache "passes vacuously, it verifies transforms nobody performed", and theloadBundleexclusion quietly re-admitted that forfrozen+load, which is the standard verify invocation. But a long-lived--bundle=loaddev server now re-transforms on every start.Keeping the cache under load requires the capture to run the cache-key path so those edges get recorded: leave the stores in place and force a full miss with a per-run
cacheVersionnonce — the mechanism already in #156 for hand-wired configs. That's strictly better on speed and stops overriding the user'scacheStoresat all, but it's a bigger change; this PR fixes the broken build first.Generated by Claude Code