From cb90230961b30457046a58c0104654b780be58af Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 28 Jul 2026 19:14:20 +0000 Subject: [PATCH] fix(metro): drop Metro's transform cache under bundle=load too `withStasis` dropped `cacheStores` for a capture or a frozen verify but left them alone under `bundle=load`, on the theory that a cache hit there costs no attestation because the transform derives from the bundle's own bytes. It costs something else: the transform cache decides which modules METRO ITSELF loads. `new Transformer` computes the global cache key only when the cache is enabled: const globalCacheKey = this._cache.isDisabled ? '' : getTransformCacheKey({...}) // metro/src/DeltaBundler/Transformer.js and `Cache.isDisabled` is just `stores.length === 0`. So that call tree -- `require(transformerPath)`, then `Transformer.getCacheKey()` reaching `metro-transform-plugins.getTransformPluginCacheKeyFiles()` and its `require.resolve()` calls -- runs in load mode and never during a capture, whose cache we had dropped. The capture cannot record edges it never resolved, so the replay fails closed on them: Failed to construct transformer: Error: Cannot find module 'metro-transform-worker' imported from node_modules/metro/src/DeltaBundler/getTransformCacheKey.js at State.getImport (stasis-core/src/state.js) at resolve (stasis-core/src/hooks.js) Metro swallows that into "Failed to construct transformer" and then dies in `Bundler.end` on the half-built instance. Whether a given resolution throws depends on which resolver it reaches -- the fail-closed getImport path, or the Module._resolveFilename shim, which falls through to disk for an edge the bundle lacks -- so it presents intermittently. Dropping the stores in every active mode makes the loaded set independent of cache state, which is the same reason a capture drops them. Measured against real Metro 0.87.0 (4 workers, `--child-process`, withStasis): * before: capture runs with stores=0, isDisabled=true, getTransformCacheKey never called; the `--lock=frozen --bundle=load` replay runs with stores=1, isDisabled=false, getTransformCacheKey CALLED -> ERR_MODULE_NOT_FOUND. * after: both run with stores=0 and never call it; the replay exits 0 and writes its bundle, 3 for 3. The capture's attested set is unchanged (105 buckets, 639 files, 1243 formats, same hashes). The bundle=load test asserted the old behaviour, with the reasoning this disproves, so it is inverted rather than deleted: a replay must drop the stores, and a caller's own stores being replaced must still be announced. Cost: a load build no longer reuses transforms. For `--lock=frozen` that is a gain -- #152 already notes a verify over a warm cache "passes vacuously, it verifies transforms nobody performed" -- but a long-lived `--bundle=load` dev server now re-transforms on every start. Keeping the cache there needs the capture to run the cache-key path instead, i.e. a per-run `cacheVersion` nonce with the stores left in place (the mechanism in #156), which is a follow-up. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01TMPW6hy7z3BTnJTJjS8k7k --- stasis-plugins/src/metro.js | 44 ++++++++++++++++++++++--------------- tests/metro.test.js | 21 ++++++++++++------ 2 files changed, 40 insertions(+), 25 deletions(-) diff --git a/stasis-plugins/src/metro.js b/stasis-plugins/src/metro.js index 4aff821..2aedbe9 100644 --- a/stasis-plugins/src/metro.js +++ b/stasis-plugins/src/metro.js @@ -172,14 +172,13 @@ export class StasisMetro { this.#resources = state?.config.resources ?? new Set() } - // True when this build's guarantees depend on Metro actually TRANSFORMING files: any active - // lockfile/bundle mode either records what the workers load (capture) or checks it (frozen). - // False when inert, when attesting nothing ('none'/'ignore' -- what useLockfile/bundle exclude), - // or under bundle=load, where the transform derives from the bundle's own bytes so a cache hit - // costs nothing. Read by withStasis to decide whether Metro's transform cache must be dropped. + // True when this build's guarantees depend on Metro actually TRANSFORMING files: a capture records + // what the workers load, frozen checks it, and load REPLAYS it -- all three break if a cached + // transform skips the work. False only when inert or attesting nothing ('none'/'ignore', what + // useLockfile/bundle exclude). Read by withStasis to decide whether to drop the transform cache. get needsTransforms() { const config = this.#state?.config - if (!config || config.loadBundle) return false + if (!config) return false return config.useLockfile || config.bundle } @@ -226,7 +225,7 @@ export class StasisMetro { "[stasis] StasisMetro: wired without withStasis(), so Metro's transform cache is outside " + `stasis's control -- a cached transform skips the worker and the toolchain it loads ` + `(${WORKER_TOOLCHAIN}) goes unattested. Wrap your config in withStasis(), or set ` + - '`cacheStores: []` yourself for capture/frozen builds.' + '`cacheStores: []` yourself for every stasis build.' ) } this.#capture(graph, preModules) @@ -452,20 +451,29 @@ export class StasisMetro { // Idiomatic Metro-config wrapper: returns a new config with `serializer.customSerializer` wired // so stasis captures the graph + preModules while your existing serializer (or Metro's default) -// still produces the bundle, and (for a capture/verify) with Metro's transform cache dropped so the -// workers really run. Pure -- returns a new object, doesn't mutate `config`. +// still produces the bundle, and with Metro's transform cache dropped so the workers really run. +// Pure -- returns a new object, doesn't mutate `config`. export function withStasis(config = {}, options = {}) { const stasis = new StasisMetro(options, { ownsConfig: true }) const existing = config.serializer?.customSerializer ?? undefined - // Metro's transform cache is a CORRECTNESS hazard here, not a speed knob. On a cache hit Metro - // returns the stored result WITHOUT calling a worker, so the worker-side toolchain is never loaded - // in any process and never reaches the root -- exactly what the constructor's --child-process - // assert exists to guarantee. The default store is a FileStore in the OS tmpdir shared with every - // other Metro run, so ONE earlier `metro build`/dev server silently decides what gets attested: a - // capture over a warm cache drops the toolchain (exit 0, no warning) and a later frozen run - // rejects the lockfile it wrote, while a frozen run over a warm cache passes vacuously (it - // verifies transforms nobody performed). Drop the stores so every file is transformed for real -- - // slower, but the attested set stops depending on what ran before. Capture is one-shot anyway. + // Metro's transform cache is a CORRECTNESS hazard here, not a speed knob: it decides which modules + // Metro itself LOADS. On a cache hit Metro returns the stored result without calling a worker, so + // the worker-side toolchain is never loaded in any process and never reaches the root -- exactly + // what the constructor's --child-process assert exists to guarantee. The default store is a + // FileStore in the OS tmpdir shared with every other Metro run, so ONE earlier `metro build`/dev + // server silently decides what gets attested: a capture over a warm cache drops the toolchain + // (exit 0, no warning) and a later frozen run rejects the lockfile it wrote, while a frozen run + // over a warm cache passes vacuously (it verifies transforms nobody performed). + // + // Dropped in EVERY active mode, load included, so the module set can't diverge between record and + // replay. `new Transformer` calls getTransformCacheKey() only when the cache is enabled + // (metro/src/DeltaBundler/Transformer.js: `this._cache.isDisabled ? '' : ...`), and that call tree + // resolves the transformer plus its plugins' cache-key files. Disabled during capture, those edges + // are never recorded; left enabled under load they resolve for real and getImport fails closed on + // a bundle that cannot know them ("Cannot find module 'metro-transform-worker' imported from + // .../getTransformCacheKey.js" -- Metro reports it as "Failed to construct transformer"). + // + // Slower, but the loaded set stops depending on what ran before. These builds are one-shot anyway. const dropCache = stasis.needsTransforms // `undefined` is Metro's own default store, which the user never chose; an array or a // `(MetroCache) => stores` factory is theirs, so replacing it is worth saying out loud. diff --git a/tests/metro.test.js b/tests/metro.test.js index efba0d6..5b32b14 100644 --- a/tests/metro.test.js +++ b/tests/metro.test.js @@ -428,8 +428,8 @@ describe('StasisMetro (spawned, concurrent)', { concurrency: CONCURRENCY }, () = // capture's lockfile is then REJECTED by a cold-cache frozen run ("observed resolution // '../../../jest-util/build/index.js' from ... processChild.js is not attested by the lockfile"). // A frozen verify over a warm cache is the mirror image: it passes vacuously, verifying transforms - // nobody performed. Hence both modes drop the stores -- a capture to RECORD what the workers load, - // a verify to CHECK it. + // nobody performed. Hence every active mode drops the stores -- a capture to RECORD what the + // workers load, a verify to CHECK it, a load to REPLAY it (see the bundle=load test below). for (const [mode, env, dropLock] of [ ['a CAPTURE', () => withOpts({ lock: 'add' }), true], ['a frozen VERIFY', () => ({ EXODUS_STASIS_LOCK: 'frozen', EXODUS_STASIS_SCOPE: 'full' }), false], @@ -472,9 +472,16 @@ describe('StasisMetro (spawned, concurrent)', { concurrency: CONCURRENCY }, () = t.assert.match(r.stdout, /^CACHE_STORES_INPUT=\["user-store"\]$/m, 'withStasis must not mutate the input config') })) - test('withStasis leaves the Metro transform cache alone under bundle=load', withTmp(async (t, tmp) => { - // Load mode derives each transform from the BUNDLE's bytes, so a cache hit costs no attestation - // (and load results are keyed apart by the transformer's getCacheKey marker). Nothing to drop. + test('withStasis drops the Metro transform cache under bundle=load too', withTmp(async (t, tmp) => { + // Load mode used to keep the cache, on the theory that a hit costs no attestation because the + // transform derives from the bundle's bytes. It costs something else: the cache decides which + // modules METRO loads. `new Transformer` calls getTransformCacheKey() only when the cache is + // enabled (Transformer.js: `this._cache.isDisabled ? '' : ...`), and that call tree resolves the + // transformer plus its plugins' cache-key files. A capture -- cache dropped -- never runs it, so + // those edges are in no lockfile or bundle, and the replay that does run it fails closed: + // "Cannot find module 'metro-transform-worker' imported from .../getTransformCacheKey.js", + // reported by Metro as "Failed to construct transformer". Confirmed against real Metro 0.87. + // Dropped in every active mode so the loaded set cannot diverge between record and replay. cpSync(fullFixture, tmp, { recursive: true }) const bundlePath = join(tmp, 'snapshot.br') const cap = await run('src/entry.js', { @@ -496,8 +503,8 @@ describe('StasisMetro (spawned, concurrent)', { concurrency: CONCURRENCY }, () = }, }) t.assert.equal(r.status, 0, `stderr: ${r.stderr}`) - t.assert.match(r.stdout, /^CACHE_STORES=\["user-store"\]$/m, "load mode must keep the caller's cache") - t.assert.doesNotMatch(r.stderr, /ignoring the `cacheStores`/u, 'nothing was overridden, so nothing to warn about') + t.assert.match(r.stdout, /^CACHE_STORES=\[\]$/m, 'a replay must load the same modules a capture did') + t.assert.match(r.stderr, /ignoring the `cacheStores` in your Metro config/, 'the override must be announced') })) test('a hand-wired capture warns that Metro\'s transform cache is out of stasis\'s hands', withTmp(async (t, tmp) => {