perf(core/vm): flag-gated shared precompile result caches for serial import + block builder, widen keccak cache

[lucca30]

What

Wires bor's existing per-block shared VM result caches (ECRECOVER + KECCAK256) into the two execution paths that currently lack them — the serial import processor and the block builder (miner/worker.go) — and widens the 64-byte-only keccak cache to cover the small variable-length keccak inputs that dominate Polygon's uncached VM compute time. All new behavior is behind a default-off flag (--precompile-cache); with the flag off, block validation and production are byte-for-byte unchanged.

Everything is content-keyed (never tx-index/order-keyed), which is what makes it correct on the build path where mempool order ≠ sealed order, and the caches are per-block / per-building-cycle (matching the existing sharedBlockCaches pattern), so fork-safety is free and there is no cross-block staleness.

Why (measured scope)

We instrumented a mainnet copy-node (content-keyed observer, never returns cached results, cannot affect consensus) and measured per-op call count × per-call time over a sampled window of mainnet blocks. Ranked by total observed time, KECCAK256 + ECRECOVER are the dominant cacheable VM compute — and both are already cached in production, but only for the import prefetch↔parallel paths, not for serial import or the builder. KECCAK256 non-64B inputs (avg ~88 B, clustered ≤ ~350 B) are ~60k calls/block and not cached at all today.

Caveats stated honestly: this is a single-window sample, not a multi-day census; the observer's raw totals include the losing racing processor (serial vs parallel), so absolute numbers are directional and should be re-confirmed with a cache-on-vs-off pprof/wall-clock cross-check. MODEXP/BN256/BLS/KZG/blake2F are fully reachable (they live in the active precompile sets) but were called ~never in-sample, so a geth-style cross-block LRU for them is not the first-dollar investment — see Future work. This is an MVP prioritization, not a proof the broader set never matters.

How

• New length-aware widened keccak store (core/vm/shared_cache.go), backing store chosen by a committed -benchmem microbench (sharded map[string]common.Hash + RWMutex: 12.5 ns/op, 0 B/0 allocs on Load, beating sync.Map-string and fixed-buckets). Per-block entry cap (stop-inserting-when-full) bounds adversarial memory.
• Exported vm.SharedResultCaches owner replacing the unexported core.sharedBlockCaches, so both core and miner can construct it. ApplyTo always wires the legacy jumpDest/keccak-64B/ecrecover caches (preserving today's import prefetch↔parallel sharing regardless of the flag); it wires the widened store + sets the flag only when extended.
• Widened opKeccak256: variable-length inputs hit the widened store behind the flag. Length-aware key (a short input padded to a wider size must never alias — keccak(x) ≠ keccak(x‖0x00)); value stored as common.Hash by value; preimage recorded on hit for every size; gas unchanged (charged by the interpreter before the opcode; a hit skips only computation).
• Serial import: shares the per-block caches behind the flag.
• Build path (own commit): one per-cycle cache set, created on genParams before the prefetch goroutine launches, shared by the build prefetcher and the sealing EVM. Only those two EVMs are wired; pre-tx/system EVMs, FinalizeAndAssemble, and Bor state-sync are intentionally excluded (documented).
• ECRECOVER override guard: the address-keyed ecrecover cache is never consulted on an EVM whose precompiles were overridden (RPC/simulate/tracer via SetPrecompiles), plus a defensive clone against cached-output aliasing.
• Metrics: vm/cache/keccak/{hit,miss,entries,bytes_total,lock_wait}, vm/cache/ecrecover/{hit,miss} (distinct from the state-reader chain/cache/* meters).

Testing

• Import differential — blocks exercising real ecrecover + keccak sizes {32,64,88,100,128,150,200} produce byte-identical state root/receipts/gas flag-on vs flag-off (serial path forced).
• Build differential — sealed block identical flag-on vs flag-off, with two identical-calldata txs forcing genuine keccak + ecrecover cache hits during building.
• Length-collision, all-sizes hit==miss, preimage-on-hit, adversarial memory cap (incl. concurrent -race), ecrecover override-exclusion + hit/miss no-alias, fork-boundary (crosses a real hardfork gate), fuzz (widened cache vs direct keccak), and -race on concurrent prefetch+seal sharing one cache set.

Result — overnight A/B on a mainnet copy-node

Validated on a mainnet copy-node in producer mode over a 12h interleaved A/B (24 alternating
30-min rounds, 12 per arm, ~26k blocks). Arm A = flag off, arm B = flag on; only
--precompile-cache differs between arms.

--precompile-cache gives a ~8.2% block-build throughput win (build-loop MGas/s, measured
from the Commit new sealing work log line's gas=/elapsed= fields). The effect is robust:
every arm-B round beat every arm-A round (no overlap; A 244.8 ± 6.4 vs B 264.8 ± 7.3 MGas/s on a
round-mean basis). It is not a block-size artifact — recomputed at matched block size across
fine gas bands, arm B wins in 10/10 bands (+6.4% to +11.7%), with the gap widening on larger
blocks, consistent with a genuine per-tx cache benefit.

This is a CPU-latency optimization on the build path, not a protocol-ceiling change: it does not
alter gas limits or block time, and translates to producer throughput only insofar as block
production is build-time-bound. Per-EVM hit attribution is not wired, so the win is not decomposed
into cross-EVM prefetcher→sealer sharing vs. intra-sealer per-block caching — both are real and
both are enabled by this flag.

Full A/B methodology, per-round data, and the measurement harness are retained internally (PoS
block-production instrumentation campaign) and available to reviewers on request.

Rollout

Flag off by default. Deploy flag-off first (verify no-op), then enable on a non-producer node for the import-path changes, and only then on a dedicated producer canary for the build-path cache (the build commit is isolated precisely because a cache bug there makes a validator produce an invalid block). Watch vm/cache/*, block-execution timers, lock_wait, and producer block validity.

Future work (tracked, not built here)

• Broaden the content-keyed per-block result cache to the rest of the precompile set (MODEXP, BN256, BLS12-381, KZG, blake2F) if/when Polygon traffic for them grows — with explicit fork/rules set-identity namespacing (geth #35388's approach), since build-prefetch and build-work headers are constructed separately.
• Evaluate a cross-block LRU only against a real cross-block recurrence measurement (the ecrecover key includes the signed message hash, so same-signer/different-message misses across blocks — cross-block reuse is unmeasured, not disproven).
• The copy-node CallObserver (not shipped here) remains the trigger to re-measure if traffic shifts.
• Memoize hot pure-EVM sub-computations from high-traffic contracts. Beyond precompiles, a
  meaningful share of build-time CPU is ordinary EVM bytecode re-executed with identical inputs by
  the busiest contracts on the chain (hot view/pure paths, fixed-point/math libraries,
  signature- and Merkle-proof verifiers written in Solidity). A content-keyed per-block result
  cache keyed on (code identity, input) over a curated allowlist of such hot code segments could
  extend the same win into pure-EVM territory. Prerequisites: (a) a profiling pass to rank
  high-recurrence pure snippets per contract from real traffic; (b) a soundness argument that each
  cached region is a genuine pure function of its inputs — no SLOAD/SSTORE, external CALL, or
  environment/opcode state dependence — before a hit may substitute for execution; (c) gas charging
  left completely untouched (a hit skips only computation, never metering). Speculative and
  unmeasured — a direction, not a committed design.

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

_{Tip: disable this comment in your organization's Code Review settings.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 54.11%. Comparing base (55cac19) to head (3771afe).

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #2318      +/-   ##
===========================================
+ Coverage    54.07%   54.11%   +0.03%     
===========================================
  Files          907      908       +1     
  Lines       162012   162136     +124     
===========================================
+ Hits         87615    87740     +125     
+ Misses       68984    68982       -2     
- Partials      5413     5414       +1     

Files with missing lines	Coverage Δ
core/blockchain.go	63.04% <100.00%> (-0.04%)	⬇️
core/vm/evm.go	46.33% <100.00%> (+4.15%)	⬆️
core/vm/instructions.go	83.94% <100.00%> (+0.84%)	⬆️
core/vm/interpreter.go	67.45% <ø> (ø)
core/vm/shared_cache.go	100.00% <100.00%> (ø)
eth/backend.go	54.41% <100.00%> (-0.21%)	⬇️
eth/ethconfig/config.go	78.94% <ø> (ø)
internal/cli/server/config.go	63.95% <100.00%> (+0.05%)	⬆️
internal/cli/server/flags.go	100.00% <100.00%> (ø)
miner/worker.go	78.36% <100.00%> (+0.49%)	⬆️

... and 23 files with indirect coverage changes

Files with missing lines	Coverage Δ
core/blockchain.go	63.04% <100.00%> (-0.04%)	⬇️
core/vm/evm.go	46.33% <100.00%> (+4.15%)	⬆️
core/vm/instructions.go	83.94% <100.00%> (+0.84%)	⬆️
core/vm/interpreter.go	67.45% <ø> (ø)
core/vm/shared_cache.go	100.00% <100.00%> (ø)
eth/backend.go	54.41% <100.00%> (-0.21%)	⬇️
eth/ethconfig/config.go	78.94% <ø> (ø)
internal/cli/server/config.go	63.95% <100.00%> (+0.05%)	⬆️
internal/cli/server/flags.go	100.00% <100.00%> (ø)
miner/worker.go	78.36% <100.00%> (+0.49%)	⬆️

... and 23 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.