fix(contains): treat unresolvable LabelOnly as 'maybe present', not absent - #13
fix(contains): treat unresolvable LabelOnly as 'maybe present', not absent#13mwaddip wants to merge 1 commit into
Conversation
a-shannon
left a comment
There was a problem hiding this comment.
Great catch and an elegant fix!
As discussed over in PR #12, failing safe and treating unresolved LabelOnly nodes as 'maybe present' is absolutely critical to protect lazily-materialized database backends from false garbage collection during tree rebalancing.
The logic is perfectly sound, and the scoped RefCell borrow workaround with the Kind enum to avoid runtime panics is very clean, idiomatic Rust. LGTM! 🚀
…bsent
`contains_recursive` previously returned false at the catch-all `else`
arm whenever it reached a non-Internal node. That branch fires for both
Leaf nodes (terminal — return false is correct) and LabelOnly nodes
that the resolver could not materialise (return false is *unsafe*).
Used by `removed_nodes()` to decide which digests to delete from
persistent storage:
for cn in &self.base.changed_nodes_buffer_to_check {
if !self.contains(cn) {
self.base.changed_nodes_buffer.push(cn.clone())
}
}
If `contains()` walks into an unresolvable subtree, returning false says
"definitely not in tree → delete it". The caller deletes the node from
storage. But the node may still be referenced from the very subtree we
couldn't resolve. The next walk into that subtree hits a LabelOnly with
the deleted digest and bails:
ERROR ergo_sync::state: apply_state failed
error=UTXO state operation failed:
operation N failed: Should never reach this point.
If in prover, this is a bug. If in verifier, this proof is wrong.
Observed in production on a Rust full-node implementation of the Ergo
mainnet at v0.4.x. After ~250 blocks of steady-state validation with
the typical "most subtrees are LabelOnly, only walked paths are
materialised" prover state, an on-disk scan revealed:
Reachable nodes: 6,353,404
Missing references: 135 (parent in storage, child digest not in NODES_TABLE)
Orphan nodes: 378,274 (in storage but unreachable from root)
Fix: distinguish Leaf from LabelOnly at the catch-all. Leaf with
non-matching label remains terminal → false. LabelOnly that the
resolver couldn't materialise → true (fail safe). At worst this leaks
orphan nodes; never silently corrupts.
Refactored to use a `Kind` enum so we can scope the immutable borrow
that does the discrimination separately from the mutable borrow that
drives the Internal-node walk.
All 22 existing tests pass unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
69765ef to
2a08c89
Compare
|
Force-pushed: rustfmt applied, folded into the existing commit so this stays a single commit.
Unrelated: |
Bug
AVLTree::contains_recursive(used byBatchAVLProver::removed_nodes) returnsfalsewhen the walk reaches any non-Internalnode whose label doesn't match the target. That catch-all fires for two distinct cases:falseis correct.LabelOnlyplaceholder the resolver couldn't materialize — we cannot conclude anything about the unresolved subtree.falseis unsafe.removed_nodes()interpretsfalseas "definitely not in tree → mark for deletion":For a persistent backend that lazily materializes nodes (most of the in-memory tree is
LabelOnlyafter a few flushes), a candidate-for-deletion's key path frequently crosses an unresolved subtree. Pre-fix, those candidates were marked for deletion even when they remained reachable through the unresolved branch. Subsequent walks into that branch hit aLabelOnlywhose digest the resolver can no longer find and bail at_ => bail!("Should never reach this point. ...")(modify_helper line 382 of master).Reproduction
A Rust full node implementing the Ergo mainnet on top of this crate (with the persistence backend from #10) ran a fresh UTXO snapshot bootstrap to mainnet tip and 250 blocks of steady-state validation. A direct on-disk scan of the resulting NODES_TABLE:
The 135 dangling references are the over-deletions caused by this bug. Each one eventually triggers an apply failure when block validation walks into the missing subtree.
Fix
Distinguish
Leaf(terminal, returnfalse) fromLabelOnly(couldn't determine, returntrueto fail safe). At worst this leaks orphan nodes; never silently corrupts.Refactored to use a
Kindenum so the immutable borrow that does the discrimination is scoped separately from the mutable borrow driving theInternalwalk (avoidsRefCell already mutably borrowed).Tests
All 22 existing tests pass unchanged. No new behaviour change in the no-storage case (where every walk lands on a real
Internal/Leaf).Relationship to other PRs
contains()issue here. Anyone usingprover.removed_nodes()with a persistent backend is affected regardless of which storage layer they use.🤖 Generated with Claude Code