fix(logs-source): follow the heads of upstreams that can serve logs - #398
Merged
Merged
Conversation
l0gun0v
marked this pull request as ready for review
October 2, 2026 09:54
The logs source announced blocks from the chain's merged head, which may come from an upstream without eth_getLogs. At that moment no upstream with the method had the block yet, so its logs were skipped for every subscriber (80-95% of ethereum blocks on drpc-core after eth_getLogs was disabled on the faster of two upstreams). - The block-update stream follows the highest head among the upstreams that fetchBlockLogs may select (available, with eth_getLogs), re-read on every chain state event and every 50 ms. - Heights that head jumps over, and the new chain after a reorg, are fetched by parent hash (eth_getBlockByHash, up to 32 blocks) and announced in order; a deeper gap or a failed fetch announces the head with the gap. - The source checks LogsCap every second as well, since blocks stop arriving once no upstream has eth_getLogs. - Metrics: logs_source_backfill_failed_total, logs_source_head_lag_blocks.
l0gun0v
force-pushed
the
logs-source-no-silent-skip
branch
from
October 2, 2026 10:27
ef40f1c to
645f727
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The local
logssubscription source announces blocks from the heads of the upstreams that can serve them, instead of the chain's merged head, and backfills heights that head jumps over.Seen on ethereum
drpc-core: the merged head came from the faster of two upstreams, which haseth_getLogsdisabled (infra-bcn #1190). At that moment the upstream with the method did not have the block yet, so 80–95% of blocks were skipped and every logs subscriber on the instance lost those events.Spec
StreamBlockUpdatesfollows the highest head among the upstreams thatfetchBlockLogsmay select —Availableand witheth_getLogs, the same matchers — not the merged head. Heads are re-read on every chain state event and every 50 ms (the chain state publishes only when the merged head moves, not when such an upstream catches up). On a tie at the top height the head already followed is kept. Logs therefore lag the chain head by the lag of the best upstream witheth_getLogs; blocks it serves are no longer skipped.parentHashwitheth_getBlockByHash(up to 32, from upstreams at that height) and announced oldest-first through the regular tracker logic. This also announces the new canonical block below a reorged tip, which was previously never emitted. A deeper gap or a failed fetch announces the head with the gap, as before, with a warning andbackfill_failed_total.fetchBlockLogs(3 attempts down the rating list, then the block is skipped), reorg handling, fan-out. The source still terminates when the chain losesLogsCap; it now also checks that every second, since blocks stop arriving once no upstream haseth_getLogs.nodecore_logs_source_backfill_failed_total{chain}andnodecore_logs_source_head_lag_blocks{chain}(merged head minus logs head).logssource is affected;newHeads, pending-tx sources and node-backed subscriptions are untouched.Changes
internal/upstreams/flow/subengine/blockupdates.goStreamBlockUpdatesfollowsbestHeadof eligible upstreams (event + 50 ms recheck);needsParent,advanceWithAncestors,BlockResolver; new metricsinternal/upstreams/flow/logs_source.gocanServeLogs(status +eth_getLogsmatchers),blockByHashResolver, periodicLogsCapcheckinternal/upstreams/flow/subengine/blockupdates_test.gobestHeadinternal/upstreams/flow/logs_source_internal_test.goeth_getLogs(end to end), backfill and its failure, reorg with the reorged-in parent, termination wheneth_getLogsis gonedocs/nodecore/08-prometheus-metrics.md,docs/nodecore/13-subscriptions.md