Skip to content

fix: stop sync health check claiming a rebuild on a built index; 0.11.1:16 → 0.11.1:17 - #80

Merged
MattDHill merged 1 commit into
masterfrom
fix/sync-health-check-false-rebuild
Aug 3, 2026
Merged

fix: stop sync health check claiming a rebuild on a built index; 0.11.1:16 → 0.11.1:17#80
MattDHill merged 1 commit into
masterfrom
fix/sync-health-check-false-rebuild

Conversation

@helix-nine

Copy link
Copy Markdown

Problem

The sync health check confirms sync positively — only a real JSON-RPC result from server.banner counts — because electrs services RPC only between indexing batches, so silence during a build is the norm. Reading silence as success reported "Fully synced" all through the initial build, which is what 0354ef8 fixed.

The converse was never handled: silence was read as an unbuilt index. Past the first success that is wrong. Indexing a single block, or the RocksDB compaction behind it, blocks the RPC loop past the 10 s read timeout on modest hardware, and the check flipped a fully-synced server to:

Electrs is building its address index. This can take several hours on first run.

Found while diagnosing a customer report of electrs "getting stuck". On their aarch64 box with a 62 GB index this fired 53 times in six days, once for 118 minutes — while electrs was up and answering wallet queries throughout. 9 of 20 flaps on one day landed while electrs was merely mid-indexing. The message invites the user to reindex a perfectly good index, which on that hardware is days of pointless work.

Fix

Record the first success in store.json (everSynced) and branch on it:

  • Retry before concluding (3 attempts once built, 1 before) — a single blip is not a sync regression. Before the first success a non-answer is expected, so retrying there would only add container-exec churn during the build.
  • Never claim a rebuild once the index exists — report that electrs is busy instead.

Behaviour before the first success is unchanged, so the property 0354ef8 established still holds: silence is still never reported as synced.

New string translated for all four non-default languages; release notes in all five.

Docs

README.md and instructions.md described a cookie-aware "Waiting for Bitcoin to finish syncing" / "Waiting for Bitcoin to start" fallback on the Electrum Server check that main.ts does not implement, and claimed the Electrum port is not served until sync completes — electrs binds the listener before it connects to bitcoind. Both corrected here alongside the new message, since they sit in the section this change rewrites.

Test plan

  1. Fresh install (or wipe the main volume) with Bitcoin still in IBD, and start Electrs.
    • Sync Progress shows "Electrs is building its address index. This can take several hours on first run." — unchanged from :16.
    • Confirm it does not show "Fully synced" at any point during the build.
  2. Let the index finish. Sync Progress reaches "Fully synced" and the Sync Complete notification fires once.
    • Verify everSynced is now true:
      cat /media/startos/data/package-data/volumes/electrs/data/store.json
  3. The regression this fixes. With the index built, make electrs miss a probe — easiest is to watch a natural block arrival on a slow disk, or start-cli package attach electrs -n electrs -- kill -STOP <electrs pid> for ~40 s and then -CONT.
    • Sync Progress must report "Electrs is not responding. It is likely busy indexing; this usually clears on its own."
    • It must never report the "building its address index / several hours" message.
    • It returns to "Fully synced" on its own once electrs answers again.
  4. Persistence across restart. Restart the service and repeat step 3.
    • The busy message (not the build message) still appears, confirming everSynced was re-seeded from store.json rather than reset.
  5. Wallet-visible behaviour is unaffected — point an Electrum wallet at the Main interface (SSL, 50002) and confirm balance/history still resolve normally through a flap.

Notes

  • Does not address the underlying stall this customer hit: electrs shares one p2p connection between query-driven block fetches and new-block announcements, so a wallet polling blockchain.scripthash.get_history on unsubscribed scripthashes starves the chain-following loop. That is upstream electrs architecture, not packageable. This PR only stops StartOS misreporting the result.
  • No migration — behaviour-only change, so current.ts is edited in place.

🤖 Generated with Claude Code

….1:16 → 0.11.1:17

The sync check confirms sync positively — only a real JSON-RPC result from
server.banner counts — because electrs services RPC only between indexing
batches, so silence during a build is the norm and reading it as success
reported "Fully synced" all through the initial build (0354ef8).

The converse was never handled: silence was read as an unbuilt index. Past
the first success that is wrong. Indexing one block, or the RocksDB
compaction behind it, blocks the RPC loop past the 10s read timeout on
modest hardware, and the check flipped a fully-synced server to "Electrs is
building its address index. This can take several hours on first run."
A customer's aarch64 box with a 62 GB index did this 53 times in six days,
once for 118 minutes — while electrs was up and answering wallet queries
the whole time. The message invites reindexing a perfectly good index.

Record the first success in store.json (everSynced) and branch on it: retry
before concluding, since one blip is not a sync regression, and never claim
a rebuild once the index exists — report that electrs is busy instead.
Behaviour before the first success is unchanged, so the property 0354ef8
established still holds.

Also corrects the health-check docs in README.md and instructions.md, which
described a cookie-aware "Waiting for Bitcoin to finish syncing" fallback on
the Electrum Server check that main.ts does not implement, and claimed the
Electrum port is not served until sync completes — electrs binds the
listener before it connects to bitcoind.
@MattDHill
MattDHill merged commit 91de13a into master Aug 3, 2026
3 checks passed
@MattDHill
MattDHill deleted the fix/sync-health-check-false-rebuild branch August 3, 2026 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants