fix: offload bolt11_decode in list_transactions to worker thread#37
Open
BenGWeeks wants to merge 1 commit into
Open
fix: offload bolt11_decode in list_transactions to worker thread#37BenGWeeks wants to merge 1 commit into
BenGWeeks wants to merge 1 commit into
Conversation
Closes lnbits#35 `_on_list_transactions` decodes each payment's BOLT11 invoice with the synchronous `bolt11_decode()` inside the asyncio event loop. On wallets with non-trivial history this blocks the event loop long enough that NWC clients hit their reply timeout (~60s) and LNbits' other async work (relay keep-alives, publishes, concurrent NWC requests) stalls. Move the decode call to a worker thread via `asyncio.to_thread` so the event loop keeps running while each invoice is parsed. Sibling event-loop-starvation fixes: - lnbits/lnbits#3918 (IN_FLIGHT payment polling backoff) - lnbits/lnbits#3925 (non-blocking relay publishes in send_nostr_dm) Together these three changes restore NWC responsiveness on busy LNDRest-backed wallets. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
BenGWeeks
added a commit
to BenGWeeks/lightning-piggy-mobile
that referenced
this pull request
Apr 12, 2026
Swap recovery service:
- Parse lockup tx hex to extract vout+amount (Boltz v2 /swap/{id} doesn't
return these fields; derive them from transaction.hex by matching the
lockup address script).
- initEccLib so bitcoinjs-lib can parse bech32m (taproot) addresses.
- Surface progress via react-native-toast-message: info toasts when
checking/claiming, success when recovered, error with full message when
recovery fails. Custom renderer lets body text wrap so long Electrum
errors (e.g. script-verify-flag-failed) are readable.
Boltz claim (work-in-progress, witness-hash mismatch debugging):
- Attempt MuSig2 key aggregation via @scure/btc-signer (MIT-licensed
alternative to AGPL boltz-core) for Taproot internal key.
- Diagnostic logging compares computed tweaked key vs actual lockup
output script. Currently doesn't match — internal-key derivation
still needs alignment with Boltz v2 (their docs say "Boltz key first"
but our sorted/claim_refund/refund_claim variants all diverge from
on-chain output key; needs boltz-core reference comparison).
LNbits container hot-fix patches (docker/patches/):
- README + apply.sh to re-apply patches after image upgrades.
- lnbits_wallets_lndrest.py — IN_FLIGHT payment polling backoff
(lnbits/lnbits#3918).
- lnbits_core_services_nostr.py — non-blocking relay publishes
(lnbits/lnbits#3925).
- lnbits_extensions_nwcprovider_tasks.py — bolt11_decode in worker
thread (lnbits/nwcprovider#37).
All three fixes target independent event-loop starvation paths that
together rendered NWC unusable on busy wallets. apply.sh requires
LNBITS_HOST to be set (no hard-coded hostname, safe for public repo).
Docs:
- TERMS.adoc: expand Taproot entry with bech32m/bc1p, add bech32m,
P2TR, swap-in/out, lockup/claim/refund tx, preimage, key-path vs
script-path, control block, tapleaf. Add @scure/btc-signer and
react-native-toast-message to dependent libraries table.
Ignore:
- .claude/ runtime state directory.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Collaborator
|
please drop the AI slop description This is not a blocker operation, it runs in constant time. invoice_data = bolt11_decode(p.bolt11)Starting a new thread for each decode might actually degrade the overall performance. invoice_data = await asyncio.to_thread(bolt11_decode, p.bolt11)Is there a before/after performance test done? |
Author
|
Feel free to fix how ever you guys see fit - you know your solution better than I do. The issue was that |
BenGWeeks
added a commit
to BenGWeeks/nwcprovider
that referenced
this pull request
May 7, 2026
Collaborator
|
I see, the TL;DR is basically that you want to yield the event loop, not to make _on_list_transactions faster. Could you please change the PR to reflect that? Or if you prefer i can commit the fix to main branch. Thanks |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #35
_on_list_transactionsintasks.pycalls the synchronousbolt11_decode()inside a Pythonforloop in an async handler. On wallets with non-trivial history this blocks the asyncio event loop long enough that:~60s) and seelist_transactionsfailOther NWC methods (
get_info,get_balance,make_invoice,lookup_invoice) are unaffected because they don't do per-record synchronous CPU work in a loop.Status
Fix
Offload each
bolt11_decodecall to a worker thread viaasyncio.to_thread, so the event loop can keep running while each invoice is parsed. This is the same pattern used in lnbits/lnbits#3925 for analogous synchronous websocket calls insend_nostr_dm.asynciois already imported; no new dependencies.Why this matters
Together with the other two fixes that are already in production testing, this closes the last of three event-loop-starvation paths that collectively rendered NWC unusable on busy LNDRest-backed wallets:
lnbits/core/services/payments.pylnbits/core/services/nostr.pynwcprovider/tasks.pyAlternatives considered
bolt11through and let the client decode. Smaller payload, no server CPU. But it changes the response schema (fieldsdescription,description_hash, and theinvoice_data.datefallback fortimestamp) and would break clients that rely on the current shape. Rejected for backward compatibility.asyncio.sleep(0)yields: works, but still runs all the CPU on the event-loop thread.asyncio.to_threadis cleaner and scales better.Related
Changes
tasks.py: wrapbolt11_decodeinasyncio.to_threadinside_on_list_transactions🤖 Generated with Claude Code