Repository navigation
fix(extension): tb thread always returns empty — getRaw() headers, angle brackets, downstream search - #4
fix(extension): tb thread always returns empty — getRaw() headers, angle brackets, downstream search#4dboeckenhoff wants to merge 5 commits into
Conversation
Three bugs in the thread endpoint and one missing search parameter:
1. getFull() doesn't expose RFC 2822 headers (References, In-Reply-To,
Message-ID) in the headers map in practice. Switch to getRaw() and
parse headers directly from the raw message bytes.
2. Message-IDs extracted from the References header include angle
brackets (<msgid@domain>) but messages.query({ headerMessageId })
expects them stripped. Add stripBrackets() before querying.
3. The algorithm only found upstream messages (those referenced by the
queried message). Replies that arrived after it don't appear in its
own References chain. Add a subject-based pass to catch downstream
messages too. Strips Re:/WG:/AW:/Fwd: prefixes and [list-name]
brackets before searching.
Also expose headerMessageId as a filter in the /messages/search POST
endpoint — messenger.messages.query() supports it but the HTTP layer
didn't pass it through.
- extension/src/thread-utils.js: extract pure functions (parseHeader, stripAngleBrackets, parseReferences, normalizeSubject, buildThreadIds) so they can be unit-tested outside Thunderbird's extension context. Also found and fixed two edge-case bugs: stripAngleBrackets and normalizeSubject now trim whitespace before applying their regexes. - test/thread-utils.test.mjs: 29 unit tests covering RFC 2822 header parsing, bracket stripping, References tokenization, subject normalization (all German/English reply prefixes, list-name brackets, stacking), and full buildThreadIds including forwarded-message bleed. - test/quick-test.mjs: 3 new integration assertions: · headerMessageId is forwarded through /messages/search · /messages/:id/thread returns an array · thread results are sorted by date and count matches length
Problem
Root causes1. The thread handler extracted Fix: parse headers from 2. Angle brackets not stripped before The Fix: strip 3. Downstream messages not found The current algorithm only finds messages referenced by the queried message Fix: after the References pass, run a subject-based Bonus:
|
Destructured only { hours, limit } from body, dropping accountId entirely.
Now extracts accountId; collects limit*5 messages up-front when filtering
(accounts are interleaved in the raw query stream), then filters by
msg.folder.accountId and slices to limit.
Previous approach post-filtered after collecting a capped batch, so accounts with low message volume (e.g. Gmail) got zero results if the batch filled up with messages from a busier account (IPP). Fix: pass accountId into collectMessages so filtering happens inside the pagination loop — keeps fetching pages until limit matching messages are found or the result set is exhausted.
|
Thank you @dboeckenhoff — your root-cause analysis (getFull() missing the headers, angle brackets in headerMessageId queries) was exactly right, and the fix ships in 1.1.0 via #25 with you as co-author. Verified against a live Thunderbird with Gmail threads. A few adjustments made during integration:
Closing in favour of #25. |
… bounded parallelism Integrates the extension parts of PR vitalio-sh#4 and PR vitalio-sh#20, with fixes: Thread (from vitalio-sh#4) - Parse References / In-Reply-To / Message-ID from getRaw() (getFull() doesn't expose them) and strip angle brackets before messages.query({ headerMessageId }) — previously `tb thread` always returned an empty list. getFull() fallback now strips brackets too. - Downstream replies found by subject are limited to exact normalized subject matches (Thunderbird's query is substring), exclude junk, and are labelled threadMatch: "subject" vs "references". - thread-utils.js is loaded by the manifest and is the single source for the parsing helpers instead of a test-only copy. Recent / search (from vitalio-sh#4) - /recent honours accountId, and now unreadOnly as well; the CLI sent both but the extension ignored them. - /messages/search accepts headerMessageId. Connection (from vitalio-sh#20, reworked) - Reconnect backoff 3s → 6s → 12s → 15s cap (PR used 60s, which would leave a restarted bridge waiting up to a minute), reset on connect, immediate retry on idle → active ("idle" permission). - Socket handlers are bound to their own socket, so a late close from a replaced socket can't null the live one (main crashed on ws.send). Throughput (from vitalio-sh#20) - read-batch, fetch, bulk tag/fetch and thread lookups run through a shared queue capped at 8 concurrent operations, preserving order. - Attachment base64 encoded in 32 KB chunks instead of byte by byte. Not taken from vitalio-sh#20: the 30s folder-info cache (stale unread counts right after mark-read; Thunderbird is the source of truth), the /debug/compose-capabilities endpoint, and the 15s resume-poll interval. test/extension.test.mjs loads the real background scripts in a vm with a mocked messenger API (39 checks); against the previous extension the thread, recent, search and reconnect checks fail. Co-authored-by: Daniel Böckenhoff <dboe@ipp.mpg.de> Co-authored-by: le-dawg <3172733+le-dawg@users.noreply.github.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019uoUWTHVsDJpMJCVNFZz5L
#3