Repository navigation
fix: channel feeds return a channel's own casts, newest first, page by page - #44
Merged
Merged
Conversation
…y page Casts by a parent URL read the casts of every channel whose URL starts with it (/channel/dev-jobs under /channel/dev): the casts-by-parent index key is the URL followed by ts_hash and fid, so a prefix scan reaches longer URLs. Those entries decoded as garbage and were dropped after filling the page, so newest-first pages (which meet them first) came back empty. Index reads now skip keys longer than an entry, in the union and legacy paths. get_casts_by_parent across shards appended each shard's page, so a page held up to page_size casts per shard, out of time order, and a shard that ran out returned no token and was read again from the start on the next page. Pages now merge by timestamp and hash, hold page_size casts, resume each shard after the last of its casts taken, mark finished shards, and end with no token once every shard is done. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
martinvr
approved these changes
Oct 29, 2026
CassOnMars
approved these changes
Oct 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Channel feeds (
/v2/farcaster/feed/channels,/v2/farcaster/feed/parent_urls,/v1/castsByParent?url=…&reverse=true) come back empty or out of order for many channels. Two bugs inget_casts_by_parentcause it.A channel's URL is a prefix of other channels' URLs
The casts-by-parent index key is
[root][parent url][ts_hash][fid], and reads scan the prefix[root][parent url]. For a URL parent that prefix also covers every channel whose URL starts with it:/~/channel/devreachesdev-jobs,devcon,developersand so on. Those entries were read asdeventries, with garbagets_hash/fid, and then dropped when no message was found. They still counted toward the page.A channel's own entries sort before its siblings' (the byte after the URL is the high byte of the timestamp, below any URL character). So oldest-first pages mostly work, but newest-first pages start in the siblings and come back empty. On haatz.quilibrium.com today:
reverse=trueforhttps://warpcast.com/~/channel/devreturns nothing, whilespiritualityandgifart(no longer-named siblings) work;feed/channels?channel_ids=farcasterreturnscasts: [].union_suffixes(index_read.rs) now skips keys whose length isn't exactly an entry's,1 + tail + TS_HASH_LENGTH + FID_BYTES. All of its callers (casts by parent, casts by mention, reactions by target, links by target, and the casts-by-parent count) document that layout. That also fixes reactions on URL targets, which have the same prefix overlap. The legacyget_casts_by_parentpath gets the same check.Siblings' keys are still stepped over during the scan. That's a key iteration with no message reads, and it stops once the page is full.
Shard pages were appended, not merged
get_casts_by_parentin server.rs readpage_sizecasts from each shard and appended the pages. This had three effects:page_size × shardscasts (feed/channels?limit=4returns 8);Noneas its token, so the next page read it again from the start. A client paging to the end saw the same casts again, and the token never ended.It now:
(timestamp, hash), which is index order, and keepspage_sizecasts;next_page_tokenonce every shard is done.The token stays the same JSON list of per-shard keys. Tokens from before this change still work.
The other multi-shard endpoints (
get_casts_by_mention, reactions and links by target…) use the same append pattern. I left them alone to keep this PR to channels. The same merge could move into a shared helper if you want it there.Tests
test_casts_by_parent_url_skips_longer_urls_with_the_same_start(cast_store_test.rs):/channel/devand/channel/dev-jobscasts in legacy and hyper stores. Readingdevin both directions, as one large page or page by page to the end, gives exactly its 3 casts in order. The hyper count is 3.test_get_casts_by_parent_hash(server_tests.rs) now checks exact pages across two shards: sizes 1, 2, 2 give the 4 replies in time order (and reversed withreverse), then no token. Before, it asserted one cast per shard per page.Both fail on
main. On the server test, the third page returns all four replies again.cargo test --lib: everything passes except the 4 storage tests that fail the same way onmain(test_storage_slot_from_rent_event,test_storage_slot_with_mix_of_units,test_user_with_low_total_storage_cannot_lend,test_commit_username_proof_messages).cargo fmt --all --checkis clean, and clippy reports nothing in the new code. I haven't run it on a synced node.