Skip to content

fix: channel feeds return a channel's own casts, newest first, page by page - #44

Merged
CassOnMars merged 1 commit into
farcasterorg:mainfrom
backmeupplz:channel-feeds
Oct 30, 2026
Merged

CassOnMars merged 1 commit into
farcasterorg:mainfrom
backmeupplz:channel-feeds

Conversation

@backmeupplz

@backmeupplz backmeupplz commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Channel feeds (/v2/farcaster/feed/channels, /v2/farcaster/feed/parent_urls, /v1/castsByParent?url=…&reverse=true) come back empty or out of order for many channels. Two bugs in get_casts_by_parent cause it.

A channel's URL is a prefix of other channels' URLs

The casts-by-parent index key is [root][parent url][ts_hash][fid], and reads scan the prefix [root][parent url]. For a URL parent that prefix also covers every channel whose URL starts with it: /~/channel/dev reaches dev-jobs, devcon, developers and so on. Those entries were read as dev entries, with garbage ts_hash/fid, and then dropped when no message was found. They still counted toward the page.

A channel's own entries sort before its siblings' (the byte after the URL is the high byte of the timestamp, below any URL character). So oldest-first pages mostly work, but newest-first pages start in the siblings and come back empty. On haatz.quilibrium.com today:

  • reverse=true for https://warpcast.com/~/channel/dev returns nothing, while spirituality and gifart (no longer-named siblings) work;
  • feed/channels?channel_ids=farcaster returns casts: [].

union_suffixes (index_read.rs) now skips keys whose length isn't exactly an entry's, 1 + tail + TS_HASH_LENGTH + FID_BYTES. All of its callers (casts by parent, casts by mention, reactions by target, links by target, and the casts-by-parent count) document that layout. That also fixes reactions on URL targets, which have the same prefix overlap. The legacy get_casts_by_parent path gets the same check.

Siblings' keys are still stepped over during the scan. That's a key iteration with no message reads, and it stops once the page is full.

Shard pages were appended, not merged

get_casts_by_parent in server.rs read page_size casts from each shard and appended the pages. This had three effects:

  • a page held up to page_size × shards casts (feed/channels?limit=4 returns 8);
  • casts weren't in time order across shards;
  • a shard that ran out returned None as its token, so the next page read it again from the start. A client paging to the end saw the same casts again, and the token never ended.

It now:

  • merges the shards' pages by (timestamp, hash), which is index order, and keeps page_size casts;
  • sets each shard's token to the key of the last of its casts on the page, or keeps its old token if it contributed none;
  • marks a finished shard with an empty token, which is skipped later;
  • returns no next_page_token once every shard is done.

The token stays the same JSON list of per-shard keys. Tokens from before this change still work.

The other multi-shard endpoints (get_casts_by_mention, reactions and links by target…) use the same append pattern. I left them alone to keep this PR to channels. The same merge could move into a shared helper if you want it there.

Tests

  • test_casts_by_parent_url_skips_longer_urls_with_the_same_start (cast_store_test.rs): /channel/dev and /channel/dev-jobs casts in legacy and hyper stores. Reading dev in both directions, as one large page or page by page to the end, gives exactly its 3 casts in order. The hyper count is 3.
  • test_get_casts_by_parent_hash (server_tests.rs) now checks exact pages across two shards: sizes 1, 2, 2 give the 4 replies in time order (and reversed with reverse), then no token. Before, it asserted one cast per shard per page.

Both fail on main. On the server test, the third page returns all four replies again.

cargo test --lib: everything passes except the 4 storage tests that fail the same way on main (test_storage_slot_from_rent_event, test_storage_slot_with_mix_of_units, test_user_with_low_total_storage_cannot_lend, test_commit_username_proof_messages). cargo fmt --all --check is clean, and clippy reports nothing in the new code. I haven't run it on a synced node.

…y page

Casts by a parent URL read the casts of every channel whose URL starts with
it (/channel/dev-jobs under /channel/dev): the casts-by-parent index key is
the URL followed by ts_hash and fid, so a prefix scan reaches longer URLs.
Those entries decoded as garbage and were dropped after filling the page, so
newest-first pages (which meet them first) came back empty. Index reads now
skip keys longer than an entry, in the union and legacy paths.

get_casts_by_parent across shards appended each shard's page, so a page held
up to page_size casts per shard, out of time order, and a shard that ran out
returned no token and was read again from the start on the next page. Pages
now merge by timestamp and hash, hold page_size casts, resume each shard
after the last of its casts taken, mark finished shards, and end with no
token once every shard is done.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@CassOnMars
CassOnMars merged commit 3a28a8a into farcasterorg:main Oct 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants