perf(api): exact keyset cursors for the new sort - #30
Merged
Conversation
The API caps cursor depth at 1000 because `paginate` locates a cursor by scanning the rows it was handed. `new` does not need that: its key, (created_at, id), never moves once a post is written, so the database can seek straight to the cursor row and return only the page. get_listing takes after_id/before_id and compares the whole key as a row value: WHERE (a.created_at, a.id) < (SELECT created_at, id FROM posts WHERE id = ?) which expresses "strictly past that row in this order" in one shot, tiebreaker included. A cursor id that no longer exists makes the subquery NULL, so the comparison is NULL and no rows come back -- a stale cursor reads as "nothing after this", which is the answer the API wants anyway. Walking backwards runs the comparison the other way in ascending order and flips the rows, so callers always get one order. The ranked sorts keep the window-and-cap treatment. That is a property of the data, not a shortcut: hot/controversial/rising rank on live vote counts, so a row's position moves between requests and a cursor into one is approximate however it is implemented. Posts.KEYSET_SORTS is the list, and get_listing asserts when a cursor arrives with a sort that is not on it, because that failure would otherwise be silent and subtly wrong. luacheck caught a real bug while writing this: `local _, cursor_id = cursor and S.parse_fullname(cursor)` truncates to one value, so cursor_id was always nil and the keyset path would never have fired. The tests still passed, because the window path handles those cases correctly -- which is why the spec now captures the filters the endpoint passes the model rather than counting queries. Both paths are a single query, so a query count cannot tell them apart. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0128hUpuk1spKzk4UHburdki
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First of the two follow-ups from the data-model review.
newdoesn't need the depth cap#29 capped cursor depth at 1000 because
paginatelocates a cursor by scanningthe rows it was handed.
newnever needed that: its key(created_at, id)isfixed the moment a post is written, so the database can seek straight to the
cursor row.
get_listingtakesafter_id/before_idand compares the whole key as a rowvalue:
That expresses "strictly past that row in this order" in one shot, tiebreaker
included — no separate
OR (created_at = ? AND id < ?)limb to get wrong.Two properties worth noting:
subquery is NULL, so the comparison is NULL, so no rows return — a stale
cursor reads as "nothing after this", which is what the API wants anyway.
beforeruns the comparison the other wayin ascending order and flips the rows, so callers always see one order.
The ranked sorts deliberately keep the cap
hot,controversialandrisingrank on live vote counts, so a row'sposition moves between requests — a cursor into one is approximate however it's
implemented, and keyset would relocate the inaccuracy rather than remove it.
Posts.KEYSET_SORTSis the list, andget_listingasserts if a cursorarrives with a sort that isn't on it. That failure would otherwise be silent and
subtly wrong, which is the worst shape for a pagination bug.
A bug luacheck caught, and what it changed about the tests
x and f()truncates to a single value, socursor_idwas alwaysniland thekeyset path would never have fired. The whole test suite still passed —
because the window path handles those requests correctly, just less efficiently.
That's a warning worth taking seriously: a green suite would not have caught it.
So the spec no longer tries to infer the path from behaviour or query counts
(both paths are a single query, so counting can't tell them apart). It captures
the filters the endpoint hands the model and asserts directly:
new+ cursor →after_idset,limit = 3(page + lookahead)hot+ cursor → noafter_id,limit = S.MAX_DEPTHVerification
Five new cases on top of #29's: a forward walk that must reproduce the full
listing exactly, a backward walk that must return the previous page, the
path-selection assertions above, a vanished cursor, and the assert firing on a
ranked sort. Full suite 413 passing, stylua and luacheck clean.
🤖 Generated with Claude Code
https://claude.ai/code/session_0128hUpuk1spKzk4UHburdki