Repository navigation
fix(search): rank results by page title, path, and heading - #212
Conversation
Docs sites with one copy of each guide per framework got poor results: title matches saturated into one more BM25 occurrence, the framework only lived in the URL, camel-case names were single tokens, every chunk repeated the page description, and one page could fill every result. - Multiply each chunk's score by how much of the query its page title, URL path, and heading cover, weighted by title precision. - Index camel-case parts and joined dotted/hyphenated compounds. - Index the page description with its first chunk only. - Add maxResultsPerPage (default 2), applied before limit; NLWeb uses 1. - Skip phrase matching for chunks that cannot reach the results. Bumps the search index format to version 4.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Warning Review limit reachedYour organization has reached its usage spending cap. Adjust your spending cap in the billing tab. Next included review available in 20 minutes. View limit details
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
📜 Recent review details
📝 Summary
Merge Risk: ⚪ Minimal · up to
|
| Check name | Status | Explanation | Resolution |
|---|---|---|---|
| Docstring Coverage |  |
Docstring coverage is 40.91% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 22 functions across 3 files. (4 skipped: … | Write docstrings for the functions missing them to satisfy the coverage threshold. |
-
Check name Status Explanation Title check The title clearly summarizes the main search-ranking change by identifying page title, path, and heading as ranking signals. Description check The description directly explains the search-ranking problems, implemented fixes, index-version change, performance results, and regression tests. Linked Issues check Check skipped because no linked issues were found for this pull request. Out of Scope Changes check Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage
-
Explanation
Docstring coverage is 40.91% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 22 functions across 3 files. (4 skipped: 4 unsupported.)
✨ Finishing Touches: 💡 1
-
📝 Generate docstrings: 💡
-
- Commit to this branch
- Create a new PR
🧪 Generate unit tests (beta)
-
- Commit to this branch
- Create a new PR
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
I’m a rabbit with queries to try,
I follow page titles and paths as they fly.
New tokens hop into the index with care,
Two chunks per page make a tidy result pair.
I nibble the changelog, then bound through the night.
Comment @coderabbitai help to get the list of available commands.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 214f17d724
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
There was a problem hiding this comment.
ℹ️ No critical issues. Two ranking and performance rough edges are inline.
Reviewed changes
I reviewed the whole PR: the new compound tokenizer, the page-match multiplier, the per-page result cap, the pruned phrase boosting, and the index v4 bump.
- Compound tokens —
ConsentBannerandNext.jsnow index their parts and their joined form. Case is lowered per token so camel-case boundaries survive. - Page-match multiplier — BM25 plus the phrase boost is scaled by title coverage × precision, title∪path coverage, and last-heading coverage, up to 5×.
- Description in the first chunk only — this also fixes the old double count, because chunk text already contained the description.
assertNlwebArtifactsis updated to match. maxResultsPerPage— defaults to 2 and is applied beforelimit. NLWeb/askpasses 1, anddedupeByPageis removed.- Pruned content boosts — phrase and proximity matching run in score order and stop once no remaining chunk can reach the results. I checked the bound: boost ≤
1.4 × 5, and the k-th selected score never drops when chunks are added, even with the per-page cap.
The search and NLWeb test suites pass (160 tests).
claude-opus-5-5 ｜ 𝕏
…cost - Count title precision per written word, so TypeScript is fully matched by typescript instead of scoring 1/3 for its camel-case parts. - Also compare phrases on the words as written, so the joined nextjs token in Next.js app no longer breaks the phrase next js app. - Refresh the pruning bound at doubling checkpoints, so refreshing stays O(n log n) when scores are flat and nothing is pruned. - Drop the dedupeByPage JSDoc left above sseEvent.
There was a problem hiding this comment.
ℹ️ No new issues. All three earlier comments are fixed, and I left one small efficiency suggestion inline.
Reviewed changes
I reviewed b056553, the commit that responds to the earlier review.
- Title precision counts written words —
fieldPrecisionnow gets one token group per word fromtokenizeWordGroups.TypeScriptis a single word, so atypescriptquery covers the whole title. The new test checks that "TypeScript" ranks above "Typescript setup". - Phrase matching on written words — when the query has two or more written words, the phrase boost also compares written words without the derived tokens. The joined
nextjsno longer breaksnext js app. The largest possible boost is stillPHRASE_MATCH_BOOST, so the pruning bound still holds. - Doubling checkpoints for the pruning bound — the bound is now recomputed at 32, 64, 128 and so on, which keeps the total recompute cost at O(n log n). Between checkpoints the bound can be out of date, but that only means fewer chunks get skipped. Results are unchanged.
- Removed the orphaned JSDoc in
ask.ts, and moved the shared filler docs into onefillerDocsfixture.
The search and NLWeb suites pass on this head: 187 tests, 0 failures.
claude-opus-5-5 ｜ 𝕏
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b05655322d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Reuse the phrase tokens for the proximity check, and test that a phrase match lifted only by the content boost survives pruning.
There was a problem hiding this comment.
✅ No new issues found. The efficiency suggestion is fixed, and the new test closes the pruning coverage gap.
Reviewed changes
I reviewed d27d100, the commit added since the last review.
- Chunk text tokenized once for phrase and proximity —
scoreContentMatchBoostnow buildstextTokensonce and uses it for both the phrase check andhasOrderedProximityMatch.normalizeForPhraseMatchis gone. Boosted chunks are tokenized at most twice now, down from three times. Behavior is unchanged. - Regression test for the pruning bound — 61 matching chunks take
applyContentMatchBoostspast its first checkpoint at 32. Only the phrase boost lifts the target page into the top 5. I checked that the test bites: with+ maxBoostremoved from the skip condition, it fails andguide-0ranks first.
The search and NLWeb suites pass on this head (188 tests, 0 failures), and tsc --noEmit is clean.
claude-opus-5-5 ｜ 𝕏
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d27d100d7e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The browser search client and MCP artifact loading only checked that the index and content matched each other, so a cached v3 pair loaded and was searched with the v4 tokenizer, quietly missing matches. Both now reject an index whose version differs from DOCS_SEARCH_INDEX_VERSION, with the same regenerate message NLWeb uses.
There was a problem hiding this comment.
✅ No new issues found. Both loaders that read a shipped index now reject a v3 index with a clear error telling the user to regenerate.
Reviewed changes
I reviewed df2c484, the only commit since the last review.
- Shared version check — the new
docsSearchIndexVersionErrorinsearch.tsreturns an error unlessindex.versionequalsDOCS_SEARCH_INDEX_VERSION. The message names the source path or URL. - Browser search client —
loadArtifactsthrows inside the cached promise, so the existingcatchdrops the cache entry and a redeployed index can load on the next call. - MCP artifact loading —
loadDocsArtifactsruns the check right after the missing-file check, before it reads the manifest. NLWeb/askalready rejected old versions inassertNlwebArtifacts, so the changeset wording matches the code. - Tests —
mcp.test.tsandnext/client.test.tseach serve a version 3 index and expect the rejection. Only these two code paths parse a generated index from outside; theprojectandsourcepaths build the index in-process.
The search, NLWeb, MCP and Next client suites pass on this head (222 tests), and tsc --noEmit is clean.
claude-opus-5-5 ｜ 𝕏
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: df2c484d7c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The page-match boost read the URL path, but only chunks with a posting could be scored, so a page whose only match was its URL never appeared: `vue` missed a page titled Quickstart at /docs/frameworks/vue/quickstart. Path words now count once as body text when the chunk lacks them, using the words as written so hyphenated segments add no joined prefix terms. countDocsSearchChunkTerms keeps the indexer and NLWeb validation on one rule.
A query such as nextjs matched Next.js through its joined token, but the excerpt looked for nextjs in text that still had the dot and fell back to the first 220 characters. Excerpts now also search the text with compound separators removed, mapped back to the original offsets.
There was a problem hiding this comment.
✅ No new issues found. A page that matches only through its URL path is now found, and the indexer and NLWeb validation use the same counting rule.
Reviewed changes
I reviewed b2c0358, the only commit since the last review.
- URL path words become postings — the new
countDocsSearchChunkTermsadds each URL path word that a chunk doesn't already contain to its body counts, once. It uses only the words as written, sotypescript-setupdoesn't add a joinedtypescriptsetupthat everytypescriptquery would prefix-match.vuenow finds the page titled "Quickstart" at/docs/frameworks/vue/quickstart. - One counting rule —
createDocsSearchIndexand NLWeb'sassertNlwebArtifactsboth callcountDocsSearchChunkTerms, so the posting validation can't drift from the indexer. NLWeb reads the URL path fromdocument[3], the same slotcreateDocsSearchIndexwrites it to. Chunk length (chunk[4]) still counts only the text tokens, so NLWeb's length and average-length checks still pass. - Docs and test — the changeset and
docs/reference/search.mdxnow describe the path postings, and the new test "finds a page whose only match is its URL" covers the case Codex raised.
The search, NLWeb, MCP and Next client suites pass on this head (223 tests).
claude-opus-5-5 ｜ 𝕏
There was a problem hiding this comment.
✅ No new issues found. When a query matches only through a joined compound like
nextjs, the excerpt now centers on the match in the original text.
Reviewed changes
I reviewed 14f4e3a, the only commit since the last review.
- Joined-compound fallback for excerpts — the new
findOriginalMatchIndextries each excerpt token against the normalized text first. If that misses, it tries the same token against a copy that drops.,-and_between two word characters, sonextjsfindsNext.js.joinCompoundsWithOffsetsbuilds that copy lazily and only once per excerpt, and its offsets still point into the original text, so the slice window is unchanged. - Regression test — "centers the excerpt on a dotted name matched through its joined form" puts
Next.jsabout 340 characters into a chunk, past the 220-character fallback excerpt. I checked that it fails when the joined match is disabled.
The search and NLWeb suites pass on this head (165 tests).
claude-opus-5-5 ｜ 𝕏
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 14f4e3afc2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| }; | ||
| // Written words only: a joined `typescriptsetup` from `typescript-setup` | ||
| // would become a prefix match for every `typescript` query. | ||
| for (const token of tokenizeWrittenWords(chunk.urlPath)) { |
There was a problem hiding this comment.
When a dotted or hyphenated name appears only in the URL, using tokenizeWrittenWords omits the joined token that the new tokenizer advertises. For example, a generic Quickstart page at /docs/frameworks/tanstack-start/quickstart returns no result for tanstackstart: the index contains only tanstack and start, so the page never becomes a candidate for its path multiplier. Fresh evidence after the earlier URL-indexing fix is this new written-only call; joined compounds need an exact-candidate path that does not also make typescriptsetup a prefix match for typescript.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Leaving this as is for now. I tried indexing the joined URL compounds (swapping tokenizeWrittenWords for tokenizeDocsSearchText on the path). Every hyphenated slug then becomes a prefix-expansion target for its first word: /docs/typescript-setup adds typescriptsetup, which the query typescript prefix-matches at 0.55, so "Typescript setup" outranks the page titled "TypeScript" again and that regression test fails. On the c15t.com eval, right-page-first drops from 32/46 to 31/46 and MRR@8 from 0.752 to 0.741.
The miss is limited to queries that type a hyphenated slug run together, such as tanstackstart, for a page whose title and text never say "TanStack Start". Fixing it without the prefix problem needs postings that match exactly and never expand by prefix, which is another index format change. The c15t site already maps tanstackstart to tanstack start with query synonyms (inthhq/monorepo#2316), which covers the real case. Happy to do the exact-only postings as a follow-up if maintainers want it.
Search on c15t.com ranks poorly now that the docs have one copy of most guides per framework (11 trees, ~28 pages each). The causes are in
searchDocs:consent categoriesranked the page with that exact title 47th, behind changelogs that repeat both words.ConsentBanner,DevToolsandNext.jswere single tokens, soconsent banner,dev toolsandnextjsmissed them.posthogreturned 8 sections of one page. NLWeb removed duplicate pages after cutting tolimit, so it returned fewer results than asked for.Changes
ConsentBanner→consentbanner,consent,banner). Short dotted or hyphenated names also index their joined form (Next.js→next,js,nextjs).maxResultsPerPageoption (default 2), applied beforelimit. NLWeb/askpasses 1 and drops its own post-limit dedupe.consent bannerdrops from 148 ms to 18 ms. Pruned and exhaustive results matched on 918 query/option combinations.The index format moves to version 4. Sites must regenerate their search artifacts, and v3 artifacts fail validation.
Results
I rebuilt the c15t.com index from its 438 live Markdown mirrors and scored 46 queries, each with the page that should come first:
mainI chose the queries and the correct pages myself, so treat these numbers as a guide. The weights are round numbers and none are specific to c15t.
Tests
Four regression tests cover camel-case and dotted tokens, a title beating a release note that repeats it, URL-only words, and the per-page cap. All four fail on
main.Need help on this PR? Tag
@codesmith-botwith what you need.