fix(search): re-add kind= checkpoint filter dropped in 4a318d3 (#194) - #199
Merged
Conversation
`GET /search?kind=content` returned byte-identical results to `kind=all` after the 2026-05-29 redeploy + DB fix — including the Stop-hook `CHECKPOINT:` drawers `kind=content` is meant to exclude. Root cause: commit 4a318d3 (2026-04-27) deliberately RETIRED the `kind=` param from /search and /context. That was sound at the time: the Phase A–E checkpoint-collection split had moved every checkpoint out of `mempalace_drawers` into `mempalace_session_recovery`, so the filter was filtering nothing. The 2026-05-29 DB rebackfill silently re-merged checkpoint drawers back into the main collection, undoing the invariant that made retiring the filter safe — checkpoints returned, but the filter that excluded them was gone. mempalace's read-side `kind=` machinery was also retired (companion 7ba28dc), so `build_where_filter` only knows wing/room/tags and can't push the filter down. Enforce it daemon-side instead: - Re-add `kind` query param (default "content") to /search and /context, with 400 on an invalid value (mirrors the pre-4a318d3 `_VALID_KINDS` contract). - `_apply_kind_filter` post-filters the response: "content" drops checkpoints, "all" is the superset, "checkpoint" keeps only them, and echoes `kind` into the `filters` block (#194 noted it was never echoed). On /search it runs before rerank so the cross-encoder ranks only the eligible pool. - `_hit_is_checkpoint` classifies on TWO OR'd signals — `topic` in {checkpoint, auto-save} OR a `CHECKPOINT:` body prefix — so it survives the dropped/empty metadata #194 reports on rebackfilled drawers. Regression test (tests/test_search_kind_filter.py): unit coverage of the classifier + filter, plus HTTP-level TestClient tests asserting a CHECKPOINT-shaped drawer is excluded by kind=content but included by kind=all (content ⊊ all), checkpoint-only mode, default-is-content, and the 400. Mirrors the SME-side invariant in techempower-org/multipass-structural-memory-eval test_kind_content_excludes_stop_hook_checkpoints. Full suite: 603 passed, 1 skipped. Closes #194 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
GET /search?kind=contentreturned byte-identical results tokind=allafter the 2026-05-29 redeploy + DB fix — including the Stop-hookCHECKPOINT:drawerskind=contentis meant to exclude. Thekindparameter was being silently ignored.Root cause
Commit
4a318d3(2026-04-27) deliberately retired thekind=param from/searchand/context. That was sound at the time: the Phase A–E checkpoint-collection split had moved every checkpoint out ofmempalace_drawersintomempalace_session_recovery, so the filter was filtering nothing (verified empirically: 763 in recovery, 0 in main).The 2026-05-29 DB rebackfill silently re-merged checkpoint drawers back into the main collection, undoing the invariant that made retiring the filter safe. Checkpoints returned to
mempalace_drawers, but the filter that excluded them was gone — sokind=content==kind=all.This is the same bug class as the #174/#175 write/read-symmetry cases: a downstream change (the DB shape) broke an assumption baked into a read surface.
Fix
mempalace's read-side
kind=machinery was also retired (companion7ba28dc), sobuild_where_filteronly knows wing/room/tags and can't push the filter down. Enforce it daemon-side instead:kindquery param (default"content") to/searchand/context, with400on an invalid value (mirrors the pre-4a318d3_VALID_KINDScontract)._apply_kind_filterpost-filters the response:contentdrops checkpoints,allis the superset,checkpointkeeps only them — and echoeskindinto thefiltersblock (bug: kind=content filter is a no-op on redeployed daemon — identical to kind=all (regression) #194 noted it was never echoed). On/searchit runs before rerank so the cross-encoder ranks only the eligible pool._hit_is_checkpointclassifies on two OR'd signals —topic ∈ {checkpoint, auto-save}OR aCHECKPOINT:body prefix — so it survives the dropped/empty metadata bug: kind=content filter is a no-op on redeployed daemon — identical to kind=all (regression) #194 reports on rebackfilled drawers.Known tradeoff (documented in the helper docstring)
This is a post-fetch filter over the
limitrows mempalace already returned, not a push-down into the over-fetched candidate pool (mempalace no longer has the machinery to push down to). So akind=contentresponse can come back with fewer thanlimitrows when checkpoints rank inside the toplimit. The correctness invariant holds (content ⊊ all, no checkpoints in content); the count is best-effort. Restoring exact-count behaviour would need either a larger callerlimitor re-adding the push-down filter to mempalace.Tests
New
tests/test_search_kind_filter.py:_hit_is_checkpoint(topic, synonym, case-insensitive, body-prefix-when-metadata-missing, nested metadata, negatives) and_apply_kind_filter(all/content/checkpoint, strict-subset, filters-echo).TestClienttests (mirroringtest_search_rerank_endpoint.py): a CHECKPOINT-shaped drawer is excluded bykind=contentbut included bykind=all(content ⊊ all), checkpoint-only mode, default-is-content, and the400.Mirrors the SME-side invariant in
techempower-org/multipass-structural-memory-eval::test_kind_content_excludes_stop_hook_checkpoints(the test that surfaced this).Full suite: 603 passed, 1 skipped.
Deploy note
Code fix only — the live familiar daemon needs a redeploy for this to take effect (the running process predates the fix). Did not restart the daemon per task constraints; team-lead handles deploy after merge.
Closes #194
🤖 Generated with Claude Code