Skip to content

fix(search): re-add kind= checkpoint filter dropped in 4a318d3 (#194) - #199

Merged
jphein merged 1 commit into
mainfrom
fix/194-kind-content-filter
May 30, 2026
Merged

fix(search): re-add kind= checkpoint filter dropped in 4a318d3 (#194)#199
jphein merged 1 commit into
mainfrom
fix/194-kind-content-filter

Conversation

@jphein

@jphein jphein commented May 30, 2026

Copy link
Copy Markdown
Collaborator

What

GET /search?kind=content returned byte-identical results to kind=all after the 2026-05-29 redeploy + DB fix — including the Stop-hook CHECKPOINT: drawers kind=content is meant to exclude. The kind parameter was being silently ignored.

Root cause

Commit 4a318d3 (2026-04-27) deliberately retired the kind= param from /search and /context. That was sound at the time: the Phase A–E checkpoint-collection split had moved every checkpoint out of mempalace_drawers into mempalace_session_recovery, so the filter was filtering nothing (verified empirically: 763 in recovery, 0 in main).

The 2026-05-29 DB rebackfill silently re-merged checkpoint drawers back into the main collection, undoing the invariant that made retiring the filter safe. Checkpoints returned to mempalace_drawers, but the filter that excluded them was gone — so kind=content == kind=all.

This is the same bug class as the #174/#175 write/read-symmetry cases: a downstream change (the DB shape) broke an assumption baked into a read surface.

Fix

mempalace's read-side kind= machinery was also retired (companion 7ba28dc), so build_where_filter only knows wing/room/tags and can't push the filter down. Enforce it daemon-side instead:

Known tradeoff (documented in the helper docstring)

This is a post-fetch filter over the limit rows mempalace already returned, not a push-down into the over-fetched candidate pool (mempalace no longer has the machinery to push down to). So a kind=content response can come back with fewer than limit rows when checkpoints rank inside the top limit. The correctness invariant holds (content ⊊ all, no checkpoints in content); the count is best-effort. Restoring exact-count behaviour would need either a larger caller limit or re-adding the push-down filter to mempalace.

Tests

New tests/test_search_kind_filter.py:

  • Unit coverage of _hit_is_checkpoint (topic, synonym, case-insensitive, body-prefix-when-metadata-missing, nested metadata, negatives) and _apply_kind_filter (all/content/checkpoint, strict-subset, filters-echo).
  • HTTP-level TestClient tests (mirroring test_search_rerank_endpoint.py): a CHECKPOINT-shaped drawer is excluded by kind=content but included by kind=all (content ⊊ all), checkpoint-only mode, default-is-content, and the 400.

Mirrors the SME-side invariant in techempower-org/multipass-structural-memory-eval::test_kind_content_excludes_stop_hook_checkpoints (the test that surfaced this).

Full suite: 603 passed, 1 skipped.

Deploy note

Code fix only — the live familiar daemon needs a redeploy for this to take effect (the running process predates the fix). Did not restart the daemon per task constraints; team-lead handles deploy after merge.

Closes #194

🤖 Generated with Claude Code

`GET /search?kind=content` returned byte-identical results to
`kind=all` after the 2026-05-29 redeploy + DB fix — including the
Stop-hook `CHECKPOINT:` drawers `kind=content` is meant to exclude.

Root cause: commit 4a318d3 (2026-04-27) deliberately RETIRED the
`kind=` param from /search and /context. That was sound at the time:
the Phase A–E checkpoint-collection split had moved every checkpoint
out of `mempalace_drawers` into `mempalace_session_recovery`, so the
filter was filtering nothing. The 2026-05-29 DB rebackfill silently
re-merged checkpoint drawers back into the main collection, undoing
the invariant that made retiring the filter safe — checkpoints
returned, but the filter that excluded them was gone.

mempalace's read-side `kind=` machinery was also retired (companion
7ba28dc), so `build_where_filter` only knows wing/room/tags and can't
push the filter down. Enforce it daemon-side instead:

- Re-add `kind` query param (default "content") to /search and
  /context, with 400 on an invalid value (mirrors the pre-4a318d3
  `_VALID_KINDS` contract).
- `_apply_kind_filter` post-filters the response: "content" drops
  checkpoints, "all" is the superset, "checkpoint" keeps only them,
  and echoes `kind` into the `filters` block (#194 noted it was
  never echoed). On /search it runs before rerank so the cross-encoder
  ranks only the eligible pool.
- `_hit_is_checkpoint` classifies on TWO OR'd signals — `topic` in
  {checkpoint, auto-save} OR a `CHECKPOINT:` body prefix — so it
  survives the dropped/empty metadata #194 reports on rebackfilled
  drawers.

Regression test (tests/test_search_kind_filter.py): unit coverage of
the classifier + filter, plus HTTP-level TestClient tests asserting a
CHECKPOINT-shaped drawer is excluded by kind=content but included by
kind=all (content ⊊ all), checkpoint-only mode, default-is-content,
and the 400. Mirrors the SME-side invariant in
techempower-org/multipass-structural-memory-eval
test_kind_content_excludes_stop_hook_checkpoints.

Full suite: 603 passed, 1 skipped.

Closes #194

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 30, 2026 03:44
@gemini-code-assist

Copy link
Copy Markdown

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@jphein
jphein merged commit 8bca2fb into main May 30, 2026
1 check failed
@jphein
jphein deleted the fix/194-kind-content-filter branch May 30, 2026 03:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: kind=content filter is a no-op on redeployed daemon — identical to kind=all (regression)

2 participants