Skip to content

[integrations] Fix enhanced-mcp search returning zero rows and captures losing embeddings - #490

Open
TomKeyser wants to merge 1 commit into
NateBJones-Projects:mainfrom
TomKeyser:contrib/TomKeyser/enhanced-mcp-filter-payload-fix
Open

[integrations] Fix enhanced-mcp search returning zero rows and captures losing embeddings#490
TomKeyser wants to merge 1 commit into
NateBJones-Projects:mainfrom
TomKeyser:contrib/TomKeyser/enhanced-mcp-filter-payload-fix

Conversation

@TomKeyser

Copy link
Copy Markdown

Contribution Type

  • Integration (/integrations)

What does this do?

Fixes three defects that make integrations/enhanced-mcp non-functional against the schemas it ships against. All search returned zero rows, captures were stored with a NULL embedding (and reported an error while doing so), and get_thought could not be called at all. All three were found by testing a fresh install built from docs/01-getting-started.md plus schemas/enhanced-thoughts.

1. All search returned zero rows

brain_search_thoughts (both semantic and text modes) and search_thoughts_text passed {exclude_restricted: true} — plus start_date/end_date — inside the RPC filter payload. But both RPCs treat that payload as a metadata containment test:

AND t.metadata @> coalesce(p_filter, '{}'::jsonb)

So the query asked Postgres for thoughts whose metadata literally contains {"exclude_restricted": true}. No thought carries that key, so every semantic and full-text query matched nothing. A comment in the code assumed older RPC versions would ignore unknown filter keys — they don't; a non-matching containment test returns zero rows.

Fix: pass only genuine metadata filters, and apply sensitivity and date filtering client-side. The semantic branch already post-filtered sensitivity_tier !== "restricted", which made the payload flag both redundant and fatal; the text paths now do the same.

2. Captures were stored with a NULL embedding

brain_capture_thought sends the embedding in p_payload, but the upsert_thought in schemas/enhanced-thoughts never reads it — embedding is not in that function's INSERT list. The handler then checked result.thought_id, while that RPC returns {id, fingerprint}, so it threw "upsert_thought returned no result" after the row had already been inserted.

The result was an error message on a write that actually succeeded, leaving a thought with a NULL embedding that semantic search can never return. Re-capturing does not heal it, because the fingerprint dedup path short-circuits before computing an embedding.

Fix: accept either key, then persist the embedding in a follow-up update — the same two-step the stock core server in server/index.ts already performs — and report explicitly if that update fails rather than claiming success.

3. get_thought could not be called

It declared id as z.number().int().min(1), but the core schema defines thoughts.id as uuid default gen_random_uuid(). No valid call was possible on a standard install. The lookup is a plain .eq(), so accepting a uuid string is sufficient.

Deliberately not fixed here

update_thought and related_thoughts carry the same integer-id declaration. related_thoughts additionally requires the knowledge-graph schema, which I don't have installed, so I couldn't verify a change to either and left them alone rather than ship untested edits. Happy to follow up if a maintainer with that schema can confirm the expected id type.

Separately, graph_search throws a raw Could not find the table 'public.entities' rather than the graceful "install required schema" message the README describes. Left as-is to keep this PR scoped.

Requirements

No new requirements. Same services as the existing integration: Supabase (with pgvector) and OpenRouter. No dependency, schema, or API changes — this is a behavior fix confined to integrations/enhanced-mcp/index.ts.

Testing

Tested on my own Open Brain instance (Supabase + pgvector, us-west-2, Postgres 17), built from docs/01-getting-started.md with schemas/enhanced-thoughts applied.

Before:

  • search_thoughts_text{"results":[]} for every query, including exact phrases present in stored content
  • brain_search_thoughts[] in both text and semantic mode
  • brain_capture_thoughtError: upsert_thought returned no result (row inserted anyway, embedding NULL)
  • get_thought → uncallable; schema rejects a uuid

After:

  • search_thoughts_text returns a phrase that semantic search scored at 0.348 and silently missed (rank 0.6)
  • brain_search_thoughts text mode returns both stored thoughts, ranked 0.665 / 0.6
  • brain_search_thoughts semantic mode returns correctly ordered results at 0.507 / 0.481
  • brain_capture_thought returns {thought_id, action: "inserted"} and the thought is retrievable semantically at 0.590
  • get_thought returns full metadata and provenance for a uuid
  • count_thoughts and brain_thought_stats unaffected, still correct

Checklist

  • I've read CONTRIBUTING.md
  • My contribution has a README.md with prerequisites, step-by-step instructions, and expected outcome (existing integration README unchanged — this is a bug fix, no doc changes needed)
  • My metadata.json has all required fields (unchanged)
  • If my contribution depends on a skill or primitive, I declared it in metadata.json and linked it in the README (no new dependencies)
  • I tested this on my own Open Brain instance
  • No credentials, API keys, or secrets are included

…es losing embeddings

Three defects made enhanced-mcp non-functional against the schemas it ships
against. All three were found by testing a fresh install built from
docs/01-getting-started.md plus schemas/enhanced-thoughts.

1. All search returned zero rows.
   brain_search_thoughts (both modes) and search_thoughts_text passed
   {exclude_restricted: true} — plus start_date/end_date — inside the RPC
   filter payload. Both match_thoughts and search_thoughts_text treat that
   payload as a metadata containment test (t.metadata @> filter), so the
   query asked for thoughts whose metadata literally contains those keys.
   No thought does, so every query matched nothing. An in-code comment
   assumed older RPCs would ignore unknown filter keys; they do not.
   Fix: pass only genuine metadata filters, and apply sensitivity and date
   filtering client-side — which the semantic branch already did, making the
   payload flag both redundant and fatal.

2. Captures were stored with a NULL embedding.
   brain_capture_thought sends the embedding in p_payload, but the
   upsert_thought in schemas/enhanced-thoughts never reads it — the column is
   not in its INSERT list. The handler then read result.thought_id while that
   RPC returns {id, fingerprint}, so it threw "upsert_thought returned no
   result" *after* the row was already inserted: an error on a write that
   succeeded, leaving a thought invisible to semantic search permanently.
   Fix: accept either key, then persist the embedding in a follow-up update
   (the same two-step the stock core server performs), and report failure
   explicitly if that update does not land.

3. get_thought could not be called.
   It declared id as z.number().int(), but the core schema defines
   thoughts.id as uuid default gen_random_uuid(). The lookup is a plain
   .eq(), so accepting a uuid string is sufficient.

Not fixed here: update_thought and related_thoughts carry the same
integer-id declaration. related_thoughts additionally requires the
knowledge-graph schema, which I do not have installed, so I could not verify
a change to either and left them alone rather than ship untested edits.

Tested on my own Open Brain instance (Supabase + pgvector, us-west-2).
Full-text now returns a phrase that semantic search scored at 0.348 and
missed; capture writes an embedding and the thought is retrievable at 0.590.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K3k7Mmb11c1yzPWCn15HVC
@github-actions github-actions Bot added the integration Contribution: MCP extension or capture source label Sep 2, 2026
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown

Hey @TomKeyser — welcome to Open Brain Source! 👋

Thanks for submitting your first PR. The automated review will run shortly and check things like metadata, folder structure, and README completeness. If anything needs fixing, the review comment will tell you exactly what.

Once the automated checks pass, a human admin will review for quality and clarity. Expect a response within a few days.

If you have questions, check out CONTRIBUTING.md or open an issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

integration Contribution: MCP extension or capture source

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant