Skip to content

[integrations] Fix smart-ingest non-dry-run commit bugs - #473

Open
symy1610 wants to merge 2 commits into
NateBJones-Projects:mainfrom
sydney-moutia:contrib/sydney-moutia/fix-smart-ingest-commit-bugs
Open

[integrations] Fix smart-ingest non-dry-run commit bugs#473
symy1610 wants to merge 2 commits into
NateBJones-Projects:mainfrom
sydney-moutia:contrib/sydney-moutia/fix-smart-ingest-commit-bugs

Conversation

@symy1610

@symy1610 symy1610 commented Aug 4, 2026

Copy link
Copy Markdown

Summary

Fixes the two bugs that block every real (non-dry-run) smart-ingest write, tracked in #379 and #468.

#379 — UUID/bigint mismatch on upsert_thought return.
upsert_thought's v_id is UUID (schemas/enhanced-thoughts/schema.sql), serialized as a JSON string. extractThoughtId() in integrations/smart-ingest/index.ts only accepted numbers (Number.isFinite), so every successful write was read back as "no thought_id" and the item marked failed — even though the row had already landed in thoughts. ingestion_items.matched_thought_id / result_thought_id were also declared bigint, so they couldn't hold a UUID even if the client accepted one, and append_thought_evidence(bigint, ...) compared a bigint parameter against thoughts.id (uuid), which can't succeed.

Fix:

  • extractThoughtId() accepts UUID strings (numeric ids still accepted, for bigint-id forks).
  • ingestion_items.matched_thought_id / result_thought_id are now uuid columns, with an idempotent migration for existing installs (safe — the bug meant these were always NULL in practice).
  • append_thought_evidence now takes p_thought_id uuid; the old bigint overload is dropped rather than left dangling.

#468 — OpenRouter extraction fails on instruction-shaped content.
response_format: { type: "json_object" } is an OpenAI-shaped parameter; the default OpenRouter model (an Anthropic model) silently ignores it, so it can still respond with prose instead of JSON — reproduced live as an HTTP 500 "OpenRouter returned invalid JSON".

Fix: per the issue's suggested fix #1, retry once with an explicit "return only JSON" correction before failing. Left a comment on response_format explaining it isn't a reliable guard on its own.

What it requires

  • Same as smart-ingest today: Supabase + pgvector, enhanced-thoughts schema applied (for upsert_thought), OpenRouter/OpenAI/Anthropic key.
  • The schema change is additive/idempotent — safe to reapply schemas/smart-ingest/schema.sql on an existing install.

Testing

Repro'd both failures against a live OB1 deployment before this fix:

For this change itself: deno check and deno lint pass on integrations/smart-ingest (the one pre-existing lint warning in _shared/helpers.ts and the deno fmt diff on index.ts both predate this PR — confirmed present on origin/main before these edits, unrelated to this change). I was not able to redeploy and re-verify end-to-end against a live Supabase instance as part of this PR (no test project available in this environment) — flagging that per CONTRIBUTING.md's testing guidance, and happy to verify against a real instance if a maintainer can point me at one, or if the original reporter (who offered to send a PR in #379) wants to confirm independently.

Closes #379, and fixes the OpenRouter side of #468 for smart-ingest specifically (the issue also covers the general "no guard on non-JSON responses" observation, which this PR's retry addresses for this call site).

…rojects#379, NateBJones-Projects#468)

Two bugs blocked every real (non-dry-run) smart-ingest write:

1. UUID/bigint mismatch (NateBJones-Projects#379). upsert_thought's v_id is UUID
   (schemas/enhanced-thoughts/schema.sql), serialized as a JSON string, but
   extractThoughtId() only accepted numbers via Number.isFinite(). Every
   successful insert was read back as "no thought_id" and marked failed,
   even though the row landed in thoughts. ingestion_items.matched_thought_id
   / result_thought_id were also declared bigint, so they couldn't hold a
   UUID even if the client accepted one. append_thought_evidence(bigint, ...)
   had the same problem — comparing a bigint parameter against thoughts.id
   (uuid) can't succeed.

   Fix: extractThoughtId() now accepts UUID strings (numbers still accepted
   for bigint-id deployments), matched_thought_id/result_thought_id are now
   uuid columns (idempotent migration included, safe because the bug meant
   these were always NULL in practice), and append_thought_evidence takes
   uuid.

2. OpenRouter extraction failures on instruction-shaped content (NateBJones-Projects#468).
   response_format: json_object is an OpenAI-shaped parameter that the
   default OpenRouter model (an Anthropic model) silently ignores, so a
   model could still respond with prose instead of JSON — reproduced with
   HTTP 500 "OpenRouter returned invalid JSON" on a real ingest.

   Fix: retry once with an explicit "return only JSON" correction before
   failing, per the issue's suggested fix NateBJones-Projects#1. Left a comment explaining why
   response_format isn't a reliable guard on its own.

Verified: deno check and deno lint pass on integrations/smart-ingest (the
one pre-existing lint warning and fmt diff are unrelated to this change,
confirmed present on origin/main). Not deployed/tested against a live
Supabase instance as part of this change — recommend running against a
real instance per CONTRIBUTING.md before merge.

Tested: reproduced both failures against a live OB1 deployment prior to
this fix (HTTP 200 with added_count: 0 / failed_count: 1 for NateBJones-Projects#379; HTTP 500
extraction_failed for NateBJones-Projects#468); this PR fixes the root causes identified in
that repro and in the linked issues, but has not yet been re-verified end
to end against a live instance since the fix.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@github-actions github-actions Bot added integration Contribution: MCP extension or capture source schema Contribution: database extension labels Aug 4, 2026
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown

Hey @sydney-moutia — welcome to Open Brain Source! 👋

Thanks for submitting your first PR. The automated review will run shortly and check things like metadata, folder structure, and README completeness. If anything needs fixing, the review comment will tell you exactly what.

Once the automated checks pass, a human admin will review for quality and clarity. Expect a response within a few days.

If you have questions, check out CONTRIBUTING.md or open an issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

integration Contribution: MCP extension or capture source schema Contribution: database extension

Projects

None yet

Development

Successfully merging this pull request may close these issues.

smart-ingest & enhanced-mcp: writes silently orphan — UUID/bigint mismatch on upsert_thought return

2 participants