[integrations] Fix silent metadata loss on OpenAI-compatible endpoints

[mike-sanlon]

The bug

extractMetadata sends response_format: { type: "json_object" }. Not every OpenAI-compatible endpoint accepts that. LM Studio rejects it with HTTP 400:

'response_format.type' must be 'json_schema' or 'text'

The response is never checked for r.ok, so the 400 body flows into d.choices[0], throws inside the try, and is swallowed by the catch, which returns the "uncategorized" fallback:

const d = await r.json();
try {
  return JSON.parse(d.choices[0].message.content);
} catch {
  return { topics: ["uncategorized"], type: "observation" };
}

capture_thought then reports success. The thought is stored with junk metadata and nothing appears in the logs. On an affected endpoint this happens for every capture, and the resulting rows are indistinguishable from thoughts genuinely categorised as "uncategorized".

The embedding and content are still correct, so search by meaning still works — which is likely why this hasn't been noticed. What breaks is everything driven by metadata: list_thoughts filtering by type/topic/person, and thought_stats.

Reproduction

Point CHAT_API_BASE at any endpoint that rejects json_object (LM Studio's /v1 is the easy one) and capture anything. It returns success; the row lands with topics: ["uncategorized"].

Changes

1. Send a real json_schema response_format. Accepted by OpenAI and by the compatible runtimes, and it enforces the shape rather than merely requesting it. Includes additionalProperties: false and lists every property in required — both mandatory under strict mode, per the structured outputs guide.

2. Check r.ok and validate the response shape. Logs status and body on failure instead of failing silently.

3. Tag the fallback with metadata_degraded: true, so degraded rows become findable and re-extractable:

   SELECT id, content FROM thoughts WHERE metadata->>'metadata_degraded' = 'true';

   It still returns the fallback rather than throwing — losing the thought because the metadata model was down is worse than storing it with weak metadata.

4. Constant-time comparison for the access key. !== short-circuits on the first differing byte, leaking key material through response timing. Length is compared first and non-secretly; that only reveals the key's length, and bailing early on a length mismatch is unavoidable for a fixed-width compare.

5. Make the similarity threshold configurable via OPEN_BRAIN_MATCH_THRESHOLD. The 0.5 default is unchanged. It is tuned for text-embedding-3-small and is a poor fit for models with a compressed similarity range — on bge-m3, 0.5 dropped 2 of 5 genuinely relevant queries in local testing. That is a config problem, not a default worth changing for everyone.

Verification

deno check clean against Deno 2.9.5. Items 1–3 exercised against a live LM Studio endpoint: captures that previously stored "uncategorized" now store correct extracted metadata, with metadata_degraded absent.

Deliberately not included

The server also accepts the key as a ?key= query parameter. Query strings land in reverse-proxy access logs, browser history, and Referer headers, so it is worth removing — but that is a breaking change for anyone relying on it, so it belongs in its own PR with maintainer input rather than buried in this one. Happy to open that separately if you want it.

[github-actions]

Hey @mike-sanlon — welcome to Open Brain Source! 👋

Thanks for submitting your first PR. The automated review will run shortly and check things like metadata, folder structure, and README completeness. If anything needs fixing, the review comment will tell you exactly what.

Once the automated checks pass, a human admin will review for quality and clarity. Expect a response within a few days.

If you have questions, check out CONTRIBUTING.md or open an issue.