Skip to content

[integrations] entity-extraction-worker: fix prerequisites, document safe pg_cron scheduling - #421

Open
hadolphs wants to merge 1 commit into
NateBJones-Projects:mainfrom
hadolphs:contrib/hadolphs/entity-worker-scheduling
Open

[integrations] entity-extraction-worker: fix prerequisites, document safe pg_cron scheduling#421
hadolphs wants to merge 1 commit into
NateBJones-Projects:mainfrom
hadolphs:contrib/hadolphs/entity-worker-scheduling

Conversation

@hadolphs

@hadolphs hadolphs commented Jul 9, 2026

Copy link
Copy Markdown

Problem

Two independent problems with integrations/entity-extraction-worker/, both found while actually deploying and scheduling it.

1. The prerequisites are wrong

- **Enhanced thoughts schema** applied — install `schemas/enhanced-thoughts`
- **Knowledge graph schema** applied — install `schemas/knowledge-graph` to create the `entities`, `edges`, ...
  • schemas/knowledge-graph does not exist in this repo. The tables the worker needs (entities, edges, thought_entities, entity_extraction_queue) are created by schemas/entity-extraction/.
  • schemas/enhanced-thoughts is not required. index.ts reads only id, content, metadata from thoughts:
    .from("thoughts")
    .select("id, content, metadata")
    and references none of sensitivity_tier / source_type / quality_score. Listing it as a prerequisite sends people to install a schema they don't need.

2. Scheduling this worker fails silently, in three separate ways

The README recommends running the worker but never shows how to schedule it. Following the repo's existing pg_cron pattern (recipes/editorial-policy/schedule.sql) produces a job that reports succeeded every single run while processing zero thoughts:

  1. net.http_post() defaults to timeout_milliseconds = 5000. This worker makes one LLM call per queued thought and takes tens of seconds — measured elapsed_ms of 26,280 (3 thoughts), 57,569 (12), 58,332 (10). Every scheduled call is abandoned mid-flight at 5 s, so nothing is ever committed.
  2. A trailing space in the Vault secret's name makes where name = 'mcp_access_key' match nothing. The sub-select returns NULL, the job sends x-brain-key: null, and the worker replies 401.
  3. net.http_post() only enqueues the request and returns a request id. pg_cron never observes the HTTP response, so cron.job_run_details logs succeeded regardless. The cron log cannot distinguish success from total failure.

Stacked together, these are near-undiagnosable. In our case the knowledge graph sat frozen for days behind a wall of green "successful" cron runs. The giveaway was that complete in entity_extraction_queue exactly matched the count from manual worker invocations — the cron had never processed a single thought.

Evidence from net._http_response, in order:

result cause
status_code = null, "Timeout of 5000 ms reached" pg_net abandoned the 26–58 s worker at 5 s
status_code = 401 trailing space in the Vault secret's name → x-brain-key: null
status_code = 200, {"processed": 0, ...} fixed

Fix

Docs only — no code changes.

  • Correct the prerequisites (entity-extraction, not knowledge-graph; note that enhanced-thoughts is not required).
  • Add a “Schedule It” step with a hardened pg_cron + pg_net job: explicit timeout_milliseconds, a raise exception guard so a missing/renamed Vault secret becomes a visible cron failure rather than a phantom success, and a limit sized to finish inside the timeout.
  • Document the queries that are actual health signalsentity_extraction_queue status counts and net._http_response.status_code (whose content column holds the worker's real JSON reply) — and state plainly that cron.job_run_details is not one.
  • Add a Troubleshooting entry for the exact symptom ("scheduled runs report success but nothing is processed").

Verification

All three claims were checked against the code on main before writing:

  • grep -n -A1 '\.from("thoughts")' index.ts.select("id, content, metadata")
  • grep -nE 'sensitivity_tier|source_type|quality_score' index.ts → no matches
  • ls schemas/knowledge-graph → does not exist

The hardened schedule in this PR is the one now running against a live brain: status_code = 200, body {"processed": 0, ...}, queue pending = 0.

@github-actions github-actions Bot added the integration Contribution: MCP extension or capture source label Jul 9, 2026
…safe pg_cron scheduling

The prerequisites were wrong in two ways:
- schemas/knowledge-graph does not exist. The tables this worker needs
  (entities, edges, thought_entities, entity_extraction_queue) come from
  schemas/entity-extraction.
- schemas/enhanced-thoughts was listed as required, but index.ts reads only
  'id, content, metadata' from thoughts and references none of its columns.

Scheduling was undocumented, and following the repo's existing pg_cron pattern
fails silently in three separate ways, each logging 'succeeded' in
cron.job_run_details while processing zero thoughts:
- net.http_post() defaults to timeout_milliseconds=5000, but the worker takes
  tens of seconds (one LLM call per thought), so every call is abandoned
  mid-flight (measured elapsed_ms: 26280 / 57569 / 58332).
- A trailing space in the Vault secret's NAME makes the lookup return NULL,
  sending 'x-brain-key: null' and yielding a 401.
- net.http_post() only enqueues the request, so pg_cron never sees the HTTP
  result; the cron log cannot distinguish success from total failure.

Adds a hardened schedule (explicit timeout + raise-on-null guard), documents the
queries that are real health signals (entity_extraction_queue status counts and
net._http_response status_code/body), and adds a troubleshooting entry.
@hadolphs
hadolphs force-pushed the contrib/hadolphs/entity-worker-scheduling branch from 4a6b95f to 19ecc8c Compare September 3, 2026 00:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

integration Contribution: MCP extension or capture source

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant