Skip to content

[schemas] Decision ledger with ranked recall - #488

Open
oSquashBlossomo wants to merge 2 commits into
NateBJones-Projects:mainfrom
oSquashBlossomo:contrib/osquashblossomo/decision-ledger
Open

[schemas] Decision ledger with ranked recall#488
oSquashBlossomo wants to merge 2 commits into
NateBJones-Projects:mainfrom
oSquashBlossomo:contrib/osquashblossomo/decision-ledger

Conversation

@oSquashBlossomo

Copy link
Copy Markdown

Contribution Type

  • Recipe
  • Schema
  • Dashboard
  • Integration
  • Skill
  • Repo improvement

What does this do?

Adds a step-indexed decisions ledger on top of the agent-memory schema: every memory_type = 'decision' write is auto-enrolled (via trigger — the write path doesn't change) with a step index, adjustable importance, an optional rationale, and dependency edges to other decisions. A ranked-recall RPC, match_decision_ledger(), scores entries by relevance × importance × step-recency × dependency degree — with a full-text fallback so recall works with no embedding service at all — and a per-policy stats view makes recall policies comparable side by side.

Why this is more than memory_type = 'decision'

The agent-memory schema already stores decision rows, but recall reaches them only through vector similarity on the linked thought. In Stefania Druga's memory-harness experiments (Sakana.ai, AI Engineer 2026), a ranked decisions ledger beat plain vector RAG on long-horizon recall on both accuracy and token cost. This sidecar adds what a ledger needs and vector search lacks: where in the task a decision happened (step_index), an explicit priority (importance), the why kept apart from the what (rationale), which decisions rest on which (agent_decision_edges), and a ranking RPC that blends those signals. Lifecycle state stays in agent_memories — the ledger reads it, never duplicates it.

Requirements

  • Working Open Brain setup, including the core capture path (upsert_thoughtpublic.thoughts) — ledger rows hang off memories whose thoughts arrive through it.
  • The agent-memory schema applied (agent_memories and its sidecar tables).
  • No external services beyond Supabase; no primitives or skills dependencies.
  • Strictly sidecar-only: public.thoughts and every agent-memory table are untouched (no ALTER on any existing table). New tables carry RLS, service_role policies, and explicit grants per the agent-memory pattern, and the migration ends with NOTIFY pgrst, 'reload schema';. Safe to run more than once.

An optional companion patch (separate, not in this PR) wires the ledger into integrations/agent-memory-api as a recall_policy: "ranked_ledger" option on /recall, with the policy recorded on every recall trace — README Step 3 documents the exact change.

The ranking is measured, not guessed

A 1024-dim port of this schema has been running under a local agent brain, and running it turned up five things the first cut of match_decision_ledger() got wrong. All five are fixed here, and the RPC's own header says why:

  1. The full-text query is disjunctive. websearch_to_tsquery and plainto_tsquery AND their terms, so a natural-language question matches a row only if that one row contains every content word in the question — 0 of 15 rows matched that way, which left the FTS arm, and with it 45% of the score, dead for exactly the no-embedding-service setup it exists to serve. The query's lexemes are ORed instead: same configuration, same stemming and stopword list, one operator changed.
  2. The full-text rank saturates rather than clipping. LEAST(ts_rank_cd * 10, 1.0) pins every non-trivial match to exactly 1.0, after which recency alone decides the order; x / (1 + x) is monotone over the whole range and bounded by 1.
  3. Every component is clamped into [0,1] before it is weighted. Cosine similarity is -1 on opposed vectors, not 0. Each row also reports relevance_source (cosine / fts / none), so a partly-embedded corpus is visible in the trace rather than silently ranking on structure alone.
  4. The dependency normaliser is a frozen parameter, not the candidate set's maximum degree. Normalising by the maximum makes a row's score depend on what else happened to match — it scored the same decision 0.333 and 0.5 across two queries. p_dependency_saturation defaults to 3.
  5. The step half-life is 30, not 120. exp(-d/120) spreads 0.0165 of the score across 15 steps, which is inert at session scale. Step distance is measured from max(p_current_step, the stream's own newest step) — defined for every row, so a caller that passes a stale step or none at all no longer floors every row to a recency of 1.0.

All weights, the half-life and the saturation point remain function parameters; these are the defaults, not a hardcoding.

Testing

Tested on my own Open Brain instance in two forms:

  • This exact artifact (1536-dim, RLS + service_role grants) installs green on a scratch Postgres with the Supabase environment present, re-runs idempotently, and is checked by a dual-install parity suite that diffs its installed schema against my production port line by line — every remaining difference is a reviewed allowlist entry. The five fixes above are no longer among them: they are shared by both installs now, which is the point of contributing them rather than keeping them local. What still differs is my deployment's own policy (a pin sidecar, a recall gate, a role model instead of RLS), not the ranking.
  • The ledger design end-to-end: the 1024-dim port has been running under a local agent brain and was exercised by a 486-row live eval (54 items × 9 recall-policy arms) with zero anomalies, including the FTS fallback path and the dependency edges.

Checklist

  • I've read CONTRIBUTING.md
  • My contribution includes a README with prerequisites, steps, and expected outcome
  • My metadata.json is complete and valid
  • Dependencies are declared and linked (agent-memory + core capture path in Prerequisites; requires.open_brain: true)
  • I've tested this on my own Open Brain instance
  • No credentials, API keys, or secrets are included

oSquashBlossomo and others added 2 commits September 1, 2026 11:29
Step-indexed decisions ledger over the agent-memory sidecar:
auto-enrollment trigger for memory_type='decision' writes, importance
and rationale columns, dependency edges, the match_decision_ledger()
ranking RPC (relevance x importance x step-recency x dependency degree,
with a full-text fallback so recall works with no embedding service),
usage feedback bookkeeping, and a per-policy recall stats view.

Sidecar-only: public.thoughts and every agent-memory table untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
match_decision_ledger() scored worse than its documentation claimed, in
five ways found while running a 1024-dim port of this schema under a
local agent brain. Each is fixed here and each is now documented in the
RPC's own header and the README's ranking section.

1. The full-text query was AND-semantics. websearch_to_tsquery ANDs its
   terms, so a natural-language question matched a row only if that one
   row contained every content word in the question: 0 of 15 rows
   matched, leaving the FTS arm — 45% of the score — dead for exactly
   the no-embedding-service setup it exists to serve. The query's
   lexemes are ORed now: same configuration, same stemming and stopword
   list, one operator changed. An all-stopword query reduces to the
   empty tsquery and ranks every row 0, quietly.

2. Relevance clipped instead of saturating. LEAST(ts_rank_cd * 10, 1.0)
   pinned every non-trivial match to exactly 1.0, after which recency
   alone decided the order. x / (1 + x) is monotone over the whole
   range, bounded by 1, and keeps the same scale constant.

3. Cosine similarity was unclamped. It is -1 on opposed vectors, not 0,
   and fed straight into the blend. Every component is now clamped into
   [0,1] before it is weighted, importance included, and each row
   reports which arm produced its relevance in a new relevance_source
   column so a partly-embedded corpus is visible rather than silent.

4. The dependency normaliser is now a parameter frozen at 3 rather than
   a bare constant, with the degree clamped at 0 below. Normalising by
   the candidate set's own maximum degree — the obvious alternative —
   makes a row's score depend on what else happened to match, and
   scored the same decision 0.333 and 0.5 across two queries.

5. The recency constant was inert. exp(-d/120) spreads 0.0165 of the
   score across 15 steps, which does nothing at session scale; the
   half-life is 30 now. Step distance is measured from max(the caller's
   step, the stream's own newest step), which is defined for every row,
   so a stale or absent caller step no longer floors every row to 1.0 —
   and that made the wall-clock fallback, and p_half_life_days with it,
   unnecessary.

The RPC is dropped by name before it is recreated, so re-running the
file cannot leave the previous signature behind for PostgREST to choose
between. Still additive only: no ALTER on public.thoughts or any
agent-memory table, RLS and service_role grants unchanged, and the
migration still ends with NOTIFY pgrst.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@github-actions github-actions Bot added the schema Contribution: database extension label Sep 1, 2026
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown

Hey @oSquashBlossomo — welcome to Open Brain Source! 👋

Thanks for submitting your first PR. The automated review will run shortly and check things like metadata, folder structure, and README completeness. If anything needs fixing, the review comment will tell you exactly what.

Once the automated checks pass, a human admin will review for quality and clarity. Expect a response within a few days.

If you have questions, check out CONTRIBUTING.md or open an issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

schema Contribution: database extension

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant