The substrate is the asset; the model is an interchangeable client.
git clone https://github.com/thomaskawas/agent-context-substrate.git
cd agent-context-substrate
make setup && make demo
Requirements: Python 3.12+, Docker, make. make setup creates a venv and
installs the dependencies, about 1.4 GB since the demo runs on CPU and skips
the CUDA wheels; allow a few minutes on a slow connection. The database
binds 127.0.0.1:5432, so stop anything already holding that port.
I build long-running projects with AI agents, and re-establishing context at the start of every session was the tax I paid most often. Uploading the same files, re-explaining the same decisions, watching the useful part of the window fill with material the model had already seen twice. A better model would ship and none of it moved across. The problem was never that the context was missing. It was that the context was not addressable: no way to ask a question and get back only what answers it, so you send everything and hope. The fix was to stop treating context as something you carry into a session and start treating it as something you query.
A reference implementation of a model-independent memory substrate for AI agents: project memory lives outside the model in a queryable, versioned store, exposed through one MCP gateway. Swap the model; keep everything. Claude today, Gemini or GPT tomorrow, same project memory underneath. Instead of pasting history back into every session, retrieval selects the small set of records the current task actually needs.
This repository is a clean-room reference implementation of that pattern, written from scratch against a synthetic corpus so every number on this page reproduces from a clean clone with the pinned dependencies. It is the pattern, not the system I run on my own work.
No API keys required: make demo runs entirely on local components. The
pinned local models (name and revision) are the default adapter profile,
local (make warmup, run automatically by make demo, pre-downloads
~180 MB once); the hermetic zero-download deterministic profile backs the
tests and the always-on CI gate.
src/acs/adapters/base.py is the thesis in code: four small interfaces every
external capability sits behind. src/acs/store/ treats memory as a system
of record (versioned, auditable, genuinely deletable). src/acs/retrieval/
is the pipeline as small, separately testable stages. eval/ is the gate:
changes ship by matching or beating baseline.lock.json, or they don't ship.
Design rationale lives in docs/adr/: nine decision records. Most are half a page; three run longer, where the argument needed the room.
Also in docs/: architecture.md for why each
layer exists, threat-model.md for the adversarial
boundaries and where each defence lives,
principles.md for the rules the rest follows from,
and build-your-own.md for the build sequence
this repo followed.
- Ingest writes both representations in one pass: embeddings for similarity, an entity/relation graph for structure. Every document is embedded; a document that states no relation contributes no edges.
- Retrieval is a four-stage pipeline: hybrid vector + full-text search with a tunable blend, multi-query fusion (RRF), cross-encoder rerank, and an entity-anchored partition where the graph reorders the reranked list so a topical reranker stops confusing one component with its similarly-named sibling. Per-consumer retrieval profiles are data, not code.
- Governance: retrieval changes ship by matching or beating a locked eval baseline. Memory is SCD2-versioned with time-travel reads, purge genuinely deletes (vectors included), and the audit log refuses mutation at the database level.
- Access: one MCP gateway serves the substrate to any MCP client, scoped per caller. A quarantined research loop fills gaps with citation-mandatory write-back.
- Portability: every provider sits behind an adapter; the embedder is pinned on purpose.
| you want to see | run |
|---|---|
| a query traced through every retrieval stage | make query Q="..." |
| the ablation benchmark (recall@5, MRR per stage) | make bench |
| a retrieval change fail CI against the locked baseline | make check |
| memory identities, and one memory's full version chain | make history |
| memory superseded, the old version still readable | make supersede LINEAGE=<id> CONTENT="..." |
| that memory read as it stood at a past time | make asof LINEAGE=<id> TS=<timestamp> |
| a subject purged, embeddings included | make purge SUBJECT=contributor-03 |
| supersession chains + entity neighbours from the graph | make graph ENTITY=CHG-4568 |
| two scoped callers on one substrate, one call refused | make gateway-client |
| the research loop fill a gap, cited and quarantined | make research then make vet |
The lineage id and the timestamps both come from make history: bare, it
lists current memories with their ids; with LINEAGE=<id> it walks one
memory's version chain and prints each version's validity window. Those
windows are ISO timestamps so that a TS= you paste back resolves to the
version you actually read, rather than one a second either side of it. Ids
are generated per clone, so yours will not match any shown here.
make purge genuinely deletes (golden-set targets included, when they belong
to the purged subject), so a benchmark against a purged corpus refuses to run
as incomplete rather than quietly reporting lower recall. make demo resets
to a clean corpus.
Every benchmark number on this page comes from an ablation over a seeded
synthetic corpus (corpus/generate.py), reproducible from a clean clone
and gated in CI (the deterministic profile on every push, the local profile
on the bench-local label) against a locked baseline that also locks the
corpus digest.
Read the deltas, not the absolutes: recall@5 / mrr@5 on a synthetic corpus
demonstrate what each stage contributes, never real-world quality. To see
where a stage earns its delta, run
.venv/bin/python eval/run_benchmark.py --by-family.
| stage | recall@5 | mrr@5 | what the stage earns |
|---|---|---|---|
| vector-only | 0.6333 | 0.4340 | the floor: similarity alone |
| +hybrid | 0.9667 | 0.5742 | recall. The lexical lane recovers what embeddings miss |
| +fusion | 0.9750 | 0.6026 | ranking, and almost no recall (recall +0.0083) |
| +rerank | 0.9917 | 0.9072 | ranking. mrr +0.3046 on a recall that barely moves |
| +graph | 0.9917 | 0.9315 | identity. An entity partition, not more retrieval |
Each stage earns a different thing, which is the argument for a pipeline
rather than for one better retriever. tests/test_readme_table.py fails if
this table and eval/baseline.lock.json ever disagree, so the table cannot
drift from the numbers the gate enforces. The gate is a floor, so a change
that improves a metric passes it and the numbers here stand until the baseline
is re-locked on purpose with make lock-baseline, which then fails this table
until it is brought up to match. Local profile, 120 golden queries,
k=5, stage-1 merge rank, corpus digest ad7bf7ca. Reproduce with
make demo.
What these numbers do and do not support, including why the model-lane comparison is a bound rather than a curve, is set out in docs/limitations.md.