Engram
Grounded, cited answers from your own documents — local or cloud, set up in one command.

---

Status: pre-alpha. Not on PyPI yet — install from source (Install). API may change before v0.1.0. Full evolution in CHANGELOG.md.

Engram is a Python library + CLI that answers questions over your own documents and cites the exact source (file and page) behind every claim, so a human can verify it. It runs fully on your machine (local models via Ollama, on-disk index, no data leaving the box) or against your own cloud API key. We never ship keys — local runs cost you nothing.

• Point it at a PDF, ask a question, get a cited answer. Built-in loaders (PDF/txt/md/docx), grounded reader, page-level citations.
• Local by contract. profile = "local" hard-fails if any component could send text off the machine — the answer to HIPAA/PHI and air-gapped setups.
• One folder of storage. LMDB + a vector index on local disk. No Postgres, no Pinecone, no server to run.
• Proven retrieval. Hybrid (dense + BM25 + RRF) + reranking + IRCoT + a grounded synthesis reader by default; an entity/fact knowledge graph and an adaptive per-query router are opt-in.

Coding agents: see AGENTS.md for an unattended setup runbook.

Quickstart — fully local (nothing leaves your machine)

Best for medical/PHI, legal, or any air-gapped use.

Prerequisites: Python 3.11–3.13 (not 3.14 yet — some native deps lack 3.14 wheels), Ollama installed and running, and ~8 GB free disk plus enough RAM to run an 8B model (llama3.1:8b ≈ 5 GB). A GPU is optional but speeds up the reader.

Engram does not download models for you. You pull them once with Ollama; engram init then detects what you have and prints ollama pull commands for anything missing. Nothing is auto-installed behind your back.

1. Install and start Ollama

Download from ollama.com and make sure it's running (launch the desktop app, or run ollama serve). Engram talks to it at localhost:11434, so it must be up for every step below.

ollama --version     # confirms Ollama is installed
ollama list          # confirms the server is running (the list may be empty)

2. Pull one embedder and one reader

ollama pull nomic-embed-text     # embedder  (~275 MB)
ollama pull llama3.1:8b          # reader    (~4.9 GB; use a 70B-class model if you have the RAM/GPU)

Check: ollama list now shows both.

3. Install Engram (from source until the PyPI release)

git clone https://github.com/Vrin-cloud/engram && cd engram
uv venv && uv pip install -e ".[local]"
# no uv?  →  python3.13 -m venv .venv && .venv/bin/pip install -e ".[local]"

Check: engram --help prints the four commands.

4. Configure for local

engram init --local

Writes engram.toml, auto-detects the models you pulled, and prints an ollama pull … line for anything still missing — run those if you see them.

5. Ingest your documents, then ask

engram ingest ./discharge_summary.pdf     # a file, a glob (./docs/*.pdf), or a directory
engram ask "Which medications should be stopped at discharge, and why?"

The first ask is slow: it loads the reader into memory and downloads the local reranker (~1 GB) once. Later questions are faster.

Expected output:

Lisinopril and glyburide should be stopped. Lisinopril was replaced by
sacubitril/valsartan, which requires a 36-hour washout (Document 1). Glyburide
was discontinued due to hypoglycemia risk in CKD (Document 1). Metoprolol was
held pending cardiology follow-up (Document 1).

Sources:
  [1] discharge_summary.pdf p.1

(route: hybrid+ircot · 41s)

Local latency depends on your hardware and model. On a laptop, an 8B reader with IRCoT runs ~30–45s per question; a GPU box, a smaller reader, or [rerank] mode = "off" are faster. Cloud readers (gpt-4o-mini) answer in a few seconds.

Confirm nothing can leave the machine:

engram doctor
#   embedder   ollama/nomic-embed-text   → localhost:11434
#   reader     ollama/llama3.1:8b        → localhost:11434
#   reranker   local                     → in-process
#   store      ./.engram                 → local disk
#   egress: none — nothing can leave this machine.

Or: cloud (your own API key)

uv pip install -e ".[llm,memory,query,loaders]"
export OPENAI_API_KEY=sk-...            # your key; add AWS creds for Cohere rerank
engram init --cloud
engram ingest ./report.pdf
engram ask "..."

Troubleshooting

Symptom	What it means / fix
engram: command not found	The venv isn't active. source .venv/bin/activate, or run python -m engram.cli ….
ConnectionError / connection refused	Ollama isn't running. Start the app (or ollama serve) and confirm with ollama list.
model "…" not found when you ingest or ask	That model isn't pulled. ollama pull <name> (the name shown in engram.toml), then retry.
pip install fails building lmdb / hnswlib / torch	You're on Python 3.14. Use 3.11–3.13: python3.13 -m venv .venv.
ConfigError: profile='local' forbids network egress	A cloud model (openai/*) or rerank.mode = "cohere" is in a local config. Set [models] to ollama/* and [rerank] mode = "local" (or "off").
engram ingest reports 0 chunks from a PDF	It's a scanned/image PDF (no text layer). OCR it first — out of scope for v1.
The first ask hangs for ~a minute	Normal on the first call only: model load + a one-time reranker download. Later asks are fast.
Answers are weak on dense tables/abbreviations	The 8B local reader is the limit. Use a larger reader (70B-class) in engram.toml; retrieval + citations don't change.

Coding agents doing setup unattended: follow AGENTS.md — same steps as ordered checks, plus decision gates.

Python API

from engram import Engram

eg = Engram()                       # reads engram.toml / env; builds everything
eg.ingest("discharge_summary.pdf")  # PDF/txt/md/docx, a glob, a dir, or Chunk[]
ans = eg.ask("What medications was the patient discharged on?")

print(ans.text)          # answer, with "Document N" cited inline
print(ans.citations)     # [Citation(document=1, source='discharge_summary.pdf', page=3, quote='...')]

The Answer object carries everything you need to verify or render a result:

Field	Meaning
.text	The answer; cites Document N inline
.citations	list[Citation(document, source, page, quote)] — maps each Document N to its source file + page
.chunk_ids	Retrieved chunk ids, in evidence order
.route	Which retrieval path ran (hybrid+ircot, hybrid+kg+ircot, …)
.latency_ms	End-to-end wall-clock for the query

str(ans) returns ans.text. Sync methods (ingest, ask) wrap async ones (aingest, aask).

Local & HIPAA

Engram is designed so PHI never has to leave the building:

• profile = "local" is a contract, not a hint. Constructing Engram() under a local profile raises ConfigError if the embedder, reader, or reranker could make a network call. You cannot accidentally ship PHI to a cloud API.
• engram doctor prints every component and its endpoint, ending in egress: none — nothing can leave this machine. — hand it to a compliance reviewer.
• Storage is one folder. LMDB holds chunks, facts, entities, and vectors on local disk; the vector index (hnswlib) and knowledge graph (networkx) are rebuilt from it on open. Back up the folder, move it between machines, delete it to forget. No external database or service.
• We never ship API keys. You point Engram at your own local models (Ollama) or your own API key. Local runs have no per-query cost and no vendor in the loop.

Cloud models can be HIPAA-compliant with a signed BAA (OpenAI and AWS both offer one) — but "the vendor doesn't train on API data" is not sufficient on its own. When in doubt, use profile = "local".

How it works

By default ask() runs hybrid retrieval (dense + BM25 fused with Reciprocal Rank Fusion, elbow-cut — no hardcoded top-k) → rerank → IRCoT (a second retrieve-and-reason round) → a grounded synthesis reader constrained to answer only from the retrieved evidence and cite each Document N. This is the robust default and is the best config for single-document Q&A.

Two opt-in modes go further on multi-document corpora:

• --mode kg / Engram(query_mode="kg") — adds an entity + fact knowledge graph: triple-vector ANN match, two-stage Personalized PageRank, and multi-hop beam search fused into retrieval. Requires ingesting with --graph.
• --mode auto — a strategic router makes one token-minimal LLM call per question and picks the capabilities that question needs (statistical parity with the full static stack at ~40% lower latency).

Configuration

engram.toml (written by engram init, hand-editable):

[models]
embedder = "ollama/nomic-embed-text"   # or "openai/text-embedding-3-small"
reader   = "ollama/llama3.1:8b"         # or "openai/gpt-4o-mini"

[rerank]
mode  = "local"    # local (in-process cross-encoder) | cohere (Bedrock) | off
model = "bge-reranker-base"

[store]
path = "./.engram"                      # LMDB + vector index live here

[runtime]
profile = "local"  # local | cloud     (local hard-fails on any egress)
mode    = "hybrid" # hybrid | kg | auto (auto = adaptive per-query router)

mode = "auto" puts the strategic router in charge per query — best on multi-document corpora with a capable reader (70B-class or cloud). On small local models it can over-route and its structured sub-calls are less reliable, so hybrid is the default.

Searched in ./engram.toml, ./.engram/engram.toml, then ~/.engram/. Environment variables override the file:

Variable	Overrides
ENGRAM_READER	[models] reader
ENGRAM_EMBEDDER	[models] embedder
ENGRAM_RERANK	[rerank] mode
ENGRAM_STORE	[store] path
ENGRAM_PROFILE	[runtime] profile
OPENAI_API_KEY	your key for cloud OpenAI models
AWS default chain	your creds for Cohere Rerank on Bedrock

Full reference: docs/configuration.md.

CLI reference

Command	What it does
engram init [--local|--cloud] [--store PATH] [--force]	Write engram.toml. Auto-detects a running Ollama; prints ollama pull hints for missing models.
engram ingest <paths…> [--graph]	Load files/globs/dirs (PDF/txt/md/docx) into the store. --graph also builds the knowledge graph (slower).
engram ask "<question>" [--mode hybrid|kg|auto] [--json]	Answer from the ingested corpus. Prints the answer, sources, and route; --json emits the full Answer.
engram doctor	Print every component → endpoint and whether any data can leave the machine.

Install

Not on PyPI yet. Package name reserved for v0.1.0.

git clone https://github.com/Vrin-cloud/engram && cd engram
uv venv && uv pip install -e ".[local]"     # fully-local stack

Extra	Brings in
local	llm, memory, query, loaders, rerank, ollama — the complete fully-local stack
llm	litellm, instructor, tenacity — the LLM provider
memory	lmdb, hnswlib, numpy, networkx, scipy — the on-disk store
query	rank-bm25 — the sparse leg of hybrid retrieval
loaders	pypdf, python-docx — PDF/DOCX ingestion
rerank	sentence-transformers — the local cross-encoder reranker
ollama	the Ollama client for local models
observability	opentelemetry — tracing spans
all	every extra

API keys (only if you choose cloud components)

Var	Used by
OPENAI_API_KEY	openai/* reader + embedder
AWS default chain	Cohere Rerank 3.5 on Bedrock (rerank.mode = "cohere")

Local (Ollama) profiles need no keys.

Measured performance

MuSiQue dev (n=200, gpt-4o-mini reader, text-embedding-3-small, Cohere Rerank 3.5):

Config	F1	EM	Notes
Hybrid + rerank	0.46	0.32	no-IRCoT default
+ IRCoT	0.54	0.40	= field SOTA at this reader
+ Knowledge graph	0.51–0.53	0.36–0.39	adds graph capabilities; no F1 lift over IRCoT
Adaptive router (--mode auto)	0.52	0.38	within run variance at ~40% lower median latency

On single-document Q&A, plain hybrid is best — the knowledge graph adds nothing there, so the default hybrid mode is both the fastest and the most accurate choice. Methodology in docs/benchmarks.md; reproduce with python -m benchmarks.musique.

Comparison to adjacent systems

System	Where it fits
Cohere / Voyage Rerank	Engram uses Cohere Rerank (or a local cross-encoder) — a building block, not a competitor
GBrain	Production hybrid-retrieval reference; Engram extends it with IRCoT + an entity/fact KG
HippoRAG 2	Closest to Engram's KG mode (OpenIE triples + PPR)
PropRAG	Two-stage PPR is ported (engram.core.kg_retrieval)
LangChain / LlamaIndex	Frameworks; Engram is a focused library that either can wrap

Codebase map

Path	What's there
src/engram/dialogue/orchestrator.py	Engram — ingest / ask (high-level) + aenrich (enrichment), Answer, Citation
src/engram/config.py	engram.toml + env resolution, profile="local" egress contract, component builders
src/engram/loaders.py	PDF/txt/md/docx → Chunk[] with page provenance
src/engram/rerank_local.py	Local cross-encoder reranker (drop-in for Cohere)
src/engram/cli.py	engram init / ingest / ask / doctor
src/engram/query/	Retrieval + IRCoT + grounded reader (answer_one, hybrid_neighbors, kg_hybrid_neighbors)
src/engram/backends/memory.py	MemoryBackend — LMDB + hnswlib + networkx graph
src/engram/dialogue/strategic_router.py	Adaptive per-query router (--mode auto)
benchmarks/	MuSiQue + custom-corpus evaluation harnesses

Deep docs: architecture · benchmarks · configuration · KG internals · LLM provider.

What's original, and what we ported

Ours: the strategic router, graph-aware retrieval planner, derived-fact engine with a 6-filter hallucination cascade, hot/cold ingest split, bi-temporal supersession, no-hardcoded-limits retrieval. Ported from our production engine (Vrin): confidence-decayed beam search, entity canonicalization, fuzzy fact dedup, multi-hop decomposition, cluster-gap cutoff. From the literature: IRCoT (Trivedi et al. 2023), query-to-triple ANN + PPR (HippoRAG 2), two-stage PPR (PropRAG), Leiden communities (GraphRAG), Noisy-OR fusion (Knowledge Vault), BM25 + dense + RRF + Cohere Rerank (standard practice).

Contributing

See CONTRIBUTING.md. Commits carry a Signed-off-by trailer per the DCO.

Security

Report vulnerabilities to vedant@vrin.cloud. See SECURITY.md.

License

Apache License 2.0. See LICENSE and NOTICE.