From b12c6f649fd56456f0d60ac252890efc90b1929b Mon Sep 17 00:00:00 2001 From: L4s4rt3 Date: Mon, 6 Jul 2026 14:38:14 +0200 Subject: [PATCH] docs(readme): add 'How Midas compares' table MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A skimmable side-by-side after 'Why Midas', built only from claims already measured and published in BENCHMARKS.md — including the aggregation weakness, stated in Midas's own table. Co-Authored-By: Claude Fable 5 --- README.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/README.md b/README.md index 1f220b8..be80311 100755 --- a/README.md +++ b/README.md @@ -55,6 +55,27 @@ bet, and that bet is what makes it cheap, private, and trustworthy: - **Proven, not asserted.** Every claim has a reproducible benchmark — *including the experiments that failed.* +## How Midas compares + +Every Midas number below is measured and reproducible from this repo; the LLM-at-ingest column +reflects the structural properties of that design class (Mem0, Zep, Hindsight) and the figures +documented in [BENCHMARKS.md](BENCHMARKS.md). + +| | **Midas** | LLM-at-ingest systems (Mem0, Zep, Hindsight) | +|---|---|---| +| LLM calls at ingest | **0** | ≥1 per session | +| Cost per message | **$0** | per-token API spend, forever | +| Data egress at ingest | **None** | every turn leaves the box | +| Ingest latency | **~16–116 ms**, local, embed-bound | ~668 ms + API round-trip | +| Recall returns | **verbatim source turn**, traceable | LLM-rewritten facts (source `recall@k` not computable) | +| Deterministic & reproducible | **yes — every number, one command** | no | +| Works fully offline | **yes** (measured end-to-end with a local Ollama reader) | no | +| LongMemEval-`s` judged answer (gpt-4o) | **0.84** | 0.84 — Observational Memory, with LLM ingest | +| Whole-conversation aggregation / summarization | ❌ **by design** — top-k retrieval can't cover it ([documented](BENCHMARKS.md)) | ✅ their structural edge | + +The last row is deliberate: Midas trades whole-conversation abilities for $0, privacy, and +auditability, and publishes the measurements that show exactly where that trade bites. + ## More than recall: a memory you can govern Finding a buried fact is table stakes. A long-horizon coding agent needs memory it can **act on