Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,10 +70,11 @@ Auth uses timing-safe token comparison. Origin allowlisting supports `"*"` wildc
| `remember_context` | Single write tool for chat and admin clients: save durable memory with optional topic, draft flag or explicit branch state, image input, artifact refs, and provenance metadata |
| `search_context` | Query → embedding → Firestore vector similarity search (cosine, top-K) with metadata and provenance origin filters |
| `fetch_context` | Retrieve one stored memory by document ID after search |
| `list_context` | Enumerate stored memories with filters and pagination |
| `deprecate_context` | Soft-delete: mark document as deprecated, record superseding document ID, and record supersession_reason ("changed" sets valid_until, "corrected" does not) |
| `consolidate_context` | Merge N related memories into one canonical active memory via LLM; deprecates all sources with `superseded_by` pointing to the merged result. Defaults to WIP queue for a topic; accepts explicit `source_ids` for targeted consolidation |

`remember_context` also accepts optional `valid_from`/`valid_until` (epoch-ms numbers) so a write can carry its temporal validity window from creation. It also accepts optional `origin`, `source_session`, `derived_from`, and `confidence` for provenance tracking (`origin` defaults to `agent_inferred` when omitted). `search_context` accepts optional `filter_origin` to filter search results by provenance origin as a post-filter (no new Firestore index).
`remember_context` also accepts optional `valid_from`/`valid_until` (epoch-ms numbers) so a write can carry its temporal validity window from creation. It also accepts optional `origin`, `source_session`, `derived_from`, and `confidence` for provenance tracking (`origin` defaults to `agent_inferred` when omitted). `search_context` and `list_context` accept optional `filter_origin` to filter memories by provenance origin as a post-filter.

### MCP Prompts

Expand All @@ -91,13 +92,13 @@ Auth uses timing-safe token comparison. Origin allowlisting supports `"*"` wildc
| `errors.ts` | ~9 | `HttpError` exception with `statusCode` field |
| `merging.ts` | ~74 | `LlmMergeClient` interface + `GeminiMergeClient` — calls Gemini to merge N memory contents into one |
| `runtime.ts` | ~107 | Dependency injection: `createRuntime()` lazily creates and caches Gemini clients, Firestore repo, service |
| `service.ts` | ~468 | `MetaCortexService` — remember/store/search/fetch/deprecate/consolidate flows |
| `service.ts` | ~533 | `MetaCortexService` — remember/store/search/fetch/deprecate/consolidate/list flows |
| `observability.ts` | ~150 | Structured tool-event and request-event logging plus Firestore-backed `memory_events` audit trail |
| `embeddings.ts` | ~195 | `GeminiEmbeddingClient` + `GeminiMultimodalPreparer` (image→text normalization for retrieval) |
| `normalize.ts` | ~8 | Shared text-normalization helper |
| `memoryRepository.ts` | ~228 | Firestore CRUD: `store()`, `search()` (findNearest + cosine), `deprecate()`, `getConsolidationQueue()` |
| `types.ts` | ~175 | Enums (`BRANCH_STATES`, `MEMORY_MODALITIES`, `MCP_TOOL_NAMES`, `PROVENANCE_ORIGINS`) and interfaces |
| `mcpServer.ts` | ~674 | MCP tool registration with Zod schemas, filtered by client's `allowedTools` and `allowedFilterStates`; also registers the `correct_memory` prompt unconditionally |
| `memoryRepository.ts` | ~320 | Firestore CRUD: `store()`, `search()` (findNearest + cosine), `deprecate()`, `getConsolidationQueue()`, `list()` |
| `types.ts` | ~201 | Enums (`BRANCH_STATES`, `MEMORY_MODALITIES`, `MCP_TOOL_NAMES`, `PROVENANCE_ORIGINS`) and interfaces |
| `mcpServer.ts` | ~764 | MCP tool registration with Zod schemas, filtered by client's `allowedTools` and `allowedFilterStates`; also registers the `correct_memory` prompt unconditionally |

### Data Flow

Expand Down
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,10 +57,10 @@ This project is set up for these workflows:

The current MCP surface is intentionally split between:

- a 3-tool client-facing contract for browser-hosted chat clients
- a client-facing contract for browser-hosted chat clients
- a smaller admin-only maintenance surface documented later in this README

That means the server currently exposes 5 MCP tools total, but normal browser clients should only see 3 of them.
That means the server currently exposes 6 MCP tools total, but normal browser clients should only see 3 of them by default.

### Client-facing tools

Expand All @@ -72,6 +72,8 @@ This is the public/browser contract:
Vector search over stored memories. Results include stable `id` values and artifact refs when available.
- `fetch_context`
Fetch one memory by `id` after `remember_context` or `search_context`.
- `list_context`
Enumerate stored memories with cursor pagination and metadata/creation-time/provenance filtering. Returns item summaries and IDs.

## Why `remember_context` Is The Write Tool

Expand Down
32 changes: 32 additions & 0 deletions firestore.indexes.json
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,38 @@
}
]
},
{
"collectionGroup": "memory_vectors",
"queryScope": "COLLECTION",
"fields": [
{
"fieldPath": "metadata.branch_state",
"order": "ASCENDING"
},
{
"fieldPath": "metadata.created_at",
"order": "DESCENDING"
}
]
},
{
"collectionGroup": "memory_vectors",
"queryScope": "COLLECTION",
"fields": [
{
"fieldPath": "metadata.branch_state",
"order": "ASCENDING"
},
{
"fieldPath": "metadata.module_name",
"order": "ASCENDING"
},
{
"fieldPath": "metadata.created_at",
"order": "DESCENDING"
}
]
},
{
"collectionGroup": "memory_vectors_eval",
"queryScope": "COLLECTION",
Expand Down
90 changes: 90 additions & 0 deletions functions/src/mcpServer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import {
buildConsolidatePayload,
buildDeprecatePayload,
buildFetchPayload,
buildListPayload,
buildRememberPayload,
buildSearchPayload,
MetaCortexService
Expand Down Expand Up @@ -405,6 +406,92 @@ export function createMetaCortexMcpServer(
);
}

if (allowedTools.has("list_context")) {
server.registerTool(
"list_context",
{
title: "List Context",
description:
"Enumerate stored memories without vector search. Supports cursor-based pagination and filtering by topic, state, origin, and creation time. Returns a JSON object with items, next_cursor, and applied_filters. Note: due to origin post-filtering, a page may contain fewer than limit items while next_cursor is non-null.",
inputSchema: {
filter_topic: z
.string()
.optional()
.describe(
"Optional topic or subsystem label to pre-filter, such as auth, billing, or kmp-networking."
),
filter_state: z
.enum(BRANCH_STATES)
.default(config.defaultFilterState)
.describe("Optional branch state filter. Defaults to active."),
filter_origin: z
.enum(PROVENANCE_ORIGINS)
.optional()
.describe(
"Optional provenance origin filter. Only returns memories whose provenance.origin matches exactly; memories without provenance metadata are excluded when this filter is set."
),
created_after: z
.number()
.optional()
.describe("Optional epoch-ms timestamp (inclusive). Only returns memories created at or after this time."),
created_before: z
.number()
.optional()
.describe("Optional epoch-ms timestamp (exclusive). Only returns memories created before this time."),
limit: z
.number()
.int()
.min(1)
.max(100)
.optional()
.describe("Max results to return. Defaults to 20."),
cursor: z
.string()
.optional()
.describe("Optional cursor for pagination (a document id). When set, starts after the cursor.")
}
},
async args => {
const requestedFilterState = args.filter_state ?? config.defaultFilterState;
const normalizedFilterTopic = normalizeOptionalText(args.filter_topic);
const requestSummary = {
filter_topic: normalizedFilterTopic,
filter_state: requestedFilterState,
filter_origin: args.filter_origin,
created_after: args.created_after,
created_before: args.created_before,
limit: args.limit,
cursor: args.cursor
};
const result = await observeToolCall(
"list_context",
requestSummary,
async () => {
if (!config.allowedFilterStates.includes(requestedFilterState)) {
throw new HttpError(
403,
`filter_state '${requestedFilterState}' is not allowed for this client`
);
}

return service.listContext(args);
},
listResult => ({
result_count: listResult.items.length,
result_ids: listResult.items.map(item => item.id),
filter_state: listResult.applied_filters.filter_state,
filter_topic: listResult.applied_filters.filter_topic,
has_next_cursor: listResult.next_cursor !== null
})
);

return {
content: [jsonTextContent(buildListPayload(result))]
};
}
);
}

if (allowedTools.has("fetch_context")) {
server.registerTool(
"fetch_context",
Expand Down Expand Up @@ -645,6 +732,9 @@ function buildServerInstructions(allowedTools: readonly McpToolName[]): string {
allowedToolSet.has("search_context")
? "Use search_context for retrieval and filter_topic to narrow by topic."
: undefined,
allowedToolSet.has("list_context")
? "Use list_context to enumerate memories with filters and pagination."
: undefined,
allowedToolSet.has("fetch_context")
? "Use fetch_context with the id returned by remember_context or search_context when you need the full stored record."
: undefined,
Expand Down
54 changes: 54 additions & 0 deletions functions/src/memoryRepository.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,15 @@ export interface SearchMemoryParams {
filterState: string;
}

export interface ListMemoryParams {
filterState: string;
filterModule?: string;
createdAfter?: number;
createdBefore?: number;
limit: number;
cursorId?: string;
}

export interface MemoryRepository {
store(params: StoreMemoryParams): Promise<{ document: MemoryDocument; created: boolean }>;
search(params: SearchMemoryParams): Promise<MemoryDocument[]>;
Expand All @@ -35,6 +44,7 @@ export interface MemoryRepository {
options?: { supersessionReason?: SupersessionReason; initiator?: "user" | "agent" }
): Promise<{ previousState: BranchState }>;
getConsolidationQueue(moduleName?: string): Promise<MemoryDocument[]>;
list(params: ListMemoryParams): Promise<{ documents: MemoryDocument[] }>;
}

interface FirestoreMemoryDocument {
Expand Down Expand Up @@ -232,6 +242,50 @@ export class FirestoreMemoryRepository implements MemoryRepository {
};
});
}

async list(params: ListMemoryParams): Promise<{ documents: MemoryDocument[] }> {
let query = this.firestore
.collection(this.collectionName)
.where("metadata.branch_state", "==", params.filterState);

if (params.filterModule) {
query = query.where("metadata.module_name", "==", params.filterModule);
}

if (typeof params.createdAfter === "number") {
query = query.where("metadata.created_at", ">=", params.createdAfter);
}

if (typeof params.createdBefore === "number") {
query = query.where("metadata.created_at", "<", params.createdBefore);
}

query = query.orderBy("metadata.created_at", "desc");

if (params.cursorId) {
const cursorSnapshot = await this.firestore
.collection(this.collectionName)
.doc(params.cursorId)
.get();

if (!cursorSnapshot.exists) {
throw new HttpError(400, "Invalid cursor");
}

query = query.startAfter(cursorSnapshot);
}

query = query.limit(params.limit);

const snapshot = await query.get();

return {
documents: snapshot.docs.map(doc => {
const data = doc.data() as FirestoreMemoryDocument;
return mapFirestoreDocument(doc.id, data);
})
};
}
}

function getDedupeExpiresAt(
Expand Down
65 changes: 65 additions & 0 deletions functions/src/service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import { HttpError } from "./errors.js";
import type { LlmMergeClient } from "./merging.js";
import { normalizeOptionalText } from "./normalize.js";
import type {
BranchState,
ConsolidateContextInput,
ConsolidateContextResult,
ConsolidationQueueInput,
Expand All @@ -17,6 +18,8 @@ import type {
DeprecateContextResult,
FetchContextInput,
FetchContextResult,
ListContextInput,
ListContextResult,
MemoryDocument,
MemoryMetadata,
RememberContextInput,
Expand Down Expand Up @@ -159,6 +162,50 @@ export class MetaCortexService {
};
}

async listContext(input: ListContextInput): Promise<ListContextResult> {
const limit = input.limit ?? 20;
if (limit < 1 || limit > 100) {
throw new HttpError(400, "Limit must be between 1 and 100");
}

const filterState = input.filter_state ?? this.config.defaultFilterState;
const filterTopic = normalizeOptionalText(input.filter_topic);

const { documents } = await this.repository.list({
filterState,
filterModule: filterTopic,
createdAfter: input.created_after,
createdBefore: input.created_before,
limit,
cursorId: input.cursor
});

const nextCursor = documents.length === limit ? documents[documents.length - 1].id : null;

let items = documents;
if (input.filter_origin) {
items = documents.filter(
doc => doc.metadata.provenance?.origin === input.filter_origin
);
}

return {
items: items.map(doc => ({
id: doc.id,
summary: summarizeMemoryContent(doc.content),
metadata: doc.metadata
})),
next_cursor: nextCursor,
applied_filters: {
filter_topic: filterTopic,
filter_state: filterState as BranchState,
filter_origin: input.filter_origin,
created_after: input.created_after,
created_before: input.created_before
}
};
}

async fetchContext(input: FetchContextInput): Promise<FetchContextResult> {
const id = resolveFetchContextId(input);
const item = await this.repository.get(id);
Expand Down Expand Up @@ -306,6 +353,24 @@ export function buildSearchPayload(result: SearchContextResult): Record<string,
};
}

export function buildListPayload(result: ListContextResult): Record<string, unknown> {
return {
items: result.items.map(item => ({
id: item.id,
summary: item.summary,
metadata: buildPublicMetadata({ metadata: item.metadata })
})),
next_cursor: result.next_cursor,
applied_filters: {
filter_topic: result.applied_filters.filter_topic ?? null,
filter_state: result.applied_filters.filter_state,
filter_origin: result.applied_filters.filter_origin ?? null,
created_after: result.applied_filters.created_after ?? null,
created_before: result.applied_filters.created_before ?? null
}
};
}

export function buildFetchPayload(result: FetchContextResult): Record<string, unknown> {
return {
item: {
Expand Down
Loading
Loading