Repository navigation
feat(memories): titles, English glosses, and search translations - #161
Conversation
A Memory gets an optional title (one line, at most 256 characters) and an optional English gloss (one line, at most 512) for content in another language. Both are held to lore-core validators and equal database CHECKs, flow through HTTP, SDK, CLI, MCP, Proposals, the context packet, and the Workspace archive (new format lore-workspace-v3), and are searched by four lexical name channels that answer a Memory's first chunk. They are never chunked or embedded. Migrations 0015-0017 add the columns, rules, Proposal fields, and name channels (0015), copy each valid metadata.title into title and validate the rules (0016), and build the Workspace-leading name indexes concurrently (0017). Schema revision 17, compatible_from 9. The web UI edits both, shows them under Properties, and shows the gloss as one muted line under the title in browse and search rows. Search also takes translations: the same question in up to three other languages, which the caller supplies. Memory search, Episode evidence search, and context packets run each beside the query and fuse them with the same RRF; the planner, feedback rounds, and reranker read only the original. HTTP takes repeated `translation` parameters, `translations` in an evidence search body, and `memoryTranslations` in a context packet. The MCP search tools ask the agent to send faithful translations. retrieval-v1 becomes suite version 3 with cross-lingual cases and +translations / +wrong-translations variants; results are recorded in docs/agent-conventions.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bun audit --audit-level=high now fails on main as well: markdown-it 15.0.2 renders one multi-line link reference definition in quadratic time (GHSA-gfr2-rv44-r23c), and vite 8.2.0, which vitest pulls in, has a Windows existence-check bypass (GHSA-587h-7wmj-h5pm). Move to markdown-it 15.0.3 (and raise its floor) and vite 8.2.4 within their ranges. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
lore_propose refused an update that changed or removed only the gloss: its refinement listed content, scope, title, and metadata but not gloss. lore_list and lore_search rows carried title and gloss without a budget, so 25 rows of maximal names, metadata, and text passed the 128,000- character output ceiling and the call returned only an error. Rows now cut a title past 300 and a gloss past 500 characters of JSON and mark titleTruncated or glossTruncated; detail and mutation responses return both whole. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
0016 picked its candidates and their trimmed titles in a CTE, then joined them into the UPDATE. Under READ COMMITTED, an UPDATE that waits on a row a request holds re-checks only its own quals against the row the request commits; the CTE's values stay from the old snapshot. So a request that titled a Memory, or rewrote or broke its metadata.title, while the copy waited lost its change: the copy wrote the old metadata.title and advanced the version again. 0016 now judges and reads each row in one UPDATE on memories, so the re-check sees the committed row. 0016 is not applied anywhere yet. bun run smoke:title-backfill proves it on PostgreSQL: a request holds three rows uncommitted, the copy waits on them, the request commits, and each request's change must win. CI runs it in the PostgreSQL job. The smoke target checks and the lock-wait helper move to scripts/checks/lib/smoke-database.ts so both smokes share them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Approved at 9f20ff8 after four review rounds. No findings remain.
Reviewed the full change from merge base 99d0070, then each pushed delta. The migration now preserves concurrent title writes and reads updated metadata. I independently ran smoke:title-backfill on PostgreSQL 18. Both targeted MCP regressions passed: gloss-only proposals and bounded 25-item responses.
Immediately before approval, the current PR head matched the reviewed commit. All six required CI checks passed: static, application/Core tests, PostgreSQL, packages, web/Cloudflare build, and All checks. Evidence: https://github.com/corespeed-io/lore/actions/runs/39681118050
The PR description records the translation-quality trade-off, lexical-only names, changed migration ETags, and archive-v3 compatibility. Reviewer session: 16ffca81-94a7-409f-801f-608f937887b2.
Summary
This PR adds three things that help people and agents find Memories across languages:
Behavior changes
Title and gloss
validateMemoryTitleandvalidateMemoryGlossin lore-core hold both fields to one rule. The database CHECKs equal those validators. The fields flow through HTTP, the SDK, the CLI (--title,--no-title,--gloss,--no-gloss), MCP, Proposals, context packets, and the Workspace archive.metadata.titleintotitle, then validates the rules. If a request changes a Memory while the copy waits on that row, the request's change wins.bun run smoke:title-backfillchecks this on PostgreSQL, and CI runs it.compatible_fromstays 9, so an older application instance keeps working during a rolling deploy.lore_listandlore_searchrows cut a title past 300 or a gloss past 500 characters of JSON, and mark it withtitleTruncatedorglossTruncated. This keeps 25 maximal rows under the 128,000-character output limit. Detail and mutation responses return both whole.Search translations
translationparameters besideq. Atranslationwithoutqis refused.translations.memoryTranslations.translationsandmemoryTranslations, and the CLI takes a repeatable--translation.lore_search,lore_search_evidence, andlore_retrieve_contexttell the agent to send a faithful translation when the Workspace may hold another language.Defaults and scope
Decisions for the PM
LORE_RERANK_MIN_SCORE.updated_at, list order, and cursors do not move.lore-workspace-v3. A release before schema revision 15 cannot import it, so upgrade the target of a Workspace move first.memoryTranslations. Three 10,000-character echoes would push Code evidence out of MCP's 128,000-character output limit.Evidence
The retrieval-v1 suite is now version 3:
+translationsvariants, and+wrong-translationsvariants that measure the risk above.Results, from Ollama
qwen3-embedding:0.6bwith no reranker or planner:This suite holds only 34 Memories. On the HaaS production corpus, English questions about notes written mostly in CJK reached only Recall@5 63% with semantic search alone.
Local checks run on the merged head:
bun run test: 1,509 tests and 59 lore-core tests passed.sdk:check,design:check, andarchitecture:checkpassed.build,packages:smoke, andsmoke:nextpassed.wrangler deploy --dry-runpassed.smoke:memory-corewith a maintenance cycle passed, anddb:migratebuilt a fresh database through 0017.Dependency fix
bun audit --audit-level=highbegan failing on main too, from two new advisories:A separate commit moves them to markdown-it 15.0.3 (and raises its floor) and vite 8.2.4, both within their ranges. All tests pass on the new versions.
Known follow-ups
🤖 Generated with Claude Code