diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 86a74c3..a69ad34 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -141,8 +141,9 @@ the UI loads — CSS, htmx, the theme script, icons — is same-origin. - [ADR-0017](docs/adr/0017-desktop-shell-wails.md) — desktop shell: Wails v2 window over the embedded web server. - [ADR-0020](docs/adr/0020-bundled-exporters-guided-setup.md) — bundled exporter toolchain in the `.app` + guided setup. - [ADR-0021](docs/adr/0021-syncthing-sync-engine.md) — device sync: supervised Syncthing engine (supersedes ADR-0018). +- [ADR-0022](docs/adr/0022-contact-merging-and-address-book-abstraction.md) — contact merging + address-book abstraction (pure-Go resolver seam, macOS provider behind a build tag). -The full set (ADR-0001–0021) lives in [`docs/adr/`](docs/adr/). +The full set (ADR-0001–0022) lives in [`docs/adr/`](docs/adr/). ## Containerization diff --git a/docs/adr/0022-contact-merging-and-address-book-abstraction.md b/docs/adr/0022-contact-merging-and-address-book-abstraction.md new file mode 100644 index 0000000..00560be --- /dev/null +++ b/docs/adr/0022-contact-merging-and-address-book-abstraction.md @@ -0,0 +1,315 @@ +# ADR-0022: Contact merging and the address-book abstraction + +- **Status:** Proposed +- **Date:** 2026-07-11 +- **Deciders:** Joe Stump +- **Related:** + - [ADR-0003 (dual-source archive with unified contacts)](0003-dual-source-archive.md) — established + `contacts` + `contact_identifiers`, auto-creation on import, and the rule + that cross-source merging is a **manual confirmation, never a heuristic**; + this ADR builds the machinery that ADR-0003 deferred to "the contacts page". + - [ADR-0011 (contact facts extraction)](0011-contact-facts-extraction.md) — prior art for + per-contact data that must survive merges and re-ingest: facts are keyed to + `contacts(id)` with idempotent, hash-deduplicated writes; merged threads + accumulate a single fact set. + - [ADR-0010 (security & privacy posture)](0010-security-privacy-posture.md) — the address-book + integration is local-only, read-only, and adds **no** network egress. + - [ADR-0020 (bundled exporters + guided setup)](0020-bundled-exporters-guided-setup.md) / + [ADR-0021 (Syncthing sync engine)](0021-syncthing-sync-engine.md) — the platform-gating + precedents: permission-probed macOS integrations behind seams, and the + `devicesync` build tag that keeps an unfinished/platform-specific feature + out of the default build (#20). +- **Tracking:** epic #8; children #9 (interface + no-op), #10 (macOS provider), + #11 (merge engine), #12 (settings UI); this ADR is #13. +- **Requirements:** [SPEC-0015 (contact merging & de-duplication)](../openspec/specs/contact-merge/spec.md) + +## Context and Problem Statement + +msgbrowse ingests identities from multiple providers (Signal, iMessage, +WhatsApp, later Telegram) into the unified contacts layer of +[ADR-0003](0003-dual-source-archive.md): `contacts(id, display_name, notes)` is +the canonical person, `contact_identifiers(contact_id, source, identifier, +UNIQUE(source, identifier))` is the source-side handle, and +`conversations.contact_id` points a 1:1 thread at its person. On import, +`UpsertConversation` (internal/store/store.go) does a transactional +get-or-create: an unseen `(source, identifier)` auto-creates a contact named +after the identifier and links it. Per-contact features already lean on this — +contact facts ([ADR-0011](0011-contact-facts-extraction.md)) key to +`contacts(id)` precisely so merged threads share one fact set. + +What is missing is the merging itself. Today the same real person shows up as +`signal:MJ`, `imessage:+15551234567`, and `whatsapp:MJ 🎸` — three contacts, +three fact sets, three journal identities. There is no engine to detect that +they match, no user-facing control to merge or split them, and no persistence +model that keeps a manual decision alive across re-ingest: `DeleteSourceData` +prunes a source's identifiers and orphaned contacts, and the next import's +get-or-create would resurrect a fresh, unmerged contact. + +On macOS, the native Contacts app is the obvious *suggestion* source — it +already maps phone numbers and emails to named people. But it is Mac-only, +permission-gated (TCC), and requires cgo bindings, so it must not become a hard +dependency of the store, the web layer, or the Linux build. + +Three decisions need capturing: the shape and injection seam of the +address-book abstraction, the platform split, and the merge/override +persistence model. + +## Decision Drivers + +- **A wrong merge corrupts derived history.** Journal digests and contact facts + propagate a bad merge forward; ADR-0003 already ruled that merging is + user-confirmed, never heuristic. The engine must default to *suggesting*. +- **Overrides must survive re-ingest.** `ReplaceConversationMessages` churns + message rowids on every import, and `DeleteSourceData` + re-enable churns + contact rowids too. The codebase's answer to rowid churn is to key durable + state by **stable identity** (message hashes for embeddings, facts, + reactions); merge decisions need the same treatment. +- **Platform integrations stay behind seams.** The web layer cannot import cgo + modules; every privileged or platform-specific capability so far is injected + (`SetDetector`, `SetEnabler`, `SetPairingSource` in internal/web) with a + documented nil/absent state. The address book must follow suit. +- **No new egress.** The Contacts lookup is a local framework call; nothing + about merging may talk to the network ([ADR-0010](0010-security-privacy-posture.md)). +- **CI and Linux build clean with `CGO_ENABLED=0`.** The default `make check` + build must not link the Contacts framework, mirroring how the `devicesync` + tag keeps Syncthing wiring out of release binaries (#20). + +## Considered Options + +### Address-book abstraction + +1. **`ContactResolver` interface in a pure-Go package, injected via a `Set…` + seam; macOS provider behind a build tag; default no-op (CHOSEN).** +2. Call the macOS Contacts framework directly from the merge engine behind + `runtime.GOOS` checks. **Rejected:** links cgo into every build, makes the + store/web layers platform-aware, and is untestable without a Mac — the exact + problems the existing seams were built to avoid. +3. Import the user's address book into the database at setup time. **Rejected:** + copies sensitive third-party PII into msgbrowse's store when a read-at-match + lookup suffices; goes stale; and turns a *hint* source into persisted state + that would then need its own sync/cleanup story. + +### Canonical-person persistence + +1. **Keep `contacts` as the canonical person; persist merge/split *decisions* + in a new identifier-keyed `contact_links` table and re-apply them in an + idempotent reconcile pass (CHOSEN).** +2. Add a `canonical_id` self-reference column on `contacts` (merge = pointing a + loser at a winner, rows never deleted). **Rejected:** every existing query + (`conversations.contact_id`, `contact_facts.contact_id`, + `ContactFactsByConversation`, sidebar joins) would need a resolve-the-alias + hop or a view; orphan cleanup in `DeleteSourceData` gets subtle; and the + alias chain still dies with the row on source delete, so it does not even + solve re-ingest survival by itself. +3. A separate `canonical_persons` table above `contacts`. **Rejected:** a second + person concept when `contacts` already *is* the canonical person + (ADR-0003); it would fork every per-contact feature (facts, journal, + conversation linking) into "which person table?". + +### Auto-matching posture + +1. **Suggest by default; exact-normalized-identifier auto-merge only as an + explicit opt-in; address book is hints-only (CHOSEN).** +2. Auto-merge on identifier equality out of the box. **Rejected:** shared family + phone numbers, recycled numbers, and stale address-book entries make exact + matches wrong often enough that ADR-0003's "manual confirmation" rule stands. + +## Decision Outcome + +### 1. `ContactResolver`: a pure-Go seam, wired like the existing ones + +A new pure-Go package `internal/contacts` defines the abstraction: + +```go +// Identifier is a normalized source-side handle. +type Identifier struct { + Kind string // "phone" | "email" | "handle" + Value string // E.164 for phones, lowercased for emails +} + +// Person is one address-book entry: a display name plus its identifiers. +type Person struct { + Name string + Identifiers []Identifier +} + +// Availability is the resolver's tri-state, mirroring the setup detector's +// permission model: absent (no provider on this platform/build), needs +// permission (provider present, OS grant missing), available. +type Availability int + +// Resolver is the address-book seam. Implementations MUST be read-only and +// MUST NOT perform network I/O. +type Resolver interface { + Availability(ctx context.Context) Availability + // People enumerates address-book entries for batch matching. An absent or + // permission-denied book returns an empty slice and no error. + People(ctx context.Context) ([]Person, error) + // LookupIdentifier returns the people matching one normalized identifier. + LookupIdentifier(ctx context.Context, id Identifier) ([]Person, error) +} +``` + +The default implementation is a **no-op resolver** (`Availability` = absent, +empty results, never an error), so the merge path degrades to +stored-identifier matching with zero conditional logic at call sites. + +Injection mirrors the established seams — and for the same reason. The web +layer cannot import the cgo desktop module, so `internal/web/enable.go` and +`internal/web/settings.go` take their privileged/platform capabilities through +`SetDetector` / `SetEnabler` / `SetPairingSource`: a `Set…` method called after +`NewServer` and **before serving begins** (handlers read the field without +locking; late wiring would race), with a documented rendered state when nothing +is wired. `ContactResolver` gets the identical contract: +`Server.SetContactResolver(contacts.Resolver)`; unset or no-op means the +settings UI renders the address-book hint option in its disabled/absent state +and the merge engine runs on stored identifiers alone. The merge engine +receives the same `Resolver` instance at construction in the cli/desktop wiring +(where `wireDeviceSync` and the Enabler are wired today). + +### 2. Platform split: macOS provider behind a build tag, Linux no-op + +The macOS Contacts provider is the only cgo consumer, so it is doubly gated, +following the `devicesync` precedent that just landed in #20 +(`internal/cli/serve_devicesync.go` / `serve_nodevicesync.go`): + +- The provider lives behind `//go:build darwin && macontacts`; a paired stub + file (`!darwin || !macontacts`) supplies a constructor returning the no-op. + The default build — and CI's `CGO_ENABLED=0 make check` — never links the + Contacts framework; the desktop shell (already cgo, [ADR-0017](0017-desktop-shell-wails.md)) + builds with the tag and wires the real provider. +- Contacts access is TCC-gated. A denied or undetermined grant makes the + provider behave exactly like the no-op for results while reporting + `Availability` = needs-permission, so the settings UI can render the same + "needs permission" guidance the setup detectors use (`internal/setup`). A + permission failure never errors the merge path. +- Identifier normalization (E.164 phones, lowercased emails) is pure Go in + `internal/contacts`, shared by the provider and the matcher, and unit-tested + in CI where the framework itself cannot run. + +### 3. Identifier matching + manual merge/split, with suggestions as the default + +The merge engine (`internal/store` methods plus matching logic in +`internal/contacts`) produces **candidates**: groups of contacts sharing a +normalized identifier value across sources, optionally augmented by +address-book grouping (two stored identifiers appearing on one `Person`) — +each candidate carrying its reason. Per ADR-0003, the address book is a +*suggestion* source, never a *decision* source: + +- **Default:** candidates are surfaced in settings for manual confirmation. + Nothing merges silently. +- **Opt-in auto-merge:** the user may enable auto-merge for exact normalized + equality on chosen identifier kinds (phone and/or email). Address-book hints + never auto-merge regardless of settings. +- **Manual merge** unions two contacts: repoint `contact_identifiers`, + `conversations.contact_id`, and `contact_facts` (dedup via the existing + `UNIQUE(contact_id, fact_hash)`) to the winner, delete the loser — the + mechanics ADR-0003 §Consequences already sketched. +- **Manual split** moves chosen identifiers off a contact onto a fresh one and + records that the affected pairs must stay apart. + +### 4. Overrides persist across re-ingest, keyed by stable identifiers + +The persistence model is a decision journal, not a pointer graph — new tables +in migration v11, no changes to `contacts` / `contact_identifiers`: + +```sql +CREATE TABLE contact_links ( + id INTEGER PRIMARY KEY, + kind TEXT NOT NULL, -- 'merge' | 'split' + origin TEXT NOT NULL, -- 'manual' | 'auto' + source_a TEXT NOT NULL, + identifier_a TEXT NOT NULL, + source_b TEXT NOT NULL, + identifier_b TEXT NOT NULL, + created_at TEXT NOT NULL, + UNIQUE(source_a, identifier_a, source_b, identifier_b) +); + +CREATE TABLE contact_merge_rules ( -- single-row settings + id INTEGER PRIMARY KEY CHECK (id = 1), + auto_merge INTEGER NOT NULL DEFAULT 0, + match_phone INTEGER NOT NULL DEFAULT 1, + match_email INTEGER NOT NULL DEFAULT 1, + use_address_book INTEGER NOT NULL DEFAULT 1, + updated_at TEXT NOT NULL +); +``` + +Links are keyed by **`(source, identifier)` pairs, not contact rowids** — the +same reasoning that keys embeddings, facts, and reactions by stable message +hashes instead of rowids: contact rowids churn (`DeleteSourceData` + re-enable +recreates them), identifiers are stable. Pairs are stored in canonical order +(`(source_a, identifier_a) < (source_b, identifier_b)`) so the `UNIQUE` +constraint dedups symmetric records, and a pair holds exactly **one current +decision**: manually merging a previously-split pair replaces the split row, +and vice versa — the latest manual action wins, and the table never contradicts +itself. A merge records the full bipartite pairing of the two contacts' +identifiers so a partially-deleted group still re-links from any surviving +pair. There is deliberately **no foreign key** from links to +`contact_identifiers`: a link whose identifier is currently absent is inert, +not invalid — it re-activates when that source is re-imported. + +An idempotent **reconcile pass** re-applies decisions after every import (and +on demand from settings): for each `merge` link whose two identifiers both +exist on different contacts, union them (winner selection is a deterministic +ordered rule: (1) exactly one contact has a user-meaningful `display_name` — +i.e. differs from all of its identifiers — that contact wins; (2) otherwise +(both or neither user-meaningful) the lower `id` wins); then, if rules enable +auto-merge, +apply exact-match merges, skipping any pair with a `split` link (**precedence: +manual split > manual merge > auto rules**), recording applied auto-merges as +`origin='auto'` links so they too survive re-ingest. Reconcile runs entirely +locally after the store write path — it is not an import side effect that adds +egress (there is none to add) and it never touches the LLM. The get-or-create +in `UpsertConversation` is untouched: it may briefly resurrect an unmerged +contact mid-import, and reconcile immediately folds it back. + +## Consequences + +### Good + +- Merge and split become first-class, reversible, durable user decisions; the + journal, facts, and transcripts address one person per human. +- Re-ingest, source disable/re-enable, and device-sync replicas + ([ADR-0021](0021-syncthing-sync-engine.md) replicas run their own ingest) + all converge to the same merged state, because decisions are identifier-keyed + and reconcile is idempotent. +- Linux, CI, and release builds carry zero cgo/Contacts surface; the seam keeps + the web layer testable with fakes exactly like the existing seams. +- Facts and future per-contact features inherit merging for free — they already + key to `contacts(id)`, and fact dedup already tolerates the union + ([ADR-0011](0011-contact-facts-extraction.md)). + +### Bad + +- A reconcile pass now runs after imports: more work in the import path + (bounded — contacts number in the hundreds, not millions) and one more + invariant ("reconcile converges, in one pass, regardless of decision order") + to test carefully. +- Merging deletes the loser's `contacts` row; if any future feature keys an + external reference to a contact id (a bookmark, or a hypothetical + contact-by-id URL — no such route exists today), that reference can dangle. + The winner-selection rule keeps ids stable where possible but not always. +- `contact_links` grows with the bipartite pairing of merged groups; a person + with many identifiers across many sources produces O(n·m) rows. Acceptable at + address-book scale, but it is bookkeeping the UI never shows directly. +- Two decisions (`macontacts` tag name, exact settings surface) are delegated to + the implementing issues and could drift; SPEC-0015 pins the behavior, not the + spellings. + +### Neutral + +- The address book is consulted live and never persisted; msgbrowse's database + gains no third-party PII beyond what imports already contain. +- Auto-merge stays off by default; users who never open settings get exactly + today's behavior plus suggestions. + +## Requirements + +Normative requirements live in +[SPEC-0015 (contact merging & de-duplication)](../openspec/specs/contact-merge/spec.md) +with design rationale in its paired +[design.md](../openspec/specs/contact-merge/design.md). Implementation is +tracked by epic #8 (children: #9 interface + no-op, #10 macOS provider, #11 +merge engine, #12 settings UI). diff --git a/docs/openspec/specs/contact-merge/design.md b/docs/openspec/specs/contact-merge/design.md new file mode 100644 index 0000000..ca2ec8e --- /dev/null +++ b/docs/openspec/specs/contact-merge/design.md @@ -0,0 +1,205 @@ +--- +status: draft +date: 2026-07-11 +implements: [ADR-0022] +--- + +# SPEC-0015 Design: Contact merging & de-duplication + +- **Capability:** contact-merge +- **Related ADRs:** [ADR-0022](../../../adr/0022-contact-merging-and-address-book-abstraction.md), + [ADR-0003](../../../adr/0003-dual-source-archive.md), + [ADR-0011](../../../adr/0011-contact-facts-extraction.md), + [ADR-0010](../../../adr/0010-security-privacy-posture.md) +- **Tracking:** epic #8; built by #9 (interface + no-op), #10 (macOS provider), + #11 (merge engine), #12 (settings UI) + +## Architecture + +The abstraction and matching logic are pure Go in `internal/contacts`; +persistence and the merge/reconcile transactions are store methods; the web +layer consumes both through the existing seam style. Only the macOS provider +touches cgo, and only tagged builds contain it. + +``` +cli / desktop wiring ──▶ contacts.Resolver (no-op | macOS provider, build-tagged) + │ │ + │ ├──▶ web.Server.SetContactResolver (availability, + │ │ hint state for settings) + │ └──▶ merge engine construction + │ +internal/contacts ── Identifier normalization (E.164, lowercase email) + │ Candidates(storedIdentifiers, resolverPeople, rules) + │ +internal/store ───── schema v11: contact_links, contact_merge_rules + │ MergeContacts / SplitContact (transactional) + │ ReconcileContacts (idempotent, post-import + on demand) + │ MergeCandidates, GetMergeRules / SetMergeRules + │ +internal/web ─────── settings merge section (boosted partial, CSP-safe, + fixed-enum result banners) → store methods +``` + +## Key design decisions + +### Decisions are identifier-keyed rows, not a pointer graph + +`contact_links` stores each merge/split decision against canonical-ordered +`(source_a, identifier_a, source_b, identifier_b)` tuples with +`UNIQUE` on the pair. This is the codebase's standard answer to rowid churn: +embeddings (v3), contact facts (v4), and reactions (v6) all key durable state +by stable message hashes because `ReplaceConversationMessages` reassigns +rowids on every import; contact rowids churn the same way under +`DeleteSourceData` + re-enable, while `(source, identifier)` is the stable +identity (`UNIQUE(source, identifier)` in `contact_identifiers` since v2). +There is deliberately no FK from links to `contact_identifiers` — an absent +identifier makes a link inert, and it re-activates when its source is +re-imported, exactly like a fact whose message hash has vanished simply loses +its jump link. + +Canonical pair ordering (`(source_a, identifier_a) < (source_b, +identifier_b)`) makes symmetric decisions collide on the UNIQUE constraint, +and "one current decision per pair" (a manual merge of a split pair replaces +the split row and vice versa) keeps the table free of contradictions without +timestamp arbitration. A merge records the full bipartite pairing of both +contacts' identifier sets so that any surviving pair re-links the group after +a partial source deletion. + +### `contacts` stays the canonical person + +Merging repoints `contact_identifiers.contact_id`, +`conversations.contact_id`, and `contact_facts.contact_id` to the winner and +deletes the loser — the mechanics [ADR-0003](../../../adr/0003-dual-source-archive.md) +§Consequences anticipated. A `canonical_id` alias column or a second person +table was rejected (ADR-0022 Considered Options): every existing query joins +`contacts(id)` directly, and an alias hop would tax all of them to solve a +problem the identifier-keyed journal solves better. Fact repointing rides the +existing dedup: `UPDATE OR IGNORE contact_facts SET contact_id = :winner WHERE +contact_id = :loser` moves unique facts, and deleting the loser contact +cascades away the duplicates the IGNORE left behind (`ON DELETE CASCADE` from +v4). All of this happens in one transaction per merge. + +### Reconcile is an idempotent post-import pass + +`UpsertConversation`'s get-or-create stays untouched: mid-import it may +resurrect an unmerged contact for a re-imported identifier, and the reconcile +pass folds it back immediately afterwards. Reconcile: + +1. applies every `merge` link whose two identifiers currently sit on + different contacts (union with the deterministic winner rule, applied as an + explicit ordered rule: (1) if exactly one contact has a user-meaningful + `display_name` — differs from all of its identifiers, i.e. was user-edited + or address-book-derived rather than auto-created — that contact wins; + (2) otherwise, when both or neither is user-meaningful, the lower `id` wins); +2. if rules enable auto-merge, merges exact normalized matches on trusted + kinds, skipping any pair carrying a `split` link (precedence: manual split + > manual merge > auto rules) and recording each applied merge as an + `origin='auto'` link so it, too, survives the next churn. + +Running it twice is a no-op by construction (all decisions applied → no +identifiers on different contacts remain for any link). It runs after every +import and on demand from settings; it performs no I/O beyond SQLite and the +optional in-memory resolver snapshot, so hooking it to the import path adds no +egress and needs no opt-in ceremony (unlike `facts`/`embed`, whose +deliberate-step rule exists because they call the LLM). + +### The resolver seam and the no-op default + +`contacts.Resolver` is minimal by demand of the matcher: availability, +`People` (batch matching), `LookupIdentifier` (spot checks / UI hints). The +no-op returns absent + empty + nil-error, so call sites carry no platform +conditionals; the merge engine treats "no resolver" and "resolver with no +matches" identically. Web wiring is `SetContactResolver` with the established +contract (wire after `NewServer`, before serving; unset renders the absent +state) because the web layer cannot import the cgo desktop module — the same +constraint that produced `SetDetector`/`SetEnabler`/`SetPairingSource` +(`internal/web/enable.go`, `internal/web/settings.go`). + +### Platform gating follows the `devicesync` precedent + +The macOS provider compiles under `//go:build darwin && macontacts` with a +paired stub for every other build, mirroring +`internal/cli/serve_devicesync.go` / `serve_nodevicesync.go` (#20): the +default `CGO_ENABLED=0 make check` build never links the Contacts framework, +and the desktop shell builds with the tag and wires the real provider. +TCC permission state maps onto the `internal/setup` detector model: +needs-permission is a rendered state, never an error. The framework call is a +thin adapter; normalization and mapping logic live in pure Go and are the +CI-tested surface (native Contacts cannot run in CI). + +### Rules live in the store, not the config file + +`contact_merge_rules` is a single-row table because the settings UI is the +owner of these values and the web layer has no config-file write path +(inventing one for three booleans would be a new, worse seam). Config keys +stay the domain of operator-set, restart-scoped concerns. Defaults are +conservative: auto-merge off, phone+email trusted once enabled, address-book +hints on (they only ever *suggest*, and the provider is permission-gated +anyway). + +## Schema (migration v11) + +```sql +CREATE TABLE contact_links ( + id INTEGER PRIMARY KEY, + kind TEXT NOT NULL, -- 'merge' | 'split' + origin TEXT NOT NULL, -- 'manual' | 'auto' + source_a TEXT NOT NULL, + identifier_a TEXT NOT NULL, + source_b TEXT NOT NULL, + identifier_b TEXT NOT NULL, + created_at TEXT NOT NULL, + UNIQUE(source_a, identifier_a, source_b, identifier_b) +); +CREATE INDEX idx_contact_links_a ON contact_links(source_a, identifier_a); +CREATE INDEX idx_contact_links_b ON contact_links(source_b, identifier_b); + +CREATE TABLE contact_merge_rules ( + id INTEGER PRIMARY KEY CHECK (id = 1), + auto_merge INTEGER NOT NULL DEFAULT 0, + match_phone INTEGER NOT NULL DEFAULT 1, + match_email INTEGER NOT NULL DEFAULT 1, + use_address_book INTEGER NOT NULL DEFAULT 1, + updated_at TEXT NOT NULL +); +``` + +Both tables are node-local derived-decision state; on device-sync replicas +([ADR-0021](../../../adr/0021-syncthing-sync-engine.md)) each node keeps its +own (the database never syncs), and reconcile makes any node converge from its +own decisions. + +## Risks / trade-offs + +- **Deleted loser ids can dangle in external references** (e.g. a bookmark to + a hypothetical contact-by-id URL — no such route exists today). Accepted: the + winner rule keeps the longest-lived/user-curated row, and contact ids are not + a stability contract today. +- **Bipartite link fan-out** is O(n·m) per merge. At address-book scale + (hundreds of contacts, a handful of identifiers each) this is trivial, and + it buys re-linking from any surviving pair. +- **Group conversations are out of scope**: `conversations.contact_id` is NULL + for groups (ADR-0003); merging affects 1:1 threads and per-contact data + only. Group-member identity mapping is a future capability. +- **Auto-merge, even opt-in, can still be wrong** (shared/recycled numbers). + Mitigated by: off by default, split-link precedence, and split being a + first-class recorded undo. + +## Testing + +- `internal/contacts`: normalization tables (E.164 variants, casing, junk), + candidate grouping with and without resolver people, reason payloads; a fake + `Resolver` covering absent / needs-permission / available. +- `internal/store`: merge transaction (identifier/conversation/fact repoint, + fact dedup on collision, loser deleted), split, decision replacement + (merge↔split on the same pair), reconcile idempotency (run twice → no-op), + precedence (split blocks auto and stored merge), re-ingest survival + (`DeleteSourceData` + re-import + reconcile converges), winner determinism, + rules round-trip and defaults. +- `internal/web`: seam contract with a fake resolver (absent / + needs-permission / available renders), settings section render, POST + merge/split happy path + fixed-enum banners, CSP compliance via the existing + template test style. +- Build gating: the tagged provider compiles only under + `darwin && macontacts`; `CGO_ENABLED=0 go build ./...` stays green with the + stub (CI already enforces this shape for `devicesync`). diff --git a/docs/openspec/specs/contact-merge/spec.md b/docs/openspec/specs/contact-merge/spec.md new file mode 100644 index 0000000..b8015df --- /dev/null +++ b/docs/openspec/specs/contact-merge/spec.md @@ -0,0 +1,189 @@ +--- +status: draft +date: 2026-07-11 +implements: [ADR-0022] +requires: [SPEC-0001] +--- + +# SPEC-0015: Contact merging & de-duplication + +- **Capability:** contact-merge +- **Target packages:** `internal/contacts` (new), `internal/store` (`schema.go` + v11, merge/reconcile methods), `internal/web` (`SetContactResolver`, settings + merge section), desktop/cli wiring +- **Related ADRs:** [ADR-0022 (contact merging & address-book abstraction)](../../../adr/0022-contact-merging-and-address-book-abstraction.md), + [ADR-0003 (dual-source archive)](../../../adr/0003-dual-source-archive.md), + [ADR-0011 (contact facts extraction)](../../../adr/0011-contact-facts-extraction.md), + [ADR-0010 (security & privacy posture)](../../../adr/0010-security-privacy-posture.md) +- **Tracking:** epic #8 (#9 interface + no-op, #10 macOS provider, #11 merge + engine, #12 settings UI; ADR/spec: #13) + +## Overview + +msgbrowse lets the user merge the same real person's identities across +providers (Signal, iMessage, WhatsApp, …) into one canonical contact, split +wrongly-merged ones, and control how candidates are detected — optionally +assisted by the native macOS address book through a pluggable, injected resolver +(the `contacts.Resolver` interface, wired via `SetContactResolver`). Merging MUST default to user-confirmed suggestions (never +silent heuristics, per [ADR-0003](../../../adr/0003-dual-source-archive.md)), +manual decisions MUST survive re-ingest and source disable/re-enable, the +address book MUST be optional on every platform (a no-op on Linux), and nothing +in this capability may perform network egress. + +## Requirements + +### REQ-0015-001: Pluggable address-book resolver with a safe absent state + +There MUST be a resolver interface — the Go identifier is `contacts.Resolver` +(package `contacts`), wired into the web layer via `SetContactResolver` — in a +pure-Go package (no cgo) exposing: a tri-state availability (absent / needs-permission / available), +enumeration of address-book people with their identifiers, and lookup of the +people matching one normalized identifier. A default no-op implementation MUST +report absent, return empty results, and never return an error, so that every +consumer works unchanged without an address book. Resolver implementations MUST +be read-only over the address book and MUST NOT perform network I/O. + +#### Scenario: No address book, merge still works +- **Given** a build with only the no-op resolver +- **When** merge candidates are computed and a manual merge is performed +- **Then** matching runs on stored identifiers alone, no error is raised, and the address-book hint surface reports itself absent. + +### REQ-0015-002: Injection seam mirroring the existing `Set…` contract + +The resolver MUST be wired into the web layer via a `Set…` method on +`web.Server` (the `SetDetector`/`SetEnabler`/`SetPairingSource` contract: +called after `NewServer` and before serving begins; handlers read the field +without locking; unset renders a documented absent state). The merge engine +MUST receive the same resolver instance at construction in the cli/desktop +wiring. The web layer MUST NOT import any cgo or platform-specific package to +consume the resolver. + +#### Scenario: Unwired resolver renders the absent state +- **Given** a server with no resolver wired +- **When** the merge settings section renders +- **Then** the address-book option appears in its disabled/absent state and every other merge control still functions. + +### REQ-0015-003: macOS provider is build-gated and permission-graceful + +The macOS Contacts provider MUST be excluded from default builds by a build tag +(with a paired stub supplying the no-op constructor, following the +`devicesync` gating precedent), so `CGO_ENABLED=0` builds and CI never link the +Contacts framework. When Contacts access is denied or undetermined (TCC), the +provider MUST behave like the no-op for results while reporting +needs-permission, consistent with the `internal/setup` permission-probe model; +a permission failure MUST NOT error the merge path. + +#### Scenario: Permission denied degrades to hints-off +- **Given** the macOS provider with Contacts access denied +- **When** candidates are computed and settings render +- **Then** matching proceeds without address-book hints, no operation errors, and the UI shows a needs-permission state for the address-book option. + +### REQ-0015-004: Normalized identifier matching produces explained candidates + +Candidate detection MUST group contacts whose identifiers are equal after +normalization — phone numbers to E.164, emails lowercased — across sources, and +MAY add candidates from address-book grouping (two stored identifiers on one +address-book person) when a resolver is available and enabled. Normalization +MUST be pure Go, shared between matcher and providers, and unit-testable +without any framework. Every candidate MUST carry a machine-readable reason +(which identifier matched, or which address-book person grouped them). +Candidates MUST NOT be silently merged by default. + +#### Scenario: Cross-source phone match is suggested, not applied +- **Given** `signal:+1 (555) 123-4567` and `imessage:+15551234567` on two contacts, default rules +- **When** candidates are computed +- **Then** the pair is listed as a candidate with the matching E.164 value as its reason, and the contacts remain unmerged. + +### REQ-0015-005: Manual merge unions the person + +A manual merge of two contacts MUST repoint the loser's +`contact_identifiers`, `conversations.contact_id`, and `contact_facts` rows to +the winner (facts deduplicating via the existing `UNIQUE(contact_id, +fact_hash)`), delete the loser's `contacts` row, and record the decision (see +REQ-0015-007). After the merge, every conversation of either former contact +MUST render the same person and one deduplicated fact set +([ADR-0011](../../../adr/0011-contact-facts-extraction.md)). + +#### Scenario: Merged threads share one fact set +- **Given** two contacts each holding a conversation and an identical extracted fact +- **When** the user merges them +- **Then** both conversations link to the surviving contact and the fact appears once. + +### REQ-0015-006: Manual split separates identifiers + +The user MUST be able to split chosen identifiers off a contact onto a new +contact. The split MUST record that the separated identifier pairs stay apart +(see REQ-0015-007), and conversations MUST follow their identifiers to the new +contact. A pair holds one current decision: manually merging a previously-split +pair MUST replace the split record, and splitting a previously-merged pair MUST +replace the merge record — the latest manual action wins. + +#### Scenario: Split pair is not re-suggested by auto-match +- **Given** a contact split into two, whose identifiers share a normalized phone value, with auto-merge enabled +- **When** reconcile and candidate detection run +- **Then** the pair is neither auto-merged nor re-applied by any stored decision, because the split record takes precedence. + +### REQ-0015-007: Overrides persist across re-ingest, keyed by stable identifiers + +Merge and split decisions MUST be persisted keyed by canonical-ordered +`(source, identifier)` pairs — never by contact rowids — with a uniqueness +constraint on the pair, an origin (`manual` | `auto`), and no foreign key to +`contact_identifiers` (a link whose identifier is absent is inert, not +invalid). A merge MUST record the full bipartite pairing of the two contacts' +identifiers. An idempotent reconcile pass MUST re-apply decisions after every +import and on demand, with precedence **manual split > manual merge > auto +rules**, and a deterministic winner chosen by an explicit ordered rule: (1) if +exactly one contact has a user-meaningful `display_name` (differs from all its +identifiers) that contact wins; (2) otherwise (both or neither user-meaningful) +the lower `id` wins. Disabling a source and +re-importing it MUST converge back to the merged state without user action. + +#### Scenario: Re-ingest cannot clobber a manual merge +- **Given** a manual merge of `signal:MJ` and `imessage:+15551234567`, then the iMessage source disabled (its identifiers and orphaned contacts pruned) and re-imported +- **When** the post-import reconcile runs +- **Then** the re-created iMessage identity is folded back onto the surviving contact, and no duplicate person remains. + +### REQ-0015-008: Merge rules are persisted settings with safe defaults + +Merge rules MUST persist in the store (a single-row settings table owned by +migration v11) and MUST cover at least: auto-merge on/off (default **off** — +suggestions only), which identifier kinds to auto-match (phone, email), and +whether to use the address book as a hint source. Enabled auto-merge MUST apply +only to exact normalized-identifier equality on the trusted kinds and MUST +record applied merges as `auto`-origin decisions; address-book hints MUST NOT +auto-merge under any setting. With default rules, behavior differs from +pre-merge msgbrowse only by the presence of suggestions. + +#### Scenario: Opt-in auto-merge applies and persists +- **Given** auto-merge enabled for phone identifiers and two contacts sharing an E.164 value +- **When** reconcile runs +- **Then** the contacts merge, the decision is recorded with origin `auto`, and a later re-ingest converges to the same merged state. + +### REQ-0015-009: Settings UI for candidates, merge/split, and rules + +The settings surface MUST let the user: review candidates with their reasons, +merge a candidate pair, split a contact, and edit the merge rules. It MUST be +rendered via the established web patterns — boosted partial with the +`*_content` template owning its `` (SPEC-0008), CSP-safe with no inline +JS, fixed-enum banner states for POST results (the `PairResult`/`UnpairResult` +pattern), and state-changing actions as same-origin POSTs. The address-book +option MUST reflect resolver availability (absent / needs-permission / +available). Merge and split MUST be presented as explicit, confirmable actions. + +#### Scenario: Merge action confirms and reports through a banner +- **Given** a listed candidate pair +- **When** the user confirms a merge +- **Then** the POST performs the merge, the page re-renders with a fixed-enum success banner naming the surviving contact, and the candidate disappears from the list. + +### REQ-0015-010: Local-only, no egress, no persisted address book + +No operation in this capability may perform network I/O +([ADR-0010](../../../adr/0010-security-privacy-posture.md)): matching, +reconcile, merge, split, and address-book lookups MUST all be local. +Address-book data MUST be consulted live and MUST NOT be persisted into the +msgbrowse database beyond user-confirmed effects (e.g. a chosen display name). + +#### Scenario: Reconcile makes no network calls +- **Given** any combination of rules, decisions, and resolver availability +- **When** import-time reconcile and candidate detection run +- **Then** no network connection is opened, and no address-book records are written to the database.