Skip to content
This repository was archived by the owner on Jul 15, 2026. It is now read-only.

Consolidate freshness, semantic-index status, and MCP config onto the Overview - #24

Merged
joestump-agent merged 6 commits into
mainfrom
feature/1-overview-consolidation
Jul 11, 2026
Merged

Consolidate freshness, semantic-index status, and MCP config onto the Overview#24
joestump-agent merged 6 commits into
mainfrom
feature/1-overview-consolidation

Conversation

@joestump-agent

@joestump-agent joestump-agent commented Jul 11, 2026

Copy link
Copy Markdown
Owner

Part of #1. Closes #1.

What

Turns the Messages landing view into a single Overview that surfaces freshness + indexing + MCP connection, instead of hopping between /status and /settings.

  1. Archive freshness with per-provider breakdowns — wires store.SourceCounts + store.LastSyncTimes so Signal/iMessage/WhatsApp each show their own message/conversation counts and last-synced stamp, alongside the global strip and NewestTS.
  2. Semantic-search indexing status — embeddings coverage (embedded vs total), last index run, and an in-progress indicator. Adds an embed_runs table (store records begin/heartbeat/finish; internal/embed writes them) and a store-level coverage query; a 10-minute heartbeat window (web.embedRunStaleAfter) drives the "in progress" vs "stalled" call.
  3. MCP server config below the stats — relocates the endpoint URL + client JSON + claude mcp add block onto the Overview, keeping the /static/copy.js copy-button affordances.
  4. Follows the SPEC-0008 boosted partial pattern (*_content define owning <title>, cheap isPartialRequest path via ArchiveStats).

Design decisions (resolving the ticket's open questions)

  • /status and /settings stay canonical — nothing redirects (the existing code comment made this the conservative default); the Overview cross-links them.
  • Device pairing stays on Settings, per the ticket.
  • Snapshots table is NOT moved here — it graduates to a dedicated Backups tab in Move encrypted DB snapshots into a dedicated Backups tab #2, which builds on this branch. Nav/shell is structured so that tab slots in cleanly.

Notes

Reviews flagged that the two extra freshness queries must not scan messages on every / render — the fix stage confirmed they ride existing indexes and gated the MCP card behind its config, plus corrected a model-agnostic-vs-model-scoped mismatch in the coverage label.

🤖 Generated with Claude Code

Generated by Claude Code

claude added 6 commits July 11, 2026 08:37
@claude
store: record embedding runs and expose semantic-index coverage
392bcc6 
Schema v11 adds embed_runs — the embeddings analogue of ingest_runs — so a
semantic-search indexing run leaves a durable trail: a row at start
(finished_at='' marks in-flight), a per-batch updated_at heartbeat, and a
terminal write with the totals or the abort error. The embed CLI and
'msgbrowse serve' are separate processes sharing one SQLite file, so this
table is how the web Overview observes a live run (fresh heartbeat), a
crashed one (stale heartbeat), or the last completed one (issue #1).

EmbeddingCoverage is the called-out store-level coverage query: one LEFT JOIN
pass over messages x the embeddings PK counting the embeddable corpus
(non-system, non-blank — the exact CountMissingEmbeddings predicate, so
Embeddable-Embedded always equals the pending count) and the subset already
carrying a vector for the model.

Part of #1

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@claude
embed: record each run in embed_runs (begin/heartbeat/finish)
75e166d 
Run now wraps the embedding loop with best-effort run recording: BeginEmbedRun
before any work, UpdateEmbedRunProgress after every batch (the liveness
heartbeat the Overview reads), and FinishEmbedRun on the way out — under
context.WithoutCancel so a Ctrl-C'd run still lands its terminal write instead
of reading as crashed forever. Recording failures log a warning and never
abort the embedding work itself.

Part of #1

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@claude
web: consolidate freshness, index status, and MCP config onto the Ove…
4ca133d 
…rview

The Messages landing screen (/) is now the single Overview (issue #1):

- Archive freshness at the top: the existing global stat strip, expanded
  with a per-provider table (Signal/iMessage/WhatsApp) wiring
  store.SourceCounts + store.LastSyncTimes — each provider's own
  conversation/message counts and last-synced stamp, in canonical source
  order, rows only for providers with data or a recorded run.
- Semantic search index card: embeddings coverage for the currently
  configured embed model (live via the LLM-tab configurator when wired),
  the last recorded index run, a live 'Indexing…' state driven by the
  embed_runs heartbeat, and honest 'Interrupted'/'Failed' states. An unset
  embed model renders a pointer to Settings → LLM, never fake zeros.
- MCP connection card below the stats: the endpoint URL, client JSON, and
  claude mcp add blocks extracted from settings_content into a shared
  mcp_connect_card define — Settings keeps rendering the identical card, so
  the two surfaces can never drift; copy.js affordances unchanged.

Deliberate decisions (the ticket's open questions): settings_subnav stays
as-is and the Overview stays outside the Settings shell; GET /status and
GET /settings remain canonical with nothing redirecting (the joestump#163 contract);
device pairing stays on Settings; snapshots stay on /status for the Backups
tab to claim later. The boosted-partial contract is untouched: index_content
still owns <title> + #main-content, and the HX-Request path still uses the
cheap ArchiveStats aggregate — the new queries are all small aggregates run
on both paths.

app.css: regenerated (the new mt-4 utility); no hand edits.

Part of #1

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@claude
web: address Overview cross-review — model-mismatch label, MCP card g…
2e51a1b 
…ate, stale window

Cross-review follow-ups on the Overview consolidation (all minor):

- Semantic-index card could silently disagree with itself after a user
  switched llm.embed_model: coverage is scoped to the configured model but
  LatestEmbedRun is model-agnostic. Surface the last run's model in the
  "Last index run" line when it differs from the configured one so the two
  halves explain rather than contradict each other.
- Move the MCP connection card outside the {{if .HasArchive}} gate on the
  Overview so a fresh, un-ingested user can wire up their client before
  importing — restoring the "same card on / and /settings" invariant.
- Raise embedRunStaleAfter 10m -> 30m: a large batch (up to 512) against a
  slow local embedding endpoint can exceed 10m per heartbeat, and misreading
  a live run as "Interrupted" invites a second concurrent embed against the
  same SQLite file — the costlier error.
- Note the accepted cost of the per-render EmbeddingCoverage full scan on the
  "/" hot path (revisit with a cached aggregate at the millions-of-messages
  target).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@claude
Rebuild app.css from the Tailwind toolchain (fix css-fresh CI check)
60d640d 
The committed stylesheet didn't match a clean 'make css' build; regenerated
with the pinned Tailwind v4.3.1 + daisyUI 5.6.3 toolchain.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@claude
Merge remote-tracking branch 'origin/main' into feature/1-overview-co…
ea09a2f 
…nsolidation
@joestump-agent
joestump-agent merged commit 1e0defa into main Jul 11, 2026
3 checks passed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Consolidate Settings + Status into the Overview (Messages) landing screen

2 participants

@joestump-agent @claude