Providers redesign, LLM API-key field, and gate device sync out of the binary - #20
Conversation
Reworked onto current main (Providers is now a Settings tab; the LLM tab and live-apply provider swap already exist upstream). LLM API key (Settings → LLM tab): - Add an API-key field to the tab. It's a write-only secret: the value is never rendered back (the form shows only whether a key is set), a blank submission keeps the current key, and it applies live like the other LLM settings (swaps the running client, no restart). - Persist it to the 0600 config file via config.SaveLLM (llm.api_key), and thread it through llm.Settings + the Applier. MSGBROWSE_LLM_API_KEY still overrides at startup. Providers page (now the /providers Settings tab): - 2×2 grid of source cards plus a Telegram "coming soon" placeholder (not wired into internal/source). - Remove the one-shot "Refresh all sources" button/route and replace it with a background auto-refresh scheduler (providers.refresh_interval, default 6h; 0 disables), reusing the per-source job guard. - "Last synced" per card from a new store.LastSyncTimes over ingest_runs. Device sync gated out of the binary (ADR-0021 / SPEC-0014): - Behind the `devicesync` build tag. The default `msgbrowse` binary links with zero internal/syncthing symbols and no engine constructors; build with `-tags devicesync` to include it. - serve + desktop wiring, `msgbrowse devices`, and doctor's device checks split into tagged/stub files behind a wireDeviceSync seam. - The Device sync UI on /settings and /status is hidden via a compile-time feature flag (web.SetDeviceSyncFeature). Both build variants pass `go build`/`go test` for the main and desktop modules. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PruXCwhbcG7C6U88x7wykZ
9cb1c7c to
39f1f06
Compare
joestump
left a comment
There was a problem hiding this comment.
Each of the three pieces is competently executed in isolation — write-only key field with a set/keep indicator, the Providers grid + scheduler reusing refreshEnabledSources under the Runner's per-source guard, build-tag stubs with feature-gated tests, and everything builds/vets/tests clean under all tag combinations ("", devicesync, devicesync desktop). But this PR bundles several load-bearing problems that need fixing before it lands. All of the below were verified against the branch, not just eyeballed.
Blocking
1. An env-only API key gets silently persisted to disk. A blank key field resolves to s.currentLLM().APIKey (llmsettings.go, keptKey path) — and at boot viper maps MSGBROWSE_LLM_API_KEY → llm.api_key (config.go), so the live key may be the env secret. Any unrelated save (say, editing the base URL) then passes it to config.SaveLLM, which now always writes llm.api_key. A user who deliberately kept the key out of files — the previously documented posture — has it copied into config.yaml, where it survives backups and dotfile syncs, and unsetting the env var no longer removes it. The old save.go guaranteed exactly the opposite ("a pre-existing llm.api_key … this code never writes"). Suggested fix: only persist a key the user explicitly typed into the form; a kept key should keep the file's value, not the resolved live value.
2. A stored key can never be cleared from the UI — and it follows you to new endpoints. Blank-means-keep leaves no removal affordance, directly contradicting SaveLLM's doc ("An empty apiKey is written as an empty value (clearing any previous key)"). Worse: repoint Base URL at a different provider/local proxy and the retained key is sent as the Authorization bearer to the new host — a credential leak to an unintended party. Needs an explicit "clear key" control (checkbox or button), and arguably a nudge when base_url changes while a key is being kept.
3. ADR-0009 / ADR-0010 still say the opposite. ADR-0009 decision 6 ("Secrets via env only", with "Secrets in the config file" listed as rejected) and ADR-0010 decision 6 ("The LLM API key is env-only … never baked into the image or a committed file") are both still Accepted and unmodified, and this PR also deletes the test that pinned "SaveLLM wrote an api_key — it must never do that". If key-in-0600-config is the product choice (it's defensible for a desktop app), amend/supersede both ADRs in this same PR so docs/adr keeps describing reality.
4. device_sync.enabled: true is silently ignored in the shipping build. The untagged wireDeviceSync stub returns nil without even a log line (serve_nodevicesync.go) — on main, serve with sync enabled either actually replicated or failed fast on an unstartable engine. And doctor_devices_stub.go reports statusPass unconditionally. Since every default artifact is untagged (make build, Dockerfile, and DESKTOP_TAGS all omit devicesync), an operator with paired replicas who upgrades gets: serve starts green, replication silently stops, doctor says PASS. Minimum fix: both stubs should check cfg.DeviceSync.Enabled and log/report a prominent warning that the configured feature is not in this binary.
5. The build tag doesn't deliver what the comments claim. internal/web still unconditionally imports internal/devsync (statusData/SetSyncMonitor/logs feed), which pulls internal/syncthing — go list -deps ./cmd/msgbrowse on the untagged tree lists both packages. The stubs' "links WITHOUT internal/devsync or internal/syncthing" claim is false today: only the wiring and UI are gated; the not-release-ready code still ships in the binary. Either sever the dependency at the web boundary (web-local status types or a tagged web file) or correct the comments and the PR's "gated out of the binary" claim.
6. "Last synced" vanishes on every card fragment swap. Only setupCards() backfills LastSynced/HasLastSynced (setup.go); the fragments rendered by the enable/refresh status poller (enable.go), /setup/recheck (recheck.go), and the disable flow (disable.go) call setupCardFor directly, so the stamp disappears at the exact moment a refresh completes and only returns on full page reload. Move the backfill into setupCardFor (or fetch LastSyncTimes there).
7. Hand-edited generated app.css. Same as #19: make css on this branch produces a diff, so the css.yml drift guard fails; and this PR's appended tail conflicts with #19's on the same minified line. Regenerate and commit; never hand-resolve that conflict.
Should-fix
8. "Last synced" advances on all-failed runs. LastSyncTimes is MAX(finished_at) with no errors filter, and per-conversation import failures still record the run row (run.Errors++ then unconditional RecordIngestRun). A broken/unparseable archive shows "Last synced just now" forever while importing nothing. (Scoped: a total exporter failure records no row and surfaces as a failed job — this is specifically the partial/parse-failure class.) Consider WHERE errors = 0 or exposing the error count on the card.
9. Issue joestump#162 footer regression. The deliberately three-way footer collapsed to two-way: with nothing detected and nothing enabled (fresh machine), /providers now claims "Enabled sources refresh automatically … use a source's Refresh icon" while every card reads Not detected and renders no Refresh control — the same footer-contradicts-cards class joestump#162 fixed. Restore the empty-state branch; also consider the copy when refresh_interval: 0 disables auto-refresh.
10. Docs not updated for the gating. README §Device sync and docs-site features/device-sync.md still walk users through device_sync.enabled, Settings → Device sync, and msgbrowse devices … — all unreachable in a default build; only config.example.yaml was touched, and the devicesync build tag is documented nowhere user-facing.
11. autocomplete="off" on the key field. Chrome/Safari ignore off on type=password; use autocomplete="new-password". Otherwise a password manager can autofill a login password that — because the field never echoes and non-blank replaces — silently becomes the API key on an unrelated save.
Non-blocking cleanups
LastSyncTimesinlinestime.Parse(time.RFC3339, …);parseRFC3339(used byLatestIngestRunon the same column) has different failure semantics — use one path.validateLLMAPIKeyis a byte-for-byte clone ofvalidateLLMModelmodulo the length constant — share avalidatePrintable(s, max).- The "Last synced" stamp is the third literal copy of the
"2006-01-02 15:04"local-format (settings.go has PairedAt/LastSeen) — worth one helper. .setup-card-coming-soon/.setup-badge-coming-soonduplicate the not-detected pair's values; andsetup_coming_soon_cardtakes the source name as a parameter while hardcoding Telegram's tint/icon — either parameterize fully or hardcode honestly.- Each 6h tick re-runs the full export per enabled source even when nothing changed (only the import is incremental) — a cheap staleness pre-check (source DB mtime vs last
finished_at) would spare large archives minutes of subprocess churn 4×/day. Relatedly, the scheduler makesingest_runsgrow unboundedly (~1.5k rows/source/year) andMAX(finished_at)has no(source, finished_at)index.
For the record, one suspicion was checked and refuted: the auto-refresh goroutine's lifecycle is handled correctly — both call sites create the ctx via signal.NotifyContext with defer stop(), no tick can fire in the failure window, and a closed store degrades to logged warnings.
🤖 Posted on behalf of @joestump by Claude.
…les) The css-fresh CI check diffs a scratch 'make css' run against the committed stylesheet, so the hand-appended setup-card rules failed it. Rebuilt with the pinned Tailwind v4.3.1 + daisyUI 5.6.3 toolchain. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YWWaS4f9mr6Yp43S4GXHE7
Review follow-up: the threat-model docs still claimed the LLM API key is env-only and never expected in a file, which this PR's Settings → LLM persistence made false. State the real posture: env always wins at startup, the Settings tab persists to the 0600 config file, and that file must never be committed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YWWaS4f9mr6Yp43S4GXHE7
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YWWaS4f9mr6Yp43S4GXHE7
…tion (#22) * docs: ADR-0022 + SPEC-0015 for contact merging & address-book abstraction Add ADR-0022 (MADR) deciding the ContactResolver seam (mirroring the SetDetector/SetEnabler/SetPairingSource injection contract), the platform split (macOS Contacts provider behind a darwin+macontacts build tag with a no-op default, per the devicesync gating precedent from #20), the suggest-by-default matching posture (ADR-0003's manual-confirmation rule), and identifier-keyed merge/split persistence (contact_links + contact_merge_rules, migration v11) that survives re-ingest via an idempotent reconcile pass -- the same stable-identity keying embeddings, facts, and reactions use against rowid churn. Add the paired SPEC-0015 (spec.md REQ-0015-001..010 with scenarios, plus design.md rationale/schema/testing) under docs/openspec/specs/contact-merge/, cross-linked to ADR-0022, epic #8, and children #9-#12, with ADR-0011 as prior art. Bump ARCHITECTURE.md's ADR range to 0022. Part of #8; closes #13. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: address cross-review findings on ADR-0022 / SPEC-0015 - ARCHITECTURE.md: list ADR-0022 in the curated ADR bullets (consistency with 0020/0021). - SPEC-0015 spec.md: pin the resolver's Go identifier as contacts.Resolver (wired via SetContactResolver) to remove a downstream contract-mismatch hazard between #9/#11/#12. - ADR-0022 / spec.md / design.md: restate the deterministic merge-winner rule as an explicit ordered rule (user-meaningful display_name wins; both/neither user-meaningful falls through to lower id). - ADR-0022 / design.md: drop the misleading /contacts/{id} URL example (no contact-by-id route exists; routes are /c/{id}), reframe as a hypothetical future reference. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
A dangling node_modules symlink pointing into a build-time scratch dir was accidentally committed via #20's CSS rebuild and inherited by every branch since. It's harmless to the pure-Go build but resolves to nothing on any other clone. Remove it and gitignore /node_modules and /.tools/ so the Tailwind CLI shim can't be committed again. Co-authored-by: Claude <noreply@anthropic.com>
* Add macOS Contacts provider (ContactResolver) behind macoscontacts tag Implements issue #10: the platform-specific address-book provider behind the contacts.Resolver seam (#9), read by the merge engine (#11) and merge settings UI (#12). internal/macoscontacts splits pure-Go logic from a thin cgo edge so the whole pipeline is testable in the CGO_ENABLED=0 CI where Contacts.framework cannot be linked: - provider.go (always compiled, no cgo): Provider satisfying contacts.Resolver, the CNAuthorizationStatus -> Availability tri-state mapping (Absent vs NeedsPermission vs Available, matching internal/setup's permission model), raw-record -> contacts.Person normalization through the shared contacts.Normalize* helpers (phone canonical, email lowercase), and the enumeration-dump parser. Resolve matches by exact (Kind,Value) equality; the KindPhone national/international cross-shape widening stays the engine's job. - backend_darwin.go (darwin && macoscontacts && cgo): the sole ~40-line Contacts.framework binding — detect-only authorization (never prompts) plus a raw contact dump the pure-Go parser consumes. - backend_stub.go (everything else): a no-provider backend, so go build ./... with no tags — and every release binary — links zero Contacts symbols and the provider behaves exactly like contacts.Unavailable. Gated by a dedicated `macoscontacts` build tag layered on the desktop build (the devicesync precedent) rather than the always-on `desktop && darwin` glue, since Contacts needs a TCC entitlement + usage string and must not be force-linked; plus a runtime GOOS==darwin guard in New. Desktop wiring mirrors the syncthing tagged/untagged split: embedded's wireContacts injects macoscontacts.New via SetContactResolver under the tag and is a no-op otherwise, leaving the web layer's contacts.Unavailable default. Tests (pure Go, fake backend): authorization mapping, availability gating, normalization/dedupe/filtering, dump parsing, Resolve/People behavior, the New runtime-guard fallback, a #20-style build-constraint proof (backendCompiledIn), and a `go tool nm` proof that the default msgbrowse binary carries no Contacts symbols. The cgo binding's C side is the only part not compile-verifiable in this Linux CI. Part of #8 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(macoscontacts): note Resolve re-enumerates, prefer People() for bulk Cross-review minor: document that Provider.Resolve is a single-lookup affordance that re-enumerates the whole address book per call, steering future bulk callers to enumerate once via People() and index locally. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
Summary
Three related pieces, reworked onto current
main(Providers is now a Settings tab; the LLM tab with live-apply already exists upstream):LLM API key (Settings → LLM tab)
config.SaveLLM(llm.api_key), threaded throughllm.Settings+ theApplier.MSGBROWSE_LLM_API_KEYstill overrides at startup. (Option A: config-file storage — chosen deliberately so a desktop user without a handy env var can set a key.)Providers page (the
/providersSettings tab)internal/source)./setup/refresh-allroute; replaced with a background auto-refresh scheduler (providers.refresh_interval, default6h,0disables) that reuses the per-source concurrency guard, so it can't collide with a manual Refresh.store.LastSyncTimes()overingest_runs.Device sync gated out of the binary (ADR-0021 / SPEC-0014)
devicesyncbuild tag. The defaultmsgbrowsebinary links with zerointernal/syncthingsymbols and no engine constructors (verified withgo tool nm); build with-tags devicesyncto include it.serve+ desktop wiring, themsgbrowse devicesnamespace, and doctor's device checks are split into tagged real / untagged stub files behind awireDeviceSyncseam.web.SetDeviceSyncFeature), so a release build renders no dead surface. New tests assert the feature-off hiding.Verification
go build ./...andgo test ./...pass for both build variants (default and-tags devicesync), across the main module and the desktop module.Note: the built
internal/web/static/app.csswas hand-mirrored frominternal/web/tailwind/input.cssbecause the Tailwind CLI is unreachable in this environment (proxy 403); runmake cssto regenerate it cleanly when online.🤖 Generated with Claude Code
https://claude.ai/code/session_01PruXCwhbcG7C6U88x7wykZ