Skip to content

Merge sessions from other machines via extra session roots (1/2: core) - #115

Open
crazy-daddy wants to merge 3 commits into
JayantDevkar:mainfrom
crazy-daddy:feature/extra-roots-core
Open

crazy-daddy wants to merge 3 commits into
JayantDevkar:mainfrom
crazy-daddy:feature/extra-roots-core

Conversation

@crazy-daddy

@crazy-daddy crazy-daddy commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Shows Claude Code sessions from several computers in one Karma dashboard, without Karma doing any syncing itself.

Users copy each other machine's ~/.claude dir onto the Karma machine with whatever they already use (Syncthing, OneDrive, Dropbox, rsync, a network share) and list those copies as extra roots:

// ~/.claude_karma/extra-roots.json  (or CLAUDE_KARMA_EXTRA_ROOTS)
[{ "name": "laptop", "path": "D:/Sync/laptop/.claude",
   "path_map": { "/home/me/code": "C:/Users/me/code" } }]

Karma reads extra roots read-only. This is a much smaller alternative to the Syncthing-based approach in #85: no daemon, no teams or pairing, no schema chain.

This is part 1 of 2. #116 is stacked on this branch: it extends coverage to the remaining views and adds a machine badge, and should merge after this one. #117 (independent, any order) adds an optional hook for exact matching on machines that don't run Karma.

How remote projects are matched

A project dir from another machine joins a local project by the first rule that matches:

  1. project_map: explicit {remote dir: local dir} in the root config
  2. karma-project.json sidecar: same git remote and subdir within the repo
  3. path_map: prefix rewrite of the remote project path
  4. Same encoded dir name (identical path on both machines)
  5. Exactly one local project with the same folder name (match_by_name, on by default; rejected when both sides have known, different git remotes)

Anything unmatched shows as its own project. Remote worktree dirs follow their main project. Karma writes a sidecar into each local git project dir on every index pass, so machines that also run Karma get exact matching automatically.

Changes

  • api/services/session_roots.py (new): root config, sidecars, matching, cached mapping with a single-flight rebuild
  • Indexer: third pass over remote dirs; new nullable sessions.source_root column (added without a reindex). A session present locally wins over a synced copy, the first root wins across roots, and an offline root (unmounted drive, paused sync) keeps its rows
  • Project.session_dirs: session listing, lookup and agents span local and synced dirs; remote-only projects appear in the project list
  • session_lookup falls back to synced dirs
  • GET /extra-roots[?refresh=true]: shows each remote dir and which rule matched it
  • SETUP.md: new "Multiple Machines" section

With no roots configured, behavior is unchanged: the registry returns early without disk I/O.

Testing

  • api/tests/test_session_roots.py: matching rules, config validation, sidecars, Project across dirs, indexer merge/dedupe/offline/removed roots, failure handling
  • Full API suite: 1951 passed. The 12 failures also fail on current main on Windows (live session store, terminal focus); unrelated
  • ruff check: no new findings

@JayantDevkar JayantDevkar left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. I read the whole thing and I like the direction: read-only roots, a canonical-name mapping layer and a cached single-flight registry are small, isolated and reversible, and this is a much better fit for the project than a daemon. I want to build on it. Two things block the merge, and a handful should change with them.

Blockers

  1. Karma writes into ~/.claude/projects even with no roots configured. db/indexer.py:963-968 calls write_sidecar for every git project on every index pass, so every existing user gets a karma-project.json in each project dir plus a git-root lookup per project per pass. That directory is Claude Code's, and the PR body says the no-roots path is unchanged. Gate the sidecar write (and the stale-sidecar delete at session_roots.py:239-243) on at least one configured root.
  2. The folder-name guess (rule 5) is on by default and only half-guarded. session_roots.py:394 rejects only when both sides have a known, different remote. The remote side only has an identity if it has a sidecar, which machines without Karma or the #117 hook never do, and on the first index pass local sidecars don't exist yet either (indexer.py:195 vs :966). So a remote api, web or demo folder merges silently into whichever local project shares the name. Default match_by_name to off, or require identity on both sides.

Should fix

  • A malformed or mid-edit extra-roots.json wipes every remote row: _load_root_entries (session_roots.py:157-165) logs and returns an empty list, _sync_extra_roots clears the cache first (indexer.py:718), and _cleanup_stale_sessions then treats every root as removed (indexer.py:780-783). Raise on parse failure so _refresh keeps the previous roots, and skip the removed-root deletion unless the config loaded cleanly.
  • A mid-copy JSONL can be indexed truncated and never re-read: sync_project (indexer.py:316) compares mtime only. cp -p and SMB can stamp the final mtime before the data lands. For remote files also compare jsonl_size (already stored).
  • Rule 2 with two local clones of the same repo and subdir: by_identity.setdefault (session_roots.py:316) picks whichever comes first in directory order. Treat a duplicated key as ambiguous and fall through.
  • GET /extra-roots (routers/extra_roots.py:19) exposes absolute host paths with no localhost guard, and ?refresh=true mutates state on a GET. Reuse _require_local from routers/desktop_app.py and make refresh a POST.
  • Root validation (session_roots.py:169-189): also reject duplicate resolved paths, a root nested inside or containing the local ~/.claude, and a path that is a file. Keep missing paths allowed, offline roots are intentional.
  • path_map (session_roots.py:278-285): prefix matching is case-insensitive on POSIX too, so /home/Me and /home/me collide, and a source of / normalises to an empty prefix that matches everything.

Nits

  • Multi-line comment blocks: config.py:63-68, session_roots.py:75-76 and 95-98, schema.py:819-820, indexer.py:313-315. We keep code comments to one line here.
  • root_for_path (session_roots.py:563-578) resolves the file and every root on each Session.from_path once roots exist; cache the resolved bases on SessionRoot.
  • Tests worth adding: malformed roots file, nested root, partial file, the endpoint, "no roots means no sidecar write".

The cachetools CI failure on the guard job was ours (a minimal pip list in the workflow); fixed on main in #120, so a rebase clears it. Once the two blockers are in I'll go through #116 and #117 with it; I've left notes on both.

Part 1 of 2 (stack). Part 2, feature/extra-roots-coverage, builds on
this branch and extends coverage to the remaining views plus a machine
badge. The sidecar hook (feature/extra-roots-hook) is independent.

Users sync other machines' ~/.claude dirs here with any tool (Syncthing,
OneDrive, rsync, ...) and list them in ~/.claude_karma/extra-roots.json or
CLAUDE_KARMA_EXTRA_ROOTS. Karma reads them read-only; no sync daemon.

Each remote project dir joins a local project by the first matching rule:
project_map, karma-project.json sidecar (git identity + subdir), path_map
prefix rewrite, same encoded name, unique folder name (guess). Unmatched
dirs show as remote-only projects. Remote worktree dirs follow their project.

- services/session_roots.py: root config, sidecars, matching, TTL cache
- indexer: third pass for remote dirs, sessions.source_root column (no
  reindex), local copy wins, first root wins, offline roots keep their rows
- Project.session_dirs: listing, lookup and agents span all dirs
- session_lookup falls back to remote dirs
- local git projects get a sidecar on every index pass
- GET /extra-roots shows how each remote dir was matched
- SETUP.md: Multiple Machines section
Blockers:
- Write karma-project.json sidecars only while at least one extra root is
  configured. Without roots Karma no longer touches ~/.claude/projects.
- match_by_name now defaults to off; the folder-name guess is opt-in per root.

Should fix:
- A malformed or mid-edit extra-roots.json raises in _load_root_entries;
  _refresh keeps the previous roots and flags config_ok=False, and the
  indexer skips the removed-root cleanup until the config loads cleanly.
  The indexer calls invalidate() (keeps the fallback) instead of clear_cache().
- Remote JSONL files are re-indexed when their size changed even if the
  mtime did not (cp -p, SMB stamp the final mtime before the data lands).
- A sidecar identity claimed by two local clones is ambiguous and no longer
  matches rule 2.
- GET /extra-roots is localhost-only via _require_local; refresh moved to
  POST /extra-roots/refresh. _require_local takes a subject for the 403 text.
- Root validation rejects duplicate resolved paths, roots overlapping the
  local ~/.claude, and paths that are files. Missing paths stay allowed.
- path_map: "/" (empty prefix) is ignored; prefixes are case-insensitive only
  for Windows paths (drive letter or UNC).

Nits:
- One-line comments in config.py, session_roots.py, schema.py.
- SessionRoot caches its resolved base; root_for_path no longer resolves
  every root per session file.
- Tests: malformed roots file, non-list file, nested/duplicate/file roots,
  path_map edge cases, ambiguous identity, partial remote file, endpoints,
  no roots means no sidecar write.
- path_map: a bare drive source ("C:/", normalised to "C:") is treated as a
  Windows prefix and matched case-insensitively.
- SessionRoot resolves its base in __post_init__, so a root built without
  resolved_base still matches symlinked paths in root_for_path.
- Tests: no roots also means no stale-sidecar delete; a malformed roots file
  on a cold start (no previous roots in memory) keeps remote rows; drive-root
  path_map case.
@crazy-daddy
crazy-daddy force-pushed the feature/extra-roots-core branch from 0779a03 to 8e8e445 Compare October 21, 2026 19:17
@crazy-daddy

Copy link
Copy Markdown
Contributor Author

Rebased onto main (picks up #120) and pushed two commits on top: 2a08bab for the review, 8e8e445 for follow-ups from a second pass.

Blockers

  1. Sidecar write gated: _update_project_summaries only calls write_sidecar when get_extra_roots() is non-empty, so the stale-sidecar unlink inside it is gated too. Without roots Karma never touches ~/.claude/projects; the Hook: project identity sidecar for multi-machine matching #117 hook is the publish path for such machines. SETUP.md says so.
  2. match_by_name defaults to off (dataclass, parser, docs). The "both sides known and different" guard stays for roots that opt in.

Should fix

  • Malformed or mid-edit extra-roots.json: _load_root_entries raises on bad JSON or a non-list; _refresh keeps the previous roots and records config_ok=False. The indexer now calls a new invalidate() (drops the mapping, keeps the last roots as fallback) instead of clear_cache(), and _cleanup_stale_sessions skips the removed-root deletion while roots_config_ok() is false. /extra-roots reports config_ok.
  • Mid-copy JSONL: sync_project also compares jsonl_size for rows with a source_root; same mtime with a different size re-indexes.
  • Two local clones with one identity: the key is dropped from by_identity, rule 2 falls through.
  • GET /extra-roots and the new POST /extra-roots/refresh both go through _require_local (it gained a subject argument for the 403 text; default message unchanged). The ?refresh query parameter is gone.
  • Root validation rejects a duplicate resolved path, a root inside or containing the local ~/.claude, and a path that is a file. Missing paths still allowed.
  • path_map: a source that normalises to an empty prefix (/) is ignored with a warning. Prefix matching folds case only when the source is a drive-letter or UNC path (including a bare C:/); POSIX sources are case-sensitive.

Nits

  • One-line comments in config.py, session_roots.py, schema.py.
  • SessionRoot resolves its base once at load (__post_init__); root_for_path no longer resolves every root per file.
  • Tests added (47 to 49 in test_session_roots.py, 14 new): malformed file keeps roots and rows, non-list file, malformed file on a cold start, nested/duplicate/file roots, path_map root and case edge cases, ambiguous identity, partial remote file re-indexed on growth, both endpoints incl. 403 on a non-loopback Host and Origin, no roots means no sidecar write and no stale-sidecar delete.

One known edge I left as is: a malformed roots file on a cold start also drops roots supplied via CLAUDE_KARMA_EXTRA_ROOTS until the file parses again (there is no previous list to fall back to). Existing rows survive and the mapping returns on the next pass. Say if you want env roots loaded independently of the file.

Verified: pytest tests/test_session_roots.py 49 passed; full API suite 2015 passed with the 6 Windows-only failures that also fail on main here (test_settings permissions and symlink, test_git_lookups_cache, test_live_session_store purge, test_terminal_focus). ruff check/format --check clean on the CI scope.

@crazy-daddy

Copy link
Copy Markdown
Contributor Author

On the two red checks: neither is in code this PR touches, and both look environmental.

I cannot rerun the failed jobs from the fork (admin rights). A rerun on your side should clear them; happy to push an empty commit instead if you prefer, though that moves #116's base again.

@crazy-daddy

Copy link
Copy Markdown
Contributor Author

Addendum: the macOS 3.10 test_timed_out_build_is_killed_even_if_it_ignores_sigterm failure also hit #116's second run (411389c) after passing its first, with no change under scripts/ between them, so it is flaky on that runner rather than tied to either PR. The tmux test passed on #116's second run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants