Repository navigation
merge: land PRs 3, 4, 5, 6 and 9 of the v7 audit stack in working_branch - #69
Merged
Merged
Conversation
…re absent Add pythonpath = . to pytest.ini so tests/conftest.py can import the tests package under a bare pytest run, and drop the duplicate [tool.pytest.ini_options] block from pyproject.toml. Guard modules that import numpy or rank_bm25 at import time with module-level pytest.importorskip, and mark only the individual tests that need tiktoken, numpy, huggingface_hub, the models-local profile or the E2B SDK, so the rest of those modules keeps running under CI's [dev,apps,graph] install. Also apply ruff format to tests/test_retrieval_runtime.py. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… Linux mypy narrows sys.platform but not os.name, so the ctypes.WinDLL and msvcrt.locking calls failed the Linux type-check job. Declare the job handle closer's type instead of ignoring the attribute error. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Pin ruff 0.16.8 (uv.lock) in the lint job and the pre-commit hook, and drop the continue-on-error legacy mypy step. Rewrite the AGENTS.md contributor commands to the gates ci.yml actually runs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
mcp 1.26.0 -> 1.30.0, cryptography 46.0.3 -> 50.0.1, urllib3 2.6.1 -> 2.8.0 and idna 3.11 -> 3.20 clear the published advisories against the locked versions. httpx is unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…th leaks Pass 1 runs the production server in-process as the loopback principal with two registered workspaces. It seeds A with canary text, records, an outcome, a relationship, a rule, a trigger, active context, cursors and selection tokens, then calls every manifest tool and workspace resource against B with A's IDs, cursors, tokens and idempotency keys. No B response may contain A's text or IDs, A's ledger tables may not change, a rejected call may not change B, and no field outside user text may carry a registered root, a storage path or the home directory. Consolidation and federated recall are refused without a directional link, and a B-side link is what makes consolidation from A possible. Pass 2 relaunches the same workspaces over HTTP with a JWT principal granted only A and requires UNAUTHORIZED_WORKSPACE for every call that targets B, including B as a consolidation or federated-recall source. A fast guard fails when a manifest tool is neither swept nor declared cross-workspace. The JWT issuer and workspace setup move into process_client.py so the access test and the sweep share them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
inventory_database opened the source with the unix-none VFS, which has no shared-memory support, so SQLite could not open any WAL database and every migrate-v7 dry run and apply failed on POSIX with 'unable to open database file'. A closed database with no -wal or -shm is now read as immutable (the main file is the whole database and nothing is created); a live WAL is read through the default VFS, whose only write is the volatile -shm wal-index. Windows keeps the win32-none VFS. The dangling-pointer guard test now matches the redacted error code; it was skipped wherever symlinks are unavailable, so it never ran before. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Fresh-workspace bootstrap staged its database under the system temp directory, which on macOS sits below the /var -> /private/var (or /tmp) symlink, so the linked-ancestry guard rejected it and every new workspace reported ACTIVE_V7_UNAVAILABLE. Resolve the temp root first. The Leiden worker capped itself with setrlimit(RLIMIT_AS), which macOS rejects because a process already reserves more address space than the cap, so every graph build ended CAPABILITY_DEGRADED. Skip the cap on macOS as on Windows; the deadline and request bounds still apply. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The default socket path was ~/.daem0nmcp/edit-bridges/<64 hex>/run/ bridge-<24 hex>.sock, about 140 bytes, beyond the 104-108 byte AF_UNIX limit, so the local edit bridge could never start on Linux or macOS. Sockets now live in <root>/run/<16 hex>/b-<24 hex>.sock, and a test bounds the suffix. Tests that open sockets use a short socket_tmp_path because the repository-local tmp_path is too deep on CI runners. A backslash in a POSIX edit path now raises NativeEditNormalizationError instead of leaking a ValueError; the apply_patch test uses a backslash separator only on Windows, where OpenCode emits one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…tors An elevated administrator's new files are owned by BUILTIN\Administrators, so the owner-only ACL check always failed for them (and on the Windows CI runners). After applying the owner-only DACL, which grants this user WRITE_OWNER, set the owner to the current user when it differs. The verification is unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…uming The test cancelled the task and released the blocked worker in the same step, so a fast worker could record the next archive before the task saw its cancellation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The dev extra now pins ruff 0.16.8 and mypy 2.3.1 (locked), CI installs the same mypy, and the pre-commit mypy mirror matches it, so following AGENTS.md reproduces the lint and type-check gates. AGENTS.md also notes the platforms the locked cryptography ships wheels for. The stdio duplicate-key docstring no longer names a specific MCP SDK version. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…k checks - Sweep a third registered root C whose storage is a byte copy of A's, so only the workspace_id predicates keep A's rows out; dropping the predicate from _load_record_row now fails the sweep. - Give B and C their own src/ file and run own-argument variants of code_index, code_search, code_todos_scan, code_refactor_propose and workspace_export, so their success responses are scanned. - Pin every call's outcome, including the cross-workspace refusals; the consolidate refusals are rechecked with B's own valid preview token after unlinking, so only the missing link can refuse them. - Scan dictionary keys, forbid the bare canary (except an exact echo of the request), and drop the field-name exemption from the host-path check: no response carries a host path in any field. - Require the core seeds to be real, and report gate-only tools and placeholder seeds next to the swept list. - Pass 2 names the tool when a denial unexpectedly succeeds, requires unchanged A and B ledgers, and matches resource denials exactly with a positive control on A. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Ownership is reassigned to the current user only when a new protected path is owned by BUILTIN\Administrators (an elevated administrator's default); any other foreign owner is still refused by verification. Unit tests replay both owners without needing elevation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…urnal A -journal file beside the source may be a hot journal that SQLite must see, so it now selects the ordinary mode=ro path like -wal and -shm do. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
models-local needs Python 3.11+ (its extra installs nothing on 3.10), so on 3.10 the registry correctly reports upgrade_python before probing packages. The missing-extra and ready-probe tests now pin 3.11 like the python-floor test pins 3.10. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The competitor ran a fixed 80 ms after the builder released the writer lock, so a slow runner could let the 100 ms lease lapse before the heartbeat renewed it. The test now polls for a lease renewed after the commit (5 s deadline) and uses a 500 ms lease with a 750 ms lock hold, so the lock still outlasts the lease as before (F-092). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The graph-generation test read system_health after a fixed 2 s sleep, so a slow runner could still see community_refresh running. It now polls every 250 ms (30 s deadline) until the refresh is no longer running and has succeeded, then makes the same assertions (F-092). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
pre_edit is now stdlib-only: outside a .daem0nmcp project it exits 0 silently; inside one it adds a one-line additionalContext reminder to call memory_recall_file and memory_preflight, and never sets permissionDecision. The stop hook reads cwd, transcript_path and session_id from its stdin event, unwraps Claude Code's nested transcript records, and shows its suggestions as a systemMessage; it never emits "decision" or exits 2. post_edit only imports the edit bridge when the project is paired. uninstall-claude-hooks with --project-path strips the pairing env keys from the project's settings.local.json; --remove-credentials also deletes the local authority credential and socket directories. The manual hooks/ scripts become exit-0 deprecation stubs and settings.json.example is gone. The bridge integration tests now drive the bridge through a test-only host driver instead of the deleted handle_pre_edit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Replace the hook sections of the README, Summon section 9 and the v7 operations guide with one table of the installed hooks, the user-level install location, and what pairing writes. Delete the README manual hooks block and tell manual-hook users to re-run install-claude-hooks. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Any INSERT failure (I/O, lock) used to become a retryable EVENT_STREAM_CONFLICT. Only sqlite3.IntegrityError is a real conflict; everything else now propagates to the caller's own mapping (F-066). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…error - A full writer/resource worker pool or a SQLite "database is locked"/busy timeout now surfaces as DATABASE_IN_USE (retryable, retry_after_ms=250) instead of a non-retryable INTERNAL_ERROR (F-020, F-027, A-011). This covers memory_store, memory_record_outcome, session_brief and memory_preflight through one is_database_busy() classifier in errors.py. - ResponseContext.internal_error logs the cause under the correlation ID the client receives (stderr via logging, never the MCP wire; the envelope is unchanged). Runtime service errors chain their cause (`from exc`) so the log shows the real failure (F-021, F-028). - memory_preflight with target arguments that do not validate stays ok:true but adds a PREFLIGHT_TOKEN_NOT_ISSUED warning naming the failing fields (field names and pydantic error types only) (F-070, F-082). - Tests: 8 parallel preflight+store flows, a held write lock, a logged induced failure, a full resource pool, and two stdio server processes firing 5 same-key and 10 distinct-key stores (F-055). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ing else Memory content, rationale, tags, context, queries, rule and trigger text, entity names and the server text that quotes them are typed UserText and are exempt from the absolute-path rule in both directions (UD-3, KD-2). Every other string on every WireModel, input and output, keeps the rule through the single is_host_absolute_path predicate, so nothing can be stored and then brick a read. Delete the drifted output guards (_path_safe_success, both _contains_raw_path copies, the resource repository's root scan, and the path checks on content in entity evolution, intelligence summaries, TODO text, capture records and edit descriptions). Event payload checks keep refusing raw path columns and host paths in relative-path keys. context.visibility is validated at the input model. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
test_path_policy stores the UD-3 examples, reads back a pre-fix record, seeds every free-text input field found from the tool models plus migrated-style memory, rule, trigger, entity and community rows with paths, then calls every read tool and resource. The isolation sweep now stores a note naming each target's root and scans for host paths only outside UserText fields, located through the response models. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Readers turn a row that fails its output model into CAPABILITY_DEGRADED, which the first version accepted as a gate. Compare each read with its outcome before any path was stored, count CAPABILITY_DEGRADED as a refusal for seeded calls, and retire the graph rebuild the inserted rows queue so entity IDs stay current. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…d 1) - resource_repository._run: a read that overran its deadline and then finished cleanly is contention, not a fault; report DATABASE_IN_USE instead of RESOURCE_REPOSITORY_UNAVAILABLE (-> PREFLIGHT_FAILED -> INTERNAL_ERROR), the chain behind the Windows CI failures (R-9, R-10). - application.py: non-pinned tools now send DATABASE_IN_USE as retryable with retry_after_ms=250, like pinned tools (R-1, R-15). - memory_store and token-bound non-pinned tools: the gate spends the preflight token before the write, so DATABASE_IN_USE carries a memory_preflight remedy and says to retry with a fresh token and the same idempotency_key (R-4). - is_database_busy uses sqlite_errorcode (BUSY/LOCKED) when available and SQLite's fixed lock messages on 3.10 (R-6); the retry hint is one constant. - governance_events INSERT: only IntegrityError is a stream conflict (R-2). - internal_error logs pydantic ValidationError links without input values (R-13); token-not-issued docstring states what is echoed (R-3/R-7/R-14). - Tests: service-level preflight/briefing contention (full pool, overrun read) through the production readers; repository overrun maps to DATABASE_IN_USE; same-token retry is TOKEN_REPLAYED with the preflight remedy; governance conflict rule; log redaction; server log tails in the two-process test's assertion messages; retry counts printed, not asserted (R-11, R-12). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- pre_edit and stop find the nearest ancestor of cwd (then of CLAUDE_PROJECT_DIR) with .daem0nmcp/, never counting the home directory whose ~/.daem0nmcp/ is Daem0n's own state. - Model-supplied paths are checked lexically (normpath + commonpath; UNC and other-drive paths dropped) and never resolved, so a \host\share path cannot trigger SMB authentication. The path in the reminder is JSON-escaped. - stop JSON-parses only the transcript tail, keeps the full record count for its anti-loop state, prunes state files untouched for a week, and words its systemMessage for the user. - install/uninstall refuse a malformed ~/.claude/settings.json instead of replacing it; settings files with a UTF-8 BOM are read. - uninstall reads everything that can fail before writing, reports partial success accurately, only strips DAEM0NMCP_PROJECT_ROOT/STORAGE_PATH from a project pairing marked, checks each credential directory for links before removing it, and says when no project was given. The CLI defaults the project to the current directory and also accepts --project-path after the subcommand. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Under load, memory_preflight overran its 2 s read and then the 15 s tool deadline (DEADLINE_EXCEEDED / INTERNAL_ERROR on CI). Stack captures at the deadline showed every resource worker inside _read_git_changes_sync: each briefing snapshot runs two git subprocesses (on Windows also a system-wide Toolhelp thread walk to resume them), and guidance discards git_changes. read_briefing_snapshot gains include_git_changes; the guidance reader passes False. session_brief still reads git changes. Also: a shared storage lock refused because another process holds it exclusively (bootstrap, migration) is DATABASE_IN_USE, not ACTIVE_V7_UNAVAILABLE; and ACTIVE_V7_UNAVAILABLE, opaque on the wire, is now logged with its cause and correlation ID. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
On POSIX a source with a WAL sidecar is read through the default VFS, whose volatile ``-shm`` write the comment already describes, so the opening sentence no longer claims no byte is written anywhere. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ion-gener' into v7/bf-09-auto-enable-installed-profiles-a
Both failed on the slowest Python 3.10 CI legs of run 35495906995 and neither touches this PR's code. `test_write_locked_builder_outlasts_heartbeat_busy_timeout` held the writer lock for a fixed 1.25 s against a 100 ms lease, so its whole margin was the 100 ms between the heartbeat's final refresh and the lease check in `_record_success`; a descheduled runner lost it and the job that had in fact completed was recorded as PROJECTION_JOB_LEASE_LOST. The builder now holds the lock until the heartbeat has provably been through its busy timeout more than once (counted on a connection subclass, so real contention is asserted rather than assumed) and the lease it was claimed under has expired, both to a bounded deadline, and the lease is long enough that the terminal update cannot lose a race it was never about. Every assertion is unchanged and one is added. Proved on python:3.10-slim by injecting a 150 ms stall into exactly that window: the previous test fails 5/5 with the CI's error, the new one passes 5/5, and the previous test passes without the stall. `test_production_ritual_and_restart` polled to a deadline for the record to become visible before the restart but asserted immediately after it, although a restarted server republishes the lexical generation in the background. Both now use one bounded poll. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…rofiles-a' into v7/bf-06-make-migrated-v6-workspaces-usab # Conflicts: # daem0nmcp/api/v7/operations.py # daem0nmcp/api/v7/pinned.py # daem0nmcp/api/v7/resource_repository.py # daem0nmcp/api/v7/runtime_services.py # daem0nmcp/retrieval/lexical_config.py # daem0nmcp/verification_v7.py
… hides
R-11/R-38 (major): `reconcile_retained_governance` read "absent from the
retained v6 rows" as "deleted in v6", and ran on the reactivate path, whose
candidate is the database that was live. So apply → author a rule or trigger
through v7 → rollback --discard-v7-writes → apply disabled every v7-authored
rule and deleted every v7-authored trigger. It now reconciles only streams
whose creating governance event was written by the migration itself.
R-32 (major, security): `_lossless_row` copied each v6 column as a
`[name, value]` pair, so the host-absolute `file_path` rode inside every
migrated event's payload and out through `workspace_export` — including from
stores migrated by this branch. `_reject_raw_paths` only inspected mapping
keys and the wire sweep exempts `payload.data`, so nothing caught it. Host
path columns, and a derived relative form that is not workspace-relative, are
now written as a `{"$host_path": {sha256,length}}` digest (self-consistent:
`_source_row_hash` is only ever compared against itself, and the original
stays in `source.snapshot.db` and the retained v6 tables). `_reject_raw_paths`
also refuses the pair-list shape, so a regression fails loudly.
R-2/R-34/R-13/R-17 (major/minor): the rollback guard counted two of the four
tables a v7 client can add user data to. One named `_POST_ACTIVATION_TABLES`
now covers `workspace_link_events` (federation) and `memory_capture_candidates`
(unapproved user content, plain inserts with no event), and the comparison is
against row counts recorded at activation rather than a wall clock, so a
backward clock step cannot hide the writes this exists to refuse.
R-33/R-36/R-39 (minor): five more readers still refused a migrated row on the
legacy `file_path` — local_state, maintenance, discovery, relationship and
code_entity — and two passed the stored relative path into `RelativePath`
unchecked. All now behave like the readers the plan named.
R-5 (minor): the four MCP resources answered "outside its workspace" for a v6
store, which is more misleading than what it replaced; they now raise
MIGRATION_REQUIRED like every other entry point.
R-12 (minor): the rollback refusal's count and retained candidate never
reached the operator, and `migrate-v7` never printed its warnings without
--json. Both do now; other migration details stay redacted to their code.
R-3 (minor): `RecordSummary.tags` required uniqueness while every tag input
allowed duplicates, so an ordinary `memory_store(tags=["a","a"])` wrote a
record no reader could render. `UniqueTags` fixes the asymmetry at the input.
R-7: a trigger recording no project root is foreign again. R-9/R-15/R-41:
removals, edits and unreconcilable trigger edits are counted and warned
separately. R-8/R-16: a row that recorded any link shape and kept none is
counted. R-14/R-43: one `sole_recorded_project_root`, normalized, for both
file links and triggers. R-18/R-40: only the three `memory_*` snapshots skip
the live manifest invariants. R-42: a pointerless store mid-creation is not
capped, so its recovery branch still heals it. R-35: `_sqlite_backup` opens
the source through `_readonly_connection`. R-10: `functools.partial` instead
of a parameter two importers never read.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
R-22 (major): the "already-migrated store" test looped over `recall["items"]`, which was empty, so both reader-side repairs were unproved — neutering `stored_relative_path` or deleting the bound clipping left it green. It now reads the affected record through `memory_search_text` with metadata (so the assertions run on values the test controls), over each shape v6 could store: `../outside`, the drive-lettered `resolved.as_posix()` fallback that `_v6_file_paths` was missing, and a POSIX absolute. It asserts no link is emitted, the host path is absent from the whole response, the tags are 32 unique names of ≤80 characters, and the excerpt of an empty v6 content is the migration marker. `test_the_bounded_summary_clips_what_v6_had_no_limit_for` pins the model itself. That test found a real defect: `RecordSummary` substitutes the marker, but four readers refused an empty stored content *before* building one, so a single empty migrated record still failed session_brief for the whole workspace — the exact failure mode PR 5 escalated. Fixed in resource_repository, record_operations (summary and search excerpt) and runtime_services. R-21 (major): `verify_v7` is asserted `verified` right after a migration and again after the first write through the server, with the table-snapshot manifests asserted present so it cannot pass vacuously. R-11/R-38: a reactivate test that creates a rule and a trigger through v7, removes the retained v6 rows their compatibility write made, rolls back with --discard-v7-writes and re-applies. Measured, and reported in the summary: the reviewers' sequence does not reproduce as written, because `rule_create` and `context_trigger_create` also write the retained v6 row, so the stream is found and left alone. The guard is still right, and this pins it. R-2/R-13/R-34: a federation link event and a pending capture candidate each make rollback refuse. R-28: the refusal and the discard warning are asserted for the count, the candidate and the flag, not for an absent substring. R-26: MIGRATION_REQUIRED over brief, recall, store, the resource surface and the transfer boundary (reached by rolling a briefed session's store back to v6). R-6: the post-repair `content_hash` move is pinned as a decision. R-24/R-25: the fixture builds all sixteen v6 ORM tables and its DDL digest is pinned — it is now byte-identical to the database the v6 code produced, 23 tables and the same sha256. R-20: a store a legacy command touched verifies after migrating. R-29 `entity_match`, R-31 the control-character tag, R-27 the single-value parametrize, R-30 `__import__`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…arries R-5: "every tool answers MIGRATION_REQUIRED" was an overclaim — store and recall are blocked behind a briefing that cannot succeed on a v6 store, so they answer COMMUNION_REQUIRED first. R-19: state positively that exports carry no host path and that the lossless v6 row lives in the migrated database rather than in the bundle. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PR 9's `_path_safe_success` accepted `list | None`; PR 5 deleted that wrapper, so the call reaches `ResponseContext.success`, whose `warnings` is a Sequence. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The pair-shape check R-32 added asked `value[0] in <frozenset>` before knowing `value[0]` was a string. Any ordinary two-element list whose first member is a list or dict — which the legacy-identity import path carries — raised `TypeError: unhashable type: 'list'`, and the catch-all turned it into IMPORT_INVALID. It now requires a string first, which is what a column name is, and the union is hoisted out of the recursion. The guard still rejects a `["file_path", …]` or escaping `["file_path_relative", …]` pair; `test_export_structured_fields_stay_guarded` now pins both halves, so neither the rejection nor the unhashable shapes can regress. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…the index **CI (ubuntu 3.11).** `test_a_database_migrated_by_the_pre_fix_code_still_reads` rewrote `memory_records.content` with direct SQL and then searched for the new marker, but `memory_search_text` is served by an FTS index built at migration time from the original content, and nothing forced or awaited a rebuild. It no longer depends on re-indexing at all: one warning row now carries every shape at once — the host-absolute `file_path`, the escaping relative form, v6's unbounded tags and empty content — and is read back through `session_brief` and the warnings resource, which select by record type. That is also the reader whose failure PR 5 escalated, so the test now exercises the path the requirement came from. **MIGRATION_REQUIRED on the resource surface — wontfix in code.** `production._active_database` raises for format 6 first and that is a `RuntimeError`, so the branch I added could never match, `format_version == 6` was unreachable with that resolver, and every reader funnels exceptions through `_capture` into the deliberate `RESOURCE_UNAVAILABLE` invariant that stops a caller enumerating workspaces by reading errors. Surfacing a distinct code there would punch a hole in that invariant for no gain, so the class, its branch, the re-raise and the assertion that could not fail are deleted, and `docs/v7-operations.md` now says the resource surface answers the invariant by design and that `session_brief` is where the reason is named. **Also:** the `RecordSummary` docstring no longer claims clipping is a no-op for a v7-written record — it names the three v6 bounds a v7 write cannot violate and separates the timeline clamp, which is not a v6-only repair. The export assertion now runs over the export pages, including an events page, rather than only the recall response. The `<empty>` substitution lives in the one place that needs it (the model, plus the bounded excerpt the model does not reach); the three copies in the readers were dead, and what those readers actually needed was to stop rejecting empty content. The reactivate test says why it deletes the retained rows. **CHANGELOG:** `_lossless_row`'s host-path digest is a payload-format change, so a store migrated by an earlier development build fails `verify-v7` with "retained legacy source provenance differs" and must be migrated again from its intact v6 source. v7 is unreleased, so no shipped store is affected. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
**Brownfield run 3403b448, PR 2 of 12** — audit-and-fix stack for the v7 core (#61/#62). Base: `v7/bf-01-make-ci-green-and-pin-workspace`. ## Plan (contract) ### PR 2: Make the Claude Code hooks remind instead of block - Depends on: PR 1 - Files: daem0nmcp/claude_hooks/pre_edit.py, daem0nmcp/claude_hooks/post_edit.py, daem0nmcp/claude_hooks/stop.py, daem0nmcp/claude_hooks/_client.py, daem0nmcp/claude_hooks/install.py, hooks/daem0n_pre_edit_hook.py, hooks/daem0n_post_edit_hook.py, hooks/daem0n_prompt_hook.py, hooks/daem0n_stop_hook.py, tests/test_claude_hooks_pre_edit.py, tests/test_claude_hooks_stop.py, tests/test_claude_hooks_post_edit.py, tests/test_claude_hooks_install.py, tests/api_v7/test_docs_hooks.py, docs/release/v7/requirements.json, docs/release/v7/inventory.md, README.md, Summon_Daem0n.md, docs/v7-operations.md - Traces to: FP-4, FP-7, F-009, F-019, F-045, F-050, F-042, F-044, F-084, F-052, F-037, F-038, A-025, A-031, A-036, UD-1 - Risk: low - Description (KD-1): 1. **`pre_edit.py` rewrite.** Stdlib + `_client.read_hook_event` only. - Take the project from `event["cwd"]`, falling back to `CLAUDE_PROJECT_DIR`. - With no `.daem0nmcp/` there, exit 0 silently. - Otherwise print `{"hookSpecificOutput":{"hookEventName":"PreToolUse","additionalContext":"<one-line reminder naming memory_preflight and memory_recall_file with the relative path>"}}` and exit 0. - Never set `permissionDecision`. - Delete `async_main`, `PreEditResult` and `handle_pre_edit`. 2. **`post_edit.py`.** Import `edit_host` lazily, only when the bridge credential env is configured. 3. **`stop.py`** (R-7). - Read `transcript_path` and `session_id` from the stdin event, and parse transcript records nested under `message`. - Emit the reminder only as a user-visible `systemMessage` JSON field or as plain stdout. Record in the PR which one Claude Code displays. - Never emit `"decision"` and never exit 2. - It still never writes memory. 4. **`_client.py`.** Delete `get_tool_input` and `get_file_path_from_input`. 5. **Uninstall cleanup** (F-038). - `uninstall-claude-hooks --project-path` strips the Daem0n env keys it added to `<project>/.claude/settings.local.json`. - With `--remove-credentials`, it also deletes `~/.daem0nmcp/edit-bridges/<authority>/`. - Both `install-claude-hooks` and `uninstall-claude-hooks` remove any `hooks/daem0n_*` command entries from `~/.claude/settings.json` (R-20; `install.py:45-48` already detects them). 6. **Legacy scripts** (R-13/R-20). Replace each `hooks/daem0n_*.py` with a stdlib-only stub. The stub prints `Daem0n: this manual hook is deprecated; run python -m daem0nmcp.cli install-claude-hooks` to stderr and exits 0. Delete the README "Manual Hooks (Legacy)" block. The PR description and CHANGELOG note tell manual-hook users to re-run `install-claude-hooks`. 7. **Doc sections.** Only the hook sections of README §Claude Code Integration, Summon §9 and v7-operations §Claude Code: one table of the 6 hooks (event, what it does, input channel, reminds only), the user-level install location, and what pairing writes. 8. **Release inventory** (R-2/R-22). Run `python scripts/v7_release_inventory.py --write` and commit the regenerated `requirements.json`/`inventory.md`, because hook entry points changed. 9. **Tests** (F-052). - stdin-driven `main()` subprocess tests for `pre_edit`: non-Daem0n dir → exit 0 and empty stdout; Daem0n project → exit 0 plus the reminder JSON. - An import-set test: no `daem0nmcp.api`/`daem0nmcp.retrieval` module loaded. - `stop` driven by a nested-`message` JSONL fixture, asserting its output never contains `"decision"`. - Each legacy stub fed a PreToolUse payload exits 0. - The uninstall test. - `test_docs_hooks.py` updated. - Acceptance: - A Claude-style Write event piped into `pre_edit` from a non-Daem0n temp dir exits 0 with empty stdout. From a Daem0n project it exits 0 with the reminder JSON. - `pre_edit` median runtime is < 150 ms (was 583 ms). - No Daem0n hook and no legacy stub can exit 2 (the tests above). - The release-inventory CI job is green. - The PR states whether `additionalContext` was confirmed visible to the model in a live session (A-036). #### Carry-over from PR 1 review (orchestrator, binding) - PR 1 (b881ca6) moved local edit-bridge sockets to `edit-bridges/run/<authority[:16]>/b-<24hex>.sock`. `uninstall-claude-hooks --remove-credentials` (step 5) must also remove that socket directory. --- ## What landed # PR 2 summary: Make the Claude Code hooks remind instead of block (Saved by the orchestrator from the implementer's reply.) Branch `v7/bf-02-make-the-claude-code-hooks-remin`, base PR 1 tip f7f024a. Worktree `D:\Daem0n-MCP\.claude\worktrees\agent-ab437c58b407838c4`. - 220f98a `fix: make the Claude Code hooks remind instead of block` - a9e7d7a `docs: describe the six remind-only Claude Code hooks and pairing` (final) ## Files changed - `daem0nmcp/claude_hooks/pre_edit.py` (rewritten); `stop.py`, `post_edit.py`, `_client.py`, `install.py`. - Outside plan file list: `daem0nmcp/cli.py` (`--remove-credentials`, `--project-path` to uninstall), `daem0nmcp/edit_host.py` (extracted `local_bridge_layout()`). - `hooks/daem0n_{pre_edit,post_edit,prompt,stop}_hook.py` → exit-0 deprecation stubs; `hooks/settings.json.example` deleted. - Tests: `test_claude_hooks_{pre_edit,stop,post_edit,install,client}.py`, `api_v7/test_docs_hooks.py`; bridge tests `test_edit_host.py`, `test_edit_bridge_transport.py`, `api_v7/test_process_edit_capture.py`; new helper `tests/native_edit_host.py`. - Docs: `README.md`, `Summon_Daem0n.md`, `docs/v7-operations.md`, `CHANGELOG.md`. ## What and why 1. **pre_edit**: stdlib + `read_hook_event` only; project from `event["cwd"]` then `CLAUDE_PROJECT_DIR`; no `.daem0nmcp/` → exit 0 silent; else prints PreToolUse `additionalContext` reminder (memory_recall_file + memory_preflight) and exits 0; path omitted if outside project; never sets `permissionDecision`; exceptions swallowed → exit 0. Deleted `async_main`, `PreEditResult`, `handle_pre_edit`. Lazy `asyncio` import in `_client.run_async`. Median runtime 87 ms (was 503 ms), 21 subprocess runs on Windows. 2. **post_edit**: `edit_host`/`native_edit` imported lazily; only runs when `DAEM0NMCP_EDIT_BRIDGE_CREDENTIAL_FILE` is set. 3. **stop**: reads `cwd`, `transcript_path`, `session_id` from stdin; state keyed by session_id; unwraps nested `message` records; emits `{"systemMessage": ...}` exit 0; never `decision`, never writes memory. Live `claude -p --output-format stream-json` run showed the systemMessage as an informational system event. 4. **_client**: deleted `get_tool_input`, `get_file_path_from_input`; env parsing folded into `get_command_from_input` (pre_bash needs it until PR 10). 5. **Uninstall**: `cli --project-path P uninstall-claude-hooks [--remove-credentials] [--dry-run]` strips pairing env keys from `<project>/.claude/settings.local.json` (credential file, runtime dir, mode, workspace-binding file, `DAEM0NMCP_PROJECT_ROOT`, `DAEM0NMCP_STORAGE_PATH`), keeping others; `--remove-credentials` deletes `edit-bridges/<authority>/` and `edit-bridges/run/<authority[:16]>/` (PR 1 carry-over). Legacy `hooks/daem0n_*` entries removed via `_is_daem0n_entry` (test added). 6. **Legacy scripts**: import only `sys`, print deprecation to stderr, exit 0. README "Manual Hooks (Legacy)" block deleted; CHANGELOG tells manual-hook users to re-run `install-claude-hooks`. 7. **Docs**: README, Summon §9, v7-operations share one six-hook table (event, behaviour, input channel, reminds-only, install location, what pairing writes); states pre_bash is inert (reads unset `TOOL_INPUT`) and post_edit hooks are no-ops. 8. **Release inventory**: `--write` produced no diff; `--check` valid (75 tools, 10 resources, 571 requirements). ## Tests and results - New pre_edit subprocess tests: non-Daem0n dir silent exit 0; Daem0n project exit 0 + reminder JSON with no permissionDecision; CLAUDE_PROJECT_DIR fallback; outside path and garbage stdin exit 0; import set excludes `daem0nmcp.api`, `retrieval`, `edit_host`, `database`, `sqlalchemy`, `pydantic`. - Stop: `main()` over stdin with nested-message JSONL; asserts no `"decision"`, systemMessage mentions memory_store/memory_record_outcome, no storage written, state file created; silent outside Daem0n project. - post_edit: unpaired project exits 0 without importing edit_host; paired project with no consumed edit exits 0. - Install/uninstall: legacy entries removed; project cleanup dry-run, env keys, both credential dirs; credentials kept without the flag. - `test_docs_hooks.py`: stubs sys-only, never exit(2); each stub fed a PreToolUse payload exits 0 with deprecation on stderr and no workspace mutation. - Hook + bridge tests: 105 passed (Windows); 86 passed (Docker python:3.12 Linux). - Full suite (Windows): 2799 passed, 121 skipped, 13 min 08 s. - ruff check/format clean; mypy api/v7 clean (also clean on claude_hooks and edit_host.py). - A-036 live check: `claude -p` (Claude Code 2.1.277) with only the new pre_edit hook in a temp dir with `.daem0nmcp/`: the Write went through and the model quoted the additionalContext. Confirmed. ## Deviations 1. Deleting `handle_pre_edit` broke bridge integration tests that used it as the native host → test-only driver `tests/native_edit_host.py::drive_native_edit` (raises instead of swallowing; two test_edit_host "fails before network" tests now assert `pytest.raises(ValueError)` and `calls == 0`). 2. `edit_host.py`: extracted `local_bridge_layout()` so uninstall computes the same dirs as provisioning; provisioning unchanged. 3. `cli.py`: `--remove-credentials`, `--project-path` passed through. 4. Deleted `hooks/settings.json.example`. 5. Deleted old pre_edit tests asserting blocking/permissive mode and `test_remote_mode_without_workspace_binding_fails_recoverably` (config error still covered by `EditHostConfig.from_environment`). ## Follow-ups (not fixed) - Claude-side edit bridge now unreachable (only pre_edit created bridge requests) → post_edit_preflight/post_edit are no-ops in Claude Code (documented). Fits UD-13 follow-up; pairing on install could be dropped then. - pre_bash still reads `TOOL_INPUT`; `block()` still in `_client` (PR 10). `run_hook_safely` maps exit 2 to 0. - Uninstall doesn't remove a remote pairing's binding file. - `daem0nmcp/__init__.py` `importlib.metadata.version()` costs ~36 ms per hook start. - README historical v6 section still says pre_edit "fails closed" (PR 12 scope). - `.gitignore` lacks `.venv/`. ## Note for PR description Manual-hook users: `hooks/daem0n_*.py` now only print a deprecation notice and exit 0. Re-run `python -m daem0nmcp.cli install-claude-hooks`; it also removes the old entries. ## Round 2 134b724 (hooks hardening: find_project_root walk-up stopping at home; lexical relative_project_path rejecting UNC/NUL/outside; json.dumps path; uninstall reads-before-writes with partial-success reporting; utf-8-sig settings reads; install refuses malformed ~/.claude/settings.json; pairing-marker-gated env strip; --project-path after subcommand; CLI cwd default; reject_linked_ancestry before rmtree; stop reads transcript tail, user-facing systemMessage, prunes state >7 days) and 27d1682 (docs: OpenCode edit gating gone; project detection). 26 items: 23 addressed, 3 wontfix (R-14 pre-existing bootstrap flake → PR 8; R-24 UD-13 follow-up list; R-25 plan amendments done by orchestrator). Security reviewer re-verified R-19..R-22 resolved. CI run 35422498587 green on all 9 legs + lint/type-check/release-inventory. UD-13 follow-up items: stop pairing on install; unregister post_edit/post_edit_preflight and strip them on install; delete tests/native_edit_host.py with the bridge; surface pre_edit stdout as OpenCode reminder; drop failClosed (incl. 5 s timeout block); fix daem0n.ts header/comments (lines 6, 216). --- ## Review # PR 2 review — merged (round 2: all addressed; security re-verified R-19..R-22; CI run 35422498587 green) Reviewer slots: general, general-2, tests, security, plan-alignment (effort 5). Item statuses: 23 addressed, 3 wontfix. CI: https://github.com/9thlevelsoftware/Daem0n-MCP/actions/runs/35422498587 (green) 🤖 Generated with [Claude Code](https://claude.com/claude-code)
… recall) (#66) **Brownfield run 3403b448, PR 4 of 12** — audit-and-fix stack for the v7 core (#61/#62). Base: `v7/bf-03-report-transient-failures-as-ret`. ## Plan (contract) ### PR 4: Garbage-collect projection generations and take git out of preflight - Depends on: PR 3 - Files: daem0nmcp/retrieval/projections.py, daem0nmcp/retrieval/specialized_projection.py, daem0nmcp/retrieval/jobs.py, daem0nmcp/retrieval/lexical_config.py, daem0nmcp/verification_v7.py, daem0nmcp/api/v7/production.py, daem0nmcp/api/v7/resource_repository.py, tests/test_retrieval_projection_growth.py, tests/api_v7/test_production.py - Traces to: FP-6, F-001, F-010, F-026, F-039, F-057, F-060, F-040, A-002, A-009, A-019 - Risk: medium - Description: 1. **Generation GC** (KD-5; R-24). - Move `_GENERATION_TABLES` from `verification_v7.py` into `retrieval/lexical_config.py`, alongside the FTS table naming for lexical and procedure. Verification imports it. - Add `collect_superseded_generations(connection, workspace_id, projection, limit=5)`. It opens its own `BEGIN IMMEDIATE`. It selects manifests with `status IN ('ready','failed')` for that workspace and projection, excluding the active manifest and the most recently deactivated (`MAX(activated_at_us)` among `ready`), oldest first, `LIMIT 5`. For each it runs `DROP TABLE IF EXISTS` on the regex-validated FTS table names, deletes the generation-table rows, and deletes the manifest. - Call it *after* the activation commit in `LexicalProjectionBuilder.rebuild` and in the specialized builder activation. - Catch lock/busy errors, log them at debug, and return; the next activation retries. - Next generation numbers stay `MAX(generation)+1`, so deleted manifests below the maximum are harmless. 2. **Lock contention** (F-060). In `rebuild`, classify lock contention with `jobs._is_lock_contention` as retryable instead of `LEXICAL_UNAVAILABLE`. 3. **Preflight** (F-040). Give `production._guidance_reader` a narrow reader in `resource_repository.py` that returns only the active rules and recent decisions/warnings, with no `_read_git_changes_sync`. `session_brief` keeps the git probe. 4. **Tests.** - 50 stores + forced drains leave ≤ 2 generations per projection in `sqlite_master`, ≤ 2 manifests per projection, and `retrieval_documents` ≤ 2 × live records. - A DB seeded with 120 stale generations shrinks over successive writes (5 per activation per projection) to ≤ 2. - A failed or never-activated generation just below the new one never causes the previously active generation to be dropped. - A GC pass while another connection holds a read transaction does not error the write. - `memory_preflight` spawns no subprocess. - Acceptance: - The growth test passes in under 10 s on 3 OSes, including POSIX in DELETE journal mode (pre-PR-8 DBs). - `verify-v7 --repair-projections` tests stay green. - The preflight profile shows no `git` calls. #### Carry-over from PR 3 (orchestrator, binding) - "Take git out of preflight" is DONE in PR 3 (987bd00: `read_briefing_snapshot(include_git_changes=...)`, guidance passes False) — it was the root cause of preflight timeouts. Drop that part of this PR; keep only the acceptance check that preflight spawns no git process, if not already covered. - Pre-existing race (seen on CI ubuntu 3.10 and locally under load): `memory_recall_entity` raises non-retryable CAPABILITY_DEGRADED when the retrieval projection doesn't yet contain every record the entity points to (`discovery_operations.py:2096`). This PR owns projection freshness: make "projection not yet caught up" a retryable outcome (e.g. DATABASE_IN_USE/RETRIEVAL_UNAVAILABLE with retry_after_ms) and make `test_process_discovery_surface` wait/poll for recall to see the records, like `test_process_client` does. - [REASSIGNED to PR 10 by the orchestrator after PR 4's general review: it shares no code or data path with generation GC and holds nothing up] Recall `bounded_excerpt` repeats short content twice (`retrieval/composer.py:561-566`). Also confirm the "LEXICAL_REBUILD_REQUIRED"/stale marker set on every write (`event_store.py:612`) always clears once the rebuild job runs (GC must not leave it stuck). --- ## What landed # PR 4 summary: Garbage-collect projection generations (and retryable entity recall) (Saved by the orchestrator from the implementer's reply.) Branch `v7/bf-04-garbage-collect-projection-gener`, base 987bd00 (PR 3 tip). Worktree `D:\Daem0n-MCP\.claude\worktrees\agent-aacf6400555e70fbc`. - 556c1b4 `fix: garbage-collect superseded projection generations` - 10bf9d6 `fix: report entity recall on a lagging lexical index as retryable` - d0773dd `test: expect collected lexical generations after reactivation` (final) ## What changed 1. **Generation GC (KD-5, R-24; F-001/F-010/F-026/F-039/F-057)**: `GENERATION_TABLES` moved to `retrieval/lexical_config.py` (verification_v7 imports it). New `retrieval/projections.py::collect_superseded_generations(connection, workspace_id, projection, limit=5)`: no-op inside a caller's transaction, for `graph` (discovery FKs), or invalid workspace id; else own `BEGIN IMMEDIATE` with 250 ms busy timeout (restored after); selects manifests `status IN ('ready','failed')`, keeping the most recently deactivated `ready` (highest `activated_at_us`, ties by generation) — active/building/rebuild_required excluded by the filter; oldest first, ≤5 per pass; per generation: `DROP TABLE IF EXISTS` FTS table (`lexical_fts_table_name`/`procedure_fts_table_name`), delete generation-table rows, delete manifest; busy/locked → debug log, other `sqlite3.Error` → warning; returns 0, never raises. Called after the activation commit in `LexicalProjectionBuilder.rebuild` and `SpecializedProjectionBuilder.rebuild` (procedure, outcome, temporal) only when the builder owns the transaction, never on dry runs or when reusing a current generation. 2. **Lock contention (F-060)**: lexical and specialized rebuilds report SQLite busy/locked (`ProjectionJobRunner._is_lock_contention`) as `DATABASE_IN_USE`; job runner re-queues instead of dead-lettering. `api/v7/operations.py` passes projection DATABASE_IN_USE through (projection_rebuild retryable, not CAPABILITY_DEGRADED). 3. **Preflight git**: done in PR 3; acceptance test added. 4. **Entity recall race**: `_memory_recall_entity` returns retryable DATABASE_IN_USE (250 ms) when entity members are missing and the lexical provider isn't `ready`; stays non-retryable CAPABILITY_DEGRADED when lexical is `ready`. `test_process_discovery_surface` polls recall with a 30 s deadline accepting only ok or retryable. ## Tests - New `tests/test_retrieval_projection_growth.py` (forces DELETE journal mode, `synchronous=OFF`): 50 spaced writes via real `drain_projection_jobs` → lexical/procedure/outcome/temporal ≤2 manifests, lexical/procedure ≤2 FTS tables matching manifests, `retrieval_documents` ≤ 2× live records; 120-generation backlog (GC patched out while seeding) shrinks by exactly 5 per activation to {active−1, active} for lexical and procedure; failed/never-activated generations dropped while previous active survives; GC while another connection holds a read transaction on an old FTS table → rebuild still active, GC returns 0 after 250 ms busy, connection clean, next activation collects; contended lexical job → `queued`, reason DATABASE_IN_USE; GC in caller transaction or for graph → no-op. Reverted-source: 5 tests fail (e.g. 51 lexical manifests left). - `test_production.py::test_memory_preflight_spawns_no_subprocess` (control: session_brief spawns git; preflight spawns nothing; fails if guidance sets include_git_changes=True). - `test_discovery_operations.py::test_memory_recall_entity_missing_members_retry_only_while_stale`. - `test_migrate_v7.py` reactivation expects `[(2,ready),(3,active)]` (gen 1 collected). - Full suite Windows 3.12: 2825 passed, 121 skipped, 0 failed (1671 s). Linux Docker 3.10: growth file 6 passed in 3.2 s (<10 s acceptance); jobs/specialized/runtime/verify_v7 (incl. --repair-projections)/discovery/production/process_discovery: 149 passed, 3 skipped. Windows related modules: 504 passed. ruff/format clean; CI mypy clean. - Entity recall race not reproduced in 12+12 parallel runs; classification rests on the carry-over diagnosis and code reading. ## Deviations 1. Outside file list: `api/v7/operations.py` (DATABASE_IN_USE passthrough), `tests/test_migrate_v7.py` (expectation). 2. Specialized builder also got the F-060 fix. 3. GC catches every `sqlite3.Error`. 4. `resource_repository.py`/`production.py` unchanged (PR 3 did the git change). ## Follow-ups (not fixed) - Graph generations still accumulate (discovery rows reference them). - `rebuild_required` manifests from verify-v7 repair never collected (by design). - Existing DBs shrink 5 generations per activation per projection; VACUUM needed to reclaim disk (PR 12 docs). - Lexical provider reads manifest then FTS table in separate implicit transactions; keeping the previous generation makes it safe for one activation; two activations between reads → "unavailable", not a crash. - `_generation_is_current_sync` in recall_entity still non-retryable CAPABILITY_DEGRADED if a graph rebuild lands mid-request. - Carry-over not addressed (added to the plan after launch): recall `bounded_excerpt` repeats short content (`retrieval/composer.py:561-566`); confirm stale/LEXICAL_REBUILD_REQUIRED marker always clears. ## Round 2 (ff9dcf5) 21 addressed, 3 wontfix (bounded_excerpt trio reassigned to PR 10), 1 wontfix (graph GC — discovery FKs; tracked follow-up). - Entity recall: `_lexical_catchup_pending_sync` — retryable only when the missing record_ids are absent from the ACTIVE lexical generation's `retrieval_documents` AND a `retrieval.projection_rebuild` job is queued/running; dead-lettered rebuild, missing FTS5, config/table mismatch, missing diagnostic, or indexed-but-unmatched → terminal CAPABILITY_DEGRADED. Kept DATABASE_IN_USE (RETRIEVAL_UNAVAILABLE isn't in the wire registry). - Growth test: `include_optional=True`, 20 writes, asserts all four projections rebuilt each drain (5.0 s Windows / 2.07 s Linux); asserts no active manifest keeps `rebuild_required_*`. - One-shot compaction: `retrieval.operations.compact_projections` + CLI `python -m daem0nmcp.cli --project-path <root> compact-projections --workspace-id ws_…` (`--no-vacuum` to skip VACUUM). Takes the exclusive storage lock (server must be stopped); loops GC per projection until 0 then VACUUMs; reports collected and before/after superseded counts. Release inventory regenerated. **PR 12 must document this as the step BEFORE VACUUM.** - Keep-rule by `ORDER BY generation DESC` (no clock); `rebuild_required` generations below active now collected (COALESCE guard); per-victim SAVEPOINT so one undroppable victim doesn't wedge GC; handlers catch `Exception`; `typeof(generation)='integer'` and delete by manifest_id; `jobs.is_lock_contention` public; single `COLLECTABLE_PROJECTIONS`; verify-v7 emits `checks["projection_generations"]`; generation-row deletion asserted for all three tables; WAL blocked-reader test; process-surface poll asserts DATABASE_IN_USE. Results: Windows full suite 2830 passed / 121 skipped (2 errors = unrelated disk-I/O flake in test_graph.py under container load; module passes alone). Linux 3.10 container green; ruff/format, CI mypy, release inventory clean. Mutations: GC disabled → 3 tests fail; GENERATION_TABLES shrunk to lexical → growth test fails. --- ## Review # PR 4 review — merged (round 2: verified by tests, general-2 and plan-alignment; CI run 35490179442 green) Reviewer slots: general, general-2, tests, security, plan-alignment (effort 5). Item statuses: 20 addressed, 4 wontfix. CI: https://github.com/9thlevelsoftware/Daem0n-MCP/actions/runs/35490179442 (green) 🤖 Generated with [Claude Code](https://claude.com/claude-code)
… OpenCode installer (#68) **Brownfield run 3403b448, PR 9 of 12** — audit-and-fix stack for the v7 core (#61/#62). Base: `v7/bf-01-make-ci-green-and-pin-workspace`. ## Plan (contract) ### PR 9: Auto-enable installed profiles and fix the Windows launcher and OpenCode installer - Depends on: PR 1 - Files: daem0nmcp/capabilities.py, daem0nmcp/api/v7/discovery_operations.py, daem0nmcp/cli.py, start_server.py, start_daem0nmcp_server.bat, daem0nmcp/opencode_install.py, tests/test_capabilities.py, tests/test_start_server.py, tests/test_opencode_install.py - Traces to: FP-6, FP-7, F-036, F-083, F-032, F-049, F-077, A-016, A-017, UD-7 - Risk: medium - Description (R-9: runtime behavior split out of the docs PR): 1. **Profiles** (KD-8, UD-7). `capabilities._configured` returns "auto" when the variable is unset: - auto → ready if all distributions are present, disabled otherwise; - explicit false → disabled; - explicit true with packages missing → degraded. The disabled-but-installed remediation says to unset the variable or set it to true. Fix the `index` remediation (`tree-sitter-language-pack`) and `discovery_operations.py:1427`. 2. **Launcher** (F-032/F-049). - `start_server.py` uses `--project`, else an existing `DAEM0NMCP_PROJECT_ROOT`, else cwd, and never overwrites an exported value. - `start_daem0nmcp_server.bat` captures `%CD%` before `cd /d "%~dp0"` (or takes `%1`) and passes `--project`. 3. **OpenCode** (F-077). `opencode_install.py` copies the four repo command files into `.opencode/commands/`, create-if-absent. 4. **Windows stdio** (A-016). - Confirm that the stdio variant of `tests/api_v7/test_process_client.py` runs, not skipped, on the Windows leg. - Run the manual Claude Code check on Windows: `claude mcp add daem0nmcp -- python -m daem0nmcp` → `session_brief` → store → recall. Record the Claude Code version and the result in the PR. PR 12 documents it. 5. **Tests.** - `test_capabilities.py` covers all four profile states. - A launcher test: an exported `DAEM0NMCP_PROJECT_ROOT` survives, and `--project` wins. - The installer creates the command files and does not overwrite existing ones. - Acceptance: - A green 3-OS CI run with `[graph]` (installed by CI) now auto-enabled across the whole suite. Any test that assumed "disabled" is fixed here. - `start_daem0nmcp_server.bat` launched from a project directory registers that project. - The PR records the A-016 result. --- ## What landed # PR 9 summary: Auto-enable installed profiles and fix the Windows launcher and OpenCode installer (Saved by the orchestrator from the implementer's reply.) Branch `v7/bf-09-auto-enable-installed-profiles-a`, base f7f024a (PR 1 tip). Worktree `D:\Daem0n-MCP\.claude\worktrees\agent-a51f5956496f2f3bf`. - 6eb091e `feat: enable an optional profile automatically when its extra is installed` - e63f1e0 `fix: make the Windows launcher serve the caller's project` - f2e6a5d `fix: install the OpenCode slash-command files` - 5e731b1 `test: disable installed profiles explicitly where a test needs them off` - 46008ec `test: import the launcher before snapshotting sys.modules` (final) ## What and why 1. **Profiles (KD-8, UD-7)**, `capabilities.py`: `_configured` returns "auto" when the variable is unset or empty (empty used to mean false). unset + installed → ready; unset + missing → disabled with `install_extra`; `false` → disabled with `enable_configuration` ("Unset X or set it to true…"), packages not probed; `true` + missing → degraded (unchanged); invalid → failed. models-local on <3.11 keeps `upgrade_python` (disabled under auto, degraded when true). All consumers go through `CapabilityRegistry`. 2. **Remediation text**: discovery_operations apps remediation ("pip install 'daem0nmcp[apps]'; turns on automatically unless DAEM0NMCP_APPS_ENABLED=false"); `cli.py index` names `tree-sitter-language-pack` and `daem0nmcp[apps]`. 3. **Launcher (F-032, F-049)**: `start_server.py` uses `--project`, else a non-empty exported `DAEM0NMCP_PROJECT_ROOT`, else cwd; never overwrites an exported value. `.bat` captures `%CD%` or `%~f1` before `cd /d "%~dp0"`, passes `--project "%DAEM0N_PROJECT%\."`; header no longer says HTTP is required; CRLF kept. Live check: real `.bat` from a temp project → `session_brief` on the project ok; on the repo's workspace → UNAUTHORIZED_WORKSPACE. 4. **OpenCode (F-077)**: the four command files packaged under `daem0nmcp/opencode_assets/commands/` (package-data `commands/*.md`); `install_opencode` writes them via `_ensure_file` (existing kept `[exists]`, `--force` overwrites, dry-run writes nothing); byte-equality test against repo `.opencode/commands`. 5. **Windows stdio (A-016) — confirmed working**: `test_process_client.py` stdio variant runs (6 passed, 0 skipped) incl. brief → preflight → store → outcome → recall → restart → recall; scripted MCP SDK stdio_client with `python -m daem0nmcp` in a fresh project (initialize 13.2 s cold, 75 tools, brief/preflight/store/recall ok; graph and apps auto-ready); real Claude Code 2.1.278 `claude mcp add` → Connected, `claude -p --mcp-config` brief/preflight/store/recall ok (entry removed afterwards). Line for PR 12: "Stdio (`python -m daem0nmcp`) works on Windows 11 with Claude Code 2.1.278; HTTP via the launcher is optional." ## Tests - `test_capabilities.py`: 5 new (auto+installed ready; auto+missing disabled/install; false+installed disabled/unset message; true+missing degraded; auto models-local on 3.10 disabled/upgrade_python). - New `test_start_server.py`: exported root survives; `--project` wins; cwd default; Windows-only: real `.bat` via `cmd /c` with python.cmd shim checks `%CD%` and a `%1` with spaces/parentheses reach `--project`. - `test_opencode_install.py`: command files created, existing `recall.md` kept, others match repo. - Tests that assumed "disabled" now set the variable false: `test_graph_operations.py`, `test_process_code_impact.py` (also new remediation text), `test_process_dreaming.py`, `test_vectors.py`. - `test_process_portability.py[*-streamable-http]`: with graph auto-on, `storage.rename()` right after HTTP server exit hit WinError 5 intermittently (1/3); passes 4/4 with graph/apps off. Test-only rename retry (15 s deadline) → 5/5. Root cause not pinned. - Isolation sweep: no EXPECTED pin changed. - Full suite (Windows 3.12, `.[dev,apps,graph]`) at 5e731b1: 2813 passed, 121 skipped, 0 failed (1263 s). First run before test fixes: 8 failed (6 "disabled" assumptions + portability rename + one-off consolidation INTERNAL_ERROR, 8/8 on rerun). ruff/format clean; mypy api/v7 clean; release inventory valid. Linux Docker python:3.10-slim affected modules: 115 passed, 5 skipped, 2 failed in test_start_server (stale module after `patch.dict(sys.modules)`) → fixed in 46008ec (3 passed, 2 skipped). ## Deviations 1. Command files packaged (pip installs have no repo) with an equality test. 2. Empty `DAEM0NMCP_<P>_ENABLED=` treated as auto. 3. Manual check via `claude mcp add`/`list` + headless `claude -p`, not interactive. 4. Edited tests outside file list (graph_operations, process_code_impact, process_dreaming, vectors, process_portability) and pyproject package-data. ## Follow-ups (not fixed) - Portability rename race on Windows with graph on (lingering handle — langgraph checkpoint / Leiden / antivirus?). - `production._capability_states` reports generic "Review the <name> capability profile." remediation in system_health. - `.opencode/plugins/daem0n.ts` also duplicated in the package (kept in sync by test). - Recall on a fresh workspace reported LEXICAL_REBUILD_REQUIRED/stale projections, and `bounded_excerpt` repeated content twice (not investigated). - Stdio cold start ~13 s on Windows. - `.gitignore` lacks `.venv/`. ## Rounds 2–3 2c6ac4d/a68b205/7df3bc2 addressed all 26 review items (22 addressed, 3 partially, 1 wontfix), then 0543b14 merged PR 3 and 2dc3d23 merged PR 4 — PR 9 now stacks on v7/bf-04. - **Root cause of the recurring export→import IMPORT_INVALID (hit CI on PRs 1, 3, 5, 9): SQLITE_BUSY, not a bad bundle.** `prepare_import` does session bookkeeping in short autocommit write transactions after `claim_import_finalization` commits; a concurrent writer (projection drain / dreaming job, busier with graph auto-on) holds the write lock past the 5 s busy_timeout (`portable_projections.py:410 _account_physical_files`). `operations._portable_failure_code()` now walks `__cause__` (only), maps busy/locked via PR 3's `errors.is_database_busy` to retryable DATABASE_IN_USE, and logs the cause on both import and export; the test retries only that code, bounded to 8 attempts/30 s and asserting success within the first 4. - Per-profile preload at build time: a broken-but-installed extra (import raises, e.g. a DLL OSError) degrades only that profile everywhere and names the module and exception type; numpy removed from the graph preload (false warning on every default start). - `capability_states` render the registry's real remediation; `sandbox_execute_python` requires the agency-e2b capability so `=false` really disables it; launcher refuses a non-existent `--project`, uses `setlocal DisableDelayedExpansion`, omits `--project` when an exported root exists, refuses System32; `_ensure_file` refuses symlinks and creates exclusively; invalid-value message mentions "leave it unset". - CHANGELOG "Behaviour change" entry (auto-on profiles, startup/download/re-embed costs, empty = auto, per-profile opt-out). Process harness dumps the server log tail on failure. - Results: Windows full suite 2854 passed / 122 skipped; discovery surface 9/9 across 3 parallel lanes after the PR 4 merge; CI run 35501238206 green on all 9 test legs + lint/type-check/release-inventory. --- ## Review # PR 9 review — merged (round 3: all addressed/wontfix; PR 3 and PR 4 merged in; CI run 35501238206 green on all 12 jobs) Reviewer slots: general, general-2, tests, security, plan-alignment (effort 5). Item statuses: 25 addressed, 1 wontfix. CI: https://github.com/9thlevelsoftware/Daem0n-MCP/actions/runs/35501238206 (green) 🤖 Generated with [Claude Code](https://claude.com/claude-code)
… everything else (#67) **Brownfield run 3403b448, PR 5 of 12** — audit-and-fix stack for the v7 core (#61/#62). Base: `v7/bf-01-make-ci-green-and-pin-workspace`. ## Plan (contract) ### PR 5: Exempt user text from the path guard, with one predicate for everything else - Depends on: PR 1 - Files: daem0nmcp/api/v7/models.py, daem0nmcp/api/v7/tools.py, daem0nmcp/api/v7/pinned.py, daem0nmcp/api/v7/runtime_services.py, daem0nmcp/api/v7/operations.py, daem0nmcp/api/v7/record_operations.py, tests/api_v7/test_path_policy.py - Traces to: FP-2, FP-3, F-005, F-006, F-025, F-008, F-061, F-064, F-071, A-006, A-012, UD-3 - Risk: medium - Description (KD-2): 1. **`UserText` type.** In `models.py`, add `UserText = Annotated[str, …]` and keep one `is_host_absolute_path()`. `WireModel`'s model validator skips fields whose annotation is `UserText` (or `list[UserText]`/`dict[str, UserText]` for tags and context values) and checks every other string, input and output. `RelativePath` keeps its validator. 2. **Annotate the user-text fields** in the input and output models (`tools.py`/`models.py`): - memory content, rationale, tags, context values; - recall queries, trigger patterns, outcome text, preflight description; - rule lists `must_do`/`must_not`/`ask_first` (`tools.py:381-383, 603-605, 1410-1423`) and entity names; - output record summaries, recall excerpts/highlights, `meta.warnings` that quote content, dreaming and community summaries, and code-index symbols. 3. **Delete the drifted guards.** Delete `_path_safe_success` and both `_contains_raw_path` copies (`pinned.py:80-110`, `runtime_services.py:124-140`). Route `operations.py:_reject_raw_paths` through `is_host_absolute_path`. 4. **Visibility** (F-061). Validate `context.visibility` as a `Literal` of the values `retrieval/projections.py` accepts. 5. **Filesystem-sink checklist** (R-23), in the PR description. List every input field that reaches `Path(...)`, `open`, `os.*` or subprocess, and confirm each is `RelativePath` (or an https URL for ingest). 6. **Tests** (`test_path_policy.py`). - Store `\\d+`, `/health`, `GET /api/v1/users` and `C:\proj\src\app.py` content as `decision` and as `warning`: the store returns a `record_id`, and brief, recall, preflight and outcome all succeed. - A pre-fix poisoned record inserted directly reads back fine. - `relative_file_path` values `C:\x` and `/etc/x` are rejected. - Round-trip property: every string accepted in a `UserText` input field validates in every output model field that can carry it. - **Mechanical coverage** (R-26). Enumerate every free-text-typed field (`str`/`MediumText`/`ShortText` aliases, including inside lists and dicts) of every write tool's input model *from the models themselves*, not from a list. Seed each with `C:\proj\x.py`, `/health` and `\d+`: rules, triggers, entities, context and memories. Also insert the same strings directly into `memory_records`, `governance_*` and entity rows, the way a migrated v6 DB carries them. Then call every read tool and resource template (reusing PR 1's harness), and assert that none returns an error. An unannotated user-derived field fails this test. - Model-wide check: a server-generated field (for example a health diagnostic) set to `C:\x` fails validation. - PR 1's sweep, including its leak assertion, stays green. - Acceptance: - The new tests pass. - `_path_safe_success` and `_contains_raw_path` no longer exist. - Exactly one absolute-path regex set remains in `daem0nmcp/api/v7`. - The checklist is in the PR description. --- ## What landed # PR 5 summary: Exempt user text from the path guard, with one predicate for everything else (Saved by the orchestrator from the implementer's reply.) Branch `v7/bf-05-exempt-user-text-from-the-path-g`, base f7f024a (PR 1 tip). Worktree `D:\Daem0n-MCP\.claude\worktrees\agent-ab8e615d18037cdb7`. - 224e7ee `fix: let user text mention paths, with one path predicate for everything else` - e6a087e `test: prove path-bearing user text round-trips through every tool` - 9ef43d1 `test: pin every read against its pre-path baseline` (final) 20 files, +1046/−344. Production: `api/v7/{models,tools,resources,pinned,runtime_services,operations,resource_repository,discovery_operations,code_entity_operations,intelligence_operations,utility_operations}.py`, `capture_candidates.py`, `edit_bridge.py`. Tests: new `tests/api_v7/test_path_policy.py`; updated `test_process_workspace_isolation_sweep.py`, `test_models.py`, `test_operations.py`, `test_pinned_handlers.py`, `test_resource_repository.py`, `test_runtime_services.py`. ## What and why 1. **One predicate (`models.py`)**: `is_host_absolute_path(str)` is the only predicate/regex set (Windows, POSIX, `file:` URI; `_WINDOWS_DRIVE` for RelativePath). `UserText = Annotated[str, _USER_TEXT]` and `UserJsonObject` (ContextJsonObject built on it). `guarded_strings(value)` walks fields by their own annotations, skipping any carrying the marker (bare/Optional/list/set/dict). `WireModel.reject_absolute_filesystem_paths` checks every guarded string on input and output. `contains_absolute_filesystem_path` kept as a thin wrapper (edit-bridge native-argument check, legacy_compat). `discovery_operations._safe_code_path` uses `_relative_path`. 2. **Annotated fields**: tools.py aliases `UserShortText`, `UserNameText`, `UserMediumText`, `UserLongText`, `OptionalUserMediumText`, `RecordContext` (unused ShortText/LongText/OptionalMediumText deleted). Inputs: memory content/rationale/context/tags/procedure steps (store, batch, capture promote); all queries; preflight description and target_arguments; edit description; outcome text; rule trigger/must_do/must_not/ask_first/warnings (create + RulePatch); trigger pattern/recall query; entity names, qualified_name; compress/verify text; sandbox code; link description/label; active-context reason; ingest topic; refactor objective; compaction summary/query; debate topic/positions; brief focus_areas. Outputs: TriggerView pattern/query, OutcomeSummary excerpt, PreflightGuidance/RuleCheckData lists, TriggerMatch.matched_value, community/graph-node/entity labels and names, TextSearchHit excerpt, compress text, DocumentSource topic, verify claims, sandbox stdout/stderr, CodeEntitySummary.qualified_name, TodoFinding.text, refactor proposal, relationship description, mermaid text, EntityEvolutionItem.summary, export event/vector/legacy-row payloads, CaptureCandidateView.proposed_record, link label, debate rounds/synthesis; models.py RecordSummary.excerpt, EvidenceItem.bounded_excerpt, RetrievalData.rendered_context, Tag; resources.py RuleView text fields, ActiveContextItem.reason. **Still guarded**: diagnostics, remediation, covenant reasons, UpdateSummary, differences, rule-evolution summary, contradiction explanation, sanitized_logs, refactor warnings, health, manifests, provenance. 3. **Drifted guards deleted**: pinned.py `_path_safe_success`, `_contains_raw_path` + regexes, guidance internal_error branch; runtime_services.py `_contains_raw_path` + 7 call sites (all on validated WireModels); `operations._reject_raw_paths` no longer scans free text — refuses non-empty `file_path`/`project_path`/`database_path` and absolute `relative_file_path`/`file_path_relative`, not descending into `context`, still refuses bytes (before, any memory mentioning `/health` broke export/versions/at-time); also removed `resource_repository._reject_canonical_root` (a warning naming the root made the resource unreadable) and path checks in `code_entity._safe_event_content`, `intelligence._safe_text`, TODO scan text, capture-candidate record, edit-preflight description (only hash stored). 4. **Visibility (F-061)**: `RecordContext` validates `context.visibility` ∈ {workspace, private, shared} (the set projections accept) on store/batch/promote; before, "team" was committed then INTERNAL_ERROR. 5. **Filesystem-sink checklist**: `code_index.relative_root`, `patterns[]`, `code_todos_scan*.relative_root`, `code_refactor_propose.relative_file_path` → RelativePath; `memory_recall_file`/`context_triggers_match.relative_file_path` → SQL/glob only; `memory_store.relative_file_path` stored never opened; `document_ingest_url.url` HttpsUrl (https, no credentials); `sandbox_execute_python.code` → remote E2B. No v7 input reaches subprocess (only fixed git invocation in resource_repository). 6. **Sweep (PR 1)**: B and C each store a warning naming `<root>/src/neutral.py`. Scan validates each response against its own model (`ApiResponse[TOOL_DATA_MODELS[tool]]` / resource document model, strict=False), then checks only `guarded_strings()` for root/home needles — exemption by annotation only. New `test_leak_check_exempts_user_text_by_annotation_only` proves a server-generated field carrying the root fails. Pinned outcomes unchanged (70 tools, 4 resources, B and C, JWT pass). ## Tests - `test_path_policy.py` (13 tests): model-level rejects `C:\x`, `/etc/x` in relative_file_path; user-text inputs accept 7 strings; guarded diagnostics/remediation with paths fail; >30 UserText output fields accept each string via WireModel probe, guarded MediumText refuses. `test_path_content_is_stored_and_read_back`: pre-fix `\\d+` warning inserted via EventStore appears in session_brief; `\\d+`, `/health`, `GET /api/v1/users`, `C:\proj\src\app.py` stored as decision and warning → brief, recall, text search, preflight, outcome succeed, 4 resources read. `test_every_free_text_field_accepts_paths_and_every_read_survives` (R-26): ~60 free-text fields across 31 tools discovered from model annotations, seeded with `C:\proj\x.py`, `/health`, `\d+`; seeded outcome equals neutral or is a non-refusal; plus direct inserts (records, rule, file trigger, entities, community); every read tool + 4 resources compared to pre-path baseline — none degrades. Mutations (guarding TriggerView.pattern on output or input; guarding EntitySummary.name) fail the test. - `test_path_policy.py` 13 passed (~75 s); sweep 3 passed (37 s); ruff/format clean; mypy api/v7 clean (native, linux); release inventory valid. - Full suite: 2815 passed, 121 skipped, 1 failed (`test_database.py::TestSQLitePragmas::test_foreign_keys_enabled`, WinError 32 on `.migrate-v7.lock` — file-lock flake with other agents on the machine; passes on rerun). Earlier `-x` run hit one `disk I/O error` in test_runtime_services; passes on rerun. CI not yet run. ## Deviations 1. Files beyond plan list (each had a user-text refusal or leftover regex found by the mechanical test): resources.py, resource_repository.py, discovery_operations.py, code_entity_operations.py, intelligence_operations.py, utility_operations.py, capture_candidates.py, edit_bridge.py. record_operations.py unchanged (file_path integrity check is PR 6). 2. PR 1 sweep changed to pass with host paths in user text. 3. Six existing tests rewritten (asserted old refusal of paths in free text). 4. Whole JSON fields exempt incl. keys (ExportEvent.payload, vector payloads, legacy_rows, proposed_record, preflight target_arguments, context); structured path keys in event payloads still refused by `_reject_raw_paths`. 5. `document_ingest_url.topic` and `memory_compact.summary` seeds only reach a pre-content refusal (acceptance checked, not storage). 6. `workspace_import` bundles not seeded. ## Follow-ups (not fixed) - Control characters accepted on input but refused on output by RecordSummary.excerpt and Tag — same stored-then-bricked pattern, not paths. - Model-wide POSIX regex refuses some valid relative paths (`src/__pycache__/x`) and globs (`src/**/foo.py`). - edit_bridge refuses native edit arguments with absolute paths, but Claude Code's Edit sends absolute file_path (PR 2 area). - Migrated absolute `memory_records.file_path` is PR 6 scope. ## Round 2 0a204d3 + 88e66ff + 797685e. 24 addressed, 1 wontfix (non-path migrated-row limits → PR 6). Key: `ErrorRemedy.arguments` → UserJsonObject (token errors no longer crash on path content); control characters read back (`_sanitized` removed from Tag/RecordSummary.excerpt/EvidenceItem.bounded_excerpt and from intelligence/code-entity checks); mechanical coverage test walks nested models from annotations, asserts on unclassified shapes, covers workspace_import via a real export→import round trip, requires non-refused neutral calls, pins GATED_READS and a ≥60 seed floor; vector payloads guarded again, ExportEvent.payload exempt only for `data`, legacy pages through `_reject_raw_paths` with RelativePath validation; leak sweep scans every string allowing root/home only in a marked UserText note; sanitized_logs UserText(4096); RelativePath skips the substring scan (globs OK); legacy_compat keeps path context; `_safe_code_path` deleted; `test_memory.py::test_v7_legacy_id_keeps_opaque_canonical_origin` updated to expect kept context (still asserts opaque ids/evidence). All 8 reviewer mutations now fail. CI run 35439309031 green on all 9 legs + lint/type-check/release-inventory. --- ## Review # PR 5 review — merged (round 2: all addressed; general/tests/plan-alignment verified; CI 35439309031 green) Reviewer slots: general, general-2, tests, security, plan-alignment (effort 5). Item statuses: 24 addressed, 1 wontfix. CI: https://github.com/9thlevelsoftware/Daem0n-MCP/actions/runs/35439309031 (green) 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Keep PR 3's retryable DATABASE_IN_USE mapping, exception chaining, and tokenless-preflight warnings. Drop the old pinned/runtime path predicates now that WireModel owns the absolute-path rule and exempts user text. Co-authored-by: Devil <9thLevelSoftware@users.noreply.github.com>
response.success expects Sequence[ApiWarning], not None. The merge replaced _path_safe_success, which previously coerced None to (). Co-authored-by: Devil <9thLevelSoftware@users.noreply.github.com>
…l error (#65) **Brownfield run 3403b448, PR 3 of 12** — audit-and-fix stack for the v7 core (#61/#62). Base: `v7/bf-01-make-ci-green-and-pin-workspace`. ## Plan (contract) ### PR 3: Report transient failures as retryable and log every internal error - Depends on: PR 1 - Files: daem0nmcp/api/v7/responses.py, daem0nmcp/api/v7/runtime_services.py, daem0nmcp/api/v7/resource_repository.py, daem0nmcp/api/v7/pinned.py, daem0nmcp/event_store.py, daem0nmcp/api/v7/errors.py, tests/api_v7/test_error_honesty.py, tests/api_v7/test_process_concurrent_writers.py, tests/api_v7/test_pinned_handlers.py - Traces to: FP-2, F-020, F-027, F-021, F-028, F-055, F-066, F-070, F-082, A-011 - Risk: low - Description (KD-6): 1. **Busy mapping.** In `SQLiteMemoryEventWriter._run_mutation` and the outcome path (`runtime_services.py:570-597, 757-770, 883-886`) and in `ResourceRepository._run` (`resource_repository.py:947-984`), map `BoundedWorkerBusyError` and `sqlite3.OperationalError` "database is locked"/"busy" to `DATABASE_IN_USE` (retryable, `retry_after_ms=250`) in `pinned._EXPECTED_SERVICE_ERRORS`. 2. **Logging.** In `ResponseContext.internal_error`, replace `del error` with `logger.exception("v7 internal error correlation_id=%s", …, exc_info=error)` to stderr. Service raises chain `from exc`. The wire envelope is unchanged. 3. **Conflict mapping.** In `event_store.py:539-542`, only `sqlite3.IntegrityError` becomes `EventStreamConflict` (F-066). 4. **Null token.** `memory_preflight` with target arguments that don't normalize keeps `ok:true`, but adds a `meta.warnings` entry with the field errors and "no preflight token issued" (F-070/F-082). 5. **Tests.** - 8 parallel preflight+store flows: every response is either ok or retryable `DATABASE_IN_USE`, and after retries there are exactly 8 records and 8 events. - A forced lock returns `DATABASE_IN_USE`. - An induced exception is logged with the correlation ID returned to the client. - Two `process_client` stdio servers on one workspace fire 5 same-key and 10 distinct-key stores concurrently: exactly one event per key and no `INTERNAL_ERROR` (F-055). - Acceptance: - The new tests pass on 3 OSes. - There is no `del error` left in `responses.py`. --- ## What landed # PR 3 summary: Report transient failures as retryable and log every internal error (Saved by the orchestrator from the implementer's reply.) Branch `v7/bf-03-report-transient-failures-as-ret`, base f7f024a (PR 1 tip). Worktree `D:\Daem0n-MCP\.claude\worktrees\agent-a0bb29ebdbe88c9ac`. - 358256f `fix: report only SQLite integrity errors as event stream conflicts` - 83a1b7b `fix: report transient contention as retryable and log every internal error` (final) ## What changed 1. **Busy mapping (F-020, F-027, A-011)** - `api/v7/errors.py`: `is_database_busy(error)` — true for `BoundedWorkerBusyError`, `code == "DATABASE_IN_USE"`, or `sqlite3.OperationalError` containing "locked"/"busy". - `runtime_services.py`: `SQLiteMemoryEventWriter._run_mutation` maps `BoundedWorkerBusyError` → `RuntimeServiceError("DATABASE_IN_USE")`; `_store_sync`/`_record_outcome_sync` catch-alls map busy → DATABASE_IN_USE (else MEMORY_STORE_FAILED/MEMORY_OUTCOME_FAILED); `_open_database` busy → DATABASE_IN_USE (else ACTIVE_V7_UNAVAILABLE); `BasicBriefingService.assemble` and `BasicPreflightService.guidance` pass DATABASE_IN_USE through instead of BRIEFING_FAILED/PREFLIGHT_FAILED. - `resource_repository.py`: `ResourceRepositoryError` optional `code`; `_run` raises `ResourceRepositoryError("DATABASE_IN_USE")` when busy; always chains the cause. - `pinned.py`: `_expected_service_failure` adds `retry_after_ms=250` for DATABASE_IN_USE (already retryable). 2. **Logging (F-021, F-028)** - `responses.py`: `del error` removed; `internal_error` logs `"v7 internal error correlation_id=%s"` with `exc_info` (error or active exception) via `logging.getLogger(__name__).error`. Goes to stderr (last-resort handler / v6 basicConfig), never the MCP wire; wire envelope unchanged; no "NoneType: None" when no exception. - `runtime_services.py`: all 28 `raise RuntimeServiceError(...) from None` → `from exc`. MCP-facing resource/middleware errors keep `from None` (tests assert `__cause__ is None`). 3. **Conflict mapping (F-066)**: `event_store.py` INSERT into `memory_events` maps only `sqlite3.IntegrityError` → `EventStreamConflict`; I/O and lock errors propagate. aiosqlite path raises the same class. 4. **Tokenless preflight (F-070, F-082)**: still `ok:true` with null token, now plus `meta.warnings` `PREFLIGHT_TOKEN_NOT_ISSUED` listing schema field names and pydantic error types only (unknown keys as `?`, values never echoed). `_path_safe_success` gained optional `warnings`. ## Tests - New `tests/api_v7/test_error_honesty.py` (in-process `create_v7_server("stdio")` + `fastmcp.Client`): 8 parallel preflight+store flows → each ok or retryable DATABASE_IN_USE with 250 ms; after retries exactly 8 records and 8 `memory.created` (13–14 busy retries per run locally). Held `BEGIN IMMEDIATE` → store returns retryable DATABASE_IN_USE after 5 s busy timeout; retry succeeds after release. Induced failure in writer transaction → INTERNAL_ERROR; response lacks the canary; log record contains the returned correlation_id and traceback shows canary + MEMORY_STORE_FAILED. Full resource-repository pool → `code == "DATABASE_IN_USE"`. - New `tests/api_v7/test_process_concurrent_writers.py` (F-055): two real stdio servers on one workspace; 5 same-key + 10 distinct-key concurrent stores → one `memory.created` per key, 11 records, exactly one non-replay among same-key, no code except DATABASE_IN_USE; bounded 120 s deadline honouring `retry_after_ms`. - `test_pinned_handlers.py`: description-only preflight checks the warning; new DATABASE_IN_USE → retryable, 250 ms. - `test_runtime_services.py`: cancelled-worker test now expects `RuntimeServiceError("DATABASE_IN_USE")` caused by `BoundedWorkerBusyError`. - `test_event_store.py`: `test_only_integrity_errors_on_insert_are_stream_conflicts`. - Reverted-source check: all 8 new/changed tests failed on old code; two-process test reproduced F-055 (INTERNAL_ERROR). - Full suite Windows 3.12: 2809 passed, 121 skipped, 0 failed (834 s). New tests 5× in a row: all pass (~21 s each). Linux Docker python:3.10-slim affected modules: 94 passed. ruff check/format clean; mypy api/v7 0 errors on win32/linux/darwin; release inventory valid. Sweep EXPECTED pins unchanged and green. macOS unverified until CI. ## Deviations 1. `tests/api_v7/process_client.py` gained `log_name` (default unchanged) so two servers don't share `stdio-server.log`. 2. IntegrityError test in `tests/test_event_store.py` (not in file list) to reuse its schema fixture. 3. DATABASE_IN_USE also flows through `_open_database` and briefing/preflight services, beyond the plan's line ranges — needed for contended session_brief/preflight (F-020's 8 concurrent briefs). 4. `logger.error(..., exc_info=...)` instead of `logger.exception` (some call sites aren't in an except block). 5. Two commits rather than one per item. ## Follow-ups (not fixed) - `application.py:286-300`: non-pinned tools send any registered error code, incl. DATABASE_IN_USE, with `retryable=False` — contention on non-pinned tools still non-retryable. One-line fix, outside file list. - `local_state_operations.py` has its own busy check (`_translate_error`) and maps a full pool to TASK_REQUIRED; could use `is_database_busy`/DATABASE_IN_USE. - Other pools map `BoundedWorkerBusyError` to inconsistent codes (F-071); recall maps full pool to retryable RETRIEVAL_UNAVAILABLE (left as is). - `.gitignore` lacks `.venv/`. ## Round 2 ce94143 (overrun-then-clean reads → retryable DATABASE_IN_USE; application.py non-pinned DATABASE_IN_USE retryable + remedy; busy memory_store carries memory_preflight remedy; sqlite_errorcode-based busy detection on 3.11+; governance_events IntegrityError-only; ValidationError logged without input values; locked_current storage lock → DATABASE_IN_USE; "workspace unavailable" responses logged with cause) and 987bd00 (root cause of preflight timeouts: guidance read no longer spawns git — `read_briefing_snapshot(include_git_changes=False)`; removed from PR 4's plan). 14 addressed, 2 wontfix (R-8 branch is reachable; R-16 deferred with F-071). Tests reviewer verified R-9..R-12 (20/20 + 12/12 loaded runs). CI run 35423993350 green after re-run of one unrelated portability flake (carried to PR 6). --- ## Review # PR 3 review — merged (round 2: all addressed; tests reviewer verified; CI 35423993350 green after flake re-run) Reviewer slots: general, general-2, tests, security, plan-alignment (effort 5). Item statuses: 14 addressed, 2 wontfix. CI: https://github.com/9thlevelsoftware/Daem0n-MCP/actions/runs/35423993350 (green) 🤖 Generated with [Claude Code](https://claude.com/claude-code)
CI run 35520333257 (ubuntu 3.11) saw `memory_preflight` "spawn" the Leiden graph worker. It does not: the recorder patches `subprocess.Popen` process-wide, and the spawn came from `daem0nmcp-projection-runtime_0` — the projection drain the *brief's* write scheduled, which with the graph profile now auto-enabled rebuilds communities and runs Leiden in its own thread. Traced it: `_drain_projection_jobs_sync` -> `jobs.run_once` -> `GraphProjectionBuilder.rebuild` -> `_community_seeds` -> `run_leiden_bounded`. Nothing on the preflight path reaches it, so PR 4's guarantee holds and preflight latency is unaffected. The fix closes the overlap rather than narrowing the assertion: the test awaits the drains the brief scheduled before clearing the recorder, so the window is quiet by construction and `assert spawned == []` stays literal — no subprocess of any kind, not "no git specifically", and with the graph profile left enabled. Deterministic across 14 runs where the unquiesced window leaked on 1 of 2. Verified it still fails on a regression: injecting a `Popen` into the per-request guidance closure fails the test both for `git` (F-040's original regression) and for any other worker. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PR 5 (#67) and PR 3 (#65, which carried PR 4 via #66) were squash-merged into v7/bf-01 after PR 1 (#63) had already been squash-merged into working_branch, so none of them reached working_branch. working_branch's tree equals d8842de (bf-01 after PR 2), an ancestor of bf-01's tip, so the merge result is bf-01's tree exactly. The one textual conflict, tests/api_v7/test_process_workspace_isolation_sweep.py, is PR 5's change against PR 1's squashed copy of the same lines; it takes bf-01's side. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UBLLzyphecQABpqVorHunX
Their content is already on this branch through the squash commits, so this merge keeps the tree unchanged and only links their history: - v7/bf-02 (PR 2, #64): tree identical to d8842de. - v7/bf-03 (PR 3, #65): tree identical to cd5842f. - v7/bf-05 (PR 5, #67): patch identical to b003c03. Linking them also gives the v7/bf-06 merge a closer merge base, since bf-06 descends from bf-05 and carries PR 3's original commits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UBLLzyphecQABpqVorHunX
v7/bf-06 (PR 6, "make migrated v6 workspaces usable") never had a pull request. It also carries PR 9 (#68) through its merge of v7/bf-09; #68 was squash-merged into v7/bf-04 after bf-04 had already been squashed into bf-03 (#66), so PR 9 never reached bf-01 or working_branch. Conflicts, all taken from bf-06's side: - api/v7/pinned.py: both sides made the tokenless-preflight warnings an empty tuple (edfaa57 on bf-03, 0bc432b on bf-06); bf-06's form drops the redundant parentheses. - retrieval/lexical_config.py, verification_v7.py: bf-06 also exports RETRIEVAL_PROJECTION_NAMES and builds _SUPPORTED_PROJECTIONS from it; the set is unchanged. - tests/api_v7/test_production.py: bf-06's copy of the preflight no-subprocess test also waits for the brief's projection drains. CHANGELOG.md: fold the second "### Changed" heading the merge produced into the first. The result equals bf-06 plus PR 2's patch (f7f024a..27d1682), and bf-01 plus only PR 6 and PR 9 files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UBLLzyphecQABpqVorHunX
bf-04's tip is #68's squash of PR 9; its tree equals v7/bf-09's tip (2dc3d23), which the v7/bf-06 merge already brought in. This merge keeps the tree unchanged and only links bf-04's history. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UBLLzyphecQABpqVorHunX
9thLevelSoftware
changed the title
A finalize claims a durable lease, so a concurrent writer that holds the SQLite write lock past the busy timeout fails the attempt as the retryable DATABASE_IN_USE and then fails its cleanup release the same way. The release error was suppressed, so the lease stayed for its full 15 minutes and every retry the error invites hit the live-lease branch of claim_import_finalization: a non-retryable TASK_REQUIRED. Keep a lease whose release failed, and release it before the next finalize of the session claims. A still-held lock then fails that release as DATABASE_IN_USE, so the client keeps retrying until the writer finishes. A live concurrent finalizer is unaffected and still gets TASK_REQUIRED. Seen in CI on #69: test_process_portability's retried finalize got TASK_REQUIRED on ubuntu-latest / Python 3.12. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UBLLzyphecQABpqVorHunX
The graph profile's projection drains can hold the write lock past the busy timeout, and the server then answers the import with the retryable DATABASE_IN_USE. The path-policy round trip asserted success on the first call; retry the preflight and import within a bounded deadline, as the portability test does. Seen in CI on #69 on windows-latest / Python 3.10. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UBLLzyphecQABpqVorHunX
Contributor
Code Review SummaryStatus: 0 Issues Found | Recommendation: Merge Overview
VerdictApprove. The two new commits ( Correctness / Safety FindingsNo correctness or safety findings.
The three defects listed in the PR body as "Known defects carried in" (task dispatcher Ponytail Review
Otherwise the diff is two patches totalling ~150 lines, both Ponytail net: 0 lines. Suggested Minimal PatchNo patch needed. Final Merge GuidanceCan merge as-is. (Already merged at Files Reviewed (2 files in the new commits)
Reviewed by minimax-m3 · Input: 71.2K · Output: 8.8K · Cached: 906.2K Review guidance: REVIEW.md from base branch |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Lands the parts of the v7 audit stack that never reached
working_branch: PR 3, 4, 5, 6 and 9. It also fixes two lock-contention races that CI exposed in that code. Afterwards, everyv7/bf-*branch tip is an ancestor ofworking_branch, so the stack branches can be deleted without losing any commits.Why they were missing
The stacked PRs were squash-merged into each other in an order that left most of the stack on intermediate branches:
working_branch?working_branchvia #63 (squashed when bf-01 sat at PR 2's squashd8842de)v7/bf-01, after #63 had already been squashedv7/bf-04, in the same second that #66 squashed bf-04 atff9dcf5(pre-PR 9) into bf-03v7/bf-06)Commits
a2297f1merge v7/bf-01 (PR 3, 4, 5).working_branch's tree equalsd8842de, an ancestor of bf-01's tip, so the result is bf-01's tree byte-for-byte. Its one textual conflict (the isolation sweep test) is PR 5's change against PR 1's squashed copy of the same lines.deab88erecord v7/bf-02, bf-03, bf-05 as merged (-s ours, tree unchanged). Their content is already in through the squash commits: bf-02's tree equalsd8842de, bf-03's tree equalscd5842f, and bf-05's patch is identical tob003c03's (samegit patch-id). Linking them also gives the next merge a closer merge base.1a45237merge v7/bf-06 (PR 6 + PR 9). Four conflicts, all taken from bf-06's side:api/v7/pinned.py: both sides made tokenless-preflightwarningsan empty tuple (edfaa57on bf-03,0bc432bon bf-06). bf-06's form drops redundant parentheses.retrieval/lexical_config.py,verification_v7.py: bf-06 exportsRETRIEVAL_PROJECTION_NAMESand builds_SUPPORTED_PROJECTIONSfrom it. The set is the same 12 names.tests/api_v7/test_production.py: bf-06's copy of the preflight no-subprocess test also waits for the brief's projection drains (f376ff3).CHANGELOG.md: folded the duplicate### Changedheading the merge produced.f949884record v7/bf-04 as merged (-s ours, tree unchanged). bf-04's tip is PR 9/12: Auto-enable installed profiles, fix the Windows launcher and OpenCode installer #68's squash, and its tree equals bf-09's tip, which step 3 brought in.Fixes for two races CI exposed on this PR
The first CI run failed two legs, each in a different test. Both failures came from the same cause: a concurrent writer held SQLite's write lock past the 5 s busy timeout, so an import finalize answered the retryable
DATABASE_IN_USE. The code involved is identical on bf-06's tip, whose CI happened not to hit the race. PR 9 is what lands it here.cf5bba5fix: let a finalize retried after DATABASE_IN_USE release its dead lease (ubuntu-latest / 3.12,test_process_portability). The failed attempt's cleanuprelease_import_finalizationhit the same lock. Its error was suppressed, so the attempt's lease stayed for its full 15 minutes, and every retry the error invites hitclaim_import_finalization's live-lease branch: a non-retryableTASK_REQUIRED. A lease whose release fails is now kept in-process and released before the session's next claim. A still-held lock fails that release asDATABASE_IN_USEagain, so the client keeps retrying. A live concurrent finalizer still getsTASK_REQUIRED, astest_concurrent_finalizer_is_lease_blocked_and_cannot_delete_winnerpins. New regression test:test_retry_after_a_contended_finalize_releases_its_abandoned_lease. It fails withTASK_REQUIREDwithout the fix.42f929ctest: retry DATABASE_IN_USE in the path-policy import round trip (windows-latest / 3.10,test_path_policy). The server answered correctly with the retryable code, but PR 5's round trip asserted success on the first call. It now retries within a bounded deadline, as PR 9's portability test does.How the result was checked
f7f024a..27d1682, applied withgit apply --3way) reproduces this head's tree exactly. The one exception is the CHANGELOG heading fold.cd5842f(run 35530809596) and bf-06f376ff3(run 35619920017).mainand everyv7/bf-*tip are ancestors of this head (git merge-base --is-ancestor).Tests run
Linux, Python 3.11,
pip install -e ".[dev,apps,graph]"(ruff 0.16.8, mypy 2.3.1):ruff check daem0nmcp/ tests/: all checks passedruff format --check daem0nmcp/ tests/: 475 files already formattedmypy daem0nmcp/api/v7 --ignore-missing-imports --follow-imports=silent: no issues in 46 files (also clean with--platform win32and--platform darwin)python scripts/v7_release_inventory.py --check: valid (75 tools, 10 resources, 572 atomic requirements)migrate-v7CLI, edit bridge/host, OpenCode installer,test_docs_hooks): 166 passed, 34 subtests passedapi_v7/test_production.py,test_verify_v7.py,api_v7/test_pinned_handlers.py, lexical/projection-growth tests): 138 passed, 24 subtests passedpytest tests/ --asyncio-mode=autoon the merge headf949884: 2913 passed, 120 skipped, 0 failed (1390 subtests passed) in 14m30s. The skips are the expected ones: optional extras not installed (tiktoken, models-local, e2b, numpy, rank_bm25), external services not configured (Valkey,E2B_API_KEY), and Windows-only tests.42f929c): 2914 passed, 120 skipped, 0 failed (1390 subtests passed) in 13m53s. The one extra pass is the new regression test, which fails withTASK_REQUIREDbeforecf5bba5and passes after.test_portable_projections,test_process_portabilityandapi_v7/test_operations: 54 passed.test_path_policy: 20 passed. ruff, format, mypy and the inventory are clean.Config and migration notes
These ride in with PR 6 and PR 9; the details are in
CHANGELOG.md:DAEM0NMCP_<PROFILE>_ENABLEDnow means "auto". To opt out, set=false.verify-v7until it is migrated again:migrate-v7 --rollback, thenmigrate-v7 --apply.migrate-v7 --rollbackrefuses withROLLBACK_WOULD_HIDE_WRITESwhen events were recorded after activation.--discard-v7-writesoverrides it.Known defects carried in, not introduced here
Reviewers flagged three defects on #65 (PR 3) as unfixed, and #65 was merged into bf-01 anyway. They are still present at this head. This PR moves the code unchanged, so they are left for a follow-up rather than mixed into the merge:
DATABASE_IN_USEenvelope ascompleted(api/v7/task_dispatcher.py:898). Idempotent resubmits are then served that cached failure (:1309-1312,:1063-1068) until the TTL prunes it.SpecializedProjectionBuildError("DATABASE_IN_USE")(graph, temporal, procedure, outcome) is not aProjectionBuildError.retrieval/operations.py:77-78therefore turns it intoPROJECTION_REBUILD_FAILED, and the caller gets a non-retryableCAPABILITY_DEGRADED(api/v7/operations.py:979)._lexical_catchup_pending_sync(api/v7/discovery_operations.py:2073-2081) matches any queued or runningretrieval.projection_rebuildjob, not only lexical ones. A graph or temporal rebuild therefore makes a missing entity retryable.Merge method
Merge with a merge commit, not a squash. PR 6's 17 commits (
1783c07..f376ff3) exist only onv7/bf-06, which never had a PR. A squash followed by deleting the branch would orphan them.🤖 Generated with Claude Code
https://claude.ai/code/session_01UBLLzyphecQABpqVorHunX