fix(watch): require deletion evidence before evicting nodes for a missing source#1795
fix(watch): require deletion evidence before evicting nodes for a missing source#1795CJNA wants to merge 1 commit into
Conversation
…sing source The rebuild reconciliation treated 'source identity not in the collected corpus' as 'source deleted' and evicted its nodes. But absence from the corpus is ambiguous: it also happens when the file still exists and merely stopped being collected — ignore rules or filters changed (e.g. the scanner newly honoring a .gitignore whose patterns cover wanted docs). On a real 27k-node graph, upgrading into gitignore-honoring scan semantics silently evicted all 655 nodes of a gitignored-but-present docs directory on the first incremental rebuild; recovery was only possible because a manual pre-upgrade backup existed. Make eviction fail-closed: before treating a corpus-absent identity as deleted, require the positive evidence — the file actually gone from disk. Excluded-but-alive files keep their nodes and a loud log line says how many were kept and why. True deletions and renames evict exactly as before (the old path is gone from disk in both cases), and a full re-extraction still purges deliberately excluded sources via the existing AST ownership rule. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Real-world verification on the graph from the original incident (26k nodes): re-disabled the All 655 nodes that were silently evicted under 0.9.13 stock survived, graph byte-level intact after restoring the ignore rules. Same rebuild with the file genuinely deleted still evicts (covered by the second test). |
#1795) _reconcile_existing_graph treated "source identity absent from the collected corpus" as deletion and evicted its nodes/edges/hyperedges. But corpus absence is ambiguous: it's also what you see when a file still exists and merely stopped being collected (ignore rules or filters changed). Upgrading into the merged- .gitignore scan semantics (#1363) mass-evicted 655 nodes from a deliberately- built, .gitignore'd docs dir whose files were present the whole time — reported as a successful rebuild. Fail-closed: before evicting a corpus-absent identity, require Path(identity) .exists() is False (identity is an absolute path). Alive-but-excluded sources are preserved (nodes, edges, hyperedges) and a loud line reports how many were kept and why. True deletions and renames still evict (old path gone from disk); a full extract --force still purges deliberate exclusions via the AST ownership rule. Existence is memoized (one stat per file that left the corpus). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Merged on v8, authored to you. Verified end-to-end: an excluded-but-alive file (added to .graphifyignore, kept on disk) survives an incremental rebuild with the |
Opening 0.9.13 accidentally renamed the 0.9.12 header instead of inserting a new section above it, merging already-released 0.9.12 fixes under 0.9.13. Split them back: 0.9.13 now lists only #1756/#1768/#1781/#1784/#1785/#1795/#1796/#1797/#1799; 0.9.12 (2026-07-10) holds the fixes shipped in that release. Date 0.9.13. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Problem
_reconcile_existing_graphtreats "source identity not in the collected corpus" as "source deleted" and evicts its nodes, edges, and hyperedges. But corpus absence is an ambiguous signal — it is also what you observe when the file still exists and merely stopped being collected because ignore rules or filters changed.Real-world incident (27k-node graph): upgrading 0.8.x → 0.9.x brought the merged-
.gitignorescan semantics (#1363). The repo's.gitignoreexcludes a local-only docs directory that had been deliberately built into the graph. On the first incremental rebuild after the upgrade, all 655 nodes from that directory — files present on disk the whole time — were silently evicted as "stale sources". The rebuild log line reported success; recovery was only possible because a manual pre-upgradegraph.jsonbackup existed.This is a vacuous-pass failure: the eviction logic reads absence of the file from a list as positive evidence of deletion.
Fix
Fail-closed eviction: before treating a corpus-absent identity as deleted, require the actual deletion evidence —
Path(identity).exists()returning False. If the file is alive, its records are preserved and one loud log line reports how many nodes from how many files were kept and why:Behavior deliberately unchanged everywhere else:
graphify extract --forceremains the intentional cleanup path. (Extending deletion-evidence semantics to full rebuilds is a larger design question, left out of scope.)Existence checks are memoized per identity, so the cost is one
statper file that left the corpus — zero for the common case.Tests
test_rebuild_code_preserves_nodes_from_excluded_but_alive_file— end-to-end incident reproduction: build a corpus with a markdown doc, then add an ignore rule covering it (file stays on disk), incremental rebuild → nodes preserved + "fail-closed: kept" message asserted.test_rebuild_code_still_evicts_when_excluded_file_is_also_deleted— the guard must not weaken true-deletion eviction.tests/test_watch.py+tests/test_build.py+tests/test_hooks.py: 151 passed, 2 skipped (all pre-existing eviction/rename/prune tests green).ruff checkclean.Sibling of #1785 (same graph, same debugging session).
🤖 Generated with Claude Code