feat(canonical-migration): --remap-existing to re-evaluate already-migrated edges (#45) - #208
Merged
Merged
Conversation
…grated edges (#45) The first-migration UPDATE is one-shot by design: its `AND NOT (properties ? 'raw_relation_type')` guard makes a second run a no-op, because every migrated edge now carries `raw_relation_type`. That's correct for the original rollout but blocks the #45 use case — after the predicate normalizer (mempalace.kg_predicate_norm) is improved, the bulk of the `other` bucket should be re-evaluated, and those edges are exactly the already-migrated ones a naive re-run skips. `--remap-existing` (default OFF) switches both halves: - Frequency read uses `coalesce(r.raw_relation_type, r.relation_type)` so the mapper re-evaluates the ORIGINAL predicate, not the prior (possibly `other`) canonical. - The apply UPDATE keys the join on the same coalesced original, DROPS the first-migration guard, PRESERVES the original (coalesce keeps an existing raw_relation_type, only backfilling it on never-migrated edges), and adds `relation_type IS DISTINCT FROM m.canonical` so only edges whose canonical actually changes are touched (no MVCC churn on no-ops). Default behavior is byte-for-byte unchanged. Same backup + paused-worker gating. Prerequisite for the #45 prod re-map (mempalace#336 improved the normalizer; this lets the existing 1.92M-edge graph reflect it without an LLM re-call). 6 new tests: remap cypher reads the original, apply drops the guard + keys on coalesced original + IS DISTINCT FROM, default keeps the guard, and a back-compat getattr-defaults-false case. 18/18 pass; ruff clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The first-migration UPDATE in
scripts/canonical_migration.pyis one-shot by design: itsAND NOT (properties ? 'raw_relation_type')guard makes a second run a no-op (every migrated edge already carriesraw_relation_type). That's correct for the original rollout but blocks the #45 use case — after the predicate normalizer (mempalace.kg_predicate_norm, improved in mempalace#336) is upgraded, the bulk of theotherbucket should be re-evaluated, and those edges are exactly the already-migrated ones a naive re-run skips.On the production graph, 55% of 1.92M RELATION edges sit in
other, each retaining its original predicate inraw_relation_type. The improved normalizer reclaims ~half of them — but only if the migration re-reads the original predicate and is allowed to overwrite an existing canonical.What
--remap-existing(default OFF) switches both halves of the migration:coalesce(r.raw_relation_type, r.relation_type)so the mapper re-evaluates the original predicate, not the prior (possiblyother) canonical.coalescekeeps an existingraw_relation_type, only backfilling it on never-migrated edges), and addsrelation_type IS DISTINCT FROM m.canonicalso only edges whose canonical actually changes are touched (no MVCC churn on no-ops).Default behavior is byte-for-byte unchanged without the flag. Same backup + paused-worker gating applies (the
--apply+--i-have-a-backupguards are untouched).Prerequisite chain
This is the mechanism for the #45 prod re-map. mempalace#336 (merged) improved the normalizer; this PR lets the existing graph reflect it without an LLM re-call (pure deterministic relabel of
raw_relation_type). The actual prod run remains gated on a backup + explicit approval.Tests
6 new tests in
TestRemapExisting: remap cypher reads the original, apply drops the guard + keys on the coalesced original + usesIS DISTINCT FROM, default keeps the guard, plus a back-compatgetattr(..., False)case for callers built before #45. 18/18 pass; ruff clean; module docstring updated with the new section + exact SQL.🤖 Generated with Claude Code