Use Babel's preferred_name for clique labels

[gaurav]

Context

NodeNorm picks a preferred label for every clique. What began as "first label in the list" has grown into a ~90-line algorithm in create_node() (conflation label-hunting, boost-prefix sorting, CHEMBL/length filtering) that tries to replicate the label Babel would pick. Babel now computes this itself and emits a clique-level preferred_name on every compendium line, so per #299 NodeNorm should just use that field.

The catch: preferred_name is clique-level and was stored nowhere in Redis — the loader persists per-CURIE l/d/t inside the id_to_eqids_db blob, and clique-level type/ic in their own DBs, but not preferred_name.

This is adjacent to #306 (consolidate all clique-level properties into one DB). Rather than the full overhaul, this takes the first step: repurpose the existing info_content_db (db 5) into a clique-property store whose value is a JSON dict. For 2.5 it holds preferred_name and ic; later (#306) type etc. move in. No new database is provisioned — db 5's values just get richer.

Not taxa, though: the clique-level taxa list is just the union of the per-identifier t fields, which the loader already persists inside the id_to_eqids_db blob. Copying it into db 5 would be redundant, so taxa are read from id_to_eqids_db instead. This is recorded in a new node_normalizer/loader/CLAUDE.md so future edits don't re-add it.

Rebased onto the loader overhaul. This originally stacked on #384; load-faster (#391) has since been merged into this branch, so it now targets main directly and includes the reorganized node_normalizer/loader/ package.

What changed (by commit)

1. Store preferred_name in Redis — load_compendium writes {"preferred_name", "ic"} JSON to db 5 for every clique (previously only IC-bearing cliques got a key). Config key name kept to avoid churn; rename to clique_props_db deferred to Redesign Redis databases to support arbitrary properties on identifiers and cliques #306. Integration test asserts the new format.
2. Use it, keep the old algorithm as a frozen fallback — create_node uses the stored preferred_name (looked up by the primary subclique's canonical id eids[0]['i'], so conflated cliques get the gene-first name). When absent — e.g. querying an older DB load — it falls through to the legacy algorithm, now marked do-not-modify and slated for deletion once every deployment loads preferred_name (Babel's replacement: babel_utils.py#L581). A new _clique_props() helper reads db 5, tolerating both the new JSON and the old bare-float format.
3. Read-path unit tests + tidy-ups — added test_normalizer.py unit tests covering create_node's new branch (see Verification), aligned the new db-5 read on encoding='utf8' with its neighbours, and fixed an "algorithem" comment typo in the frozen fallback (Copilot review).

Verification

• pytest -m "not integration" → 29 passed (query fixtures have no preferred_name, so they exercise the fallback unchanged — no regression). New test_normalizer.py unit tests exercise the read path directly (no Redis — only the conflation fallback touches it):
  • stored preferred_name is used verbatim for the label (and bypasses the fallback's CHEMBL/length filters)
  • an empty "" preferred_name falls through to the legacy algorithm
  • _clique_props() tolerates the new JSON dict and the legacy bare-float/int formats
  • under conflation, get_normalized_nodes returns the leading sub-clique's identifier and preferred_name (mock-Redis: a gene/protein clique resolves to the gene's name, not the queried protein's)
• pytest -m integration (real Redis testcontainer) → passes. The loader test covers all three clique properties end-to-end:
  • preferred_name — empty "" (Cell) and populated (Disease) via db 5
  • ic — string "100" (Cell) and float 100.0 (Disease) via db 5, tolerating both source formats
  • taxa — a human-taxon clique (PhenotypicFeature) whose per-identifier t field round-trips via id_to_eqids_db, confirming taxa are retrievable without a db-5 copy
• Gene/protein conflation, end-to-end on real Babel data (the CDK2 clique: NCBIGene:1017 conflated with four UniProtKB proteins, gene-first) — two integration tests over the same freshly-loaded Redis:
  • loading — every clique member maps to the gene-first list in gene_protein_db, and the gene's preferred_name "CDK2" lands in db 5
  • querying — the real frontend RedisConnectionFactory is attached to the loaded Redis and get_normalized_nodes is driven directly: any protein in the clique normalizes to NCBIGene:1017 / "CDK2" under conflation, and keeps its own identity/name without it

Scope / follow-ups

• Renaming info_content_db → clique_props_db and migrating type (etc.) into it: deferred to Redesign Redis databases to support arbitrary properties on identifiers and cliques #306. The JSON value makes each a one-liner on both sides. taxa is deliberately excluded — it stays in id_to_eqids_db (see above).
• ic type is inconsistent in the source data — Babel emits it as a JSON string in some compendia ("100") and a number in others (100.0), and the loader passes it through untyped, so db 5 stores whichever came in. _clique_props() tolerates both. Normalizing to float on write is a candidate cleanup if any consumer starts doing math on it.
• preferred_name_boost_prefixes / demote_labels_longer_than config stays (used only by the frozen fallback); delete alongside the fallback.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR shifts NodeNormalization’s clique labeling to use Babel’s clique-level preferred_name field, storing it in Redis (db 5) alongside ic as a JSON “clique properties” document. It also preserves the legacy preferred-label algorithm as a fallback for older Redis loads that lack the new db-5 JSON format.

Changes:

• Loader now writes a clique-properties JSON value ({"preferred_name", "ic"}) to info_content_db (db 5) for every clique.
• Normalizer reads preferred_name from db 5 and uses it in create_node() when present, falling back to the legacy label-selection algorithm otherwise.
• Integration test expanded to validate preferred_name + ic JSON round-tripping, and adds a taxa round-trip assertion via id_to_eqids_db.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
node_normalizer/loader/loader.py	Writes clique-level properties JSON (preferred_name, optional ic) into db 5 for every clique.
node_normalizer/normalizer.py	Adds _clique_props() parser; reads preferred_name from db 5 and threads it into create_node() label selection.
tests/test_loader_integration.py	Extends integration load coverage to assert db-5 JSON clique props and taxa retrieval via id_to_eqids_db.
tests/resources/Disease.txt	Adds a test compendium line that includes preferred_name + numeric ic.
tests/resources/PhenotypicFeature.txt	Adds a test compendium line that includes preferred_name + per-identifier t taxa.
node_normalizer/loader/CLAUDE.md	Documents the intentional choice not to store clique-level taxa in db 5.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.