Skip to content

Commit e39746e

Browse files
gauravclaude
gaurav
and
claude
committed
Add a nodenorm skill so coding agents can use this API correctly
PR #403 fixed the reference problem — every endpoint now carries a real description and the response schema is documented. What an agent still lacks is judgement: when to reach for NodeNorm at all, which conflation flags to set, how to batch a whole column of identifiers, and the handful of behaviours that silently produce wrong answers rather than errors. This is that document. Every factual claim in it was measured against the live RENCI instance rather than written from memory, including the equivalent- identifier counts used to illustrate what conflation actually does. Three things it exists to prevent, all of which fail silently: - GET and POST disagree on the conflation defaults, so the same query returns 206 equivalent identifiers one way and 30 the other (#398). The rule taught is to set both flags on every request. - /get_setid takes `conflation` (singular) on GET and `conflations` (plural) in the POST body, defaults to no conflation unlike GET /get_normalized_nodes, and silently ignores the wrong name — returning HTTP 200 and an unconflated hash. Anyone comparing a gene against its protein this way concludes they are different concepts. - A conflated clique routinely holds several identifiers sharing one prefix (the DMD clique has three ENSEMBL entries: one gene, two proteins), so matching on prefix alone is a coin flip. The recipe uses individual_types and filters on type. Format is skills/nodenorm/SKILL.md with YAML frontmatter, which is what Claude Code actually discovers; the body is plain Markdown usable by any agent. The `description` names concrete CURIE prefixes, since that is what the model matches on when deciding whether the skill is relevant. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 9d2fa37 commit e39746e

1 file changed

Lines changed: 241 additions & 0 deletions

File tree

‎skills/nodenorm/SKILL.md‎

Lines changed: 241 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,241 @@
1+
---
2+
name: nodenorm
3+
description: Normalize biomedical identifiers (CURIEs) using the Translator NodeNorm API — find the preferred identifier for a concept, list its equivalent identifiers in other databases, and get its Biolink type. Use when working with CURIEs such as MESH:D014867, NCBIGene:1756, CHEBI:15377, UniProtKB:P11532 or MONDO:0005148, to connect an identifier to another data source, to normalize a whole column of identifiers, or to check whether two identifiers refer to the same concept.
4+
---
5+
6+
# Normalizing biomedical identifiers with NodeNorm
7+
8+
NodeNorm answers the question **"what is this identifier?"** Given a CURIE, it returns the preferred
9+
identifier for that concept, every equivalent identifier it knows about, and the concept's Biolink
10+
types.
11+
12+
- **Base URL:** `https://nodenormalization-sri.renci.org/`
13+
- **Machine-readable spec:** `GET /openapi.json` — the authority on parameter names and defaults.
14+
Prefer it over `/docs`, which is a JavaScript-rendered Swagger UI and near-useless to read.
15+
- The one endpoint that matters is `/get_normalized_nodes`. Everything else is supporting.
16+
17+
## Is this the right service?
18+
19+
| You have | Use |
20+
|---|---|
21+
| An identifier (`CHEBI:15377`, `NCBIGene:1756`) | **NodeNorm** — this skill |
22+
| A name or string (`"aspirin"`, `"type 2 diabetes"`) | **NameRes** (`https://name-resolution-sri.renci.org/`), then normalize the CURIE it returns |
23+
24+
If you have free text, resolve it to a CURIE with NameRes first. NodeNorm will not look up names.
25+
26+
## One identifier
27+
28+
```
29+
GET https://nodenormalization-sri.renci.org/get_normalized_nodes?curie=MESH:D014867
30+
```
31+
32+
Repeat `curie=` for a handful of identifiers. Response:
33+
34+
```json
35+
{
36+
"MESH:D014867": {
37+
"id": { "identifier": "CHEBI:15377", "label": "Water" },
38+
"equivalent_identifiers": [
39+
{ "identifier": "CHEBI:15377" },
40+
{ "identifier": "PUBCHEM.COMPOUND:962" },
41+
{ "identifier": "UNII:059QF0KO0R" }
42+
],
43+
"type": ["biolink:SmallMolecule", "biolink:MolecularEntity", "biolink:ChemicalEntity"],
44+
"information_content": 47.5
45+
}
46+
}
47+
```
48+
49+
### Reading the response
50+
51+
- **`id.identifier`** — the preferred identifier. Two CURIEs that normalize to the same
52+
`id.identifier` refer to the same concept. This is what you store, join on, and send downstream.
53+
- **`id.label`** — the preferred name. **Not necessarily the label of `id.identifier`**; for
54+
chemicals especially, the name may come from a different member of the clique.
55+
- **`equivalent_identifiers`** — every identifier for this concept, in the Biolink Model's preferred
56+
prefix order. **This is the field that answers "connect this to another database."** Note that
57+
**`label` is often absent** on individual entries, so read it with `.get("label")` rather than
58+
`["label"]`.
59+
- **`type`** — Biolink classes, most specific first.
60+
- **`information_content`** — 0.0 (broad concept) to 100.0 (very specific). Absent when unknown.
61+
62+
Optional flags: `description=true` adds descriptions (many identifiers have none),
63+
`individual_types=true` adds a `type` to each equivalent identifier, `include_taxa=true` (the
64+
default) adds a `taxa` array of NCBITaxon CURIEs both at the top level and on the individual
65+
entries that have one.
66+
67+
Use GET for one or two identifiers you are inspecting by hand. For anything programmatic use POST —
68+
a long `curie=` list will eventually exceed the server's URL length limit, and POST has no such
69+
ceiling.
70+
71+
## Many identifiers
72+
73+
Use POST. It is fast — 1000 identifiers in roughly 2 seconds — so **never loop over GET requests**.
74+
75+
```
76+
POST https://nodenormalization-sri.renci.org/get_normalized_nodes
77+
Content-Type: application/json
78+
79+
{
80+
"curies": ["MESH:D014867", "NCBIGene:1756", "RUBBISH:1234"],
81+
"conflate": true,
82+
"drug_chemical_conflate": false
83+
}
84+
```
85+
86+
The response is keyed by input CURIE, with the same value shape as the GET method.
87+
88+
- **Deduplicate before sending.** The response is a dictionary, so duplicates buy you nothing.
89+
- **Chunk at around 1000 per request.** Larger batches work but tie up a shared public service.
90+
- **An identifier that cannot be normalized comes back as `null`** — the key is present with a null
91+
value, it is *not* omitted. Check for null values, not for missing keys.
92+
- **Response keys echo your input exactly**, including any surrounding whitespace. Look results up
93+
by the exact string you sent, not a cleaned-up version.
94+
95+
## Conflation — decide this deliberately
96+
97+
Conflation merges concepts that are not strictly identical but are often treated as one. It changes
98+
both the preferred identifier and the size of `equivalent_identifiers`, sometimes dramatically.
99+
100+
| Flag | Merges | Preferred identifier becomes |
101+
|---|---|---|
102+
| `conflate` | A gene with the protein it encodes | the **gene** |
103+
| `drug_chemical_conflate` | A drug with its active ingredient | the **active ingredient** |
104+
105+
Real numbers from the live service:
106+
107+
- `NCBIGene:1756` — 5 equivalent identifiers unconflated, **22** with `conflate=true`
108+
(5 `biolink:Gene` + 17 `biolink:Protein`).
109+
- `MESH:D014867` (water) — 30 equivalent identifiers unconflated, **206** with
110+
`drug_chemical_conflate=true`.
111+
112+
### Always set both flags explicitly
113+
114+
**The defaults differ between GET and POST.** The same query returns different results:
115+
116+
```
117+
GET /get_normalized_nodes?curie=MESH:D014867 → 206 equivalent identifiers
118+
POST /get_normalized_nodes {"curies":["MESH:D014867"]} → 30 equivalent identifiers
119+
```
120+
121+
GET defaults both flags to true; the POST body defaults `drug_chemical_conflate` to false. This is a
122+
known bug ([NodeNormalization#398](https://github.com/NCATSTranslator/NodeNormalization/issues/398)).
123+
Until it is fixed, **set `conflate` and `drug_chemical_conflate` explicitly on every request** and
124+
you will not be caught by it.
125+
126+
### Which setting do you want?
127+
128+
**Conflation changes which question you are asking, not just how many rows come back.**
129+
`conflate=false` asks *"what is this protein?"*; `conflate=true` asks *"what gene is this protein a
130+
product of?"* Both are valid; picking the wrong one gives you a confident, error-free, useless
131+
answer.
132+
133+
- **Matching an existing knowledge graph** (Translator, ROBOKOP) — use the conflation that graph was
134+
built with. If you do not know, try both and see which one's identifiers appear in the target.
135+
- **Crossing between a gene and a protein** — you need `conflate=true`. A UniProtKB accession
136+
unconflated has no gene identifiers in its clique at all, so a gene lookup will silently come back
137+
empty rather than failing.
138+
- **Asking about one specific molecule or one specific protein as itself** — turn conflation off, so
139+
you get that concept rather than a merged one.
140+
- **Linking loosely across sources** — turn it on for the widest set of equivalents.
141+
142+
Conflated results are a **single flat list** — all the members of the first clique, then the second,
143+
with no marker between them. Use `individual_types=true` to tell gene from protein.
144+
145+
## Recipes
146+
147+
**Connect an identifier to another database.** Normalize it with `individual_types=true`, then look
148+
in `equivalent_identifiers` for the prefix you want — **and filter on `type`, never on prefix
149+
alone.**
150+
151+
```
152+
GET /get_normalized_nodes?curie=UniProtKB:P11532&conflate=true&drug_chemical_conflate=false&individual_types=true
153+
→ id.identifier = NCBIGene:1756 ("DMD"), plus HGNC:2928, OMIM:300377, and three ENSEMBL entries:
154+
ENSEMBL:ENSG00000198947 biolink:Gene ← the Ensembl *gene*
155+
ENSEMBL:ENSP00000288447 biolink:Protein
156+
ENSEMBL:ENSP00000288447.4 biolink:Protein
157+
```
158+
159+
A conflated clique routinely contains **several identifiers sharing one prefix**, because it holds
160+
both halves of a gene/protein or drug/chemical pair — and they may be versioned duplicates too.
161+
Taking the first `ENSEMBL:` you find gets the right answer here only by luck of ordering. Decide
162+
which one you want by its `type`.
163+
164+
**Normalize a column in a file.** Read the column, deduplicate, POST in chunks of 1000, build a
165+
`{input → id.identifier}` map, and report the nulls separately — they are usually retired
166+
identifiers, typos, or prefixes NodeNorm does not cover.
167+
168+
**Are these two identifiers the same concept?** Normalize both with identical conflation settings
169+
and compare `id.identifier`. This is the reliable way; prefer it. See `/get_setid` below if you want
170+
a single comparable hash for a whole set.
171+
172+
**What kind of thing is this?** Read `type[0]` — the most specific Biolink class.
173+
174+
## `/get_setid` — a stable hash for a set of identifiers
175+
176+
Normalizes a set of CURIEs, deduplicates, sorts, and hashes the result. Two sets containing the same
177+
concepts produce the same hash, whatever order or spelling of identifiers you started from. Useful
178+
for comparing or caching sets of concepts without storing the members.
179+
180+
```
181+
GET /get_setid?curie=MESH:D014867&curie=NCBIGene:1756&conflation=GeneProtein&conflation=DrugChemical
182+
```
183+
184+
**This endpoint does not take the same conflation parameters as `/get_normalized_nodes`, and it will
185+
not tell you when you get them wrong.**
186+
187+
- The parameter is **`conflation`** (singular), repeated once per conflation, with the values
188+
`GeneProtein` and `DrugChemical` — not the `conflate` / `drug_chemical_conflate` booleans used
189+
everywhere else.
190+
- **It defaults to no conflation at all**, the opposite of GET `/get_normalized_nodes`.
191+
- **Wrong parameter names are silently ignored.** `conflations=GeneProtein` (plural) and
192+
`conflate=true` both return HTTP 200 with an unconflated hash and no warning:
193+
194+
```
195+
/get_setid?curie=UniProtKB:P11532 → uuid:e9a5e65f… (UniProtKB:P11532)
196+
/get_setid?curie=UniProtKB:P11532&conflation=GeneProtein → uuid:e020b733… (NCBIGene:1756)
197+
/get_setid?curie=UniProtKB:P11532&conflations=GeneProtein → uuid:e9a5e65f… silently unconflated
198+
```
199+
200+
Always check the `conflations` field echoed back in the response — if it is `[]` when you asked for
201+
a conflation, your parameter name was wrong.
202+
203+
One request is **one set**, not a batch. To hash several sets in one call, POST a list of
204+
`{"curies": [...], "conflations": [...]}` objects — note that the POST body field *is* `conflations`
205+
(plural), differing from the GET query parameter.
206+
207+
The response also carries `normalized_curies` (what the hash was built from — check this if a result
208+
surprises you), `normalized_string`, and `error`.
209+
210+
## Gotchas
211+
212+
- **Lookups are case-insensitive and whitespace-trimmed.** `" mesh:d014867 "` resolves fine. But
213+
the response key is still the string you sent.
214+
- **A missing identifier may be missing by design.** Babel keeps only identifiers whose prefix is
215+
valid for the concept's Biolink type and drops the rest, so a CURIE you expected can be
216+
legitimately absent from a clique.
217+
- **The data is a fixed snapshot**, not a live database. `GET /status` reports `babel_version` (a
218+
build name like `2025sep1`). Record it if you need reproducible results — a later build may
219+
normalize the same identifier differently.
220+
- **Information content from builds before `2025sep1` is unreliable** (it was compared as a string,
221+
so some values are wrongly 100). Check `/status` first.
222+
- **This is a shared public service.** Batch with POST; do not hammer it with parallel requests.
223+
224+
## Other endpoints
225+
226+
| Endpoint | Purpose |
227+
|---|---|
228+
| `GET /status` | Which Babel build and Biolink Model version this instance serves |
229+
| `GET /get_allowed_conflations` | The conflation names this instance supports |
230+
| `GET/POST /get_setid` | A stable hash for a set of CURIEs — see above, its parameters differ |
231+
| `GET /get_semantic_types` | Every Biolink type present in this instance |
232+
| `GET /get_curie_prefixes` | CURIE prefix counts per Biolink type (approximate) |
233+
| `POST /query`, `/asyncquery` | Normalize a whole TRAPI message — **deprecated**, do not use |
234+
235+
## Why cliques look the way they do
236+
237+
The identifiers, preferred names, types and information content all come from
238+
[Babel](https://github.com/NCATSTranslator/Babel), which is what decides that two identifiers are
239+
equivalent. If a clique looks wrong — two concepts merged, or one concept split in two — that is a
240+
Babel issue, not a NodeNorm one. See
241+
[Where NodeNorm's data comes from](https://github.com/NCATSTranslator/NodeNormalization/blob/master/documentation/Babel.md).

0 commit comments

Comments
 (0)