feat: result-language preference — return Overture name variants (#410)

[chuofringer]

Summary

Closes #410. Adds a result-language preference: geocode/geocode_detailed, resolve_place, and place_details can now return Overture's language-tagged names.common variant instead of a row's primary name.

• lang field on the stored preferences() document (2-3 lowercase letters, validated by shape the way mode is validated by membership).
• Per-call lang override on geocode, resolve_place, and place_details — the four name-heavy answer tools named in the issue. Per-call wins over the stored preference.
• When a variant exists for the matched row and language: name becomes the variant, name_primary is added only when it differs from the primary. No variant, or no lang at all → byte-identical to the pre-Result-language preference: return Overture name variants in the user's language #410 answer, no note spam.
• Never transliterates or invents — a variant only ever comes from Overture's own names.common map.

North-star demand: pelias/pelias#979 (structured search for cities in different language variants), pelias/pelias#967 (stale romanization). Competitive: Nominatim's accept-language.

Investigation findings (asked for in the task)

• Where language tags already lived: geocode.py's Search Overture's alternate names: geocode should match names.common, not just names.primary #214/Typo-tolerant + multilingual POI name search: extend the #214/#215 tiers from divisions to places #373 alt-name machinery already reads names.common, but both existing tables discard the language key — _materialize_alt_names_table groups by (id, folded-spelling) across every language on purpose (search-only, dedup by spelling). There was no existing table that answered "does this id have a variant for this specific language."
• Fixture state: tests/fixtures/divisions.parquet already carried real names.common data from the Search Overture's alternate names: geocode should match names.common, not just names.primary #214 exonym corpus (München→{en: Munich, ...}, Tokyo, Moscow, Vienna, Bratislava) — no fixture change needed there, only new query-layer plumbing. tests/fixtures/places.parquet had no names.common column at all (confirmed via DESCRIBE; also documented in test_poi_name_tiers.py's docstring). Both fixture-builder scripts (build_fixture.py, build_geocode_fixture.py) are fully local/synthetic (no live S3 scan) — extending them was the cheap, honest path, so I extended build_fixture.py to write a common MAP column (empty on every existing row, so no prior test's counts move) plus one new, deliberately isolated fixture row ("Kaffeehaus Wien" at an unused coordinate) carrying real variants for the place_details tests.

Query-cost impact

• Divisions (geocode/geocode_detailed, and resolve_place's division-kind candidates): a new small local lang_names.parquet table, materialized alongside the existing Search Overture's alternate names: geocode should match names.common, not just names.primary #214 alt-name table from the same names.common scan — one more one-time build pass per dataset materialization, not a per-request cost. At request time, applying lang is one indexed join keyed by the page of result ids (_lang_variants_for), never a query per row, and it's skipped entirely when lang isn't requested.
• Places (place_details): piggybacks the variant onto the same single-row SQL query already reading names — one extra selected column (names.common[$lang]), not a second scan. Falls back to no-variant (never fails the call) via the same try/except convention Typo-tolerant + multilingual POI name search: extend the #214/#215 tiers from divisions to places #373's alt-name tier already uses for this optional, unprobeable nested field.

Deviations from the issue's literal scope

• find_places rows keep primary names (the issue itself draws this line: "find_places rows keep primary names this round to bound scope"). Since resolve_place's place-kind candidates are sourced from find_places internally, they inherit that same scope line and are unaffected by lang — only resolve_place's division-kind candidates (via its internal geocode() call) get the swap. Documented in both the code and the CHANGELOG.
• geocode/geocode_detailed's own places-fallback rows (type: "place", found via _query_places_fallback when no division matches) are also left unenriched this round, for the same reason and to keep the change bounded — a places-side names.common lookup there would need its own query-cost design, which felt like scope creep beyond the issue's "M" effort sizing. Covered by test_places_fallback_rows_are_unaffected_by_lang.

Test plan

• uv run pytest -q — 2402 passed (2368 baseline + 34 new, tests/test_lang_preference.py), 15 deselected (@live)
• uv run ruff check . — clean
• Preferences round-trip incl. validation (junk rejected, case/whitespace normalized, 2-3 letter shape)
• Per-call override beats stored preference (all three tools)
• Variant swap + name_primary presence rule (present only when it differs)
• No-variant passthrough (byte-identical, no name_primary, no note)
• No lang configured at all → byte-identical answers (explicit guardrail tests)
• resolve_place's internal resolve-cache is keyed on lang too, so a lang="en" call can't leak into a later unlangauged call
• Offline only — no @live tests added
• docs/REFERENCE.md (preferences tool/resource rows + profile section), CHANGELOG.md ### Added, resources.py's resource description, and the generated schema docs (docs/benchmarks.md, docs/benchmarks-vs.md via token_efficiency.py --write / competitor_comparison.py --write) all regenerated/updated

House-style checklist

• Focused, one change (Result-language preference: return Overture name variants in the user's language #410)
• CI-equivalent green locally: ruff + full offline suite
• References Result-language preference: return Overture name variants in the user's language #410
• Response-shape change called out: name/name_primary on geocode, resolve_place, place_details rows — additive only (new optional field, existing name field unchanged in shape, byte-identical when lang is omitted)

🤖 Generated with Claude Code

[chuofringer]

Lead review (loop)

Re-ran locally: 2402 passed (+34), ruff clean.

Verified in review:

• The two-table split is the right design: the Search Overture's alternate names: geocode should match names.common, not just names.primary #214 alt-name table folds spellings for searching and discards language; the new lang_names.parquet keeps (id, lang) for serving — same source column, different query patterns, neither table carries dead weight. Materialization is one-time alongside the existing pass; per-request cost is a single indexed join on the result page, skipped entirely when no lang is in play.
• lang validation is a shape check (2-3 lowercase letters), correctly NOT a closed-list membership check — Overture's names.common keys aren't a fixed registry.
• name_primary only-when-different, no-variant passthrough, per-call-beats-preference, and no-lang byte-identical are all tested.
• The honest scope gaps (find_places rows, geocode's places-fallback rows) are pinned by explicit tests rather than left silent — exactly right.
• Fixture extension went through scripts/build_fixture.py (fully local, no S3), with existing-row counts untouched; divisions fixture already carried real Search Overture's alternate names: geocode should match names.common, not just names.primary #214 exonyms.
• Table build failure degrades to primary-names-only with one log line, mirroring the alt-name table's failure posture.

Acceptance criteria of #410 met. Ready for owner merge (loop does not merge).