Two served event titles are mojibake: UTF-8 arrows decoded as CP1252, invisible to every gate we had

[PipFoweraker]

Found by writing the missing producer for all_events.json (#52). Filing separately because it crosses repos and because the mechanism matters more than the two records.

The finding

Two records exist in three mutually inconsistent forms, and all three of this repo's documented Windows text landmines are visible in that one pair:

Where	Title fragment	What happened
data/raw/events/institutional_decay_events.json	UK AI Safety Institute <U+2192> ...	correct
data/raw/events/historical_events.json	UK AI Safety Institute ? ...	the ? fallback -- the exact shredding CLAUDE.md warns fix_ascii.py would do to every arrow
data/serveable/api/timeline_events/all_events.json (served)	UK AI Safety Institute <U+00E2><U+2020><U+2019> ...	mojibake

Affected ids: uk_ai_safety_to_security_2025, us_aisi_to_caisi_2025.

Verified rather than inferred. The served string round-trips exactly:

served.encode("cp1252").decode("utf-8") == correct_title   # True, both records

U+2192 was encoded UTF-8 (E2 86 92) and decoded as CP1252, yielding U+00E2, U+2020, U+2019. That is the transcoding species from coordination#10, and its defining property is that it injects plausible printable characters. Every structural check passes. The control-character scan passes -- all three codepoints are above 32. The ASCII gate passes on the file, because json.dump(ensure_ascii=True) escapes them to \uXXXX and the bytes on disk are ASCII.

A well-formed file that is wrong.

Correcting my own overstatement before it propagates

The commit message for the producer says this "reached 1,194 public pages". That is wrong and I am correcting it here rather than letting it be quoted back. The file is read by the sync that writes ~1,194 pages; the corruption is in two records, so two event pages carry it, plus any index or search surface that renders those titles.

Scope, measured: 2 of 1,194 records. No other record in the file carries a mojibake signature (checked for â€, â, â† across all 1,194).

Fixed here, in #67

Repair is neither damaged form. Per CLAUDE.md: an explicit substitution map that errors on unmapped characters, never a ? fallback. U+2192 becomes ->.

1,194 ids identical, two titles changed, no other record moves.

Why nothing caught it, which is the part worth keeping

A file with no producer has nothing to be compared against. --check on every other collection asserts the committed output equals a fresh build; this collection had no build, so there was no second opinion to disagree with. check_invariants.py gained timeline_events coverage on 2026-08-06 and asserts counts, which were correct throughout -- the record count of a corrupted record is still 1.

It is also a clean instance of the two-clause check rule: the observation has to come from outside the system under test. Every guard we had was reading the served file and asking whether it was well-formed. It was. Only comparing it to its own sources could tell you it was wrong, and until today nothing could produce that comparison.

Downstream, needing action in other repos

• pdoom1-website -- two event pages render the corrupted title. They regenerate from the daily sync, so merging Install the three lessons: dereference citations, publish the taxonomy, and give all_events.json a producer #67 fixes them on the next run provided those two ids are in the regenerating 1,194 and not among the ~1,000 orphans. Worth confirming, not assuming.
• pdoom1 -- godot/data/historical_events.json is a 1,194-record copy taken from this file and carries the same two corrupted titles. It does not regenerate; someone has to re-sync or patch.

Cheap detector, since the expensive one now exists

coordination#10 already gives the signature grep: â€, â,  . That is worth running across every repo's data, not just this one -- the corruption class is a Windows PowerShell 5.1 property, not a pdoom-data property, and four repos share the workstation.

Refs #52, #67, coordination#10, coordination#31.