📊 WYSK edits for incomes / Gini / Palma MDIMs (draft — no changes yet) by JoeHasell · Pull Request #6540 · owid/etl

JoeHasell · 2026-07-28T10:10:56Z

Written by Claude Fable 5 — Joe at the wheel.

Draft working PR for editing the "What you should know about this data" texts of the incomes, Gini, and Palma ratio MDIMs (wb/latest/incomes_pip, wb/latest/gini_pip, wb/latest/palma_ratio_pip), following the rules-doc workflow from #6474.

It intentionally contains no metadata changes yet — it exists so its staging server provides a clean, current baseline before editing begins. Proposed text changes will be pushed here as they're agreed in the shared rules document, and can be reviewed view-by-view with the Metadata Diff tool on this branch's staging server:

http://staging-site-incomes-inequality-wysk/etl/wizard/metadata-diff

Note: the diff currently shown against master for this PR is the Metadata Diff tool and skill themselves, carried from #6474 so the review tooling is available on this branch. Those files will drop out of this PR once #6474 merges; the metadata changes will be the real payload here.

Generated by Claude Code

Temporary test: prepend 'JOE TEST!!' to the first WYSK bullet of the 'Number in poverty, $4.20 a day, no breaks' view only, via a dimension-filtered common_views metadata override. To be reverted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EX3ywf7wME88SEttDZ35jP

New standalone Wizard app to review how a branch changes the metadata texts end users see on MDIMs, view by view, against production: - Blast radius: horizontal tree of all views following the MDIM's control order, colored by whether any view underneath changed, with hover previews of the diff, collapse/expand, and a changed-only filter. - View diff: side-by-side production/staging comparison of changed texts with word-level highlights, navigable via the MDIM's controls, with links to the view in the prod admin and on this staging server. Unlike the config diff in chart-diff, this compares the rendered texts (indicator metadata merged with view-level overrides, replicating the site's lodash merge), so it also catches changes coming from garden step templates. Read-only; separate app so chart-diff is untouched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EX3ywf7wME88SEttDZ35jP

…f links - The site's merge (Util.ts) overwrites arrays completely instead of merging element-wise, so a view-level description_key override replaces the whole bullet list. The diff engine now replicates that — the previous element-wise assumption made the staging side of the diff show bullets that the real page had actually dropped. - Update the poverty test override accordingly: it now carries all five bullets (first one prefixed with the test marker), so only the first bullet changes on the rendered page, as intended. - Blast radius: the component now resizes its own iframe to fit the content (the fixed-height container squished the tree once unchanged views were shown), scrolling internally past 4000px. - Leaves are real links (target=_top) to the View diff page instead of JS navigation, with a visible link arrow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EX3ywf7wME88SEttDZ35jP

- Plain click on a leaf was silently blocked: the component sandbox only allows user-initiated popups, not top navigation. Links now open in a new tab (which is also the nicer behavior for scanning views). - Move the usage hint inside the component; Streamlit-rendered text below the self-resizing iframe overlapped the tree. Also set the iframe height attribute so Streamlit's layout tracks the resize. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EX3ywf7wME88SEttDZ35jP

Delete 'Many people, today and in the past, have no monetary income.' from the description_key_not_market_income bullet, in both places the text lives: the garden template (flows to all indicator metadata and auto-generated MDIM views) and its duplicate in the poverty MDIM config's definitions (used by the hand-authored multi-line views). Also remove the temporary JOE TEST view override from the earlier dry run — it would otherwise pin the old wording on that view. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EX3ywf7wME88SEttDZ35jP

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EX3ywf7wME88SEttDZ35jP

pandas to_dict('records') returns Hashable-keyed dicts; coerce keys to str to satisfy the declared dict[int, dict[str, Any]] return type. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EX3ywf7wME88SEttDZ35jP

Guides an author from a shared rules doc (Google Doc, two tabs: Input to Claude / Output from Claude) through elicitation and a structured restatement of the rules, to an ETL PR with a rules snapshot committed alongside the YAML and a staging review via the Metadata Diff wizard app. Template doc link still to be filled in. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EX3ywf7wME88SEttDZ35jP

- The author writes the texts; Claude elicits structure, flags gaps, and implements — it drafts wording only when explicitly asked, as a suggestion. Restatements must carry the author's wording verbatim. - Drop the anchor-text-plus-tweaks framing: the rules section is free-form and can have complex structure; the block+composition restatement shape is a starting point, not a requirement. - Wire in the team template doc link and the renamed tabs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EX3ywf7wME88SEttDZ35jP

…howcase - Restore the previously deleted lead-in sentence (garden + config copy), returning most views to the production baseline. - Reword one sentence in the config's extreme-poverty-additional definition (affects the 12 hand-authored multi-line views only). - Two single-view overrides: add an explanatory first bullet to the $1/day number view; reword the first bullet of the $30/day share view. Predicted blast radius: 14 of 44 views. Also fixes a stale tab name in the mdim-metadata-helper skill. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EX3ywf7wME88SEttDZ35jP

Restore poverty_pip.config.yml to its original state (matching production) so the metadata-diff tool starts from a clean slate for demos. The Wizard app, skill, and registry entry are unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EX3ywf7wME88SEttDZ35jP

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EX3ywf7wME88SEttDZ35jP

…ter) Comparing against production answers 'what will readers see change?', but inherits noise whenever production lags behind master (e.g. a failed sync-charts-to-production run). The new toggle compares against staging-site-master instead, isolating exactly what the branch changes. Baseline choice is URL-persisted and reflected in labels and links. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EX3ywf7wME88SEttDZ35jP

owidbot · 2026-07-28T10:12:07Z

Quick links (staging server):

Site Dev	Site Preview	Admin	Wizard	Docs	Docs Preview

Login: ssh owid@staging-site-incomes-inequality-wysk

chart-diff: ✅

No charts for review.

data-diff: ❌ 1 changed — full report

~ garden/un/2026-02-03/ilostat
    ~ ilostat.gender_wage_gap_by_occupation (changed metadata)

Edited: 2026-08-05 13:23:45 UTC
Execution time: 20.82 seconds

…iff tool

…ts/other MDIMs Add a blast-radius signal to the Metadata Diff tool. For each MDIM view, diff the indicator layer (raw variables metadata, before MDIM overrides) between staging and baseline: a field that changed there is shared and propagates to every standalone chart and other MDIM that uses the indicator, whereas an MDIM-level override is contained. The View diff page shows a yes/no flag with an expandable list of affected charts (deep-linking into chart-diff) and other MDIMs, per-field 'shared vs MDim-only' tags, and the tree leaves get an '↗ affects N charts/MDIMs' marker. Explorers are intentionally out of scope (being phased out).

Add a 🟡 marker in the View diff page dimension dropdowns on any control option from which a changed view is reachable, so the user can navigate straight to the changes instead of hunting through every combination.

Demonstrate the blast-radius flag: editing description_key_mean at the garden (indicator) level spills to every chart using incomes.mean, unlike the previous MDim-only view override.

…y axis - Add a 'Jump to a changed view' selectbox that drives the controls via a callback, so users reach changes directly instead of hunting; move the 🟡 marker to a prefix so it survives the selectbox's truncation, and widen the controls. - Count charts that use the indicator on ANY axis (drop the y-only filter), so scatter plots using it on x/color are included in the blast radius.

Indicator-level edit (description_key_gini) to exercise the blast-radius flag on the three gini MDims: it spills to the gini charts (7 wb / 6 wid / 3 lis) and shows in every gini MDim view.

- Blast radius toolbar: stack it into rows — checkbox+legend, then the Controls line, then the views-changed summary. - Rename user-facing 'MDIM' → 'MDim' (menu label, messages, page description). - Narrower MDim selection box. - Blast radius mode emoji 🌳 → 💥. - Hover preview keeps a few more words after the change (count visible text, not tags).

…drop arrow) - Shared-metadata callout now leads with a ⚠️ triangle and puts a 'Show N affected charts' popover button right next to the message (replacing the expander). - Drop the misleading ↗ arrow from the per-field 'shared' tag. - Render description_key by its real structure: a key stored as a '- a\n- b' markdown string (or JSON list) shows as bullets; genuine prose shows as prose.

The 'affected charts' popover now renders a self-contained component: - hover a chart to preview the actual metadata change (like the blast-radius tree) — the change chart-diff can't show, since it's a garden-template edit not a config diff; - paginate 10 per page for long lists; - explain that links open each chart on this staging server, where the change is live.

…data-page links, drop per-field tag - Shared-metadata callout back to a yellow st.warning box (matching the status box), button beside it. - Narrower 'jump to a changed view' selectbox. - Header links relabeled 'Data page — …' so it's clear they open the indicator data page. - Drop the per-field 'shared / MDim-only' tag.

- Flag affected charts where the WYSK isn't actually shown to readers: scatter / multi-indicator charts have no single-indicator data page, so a description_key edit isn't visible there (approximated by distinct-variable count). Marked '⚠ no data page' with a summary note. - Make explicit that charts always show the indicator's own text and can't be overridden individually, unlike MDim views. - Shorten the affected-charts header; MDim selector help explains select/type + the ✏️; jump label reads 'Changes detected — jump to a changed view (N)'.

…w diff page Add a 'Next change ▶' button that steps through changed views relative to the current one (cycling at the end), mark each changed view 🟢 once opened (🟡 until then), and show an 'X/N reviewed' count in the jump box so you can see which changes you've already looked at. Reword the dots caption to point at the Next button.

Add a '📋 Review' mode that gathers an MDim's changes into distinct-text groups (core.group_changes: identical shared edits collapse to one row, ranked by reach), each with its scope + blast radius, a side-by-side diff, and per-change sign-off (Pending/Approve/Flag) + comment. A 'Review summary' expander compiles the sign-off + comments into a copy-pasteable Markdown punch-list for the author. Sign-off is session-held for now; DB persistence + gate is the next slice. Pure logic tested.

…de popover, viewed dots, mode captions - Shared-indicator banner is now orange (theme-safe tint) instead of yellow. - 'Change only this view (MDim override)' moved to a prominent on-demand popover in the right column next to the affected-charts button (opens only when clicked), replacing the checkbox. - Control dots turn 🟢 once the change they lead to has been viewed (🟡 until then); caption explains viewed ≠ approved. - Mode selector promoted to full-width with per-mode captions explaining what each does.

…override as decision checkbox - Review mode: each change is an expander that collapses once decided — approve collapses at once, 🚩 flag stays open until its comment is written (flag-then-type), collapsed header shows status + 💬. - MDim picker: a namespace filter so a review can be scoped to one dataset area, not every MDim. - 'Change only this view' is back to a checkbox (it records the override decision for the PR) in the right column next to the affected-charts button; the exact edits/code sit behind an on-demand popover.

…ase C, slice 2) Reviews now persist in a self-contained metadata_review table on the staging DB (analogous to chart_diff_approvals), so sign-off survives sessions and is shared. Each decision is bound to a content hash of the exact text (core.change_group_identity): the stable change_key finds the row, the content_hash locks it — any later edit makes the stored decision 'stale' and reverts it to Pending (mirrors Chart Diff's approval invalidation). Review mode gains a lock-in gate summary (all-approved = signed off; else pending/flagged/stale counts), a stale banner + re-review prompt, and an audit line (signed off by / when). Radios/comments read from and write to the DB on change.

…tons, dots caption tidy - View diff: status line is just 'N fields changed' (yellow); the per-env page link moves onto each column header (WYSK → data page ↗, chart FAUST → chart ↗), so each side links to its own page. - Mode selector is now a segmented control (active button highlighted, others faded); the per-mode explanations move to a caption line below. - The 🟡/🟢 dots caption only shows when there are changes to jump to, and breaks onto a second line.

… in Review Guardrail against silently repainting charts: a shared change that fans out no longer has a bare 'Approve' — its sign-off is 'Apply to all (N charts · M MDims)' (reach in the label, a conscious act) vs 'Scope to this view' (records a scoped decision + shows the two-edit override guidance). Scoped blocks the gate like flagged. Non-fan-out changes keep a plain Approve. Also add per-env data-page links (data page ↗ / chart ↗) on the Review column headers, per group's representative view.

…irst bullet — test edit

…ff gets the same styling - Blast-radius leaf links now use an absolute {wizard_url}/metadata-diff?… URL instead of a relative '?query' — the relative form resolved against the component iframe's own origin, so the new tab opened a broken URL. Now it lands on View diff with that view's dimensions selected. - Individual-chart view-diff now injects DIFF_CSS (it was missing), so it renders with the same separation/columns + green change highlights as the MDim view diff.

The Charts tab now shows a DB-persisted, content-bound sign-off below the diff — same lock-in + gate as the MDim review (change_group_identity + metadata_review), keyed by the chart slug. A shared change still gets the reach-explicit 'Apply to all (N charts…)' guardrail, but with NO 'scope to this view' option (a standalone chart can't be overridden). Stale detection, audit line, and gate summary all carry over. _signoff_options gains allow_scope.

…ts/rejects Split the roles. The AUTHOR sets scope on the View diff — each shared field gets an 'Applies to: all charts & views / only this view' toggle, persisted in a new metadata_scope table (view-agnostic, content-bound key via core.text_change_key), with the override code on demand when scoped. The REVIEWER (Review tab + Charts tab) is shown that decision and only Approves/Flags — the reach-explicit 'Apply to all vs scope' choice is gone from sign-off. Chart review restructured: each field is a collapsible holding its diff + the decision (collapse on decision), no comment box.

The BERTHA / '(metadata-diff test edit)' markers on 7 garden + MDIM metadata files were scaffolding to exercise the metadata-diff tool end-to-end. Restore them to master so the branch ships only the tool.

- Review tab: reframe as an optional pass ('Review pending — N of N'), drop the 'must be approved / locked in' gate language; it never gated the PR. - Add a second output next to the review summary: a 'PR brief' that lists each change with its exact target (shared garden .meta.yml vs a scoped MDim override) and the all-charts/only-this-view decision — Markdown for now, structured to drive the actual PR. - View diff: move the author scope toggle directly under the affected- charts button; default it to 'only this view' (conservative). - Affected-charts list: 'preview the change' / 'open its data page', and drop the confusing Chart Diff link.

…e MDim - Individual-chart review: add a 'PR brief' output (each change → its target: shared indicator garden .meta.yml vs the chart's own config), matching the MDim review, so chart changes can drive a PR. - Chart review: same optional-review framing as MDims ('Review pending — N of N'), dropping the 'must sign off / lock in' language. - MDim picker opens on a clean slate (nothing selected) with a blue prompt, mirroring the chart tab. - Chart picker copy: 'changes to its data page' → 'changes to its metadata' (multi-indicator charts have no data page).

- Review intro: drop 'optional / doesn't gate' — the review is the path to the outputs. New copy: iterate with the author, then share comments or create a PR at the end. - Scope label: drop the confusing '(other surfaces revert)' parenthetical.

- Group the brief by review decision with an explicit legend so the rules never need re-explaining: ✅ Approve → add to PR · 🚩 Flag → hold (not added) · ⏳ Pending → do nothing. - Approved changes are turnkey: pastable YAML value + a best-guess file/key resolved from the indicator catalogPath (garden dir, tables.<t>.variables. <short>), or a scoped .py override. The value is exact; the location is flagged as a best guess to confirm against the metadata build. - Thread catalogPath from variables through ViewBundle → ViewDiff → ChangeGroup; add parse_catalog_path + yaml_field_snippet (ruamel) + tests. - Same treatment for the individual-chart brief (chart-config changes marked as NOT an ETL edit).

…ator Adds four general description_key bullets to gender_wage_gap_by_occupation (what the gap measures and its sign convention, the unadjusted caveat, the paid-employees-only scope, and an ILOSTAT comparability note). Placed around the existing occupation-specific bullet, which renders empty for the Total occupation — so the gender-gap-in-average-wages chart, which had no WYSK, now shows these.

- description_key rendered as one joined "- a\n- b" string (grapher-channel serialization) instead of a YAML list; normalize via as_bullets before dump. - Best-guess key pointed at the flattened grapher column (name__dim_value...); strip the flattening suffix to the base garden key, which is where the Jinja-templated indicator is authored. - st.code's built-in copy icon relies on the async Clipboard API, which no-ops in non-secure/iframed staging contexts; add a copy button with an execCommand fallback plus a clipboard-free .md download.

@codex

…sage hint - PR brief now ends with a 'Ship it' section: the quality-check + review checklist (blast radius, make check, typos/spacing/style, adversarial claims, staging rebuild+verify, @codex) and a ready-to-paste PR body — a complete spec, not just a list of edits. - Reviewer sign-off no longer defaults to the container OS user ('owid'); add a sidebar reviewer field (defaulting to a detected identity) recorded with each decision. - Brief header + expander captions tell the reviewer to paste it to Claude Code and ask for the PR.

Single-indicator chart data pages render blank on a staging server by default, so the chart-review 'data page' links showed nothing. Point them at the admin chart preview with forceDatapage=true (using the chart id the flow already resolves), which forces the data page so WYSK/description_key edits are actually visible. Works on production too.

admin_site already ends in /admin, so the preview links were built as /admin/admin/charts/... and failed. Drop the extra segment.

Sync this branch's test WYSK to the reviewed version on PR #6611 (detailed occupation; drop informal-employment exclusion).

'sign off, comment & prepare a PR' — the review's endpoint is the PR brief.

The reviewer identity now lives on the page in both the MDim and chart review sections (recorded with each sign-off), instead of the sidebar.

- Review (MDim + chart) now renders only the WYSK bullets that changed, not the whole description_key list; View diff keeps the full list for context. - Remove the on-page reviewer field (and the sidebar one) — better nothing for now; sign-offs record without a name until we design this properly. - Drop the '(all charts / only this view)' bracket from the MDim PR-brief expander title.

- Rename the description_key field label to WYSK. - Drop the chart count from the review change header; fold it into the scope caption as a consequence ('N charts also use this indicator → all will change' / '→ scoped to this MDim only, they keep their text'). - Remove the override-code popover from the scope toggle (the PR brief already carries the override snippet). - Remove the 🟡/🟢 markers from the MDim control dropdowns; keep them only in the jump-to-changes menu, and reword its legend.

MDim views aren't added in a text-review branch, so is_new views don't occur here — remove the 🆕 marker and its legend entry.

Scope decision now defaults to 'only this view' and locks 'apply to all' until the author expands the affected chart/MDim list — so applying a shared change everywhere is a deliberate, informed choice. The affected list moves from the impact popover into that review checkbox (charts keep the popover, which have no scope decision). A note under the toggle explains the gate.

Move the 'show affected charts' reveal into the orange shared-metadata banner as a show/hide button (not a checkbox). Opening it marks the indicator reviewed, which unlocks 'apply to all' in the scope toggle below; default stays 'only this view'. A saved 'apply to all' starts unlocked.

claude added 16 commits July 17, 2026 13:30

💄🤖 metadata-diff: apply ruff formatting

b75eac0

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EX3ywf7wME88SEttDZ35jP

🔨🤖 revert unintended uv.lock churn from local test runs

ebaf6d0

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EX3ywf7wME88SEttDZ35jP

🔨🤖 revert unintended root uv.lock churn from local env build

efacc34

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EX3ywf7wME88SEttDZ35jP

🔀 merge current master to re-baseline staging against current production

c1e783c

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EX3ywf7wME88SEttDZ35jP

🔀 merge current master to re-baseline against latest production texts

efe750c

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EX3ywf7wME88SEttDZ35jP

claude added 13 commits July 28, 2026 10:35

📊🤖 Test edits to incomes/poverty MDim metadata to exercise metadata-d…

a456b45

…iff tool

📊🤖 Test: prepend BERTHA to first WYSK bullet of incomes_pip mean views

ab14692

📊🤖 Test: move BERTHA WYSK edit from MDim override to the mean indicator

61d42f7

Demonstrate the blast-radius flag: editing description_key_mean at the garden (indicator) level spills to every chart using incomes.mean, unlike the previous MDim-only view override.

💄🤖 metadata-diff: revert view-diff controls to compact one-line (4 cols)

1104f59

📊🤖 Test: prepend BER to the gini WYSK bullet in wb/wid/lis gardens

405afe1

Indicator-level edit (description_key_gini) to exercise the blast-radius flag on the three gini MDims: it spills to the gini charts (7 wb / 6 wid / 3 lis) and shows in every gini MDim view.

claude added 30 commits August 4, 2026 17:50

📊🤖 wto_trade_growth: add BERTHA marker to volume_index short desc + f…

79a7e09

…irst bullet — test edit

🔨🤖 metadata-diff: revert test-marker edits to status quo

2f0361d

The BERTHA / '(metadata-diff test edit)' markers on 7 garden + MDIM metadata files were scaffolding to exercise the metadata-diff tool end-to-end. Restore them to master so the branch ships only the tool.

🐛🤖 metadata-diff: fix doubled /admin in chart preview URL

78face2

admin_site already ends in /admin, so the preview links were built as /admin/admin/charts/... and failed. Drop the extra segment.

✨🤖 ilostat: match gender wage gap WYSK to the final PR wording

89170cd

Sync this branch's test WYSK to the reviewed version on PR #6611 (detailed occupation; drop informal-employment exclusion).

💄🤖 metadata-diff: Review mode caption names the PR outcome

d77b60e

'sign off, comment & prepare a PR' — the review's endpoint is the PR brief.

💄🤖 metadata-diff: tidy baseline labels (Production; Master)

618d2d8

💄🤖 metadata-diff: move reviewer field from sidebar to the review page

eebdf87

The reviewer identity now lives on the page in both the MDim and chart review sections (recorded with each sign-off), instead of the sidebar.

💄🤖 metadata-diff: drop '🆕 new view' from the jump menu

5303964

MDim views aren't added in a text-review branch, so is_new views don't occur here — remove the 🆕 marker and its legend entry.

Navigation Menu

Uh oh!

Uh oh!

📊 WYSK edits for incomes / Gini / Palma MDIMs (draft — no changes yet) - #6540

📊 WYSK edits for incomes / Gini / Palma MDIMs (draft — no changes yet)#6540
JoeHasell wants to merge 70 commits into
masterfrom
incomes-inequality-wysk

JoeHasell commented Jul 28, 2026

Uh oh!

owidbot commented Jul 28, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

JoeHasell commented Jul 28, 2026

Uh oh!

owidbot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

owidbot commented Jul 28, 2026 •

edited

Loading