Skip to content

🎉 Add WID wealth composition of households and NPISH - #6755

Draft
paarriagadap wants to merge 1 commit into
masterfrom
data-wid-wealth-composition
Draft

🎉 Add WID wealth composition of households and NPISH#6755
paarriagadap wants to merge 1 commit into
masterfrom
data-wid-wealth-composition

Conversation

@paarriagadap

@paarriagadap paarriagadap commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Written by Claude Opus 5 — @paarriagadap at the wheel.

Adds the wealth composition of the households and NPISH sector to the World Inequality Database dataset — the breakdown the GC Wealth Project's "Mountains of Assets" Country View chart shows, built from WID's own balance-sheet series.

We already publish WID's distributional wealth data (shares, Gini, averages and thresholds of household net wealth) but nothing about what that wealth is made of. These series exist in WID; they are aggregate balance-sheet variables rather than distributional ones, and the "share of national income" form is a series type (the w prefix) rather than a separate concept, which is why they are easy to miss on the site.

Four new indicators

All expressed as a share of net national income, for the households and NPISH sector combined (which WID calls the private sector).

Indicator WID code Rows Entities Years
Net wealth wpweal 21,094 232 1800–2025
Debt wpwdeb 10,557 231 1980–2025
Housing and land wpwhou 10,626 231 1980–2025
Financial assets and business capital wpwbus + wpwfin 10,615 231 1980–2025

Net wealth reaches back to 1800 while the three components start in 1980 because that is WID's own limit: the component series are served from 1980 for every country, in both the household and the household-plus-NPISH sector, and in every series type. Verified directly against the API with no percentile, age or population filter.

The category definitions follow the GC Wealth Project's, which states the WID formulas in the Metadata tab of its chart. WID publishes financial assets and business assets as two series; they are added together in garden.

2025 values (% of national income): United States 653 net wealth (97 debt, 228 housing and land, 522 financial and business) · France 593 (83 / 347 / 329) · Japan 731 (81 / 239 / 574) · Chile 352 (57 / 175 / 234).

Implementation

  • wid_indices.do gains a section for the aggregate balance sheet, plus a aggregate_wealth_only toggle so it can be rebuilt on its own in a few minutes rather than re-running the distributional extraction that takes hours. No PPP conversion applies: the w series is already a unitless ratio.
  • One new snapshot inside the existing 2026-06-18 version, with its own script (the existing one refuses to run unless all five of its CSVs are present). No version bump, so existing WID variable IDs are untouched — no ghost variables, no chart remapping.
  • Garden builds a wealth_aggregates table with a wealth_type dimension, and asserts WID's balance-sheet identity, the sign each category can take, that the identity check has not gone vacuous, and the magnitude of the ratios.
  • No extrapolated dimension on this table: with and without extrapolations these series are bit-identical across all 341,475 rows WID returns, so the dimension would have doubled the indicators for nothing.

Verification

  • Cross-checked against the GC Wealth Project's published warehouse (their Wealth Topography CSV, p-hn-agg-* varcodes, source WID_topo). Over 3,000 shared country-years the median divergence is 0.26–0.67 percentage points of net wealth; United States 2015 matches to a decimal. Divergence above 2 pp is confined to Greece, Russia, Singapore, Denmark, Uruguay, Finland, South Africa, China and the Netherlands — their snapshot carries the components for only 26 of 70 countries and ends in 2022, and where it disagrees our levels are the ones consistent with WID's current release and with known macro history.
  • Balance-sheet identity holds to 0.02 pp; wpweal equals mpweal / mnninc to 5e-5, confirming the denominator is net national income.
  • etl diff REMOTE data/ --include world_inequality_database: every pre-existing table byte-identical, only the new table added.
  • The do-file was also run end to end with the toggle off, in a scratch directory, to confirm it still produces all six CSVs.
  • make check, version-tracker, codespell clean. 773 tests pass; the 4 failures (test_app_indicator_upgrade, test_app_producer_analytics, 2× test_explorer_migration) fail identically on clean master — they need a local MySQL.

Still open

  • No chart ships with this PR — it publishes the four indicators only. Building charts or MDim views on top of them is a separate decision. One design constraint worth recording for whoever picks it up: debt cannot be stacked with the asset categories, because it is a liability and grapher stacks only positive series, so a composition chart wants the two asset categories stacked with net wealth and debt shown separately.
  • Italy 1942–1944 has negative net wealth in WID's own datampweal itself is negative there, with national income staying positive. It is 3 rows of 21,094, but yAxis.min: 0 means a zero-baseline chart will not show the dip. Kept at 0 to match every other indicator in this dataset rather than special-casing one; happy to switch it to auto if a reviewer prefers.
  • A pre-existing outdated pattern in the meadow step could not be modernized. ds_regions["regions"].reset_index() cannot become ds_regions.read("regions"): the latter returns nullable string dtypes, which stops the country merge from matching and makes unmatched rows read "<NA>" instead of the "nan" sentinel harmonize_countries filters on, so rows that should be dropped survive and the index gains duplicates. Reverted with a NOTE; fixing it properly means changing that sentinel for all five tables.
  • Nobody has assessed whether these four indicators belong in the wealth_wid MDim, or whether the separate financial and business components are worth publishing on their own. Both raw series are already in the snapshot, so splitting them later is a small change.

Four new indicators for the balance sheet of the households and NPISH sector
combined, as a share of national income: net wealth, debt, housing and land,
and financial assets and business capital.

These are WID's aggregate balance-sheet series (the `w` series type, a ratio to
net national income), which sit outside the distributional data the pipeline
already carries. The extraction is a new section of `wid_indices.do`, guarded by
a toggle so it can be rebuilt on its own in minutes without re-running the
distributional extraction that takes hours.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@owidbot

owidbot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Quick links (staging server):

Site Dev Site Preview Admin Wizard Docs Docs Preview

Login: ssh owid@staging-site-data-wid-wealth-composition

chart-diff: ✅ No charts for review.
data-diff: ❌ 1 changed · 1 identical — full report
~ garden/wid/2026-06-18/world_inequality_database
    + wealth_aggregates (new table, 1 column(s))

= 1 compared dataset(s) turned out identical; details in the full report

Automatically updated datasets matching excess_mortality|covid|fluid|flunet|country_profile|garden/ihme_gbd/2019/gbd_risk are not included. Run locally with etl diff REMOTE data/ --include <dataset> --verbose.

Edited: 2026-08-24 17:51:26 UTC
Execution time: 4.48 seconds

@paarriagadap
paarriagadap marked this pull request as draft August 28, 2026 10:30
paarriagadap added a commit that referenced this pull request Aug 28, 2026
Population was a standalone one-table dataset with its own snapshot
script, meadow and garden steps. It is now the `population` table of
wid/2026-06-18/world_inequality_database, so any garden step reads it
from the same dataset as the rest of WID.

The extraction is consolidated too: the npopul block moves into
wid_indices.do, and its CSV joins the FILES map in the snapshot script,
so one .do and one .py cover all six WID snapshots. The block records why
unit i is correct -- the population dimension exists only for distributed
series, so aptinc992j / npopul992i is the right pairing.

The .dvc moves to 2026-06-18 keeping its md5. Snapshot storage is
content-addressed, so it resolves to the blob already uploaded: no Stata
re-run and no re-upload, and the data is unchanged.

Verified: the population table is identical to the standalone one it
replaces (55,581 rows, 249 countries, zero differing values, origins
preserved), and every 2023 between share is unchanged to four decimals.

Note for #6755, which also edits these four files: the meadow, garden,
garden .meta.yml and wid_indices.do changes here are additive, so the
conflict should resolve mechanically.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants