📊 energy: Update IRENA data - #6546
Draft
pabloarosado wants to merge 21 commits into
Draft
Conversation
Contributor
|
Quick links (staging server):
Login: chart-diff: ❌
data-diff: ❌ 2 changed · 4 new — full report~ garden/economics/2026-08-21/owid_deflator (new version)
~ dataset metadata changed
~ owid_deflator (changed table metadata)
~ owid_deflator.country dim (new data, removed data)
+ New values: 208 / 11,785 (1.76%)
2025: Andorra
2025: Antigua and Barbuda
2025: Argentina
(+205 more)
- Removed values: 10 / 11,785 (0.08%)
1979: Djibouti
1980: Djibouti
1981: Djibouti
(+7 more)
~ owid_deflator.year dim (new data, removed data)
+ New values: 208 / 11,785 (1.76%)
Andorra: 2025
Antigua and Barbuda: 2025
Argentina: 2025
(+205 more)
- Removed values: 10 / 11,785 (0.08%)
Djibouti: 1979
Djibouti: 1980
Djibouti: 1981
(+7 more)
~ owid_deflator.cpi (new data, changed data)
+ New values: 208 / 11,785 (1.76%)
Andorra 2025: NaN
Antigua and Barbuda 2025: 144.1938934326172
Argentina 2025: NaN
(+205 more)
- Removed values: 10 / 11,785 (0.08%)
Djibouti 1979: 82.38568878173828
Djibouti 1980: 92.31977081298828
Djibouti 1981: 97.59803009033203
(+7 more)
~ Changed values: 394 / 11,785 (3.34%)
Barbados 2023: 170.89007568359375 → 143.23291015625
Barbados 2024: 170.1001739501953 → 145.3046875
Barbados 2022: 155.64883422851562 → 138.80880737304688
(+391 more)
~ owid_deflator.gdp_deflator (new data, changed data)
+ New values: 208 / 11,785 (1.76%)
Andorra 2025: 127.73190307617188
Antigua and Barbuda 2025: 128.0959014892578
Argentina 2025: 114689.8203125
(+205 more)
- Removed values: 10 / 11,785 (0.08%)
Djibouti 1979: NaN
Djibouti 1980: NaN
Djibouti 1981: NaN
(+7 more)
~ Changed values: 3,459 / 11,785 (29.35%)
Venezuela 2011: 2334.928466796875 → 2.6678710796090854e-08
Venezuela 2013: 3608.715576171875 → 4.4792439979346454e-08
Venezuela 2012: 2663.206298828125 → 3.217236610453256e-08
(+3,456 more)
~ owid_deflator.gdp_deflator_linked (new data, changed data)
+ New values: 208 / 11,785 (1.76%)
Andorra 2025: 127.73190307617188
Antigua and Barbuda 2025: 128.0959014892578
Argentina 2025: 114689.8203125
(+205 more)
- Removed values: 10 / 11,785 (0.08%)
Djibouti 1979: NaN
Djibouti 1980: NaN
Djibouti 1981: NaN
(+7 more)
~ Changed values: 2,226 / 11,785 (18.89%)
Venezuela 2011: 2334.928466796875 → 3.27273994571442e-08
Venezuela 2013: 3608.715576171875 → 5.4947935979043905e-08
Venezuela 2012: 2663.206298828125 → 3.9466595325166054e-08
(+2,223 more)
~ garden/wb/2026-08-21/wdi_prices (new version)
~ dataset metadata changed
~ wdi_prices (changed table metadata)
~ wdi_prices.country dim (new data, removed data)
+ New values: 208 / 11,785 (1.76%)
2025: Andorra
2025: Antigua and Barbuda
2025: Argentina
(+205 more)
- Removed values: 10 / 11,785 (0.08%)
1979: Djibouti
1980: Djibouti
1981: Djibouti
(+7 more)
~ wdi_prices.year dim (new data, removed data)
+ New values: 208 / 11,785 (1.76%)
Andorra: 2025
Antigua and Barbuda: 2025
Argentina: 2025
(+205 more)
- Removed values: 10 / 11,785 (0.08%)
Djibouti: 1979
Djibouti: 1980
Djibouti: 1981
(+7 more)
~ wdi_prices.cpi (new data, changed data)
+ New values: 208 / 11,785 (1.76%)
Andorra 2025: NaN
Antigua and Barbuda 2025: 144.1938934326172
Argentina 2025: NaN
(+205 more)
- Removed values: 10 / 11,785 (0.08%)
Djibouti 1979: 82.38568878173828
Djibouti 1980: 92.31977081298828
Djibouti 1981: 97.59803009033203
(+7 more)
~ Changed values: 394 / 11,785 (3.34%)
Barbados 2023: 170.89007568359375 → 143.23291015625
Barbados 2024: 170.1001739501953 → 145.3046875
Barbados 2022: 155.64883422851562 → 138.80880737304688
(+391 more)
~ wdi_prices.gdp_deflator (new data, changed data)
+ New values: 208 / 11,785 (1.76%)
Andorra 2025: 127.73190307617188
Antigua and Barbuda 2025: 128.0959014892578
Argentina 2025: 114689.8203125
(+205 more)
- Removed values: 10 / 11,785 (0.08%)
Djibouti 1979: NaN
Djibouti 1980: NaN
Djibouti 1981: NaN
(+7 more)
~ Changed values: 3,459 / 11,785 (29.35%)
Venezuela 2011: 2334.928466796875 → 2.6678710796090854e-08
Venezuela 2013: 3608.715576171875 → 4.4792439979346454e-08
Venezuela 2012: 2663.206298828125 → 3.217236610453256e-08
(+3,456 more)
~ wdi_prices.gdp_deflator_linked (new data, changed data)
+ New values: 208 / 11,785 (1.76%)
Andorra 2025: 127.73190307617188
Antigua and Barbuda 2025: 128.0959014892578
Argentina 2025: 114689.8203125
(+205 more)
- Removed values: 10 / 11,785 (0.08%)
Djibouti 1979: NaN
Djibouti 1980: NaN
Djibouti 1981: NaN
(+7 more)
~ Changed values: 2,226 / 11,785 (18.89%)
Venezuela 2011: 2334.928466796875 → 3.27273994571442e-08
Venezuela 2013: 3608.715576171875 → 5.4947935979043905e-08
Venezuela 2012: 2663.206298828125 → 3.9466595325166054e-08
(+2,223 more)
+ garden/energy/2026-07-28/photovoltaic_cost_and_capacity (new dataset: 1 table(s), 4 column(s))
+ garden/irena/2026-07-28/renewable_energy_patents (new dataset: 2 table(s), 28 column(s))
+ garden/irena/2026-07-28/renewable_energy_statistics (new dataset: 1 table(s), 27 column(s))
+ garden/irena/2026-07-28/renewable_power_generation_costs (new dataset: 2 table(s), 8 column(s))Automatically updated datasets matching excess_mortality|covid|fluid|flunet|country_profile|garden/ihme_gbd/2019/gbd_risk are not included. Run locally with Edited: 2026-08-28 14:19:37 UTC |
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
pabloarosado
force-pushed
the
data-irena-update
branch
from
August 21, 2026 14:58
f6376f9 to
fe73cf8
Compare
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…script Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…description Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4c96936b20
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…der the DAG Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Updates IRENA energy datasets to their 2026 releases and refreshes the supporting inflation-adjustment pipeline.
Changes:
- Reworks IRENA capacity, cost, and patent ingestion.
- Combines historical cost releases using the US GDP deflator.
- Updates WDI inputs, downstream photovoltaic data, Lazard, and DAG wiring.
Optional add-ons not run: the token-heavy empty-entity audit and adversarial review (~25–45 web calls).
Reviewed changes
Copilot reviewed 29 out of 41 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
dag/economics.yml |
Adds the 2026 deflator chain. |
dag/energy.yml |
Wires updated IRENA and downstream steps. |
etl/steps/data/garden/economics/2026-08-21/owid_deflator.meta.yml |
Defines deflator metadata. |
etl/steps/data/garden/economics/2026-08-21/owid_deflator.py |
Produces the refreshed deflator. |
etl/steps/data/garden/energy/2026-07-28/photovoltaic_cost_and_capacity.meta.yml |
Updates photovoltaic metadata. |
etl/steps/data/garden/energy/2026-07-28/photovoltaic_cost_and_capacity.py |
Combines photovoltaic cost and capacity. |
etl/steps/data/garden/irena/2026-07-28/renewable_energy_patents.countries.json |
Maps patent entities. |
etl/steps/data/garden/irena/2026-07-28/renewable_energy_patents.meta.yml |
Defines patent metadata. |
etl/steps/data/garden/irena/2026-07-28/renewable_energy_patents.py |
Processes and aggregates patents. |
etl/steps/data/garden/irena/2026-07-28/renewable_energy_statistics.countries.json |
Maps capacity entities. |
etl/steps/data/garden/irena/2026-07-28/renewable_energy_statistics.meta.yml |
Defines capacity metadata. |
etl/steps/data/garden/irena/2026-07-28/renewable_energy_statistics.py |
Remaps and aggregates capacity data. |
etl/steps/data/garden/irena/2026-07-28/renewable_power_generation_costs.countries.json |
Maps cost entities. |
etl/steps/data/garden/irena/2026-07-28/renewable_power_generation_costs.meta.yml |
Documents cost indicators. |
etl/steps/data/garden/irena/2026-07-28/renewable_power_generation_costs.py |
Combines and deflates cost releases. |
etl/steps/data/garden/lazard/2025-09-17/levelized_cost_of_energy.py |
Repoints Lazard’s deflator logic. |
etl/steps/data/garden/wb/2026-08-21/wdi_prices.countries.json |
Maps WDI entities. |
etl/steps/data/garden/wb/2026-08-21/wdi_prices.meta.yml |
Defines WDI price metadata. |
etl/steps/data/garden/wb/2026-08-21/wdi_prices.py |
Harmonizes WDI price series. |
etl/steps/data/grapher/energy/2026-07-28/photovoltaic_cost_and_capacity.py |
Exports photovoltaic indicators. |
etl/steps/data/grapher/irena/2026-07-28/renewable_capacity.py |
Adds gigawatt capacity indicators. |
etl/steps/data/grapher/irena/2026-07-28/renewable_capacity_by_technology.py |
Builds technology-selection indicators. |
etl/steps/data/grapher/irena/2026-07-28/renewable_energy_patents.py |
Exports patent indicators. |
etl/steps/data/grapher/irena/2026-07-28/renewable_power_generation_costs.py |
Exports cost indicators. |
etl/steps/data/meadow/irena/2026-07-28/renewable_energy_patents.py |
Loads the patent snapshot. |
etl/steps/data/meadow/irena/2026-07-28/renewable_energy_statistics.py |
Parses the restructured capacity workbook. |
etl/steps/data/meadow/irena/2026-07-28/renewable_power_generation_costs.py |
Extracts LCOE and module prices. |
etl/steps/data/meadow/wb/2026-08-21/wdi_prices.py |
Combines WDI price snapshots. |
snapshots/irena/2026-07-28/renewable_energy_patents.csv.dvc |
Records patent source metadata. |
snapshots/irena/2026-07-28/renewable_energy_patents.py |
Imports the manual patent download. |
snapshots/irena/2026-07-28/renewable_energy_statistics.xlsx.dvc |
Records capacity source metadata. |
snapshots/irena/2026-07-28/renewable_power_generation_costs.xlsx.dvc |
Records cost source metadata. |
snapshots/wb/2026-08-21/wdi_cpi.zip.dvc |
Adds the CPI snapshot. |
snapshots/wb/2026-08-21/wdi_gdp.zip.dvc |
Adds the GDP deflator snapshot. |
snapshots/wb/2026-08-21/wdi_gdp_linked.zip.dvc |
Adds the linked deflator snapshot. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Replaces the in-place edit of the 2025-09-17 step with a new 2026-08-28 version, and removes the old chain. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
pabloarosado
changed the title
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
pabloarosado
changed the title
…29 deflator steps Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tracks: owid/owid-issues#2254
Summary
Updates the three IRENA datasets to the 2026 releases, plus the OWID deflator they depend on:
irena/2026-07-28/renewable_energy_statistics): the 2026 file was restructured (wide → long format, a new technology taxonomy per the file's "Tech Mapping" sheet, abbreviated country labels), so the meadow and garden steps were rewritten. Output is unchanged in shape: same 27 indicators, same entities, 2025 added. The new country mapping was derived from the file's ISO3 codes and reproduces the previous 224 target countries exactly. IRENA dropped Russia's off-grid capacity entirely in this release (verified in the raw file).irena/2026-07-28/renewable_power_generation_costs): now combines three releases, since each older one holds data the newer ones dropped (pre-2010 history from the 2023 report; 21 onshore-wind and 3 solar country series plus the module price index from the 2024 report). The new release reports LCOE in USD/MWh (converted to the published USD/kWh in meadow) and in 2025 USD.irena/2026-07-28/renewable_energy_patents): February 2026 edition, data through 2025 (2023 revised +33%, 2024 filled in from 17k to 168k classified patents, as expected from publication lag). The source renamed itsCountrycolumn. The long-standing silent exclusion of rows without a technology classification (sector-level climate-mitigation counts, 58% of raw filed patents) is now explicit in the garden step, with assertions; output values are unchanged.economics/2026-08-21/owid_deflator+wb/2026-08-21/wdi_prices): updated to WDI release 129 so the deflator covers 2025. Lazard's LCOE step is untouched and still uses the 2025-08-29 deflator, so that version stays in the DAG for now.The superseded IRENA, photovoltaic and deflator steps were removed and recorded in
dag/archive/; the two older IRENA cost meadow steps stay active because the new cost garden step reads them.Inflation-adjustment fix
When combining cost releases (each in constant US dollars of a different year), older values are now restated with a single US GDP-deflator factor for all countries, reverting the per-country deflators introduced in #4952 "📊 energy: Fix inflation adjustment in IRENAs data" and returning to the approach of #4912 "📊 energy: Update IRENA renewable generation costs". Dividing identical country-years across consecutive IRENA releases shows the producer itself restates every country by the same ~1.02 factor (Argentina 1.021, Turkey 1.019, US 1.018), so the values are plain constant USD. Effects on published data: Argentina's pre-2010 onshore-wind LCOE drops ×0.34 (2008: $0.56 → $0.19/kWh), Turkey's ×0.67, South Korea 2004-2009 is restored, and all other countries move only a few percent.
Not covered in this PR
photovoltaic_cost_and_capacityshows cumulative capacity through 2025 but cost through 2024, for the same reason.