Inf-1: ETL vignette — convert WB Official Boundaries to per-admin-level parquet and upload to WB Data Catalog

[EBukin]

What to build

A data-raw/ vignette (R script) run once by WB package maintainers whenever the WB Official Boundaries dataset is updated. It downloads the source GeoJSONs/shapefiles, converts them to three global parquet files (one per admin level), and a per-country hex-grid parquet at H3 resolution 6. These files are then uploaded to the WB Data Catalog as dataset resources and become the backing store for get_country_shapes() (Inf-4).

This is not end-user code. It runs in a controlled WB environment with write access to the Data Catalog.

Output parquet files

Three boundary parquets (one per admin level, all countries in one file):

• wb_boundaries_admin0.parquet
• wb_boundaries_admin1.parquet
• wb_boundaries_admin2.parquet

Schema per file:

Column	Type	Notes
iso3	string	ISO-3 country code — primary filter key
admin_level	int	0, 1, or 2
pcod	string	Unique polygon code (maps to admin<N>Pcod in the package)
name	string	English polygon name (maps to admin<N>Name)
name_local	string	Local-language name (null if unavailable)
parent_pcod	string	Parent-level pcod; null for admin0
area_km2	double	Pre-computed area in km²
geometry	binary	WKB (Well-Known Binary) polygon geometry
wb_source_version	string	WB Boundaries dataset version tag

Geometry stored as raw WKB binary column (compact; readable with sf::st_as_sfc()). Arrow predicate pushdown on iso3 means get_country_shapes("RWA") reads only Rwanda rows from each file.

One hex-grid parquet per country (pre-computed H3 resolution 6 grid + spatial join to admin boundaries):

• wb_hex_r6_{iso3}.parquet or partitioned as wb_hex_r6/iso3={iso3}/data.parquet

Schema:

Column	Type	Notes
admin9Pcod	string	H3 cell ID at resolution 6
admin0Pcod	string	Parent country code
admin1Pcod	string	Parent admin1 pcod
admin2Pcod	string	Parent admin2 pcod (null if country has no admin2)
area_km2	double	Cell area
resolution	int	Always 6 in v1
geometry	binary	WKB hex polygon

ETL steps (within the vignette)

1. Download each admin-level GeoJSON/shapefile from WB Official Boundaries
2. Standardise column names to the schema above; compute area_km2; derive parent_pcod via spatial join of centroids where not already present in source
3. Convert geometry to WKB binary: sf::st_as_binary(geometry)
4. Write to parquet using arrow::write_parquet() — one file per admin level
5. For each country: build H3 resolution-6 hex grid via make_hex_grid() + make_admin_lookup() and write hex parquet
6. Upload all files to WB Data Catalog as dataset resources; record the base URL

Dependencies

New package dependencies for this ETL script (not for end users — Suggests or script-only):

• arrow (already in pipeline for hex fetch)
• sf
• h3jsr (already used for hex grid)

Acceptance criteria

• data-raw/etl-wb-boundaries.R script exists and is runnable by a WB maintainer with Data Catalog write access
• Three admin-level parquet files produced with the schema above; readable with arrow::open_dataset() + dplyr::filter(iso3 == "RWA")
• Geometry column is WKB binary; sf::st_as_sfc() recovers valid polygons
• Hex parquets produced for all countries with admin2 coverage; H3 parent Pcod join is correct (spot-checked against Rwanda)
• Source version recorded in wb_source_version column
• Script is idempotent (re-run produces the same output for the same source version)
• A README note in data-raw/ explains when and how to re-run this script

Blocked by

None — can start immediately.