Source layer "park" does not exist on source "openmaptiles"

[3nprob]

When running a build using a custom bbike export from extract.bbike.org, the park layer is missing, resulting in broken rendering of affected areas and the following errors in browser console:

Error: Source layer "park" does not exist on source "openmaptiles" as specified by style layer "park".
Error: Source layer "park" does not exist on source "openmaptiles" as specified by style layer "park_outline".

Reproduction

1. Create custom area
   • Put builds/MyArea/.env in place with custom vars
   • Put MyArea.osm.pbf in place
2. Build docker images as in GHA workflow
3. Build data assets
   • cd builds/MyArea && bin/build builds/MyArea
     • I see the planetiler container pull some endpoints of the format https://onthegomap.github.io/planetiler-demo/#{z}/{lat}/{lon}/ during this step
4. Start local app
   • cd builds/MyArea && docker-compose up -f ../../docker-compose.yaml --env-file .env up -d
     • (Run docker exec -it -u0 headway-pelias-elasticsearch-1 chown 1000:1000 /usr/share/elasticsearch/data to fix elasticsearch crashlooping on missing permissions)

Env vars to use locally mirrored/forked sources for all assets are set both in builds/MyArea/.env and the local environment:

HEADWAY_PLANETILER_FIXTURES_URL=...
HEADWAY_IMAGE_REPO=...
HEADWAY_TILES_URL=...
HEADWAY_PBF_URL=...
HEADWAY_AREA=MyArea

planetiler log snip:

INF [archive] - Starting...
INF [archive] - Unable to load tile weights from data/tile_weights.tsv.gz, falling back to top 100k
[...]
INF [archive:write] - Shallow tiles written: 411
INF [archive:write] - Tile data written: 411 (0% omitted) 
INF [archive:write] - Unique tile hashes: 0 
[...]
    last tile: ../..../... (z14 95%) https://onthegomap.github.io/planetiler-demo/#... 
DEB [archive] - Tile stats:
DEB [archive] - Biggest tiles (gzipped)
[...]
DEB [archive] - Max tile sizes

#[list of water,boundary,aeroway,... but not park]

DEB [archive] -    # tiles: 411
DEB [archive] -  # features: 233,036
[...]
FINISHED!
----------------------------
data errors:
  osm_multipolygon_missing_way      201
  render_snap_fix_input    160
  osm_boundary_missing_way    142
  feature_polygon_osm_invalid_multipolygon_empty_after_fix    2
  omt_park_area_osm_invalid_multipolygon_empty_after_fix    1
  merge_snap_fix_input    1

Observations

• The issue is not present for the same area on https://maps.earth, where it renders with parks as expected
• /tileserver/styles/basic/style.json
  • On maps.earth and the local app, layers contains park.
  • This is controlled by services/tileserver/styles/basic/style.json
• /tileserver/data/default.json
  • On maps.earth, vector_layers contains park and park_outline
  • On local app, vector_layers does not contain park or park_outline. Otherwise the field looks identical to the maps.earth one.
  • It is not clear to me yet exactly at which point in the lifecycle default.json gets created (or if part of it's still somehow sourced externally)
    • Understanding the data-build lifecycle better would help. E.g. exactly what's the role of https://github.com/headwaymaps/headway-data and how were those files sourced/generated?
  • version is 3.15.0 on local app; 3.14.0 on maps.earth
  • planetiler:version is the same (0.7.0)
    • Using a newer planetiler version (0.8.4) does not make a difference wrt the issue

Not clear if this is due to an application bug, misconfiguration, or broken data. I did redo the whole pipe on separate machines and exports with identical results.

Similar issue:

• Getting: Error: Source layer "aerodrome_label" does not exist on source "openmaptiles" ..... mapbox/mapbox-gl-js#10120
  • Removing park and park_outline from style.json as suggested by one comment removes the entries from the /tileserver/styles/basic/style.json result but does seem not change behavior otherwise.

[3nprob]

The missing weights file was a red herring. If I change dagger/main.go to mount https://github.com/onthegomap/planetiler/blob/main/layerstats/top_osm_tiles.tsv.gz as data/tile_weights.tsv.gz for the Mbtiles stage, the error goes away but the end result is the same with park still missing.

Another error in the data-build logs, slightly further up for valhalla and possibly also unrelated:

Container.withMountedDirectory(
  path: "/tiles"
  source: Container.directory(path: "/tiles"): Directory!
): Container!
Container.withMountedDirectory DONE
Contaner.withExec(args: ["sh", "-c", "valhalla_export_edges -c valhalla.json > /tiles/polylines.0sv"]): Container!
Container.file(path: "/tiles/polylines.0sv") File!
Contaner.withExec(args: ["sh", "-c", "valhalla_export_edges -c valhalla.json > /tiles/polylines.0sv"]): Container!
[ERROR] (stat): /data/valhalla/tiles.tar No such file or directory
[WARN] Tile extract could not be loaded
[WARN] (stat): /data/valhalla/traffic.tar: No such file or directory
[WARN] Traffic tile extract could not be loaded
[INFO] Enumerating edges...

[michaelkirk]

I don't immediately know the cause of this, but I can explain this part:

It is not clear to me yet exactly at which point in the lifecycle /tileserver/data/default.json gets created (or if part of it's still somehow sourced externally)

The tileserver is started with a config - see services/tileserver/configure_run.sh

The config contents includes a list of data sources. The default one for us is the planetiler generated mbtiles.

/tileserver/data/default.json is a dynamically generated response from the tileserver based on the contents of that mbtiles.

[michaelkirk]

exactly what's the role of https://github.com/headwaymaps/headway-data and how were those files sourced/generated?

I've added a README just now: https://github.com/headwaymaps/headway-data

I don't think it's related to this issue.

Note how the parks layer (from style.json) is from the openmaptiles source, which, if you go to your data definitions at the top of the style file, comes from the "default" mbtiles (aka the planetiler generated vector set), not the terrain or landcover sources.

[3nprob]

@michaelkirk Thanks for the pointers!

Note how the parks layer (from style.json) is from the openmaptiles source, which, if you go to your data definitions at the top of the style file, comes from the "default" mbtiles (aka the planetiler generated vector set), not the terrain or landcover sources.

I'm with you so far but where does the planetiler instance get those assets from? Looking inside https://data.maps.earth.example.com/planetiler_fixtures/sources.tar:

$ wcurl https://data.maps.earth.example.com/planetiler_fixtures/sources.tar
$ tar -tf sources.tar
lake_centerline.shp.zip
natural_earth_vector.sqlite.zip
water-polygons-split-3857.zip

I don't spot anything that seems related inside those zip files.

[ellenhp]

It seems like "park" in openmpatiles might have a much narrower meaning than what people would colloquially consider parks to be. https://github.com/openmaptiles/openmaptiles/blob/master/layers/park/mapping.yaml#L98-L103

Anything tagged leisure=park instead seems to be mapped to the "landcover" layer: https://github.com/openmaptiles/openmaptiles/blob/master/layers/landcover/mapping.yaml

@3nprob does your OSM extract have anything that would get mapped to "park", like a nature reserve or something? If not, something (planetiler possibly?) could be pruning the entire layer from the schema.

[3nprob]

Thanks, I think you're on to it @ellenhp !

does your OSM extract have anything that would get mapped to "park", like a nature reserve or something? If not, something (planetiler possibly?) could be pruning the entire layer from the schema.

Looking with osmium show MyArea.osm.pbf, I see some leisure=park but no leisure=nature_reserve, boundary=national_park or boundary=protected_area.

Now to figure out how to use forked openmaptiles

[ellenhp]

IMO if you want to silence the error it makes the most sense to figure out how to prevent planetiler from pruning the layer, rather than making any schema changes.