Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
ebe5023
feat(crazy-robotaxi): add semantic map pipeline
aidanfnv Aug 14, 2026
53800fb
fix(crazy-robotaxi): use full authored map for fares
aidanfnv Aug 14, 2026
59d6827
fix(crazy-robotaxi): render authored road dividers
aidanfnv Aug 14, 2026
6ce5f66
tune(crazy-robotaxi): widen starter map lanes
aidanfnv Aug 14, 2026
c3a3d81
refactor(crazy-robotaxi): separate lanes from curbs
aidanfnv Aug 15, 2026
f40287f
tune(crazy-robotaxi): strengthen curb rebound
aidanfnv Aug 15, 2026
89e4d4b
fix(crazy-robotaxi): clarify automotive road conditioning
aidanfnv Aug 15, 2026
6cdd750
tune(crazy-robotaxi): retain momentum on curb glances
aidanfnv Aug 15, 2026
f9b1d3b
feat(crazy-robotaxi): add parking destination elements
aidanfnv Aug 15, 2026
7f87722
fix(crazy-robotaxi): prevent overlapping map elements
aidanfnv Aug 15, 2026
849f704
feat(crazy-robotaxi): recreate boulevard district
aidanfnv Aug 17, 2026
299d194
fix(crazy-robotaxi): encode parking lots as roadnet masks
aidanfnv Aug 17, 2026
efb3fa2
feat(crazy-robotaxi): expand boulevard district
aidanfnv Aug 17, 2026
bc7c861
fix(crazy-robotaxi): disambiguate parking lots
aidanfnv Aug 17, 2026
bda9079
feat(crazy-robotaxi): externalize map renderer and game settings
aidanfnv Aug 17, 2026
733afc7
fix(crazy-robotaxi): hydrate HUD renderer arguments
aidanfnv Aug 17, 2026
1605609
feat(crazy-robotaxi): expand eastern map district
aidanfnv Aug 17, 2026
a622648
feat(crazy-robotaxi): recreate eastern map district
aidanfnv Aug 17, 2026
6000c7d
fix(crazy-robotaxi): trace original map topology
aidanfnv Aug 17, 2026
4570780
feat(game-map): use node graph authoring
aidanfnv Aug 18, 2026
ceefbd6
fix(game-map): stitch node geometry cleanly
aidanfnv Aug 18, 2026
1d3d4bf
refactor(game-map): make YAML maps authoritative
aidanfnv Aug 19, 2026
ade91fb
refactor(game-ui): remove map picker
aidanfnv Aug 19, 2026
cc2711b
refactor(game-engine): remove inherited out-of-bounds behavior
aidanfnv Aug 19, 2026
7d1c538
Fix map joins and clarify curve schema
aidanfnv Aug 19, 2026
aa09367
feat(game-map): add author-friendly road paths
aidanfnv Aug 19, 2026
af5a4fd
fix(game-map): infer intersection footprints
aidanfnv Aug 19, 2026
87fbaec
fix(game-map): separate road boundaries from curbs
aidanfnv Aug 19, 2026
81d5c20
feat(game-map): add curve-length road joints
aidanfnv Aug 19, 2026
bbce78e
feat(game-map): infer parking lot accesses
aidanfnv Aug 19, 2026
79bdf50
feat(game-map): recreate boulevard district
aidanfnv Aug 20, 2026
f8cf888
Tighten up minimal demo map
aidanfnv Aug 20, 2026
6e87115
Correct lot distance
aidanfnv Aug 20, 2026
38e5f29
feat(game-map): add node-owned lane transitions
aidanfnv Aug 20, 2026
06bef5d
feat(game-map): build node footprints from boundaries
aidanfnv Aug 20, 2026
3fb4194
feat(game-map): infer compact road joint curves
aidanfnv Aug 20, 2026
89d2fe7
refactor(game-map): simplify node poses and curb defaults
aidanfnv Aug 20, 2026
80a1364
fix(game-map): reject underspecified intersections
aidanfnv Aug 21, 2026
8734512
feat(game-map): render missing spawn images
aidanfnv Aug 21, 2026
cf6f4c4
feat(game-map): add authored NPC traffic
aidanfnv Aug 21, 2026
9abf8ea
feat(game-map): generate traffic to requested count
aidanfnv Aug 21, 2026
4c48386
Activate map actors by graph vicinity
aidanfnv Aug 21, 2026
2b7817b
Index map vicinity polygon lookups
aidanfnv Aug 21, 2026
fc79f01
Expand map actor vicinity symmetrically
aidanfnv Aug 22, 2026
80871cf
Crop screenshot
aidanfnv Aug 22, 2026
502563c
Improve world-model driving motion recovery
aidanfnv Aug 22, 2026
7907830
Remove synthetic model-view recovery
aidanfnv Aug 22, 2026
22c91ba
More traffic, use screenshot
aidanfnv Aug 24, 2026
9f9c100
Unify map traffic runtime state
aidanfnv Aug 24, 2026
d1ac4a6
Anchor active traffic to physical progress
aidanfnv Aug 24, 2026
0bda097
Smooth NPC intersection turns
aidanfnv Aug 24, 2026
6d35ad1
Move map scenarios into test fixtures
aidanfnv Aug 25, 2026
f92deda
Optimize active traffic route projection
aidanfnv Aug 25, 2026
e6ab3f7
Bound NPC collision recovery settling
aidanfnv Aug 25, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
114 changes: 104 additions & 10 deletions apps/crazy_robotaxi/README.md
Original file line number Diff line number Diff line change
@@ -1,34 +1,128 @@
# Crazy Robotaxi

Crazy Robotaxi is a standalone game built on `omnidreams-game-engine` and the
legacy OmniDreams inference session. It does not import or modify the
Interactive Drive demo.
OmniDreams inference session.

Launch the native game:

```bash
flashdreams-run crazy-robotaxi
```

Select the bundled performance manifest and load the scene immediately:
Select the bundled performance manifest:

```bash
flashdreams-run crazy-robotaxi \
--world-model-manifest example_world_model_perf.yaml \
--auto-start True
--world-model-manifest example_world_model_perf.yaml
```

`flashdreams-run` reserves `--manifest` for its launch-manifest format. Use
`--world-model-manifest` for the legacy OmniDreams model manifest, or use the
dedicated `crazy-robotaxi` executable below. Runner booleans use explicit
`True` / `False` values because the shared FlashDreams CLI disables implicit
boolean flag conversion.
`--world-model-manifest` for the OmniDreams model manifest. Runner
booleans use explicit `True` / `False` values because the shared FlashDreams
CLI disables implicit boolean flag conversion.

The dedicated entry point exposes the complete legacy option surface:
The dedicated entry point exposes the application options:

```bash
crazy-robotaxi --help
```

Use `--stream-mjpeg HOST:PORT` with either entry point to run the browser HUD
instead of opening a local Vulkan window.

## Configuration files

The standalone game keeps its portable configuration in three independent,
strict YAML documents:

- `*.robotaxi.yaml` describes one map, including topology, geometry, profiles,
compiler settings, spawns, and visual seed variants.
- `default_renderer.yaml` describes primary-camera and BEV rendering.
- `default_game.yaml` describes rules, scoring, controls, taxi dimensions, and
arcade physics.

The packaged renderer and game files are used when no path is supplied. Select
edited copies independently:

```bash
flashdreams-run crazy-robotaxi \
--renderer-config /path/to/renderer.yaml \
--game-config /path/to/game.yaml
```

Existing explicit CLI tuning flags override YAML values. The world-model
manifest remains separate because it configures inference rather than the map,
renderer, or game. All three YAML formats reject missing and unknown fields.

## Node-graph game maps

Crazy Robotaxi maps use schema version 1 of the engine's node-graph format.
Roads are topological edges; intersections, road joints, cul-de-sacs, parking
lots, and driveways are explicitly posed graph nodes. Road joints provide
minimal tangent-continuous degree-two bends and own lane-count or lane-width
tapers between otherwise uniform road edges. Intersections have at least three
road arms and can apply the same taper to each inferred through-road pair
independently. Node footprints follow the bearings and centerline tangents of
the roads connected to them.

The complete, authoritative format is documented in
[`../omnidreams_game_engine/NODE_GRAPH_MAP_FORMAT.md`](../omnidreams_game_engine/NODE_GRAPH_MAP_FORMAT.md).
The bundled `minimal_loop.robotaxi.yaml` is a compact working example, while
`boulevard_district.robotaxi.yaml` recreates the original scene's surface-street
layout at its source scale, including the curved arterial split, neighborhood
grid, eastern commercial loops, cul-de-sacs, and parking lots. The elevated
highway and its on-ramps in the northwest are intentionally omitted.

Select a map with either entry point:

```bash
flashdreams-run crazy-robotaxi \
--map /path/to/city.robotaxi.yaml
```

Validate a map or produce top-down and spawn-camera previews without loading a
model:

```bash
crazy-robotaxi-map validate /path/to/city.robotaxi.yaml
crazy-robotaxi-map preview /path/to/city.robotaxi.yaml --output city.svg
crazy-robotaxi-map preview-spawn /path/to/city.robotaxi.yaml \
--spawn taxi_start --output taxi_start.png
```

The engine validates the topology, compiles roads and inferred parking-access
geometry, creates curb colliders, and derives directed navigation lanes. That
lane graph powers fare reachability and future NPC routing; it does not limit
where the player's taxi may physically drive. Parking-lot fare points are
sampled anywhere inside the authored polygon, while their route distance stops
at the connected driveway or intersection.

The engine's private ClipGT archive is cached under
`$FLASHDREAMS_CACHE_DIR/omnidreams-game-engine/game-maps/`. The cache key includes the YAML,
resolved geometry, compiler implementation, and referenced seed assets, so
map or compiler edits rebuild it at the next load. The archive is runtime
output, not an authoring format. Pass `--force-map-recompile` when launching
the game to rebuild each selected map once for that process even if its cache
entry is valid. The standalone equivalent is:

```bash
crazy-robotaxi-map compile /path/to/city.robotaxi.yaml --force-map-recompile
```

Parking-lot surfaces are emitted as green `ROI_POLYGON_ROADNET_MASK` regions.
Parking-space dividers remain intentionally omitted because ClipGT would encode
them as ordinary lane lines, which can condition the model to produce bike or
turn lanes. Lots do not contain inferred navigation aisles or turnarounds.

Each visual variant may omit `image`. In that case, map compilation and scene
selection use a deterministic synthetic view projected from the spawn through
the runtime front camera. It aligns roads, boundaries, curbs, and markings but
does not synthesize buildings, vegetation, traffic, or other scenery. Authors
can inspect that fallback with `preview-spawn` before choosing or generating a
checked-in image.

The bundled maps reuse the existing OmniDreams seed image. Their authored
geometry is not expected to match that image exactly, so the first generated
frames may visibly adjust toward the selected semantic map. Static prompts
describe visual setting and atmosphere rather than duplicating map topology;
the BEV remains the source of truth as maps change.
12 changes: 12 additions & 0 deletions apps/crazy_robotaxi/crazy_robotaxi/alignment_diagnostics.py
Original file line number Diff line number Diff line change
Expand Up @@ -267,6 +267,8 @@ def _frame_telemetry(frame: PresentedFrame, sequence: int) -> dict[str, object]:
physx_position = (
np.full(3, np.nan, dtype=np.float32) if debug is None else debug.ego_position_m
)
command = frame.driver_command
motion = frame.model_motion_metrics or {}
return {
"sequence": sequence,
"timestamp_us": int(frame.timestamp_us),
Expand Down Expand Up @@ -295,6 +297,16 @@ def _frame_telemetry(frame: PresentedFrame, sequence: int) -> dict[str, object]:
state.x_m - float(physx_position[0]),
state.y_m - float(physx_position[1]),
),
"command_throttle": math.nan if command is None else command.throttle,
"command_brake": math.nan if command is None else command.brake,
"command_steer": math.nan if command is None else command.steer,
"command_reverse": False if command is None else command.reverse,
"impact_kind": frame.impact_kind or "",
"motion_axis": motion.get("axis", ""),
"motion_mismatched": motion.get("mismatched", False),
"condition_motion_px": motion.get("condition_component_px", math.nan),
"generated_motion_px": motion.get("generated_component_px", math.nan),
"motion_check_ms": motion.get("elapsed_ms", math.nan),
}


Expand Down
58 changes: 34 additions & 24 deletions apps/crazy_robotaxi/crazy_robotaxi/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@

import argparse
from dataclasses import replace
from functools import partial
from pathlib import Path
from typing import Any

Expand All @@ -31,8 +32,8 @@
)
from omnidreams_game_engine.backends.base import RenderBackend
from omnidreams_game_engine.config import AppConfig
from omnidreams_game_engine.game_map.vicinity import GameMapVicinityResolver
from omnidreams_game_engine.simulation.ground_snap import GroundSnapper
from omnidreams_game_engine.simulation.map_bounds import MapBounds
from omnidreams_game_engine.types import (
SceneBundle,
TrajectoryChunk,
Expand All @@ -49,6 +50,7 @@
TaxiGameConfig,
TaxiGameController,
)
from crazy_robotaxi.game_settings import load_game_settings
from crazy_robotaxi.high_scores import (
default_high_scores_path,
)
Expand Down Expand Up @@ -130,37 +132,37 @@ def __init__(
self._presenter_config = presenter_config
self._reference_route_world: Any | None = None
self._navigation_lanes: tuple[Any, ...] = ()
self._fare_regions: tuple[Any, ...] = ()
self._vicinity_resolver: GameMapVicinityResolver | None = None
self._ground_snapper: GroundSnapper | None = None
self._map_bounds: MapBounds | None = None
self._enclosure_segments_world = np.empty((0, 2, 3), dtype=np.float32)
self._curb_segments_world = np.empty((0, 2, 3), dtype=np.float32)

def configure_presenter(self, presenter: Any) -> None:
"""Configure application presentation before scene loading."""
configure = getattr(presenter, "configure_taxi_hud", None)
if callable(configure):
configure(self._presenter_config)
configure(self._presenter_config, self._config.vehicle)

def load_scene(self, scene: SceneBundle, map_bounds: MapBounds | None) -> None:
def load_scene(self, scene: SceneBundle) -> None:
"""Accept scene data already loaded by Interactive Drive."""
scene_data = load_scene_data(scene)
self._reference_route_world = scene_data.reference_route_world
self._navigation_lanes = scene_data.navigation_lanes
self._enclosure_segments_world = scene_data.enclosure_segments_world
self._ground_snapper = _build_taxi_ground_snapper(scene)
self._map_bounds = map_bounds
self._fare_regions = scene_data.fare_regions
assert scene.game_map is not None
self._vicinity_resolver = GameMapVicinityResolver(scene.game_map)
self._curb_segments_world = scene_data.curb_segments_world
self._ground_snapper = _build_taxi_ground_snapper(scene, self._config)
logger.info(
"[crazy-robotaxi] play-area enclosure: perimeter_segments={}",
len(scene_data.perimeter_segments_world),
"[crazy-robotaxi] compiled curb segments={}",
len(scene_data.curb_segments_world),
)

def configure_scene_presenter(self, presenter: Any, scene: SceneBundle) -> None:
"""Publish camera calibration to an application-aware presenter."""
configure = getattr(presenter, "configure_taxi_camera", None)
if callable(configure):
configure(scene.selected_camera)
configure_enclosure = getattr(presenter, "configure_taxi_enclosure", None)
if callable(configure_enclosure):
configure_enclosure(self._enclosure_segments_world)

def rollout_spec(
self,
Expand All @@ -178,8 +180,7 @@ def rollout_spec(
physics_world_factory=lambda active_scene, vehicle: TaxiPhysicsWorld(
active_scene,
vehicle,
traffic_density=self._config.traffic_density,
enclosure_segments_world=self._enclosure_segments_world,
curb_segments_world=self._curb_segments_world,
),
physics_step_fn=step_taxi_physics_world,
visual_flare_enabled=False,
Expand All @@ -198,10 +199,11 @@ def create_runtime(
scene_id=scene.scene_id,
reference_route_world=self._reference_route_world,
navigation_lanes=self._navigation_lanes,
fare_regions=self._fare_regions,
initial_state=simulation.current_state,
config=self._config,
initial_camera=scene.selected_camera,
map_bounds=self._map_bounds,
vicinity_resolver=self._vicinity_resolver,
)
return CrazyRobotaxiRuntime(controller, self._keyboard)

Expand Down Expand Up @@ -256,9 +258,11 @@ def taxi_config_from_args(args: argparse.Namespace) -> TaxiGameConfig:
if args.taxi_highscores is not None
else default_high_scores_path()
)
return TaxiGameConfig(
enabled=True,
traffic_density=float(args.traffic_density),
config = getattr(args, "_game_settings", None) or load_game_settings(
args.game_config
)
return replace(
config,
seed=None if args.taxi_seed is None else int(args.taxi_seed),
high_scores_path=high_scores_path,
alignment_diagnostics_enabled=(
Expand All @@ -267,20 +271,26 @@ def taxi_config_from_args(args: argparse.Namespace) -> TaxiGameConfig:
)


def _build_taxi_ground_snapper(scene: SceneBundle) -> GroundSnapper | None:
def _build_taxi_ground_snapper(
scene: SceneBundle, config: TaxiGameConfig
) -> GroundSnapper | None:
if scene.ground_mesh_vertices is None or scene.ground_mesh_faces is None:
return None
return GroundSnapper(
scene.ground_mesh_vertices,
scene.ground_mesh_faces,
max_absolute_rotation_deg=10.0,
invalid_sample_handler=settle_invalid_ground_attitude,
max_absolute_rotation_deg=config.ground_snap_max_absolute_rotation_deg,
invalid_sample_handler=partial(
settle_invalid_ground_attitude,
settle_fraction=config.ground_snap_settle_fraction,
),
)


def settle_invalid_ground_attitude(state: VehicleState) -> VehicleState:
def settle_invalid_ground_attitude(
state: VehicleState, *, settle_fraction: float = 0.25
) -> VehicleState:
"""Ease stale ground attitude toward level after an invalid Taxi sample."""
settle_fraction = 0.25
pitch = state.pitch_rad * (1.0 - settle_fraction)
roll = state.roll_rad * (1.0 - settle_fraction)
if abs(pitch) < 1.0e-4:
Expand Down
12 changes: 4 additions & 8 deletions apps/crazy_robotaxi/crazy_robotaxi/assets/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# Assets

This directory holds the unpacked-scene-bundle loader
(`scene_bundle.py`) plus the bundled HUD control sprites under
This directory holds the bundled HUD control sprites under
`wheel_and_pedals/`.

## `wheel_and_pedals/`
Expand All @@ -20,10 +19,7 @@ the brake PNGs are also accepted under AlpaSim's `break_*.png`
spelling. When a sprite is missing, the HUD falls back to a
CPU-rendered vector wheel / fill-bar pedals.

## Scenes
## Maps

Scene USDZs themselves are staged into the shared `omnidreams` scene
cache under `$FLASHDREAMS_CACHE_DIR/omnidreams-scenes/`, **not** here.
See `omnidreams.scenes` and `omnidreams-prepare` for how staging
works; both the desktop demo and the WebRTC server consume from the
same cache root.
Crazy Robotaxi's `.robotaxi.yaml` maps live under `crazy_robotaxi/maps/`.
Seed images referenced by a map may be map-relative files or packaged assets.
Loading
Loading