Skip to content

Add sim detection publisher for automated training data collection - #518

Open
Br-Charb wants to merge 3 commits into
mainfrom
br-charb/automated-training-data
Open

Add sim detection publisher for automated training data collection#518
Br-Charb wants to merge 3 commits into
mainfrom
br-charb/automated-training-data

Conversation

@Br-Charb

Copy link
Copy Markdown
Contributor

Closes #491

What this does

Adds sim_detection_publisher.py — a ROS2 node that projects the 3D positions of competition elements into 2D bounding boxes on the front camera image, enabling two things:

  1. Training data collection (collect_data:=true): as the sub moves through sim, automatically saves each camera frame as a PNG alongside a YOLO .txt label file to ~/sim_training_data/
  2. Sim missions without YOLO (publish_detections:=true): publishes ground-truth DetectionArray on /yolo/detections at 30Hz so missions can run in sim without a trained model

How to run

Step 1 — launch the sim as normal:

ros2 launch subjugator_bringup gazebo.launch.py world:=robosub_2025.world

Step 2 — in a new terminal, run the node:

source install/setup.bash && export RMW_IMPLEMENTATION=rmw_fastrtps_cpp

ros2 run subjugator_vision sim_detection_publisher.py --ros-args \
  -p collect_data:=true \
  -p publish_detections:=false

Drive the sub around in sim and annotated images will save automatically to ~/sim_training_data/.

World configs

Each world file has a matching yaml config with the correct object positions. Make sure the world you launch matches the config you pass:

World Config
robosub_2025.world sim_objects.yaml (default, no need to pass)
task1_2026.world sim_objects_task1_2026.yaml
task4_2026_v1.world sim_objects_task4_2026_v1.yaml
task4_2026_v2.world sim_objects_task4_2026_v2.yaml

To use a non-default config:

ros2 run subjugator_vision sim_detection_publisher.py --ros-args \
  -p collect_data:=true \
  -p config_file:=/path/to/mil2/src/subjugator/gnc/subjugator_vision/config/sim_objects_task4_2026_v1.yaml

Files changed

  • scripts/sim_detection_publisher.py — main node
  • config/sim_objects.yaml — object positions for robosub_2025.world
  • config/sim_objects_task1_2026.yaml — start gate for task1_2026.world
  • config/sim_objects_task4_2026_v1.yaml — torpedo board for task4_2026_v1.world
  • config/sim_objects_task4_2026_v2.yaml — torpedo board for task4_2026_v2.world
  • CMakeLists.txt / package.xml — install the new script and configs
  • subjugator_bridge.yaml — added camera_info bridge entries for both cameras

@Br-Charb

Br-Charb commented Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

Finished automation of training data collection. Altered the images taken per second of use and added the automation script to generate the yaml files from the world file. Next step is to wait for review and possibly use this data to train the YOLO model.
image

@wingdeans

Copy link
Copy Markdown
Contributor
image

Unfortunately these bounding boxes aren't tight enough to train on. I see you've committed the yaml publisher. Not sure but I think this could be done with better by using the Gazebo Scene message instead of parsing the urdf. If it's not too much hassle you could also get the Gazebo message directly instead of through the ROS bridge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

automatically collect training data from sim

2 participants