Multi-Robot Semantic Mapping Fusion Experiments

📢 Accepted at the 13th International Conference of Control, Dynamic Systems, and Robotics (CDSR 2026) 📅 June 07 - 09, 2026 | 📍 Barcelona, Spain

A comprehensive research platform for evaluating multi-robot semantic mapping fusion strategies in outdoor environments. This repository contains simulation environments, mapping pipelines, and experimental data for comparing early, mid, and late fusion approaches in heterogeneous robot teams.

🎥 Demo Video

Click the image above to watch the demonstration video

📋 Table of Contents

• Overview
• Key Features
• Fusion Strategies
• Custom ROS2 Packages
• Citation
• Contact
• License
• Acknowledgments

🔍 Overview

This project investigates collaborative semantic mapping for multi-robot systems operating in complex outdoor environments (forests, unstructured terrain). The system integrates:

• Heterogeneous Robot Simulation (UAVs + UGVs) using Gazebo Fortress
• Real-time Semantic Segmentation from RGB-D sensors
• Point Cloud Compression using Cloudini library
• Probabilistic 3D Mapping with SSMAP/DSSMAP (Sufficient Statistics Mapping)
• Multi-robot Map Fusion with temporal-geometric consistency

The experiments compare three fusion architectures:

1. Early Fusion: Raw semantic point clouds exchanged between robots
2. Mid Fusion: Compressed point clouds with pose information
3. Late Fusion: Incremental binary map differences

✨ Key Features

Simulation Environment

• ✅ OHMS Simulator: Outdoor Heterogeneous Multi-Robot System simulator with Gazebo Fortress
• ✅ Semantic World Models: Annotated forest environments with 10+ semantic classes
• ✅ Multi-robot Support: 2+ robots (UAV/UGV configurations)
• ✅ Ground Truth Generation: World query system for semantic label validation

Semantic Mapping Pipeline

• ✅ Semantic Point Cloud Generation: Fusion of depth + segmentation data
• ✅ Cloudini Compression: 5-10x compression with minimal accuracy loss
• ✅ SSMAP/DSSMAP: Beta-Dirichlet probabilistic occupancy + semantic modeling
• ✅ Real-time Visualization: RViz markers with confidence-based coloring
• ✅ Binary Map Serialization: Efficient LZ4-compressed map format

Experimental Analysis

• ✅ Bandwidth Utilization: Message size comparison across fusion strategies
• ✅ Memory Consumption: CPU/RAM profiling with rotop system monitor
• ✅ Map Quality Metrics: Semantic accuracy, occupancy precision
• ✅ Jupyter Notebooks: Comprehensive data visualization and statistical analysis

🔄 Fusion Strategies

Early Fusion

Data Exchanged: Raw semantic point clouds (PointCloud2 + pose)

Advantages:

• Full spatial resolution preserved
• Maximum information content
• Flexible downstream processing

Disadvantages:

• High bandwidth (~0.7 MB per message)
• Network congestion in multi-robot scenarios
• Large storage requirements

Use Case: High-bandwidth local networks, maximum accuracy required

---

Mid Fusion

Data Exchanged: Cloudini-compressed point clouds (CompressedPointCloud2 + pose)

Advantages:

• 80-90% bandwidth reduction (~0.09 MB per message)
• Configurable compression-accuracy tradeoff
• Maintains point-level granularity
• Fast compression/decompression (<5 ms)

Disadvantages:

• Lossy spatial quantization (configurable resolution)
• Additional processing overhead
• Still point-cloud based (not map-based)

Use Case: Bandwidth-limited networks, moderate compression needs

---

Late Fusion

Data Exchanged: Incremental binary map differences (DssMapBinary)

Advantages:

• Minimal bandwidth (~0.03-0.05 MB incremental updates)
• Map-level representation already computed
• Temporal-geometric fusion algorithms
• Efficient for static environments

Disadvantages:

• Loses point-level information
• Requires map differencing logic
• More complex fusion algorithms
• Less flexible for downstream tasks

Use Case: Very low bandwidth, long-term deployment, static scenes

📦 Custom ROS2 Packages

All custom packages are included in this repository:

• cloudini: Point cloud compression (GitHub)
• sempcl: Semantic point cloud generation
• dssmap: Distributed Sufficient Statistics Mapping
• ohms_sim: Outdoor Heterogeneous Multi-Robot System simulator
• point_cloud_interfaces: Custom message definitions

📄 Citation

If you use this work in your research, please cite:

@inproceedings{boralessa2026dssmap,
  title     = {{DSS-MAP}: Distributed Metric--Semantic Mapping with Conjugate Sufficient Statistics and Gossip Consensus},
  author    = {Boralessa, Kalhan and Adama, David and Lotfi, Ahmad and Ferreira, Jo{\~a}o Filipe},
  booktitle = {Proceedings of the 13th International Conference of Control, Dynamic Systems, and Robotics (CDSR 2026)},
  address   = {Barcelona, Spain},
  month     = jun,
  year      = {2026},
  note      = {Paper No. XXX, DOI: TBA}
}

📧 Contact

For questions or collaboration inquiries:

• Organization: RAISE-NTU Group CIA Lab NTU
• Repository: github.com/RAISE-NTU/fusion_experiment

📝 License

This project is licensed under the Apache 2.0 License - see individual package LICENSE files for details.

🙏 Acknowledgments

• Cloudini Library: Davide Faconti (GitHub)
• Bonxai Voxel Grid: Efficient sparse 3D data structure
• ROS2 Community: Open-source robotics framework
• Gazebo Simulator: 3D robot simulation environment

---

Last Updated: November 2025
Maintained by: RAISE-NTU Group CIA Lab NTU