MCJESP

Monte-Carlo Joint Equilibrium Search for a Policy in Dec-POMDPs

This is a C++ implementation of the MCJESP algorithm [1]. It aims to solve Dec-POMDP infinite-horizon problems by finding Nash-equilibrium solutions, only requiring black-box Dec-POMDP simulators.

[1] Monte-Carlo Search for an Equilibrium in Dec-POMDPs Yang You, Vincent Thomas, Francis Colas and Olivier Buffet. Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence (UAI 2023), 31 Jul-04 Aug, 2023.

Installation

Tested systems:

• Linux
• Mac OS X

Dependencies:

• cmake >= 2.8
• C++ 11 is required

Once you have all required dependencies, you can simply execute the following commands from the project's main folder:

cmake ./
make

Quick Start

The generated executable file is called "RunExperiments". To run the program with Dec-POMDP benchmarks, just type:

./RunExperiments

Help Manual is also provided by typing:

./RunExperiments -h

or

./RunExperiments --help

A cleaning shell script is written for cleaning the files generated by CMake. To execute it, just run：

sh cleanCMAKE.sh

The result policy files are located in the "./TempFiles" folder which contains all the final policies at each restart.

Model and solve a custom Dec-POMDP problem

To model and solve a custom Dec-POMDP problem, please apply the following two steps:

1. Model your custom Dec-POMDP problem by inheriting the SimInterface class

   • This file is located at "./Include/SimInterface.h".
   • If there is an explicit Dec-POMDP file, you can also use SimModel (".Include/SimModel.h") to load it, it will be then read as a SimInterface object.

2. Load your custom Dec-POMDP and call MCJESP to solve it

   • A demo main file is provided at "./demo_main.cpp".

Reference

@InProceedings{pmlr-v216-you23a,
  title = 	 {Monte-{C}arlo Search for an Equilibrium in {Dec-POMDPs}},
  author =       {You, Yang and Thomas, Vincent and Colas, Francis and Buffet, Olivier},
  booktitle = 	 {Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence},
  pages = 	 {2444--2453},
  year = 	 {2023},
  volume = 	 {216},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {31 Jul--04 Aug},
  publisher =    {PMLR}
}