Can hepo run on multiple gpus?

[Charlie0257]

@williamd4112 @ChangLee0903 Thanks for your great work.

I have some issues about the hepo——

1. Can hepo run on multiple gpus?
2. How to save the checkpoint of hepo during the training?
3. How to load the ckpt to visualize the performance of hepo?

Thanks for any help:)
Best,
Charlie

[ChangLee0903]

Hi @Charlie0257,

Thanks for your interest and for reaching out!

Here are answers to your questions:

1. We haven't run experiments using multiple GPUs since our model is relatively small (just a 3-layer MLP), making multi-GPU training typically unnecessary.

2. You can refer to the function def save located in:

   • hepo/isaacgymenvs/rl_games/algos_torch/hepo_continuous.py, or
   • hepo/isaacgymenvs/rl_games/algos_torch/a2c_continuous.py.

3. We primarily rely on the return values logged during training for most measurements, so you can download the performance curves directly from W&B for visualization purposes. Currently, we don't explicitly save separate visualization files. However, if needed, you can use the saved checkpoint to run additional rollouts and measure the returns for each episode manually.

Feel free to reach out if you have more questions!