Actor Design for DRL Agents in Agent-Based Electricity Market Simulations

This repository extends the ASSUME framework with a comparison of different actor designs for deep reinforcement learning (DRL) agents in electricity market simulations, as presented at EEM 2026.

For general ASSUME documentation, installation, and tutorials, see README.md.

---

What This Extension Adds

The paper introduces and benchmarks the following actor architectures for DRL bidding agents:

Actor Architecture	Key idea
MLP (baseline)	Standard multi-layer perceptron
LSTM	Recurrent actor with temporal memory
MHSA	Multi-head self-attention over observation sequences

Action Decoding	Key idea
Price (baseline)	Direct decoding of action to bid price
MC-Factor	Markup factor on marginal cost
D-MC-Factor	Distributed variant of MC-Factor for improved exploration

Day-ahead market (DAM) variants of MLP, LSTM, MHSA, MC-Factor, and D-MC-Factor are also included.

The new strategy classes live in assume/strategies/learning_strategies.py:

• EnergyLearningMCFactorStrategy
• EnergyLearningMCFactorDistributionStrategy
• EnergyLearningDAMStrategy
• EnergyLearningMCFactorDAMStrategy
• EnergyLearningMCFactorDistributionDAMStrategy

---

Installation

Exact paper environment (CUDA 12.1 / HPC — recommended for full reproduction):

git clone https://github.com/<your-username>/assume-eem-actor-design.git
cd assume-eem-actor-design
pip install -r requirements.txt

General use (CPU or any CUDA version):

git clone https://github.com/<your-username>/assume-eem-actor-design.git
cd assume-eem-actor-design
pip install -e .[learning]

---

Running the Paper Cases

All publication scenarios are defined in run_actor_design_cases.py.

python run_actor_design_cases.py

Edit the cases list at the bottom of the file to select which scenario group to run. Results are stored in a local SQLite database by default (examples/local_db/assume_db_actor_design.db). Switch to data_format = "timescale" with docker compose up -d assume_db for PostgreSQL + Grafana dashboards.

Scenario overview

Investigated actor variants are first tested in single-step (hourly) markets, indicated as EOM below. Then, the potential is tested for multi-step bidding (24 hours), being more closely aligned with European day-ahead markets (DAM).

Scenario group	Market	Description
02a	EOM	Single-agent scenario with market power
02b	EOM	Multi-agent (5 RL power plants) scenario with rare joint market power
02c	EOM	Multi-agent (10 RL power plants) scenario without market power, fully competetive
02a_dam, 02b_dam	DAM	Day-ahead market variants of 02a / 02b
example_03a	EOM	Full-year 2019 German reference scenario

---