Resilient Multi-Agent Reinforcement Learning with Adversarial Value Decomposition

Title data

Phan, Thomy ; Belzner, Lenz ; Gabor, Thomas ; Sedlmeier, Andreas ; Ritz, Fabian ; Linnhoff-Popien, Claudia:
Resilient Multi-Agent Reinforcement Learning with Adversarial Value Decomposition.
In: Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 35 (2021) Issue 13 . - pp. 11308-11316.
ISSN 2159-5399
DOI: https://doi.org/10.1609/aaai.v35i13.17348

Project information

Abstract in another language

We focus on resilience in cooperative multi-agent systems, where agents can change their behavior due to udpates or failures of hardware and software components. Current state-of-the-art approaches to cooperative multi-agent reinforcement learning (MARL) have either focused on idealized settings without any changes or on very specialized scenarios, where the number of changing agents is fixed, e.g., in extreme cases with only one productive agent. Therefore, we propose Resilient Adversarial value Decomposition with Antagonist-Ratios (RADAR). RADAR offers a value decomposition scheme to train competing teams of varying size for improved resilience against arbitrary agent changes. We evaluate RADAR in two cooperative multi-agent domains and show that RADAR achieves better worst case performance w.r.t. arbitrary agent changes than state-of-the-art MARL.