Attention-Based Recurrency for Multi-Agent Reinforcement Learning under State Uncertainty

Phan, Thomy; Ritz, Fabian ; Nüßlein, Jonas ; Kölle, Michael; Gabor, Thomas und Linnhoff-Popien, Claudia (2023): Attention-Based Recurrency for Multi-Agent Reinforcement Learning under State Uncertainty. AAMAS 2023: International Conference on Autonomous Agents and Multiagent Systems, London, United Kingdom, 29. Mai - 02. Juni 2023. In: AAMAS '23: Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems, Richland: International Foundation for Autonomous Agents and Multiagent Systems. S. 2839-2841

Abstract

State uncertainty poses a major challenge for decentralized coordination. However, state uncertainty is largely neglected in multi-agent reinforcement learning research due to a strong focus on state-based centralized training for decentralized execution (CTDE) and benchmarks that lack sufficient stochasticity like StarCraft Multi-Agent Challenge (SMAC). In this work, we propose Attention-based Embeddings of Recurrence In multi-Agent Learning (AERIAL) to approximate value functions under agent-wise state uncertainty. AERIAL uses a learned representation of multi-agent recurrence, considering more accurate information about decentralized agent decisions than state-based CTDE. We then introduce MessySMAC, a modified version of SMAC with stochastic observations and higher variance in initial states, to provide a more general and configurable benchmark. We evaluate AERIAL in a variety of MessySMAC maps, and compare the results with state-based CTDE.

• Dokumententyp: Konferenzbeitrag (Paper)
• Keywords: dec-pomdp ; multi-agent learning ; recurrence ; state uncertainty
• Fakultät: Mathematik, Informatik und Statistik > Informatik
• Themengebiete: 000 Informatik, Informationswissenschaft, allgemeine Werke > 004 Informatik
• ISBN: 978-1-4503-9432-1
• Ort: Richland