Architectural Influence on Variational Quantum Circuits in Multi-Agent Reinforcement Learning: Evolutionary Strategies for Optimization

Kölle, Michael; Schneider, Karola; Egger, Sabrina; Topp, Felix; Phan, Thomy; Altmann, Philipp ; Nüßlein, Jonas und Linnhoff-Popien, Claudia (2025): Architectural Influence on Variational Quantum Circuits in Multi-Agent Reinforcement Learning: Evolutionary Strategies for Optimization. ICAART 2024: International Conference on Agents and Artificial Intelligence, Rome, Italy, 24. - 26. Februar 2024. Rocha, Ana Paula; Steels, Luc und Herik, Jaap van den (Hrsg.): In: Agents and Artificial Intelligence: 16th International Conference, ICAART 2024, Bd. 1 Cham: Springer Nature Switzerland. S. 50-79

Abstract

In recent years, Multi-Agent Reinforcement Learning (MARL) has found application in numerous areas of science and industry, such as autonomous driving, telecommunications, and global health. Nevertheless, MARL suffers from, for instance, an exponential growth of dimensions. Inherent properties of quantum mechanics help to overcome these limitations, e.g., by significantly reducing the number of trainable parameters. Previous studies have developed an approach that uses gradient-free quantum Reinforcement Learning and evolutionary optimization for variational quantum circuits (VQCs) to reduce the trainable parameters and avoid barren plateaus as well as vanishing gradients. This leads to a significantly better performance of VQCs compared to classical neural networks with a similar number of trainable parameters and a reduction in the number of parameters by more than 97% compared to similarly good neural networks. We extend an approach of Kölle et al. by proposing a Gate-Based, a Layer-Based, and a Prototype-Based concept to mutate and recombine VQCs. Our results show the best performance for mutation-only strategies and the Gate-Based approach. In particular, we observe a significantly better score, higher total and own collected coins, as well as a superior own coin rate for the best agent when evaluated in the Coin Game environment.

• Dokumententyp: Konferenzbeitrag (Paper)
• Keywords: Quantum reinforcement learning ; Multi-agent systems ; Evolutionary optimization ; Variational quantum circuits ; Architecture search
• Fakultät: Mathematik, Informatik und Statistik > Informatik
• Themengebiete: 000 Informatik, Informationswissenschaft, allgemeine Werke > 004 Informatik
• Ort: Cham
• Sprache: Englisch