Optimizing Variational Quantum Circuits Using Metaheuristic Strategies in Reinforcement Learning

Kölle, Michael; Seidl, Daniel; Zorn, Maximilian ; Altmann, Philipp ; Stein, Jonas und Gabor, Thomas (2024): Optimizing Variational Quantum Circuits Using Metaheuristic Strategies in Reinforcement Learning. QCE 2024: IEEE International Conference on Quantum Computing and Engineering, Montréal, Canada, 15.- 20. September 2024. Culhane, Candace; Byrd, Greg; Muller, Hausi; Alexev, Yuri und Sheldon, Sarah (Hrsg.): In: Proceedings Volume II of III IEEE Quantum Week 2024, Los Alamitos: IEEE Computer Society. S. 323-328

Abstract

Quantum Reinforcement Learning (QRL) offers potential advantages over classical Reinforcement Learning, such as compact state space representation and faster convergence in certain scenarios. However, practical benefits require further validation. QRL faces challenges like flat solution landscapes, where traditional gradient-based methods are inefficient, necessitating the use of gradient-free algorithms. This work explores the integration of metaheuristic algorithms — Particle Swarm Optimization, Ant Colony Optimization, Tabu Search, Genetic Algorithm, Simulated Annealing, and Harmony Search — into QRL. These algorithms provide flexibility and efficiency in parameter optimization. Evaluations in 5× 5 MiniGrid Reinforcement Learning environments show that, all algorithms yield nearoptimal results, with Simulated Annealing and Particle Swarm Optimization performing best. In the Cart Pole environment, Simulated Annealing, Genetic Algorithms, and Particle Swarm Optimization achieve optimal results, while the others perform slightly better than random action selection. These findings demonstrate the potential of Particle Swarm Optimization and Simulated Annealing for efficient QRL learning, emphasizing the need for careful algorithm selection and adaptation.

• Dokumententyp: Konferenzbeitrag (Paper)
• Keywords: Metaheuristics ; Stability criteria ; Reinforcement learning ; Simulated annealing ; Circuit stability ; Particle swarm optimization ; Quantum circuit ; Robots ; Genetic algorithms ; Testing
• Themengebiete: 000 Informatik, Informationswissenschaft, allgemeine Werke > 004 Informatik
• ISBN: 979-8-3315-4137-8
• Ort: Los Alamitos
• Sprache: Englisch