Specification Aware Multi-Agent Reinforcement Learning

Ritz, Fabian ; Phan, Thomy; Müller, Robert ; Gabor, Thomas; Sedlmeier, Andreas; Zeller, Marc; Wieghardt, Jan; Schmid, Reiner; Sauer, Horst; Klein, Cornel and Linnhoff-Popien, Claudia (2022): Specification Aware Multi-Agent Reinforcement Learning. 13th International Conference, ICAART 2021, Virtual Event, February 4–6, 2021. Rocha, Ana Paula; Steels, Luc and Herik, Jaap van den (eds.) : In: Agents and Artificial Intelligence. 13th International Conference, ICAART 2021, Virtual Event, February 4–6, 2021, Revised Selected Papers, Lecture Notes in Computer Science Vol. 13251 Cham: Springer. pp. 3-21

Abstract

Engineering intelligent industrial systems is challenging due to high complexity and uncertainty with respect to domain dynamics and multiple agents. If industrial systems act autonomously, their choices and results must be within specified bounds to satisfy these requirements. Reinforcement learning (RL) is promising to find solutions that outperform known or handcrafted heuristics. However in industrial scenarios, it also is crucial to prevent RL from inducing potentially undesired or even dangerous behavior. This paper considers specification alignment in industrial scenarios with multi-agent reinforcement learning (MARL). We propose to embed functional and non-functional requirements into the reward function, enabling the agents to learn to align with the specification. We evaluate our approach in a smart factory simulation representing an industrial lot-size-one production facility, where we train up to eight agents using DQN, VDN, and QMIX. Our results show that the proposed approach enables agents to satisfy a given set of requirements.

• Item Type: Conference or Workshop Item (Paper)
• Faculties: Mathematics, Computer Science and Statistics > Computer Science
• Subjects: 000 Computer science, information and general works > 004 Data processing computer science
• ISSN: 0302-9743
• Place of Publication: Cham
• Language: English