Quantifying Multimodality in World Models

Sedlmeier, Andreas; Kölle, Michael; Müller, Robert ; Baudrexel, Leo und Linnhoff-Popien, Claudia (2022): Quantifying Multimodality in World Models. 14th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART, Vienna, Austria, 03. Februar 2022 - 05. Februar 2022. In: Proceedings of the 14th International Conference on Agents and Artificial Intelligence, Bd. 1 SciTePress. S. 367-374

Abstract

Model-based Deep Reinforcement Learning (RL) assumes the availability of a model of an environment’s underlying transition dynamics. This model can be used to predict future effects of an agent’s possible actions. When no such model is available, it is possible to learn an approximation of the real environment, e.g. by using generative neural networks, sometimes also called World Models. As most real-world environments are stochastic in nature and the transition dynamics are oftentimes multimodal, it is important to use a modelling technique that is able to reflect this multimodal uncertainty. In order to safely deploy such learning systems in the real world, especially in an industrial context, it is paramount to consider these uncertainties. In this work, we analyze existing and propose new metrics for the detection and quantification of multimodal uncertainty in RL based World Models. The correct modelling & detection of uncertain future states lays the foundation for handling critical situations in a safe way, which is a prerequisite for deploying RL systems in real-world settings.

• Dokumententyp: Konferenzbeitrag (Paper)
• Keywords: uncertainty; multimodality; world models; model-based deep reinforcement learning; mixture-density networks
• Themengebiete: 000 Informatik, Informationswissenschaft, allgemeine Werke > 004 Informatik
• ISBN: 978-989-758-547-0; ISSN 2184-433X
• Sprache: Englisch