Quantifying Aleatoric and Epistemic Uncertainty: A Credal Approach

Hofman, Paul; Sale, Yusuf und Hüllermeier, Eyke (26. July 2024): Quantifying Aleatoric and Epistemic Uncertainty: A Credal Approach. ICML 2024 Workshop on Structured Probabilistic Inference & Generative Modeling (SPIGM), Vienna, Austria, 26. July 2024. Vienna, Austria:

Abstract

Uncertainty representation and quantification are paramount in machine learning, especially in safety-critical applications. In this paper, we propose a novel framework for the quantification of aleatoric and epistemic uncertainty based on the notion of credal sets, i.e., sets of probability distributions. Thus, we assume a learner that produces (second-order) predictions in the form of sets of probability distributions on outcomes. Practically, such an approach can be realized by means of ensemble learning: Given an ensemble of learners, credal sets are generated by including sufficiently plausible predictors, where plausibility is measured in terms of (relative) likelihood. We provide a formal justification for the framework and introduce new measures of epistemic and aleatoric uncertainty as concrete instantiations. We evaluate these measures both theoretically, by analysing desirable axiomatic properties, and empirically, by comparing them in terms of performance and effectiveness to existing measures of uncertainty in an experimental study.

• Item Type: Conference or Workshop Item (Paper)
• Faculties: Mathematics, Computer Science and Statistics > Computer Science > Artificial Intelligence and Machine Learning
• Subjects: 000 Computer science, information and general works > 004 Data processing computer science
• Place of Publication: Vienna, Austria