Aleatoric and Epistemic Uncertainty in Conformal Prediction

Sale, Yusuf; Javanmardi, Alireza und Hüllermeier, Eyke (2025): Aleatoric and Epistemic Uncertainty in Conformal Prediction. Symposium on Conformal and Probabilistic Prediction with Applications, London, UK, 10. - 12. May 2025. Nguyen, Khuong An; Luo, Zhiyuan; Papadopoulos, Harris; Löfström, Tuwe; Carlsson, Lars und Boström, Henrik (Hrsg.): Bd. 266 PMLR. S. 784-786

Abstract

Recently, there has been a particular interest in distinguishing different types of uncertainty in supervised machine learning (ML) settings (Hullermeier and Waegeman, 2021). Aleatoric uncertainty captures the inherent randomness in the data-generating process. As it represents variability that cannot be reduced even with more data, it is often referred to as irreducible uncertainty. In contrast, epistemic uncertainty arises from a lack of knowledge about the underlying data-generating process, which–in principle–can be reduced by acquiring additional data or improving the model itself (viz. reducible uncertainty). In parallel, interest in conformal prediction (CP)–both its theory and applications–has become equally vigorous. Conformal Prediction (Vovk et al., 2005) is a model-agnostic framework for uncertainty quantification that provides prediction sets or intervals with rigorous statistical coverage guarantees. Notably, CP is distribution-free and makes only the mild assumption of exchangeability. Under this assumption, it yields prediction intervals that contain the true label with a user-specified probability. Thus, CP is seen as a promising tool to quantify uncertainty. But how is it related to aleatoric and epistemic uncertainty? In particular, we first analyze how (estimates of) aleatoric and epistemic uncertainty enter into the construction of vanilla CP–that is, how noise and model error jointly shape the global threshold. We then review “uncertainty-aware” extensions that integrate these uncertainty estimates into the CP pipeline.

• Dokumententyp: Konferenzbeitrag (Paper)
• Fakultät: Mathematik, Informatik und Statistik > Informatik > Künstliche Intelligenz und Maschinelles Lernen
• Themengebiete: 000 Informatik, Informationswissenschaft, allgemeine Werke > 004 Informatik