Pitfalls of Epistemic Uncertainty Quantification through Loss Minimisation

Bengs, Viktor ; Hüllermeier, Eyke und Waegeman, Willem (28. November 2022): Pitfalls of Epistemic Uncertainty Quantification through Loss Minimisation. Thirty-eighth Annual Conference on Neural Information Processing Systems (NeurIPS 2024), New Orleans, USA, 28 November - 9 December 2022.

Abstract

Uncertainty quantification has received increasing attention in machine learning in the recent past. In particular, a distinction between aleatoric and epistemic uncertainty has been found useful in this regard. The latter refers to the learner's (lack of) knowledge and appears to be especially difficult to measure and quantify. In this paper, we analyse a recent proposal based on the idea of a second-order learner, which yields predictions in the form of distributions over probability distributions. While standard (first-order) learners can be trained to predict accurate probabilities, namely by minimising suitable loss functions on sample data, we show that loss minimisation does not work for second-order predictors: The loss functions proposed for inducing such predictors do not incentivise the learner to represent its epistemic uncertainty in a faithful way.

• Dokumententyp: Konferenzbeitrag (Paper)
• Fakultät: Mathematik, Informatik und Statistik > Informatik > Künstliche Intelligenz und Maschinelles Lernen
• Themengebiete: 000 Informatik, Informationswissenschaft, allgemeine Werke > 000 Informatik, Wissen, Systeme
• Sprache: Englisch