Logo Logo
Switch Language to German

Bengs, Viktor ORCID logoORCID: https://orcid.org/0000-0001-6988-6186; Hüllermeier, Eyke ORCID logoORCID: https://orcid.org/0000-0002-9944-4108 and Waegeman, Willem ORCID logoORCID: https://orcid.org/0000-0002-5950-3003 (2023): On Second-Order Scoring Rules for Epistemic Uncertainty Quantification. 40th International Conference on Machine Learning (ICML 2023), Hawaii, USA, 23-29 July, 2023. Krause, Andreas; Brunskill, Emma; Cho, Kyunghyun; Engelhardt, Barbara; Sabato, Sivan and Scarlett, Jonathan (eds.) : In: Proceedings of the 40th International Conference on Machine Learning, Vol. 202 PMLR. pp. 2078-2091

Full text not available from 'Open Access LMU'.


It is well known that accurate probabilistic predictors can be trained through empirical risk minimisation with proper scoring rules as loss functions. While such learners capture so-called aleatoric uncertainty of predictions, various machine learning methods have recently been developed with the goal to let the learner also represent its epistemic uncertainty, i.e., the uncertainty caused by a lack of knowledge and data. An emerging branch of the literature proposes the use of a second-order learner that provides predictions in terms of distributions on probability distributions. However, recent work has revealed serious theoretical shortcomings for second-order predictors based on loss minimisation. In this paper, we generalise these findings and prove a more fundamental result: There seems to be no loss function that provides an incentive for a second-order learner to faithfully represent its epistemic uncertainty in the same manner as proper scoring rules do for standard (first-order) learners. As a main mathematical tool to prove this result, we introduce the generalised notion of second-order scoring rules.

Actions (login required)

View Item View Item