Second-Order Uncertainty Quantification: A Distance-Based Approach

Sale, Yusuf; Bengs, Viktor ; Caprio, Michele und Hüllermeier, Eyke (July 2024): Second-Order Uncertainty Quantification: A Distance-Based Approach. 41st International Conference on Machine Learning (ICML 2024), Vienna, Austria, 21. - 27. July 2024. In: Proceedings of the 41st International Conference on Machine Learning, Proceedings of Machine Learning Research Vol. 235 PMLR. pp. 43060-43076

Abstract

In the past couple of years, various approaches to representing and quantifying different types of predictive uncertainty in machine learning, notably in the setting of classification, have been proposed on the basis of second-order probability distributions, i.e., predictions in the form of distributions on probability distributions. A completely conclusive solution has not yet been found, however, as shown by recent criticisms of commonly used uncertainty measures associated with second-order distributions, identifying undesirable theoretical properties of these measures. In light of these criticisms, we propose a set of formal criteria that meaningful uncertainty measures for predictive uncertainty based on second-order distributions should obey. Moreover, we provide a general framework for developing uncertainty measures to account for these criteria, and offer an instantiation based on the Wasserstein distance, for which we prove that all criteria are satisfied.

• Item Type: Conference or Workshop Item (Paper)
• Faculties: Mathematics, Computer Science and Statistics > Computer Science > Artificial Intelligence and Machine Learning
• Subjects: 000 Computer science, information and general works > 000 Computer science, knowledge, and systems
• Language: English