Logo Logo
Help
Contact
Switch Language to German

Nguyen, Vu-Linh; Shaker, Mohammad Hossein und Hüllermeier, Eyke ORCID logoORCID: https://orcid.org/0000-0002-9944-4108 (January 2022): How to measure uncertainty in uncertainty sampling for active learning. In: Machine Learning, Vol. 111, No. 1: pp. 89-122 [PDF, 7MB]

[thumbnail of s10994-021-06003-9.pdf]
Preview
Creative Commons Attribution
Published Version
Download (7MB)

Abstract

Various strategies for active learning have been proposed in the machine learning literature. In uncertainty sampling, which is among the most popular approaches, the active learner sequentially queries the label of those instances for which its current prediction is maximally uncertain. The predictions as well as the measures used to quantify the degree of uncertainty, such as entropy, are traditionally of a probabilistic nature. Yet, alternative approaches to capturing uncertainty in machine learning, alongside with corresponding uncertainty measures, have been proposed in recent years. In particular, some of these measures seek to distinguish different sources and to separate different types of uncertainty, such as the reducible (epistemic) and the irreducible (aleatoric) part of the total uncertainty in a prediction. The goal of this paper is to elaborate on the usefulness of such measures for uncertainty sampling, and to compare their performance in active learning. To this end, we instantiate uncertainty sampling with different measures, analyze the properties of the sampling strategies thus obtained, and compare them in an experimental study.

Actions (login required)

View Item View Item