Conformal Prediction without Nonconformity Scores

Hanselle, Jonas ; Javanmardi, Alireza; Oberkofler, Tobias Florin; Sale, Yusuf und Hüllermeier, Eyke (22. Juli 2025): Conformal Prediction without Nonconformity Scores. Proceedings of the Forty-first Conference on Uncertainty in Artificial Intelligence, Rio de Janeiro, Brazil, 22. - 24. July, 2025. Chiappa, Silvia und Magliacane, Sara (Hrsg.): Bd. 286 PMLR. S. 1626-1639

Abstract

Conformal prediction (CP) is an uncertainty quantification framework that allows for constructing statistically valid prediction sets. Key to the construction of these sets is the notion of a nonconformity function, which assigns a real-valued score to individual data points: only those (hypothetical) data points contribute to a prediction set that sufficiently conform to the data. The point of departure of this work is the observation that CP predictions are invariant against (strictly) monotone transformations of the nonconformity function. In other words, it is only the ordering of the scores that matters, not their quantitative values. Consequently, instead of scoring individual data points, a conformal predictor only needs to be able to compare pairs of data points, deciding which of them is the more conforming one. This suggests an interesting connection between CP and preference learning, in particular learning-to-rank methods, and makes CP amenable to training data in the form of (qualitative) preferences. Elaborating on this connection, we propose methods for preference-based CP and show their usefulness in real-world classification tasks.

• Dokumententyp: Konferenzbeitrag (Paper)
• EU Funded Grant Agreement Number: 101073307
• Fakultät: Mathematik, Informatik und Statistik > Informatik > Künstliche Intelligenz und Maschinelles Lernen
• Themengebiete: 000 Informatik, Informationswissenschaft, allgemeine Werke > 004 Informatik