The Residual-based Predictiveness Curve - A Visual Tool to Assess the Performance of Prediction Models

Casalicchio, Giuseppe; Bischl, Bernd; Boulesteix, Anne-Laure und Schmid, Matthias (9. Januar 2015): The Residual-based Predictiveness Curve - A Visual Tool to Assess the Performance of Prediction Models. Department of Statistics: Technical Reports, Nr. 178

Abstract

It is agreed among biostatisticians that prediction models for binary outcomes should satisfy two essential criteria: First, a prediction model should have a high discriminatory power, implying that it is able to clearly separate cases from controls. Second, the model should be well calibrated, meaning that the predicted risks should closely agree with the relative frequencies observed in the data. The focus of this work is on the predictiveness curve, which has been proposed by Huang et al. (Biometrics 63, 2007) as a graphical tool to assess the aforementioned criteria. By conducting a detailed analysis of its properties, we review the role of the predictiveness curve in the performance assessment of biomedical prediction models. In particular, we demonstrate that marker comparisons should not be based solely on the predictiveness curve, as it is not possible to consistently visualize the added predictive value of a new marker by comparing the predictiveness curves obtained from competing models. Based on our analysis, we propose the ``residual-based predictiveness curve'' (RBP curve), which addresses the aforementionened issue and which extends the original method to settings where the evaluation of a prediction model on independent test data is of particular interest. Similar to the predictiveness curve, the RBP curve reflects both the calibration and the discriminatory power of a prediction model. In addition, the curve can be conveniently used to conduct valid performance checks and marker comparisons.

• Dokumententyp: Paper
• Keywords: Calibration, Classification, Discrimination, Predictiveness curve, Risk prediction
• Fakultät: Mathematik, Informatik und Statistik > Statistik > Technische Reports
• Themengebiete: 500 Naturwissenschaften und Mathematik > 510 Mathematik
• Sprache: Englisch