Danger: High Power! – Exploring the Statistical Properties of a Test for Random Forest Variable Importance

Strobl, Carolin and Zeileis, Achim (30. January 2008): Danger: High Power! – Exploring the Statistical Properties of a Test for Random Forest Variable Importance. Department of Statistics: Technical Reports, No.17

Abstract

Random forests have become a widely-used predictive model in many scientific disciplines within the past few years. Additionally, they are increasingly popular for assessing variable importance, e.g., in genetics and bioinformatics. We highlight both advantages and limitations of different variable importance scores and associated testing procedures, especially in the context of correlated predictor variables. For the test of Breiman and Cutler (2008), we investigate the statistical properties and find that the power of the test depends both on the sample size and the number of trees, an arbitrarily chosen tuning parameter, leading to undesired results that nullify any significance judgments. Moreover, the specification of the null hypothesis of this test is discussed in the context of correlated predictor variables.

• Item Type: Paper (Technical Report)
• Keywords: feature selection, variable importance, permutation tests
• Subjects: Mathematics, Computer Science and Statistics > Statistics > Technical Reports
• Language: English