Selection and fusion of categorical predictors with L₀-type penalties

Oelker, Magret-Ruth; Pößnecker, Wolfgang and Tutz, Gerhard (2015): Selection and fusion of categorical predictors with L₀-type penalties. In: Statistical Modelling, Vol. 15, No. 5: pp. 389-410

Abstract

In regression modelling, categorical covariates have to be coded. Depending on the number of categorical covariates and on the number of levels they have, the number of coefficients can become huge. To reduce the model complexity, coefficients of similar categories should be fused and coefficients of non-influential categories should be set to zero. To this end, Lasso-type penalties on the differences of coefficients are a standard approach. However, the clustering/selection performance of this approach is sometimes poor–especially when the adaptive weights are badly conditioned or not existing. In some situations, there is no incentive to cluster similar categories. To overcome this, a L0 penalty on the differences of coefficients is proposed, whereby the L0 ‘norm’ is defined as the number of non-zero entries in a vector. The proposed penalty favours to find clusters of categories that share the same effect on the response variable while the estimation accuracy is comparable to Lasso-type penalties. Numerical experiments within the framework of generalized linear models are promising. For illustration, data on the unemployment rates in Germany is analyzed.

• Item Type: Journal article
• Faculties: Mathematics, Computer Science and Statistics > Statistics; Mathematics, Computer Science and Statistics > Statistics > Chairs/Working Groups > Seminar for Applied Stochastic
• Subjects: 500 Science > 510 Mathematics
• Alliance/National Licence: This publication is with permission of the rights owner freely accessible due to an Alliance licence and a national licence (funded by the DFG, German Research Foundation) respectively.
• Language: English