Feature Selection Guided by Structural Information

Slawski, Martin; zu Castell, Wolfgang and Tutz, Gerhard (09. March 2009): Feature Selection Guided by Structural Information. Department of Statistics: Technical Reports, No.51

Abstract

In generalized linear regression problems with an abundant number of features, lasso-type regularization which imposes an l1-constraint on the regression coefficients has become a widely established technique. Crucial deficiencies of the lasso were unmasked when Zhou and Hastie (2005) introduced the elastic net. In this paper, we propose to extend the elastic net by admitting general nonnegative quadratic constraints as second form of regularization. The generalized ridge-type constraint will typically make use of the known association structure of features, e.g. by using temporal- or spatial closeness. We study properties of the resulting ’structured elastic net’ regression estimation procedure, including basic asymptotics and the issue of model selection consistency. In this vein, we provide an analog to the so-called ’irrepresentable condition’ which holds for the lasso. An oracle property is established by incorporating a scaled l1-constraint. Moreover, we outline algorithmic solutions for the structured elastic net within the generalized linear model family. The rationale and the performance of our approach is illustrated by means of simulated- and real world data.

• Item Type: Paper (Technical Report)
• Keywords: generalized linear model, regularization, sparsity, p >> n, lasso, elastic net, random fields, consistency, epiconvergence, model selection, signal regression
• Subjects: Mathematics, Computer Science and Statistics > Statistics > Technical Reports
• Dewey Classification: 600 Natural sciences and mathematics > 510 Mathematics