Generalized semiparametrically structured mixed models.
Sonderforschungsbereich 386, Discussion Paper 251
Generalized linear mixed models are a common tool in statistics which extends generalized linear models to situations where data are hierarchically clustered or correlated. In this article the simple but often inadequate restriction to a linear form of the predictor variables is dropped. A class of semiparametrically structured models is proposed in which the predictor decomposes into components that may be given by a parametric term, an additive form of unspecified smooth functions of covariates, varying-coefficient terms or terms which vary smoothly (or not) across the repetitions in a repeated measurement design. The class of models is explicitly designed as an extension of multivariate generalized mixed linear models such that ordinal responses may be treated within this framework. The modelling of smooth effects is based on basis functions like e.g. B-splines or radial basis functions. For the estimation of parameters a penalized marginal likelihood approach is proposed which may be based on integration techniques like Gauss-Hermite quadrature but may as well be used within the more recently developed nonparametric maximum likelihood approaches. For the maximization of the penalized marginal likelihood the EM-algorithm is adapted. Moreover, an adequate form of cross-validation is developed and shown to work satisfactorily. Various examples demonstrate the flexibility of the class of models.