Bayesian Linear Regression

Walter, Gero and Augustin, Thomas (9. September 2009): Bayesian Linear Regression. Different Conjugate Models and Their (In)Sensitivity to Prior-Data Conflict. Department of Statistics: Technical Reports, No.69

Abstract

The paper is concerned with Bayesian analysis under prior-data conflict, i.e. the situation when observed data are rather unexpected under the prior (and the sample size is not large enough to eliminate the influence of the prior). Two approaches for Bayesian linear regression modeling based on conjugate priors are considered in detail, namely the standard approach also described in Fahrmeir, Kneib & Lang (2007) and an alternative adoption of the general construction procedure for exponential family sampling models. We recognize that - in contrast to some standard i.i.d. models like the scaled normal model and the Beta-Binomial / Dirichlet-Multinomial model, where prior-data conflict is completely ignored - the models may show some reaction to prior-data conflict, however in a rather unspecific way. Finally we briefly sketch the extension to a corresponding imprecise probability model, where, by considering sets of prior distributions instead of a single prior, prior-data conflict can be handled in a very appealing and intuitive way.

• Item Type: Paper
• Form of publication: Submitted Version
• Keywords: Linear regression; conjugate analysis; prior-data conflict; imprecise probability
• Faculties: Mathematics, Computer Science and Statistics > Statistics > Technical Reports
• Subjects: 300 Social sciences > 310 Statistics; 500 Science > 500 Science
• Language: English

References

Augustin, T., Coolen, F. P., Moral, S. & Troffaes, M. C. (eds) (2009). ISIPTA’09: Proceedings of the Sixth International Symposium on Imprecise Probability: Theories and Applications, Durham University, Durham, UK, July 2009, SIPTA.

Augustin, T. & Hable, R. (2009). On the impact of robust statistics on imprecise probability models: a review, ICOSSAR’09: The 10th International Conference on Structural Safety and Reliability, Osaka. To appear.

Bernard, J.-M. (2009). Special issue on the Imprecise Dirichlet Model. International Journal of Approximate Reasoning.

Bernardo, J. M. & Smith, A. F. M. (1994). Bayesian Theory, Wiley, Chichester.

Bousquet, N. (2008). Diagnostic of prior-data agreement in applied bayesian analysis, 35: 1011–1029.

Coolen-Schrijner, P., Coolen, F., Troffaes, M. & Augustin, T. (2009). Special Issue on Statistical Theory and Practice with Imprecision, Journal of Statistical Theory and Practice 3.

de Cooman, G., Vejnarova, J. & Zaffalon, M. (eds) (2007). ISIPTA’07: Proceedings of the Fifth International Symposium on Imprecise Probabilities and Their Applications, Charles University, Prague, Czech Republic, July 2007, SIPTA.

Ellsberg, D. (1961). Risk, ambiguity, and the Savage axioms, Q. J. Econ. pp. 643–669.

Evans, M. & Moshonov, H. (2006). Checking for prior-data conflict, Bayesian Analysis 1: 893–914.

Fahrmeir, L. & Kaufmann, H. (1985). Consistency and asymptotic normality of the maximum-likelihood estimator in generalized linear-models, Annals of Statistics 13: 342–368.

Fahrmeir, L. & Kneib, T. (2006). Structured additive regression for categorial space-time data: A mixed model approach, Biometrics 62: 109–118.

Fahrmeir, L. & Kneib, T. (2009). Propriety of posteriors in structured additive regression models: Theory and empirical evidence, Journal of Statistical Planning and Inference 139: 843–859.

Fahrmeir, L., Kneib, T. & Lang, S. (2007). Regression. Modelle, Methoden und Anwendungen, Springer, New York.

Fahrmeir, L. & Raach, A. (2007). A Bayesian semiparametric latent variable model f¨ur mixed responses, Psychometrika 72: 327–346.

Fahrmeir, L. & Tutz, G. (2001). Multivariate Statistical Modelling Based on Generalized Linear Models, Springer.

Higgins, J. P. T. & Whitehead, A. (1996). Borrowing strength from external trials in a meta-analysis, Statistics in Medicine 15: 2733–2749.

Hsu, M., Bhatt, M., Adolphs, R., Tranel, D. & Camerer, C. F. (2005). Neural systems responding to degrees of uncertainty in human decision-making, Science 310: 1680–1683.

Huber, P. J. & Strassen, V. (1973). Minimax tests and the Neyman-Pearson lemma for capacities, The Annals of Statistics 1: 251–263.

Kauermann, R., Krivobokova, T. & Fahrmeir, L. (2009). Some asymptotic results on generalized penalized spline smooting, J. Roy. Statist. Soc. Ser. B 71: 487–503.

Klir, G. J. & Wierman, M. J. (1999). Uncertainty-based Information. Elements of Generalized Information Theory, Physika, Heidelberg.

Kneib, T. & Fahrmeir, L. (2007). A mixed model approach for geoadditive hazard regression for intervalcensored survival times, 34: 207–228.

Kyburg, H. (1987). Logic of statistical reasoning, in S. Kotz, N. L. Johnson & C. B. Read (eds), Encyclopedia of Statistical Sciences, Vol. 5, Wiley-Interscience, New York, pp. 117–122.

O’Hagan, A. (1994). Bayesian Inference, Vol. 2B of Kendall´s Advanced Theory of Statistics, Arnold, London.

Quaeghebeur, E. & de Cooman, G. (2005). Imprecise probability models for inference in exponential families, in F. G. Cozman, R. Nau & T. Seidenfeld (eds), ISIPTA ’05: Proc. 4th Int. Symp. on Imprecise Probabilities and Their Applications, pp. 287–296.

Rios Insua, D. & Ruggeri, F. (eds) (2000). Robust Bayesian Analysis, Springer, New York.

Scheipl, F. & Kneib, T. (2009). Locally adaptive Bayesian P-splines with a normal-exponential-gamma prior, Computational Statistics & Data Analysis 53: 3533–3552.

Walley, P. (1991). Statistical Reasoning with Imprecise Probabilities, Chapman and Hall, London.

Walley, P. (1996). Inferences from multinomial data: learning about a bag of marbles, Journal of the Royal Statistical Society. Series B. Methodological 58: 3–57.

Walter, G. (2006). Robuste Bayes-Regression mit Mengen von Prioris — Ein Beitrag zur Statistik unter komplexer Unsicherheit. Diploma thesis, Department of Statistics, LMU Munich. http://www.stat.uni-muenchen.de/˜thomas/team/diplomathesis_GeroWalter.pdf.

Walter, G. & Augustin, T. (2009). Imprecision and prior-data conflict in generalized Bayesian inference., Journal of Statistical Theory and Practice 3: 255–271.

Walter, G., Augustin, T. & Peters, A. (2007). Linear regression analysis under sets of conjugate priors, in G. de Cooman, J. Vejnarov´a & M. Zaffalon (eds), ISIPTA ’07: Proc. 5th Int. Symp. on Imprecise Probabilities and Their Applications, pp. 445–455.

Weichselberger, K. (2001). Elementare Grundbegriffe einer allgemeineren Wahrscheinlichkeitsrechnung I. Intervallwahrscheinlichkeit als umfassendes Konzept, Physika, Heidelberg.