Dynamic Neural Regression Models

Briegel, T. und Tresp, V. (2000): Dynamic Neural Regression Models. Sonderforschungsbereich 386, Discussion Paper 181

Abstract

We consider sequential or online learning in dynamic neural regression models. By using a state space representation for the neural network' s parameter evolution in time we obtain approximations to the unknown posterior by either deriving posterior modes via the Fisher scoring algorithm or by deriving approximate posterior means with the importance sampling method. Furthermore, we replace the commonly used Gaussian noise assumption in the neural regression model by a more flexible noise model based on the Student t-density. Since the t-density can be interpreted as being an infinite mixture of Gaussians, hyperparameters such as the degrees of freedom of the t-density can be learned from the data based on an online EM-type algorithm. We show experimentally that our novel methods outperform state-of-the art neural network online learning algorithms like the extended Kalman filter method for both, situations with standard Gaussian noise terms and situations with measurement outliers.

• Dokumententyp: Paper
• Fakultät: Mathematik, Informatik und Statistik > Statistik > Sonderforschungsbereich 386 ; Sonderforschungsbereiche > Sonderforschungsbereich 386
• Themengebiete: 500 Naturwissenschaften und Mathematik > 510 Mathematik
• Sprache: Englisch