Dynamic Neural Regression Models

Briegel, T. and Tresp, V. (2000): Dynamic Neural Regression Models. Collaborative Research Center 386, Discussion Paper 181

Abstract

We consider sequential or online learning in dynamic neural regression models. By using a state space representation for the neural network' s parameter evolution in time we obtain approximations to the unknown posterior by either deriving posterior modes via the Fisher scoring algorithm or by deriving approximate posterior means with the importance sampling method. Furthermore, we replace the commonly used Gaussian noise assumption in the neural regression model by a more flexible noise model based on the Student t-density. Since the t-density can be interpreted as being an infinite mixture of Gaussians, hyperparameters such as the degrees of freedom of the t-density can be learned from the data based on an online EM-type algorithm. We show experimentally that our novel methods outperform state-of-the art neural network online learning algorithms like the extended Kalman filter method for both, situations with standard Gaussian noise terms and situations with measurement outliers.

• Item Type: Paper
• Faculties: Mathematics, Computer Science and Statistics > Statistics > Collaborative Research Center 386; Special Research Fields > Special Research Field 386
• Subjects: 500 Science > 510 Mathematics
• Language: English