Affine Feedforward Stochastic (AFS) Neural Network

Résumé

The aim of this paper is to link the machine learning method of multilayer perceptron (MLP) neural network with the classical analysis of stochastic state space models. We consider a special class of state space models with multiple layers based on affine conditional Laplace transforms. This new class of Affine Feedforward Stochastic (AFS) neural network provides closed form recursive formulas for recursive filtering of the state variables of different layers. This approach is suitable for online inference by stochastic gradient ascent optimization and for recursive computation of scores such as backpropagation. The approach is extended to recurrent neural networks and identification issues are discussed.