Kernel Predictive Linear Gaussian Models for Nonlinear Stochastic Dynamical Systems

Dynamical systems form an integral and important part of many engineering and mathematical disciplines, and have been used to model a bewildering array of phenomena. The recent Predictive Linear Gaussian model improves upon traditional linear dynamical system models by using a predictive representation of state, which makes consistent parameter estimation possible without any loss of modeling power and while using fewer parameters. This model subsumes classical models such as the celebrated Kalman filter and ARMA models.

Modeling many important and useful dynamical systems requires nonlinear methods. We therefore present two possible extensions of the Predictive Linear Gaussian model to the nonlinear case. Both methods involve the use of radial basis function networks to approximate a stochastic, nonlinear dynamical system, and admit closed form solutions to the state update equations due to conjugacy between the dynamics and the state representation. We also explore efficient sigma-point approximations to both algorithms, and present parameter estimation routines for all algorithms. We empirically compare both models (and their approximations) to the original PLG and discuss the relative advantages of each.