GP-BayesFilters: Bayesian Filtering Using Gaussian Process Prediction and Observation Models Jonathan Ko and Dieter Fox Dept. of Computer Science & Engineering, University of Washington, Seattle, WA Abstract— Bayesian filtering is a general framework for re- observation models can be combined with particle filters (GP- cursively estimating the state of a dynamical system. The most PF) and extended Kalman filters (GP-EKF). The development common instantiations of Bayes filters are Kalman filters (ex- of GP-EKFs requires a linearization of GP regression models, tended and unscented) and particle filters. Key components of each Bayes filter are probabilistic prediction and observation which we derive in this paper. We furthermore perform a models. Recently, Gaussian processes have been introduced as a thorough comparison of the performance of the different filters non-parametric technique for learning such models from training based on simulation experiments and data collected by a data. In the context of unscented Kalman filters, these models robotic blimp. have been shown to provide estimates that can be superior to This paper is organized as follows. After providing the those achieved with standard, parametric models. In this paper we show how Gaussian process models can be integrated into necessary background on Bayesian filtering and Gaussian other Bayes filters, namely particle filters and extended Kalman processes, we introduce the different instantiations of GP- filters. We provide a complexity analysis of these filters and BayesFilters in Section III. Section IV presents the experi- evaluate the alternative techniques using data collected with an mental evaluation. We conclude in Section V. autonomous micro-blimp. II. BACKGROUND OF GP-BAYESFILTERS I. INTRODUCTION Before we describe the generic GP-BayesFilter, let us dis- Estimating the state of a dynamical system is a fundamen- cuss the basic concepts underlying Bayes filters and Gaussian tal problem in robotics. The most successful techniques for processes. state estimation are Bayesian filters such as particle filters A. Bayes Filters or extended and unscented Kalman filters [13]. Bayes filters recursively estimate posterior probability distributions over Bayes filters recursively estimate posterior distributions over the state of a system. The key components of a Bayes filter the state xk of a dynamical system conditioned on all sensor are the prediction and observation models, which probabilis- information collected so far: tically describe the temporal evolution of the process and the measurements returned by the sensors, respectively. Typically, p(xkjz1:k; u1:k−1) / Z these models are parametric descriptions of the involved p(zkjxk) p(xkjxk−1; uk−1) p(xk−1jz1:k)dxk−1 (1) processes [13]. However, parametric models are not always able to capture all aspects of a dynamical system. Here z1:k and u1:k−1 are the histories of sensor measure- To overcome the limitations of parametric models, re- ments and controls obtained up to time k. The term p(xk j searchers have recently introduced non-parametric, Gaussian xk−1; uk−1) is the prediction model, a probabilistic model process (GP) regression models [12] to learn prediction and of the system dynamics. p(zk j xk), the observation model, observation models for dynamical systems. GPs have been describes the likelihood of making an observation zk given the applied successfully to the problem of learning predictive state xk. Typically, these models are parametric descriptions state models [3, 4, 9]. The fact that GP regression models of the underlying processes, see [13] for several examples. provide uncertainty estimates for their predictions allows them In GP-BayesFilters, both prediction and observation models to be readily incorporated into particle filters as observation are learned from training data using non-parametric, Gaussian models [2] or as improved sampling distributions [10]. Ko and process regression. colleagues introduced GP-UKFs, which combine GP predic- tion and observation models with unscented Kalman filters. B. Gaussian Processes for Regression Using data collected with a robotic blimp they demonstrated Gaussian processes (GP) are a powerful, non-parametric that GP-UKFs outperform parametric unscented Kalman filters tool for learning regression functions from sample data. Key and that the performance of GP-UKFs can be increased by advantages of GPs are their modeling flexibility, their ability to combining GP models with parametric models [7]. provide uncertainty estimates, and their ability to learn noise In this paper we investigate the integration of Gaussian and smoothness parameters from training data [12]. Processes (GP) into different forms of Bayes filters. In addition A Gaussian process represents posterior distributions over to GP-UKFs, the previously introduced combination with functions based on training data. To see, assume we have a set unscented Kalman filters, we show how GP prediction and of training data, D = hX; yi, where X = [x1; x2; :::; xn] is a matrix containing d-dimensional input examples xi, and y = The resulting GP prediction and observation models are then [y1; y2; :::; yn] is a vector containing scalar training outputs yi. A GP assumes that the data is drawn from the noisy process p(xkjxk−1; uk−1) ≈ N (GPµ([xk−1; uk−1];Dp); GPΣ([xk−1; uk−1];Dp)) (8) yi = f(xi) + ; (2) and where " is zero mean, additive Gaussian noise with variance σ2 . Conditioned on training data D = hX; yi and a test input n p(zkjxk) ≈ N (GPµ(xk;Do); GPΣ(xk;Do)) ; (9) x∗, a GP defines a Gaussian predictive distribution over the output y∗ with mean respectively. The reader may notice that while these models are Gaussians, both the means and variances are non-linear T 2 −1 GPµ (x∗;D) = k∗ [K + σnI] y (3) functions of the input and training data. Furthermore, the and variance locally Gaussian nature of these models allows a very natural integration into different instantiations of Bayes filters, as we T 2 −1 GPΣ (x∗;D) = k(x∗; x∗) − k∗ K + σnI k∗: (4) will describe in Section III. GPs are typically defined for scalar outputs, and GP- Here, k is the kernel function of the GP, k is a vector defined ∗ BayesFilters represent models for vectorial outputs by learning by kernel values between x and the training inputs X, and ∗ a different GP for each output dimension. As a result, the K is the n × n kernel matrix of the training input values; that noise covariances GP are diagonal matrices. Another issue to is, k [i] = k(x ; x ) and K[i; j] = k(x ; x ). Note that the Σ ∗ ∗ i i j consider is that GPs assume a zero mean prior over the outputs prediction uncertainty, captured by the variance GP , depends Σ of the functions. A direct ramification of this assumption is that on both the process noise and the correlation between the test the GP predictions tend towards zero as the distance between input and the training data. the test input and the training data increases. In practice, this The choice of the kernel function depends on the applica- problem can be reduced by collecting sufficient training data tion, the most widely used being the squared exponential, or covering possible states, controls, and observations, and by Gaussian, kernel: incorporating parametric models into the GP, as shown in [7]. 0 2 − 1 (x−x0)W (x−x0)T k(x; x ) = σ e 2 ; (5) f III. INSTANTIATIONS OF GP-BAYESFILTERS 2 where σf is the signal variance. The diagonal matrix W We will now show how GP models can be incorporated into defines the smoothness of the process along the different input different instantiations of Bayes filters. Specifically, we present dimensions. algorithms for GP integration into particle filters, extended The GP parameters θ = [W; σf ; σn], describing the kernel Kalman filters, and unscented Kalman filters. For notation, function (5) and the process noise (2), respectively, are called we will stick close to the versions presented in [13]. the hyperparameters of the Gaussian process. These hyperpa- rameters can be learned by maximizing the log likelihood of A. GP-PF: Gaussian Process Particle Filters the training data using numerical optimization techniques such Particle filters are sample-based implementations of Bayes as conjugate gradient descent [12]. filters. The key idea of particle filters is to represent posteriors over the state x by sets X of weighted samples: C. Learning Prediction and Observation Models with GPs k k m (m) Gaussian process regression can be applied directly to Xk = fhxk ; wk i j m = 1;:::;Mg the problem of learning prediction and observation models m (m) Here each xk is a sample (or state), and each wk is required by the Bayes filter (1). The training data for each a non-negative numerical factor called importance weight. GP is a set of input-output relations. The prediction model Particle filters update posteriors according to a sam- x u maps the state and control, ( k, k), to the state transition pling procedure [13]. Table I shows how this proce- ∆x = x −x k k+1 k. The next state can then be found by adding dure can be implemented with GP prediction and obser- the state transition to the previous state. The observation vation models. In Step 4, the state at time k is sam- model maps from the state, xk, to the observation, zk. The m pled based on the previous state xk−1 and control uk−1, appropriate form of the prediction and observation training using the GP prediction model defined in (8). Here, data sets is thus m GP([xk−1; uk−1];Dp) is short for the Gaussian represented by 0 m m Dp = h(X; U);X i (6) N GPµ([xk−1; uk−1];Dp); GPΣ([xk−1; uk−1];Dp) . Note D = hX; Zi ; (7) that the covariance of this prediction is typically different for o each sample, taking the local density of training data into where X is a matrix containing ground truth states, and X0 = account.