DOCSLIB.ORG

GP-Bayesfilters: Bayesian Filtering Using Gaussian Process Prediction and Observation Models

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

GP-BayesFilters: Bayesian Filtering Using Gaussian Process Prediction and Observation Models Jonathan Ko and Dieter Fox Dept. of Computer Science & Engineering, University of Washington, Seattle, WA Abstract— Bayesian filtering is a general framework for re- observation models can be combined with particle filters (GP- cursively estimating the state of a dynamical system. The most PF) and extended Kalman filters (GP-EKF). The development common instantiations of Bayes filters are Kalman filters (ex- of GP-EKFs requires a linearization of GP regression models, tended and unscented) and particle filters. Key components of each Bayes filter are probabilistic prediction and observation which we derive in this paper. We furthermore perform a models. Recently, Gaussian processes have been introduced as a thorough comparison of the performance of the different filters non-parametric technique for learning such models from training based on simulation experiments and data collected by a data. In the context of unscented Kalman filters, these models robotic blimp. have been shown to provide estimates that can be superior to This paper is organized as follows. After providing the those achieved with standard, parametric models. In this paper we show how Gaussian process models can be integrated into necessary background on Bayesian filtering and Gaussian other Bayes filters, namely particle filters and extended Kalman processes, we introduce the different instantiations of GP- filters. We provide a complexity analysis of these filters and BayesFilters in Section III. Section IV presents the experi- evaluate the alternative techniques using data collected with an mental evaluation. We conclude in Section V. autonomous micro-blimp. II. BACKGROUND OF GP-BAYESFILTERS I. INTRODUCTION Before we describe the generic GP-BayesFilter, let us dis- Estimating the state of a dynamical system is a fundamen- cuss the basic concepts underlying Bayes filters and Gaussian tal problem in robotics. The most successful techniques for processes. state estimation are Bayesian filters such as particle filters A. Bayes Filters or extended and unscented Kalman filters [13]. Bayes filters recursively estimate posterior probability distributions over Bayes filters recursively estimate posterior distributions over the state of a system. The key components of a Bayes filter the state xk of a dynamical system conditioned on all sensor are the prediction and observation models, which probabilis- information collected so far: tically describe the temporal evolution of the process and the measurements returned by the sensors, respectively. Typically, p(xkjz1:k; u1:k−1) / Z these models are parametric descriptions of the involved p(zkjxk) p(xkjxk−1; uk−1) p(xk−1jz1:k)dxk−1 (1) processes [13]. However, parametric models are not always able to capture all aspects of a dynamical system. Here z1:k and u1:k−1 are the histories of sensor measure- To overcome the limitations of parametric models, re- ments and controls obtained up to time k. The term p(xk j searchers have recently introduced non-parametric, Gaussian xk−1; uk−1) is the prediction model, a probabilistic model process (GP) regression models [12] to learn prediction and of the system dynamics. p(zk j xk), the observation model, observation models for dynamical systems. GPs have been describes the likelihood of making an observation zk given the applied successfully to the problem of learning predictive state xk. Typically, these models are parametric descriptions state models [3, 4, 9]. The fact that GP regression models of the underlying processes, see [13] for several examples. provide uncertainty estimates for their predictions allows them In GP-BayesFilters, both prediction and observation models to be readily incorporated into particle filters as observation are learned from training data using non-parametric, Gaussian models [2] or as improved sampling distributions [10]. Ko and process regression. colleagues introduced GP-UKFs, which combine GP predic- tion and observation models with unscented Kalman filters. B. Gaussian Processes for Regression Using data collected with a robotic blimp they demonstrated Gaussian processes (GP) are a powerful, non-parametric that GP-UKFs outperform parametric unscented Kalman filters tool for learning regression functions from sample data. Key and that the performance of GP-UKFs can be increased by advantages of GPs are their modeling flexibility, their ability to combining GP models with parametric models [7]. provide uncertainty estimates, and their ability to learn noise In this paper we investigate the integration of Gaussian and smoothness parameters from training data [12]. Processes (GP) into different forms of Bayes filters. In addition A Gaussian process represents posterior distributions over to GP-UKFs, the previously introduced combination with functions based on training data. To see, assume we have a set unscented Kalman filters, we show how GP prediction and of training data, D = hX; yi, where X = [x1; x2; :::; xn] is a matrix containing d-dimensional input examples xi, and y = The resulting GP prediction and observation models are then [y1; y2; :::; yn] is a vector containing scalar training outputs yi. A GP assumes that the data is drawn from the noisy process p(xkjxk−1; uk−1) ≈ N (GPµ([xk−1; uk−1];Dp); GPΣ([xk−1; uk−1];Dp)) (8) yi = f(xi) + ; (2) and where " is zero mean, additive Gaussian noise with variance σ2 . Conditioned on training data D = hX; yi and a test input n p(zkjxk) ≈ N (GPµ(xk;Do); GPΣ(xk;Do)) ; (9) x∗, a GP defines a Gaussian predictive distribution over the output y∗ with mean respectively. The reader may notice that while these models are Gaussians, both the means and variances are non-linear T 2 −1 GPµ (x∗;D) = k∗ [K + σnI] y (3) functions of the input and training data. Furthermore, the and variance locally Gaussian nature of these models allows a very natural integration into different instantiations of Bayes filters, as we T 2 −1 GPΣ (x∗;D) = k(x∗; x∗) − k∗ K + σnI k∗: (4) will describe in Section III. GPs are typically defined for scalar outputs, and GP- Here, k is the kernel function of the GP, k is a vector defined ∗ BayesFilters represent models for vectorial outputs by learning by kernel values between x and the training inputs X, and ∗ a different GP for each output dimension. As a result, the K is the n × n kernel matrix of the training input values; that noise covariances GP are diagonal matrices. Another issue to is, k [i] = k(x ; x ) and K[i; j] = k(x ; x ). Note that the Σ ∗ ∗ i i j consider is that GPs assume a zero mean prior over the outputs prediction uncertainty, captured by the variance GP , depends Σ of the functions. A direct ramification of this assumption is that on both the process noise and the correlation between the test the GP predictions tend towards zero as the distance between input and the training data. the test input and the training data increases. In practice, this The choice of the kernel function depends on the applica- problem can be reduced by collecting sufficient training data tion, the most widely used being the squared exponential, or covering possible states, controls, and observations, and by Gaussian, kernel: incorporating parametric models into the GP, as shown in [7]. 0 2 − 1 (x−x0)W (x−x0)T k(x; x ) = σ e 2 ; (5) f III. INSTANTIATIONS OF GP-BAYESFILTERS 2 where σf is the signal variance. The diagonal matrix W We will now show how GP models can be incorporated into defines the smoothness of the process along the different input different instantiations of Bayes filters. Specifically, we present dimensions. algorithms for GP integration into particle filters, extended The GP parameters θ = [W; σf ; σn], describing the kernel Kalman filters, and unscented Kalman filters. For notation, function (5) and the process noise (2), respectively, are called we will stick close to the versions presented in [13]. the hyperparameters of the Gaussian process. These hyperpa- rameters can be learned by maximizing the log likelihood of A. GP-PF: Gaussian Process Particle Filters the training data using numerical optimization techniques such Particle filters are sample-based implementations of Bayes as conjugate gradient descent [12]. filters. The key idea of particle filters is to represent posteriors over the state x by sets X of weighted samples: C. Learning Prediction and Observation Models with GPs k k m (m) Gaussian process regression can be applied directly to Xk = fhxk ; wk i j m = 1;:::;Mg the problem of learning prediction and observation models m (m) Here each xk is a sample (or state), and each wk is required by the Bayes filter (1). The training data for each a non-negative numerical factor called importance weight. GP is a set of input-output relations. The prediction model Particle filters update posteriors according to a sam- x u maps the state and control, ( k, k), to the state transition pling procedure [13]. Table I shows how this proce- ∆x = x −x k k+1 k. The next state can then be found by adding dure can be implemented with GP prediction and obser- the state transition to the previous state. The observation vation models. In Step 4, the state at time k is sam- model maps from the state, xk, to the observation, zk. The m pled based on the previous state xk−1 and control uk−1, appropriate form of the prediction and observation training using the GP prediction model defined in (8). Here, data sets is thus m GP([xk−1; uk−1];Dp) is short for the Gaussian represented by 0 m m Dp = h(X; U);X i (6) N GPµ([xk−1; uk−1];Dp); GPΣ([xk−1; uk−1];Dp) . Note D = hX; Zi ; (7) that the covariance of this prediction is typically different for o each sample, taking the local density of training data into where X is a matrix containing ground truth states, and X0 = account.
Recommended publications
  • Scalable Nonparametric Bayesian Inference on Point Processes with Gaussian Processes

    Scalable Nonparametric Bayesian Inference on Point Processes with Gaussian Processes

    Scalable Nonparametric Bayesian Inference on Point Processes with Gaussian Processes Yves-Laurent Kom Samo [email protected] Stephen Roberts [email protected] Deparment of Engineering Science and Oxford-Man Institute, University of Oxford Abstract 2. Related Work In this paper we propose an efficient, scalable Non-parametric inference on point processes has been ex- non-parametric Gaussian process model for in- tensively studied in the literature. Rathbum & Cressie ference on Poisson point processes. Our model (1994) and Moeller et al. (1998) used a finite-dimensional does not resort to gridding the domain or to intro- piecewise constant log-Gaussian for the intensity function. ducing latent thinning points. Unlike competing Such approximations are limited in that the choice of the 3 models that scale as O(n ) over n data points, grid on which to represent the intensity function is arbitrary 2 our model has a complexity O(nk ) where k and one has to trade-off precision with computational com- n. We propose a MCMC sampler and show that plexity and numerical accuracy, with the complexity being the model obtained is faster, more accurate and cubic in the precision and exponential in the dimension of generates less correlated samples than competing the input space. Kottas (2006) and Kottas & Sanso (2007) approaches on both synthetic and real-life data. used a Dirichlet process mixture of Beta distributions as Finally, we show that our model easily handles prior for the normalised intensity function of a Poisson data sizes not considered thus far by alternate ap- process. Cunningham et al.
  • Deep Neural Networks As Gaussian Processes

    Deep Neural Networks As Gaussian Processes

    Published as a conference paper at ICLR 2018 DEEP NEURAL NETWORKS AS GAUSSIAN PROCESSES Jaehoon Lee∗y, Yasaman Bahri∗y, Roman Novak , Samuel S. Schoenholz, Jeffrey Pennington, Jascha Sohl-Dickstein Google Brain fjaehlee, yasamanb, romann, schsam, jpennin, [email protected] ABSTRACT It has long been known that a single-layer fully-connected neural network with an i.i.d. prior over its parameters is equivalent to a Gaussian process (GP), in the limit of infinite network width. This correspondence enables exact Bayesian inference for infinite width neural networks on regression tasks by means of evaluating the corresponding GP. Recently, kernel functions which mimic multi-layer random neural networks have been developed, but only outside of a Bayesian framework. As such, previous work has not identified that these kernels can be used as co- variance functions for GPs and allow fully Bayesian prediction with a deep neural network. In this work, we derive the exact equivalence between infinitely wide deep net- works and GPs. We further develop a computationally efficient pipeline to com- pute the covariance function for these GPs. We then use the resulting GPs to per- form Bayesian inference for wide deep neural networks on MNIST and CIFAR- 10. We observe that trained neural network accuracy approaches that of the corre- sponding GP with increasing layer width, and that the GP uncertainty is strongly correlated with trained network prediction error. We further find that test perfor- mance increases as finite-width trained networks are made wider and more similar to a GP, and thus that GP predictions typically outperform those of finite-width networks.
  • Gaussian Process Dynamical Models for Human Motion

    Gaussian Process Dynamical Models for Human Motion

    IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 30, NO. 2, FEBRUARY 2008 283 Gaussian Process Dynamical Models for Human Motion Jack M. Wang, David J. Fleet, Senior Member, IEEE, and Aaron Hertzmann, Member, IEEE Abstract—We introduce Gaussian process dynamical models (GPDMs) for nonlinear time series analysis, with applications to learning models of human pose and motion from high-dimensional motion capture data. A GPDM is a latent variable model. It comprises a low- dimensional latent space with associated dynamics, as well as a map from the latent space to an observation space. We marginalize out the model parameters in closed form by using Gaussian process priors for both the dynamical and the observation mappings. This results in a nonparametric model for dynamical systems that accounts for uncertainty in the model. We demonstrate the approach and compare four learning algorithms on human motion capture data, in which each pose is 50-dimensional. Despite the use of small data sets, the GPDM learns an effective representation of the nonlinear dynamics in these spaces. Index Terms—Machine learning, motion, tracking, animation, stochastic processes, time series analysis. Ç 1INTRODUCTION OOD statistical models for human motion are important models such as hidden Markov model (HMM) and linear Gfor many applications in vision and graphics, notably dynamical systems (LDS) are efficient and easily learned visual tracking, activity recognition, and computer anima- but limited in their expressiveness for complex motions. tion. It is well known in computer vision that the estimation More expressive models such as switching linear dynamical of 3D human motion from a monocular video sequence is systems (SLDS) and nonlinear dynamical systems (NLDS), highly ambiguous.
  • Modelling Multi-Object Activity by Gaussian Processes 1

    Modelling Multi-Object Activity by Gaussian Processes 1

    LOY et al.: MODELLING MULTI-OBJECT ACTIVITY BY GAUSSIAN PROCESSES 1 Modelling Multi-object Activity by Gaussian Processes Chen Change Loy School of EECS [email protected] Queen Mary University of London Tao Xiang E1 4NS London, UK [email protected] Shaogang Gong [email protected] Abstract We present a new approach for activity modelling and anomaly detection based on non-parametric Gaussian Process (GP) models. Specifically, GP regression models are formulated to learn non-linear relationships between multi-object activity patterns ob- served from semantically decomposed regions in complex scenes. Predictive distribu- tions are inferred from the regression models to compare with the actual observations for real-time anomaly detection. The use of a flexible, non-parametric model alleviates the difficult problem of selecting appropriate model complexity encountered in parametric models such as Dynamic Bayesian Networks (DBNs). Crucially, our GP models need fewer parameters; they are thus less likely to overfit given sparse data. In addition, our approach is robust to the inevitable noise in activity representation as noise is modelled explicitly in the GP models. Experimental results on a public traffic scene show that our models outperform DBNs in terms of anomaly sensitivity, noise robustness, and flexibil- ity in modelling complex activity. 1 Introduction Activity modelling and automatic anomaly detection in video have received increasing at- tention due to the recent large-scale deployments of surveillance cameras. These tasks are non-trivial because complex activity patterns in a busy public space involve multiple objects interacting with each other over space and time, whilst anomalies are often rare, ambigu- ous and can be easily confused with noise caused by low image quality, unstable lighting condition and occlusion.
  • Financial Time Series Volatility Analysis Using Gaussian Process State-Space Models

    Financial Time Series Volatility Analysis Using Gaussian Process State-Space Models

    Financial Time Series Volatility Analysis Using Gaussian Process State-Space Models by Jianan Han Bachelor of Engineering, Hebei Normal University, China, 2010 A thesis presented to Ryerson University in partial fulfillment of the requirements for the degree of Master of Applied Science in the Program of Electrical and Computer Engineering Toronto, Ontario, Canada, 2015 c Jianan Han 2015 AUTHOR'S DECLARATION FOR ELECTRONIC SUBMISSION OF A THESIS I hereby declare that I am the sole author of this thesis. This is a true copy of the thesis, including any required final revisions, as accepted by my examiners. I authorize Ryerson University to lend this thesis to other institutions or individuals for the purpose of scholarly research. I further authorize Ryerson University to reproduce this thesis by photocopying or by other means, in total or in part, at the request of other institutions or individuals for the purpose of scholarly research. I understand that my dissertation may be made electronically available to the public. ii Financial Time Series Volatility Analysis Using Gaussian Process State-Space Models Master of Applied Science 2015 Jianan Han Electrical and Computer Engineering Ryerson University Abstract In this thesis, we propose a novel nonparametric modeling framework for financial time series data analysis, and we apply the framework to the problem of time varying volatility modeling. Existing parametric models have a rigid transition function form and they often have over-fitting problems when model parameters are estimated using maximum likelihood methods. These drawbacks effect the models' forecast performance. To solve this problem, we take Bayesian nonparametric modeling approach.
  • Gaussian-Random-Process.Pdf

    Gaussian-Random-Process.Pdf

    The Gaussian Random Process Perhaps the most important continuous state-space random process in communications systems in the Gaussian random process, which, we shall see is very similar to, and shares many properties with the jointly Gaussian random variable that we studied previously (see lecture notes and chapter-4). X(t); t 2 T is a Gaussian r.p., if, for any positive integer n, any choice of coefficients ak; 1 k n; and any choice ≤ ≤ of sample time tk ; 1 k n; the random variable given by the following weighted sum of random variables is Gaussian:2 T ≤ ≤ X(t) = a1X(t1) + a2X(t2) + ::: + anX(tn) using vector notation we can express this as follows: X(t) = [X(t1);X(t2);:::;X(tn)] which, according to our study of jointly Gaussian r.v.s is an n-dimensional Gaussian r.v.. Hence, its pdf is known since we know the pdf of a jointly Gaussian random variable. For the random process, however, there is also the nasty little parameter t to worry about!! The best way to see the connection to the Gaussian random variable and understand the pdf of a random process is by example: Example: Determining the Distribution of a Gaussian Process Consider a continuous time random variable X(t); t with continuous state-space (in this case amplitude) defined by the following expression: 2 R X(t) = Y1 + tY2 where, Y1 and Y2 are independent Gaussian distributed random variables with zero mean and variance 2 2 σ : Y1;Y2 N(0; σ ). The problem is to find the one and two dimensional probability density functions of the random! process X(t): The one-dimensional distribution of a random process, also known as the univariate distribution is denoted by the notation FX;1(u; t) and defined as: P r X(t) u .
  • Gaussian Markov Processes

    Gaussian Markov Processes

    C. E. Rasmussen & C. K. I. Williams, Gaussian Processes for Machine Learning, the MIT Press, 2006, ISBN 026218253X. c 2006 Massachusetts Institute of Technology. www.GaussianProcess.org/gpml Appendix B Gaussian Markov Processes Particularly when the index set for a stochastic process is one-dimensional such as the real line or its discretization onto the integer lattice, it is very interesting to investigate the properties of Gaussian Markov processes (GMPs). In this Appendix we use X(t) to define a stochastic process with continuous time pa- rameter t. In the discrete time case the process is denoted ...,X−1,X0,X1,... etc. We assume that the process has zero mean and is, unless otherwise stated, stationary. A discrete-time autoregressive (AR) process of order p can be written as AR process p X Xt = akXt−k + b0Zt, (B.1) k=1 where Zt ∼ N (0, 1) and all Zt’s are i.i.d. Notice the order-p Markov property that given the history Xt−1,Xt−2,..., Xt depends only on the previous p X’s. This relationship can be conveniently expressed as a graphical model; part of an AR(2) process is illustrated in Figure B.1. The name autoregressive stems from the fact that Xt is predicted from the p previous X’s through a regression equation. If one stores the current X and the p − 1 previous values as a state vector, then the AR(p) scalar process can be written equivalently as a vector AR(1) process. Figure B.1: Graphical model illustrating an AR(2) process.
  • FULLY BAYESIAN FIELD SLAM USING GAUSSIAN MARKOV RANDOM FIELDS Huan N

    FULLY BAYESIAN FIELD SLAM USING GAUSSIAN MARKOV RANDOM FIELDS Huan N

    Asian Journal of Control, Vol. 18, No. 5, pp. 1–14, September 2016 Published online in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/asjc.1237 FULLY BAYESIAN FIELD SLAM USING GAUSSIAN MARKOV RANDOM FIELDS Huan N. Do, Mahdi Jadaliha, Mehmet Temel, and Jongeun Choi ABSTRACT This paper presents a fully Bayesian way to solve the simultaneous localization and spatial prediction problem using a Gaussian Markov random field (GMRF) model. The objective is to simultaneously localize robotic sensors and predict a spatial field of interest using sequentially collected noisy observations by robotic sensors. The set of observations consists of the observed noisy positions of robotic sensing vehicles and noisy measurements of a spatial field. To be flexible, the spatial field of interest is modeled by a GMRF with uncertain hyperparameters. We derive an approximate Bayesian solution to the problem of computing the predictive inferences of the GMRF and the localization, taking into account observations, uncertain hyperparameters, measurement noise, kinematics of robotic sensors, and uncertain localization. The effectiveness of the proposed algorithm is illustrated by simulation results as well as by experiment results. The experiment results successfully show the flexibility and adaptability of our fully Bayesian approach ina data-driven fashion. Key Words: Vision-based localization, spatial modeling, simultaneous localization and mapping (SLAM), Gaussian process regression, Gaussian Markov random field. I. INTRODUCTION However, there are a number of inapplicable situa- tions. For example, underwater autonomous gliders for Simultaneous localization and mapping (SLAM) ocean sampling cannot find usual geometrical models addresses the problem of a robot exploring an unknown from measurements of environmental variables such as environment under localization uncertainty [1].
  • Mean Field Methods for Classification with Gaussian Processes

    Mean Field Methods for Classification with Gaussian Processes

    Mean field methods for classification with Gaussian processes Manfred Opper Neural Computing Research Group Division of Electronic Engineering and Computer Science Aston University Birmingham B4 7ET, UK. opperm~aston.ac.uk Ole Winther Theoretical Physics II, Lund University, S6lvegatan 14 A S-223 62 Lund, Sweden CONNECT, The Niels Bohr Institute, University of Copenhagen Blegdamsvej 17, 2100 Copenhagen 0, Denmark winther~thep.lu.se Abstract We discuss the application of TAP mean field methods known from the Statistical Mechanics of disordered systems to Bayesian classifi­ cation models with Gaussian processes. In contrast to previous ap­ proaches, no knowledge about the distribution of inputs is needed. Simulation results for the Sonar data set are given. 1 Modeling with Gaussian Processes Bayesian models which are based on Gaussian prior distributions on function spaces are promising non-parametric statistical tools. They have been recently introduced into the Neural Computation community (Neal 1996, Williams & Rasmussen 1996, Mackay 1997). To give their basic definition, we assume that the likelihood of the output or target variable T for a given input s E RN can be written in the form p(Tlh(s)) where h : RN --+ R is a priori assumed to be a Gaussian random field. If we assume fields with zero prior mean, the statistics of h is entirely defined by the second order correlations C(s, S') == E[h(s)h(S')], where E denotes expectations 310 M Opper and 0. Winther with respect to the prior. Interesting examples are C(s, s') (1) C(s, s') (2) The choice (1) can be motivated as a limit of a two-layered neural network with infinitely many hidden units with factorizable input-hidden weight priors (Williams 1997).
  • When Gaussian Process Meets Big Data

    When Gaussian Process Meets Big Data

    IEEE 1 When Gaussian Process Meets Big Data: A Review of Scalable GPs Haitao Liu, Yew-Soon Ong, Fellow, IEEE, Xiaobo Shen, and Jianfei Cai, Senior Member, IEEE Abstract—The vast quantity of information brought by big GP paradigms. It is worth noting that this review mainly data as well as the evolving computer hardware encourages suc- focuses on scalable GPs for large-scale regression but not on cess stories in the machine learning community. In the meanwhile, all forms of GPs or other machine learning models. it poses challenges for the Gaussian process (GP) regression, a d n well-known non-parametric and interpretable Bayesian model, Given n training points X = xi R i=1 and their { ∈ n } which suffers from cubic complexity to data size. To improve observations y = yi = y(xi) R , GP seeks to { ∈ }i=1 the scalability while retaining desirable prediction quality, a infer the latent function f : Rd R in the function space variety of scalable GPs have been presented. But they have not (m(x), k(x, x′)) defined by the7→ mean m(.) and the kernel yet been comprehensively reviewed and analyzed in order to be kGP(., .). The most prominent weakness of standard GP is that well understood by both academia and industry. The review of it suffers from a cubic time complexity (n3) because of scalable GPs in the GP community is timely and important due O to the explosion of data size. To this end, this paper is devoted the inversion and determinant of the n n kernel matrix × to the review on state-of-the-art scalable GPs involving two main Knn = k(X, X).
  • Stochastic Optimal Control Using Gaussian Process Regression Over Probability Distributions

    Stochastic Optimal Control Using Gaussian Process Regression Over Probability Distributions

    Stochastic Optimal Control Using Gaussian Process Regression over Probability Distributions Jana Mayer1, Maxim Dolgov2, Tobias Stickling1, Selim Özgen1, Florian Rosenthal1, and Uwe D. Hanebeck1 Abstract— In this paper, we address optimal control of non- In this paper, we will refer to the term value function without linear stochastic systems under motion and measurement uncer- loss of generality. tainty with finite control input and measurement spaces. Such A solution to a stochastic control problem can be found problems can be formalized as partially-observable Markov decision processes where the goal is to find policies via dy- using the dynamic programming (DP) algorithm that is based namic programming that map the information available to the on Bellman’s principle of optimality [1]. This algorithm relies controller to control inputs while optimizing a performance on two key principles. First, it assumes the existence of criterion. However, they suffer from intractability in scenarios sufficient statistics in form of a probability distribution that with continuous state spaces and partial observability which encompasses all the information up to the moment of policy makes approximations necessary. Point-based value iteration methods are a class of global approximate methods that regress evaluation. And second, it computes the policy by starting at the value function given the values at a set of reference points. the end of the planning horizon and going backwards in time In this paper, we present a novel point-based value iteration to the current time step and at every time step choosing a approach for continuous state spaces that uses Gaussian pro- control input that minimize the cumulative costs maintained cesses defined over probability distribution for the regression.
  • The Variational Gaussian Process

    The Variational Gaussian Process

    Published as a conference paper at ICLR 2016 THE VARIATIONAL GAUSSIAN PROCESS Dustin Tran Harvard University [email protected] Rajesh Ranganath Princeton University [email protected] David M. Blei Columbia University [email protected] ABSTRACT Variational inference is a powerful tool for approximate inference, and it has been recently applied for representation learning with deep generative models. We de- velop the variational Gaussian process (VGP), a Bayesian nonparametric varia- tional family, which adapts its shape to match complex posterior distributions. The VGP generates approximate posterior samples by generating latent inputs and warping them through random non-linear mappings; the distribution over random mappings is learned during inference, enabling the transformed outputs to adapt to varying complexity. We prove a universal approximation theorem for the VGP, demonstrating its representative power for learning any model. For inference we present a variational objective inspired by auto-encoders and perform black box inference over a wide class of models. The VGP achieves new state-of-the-art re- sults for unsupervised learning, inferring models such as the deep latent Gaussian model and the recently proposed DRAW. 1 INTRODUCTION Variational inference is a powerful tool for approximate posterior inference. The idea is to posit a family of distributions over the latent variables and then find the member of that family closest to the posterior. Originally developed in the 1990s (Hinton & Van Camp, 1993; Waterhouse et al., 1996; Jordan et al., 1999), variational inference has enjoyed renewed interest around developing scalable optimization for large datasets (Hoffman et al., 2013), deriving generic strategies for easily fitting many models (Ranganath et al., 2014), and applying neural networks as a flexible parametric family of approximations (Kingma & Welling, 2014; Rezende et al., 2014).