Symbolic Variable Elimination for Discrete and Continuous Graphical Models
Total Page:16
File Type:pdf, Size:1020Kb
Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence Symbolic Variable Elimination for Discrete and Continuous Graphical Models Scott Sanner Ehsan Abbasnejad NICTA & ANU ANU & NICTA Canberra, Australia Canberra, Australia [email protected] [email protected] Abstract Probabilistic reasoning in the real-world often requires inference in continuous variable graphical models, yet there are few methods for exact, closed-form inference when joint distributions are non-Gaussian. To address this inferential deficit, we introduce SVE – a symbolic Figure 1: The robot localization graphical model and all condi- extension of the well-known variable elimination al- tional probabilities. gorithm to perform exact inference in an expressive class of mixed discrete and continuous variable graphi- and xi are the observed measurements of distance (e.g., us- cal models whose conditional probability functions can ing a laser range finder). The prior distribution P (d) is uni- be well-approximated as oblique piecewise polynomi- form U(d; 0; 10) while the observation model P (xijd) is a als with bounded support. Using this representation, we mixture of three distributions: (red) is a truncated Gaussian ex- show that we can compute all of the SVE operations representing noisy measurements x of the actual distance d actly and in closed-form, which crucially includes defi- i nite integration w.r.t. multivariate piecewise polynomial within the sensor range of [0; 10]; (green) is a uniform noise functions. To aid in the efficient computation and com- model U(d; 0; 10) representing random anomalies leading pact representation of this solution, we use an extended to any measurement; and (blue) is a triangular distribution algebraic decision diagram (XADD) data structure that peaking at the maximum measurement distance (e.g., caused supports all SVE operations. We provide illustrative re- by spurious deflections of laser light or the transparency of sults for SVE on probabilistic inference queries inspired glass). by robotics localization and tracking applications that While our focus in this paper is on exact inference in mix various continuous distributions; this represents the general discrete and continuous variable graphical models first time a general closed-form exact solution has been and not purely on this robotics localization task, this ex- proposed for this expressive class of discrete/continuous graphical models. ample clearly motivates the real-world need for reasoning with complex distributions over continuous variables. How- ever, in contrast to previous work that has typically resorted Introduction to (sequential) Monte Carlo methods (Thrun et al. 2000; Doucet, De Freitas, and Gordon 2001) to perform inference Real-world probabilistic reasoning is rife with uncertainty in this model (and dynamic extensions) via sampling, our over continuous random variables with complex (often non- point of departure in this work is to seek exact, closed- linear) relationships, e.g., estimating the position and pose form solutions to general probabilistic inference tasks in of entities from measurements in robotics, or radar track- these graphical models, e.g., arbitrary conditional probabil- ing applications with asymmetric stochastic dynamics and ity queries or conditional expectation computations. complex mixtures of noise processes. While closed-form ex- act solutions exist for inference in some continuous variable To achieve this task, we focus on an expressive class of graphical models, such solutions are largely limited to rela- mixed discrete and continuous variable Bayesian networks tively well-behaved cases such as joint Gaussian models. On whose conditional probability functions can be expressed the other hand, the more complex inference tasks of track- as oblique piecewise polynomials with bounded support ing with real-world sensors involves underlying distributions (where oblique piece boundaries are represented by con- that are beyond the reach of current closed-form exact solu- junctions of linear inequalities). In practice, this represen- tions. Take, for example, the robot sensor model in Figure 1 tation is sufficient to represent or arbitrarily approximate a wide range of common distributions such as the follow- motivated by (Thrun et al. 2000). Here d 2 R is a vari- able representing the distance of a mobile robot to a wall ing: uniform (piecewise constant), triangular (piecewise lin- ear), truncated normal distributions1 (piecewise quadratic or Copyright c 2012, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. 1In practice, measurements have natural ranges that make trun- 1954 k quartic), as well as all mixtures of such distributions as ex- Y p(d; x1; : : : ; xk) = p(d) p(xijd) emplified in P (xijd) above (since a sum of piecewise poly- nomials is still piecewise polynomial). While polynomials i=1 k directly support the integrals we will need to compute vari- Y = (d) Ψ (x ; d) (2) able elimination (Zhang and Poole 1996) in closed-form, d xi i i=1 computing the definite integrals of polynomials w.r.t. arbi- where quite simply, Ψ (d) := p(d) and Ψ (x ; d) := trary linear piece boundaries turns out to be a much more d xi i difficult task achieved through novel symbolic methods that p(xijd). Here, Z = 1 and is hence omitted since joint distri- underlie the key contribution of symbolic variable elimina- butions represented by Bayesian networks marginalize to 1 tion (SVE) that we make in this paper. by definition. In the following sections, we present our graphical model Variable Elimination Inference framework using a case representation of probability dis- tributions followed by a description of our SVE proce- Given a joint probability distribution p(b; x) defined by a dure. After this, we introduce the extended algebraic deci- factor graph, our objective in this paper will be to perform sion diagram (XADD) — a practical data structure for effi- two types of exact, closed-form inference: (Conditional) Probability Queries: for query variables ciently and compactly manipulating the case representation q and disjoint (possibly empty) evidence variables e drawn — and apply XADD-based SVE to robotics localization and from a subset of (b; x), we wish to infer the exact a tracking tasks demonstrating exact closed-form inference form of p(qje) as a function of q and e. This can be for these problems. achieved by the following computation, where for nota- tional convenience we assume variables are renamed s.t. Discrete and Continuous Graphical Models fb [ xg n fq [ eg = (b1; : : : ; bs; x1; : : : ; xt) and q = (b ; : : : ; b 0 ; x ; : : : ; x 0 ): Factor Graph Representation s+1 s t+1 t p(qje) = (3) A discrete and continuous graphical model compactly rep- P P R R Q ··· ··· t Ψf (bf ; xf ) dx1 : : : dxt resents a joint probability distribution p(b; x) over an as- b1 bs R f2F = P P R R Q ··· ··· t0 Ψf (bf ; xf ) dx1 : : : dxt0 signment (b; x) = (b1; : : : ; bn; x1; : : : ; xm) to respective b1 bs0 R f2F 2 random variables (B; X) = (B1;:::;Bn;X1;:::;Xm). 1 The Z from (1) would appear in both the numerator and Each bi (1 ≤ i ≤ n) is boolean s.t. bi 2 f0; 1g and each denominator and hence cancels. xj (1 ≤ j ≤ m) is continuous s.t. xj 2 R. (Conditional) Expectation Queries: for a continuous As a general representation of both directed and undi- query random variable Q and disjoint evidence assignment rected graphical models, we use a factor graph (Kschis- e drawn from a subset of (b; x), we wish to infer the ex- chang, Frey, and Loeliger 2001) representing a joint prob- 3 ability p(b; x) as a product of a finite set of factors F , i.e., act value of E[Qje]. This can be achieved by the following computation, where p(qje) can be pre-computed according 1 Y p(b; x) = Ψ (b ; x ): (1) to (3): Z f f f Z 1 f2F E[Qje] = [q · p(qje)] dq (4) Here, bf and xf denote the subset of variables that par- q=−∞ ticipate in factor f and Ψf (bf ; xf ) is a non-negative, real- Variable elimination (VE) (Zhang and Poole 1996) is a valued potential function that can be viewed as gauging the simple and efficient algorithm given in Algorithm 1 that ex- local compatibility of assignments bf and xf . The func- ploits the distributive law to efficiently compute each elimi- tions Ψ may not necessarily represent probabilities and P R f nation step (i.e., any b or x in (3) and (4)) by first factor- hence a normalization constant Z is often required to ensure i j P R izing out all factors independent of the elimination variable; b x p(b; x)dx = 1. this prevents unnecessary multiplication of factors, hence We will specify all of our algorithms on graphical models minimizing the size and complexity of each elimination op- in terms of general factor graphs, but we note that Bayesian eration. If the factor representation is closed and computable networks represent an important modeling formalism that we will use to specify our examples in this paper. For the under the VE operations of multiplication and marginaliza- Bayesian network directed graph in the introductory exam- tion then VE can compute any (conditional) probability or ple, the joint distribution for n variables is represented as expectation query in (3) and (4). Closed-form, exact solu- the product of all variables conditioned on their parent vari- tions for VE are well-known for the discrete and joint Gaus- ables in the graph and can be easily converted to factor graph sian cases; next we extend VE to expressive piecewise poly- form, i.e., nomial discrete/continuous graphical models. cated variants of distributions appropriate, e.g., a range finder that exhibits Gaussian distributed measurement errors but only returns Symbolic Variable Elimination in measurements in the range [0; 10] may be well-suited to a truncated Discrete/Continuous Graphical Models Normal distribution observation model.