Density Approximation Based on Dirac Mixtures with Regard to Nonlinear Estimation and Filtering Oliver C. Schrempf, Dietrich Brunn, Uwe D. Hanebeck Intelligent Sensor-Actuator-Systems Laboratory Institute of Computer Science and Engineering Universitat¨ Karlsruhe (TH), Germany [email protected], [email protected], [email protected] Abstract— A deterministic procedure for optimal approxi- approximate an arbitrary density function. The proposed mation of arbitrary probability density functions by means of method differs from the deterministic type of particle filters Dirac mixtures with equal weights is proposed. The optimality in [8] as a distance measure is employed to find an optimal of this approximation is guaranteed by minimizing the distance of the approximation from the true density. For this purpose approximation of the true density. However, typical distance a distance measure is required, which is in general not well measures quantifying the distance between two densities are defined for Dirac mixtures. Hence, a key contribution is to not well defined for the case of Dirac mixtures. Examples compare the corresponding cumulative distribution functions. are the Kullback–Leibler distance [9] and integral quadratic This paper concentrates on the simple and intuitive in- distances between the densities. Hence, in this paper the tegral quadratic distance measure. For the special case of a Dirac mixture with equally weighted components, closed– corresponding cumulative distribution functions of the true form solutions for special types of densities like uniform and density and its approximation are compared in order to define Gaussian densities are obtained. Closed–form solution of the an optimal Dirac Mixture approximation. This can be viewed given optimization problem is not possible in general. Hence, as a reversal of the procedure introduced in [10], where a another key contribution is an efficient solution procedure for distribution distance, in that case the Kolmogorv–Smirnov arbitrary true densities based on a homotopy continuation approach. test statistic, is used to calculate optimal parameters of a In contrast to standard Monte Carlo techniques like particle density given observed samples. filters that are based on random sampling, the proposed ap- Here, we apply the integral quadratic distance between the proach is deterministic and ensures an optimal approximation cumulative distributions, which is simple and intuitive. Other with respect to a given distance measure. In addition, the num- possible distribution distances include the Kolmogorov– ber of required components (particles) can easily be deduced by application of the proposed distance measure. The resulting Smirnov distance [11], or the Cramer–von´ Mises distance approximations can be used as basis for recursive nonlinear [12], [13]. For the special case of a Dirac mixture with filtering mechanism alternative to Monte Carlo methods. equally weighted components, closed–form solutions for spe- cial types of densities like uniform and Gaussian densities are I. INTRODUCTION obtained. The more general case for non-equally weighted Bayesian methods are very popular for dealing with sys- components is discussed in [14]. Since a closed–form solu- tems suffering from uncertainties and are used in a wide tion of the given optimization problem is not possible in range of applications. For nonlinear systems, unfortunately general, an efficient solution procedure for arbitrary true the processing of the probability density functions involved densities based on a homotopy continuation approach similar in the estimation procedure, typically cannot be performed to the approach introduced in [15] is applied. exactly. This effects especially the type of density in recur- The approximations yielded by the approach presented sive processing, which changes and increases the complexity. in this paper can immediately be used for implementing a Hence, nonlinear estimation in general requires the approxi- recursive nonlinear filter that could serve as an alternative to mation of the underlying true densities by means of generic the popular particle filters. In contrast to a standard particle density types. representation, the proposed approach provides an optimal In literature different types of parametric continuous den- approximation with respect to a given distance measure. sities have been proposed for approximation, including Gaus- Furthermore, the approach is deterministic, since no random sian mixtures [1], Edgeworth series expansions [2], and ex- numbers are involved. In addition, the number of required ponential densities [3]. Furthermore, discrete approximations components (particles) can easily be deduced by taking the are very popular. A well known approach is to represent the distance measure presented into account. true density by means of a set of samples [4]. This is used The paper is organized as follows. After the problem by the class of particle filters [5]. Typically, the locations and formulation in Section II, the conversion of the approx- weights of the particles are determined by means of Monte imation problem into an equivalent optimization problem Carlo techniques [6], [7]. is explained in Section III. Closed–Form solutions of this In this paper we provide a different view on such a discrete optimization problem for special types of densities are given representation. The given data points are interpreted as a in Section IV. A general solution approach for the case of mixture of Dirac delta components in order to systematically arbitrary densities is then given in Section V. Conclusions and a few remarks about possible extensions and future work 1 are given in Section VI. → It is important to note that this paper is restricted to the ) 0.5 x ( case of scalar random variables. Furthermore, the focus is F on the special case of Dirac mixtures with equally weighted 0 −0.2 0 0.2 0.4 0.6 0.8 1 1.2 components. This dramatically simplifies the derivations and x → allows for closed-form approximations in some important 1 cases. → ) 0.5 x II. PROBLEM FORMULATION ( ˜ F We consider a given density function f(x). The goal is to 0 −0.2 0 0.2 0.4 0.6 0.8 1 1.2 approximate this density by means of a Dirac mixture given x → by 1 L f(x, η)= w δ(x − x ) , → i i (1) ) 0.5 x i=1 ( F where in the context of this paper, the weighting factors 0 −0.2 0 0.2 0.4 0.6 0.8 1 1.2 wi are assumed to be equal and given by wi =1/L. The x → parameter vector η contains the positions of the individual Dirac functions according to Fig. 1. Approximation of the uniform distribution for a different number L =5 L =10 L =15 T of components , , and . η = x1,x2,...,xL . For the remainder of this paper, it is assumed that the Theorem III.1 The optimal parameters xi, i =1,...,L of positions are ordered according to the Dirac mixture approximation f(x, η) according to (1) of ˜ x1 <x2 <...<xL−1 <xL . a given density f(x) with respect to the distance measure (3) are obtained by solving Our goal is to minimize a certain distance measure G be- f˜(x) f(x, η) ˜ 2 i − 1 tween the given density and its approximation . F (xi)= , (4) However, standard measures of deviation are not well defined 2 L for Dirac mixture densities. for i =1,...,L. III. THE OPTIMIZATION APPROACH PROOF. The necessary condition for a minimum of the distance G(η) G(η) The first key idea is to reformulate the above approxi- measure is satisfied by the roots of the derivative of with respect to the parameter vector η according to mation problem as an optimization problem by minimizing ˜ a certain distance between the true density f(x) and its ∂G(η) =0 . approximation f(x). Instead of comparing the densities di- ∂η rectly, which does not make sense for Dirac Delta functions, x the corresponding (cumulative) distribution functions are For the individual parameters i we obtain Z ! employed for that purpose. ∂G(η) ∞ XL ˜ The distribution function corresponding to the true density = −2 F (x) − wj H(x − xj ) δ(x − xi) dx ∂xi ˜ −∞ j=1 f(x) is given by x for i =1,...,L. Using the fundamental property of the Dirac delta F˜(x)= f˜(t) dt , function gives −∞ ! ∂G(η) XL ˜ the distribution function corresponding to the Dirac mixture = −2 F (xi) − wj H(xi − xj ) . ∂xi approximation can be written as j=1 x L Evaluating the Heaviside function and setting the result to zero F (x, η)= f(t, η) dt = wiH(x − xi) , (2) finally gives the desired result −∞ i=1 ˜ 2 i − 1 ! F (xi) − =0 where H(.) denotes the Heaviside function defined as 2 L ⎧ i =1,...,L ⎨ 0,x<0 for . H(x)= 1 ,x=0 . ⎩ 2 1,x>0 IV. SOLUTION FOR SPECIAL CASES A distance measure can then given by For illustrating the usefulness of the result given in The- ∞ 2 orem III.1, we consider two special types of densities that G(η)= F˜(x) − F (x, η) dx . (3) admit a closed–form solution for the desired parameters of −∞ the Dirac mixture approximation. A. Special Case: Uniform Density → → ) Without loss of generality, we consider the uniform density ) x x ( ⎧ ( f ⎨ 0,x<0 F ˜ f(x)= 1, 0 ≤ x<1 . (5) −2 0 2 −2 0 2 ⎩ 0,x≥ 1 x → x → → The corresponding distribution function is given by → ) ⎧ ) x x ( ⎨ 0,x<0 ( f F˜(x)= x, 0 ≤ x<1 . F ⎩ 1,x≥ 1 −2 0 2 −2 0 2 x → x → ˜ Hence, in (4) we have F (xi)=xi, and the parameters of → → ) the Dirac mixture approximation are immediately given by ) x x ( ( f 2 i − 1 F x = i 2 L (6) −2 0 2 −2 0 2 x → x → for i =1,...,L. F˜(x) The true distribution and its Dirac mixture approx- Fig.