VISUAL TARGET TRACKING USING IMPROVED AND COMPUTATIONALLY EFFICIENT PARTICLE FILTERING Y Zhait, M. Yearyt, J.-C. Noyer tt, J. Havlicekt, S. Nematit, and P. Lanvin tt tSchool of Electrical and Computer Engineering University of Oklahoma, Norman, OK USA ttLaboratoire d' Analyse des Systemes du Littoral Universite du Littoral Cote d'Opale, Calais Cedex, France E-mail: [email protected], [email protected] ABSTRACT filter uses a set of weighted samples (called particles) to ap- In this paper, we present a new particle filtering (PF) al- proximate the distribution of the target motion states based gorithm for visual target tracking where Galerkin's projec- on system measurements. At each iteration, the particles are tion method is used to generate the proposal distribution. drawn from the proposal distribution, which ideally should Galerkin's method is a numerical approach to approximate the derive its support from the true state distribution. Tradition- solution of a partial differential equation (PDE). By leverag- ally, the state transition prior is used as the proposal distri- ing this method in concert with L2 theory and the FFT, we ob- bution in standard PF algorithms such as, e.g., the conden- tain a new proposal which directly approximates the true state sation algorithm [4]. But due to the fact that this proposal posterior distribution and is fundamentally different from var- does not account for the currently observed data, it is prone to ious local linearizations or Kalman filter-based proposals. We loss of track lock when a target exhibits unpredictable kine- matics. Other improvements include the localization based apply this improved PF algorithm to track a human head in a video sequence. As predicted by theory and demonstrated by method [2], the unscented particle filter [5], and the state par- tition based method these methods are based our experimental results, this new algorithm is highly effec- [7]. However, tive for tracking targets which exhibit complex kinematics. on assumptions of Gaussianity, depend on a linearization pro- The new proposal distribution given here captures the high cess, or both, which can result in nontrivial modelling errors. probability area in the state space, thereby gleaning increased In this paper, we propose a new particle filter algorithm support from the true posterior distribution. in which Galerkin's projection method is used to generate the proposal distribution. The rationale behind Galerkin's method Index Terms- TV surveillance systems, nonlinear fil- is to assume that the state posterior distribution is in L2 (Rn). ters, computer vision Then, this distribution can be approximated by its projection onto a finite set of orthogonal basis vectors. In addition, by 1. INTRODUCTION choosing a special set of exponential basis vectors, the pro- jections can be approximated by the computationally efficient The objective of visual target tracking is to estimate the po- FFT. This approach does not require any local linearization sition and velocity of an object given observations acquired of the nonlinear systems and also does not require imposing from a video sequence. This task is important in many appli- any Gaussianity assumption on the system state distribution. cations including, e.g., surveillance, visual servoing, gestu- Thus, it is fundamentally different from the various Kalman ral human-machine interfaces, motion capture, robotics, and filter based PF algorithms such as those in [2,5,7]. many others [1,2]. Recently, a new tracking algorithm called particle filtering has captured international attention because 2. PROBLEM FORMULATION of its exceptional performance in nonlinear and non-Gaussian tracking problems [3-6]. This is in part due to the increas- Target tracking is often formulated as a state estimation prob- ing availability of low cost, high performance computing plat- lem. Particle filtering is a nonlinear state estimation tech- forms capable of accomodating the significant complexity of nique which is based on sequential importance sampling and practical particle filtering algorithms. In essence, a particle Bayesian inference. Consider the state space model: xt This work was supported in part by the U.S. Army Research Laboratory f(xt-1) +wt- 1 and yt = h(xt) +vt, where xt and yt denote and the U.S. Army Research Office under grant W91 lNF-04-1-0221. the hidden states and the measurements, respectively. Both 1-4244-0481-9/06/$20.00 C2006 IEEE 1757 ICIP 2006 f(0) and h(.) could be nonlinear functions, wt and vt denote orthogonal basis for L2 (Rn) and where cl (t) is the projection the process and measurements noises. Our estimation goal of p(X, t) onto Xl (x) at time t defined by is to evaluate the posterior state distribution p(xt lYl:t) which is governed by the Chapman-Kolmogorov equation and the (p(x, t), li(x) Jp(x,t)ol(x)*dx . (5) Bayes' formula given as follows [3]: Our objective is to find an approximation p(x, t) of p(x, t) P(XtdYi:t- ) = p(xtlxt_i)p(xt_j1yi:t_ )dxt_1, (1) such thatp(x, t) = LNv l cl(t) 01(X). The approximation error in p arises from the use of only P(Yt t-l ) N -1 as opposed to an infinite number of basis elements q5. P(XtYi:t) = Xt)P(xt iY ) fp(yt xt)p(xt Yj:t_ )dxt (2) The projections cl(t),I = 0,-- ,N -1, are the values to be determined. With this setup, we project Y(x, t) onto the In the framework of particle filtering, the state posterior dis- subspace span{q$(X)}IN- as: Y(x,t), 1(x)) = 0, 1 tribution can be approximated by a set of weighted samples 0, * *, N- 1. Here, instead of solving the original problem (also called particles) denoted by x(i), see [2] for details: Y(x, t) = 0, we solve it's projection, which is a collection of Ns N ordinary differential equations (ODE). Next we apply this P(xd Y1 t) ~ E z(0)(xt - x()), (3) method to the nonlinear state estimation problem defined in i=l Section 2. First we assume p(Xt lYit 1) L=o1 Cl(t) i, where where = (i)/ 1 is the normalized importance D(i) zN w) cl (t) will be determined later. for notational simplicity, we weight, and c(') is given as: drop the variable x. We apply Galerkin's method to equation (2) by projecting it onto the span{ql (X)}IN0 as: (i 1 (i t1) N-1 t =) t( 1 t);(yq(x(')Y ~xj')PXlxt 4 -I,yl:t)(4 (P(Xt Yl:t), k) =E Cl(t)(, k) 1=0 Equation (3) is actually the discretized posterior distribution. In addition, the distributions P(Yt ) and p(X(i) Ix(i)) rep- ELN cl (t) (P(Yt Xt)9l, 9k (6) resent the system's likelihood and the state transition prior, EL=o (t) (P(Yt Xt),O*)1 respectively. At each iteration, the particles (or samples) are where k = o,... , N -1. Eq. (6) is the "projection version" drawn from a proposal distribution x(i) q(xt x i)1', Yo t) of the Bayes' formula given in (2). For simplification, (6) can which is effective provided that it has sufficient support from be written in a matrix form as the true posterior distribution. Choosing a good proposal is a crucial and challenging step in designing a particle filter. C(t) = vC(/)'( (7) 3. NEW TRACKING ALGORITHM where Yt is an N x N matrix with the elements [Ytlk,l = (P(Yt |Xt)>i, Ok) The variables C(t), C(t) and vt are N x 1 As indicated in Section 1, the traditional choice of the pro- vectors, with [vt]l = (p(yt xt), 0*). We choose the exponen- posal cannot provide enough support from the posterior when x tial basis as = a where a and b tracking a target with complex kinematics. This is because the l(x) exp (j271 )', state transition prior does not include the most recent mea- are the integral limits. It was shown in [8] that with this basis surement information. In this section, we introduce an im- the inner product can be approximated with an FFT as proved technique, based on Garlerkin's method, for designing 0o) the proposal distribution. (P(x), 3.1. Galerkin's Method and Computational Efficiency N FFT[p(x)] F (P(X;), ON-1) 1 Galerkin's method is ubiquitous in the solution of PDEs, and in fact forms the basis for the finite element, finite difference, and boundary element methods. Galerkin's method is a dis- p(x), o1) cretization procedure that represents the solution in terms of [ . /b IFFT[p(x)] an orthogonal function expansion where each trial function _ (P(), ON*-1) _ satisfies certain boundary conditions [8]. Let Y(x, t) = 0 de- notes a PDE, which is a function of the temporal variable t Then, by using the FFT, (7) can be approximated as and spatial variable x. Assume p(x, t) C L2(IRJ). is the solu- [Yttli( a(/N)b FFT [p(yt xt) 475i] (8) tion of the PDE, such that it can be decomposed according to vt b IFFT [p(yt|xt)] p(x,t) = E c01Ej(t)0j(x), where {$l(x)}-0 is a complete (9) 1758 By applying Galerkin's method to equation (1) in a similar 4. LABORATORY EXPERIMENTS way, cl (t) can be obtained according to As an illustrative example, we apply the improved PF algo- Cl(t) -(vb a/V)IFFT1 [cl(t-I)FFT,[p(xt xt- )]] (10) rithm to track a human in a video sequence by using the fol- where the FFT, [.] represents the Ith bin of the FFT of the lowing state space model: argument.