Exact Bayesian and Particle Filtering of Stochastic Hybrid Systemssystemssystems

UNCLASSIFIED Nationaal Lucht--- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR

Executive summary

Exact Bayesian and particle filtering of stochastic hybrid systemssystemssystems

Problem area Description of work Report no. In literature on Bayesian filtering of This report considers filtering of NLR-TP-2006-691 stochastic hybrid systems most stochastic hybrid systems that go studies are limited to Markov jump beyond the well known Markov Author(s) systems. The main exceptions are jump system. First the non- H.A.P. Blom, E.A. Bloem approximate Bayesian filters for Markovian jump system studied in Report classification semi-Markov jump linear systems. this report is formally defined. Next Unclassified These studies showed that nonlinear the exact Bayesian filter recursion filtering becomes much more for this system is developed and the Date challenging under non-Markov implication of going beyond September 2006 jumps. This challenge however does Markov jumps is shown. not apply to particle filtering of Subsequently a novel particle filter Knowledge area(s) stochastic hybrid systems. In for jump nonlinear systems is Advanced (sensor-) information practice, non-Markov jumps rather developed which is referred to as processing are the rule, not the exception. For the Interacting Multiple Model example, on an airport, the particle filter (IMMPF). For probability at which a taxiing comparison Monte Carlo aircraft makes a maneuver depends simulations are performed. heavily on its position; e.g. when Descriptor(s) taxiing near a crossing on the Results and conclusions Stochastic hybrid systems airport, the probability of starting a Through Monte Carlo simulations Bayesian filtering turn is relatively high, whereas IMMPF has been tested and Particle filtering outside these areas this probability compared with standard Particle State dependent switching may be very small. A similar Filter (PF) and IMM. The results Jump-nonlinear systems difference applies to the probability show that the IMMPF performs Non-Markov jumps Maneuvering target tracking very well, even in cases where the of an aircraft making a turn when it is flying near a waypoint versus standard PF or IMM have problems. flying halfway two waypoints. Similar non-Markov jump behavior Applicability also applies to other traffic The applicability of the work modalities, and to any other comprises the implementation of intelligently controlled system. the resulting filtering algorithms in Nevertheless in target tracking, a multitarget tracker, in particular particle filtering studies have the Advanced suRveillance Tracker continued to focus on Markov jump And Server ARTAS. systems

Nationaal Lucht- en Ruimtevaartlaboratorium, National Aerospace Laboratory NLR

Anthony Fokkerweg 2, 1059 CM Amsterdam, P.O. Box 90502, 1006 BM Amsterdam, The Netherlands UNCLASSIFIED Telephone +31 20 511 31 13, Fax +31 20 511 32 10, Web site: www.nlr.nl Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR

NLR-TP-2006-691

Exact Bayesian and particle filtering of stochastic hybrid systems

H.A.P. Blom and E.A. Bloem

This report contains an article to appear in IEEE Transaction on Aerospace and Electronic Systems.

This report may be cited on condition that full credit is given to NLR and the authors.

Customer: National Aerospace Laboratory NLR Working Plan number: 2005 AT.1.C Owner: National Aerospace Laboratory NLR Division: Air Transport Distribution: Unlimited Classification title: Unclassified September 2006 Approved by: Author Reviewer Managing department Anonymous peer reviewers

Exact Bayesian and Particle Filtering of Stochastic Hybrid Systems

Henk A.P. Blom Senior Member IEEE & Edwin A. Bloem

challenging under non-Markov jumps. This challenge Abstract —The standard way of applying particle filtering to however does not apply to particle filtering of stochastic stochastic hybrid systems is to make use of hybrid particles, hybrid systems [5]. Hence there is no reason to continue where each particle consists of two components, one assuming focusing on Markov jump systems. In practice, non-Markov Euclidean values, and the other assuming discrete mode values. This paper develops a novel particle filter for a jumps rather are the rule, not the exception. For example, discrete-time stochastic hybrid system. The novelty lies in the on an airport, the probability at which a taxiing aircraft use of the exact Bayesian equations for the conditional mode makes a maneuver depends heavily on its position; e.g. probabilities given the observations. Therefore particles are when taxiing near a crossing on the airport, the probability needed for the Euclidean valued state component only. The of starting a turn is relatively high, whereas outside these novel particle filter is referred to as the Interacting Multiple areas this probability may be very small. A similar Model (IMM) particle filter because it incorporates a filter step which is of the same form as the interaction step of the difference applies to the probability of an aircraft making a IMM algorithm. Through Monte Carlo simulations, it is turn when it is flying near a way-point versus flying halfway shown that the IMM particle filter has significant advantage two way-points. Similar non-Markov jump behavior also over the standard particle filter, in particular for situations applies to other traffic modalities, and to any other where conditional switching rate or conditional mode intelligently controlled system. Nevertheless in target probabilities have small values. tracking, particle filtering studies have continued to focus

on Markov jump systems [6] – [19]. Keywords: Stochastic hybrid systems, Bayesian filtering, Particle filtering, state dependent switching, jump-nonlinear systems, non-Markov jumps, maneuvering target tracking. In nonlinear filtering studies, the Sampling Importance Resampling (SIR) based approach [20] has surfaced as the I. INTRODUCTION baseline particle filter. It has shown to form an elegant and The paradigm of particle filtering has stimulated a general approach towards the numerical evaluation of the renewed interest in exact Bayesian filtering of non-linear conditional density solution of the Chapman-Kolmogorov- stochastic systems. In line with this, [1] recently developed Bayes (CKB) filter recursion [21] – [24]. For a large class the exact Bayesian filter for Markov jump non-linear of problems it has been shown that increasing the number of systems. The aim of the current paper is to extend the exact particles ensures weak sense convergence of the Bayesian filter characterization to the much larger class of approximation density to the exact conditional density at a non-Markov jump non-linear systems, and subsequently to rate that is independent of or increases linearly with the exploit this exact Bayesian characterization for the state dimension of the process to be estimated [25] – [27]. development of a novel particle filter for stochastic hybrid This means that the SIR particle filter significantly relaxes systems. the curse of dimensionality of CKB filtering. Practically, In literature on Bayesian filtering of stochastic hybrid the SIR particle filter has found a large variety of useful systems most studies are limited to Markov jump systems. applications, including some where established non-linear The main exceptions are approximate Bayesian filters for filtering approaches do not work at all, such as the visual semi-Markov jump linear systems [2] – [4]. These studies tracking example of [28] and the track-before-detect showed that nonlinear filtering becomes much more example of [29]. Because of its generality and theoretical validity, the SIR particle filter has become a useful reference in numerically approximating CKB filter Manuscript received October 25, 2004; revised October 7, 2005; performance [18]. released for publication March, 2006 This work was supported in part by the European Commission project HYBRIDGE, IST-2001-32460. The SIR particle filter is also capable in approximating Both authors are with the National Aerospace Laboratory NLR, P.O. Box the CKB equations of a stochastic hybrid Markov process 90502, 1006BM Amsterdam, The Netherlands (phone: +31 20 5113544; θ n θ fax: +31 20 5113210; e-mail: [email protected], [email protected]). {xt , t } , with xt assuming values in R , and t assuming

θ values in a finite set M of possible modes [5]. Let {xt , t } work well without the need of ordering the samples be the hidden state process to be estimated from noisy according to their weights. • Boosting diversity by regularization, i.e. prior to observations, then the SIR particle filter use NP particles, resampling, replace each Dirac measure in the jθ j particle j of which has two components (xt , t ) at moment empirical distribution by an absolutely continuous j θ j t, with xt assuming an Euclidean value and t assuming a distribution. This approach has largely been mode value. Each SIR particle filter cycle from t-1 to t developed for particle filtering of stochastic consists of three steps: processes that, without regularization, do not satisfy the regularity conditions under which the • Evolution: For each of the N particles at moment t-1, p weak convergence to the exact conditional density jθ j is satisfied [12]. A valuable alternative is to insert draw a new hybrid particle (xt , t ) according to the Chapman-Kolmogorov transition kernel; a Monte Carlo Markov Chain (MCMC) move in each SIR cycle [31], [11]. • Correction: For each of the N particles evaluate µ j p t • Importance density based resampling. This means as the likelihood of the measurement at moment t; that particle resampling anticipates how the jθ j µ j given (xt , t ) and normalize the resulting t ’s. weights per particle will evolve over the next prediction and correction steps. Each new particle • Resampling: Draw N independently identically p µ j sample receives a weight t that compensates for jθ j distributed (i.i.d.) hybrid particle values (xt , t ) , with this importance resampling. The optimal µ j importance density has been characterized by from the sum of t weighted Dirac measures at Doucet et al. [32]. Because exact implementation (x j ,θ j ) . t t is hard, approximations are needed. Best known is the auxiliary resampling method by [33]. Other Successful application of this hybrid state version of the relevant approximate importance resampling SIR particle filter has for example been shown for target methods have been developed by [32], [34] – [36]. tracking [6] – [7], signal processing [11] and failure • Rao-Blackwellization. This approach collects all monitoring and diagnosis [5]. In none of these example state components into two vectors. One or more applications the conditional mode probabilities and/or state components that satisfy a linear equation and switching rates assumed very small values. Otherwise, there that do not appear in the measurement equation, might have been very few (or zero) particles for one or are placed in the first vector. The other state more of the mode values, and then the empirical density components are placed in the second vector. For spanned by the particles with such a mode value does not the second vector, all filtering steps are performed form an accurate approximation of the mode probability or through particles [37]. Per particle, the conditional the corresponding mode conditional density. A brute force density of the first vector follows from a Kalman approach in compensating for sample degeneracy would be filter, in which the value of the particle plays the to increase the number of particles. The aim of this paper is role of the measurement, e.g. [9], [32], [38] – [40]. to develop a more elegant approach in improving the SIR particle filter for stochastic hybrid systems. Typically an improvement of the SIR particle filter is used to reduce the number of particles needed to realize Degeneration of samples is a well known phenomenon in good filtering performance. Recently, [19] compared the particle filtering. Hence in literature various mitigation effectiveness of importance density resampling and Rao- approaches have been developed, i.e.: Blackwellization SIR improvement methods with the • Less frequent resampling, i.e. perform resampling baseline SIR particle filter. For some specific examples of only when the ”effective sample size” drops below tracking a frequently maneuvering target in clutter, a certain threshold [30]. importance density resampling in combination with less • Better resampling methods. [23] showed that a frequent resampling showed to be most effective in resampling method which systematically draws one mitigating sample degeneracy and without significantly random sample per group of samples having increasing the computational load per particle.

weight of 1/ N p reduces the chance of sample degeneracy. Another good alternative is residual All improvements of the SIR particle filter mentioned resampling [30]. A complementary advantage of above have been developed with focus on Euclidean valued systematic and residual resampling is that they state estimation. This suggests there is room for improvement of the SIR particle filter with focus on the

finite valued state component. To make this work, we apply Furthermore, a and h are measurable mappings of ′ ′ a novel kind of Rao-Blackwellization to a stochastic hybrid M× Rn × R n into Rn and M× Rn × R m into Rm process. We collect all Euclidean valued components in one respectively, and c is a measurable mapping of vector, and place the mode component only in the other M×Rn × [0,1] into M . The mappings a, c and h are time- vector. Subsequently we perform particle filtering to the Euclidean valued vector and apply the exact filter equations invariant for notational simplicity only. to the mode probabilities. Exactly the same splitting of mode and state components is used by the Rao- The filtering problem is to estimate the joint conditional n density-probability pθ ( x ,θ ) , x ∈R ,θ ∈ M , of the pair Blackwellization application to a Markov jump linear xt, t | Y t system [32], but with the allocation of particles and exact (x ,θ ) given the sequence of observations Y={ yst; ≤ } . filter equations the other way around. Independently of each t t t s other, [17] and [41] have recognized the value of this This problem is addressed in the sequel of this paper. In approach, with the latter including non-Markov jumps. preparation for this, in this section we characterize some θ Their particle filters for estimating the conditional densities properties of the solution {t ,x t , y t } of system (1)-(3). θ of xt given the mode t are equivalent. However, the Firstly, by an iterative way of working from t = 0 to t = 1, equations for the estimation of conditional mode = = probabilities differ as a result of the Markov/non-Markov then to t 2 , and so on to t T , it can be verified that for θ jump difference. The current paper gives a full exposition every initial condition (0 ,x 0 ) the system of equations (1)- θ of the more general approach developed by [41]. (3) has a unique measurable solution {t ,x t , y t } assuming values in M× Rn × R m × [0,T ] , T<∞. The paper is organized as follows. Section II formally defines the non-Markovian jump system studied in this θ paper. Section III develops the exact Bayesian filter Secondly, according to equation (2), t is defined as a θ recursion for this system, and shows the implication of function of t−1 , xt−1 and ut . This implies that the going beyond Markov jumps. Section IV develops a novel evolution of the process { θt} depends of { xt}. Hence the particle filter for jump nonlinear systems. This novel process { θt} is not a Markov process. particle filter has an interaction step which follows from the initial derivation [42] of the Interacting Multiple Model Thirdly, the joint process { xt,θt} can be shown to be a (IMM) filter algorithm. For this reason, the novel particle Markov process. Substituting (2) into (1): filter is referred to as the IMM particle filter. Section V xac= ( (θ , xuxw ,) , , ) (4) performs Monte Carlo simulations to compare the novel t ttttt−1 − 1 − 1 particle filters with the SIR particle filter and with the IMM; Together with (2), this implies that ( θt, xt) is a measurable θ for a fair comparison with IMM, the example considered is function of t−1 , xt−1 , ut and wt , for every t>0 . By a a Markov-jump linear system. Through parameter variation, repeated application of this functional relation, it follows the different effects of rare and non-rare switching θ that for every s>t>0, (s ,x s ) is a measurable function of probability is discussed. Finally, Section VI draws θ , x , (u ,..., u ) and (w ,..., w ) . This implies that the conclusions. t t t s t s θ joint process {xt , t } satisfies the Markov property. II. FORMULATION OF THE PROBLEM Π We consider the following system of stochastic Fourthly, we characterize the transition probability ηθ of difference equations, on [ 0,T ], T<∞, θ= η θ= θ a jump from t−1 to t in terms of properties of x= a(θ , x− , w ) (1) t tt1 t xt−1 and ut , Application of the total probability theorem θ= θ tc( ttt−1, x − 1 , u ) (2) and subsequent evaluation, using (2), yields:

y= h(θ , xv , ) (3) pθθ(|,)θη xp= θθ (,|,) u θη xdu t ttt ttt|,−−11x∫ u tttt ,|, −− 11 x R where the pair ( θt, xt) represents the hybrid system state, and yt represents the observation at moment t, { wt} and { vt} are = pθ θ (|,,)θ η xupudu () ∫ tttt|−1 ,x − 1 , u u t independent sequences of i.i.d. standard Gaussian variables R ′ ′ of dimension n and m respectively, { ut} is an {wt , v t } - = χ() θ η ,c ( , xu , ) pu ( udu ) (5) independent sequence of i.i.d. standard uniform random ∫ t [0,1] × n variables, {wt , v t , u t } is independent of the M R valued where χ a 0-1 indicator with χ(, θ θ ′ )= 1 iff θ= θ ′ . This initial condition ( θ0, x0), with M a set of M discrete modes.

means that it makes sense to define the state-dependent c(η,, xu) = Off , if η = Off , x <+> RHu , ε Π mode transition probabilities ηθ (x ) , as follows: =On , if η = Off , x <+≤ R H , u ε 2 n =η = ≥+> ε Π∆ηθ()xp θθ (|,),θη x (,) θη ∈∈M , x R On , if Off , x R H , u t| t−1 , x t − 1 =Off, if η = Off , x ≥+≤ R Hu , ε = χ() θ,c ( η , xu , ) p ( udu ) (6) ∫ ut =On , if η = On , x >−> R H , u ε [0,1] =Off, if η = On , x >−≤ R H , u ε

=η = ≤−> ε Fifthly, from (6) we can see that if c(θ , x , u ) is x-invariant Off, if On , x R H , u =On , if η = On , x≤ R − Hu , ≤ ε then Πηθ (x ) is x-invariant and { θt} is a finite state Markov For the transition probability this implies: process. Πηθ ()1x =−ε , if x ≥+ R H , η = Off , θ = On In return for loosing {θ }’s Markov property, the advantage =ε, if x <+= R H , η Off , θ = On t of working with an x-variant c(θ , x , u ) in (2) is the =−1ε , if x ≤ R -, H η = On , θ = Off capability to define a mode valued jump process {θt} that =ε, if x > R - H , η == On , θ Off incorporates stochastic hybrid processes where both the θ process { t} influences the evolution of process { xt} and In example 1 the mode switching is determined in a the process { xt} influences the evolution of process { θt}. In deterministic way by the evolution of { xt}, and the i.i.d. order to illustrate this effect, we give two examples where sequence { u } has no influence on this at all. This { x }- mode switching depends on the evolution of { x }. t t t determined mode switching character is totally different from the well known Markov kind of { u }-determined mode t switching. In example 2 the pure { x }-determined mode Example 1 : We consider an open tank receiving an input t switching behavior is combined with some random { u }- flow of liquid from above. Liquid can be taken out / left in t determined mode switching. the tank by switching a pump on/off. The pumping mode is captured by a process { θ } assuming values in {On,Off} . t In [5] it is demonstrated that filtering for stochastic hybrid The level of liquid in the tank is captured by an Euclidean systems, which incorporate the example 1 kind of xt - valued process { xt}. Assume the desired reference level of liquid in the tank is R. In order to avoid that the pump is dependent switching, is well handled by the SIR particle rapidly switching on and off, the switching takes into filter. For example 2, however, a SIR particle filter with N p ε = ε >> account a hysteresis of strictly positive size H in level. particles works well when 0 or when 1 / N p , but Hence a pump in On -mode is switched to Off as soon as the <ε < not when 0 1 / N p . Such discontinuity in behavior is level x , hits the level R-H. Similarly, an Off -mode pump is t rather unnatural, and asks for the development of an switched to On as soon as the level x hits the level R+H. t appropriate improvement of the SIR particle filter. Prior to Hence the mapping c satisfies: developing an adequate solution for this in section IV, we c(η, xu ,) = Off , if η = Off AND xRH <+ first characterize the exact Bayesian filter recursion from =On , if η = Off AND x ≥+ R H the general CKB equations. =On , if η = On AND x >− R H =Off, if η = On AND x ≤− R H III. EXACT BAYESIAN FILTER Substituting this into eq. (6) and evaluation yields: Πηθ ()x = 1, if x ≥+ R H , η = Off , θ = On In this section we develop the exact recursive equations for the joint conditional density-probability pθ ( x ,θ ) , =0, if x <+ R H , η = Off , θ = On xt, t | Y t σ =1, if x ≤ R - H , η = On , θ = Off where Yt denotes the -algebra generated by

=0, if x > R - H , η = On , θ = Off measurements yt up to and including moment t . For short

we refer to pθ ( x ,θ ) as the conditional density. xt, t | Y t Example 2 : For the same tank example we now assume Because { x ,θ } is a Markov process, the characterization of pump switching errors happen at a rate ε . This can be t t this conditional density consists of two steps: modeled by the c-mapping as follows:

• Chapman-Kolmogorov (CK) evolution from t −1 to t, pθ(,) xθ= p θ (|,) xxp′ θ θ (,) xdx ′ θ ′ (10) xYttt,|−1∫ xx ttt |, − 1 xY ttt −− 11 ,| i.e. characterize p,θ ( x ,θ ) as a function of xt t| Y t −1 Rn p,θ ( x ,θ ) . xt−1 t − 1| Y t − 1 Substitution of (8) into (10) yields • Bayes measurement update, i.e. characterize ,θ ,θ   px,θ | Y ( x ) as a function of px,θ | Y − ( x ) . t t t t t t 1 pθ(,) xθ= p θ (|,). xx θ′ Π ηθ (') xp θ (,) xdx ′ η  ′ xYt,| t t−1∫ xx t |, t t − 1 ∑ xYt−1 ,|, t − 1 t − 1 η∈  Rn M  The Bayes measurement update works similar as for (11) measurements of a Markov jump non-linear system, e.g. [1]. For the characterisation of the CK step of the non- Together with Bayes theorem for the measurement update Markov jump system, however, this derivation path falls step, (11) yields (12) in the Theorem below. short. Hence, we follow other derivation paths. One derivation path uses the law of total probability, and is Theorem 1 based on [41]. The second derivation path uses the Let h(,θ x , w ) be such that the density pθ ( y | x ,θ ) is Chapman-Kolmogorov equation directly, and is based on yt| x t , t n [14]. First we show the law of total probability derivation measurable for all (θ ,x ) ∈ M × R , let (θ0,x 0) admit a path, then we formally state the result in Theorem 1, measurable density pθ (θ , x ) , and let the transition including the assumptions adopted. Next we formally prove 0,x 0 θ Theorem 1 following the Chapman-Kolmogorov derivation kernel of the Markov process { t,x t} admit a measurable transition density pθ θ (, xθ | x ′ ,) η , for all path. xttt, | x −1 , t − 1 (η ,x′ ) , (θ ,x ) ∈M × Rn , then the conditional density By law of total probability satisfies pxθ(,)θ= p θ θ (,,) x θ η = xYttt−−11,|∑ xY tttt −−− 111 ,,| θ= , θ θ ′ η∈M pxY,|θ(,) x p yxt |, θ (| yx ) p xx |, θ (|,) xx (7) ttt ttt∫ ttt −1 Rn = pθ θ(|,)θ xp η θ (,) x η ∑ tttt|,,xY−−−111 xY ttt −−− 111 ,| η∈ .Πηθ (x ') p θ ( x′ ,η )  dx ′ / c (12 ) M ∑ xt−1, t − 1 |, Y t − 1  t η∈M with c a normalization constant. Because ut is independent of Yt−1 , from eq. (2) follows t θ that t is conditionally independent of Yt−1 given Proof: (using Chapman-Kolmogorov) (θ ,x ) . Hence (7) yields t−1 t − 1 Because pθ ( y | x ,θ )∈ (0, ∞ ) for all (y ,θ , x ) ∈ yt| x t , t pθ(,) xθ= p θθ (|,) θη xp θ (,) x η xYttt−−11,|∑ ttt |, x −− 11 xY ttt −−− 111 ,| Rn×M × R n η∈M , application of (generalised) Bayes theorem (e.g. [43], p.129) yields:

= Π ηθ (x ) p θ (, x η ) (8) ∑ xt−1, t − 1 | Y t − 1 ,=θ , θ , θ pxY,θ|() x p yxt | , θ (|) yxp xY , θ | − ()/ xc t (13) η∈M ttt ttt ttt 1 θ Also by law of total probability: Because {xt , t } is a Markov process, the evolution from t −1 to t satisfies the Chapman-Kolmogorov equation pθ(,) xθ= p θ (,,) xxdx θ ′ ′ = xYttt,|−1∫ xxY tttt ,,| − 1 − 1 Rn pθ(,) xθ= p θθ (,|,) xxp θη′ θ (,) xdx ′ η ′ (9) xYttt,|−1 ∫ ∑ xx tttt ,|,−−11 xY ttt −−− 111 ,| η∈ = ′θ ′ θ ′ Rn M pxxY|,,θ(|,) xxp xY ,| θ (,) xdx ∫ tttt−−11 ttt −− 11 (14) Rn Factoring the Markov transition density in (14) yields Because wt is independent of Yt−1 , from (4) follows that x is conditionally independent of Y given (x ,θ ) . t t−1 t−1 t − 1 pθθ(,|,) xxpθη′= θθ (|,,) xxp θη ′ θθ (|,) θη x ′ xxtttt,|,−−11 xx tttt |,, −− 11 ttt |, x −− 11 Hence (9) yields

=pθ θ ( xx |θ ,′ , η ) Π ηθ ( x ') (15) xttt| , x −1 , t − 1

θ Because wt is independent of t−1 , from (1) follows that Remark 1: If c is x-invariant, a and h are linear in ( x,w) θ θ and ( x,v ) respectively, p θ (. |θ ) is Gaussian for all θ, xt is conditionally independent of t−1 given xt−1 and t . x0| 0 , Y 0 t+ 1 Using this in (15) yields then p θ (. |θ ) is a mixture of N Gaussian densities xt| t , Y t [44] – [45]. If we stick to all these conditions with the pθθ(,|,) xxθ′ η= p θ (|,) xx θ ′ Π ηθ (') x (16) xxtttt,|,−1 − 1 xx ttt |, − 1 exception of the x-invariant c, then the appearance of the Πηθ (x ) in (12) destroys this exact Gaussian mixture Substituting (16) into (14) and shifting summation yields solution. The best hope for some novel Gaussian mixture solution then is that Πηθ (x ) has a Gaussian shape (or the pθ(,) xθ= p θ (|,). xx θ ′ xYttt,|−1 ∫ xx ttt |, − 1 shape of a finite Gaussian mixture) for every η, θ . R n (17)   .Π (xp ') ( x′ ,η )  dx ′ For the IMM particle filter development it is relevant to ∑ ηθx−, θ − | Y − t1 t 1 t 1  η∈M  decompose the exact recursive filter equation in Theorem 1 into a sequence of basic transitions. The following sequence Substituting this into (13) yields (12). Q.E.D. of transitions defines such a decomposition:

Eq. (12) shows that state dependent mode switching Mode Switching → pθ  t|Y t −1 probabilities have a multiplicative effect on the mode- p θ  xt−1, t − 1 | Y t − 1 →State Interaction p θ switching-conditional evolution of the Euclidean valued  xt−1| t , Y t − 1 state xt . For hybrid stochastic processes this means there State Prediction pθ→ p θ are two kinds of multiplication of conditional densities with xYt−1,| t t − 1 xYt ,| t t − 1 Correction other densities. One is due to Bayesian updating of pθ→ p θ xYt,| t t−1 xYt ,| t t measurements and the other is due to state dependent switching between modes The output of the first two transitions is integrated through the following equation: If c(θ , x , u ) is x-invariant then { θ } is a Markov process , t and the transition probability Πηθ (x ) is x-invariant. Then pθ(,) xpθ= θ (|) xp θ θ () θ (19) xYttt−−11,| xY ttt −− 11 |, tt | Y − 1 the term Π (x ) in (12) can be shifted out of the ηθ integration over dx ′ , and we get the Corollary below. We follow the above four transitions in developing the IMM particle filter, and do this in two steps. As first step Corollary 1 (for Markov jump systems) we characterize the four transitions using results from Let the assumptions of Theorem 1 hold true, and let section III. c(θ , x , u ) be x-invariant for (θ ,x ) ∈ M × Rn , then Mode switching: The mode switching transition θ= , θ characterizes how the conditional mode probabilities evolve pxY,θ |(,) x p yx | , θ (| yx t ) ttt ttt from t −1 to t . By law of total probability   ′ ′ θ= () θ .Πηθp θ ( xxp |θ , ) θ ( xdxc , η ) ' / (18) pθ|Y( ) p x , θ | Y x , dx ∑ ∫ xxttt| ,−1 xY ttt −−− 111 , |, t tt−1∫ ttt − 1 − 1 η∈   n M  Rn  R Substitution of (8) and subsequent evaluation yields with c a normalization constant t pθ()θ= Π ηθ () x p θ () x , η dx = t|Y t−1 ∫ ∑ xt−1 , t − 1 | Y t − 1 η∈ Rn M Comparison of (18), for Markov jump systems, with (12),   for non-Markov jump systems, shows a relative small =pθ() x|η Π ηθ () x dx ⋅ p θ () η  (20) Π ∑ ∫ xttt−−−111| , Y tt −− 11 | Y difference: ηθ is outside and inside the integration over η∈   M Rn  x′∈ Rn respectively. Because of this difference, under the non-Markov jump situation it is no longer possible to make State interaction: The state interaction transition µ explicit use of IMM’s mixing probabilities t,η |θ in the characterizes how the conditional state densities are being derivation of the exact Bayesian filter recursion (12). This mixed under influence of mode switching. By dividing both left and right hand terms in (8) by pθ (θ ) we get for derivation path has been followed in [1] to get recursion t|Y t −1

(18). pθ (θ )> 0 : t|Y t −1

p(|) xθ= Π () xp (|)/ xp η  () θ S  xY−|,θ − ∑ ηθ xY−−− ,| θ θ | Y − (21) η,j η , j t1 t t 1 ttt111  tt 1 pɶ (θ ) =Π() x µ δ () xxdx −=  η∈ θ|Y − ∑ ηθ ∑ t−1 t − 1 M t t 1 ∫   n η∈M j = 1  Eq. (21) represents the IMM interaction step at the level of R S the mode-conditional density. For Markov jump linear η η = Π ()x ,jµ , j (25) systems this form has been derived for the first time in [42] ∑ ∑ ηθ t−1 t − 1 η∈ = eq. (20). It is also derived in [46], section II, following M j 1 another path of derivation. In later expositions on IMM, e.g. [47], p.461, [48], p.453, eq. (21) is derived in an This forms the mode switching step of the IMM particle approximate way only. filter.

Remark 2 : If the condition pθ (θ )> 0 is not satisfied Interaction resampling step of IMMPF : t|Y t −1 Next, substituting approximation (24) into (21) and for one or more θ -values then a practical way of handling subsequent evaluation yields: this is to skip evaluation of eq. (21) and assume an arbitrarily bounded density for the corresponding θ S  pθ ( x | ) ’s. η,j η , j xt−1| t , Y t − 1 pxɶ (|)θ=Π() x µδ() xxp −  /()ɶ θ = x−|θ , Y − ∑ ηθ ∑ t−1 t − 1 θ | Y − t1 t t 1   t t 1 η∈M j = 1  State prediction : The state prediction transition S characterizes the evolution of mode conditional state η,j η , j η , j = Πηθ()()x−µ − δ xx − − / pɶ θ ( θ ) (26) ∑∑ t1 t 1 t 1 t|Y t −1 densities from t −1 to t . From (1) we get η∈M j = 1 ′ ′ θ pθ (|,) xxθ= δ() xaxwpwdw − () θ ,, () (22) Eq. (26) makes clear that pɶ x|θ , Y ( x | ) is an empirical xt| x t−1 , t ∫ w t t−1 t t − 1 [0,1] density spanned by NP = MS particles, whereas (24) θ shows that pɶ x|θ , Y ( x | ) was an empirical density Substituting this in (10) yields t−1 t − 1 t − 1 spanned by S particles only. A logical way to reduce the pθ (,) xθ= δ() xaxwpwdw −() θ ,,′ () ⋅ xt, t | Y t−1 ∫ ∫ w t ɶ θ number of particles to span px−|θ , Y − ( x | ) is to perform a Rn [0,1] t1 t t 1 resampling of eq. (26). This forms the interaction .pθ () x′ ,θ dx ′ (23) xt−1, t | Y t − 1 resampling step of the IMM particle filter.

Correction: The correction transition is characterized by Remark 3: Choosing the moment of interaction as the right (13). moment of resampling is similar to in IMM doing the hypothesis merging step in combination with the interaction IV. IMM PARTICLE FILTER step rather than after measurement update [46]. Now we use the characterizations of the four transitions for the development of the IMM particle filter (IMMPF, see Prediction step of IMMPF Table 3). Substituting the resampled version of pɶ θ ( x |θ ) and xt−1| t , Y t − 1 θ pwɶ ( ) =δ ( ww − , j ) into (22) and subsequent evaluation At moment t −1 the IMM particle filter starts with the set wt t θ,jµ θ , j θ ∈ ∈ yields: of weighted particles {xt−1, t − 1 ;M , j {1,.., S } } , thus θ , j pɶ θ (,) xθ=− δ( xaxw() θ ,,′ ) δ ( wwdw −⋅) with a total number of NP = MS particles. This set of xt, t | Y t −1 ∫ ∫ t n n ′ weighted particles spans the empirical density R R S S θ , j θ , j θ,j θ , j ⋅µ δ ()′ − = ɶ θ= µ δ () − ∑ t x x t−1 px−,θ − | Y − ( x , ) t−1 xx t − 1 (24) t1 t 1 t 1 ∑ j=1 j=1 S θ θ , j θ,j θ , j as an approximation of the exact density px,θ | Y ( x , ) . =µ δ − θ = t−1 t − 1 t − 1 ∑ t ()xa() , xt−1 , w t − 1 j=1 Mode Switching step of IMMPF : S =µθ,j δ − θ , j Substituting approximation (24) into (20) and subsequent ∑ t()x x t (27.a) evaluation yields: j=1 with θ,j= θ θ , j θ , j xt a( , xt−1 , w t ) (27.b)

∂g(θ ,x , y )  pθ (|,) yxθ= pgxy ((,,)) θ This forms the prediction step of the IMM particle filter. yttt| x , v t   ∂y 

Correction step of IMMPF The main changes of the IMMPF over the SIR PF are: Substituting eq. (27) into (13) and subsequent evaluation − yields Resample fixed number of particles per mode; − θ θ S Probabilities for {t } instead of particles for {t } ; θ,j θ , j pɶ θ(,) xpyxθ= θ (|,) θµδ () xxc − / = − xYttt, | yx ttt | ∑ t tt Resampling after interaction/mixing rather than after j=1 measurement update. S =µθ,j δ − θ , j ∑ t()x x t/ c t (28.a) j=1 θ,j θ , j θ , j with µ= µpθ (| yx ,)/ θ c (28.b) Table 1. SIR Particle Filter (SIR PF) cycle t tyxt| t , t t t SIR pɶθ→ p ɶ θ xttt−1,| − 1 Y − 1 xY ttt ,| This forms the correction step of the IMM particle filter. j j j n Particles {µ−∈[0,1], θ − ∈M ,x − ∈= R ; j 1,..., N } t1 t 1 t 1 p N p Output step of IMMPF j j j pxɶ θ (,)θ= µχθθ− (,)( − δ xx − − ) Because we have particle values for x only, we next xt−1, t − 1 | Y t − 1 ∑ t1 t 1 t 1 t j=1 determine the output equations of the IMM particle filter. For j=1,..., N : From the law of total probability we have: p • Generate w j and u j i.i.d. from p( w ) and p( u ) t t wt ut pɶθ()θ= p ɶ θ (,) x θ dx (29) tt|Y∫ x ttt , | Y • Evolution: n R θj= θ j j j tc( t−1, x t − 1 , u t ) where pɶθ (θ ) is an approximation of pθ (θ ) . t|Y t t|Y t j= θ jj j Substitution of (28.a) into (29) and subsequent evaluation xt a( t, xwt−1 , t ) yields • Correction: µµj=j ⋅ θ jj θ jjjjT θ tt−1 Nyh mttt{ ;( , xg ),( tttt , xg )( , x )} / c t S θ,j θ , j N pɶθ (θ ) = µ δ () xxdx − = p t|Y t ∫ ∑ t t j = µ = Rn j 1 with ct such that ∑ t 1 = S j 1 = µθ , j • ∑ t (30) Resampling: j=1 µ j = t1/ N p

N p Through dividing the left and right hand terms of (28.a) by jjθ µχθθδ j j− j ()xtt,~∑ t() ,() t x x t pɶθ (θ ) and subsequent evaluation we get for = t|Y t j 1 pɶθ (θ )> 0 : t|Y t

S θ,j θ , j pxɶθ (|)θµδ=() xxp − /() ɶ θ θ (31) xYt| t , t ∑ t tt| Y t j=1 Substitution of (30) into (31) yields: S S θ,j θ , j θ , j pxɶ θ (|)θ= µδ() xx − / µ (32) xYt| t , t −1 ∑ t t ∑ t j=1 j = 1 Equations (30) and (32) characterize the output step of the IMM particle filter.

Remark 4: If y= h(θ, xv , ) has for each θ, x an inverse v= g(θ, xy , ) which is differentiable in y, then (e.g. Kitagawa, 1996):

Table 2. IMM Particle Filter (IMMPF) cycle Table 4. Hybrid Particle Filter (HPF) cycle IMM Particle Filter (IMMPF) cycle Hybrid Particle Filter (HPF) cycle Particles Particles µθ,j∈ θ , j ∈∈=n θ µθ,j∈ θ , j ∈∈=n θ { t−1[0,1],x t − 1 R , M ; jNM 1,...,p / } { t−1[0,1],x t − 1 R , M ; jNM 1,...,p / }

Np / M Np / M θ,j θ , j θ,j θ , j pɶ θ ( x ,θ ) = µ− δ () xx − − pɶ θ ( x ,θ ) = µ− δ () xx − − xt−1, t − 1 | Y t − 1 ∑ t1 t 1 xt−1, t − 1 | Y t − 1 ∑ t1 t 1 j j=1 • Mode switching: • Mode switching: θ Np / M , j η η ut−1 ~ pu ( u ) γθθ()∆pɶ () = Π () x ,j µ , j t tθ| Y − ∑ ∑ ηθ t−1 t − 1 t t 1 θθ,j= θ θ , j θ , j η∈M j = 1 tc( , xt−1 , u t ) • Interaction resampling: • Prediction: µθ , j = γ θ ∈ θ ∈ θ , j t t( )M / N p , j{1,..., Np / M } , M w~ p ( w ) i.i.d., θ ∈M, j ∈ {1,., N / M } t w t p θ,j θ , j θ θθ θ If γ( θ )= 0 then x−= x − , otherwise ,j= θ , j, j , j t t1 t 1 xt a( t, xwt−1 , t ) Np / M θ,j ηηη ,,, jjj • Correction: x~Πηθ ()() xµδ xx − /() γθ t−1∑ ∑ ttt −−− 111 t θ,jθ,j θ , j θ , j η∈ = µ= µ− ⋅ pθ (| yx , θ )/ c M j 1 tt 1 yxt| t , t tttt t

• Prediction: Np / M θ θ , j µ , j = w~ p ( w ) i.i.d. for (θ ,j )∈M × {1,..., N / M } with ct such that ∑ ∑ t 1 t w t p j=1 θ ∈ M θ,j= θ θ , j θ , j xt a( , xt−1 , w t ) • Resampling: N/ M • Correction: p γθ= µχθθθ,j η , j θ,j θ , j θ , j t( ) t() t , µ= µ ⋅ pθ ( yx | ,)/ θ c ∑ ∑ t t yxt| t , t tt t j=1 η ∈ M Np / M µθ , j = γ θ µθ , j = t t( )M / N p with ct such that ∑ ∑ t 1 N/ M j=1 θ ∈ M p θ,jµδθ θθ ,, jj θδ− θ , j γθ • Output: xt~∑ ∑ t() t ,() x x ttt /() t j=1 η ∈ M Np / M θ , j γθ()∆pɶθ () θ = µ θ ∈ × { } γ θ > tt| Y t ∑ t i.i.d. for (,)jM 1,..., Np / M if t ( ) 0 j=1

Np / M θ,j θ , j pxɶ θ (|)θ= µδ() xx − /() γθ if γ( θ )> 0 xYt| t , t ∑ t tt t Table 3 provides an overview of similarities and differences j=1 between SIR PF applied to stochastic hybrid systems and

IMM PF. In addition, table 3 extends this comparison to a

version of the SIR particle filter that has shown to work

well for multi-target tracking [14]. We refer to this scheme

(see Table 4) as Hybrid Particle Filter (HPF).

Table 3. Comparison of particle filter characteristics SIR PF HPF IMMPF

(table 1) (table 4) (table 2) V. MONTE CARLO SIMULATIONS Memorize In this section some Monte Carlo simulation results are θ Yes No No t -values given for the IMM Particle Filter (IMMPF), the standard Mode- Simulation Simulation Analytical Particle Filter (PF) and the IMM algorithm. In addition we switching also give simulation results for a Hybrid Particle Filter -Prediction Simulation Simulation Simulation xt (HPF) which differs from the standard PF by resampling a Correction Standard Standard Standard fixed number of particles per mode. For each of the particle Resampling After Combined with After correction timing correction Interaction filters we used a total of Np=10000 and Np=1000 particles Fixed number Fixed number respectively. The simulations primarily aim at gaining Resampling Equal weights of particles per of particles per type insight in the behavior and performance of the filters in case mode mode of less frequent switching. In the example scenarios there is

an object moving with two possible modes, i.e. M = 2 . TABLE I SCENARIO PARAMETER VALUES One mode is constant velocity and the other mode is σ σ τ τ T Scenario α a m 1 2 S constant acceleration. The object starts with zero velocity [m/s 2] [m] [s] [s] [s] and continues this for 40 scans. After scan 40 the object 1 0.9 50 30 50 5 1 starts to accelerate with at a value equal to the standard 2 0.9 50 30 5000 5 1 σ deviation a of acceleration values. In scenarios 1 and 2 3 0.9 1 30 50 5 1 4 0.9 1 30 5000 500 1 the object continues with constant velocity after scan 60, while in scenarios 3 and 4 the object continues accelerating.

In each simulation, the filters start with perfect estimates TABLE II and run for 100 scans. COMPUTATIONAL LOAD PER SCAN (10 -3 s) The model considered is a Markov jump linear system: IMM PF HPF IMMPF Np =θ + θ 4 xAt() tt x−1 B () tt w 10 4 138 115 96 3 = + σ 10 4 19 13 11 yt[1 0 0] x tmt v

with M = {1,2} and Scenario 1: In this scenario, the target accelerates with 5g 1 2  between 40 s and 60 s. The model used by the particle 1Ts 0  1 Ts T s   2  filters expect accelerations to happen about once per A(1)= 0 1 0 , A(2)= 0 1 T 4   s  minute. With N =10 particles, all three particle filters   p 0 0 0  0 0 α  perform similarly well; they converge to a lower value 0  0  during uniform motion than IMM does. As a side effect, the     peak RMS error at the start of acceleration is for the particle B(1)= σ 0 , B(2)= σ 0 a   a   filters slightly higher than it is for IMM. These results agree 1  2    1−α  well with those in [6] – [7]. Reduction of the number of 3 particles to Np=10 affects PF dramatically, but has − Ts T s  1 τ τ negligible impact on HPF and IMMPF. Π = 1 1  T T  s− s τ1 τ  2 2  Scenario 2: In this scenario, the target accelerates with 5g between 40 s σ where a represents the standard deviation of acceleration and 60 s. The model used by the particle filters expect σ accelerations to happen less than once per hour. noise, m represents the standard deviation of the With N =10 4 particles, IMMPF performs marginally better measurement error, T is the time duration between two p s than IMM does, while PF performs dramatically worse. − τ τ successive observation moments t 1 and t, 1 and 2 are HPF performs significantly worse during the initial the mean durations of modes 1 and 2 respectively, the acceleration period only. Reduction of the number of 3 parameter α ∈ (0,1] allows the acceleration in mode 2 to particles to Np=10 has a negative effect on the convergence vary randomly in time. Table I gives the scenario parameter during constant velocity for all three particle filters. values that are being used for the Monte Carlo simulations. Moreover, during the period of acceleration, PF and HPF For each of the scenarios Monte Carlo simulations worsen dramatically in performance. containing 100 runs have been performed for each of the filters. To make the comparison more meaningful, for all Scenario 3: filters the same random number streams were used. The The target accelerates with 0.1g after 40 s. The model used results of the Monte Carlo simulations of the four scenarios by the particle filters expect accelerations to happen about 4 are shown as follows: once per minute. With Np=10 particles, all three particle − filters perform equally well, and significantly better than The position RMS errors in figures 1 through 4. 3 − The speed RMS errors in figures 5 through 8. IMM does. Reduction of the number of particles to Np=10 has a clear negative effect for the standard PF, but does not − The acceleration RMS errors in figures 9 through 12 affect IMMPF and HPF. − The computational load in Table II.

100 100 IMM IMM 90 90 PF HPF PF HPF HPF 80 80 PF IMM PF IMM PF IMM PF PF 70 70 IMM

60 60

50 PF's 50 IMM 40 40 HPF positionerror RMS positionerror RMS 30 IMM IMM 30 20 20 IMM PF IMM & 10 10 IMM PF PF's PF's Other three 0 20 40 60 80 100 0 20 PF's 40 60 80 100 time (seconds) time (seconds) 4 4 a. Np=10 particles a. Np=10 particles

100 100 IMM IMM HPF 90 PF 90 PF PF HPF HPF 80 80 IMMPF IMM PF IMM PF PF 70 70 PF IMM 60 PF 60 HPF HPF 50 50 IMM PF PF IMM 40 40

positionerror RMS 30 positionerror RMS 30 HPF & HPF IMM IMM PF 20 20 IMM PF 10 10 IMM & IMM Other three PF IMM IMM PF PF's 0 20 40 60 80 100 0 20 40 60 80 100 time (seconds) time (seconds) 3 3 b. Np=10 particles b. Np=10 particles

Figure 1. Scenario 1. The target accelerates with 5g between 40 s Figure 2. Scenario 2. The target accelerates with 5g between 40 s σ = σ = and 60 s. The particle filter parameters are a 5g , and 60 s. The particle filter parameters are a 5g , τ = τ = τ = τ = 1 50s and 2 5s . 1 5000s and 2 5s .

Scenario 4: Summary of Monte Carlo simulation results: 4 The target accelerates with 0.1 g after 40 s. The model used With Np=10 particles, all three particle filters perform by the particle filters expect accelerations to happen less better than IMM for scenarios 1 and 3. For scenarios 2 and 4 than once per hour. With Np=10 particles, all four filters, 4 however, IMM and IMMPF perform similarly well, while except the standard PF, perform similarly well. The the standard PF performs less good on sudden acceleration, standard PF performs dramatically worse during constant and the HPF response is less good for acceleration in 3 3 acceleration. Reduction of the number of particles to 10 scenario 2 only. With Np=10 particles the performance of has a clear negative effect for the standard PF and the HPF, PF degrades for all scenarios, of HPF for scenarios 2 and 4, but not for the IMMPF. and of IMMPF for scenario 2 only.

100 100 IMM IMM 90 PF 90 PF HPF HPF 80 80 IMM PF PF IMM PF 70 70

60 60

50 50 PF's 40 IMM 40

positionerror RMS 30 positionerror RMS 30

20 IMM 20

10 10 IMM & HPF & IMM PF 0 20 40 60 80 100 0 20 40 60 80 100 PF's time (seconds) time (seconds) 4 4 a. Np=10 particles a. Np=10 particles

100 100 IMM IMM 90 90 PF PF HPF 80 80 HPF IMM PF IMM PF PF 70 70

60 60

50 50 HPF & PF's HPF 40 PF IMMPF 40

positionerror RMS 30 positionerror RMS 30

20 IMM 20 IMM 10 10 IMM IMM & PF's IMM PF 0 20 40 60 80 100 0 20 40 60 80 100 time (seconds) time (seconds) 3 3 b. Np=10 particles b. Np=10 particles

Figure 3. Scenario 3. The target accelerates with 0.1g after 40 s. Figure 4. Scenario 4. The target accelerates with 0.1 g after 40 s. σ = τ = σ = τ = The particle filter parameters are a 0.1g , 1 50s The particle filter parameters are a 0.1g , 1 5000s τ = τ = and 2 5s . and 2 500s .

VI. CONCLUDING REMARKS Hence IMMPF is the preferred particle filter for stochastic In this paper we considered filtering of stochastic hybrid hybrid systems. systems that go beyond the well known Markov jump system. We derived the exact Bayesian filter and used this Next we compare IMMPF with IMM. For the regular for the development of a novel particle filter for discrete switching scenarios 1 and 3, the IMMPF has some time stochastic hybrid systems. Because of its similarity performance advantage over IMM, also when the number of 3 with the interaction step of IMM, this novel particle filter is particles is down to 10 . The computational load of IMMPF referred to as IMM particle filter (IMMPF). Through MC is then three times higher than the load of IMM. However, simulations for four scenarios, IMMPF has been tested and for the scenarios with infrequent mode switching rate compared with standard PF and IMM. With 10 4 particles, (scenarios 2 and 4), the IMM performs similarly well as the the IMMPF performs well for all four scenarios. The IMMPF. In this case the main advantage over IMM is that computational load is 25 times the load of IMM. The IMMPF incorporates various kinds of deviations from the computational load of the standard PF is even higher. As Markov jump linear IMM (i.e. non-linear and non-Markov expected, the IMMPF works well for all four scenarios mode switching). including ones where standard PF or IMM has problems. Because IMMPF uses particles for the conditional densities

given the mode, it is likely that several of the SIR [20] D.B. Rubin, “Using the SIR algorithm to simulate posterior improvement methods referred to in the introduction of this distributions,” Eds. J.M. Bernardo et al., Bayesian Statistics 3, Oxford Univ. Press, 1988, pp. 395-402. paper, combine quite well with IMMPF, and are expected [21] N.J. Gordon, D. J. Salmond, A. F. M. Smith, “Novel approach to to allow a significant reduction of the number of particles. nonlinear/non-Gaussian Bayesian state estimation,” IEE This is topic of research in a follow-up study. Proceedings-F, Vol. 140, pp. 107-113, 1993. [22] P. Del Moral, Nonlinear filtering: interacting particle solution, Markov Processes and Related Fields, Vol. 2 (1996), pp. 555-580. ACKNOWLEDGEMENT [23] G. Kitagawa, Monte Carlo filter and smoother for non-Gaussian nonlinear state space models, J. Computational and Graphical The authors would like to thank anonymous reviewers for Statistics, Vol. 5 (1996), pp. 1-25. their valuable suggestions in improving the paper. [24] A. Doucet, “On sequential simulation-based methods for Bayesian filtering,” Technical report CUED / F-INFENG / TR-310, Univ. of Cambridge, UK, 1998. REFERENCES [25] P. Del Moral, A. Guionnet, Large deviation for interacting particle [1] Y. Bar-Shalom, S. Challa, H.A.P. Blom, IMM estimator versus systems - Application to nonlinear filtering problems, Stochastic optimal estimator for hybrid systems, IEEE Tr. Aerospace and Processes and their Applications, Vol. 78 (1998), pp. 69-95. Electronic Systems, Vol. 41 (2005), pp.986-991. [26] D. Crisan, A. Doucet, A survey of convergence results on particle [2] V.J. Mathews, J.K. Tugnait, Detection and estimation with fixed lag filtering methods for practitioners, IEEE Tr. Signal Processinbg, for abruptly changing systems, IEEE Tr. Aerospace and Electronic Vol. 50 (2002), pp. 736-746. Systems, Vol. 19 (1983), pp. 730-739. [27] F.E. Daum, J. Huang, The curse of dimensionality for particle filters, [3] L. Campo, P. Mookerjee, Y. Bar-Shalom, State Estimation for Proc. IEEE Conf. on Aerospace, Big Sky, MT, March 2003. Systems with a Sojourn-Time-Dependent Markov Switching Model, [28] M. Isard, A. Blake, Condensation – Conditional density propagation IEEE Tr. Automatic Control, Vol. 36 (1991), pp. 238-243. for visual tracking, J. of Computer Vision, Vol. 29 (1998), pp. 5-28. [4] H.A.P. Blom, Hybrid state estimation for systems with semi-Markov [29] Y. Boers, J.N. Driessen, F. Verschure, W.P.M.H. Heemels, A. switching coefficients, Proc. 1st European Control Conf., Grenoble, Juloski, A multi target track before detect application, Proc.IEEE 1991, pp. 1132-1137. CVPR Conf, Workshop on multi object tracking, Madison, WI, June [5] X. Koutsoukis, J. Kurien, F. Zhao, “Monitoring and Diagnosis of 2003. hybrid systems using particle filtering methods,” Proc. Mathematical [30] J.S. Liu, R. Chen, Sequential Monte Carlo methods for dynamical Theory of Networks and Systems (MTNS), 2002. systems, J. American Statistical Association, Vol. 93 (1998), pp. [6] S. McGinnity, G. W. Irwin, “Multiple Model Bootstrap Filter for 1032-1044. Maneuvering Target Tracking,” IEEE Tr. on Aerospace and [31] W.R. Gilks, C. Berzuini, Following a moving target – Monte Carlo Electronic System s, Vol. 36, 2000, pp. 1006-1012. inference for dynamic Bayesian models, J. Royal Statistical Society [7] S. McGinnity, G. W. Irwin, “Maneuvering Target Tracking using a B, Vol. 63 (2001), pp. 127-146. Multiple-Model Bootstrap Filter,” Eds. A. Doucet, N. de Freitas and [32] A. Doucet, S. Godsill, C. Andrieu, On sequential Monte Carlo N. Gordon, Sequential Monte Carlo Methods in Practic e, Springer sampling methods for Bayesian filtering, Statistics and Computing, 2001, pp. 479-497. Vol. 10 (2000), pp. 197-208. [8] D.S. Angelova, T.A. Semerdjiev, V.P. Jilkov, E.A. Semerdjiev, [33] M.K. Pitt, N. Shephard, Filtering via simulation: auxiliary particle Application of Monte Carlo method for tracking maneuvering target filters, J. of American Statistical Association, Vol. 94 (1999), pp. in clutter, Mathematics and Computers in Simulation, Vol. 1851 590-599. (2000), pp. 1-9. [34] M. Hürzeler, H.R. Künsch, Monte Carlo approximations for general [9] R. Chen, J.S. Liu, Mixture Kalman filters, J. Royal Statist. Soc. B, state-space models, J. Computational and Graphical Statistics, Vol. Vol. 62, Part 3 (2000), pp. 493-508. 7 (1998), pp. 175-193. [10] A. Doucet, A. Logothetis, V. Krishnamurthy, Stochastic sampling [35] M.S. Arulampam, S. Maskell, N. Gordon, T. Clapp, A tutorial on algorithms for state estimation of jump Markov linear systems, IEEE particle filters for online nonlinear/non-Gaussian Bayesian tracking, Tr. Automatic Control, Vol. 45 (2000), pp. 188-202. IEEE Tr. Sgnal Processing, Vol. 50 (2002), pp. 174-188. [11] A. Doucet, N. J. Gordon, V. Krishnamurthy, “Particle Filters for [36] Y. Rui, Y. Chen, Better proposal distributions: object tracking using State Estimation of Jump Markov Linear Systems,” IEEE Tr. on unscented particle filter, Proc. IEEE 2001, pp. II-786-793. Signal Processin g, Vol. 49, 2001, pp. 613-624. [37] G. Casella, C. Robert, Rao-Blackwellisation of sampling schemes, [12] C. Musso, N. Oudjane, F. Le Gland, “Improving Regularised Particle Biometrika, Vol. 83 (1996), pp. 81-94. Filters,” Eds. A. Doucet, N. de Freitas and N. Gordon, Sequential [38] F. Gustafsson, F. Gunnarsson, N. Bergman, U. Forssell, J. Jansson, Monte Carlo Methods in Practic e, Springer 2001, pp. 247-271. R. Karlsson, P.-J. Nordlund, Particle filters for positioning, [13] C. Andrieu, M. Davy, A. Doucet, Efficient particle filtering for jump navigation and tracking, IEEE Tr. SP, Vol. 50 (2002), pp. 425-437. Markov systems. Application to time-varying autoregressions, IEEE [39] S. Maskell, M. Rollason, N. Gordon, D. Salmond, Efficient particle Tr. Signal Processing, Vol. 51 (2003), pp. 1762-1770. filtering for multiple target tracking with application to tracking in [14] H.A.P. Blom, E.A. Bloem, “Joint IMMPDA Particle filter,” Proc. structured images, Proc. SPIE, Signal and Data Processing of Small 6th Int. Conf. on Information Fusion, 2003, pp. 785-792. Targets, Vol. 4728, April 2002. [15] Y. Boers, H. Driessen, Hybrid state estimation: a target tracking [40] M.R. Morelande, S. Challa, An algorithm for tracking group targets, application, Automatica, Vol. 38 (2002), pp. 2153-2158. Proc. Workshop on Multiple Hypothesis Tracking: A tribute to S. [16] Y. Boers, J.N. Driessen, Interacting multiple model particle filter, Blackman, San Diego, May 2003. IEE Proc. Radar Sonar Navig., Vol. 150 (2003), pp. 344-349. [41] H.A.P. Blom, E.A. Bloem, “Particle filtering for stochastic hybrid [17] H. Driessen, Y. Boers, An Efficient Particle Filter for Jump Markov systems,” Proc. IEEE CDC04, Bahamas, December 2004. Nonlinear Systems, Proc. IEE Colloquium on Target Tracking, [42] H.A.P. Blom, “An efficient filter for abruptly changing systems,” March 2004. Proc. of the 23rd IEEE CDC , 1984, pp.656-658. [18] B. Ristic, S. Arulampalam, N. Gordon, Beyond the Kalman filter - [43] J.M. Bernardo, A.F.M. Smith, Bayesian theory, Wiley, 1994. Particle filters for tracking applications, Artech House, 2004. [44] J.K. Tugnait, A.H. Haddad, “A detection-estimation scheme for state [19] M. Morelande, S. Challa, Maneuvering target tracking in clutter estimation in switching environments,” Automatica, Vol. 15 (1979), using particle filters, IEEE Tr. AES, Vol. 41 (2005), pp. 252-270. pp. 477-481.

[45] R.J. Elliott, F. Dufour, D.D. Sworder, Exact hybrid filters in discrete time, IEEE Tr. AC, Vol. 41 (1996), pp. 1807-1810. [46] H.A.P. Blom, Y. Bar-Shalom, “The Interacting Multiple Model algorithm for systems with Markovian switching coefficients ,” IEEE Tr. on Automatic Control , Vol. 33 (1988), pp. 780-783. [47] Y. Bar-Shalom, X. R. Li, Estimation and tracking: principles, techniques and software, Artech House, 1993. [48] Y. Bar-Shalom, X.R. Li, T. Kirubarajan, Estimation, tracking and navigation, Wiley-Interscience, New York, 2001.

8 10 PF IMM IMM IMM IMM 9 7 PF PF's PF HPF 8 HPF 6 PF's IMM PF IMM PF 7

5 6 PF

4 5 HPF PF 3 4 speed RMSerror speed RMSerror speed 3 IMM 2 IMM PF 2 IMM IMM IMM 1 1 IMM HPF & IMM PF 0 0 0 20 40 60 80 100 0 20 40 60 80 100 PF's time (seconds) PF's PF's time (seconds) a. 10 4 particles a. 10 4 particles

8 10 HPF IMM IMM IMM IMM 9 7 PF PF PF PF's HPF IMM PF HPF 8 PF 6 IMM PF IMM PF 7 5 PF HPF 6 HPF

4 5

3 4 speed RMSerror speed IMM PF RMSerror speed 3 2 HPF IMM IMM PF HPF PF 2 IMM HPF & IMM PF IMM IMM PF 1 1

0 IMM PF 0 0 20 40 60 80 100 0 20 40 60 80 100 PF's IMM IMM & PF & time (seconds) PF time (seconds) IMM b. 10 3 particles b. 103 particles

Figure 5. Scenario 1. The target accelerates with 5g between 40 s Figure 6. Scenario 2. The target accelerates with 5g between 40 s σ = σ = and 60 s. The particle filter parameters are a 5g , and 60 s. The particle filter parameters are a 5g , τ = τ = τ = τ = 1 50s and 2 5s . 1 5000s and 2 5s .

0.9 1.4 IMM IMM IMM PF's 0.8 PF's PF PF PF's 1.2 HPF HPF 0.7 PF IMM PF IMM PF 1 0.6 IMM 0.5 0.8

0.4 0.6 IMM

speed RMSerror speed 0.3 RMSerror speed 0.4 0.2 IMM 0.2 0.1 IMM & HPF & IMM PF 0 0 0 20 40 60 80 100 0 20 40 60 80 100 PF's time (seconds) time (seconds) a. 10 4 particles a. 10 4 particles

0.9 3 IMM IMM IMM PF's 0.8 PF PF's HPF & PF 2.5 HPF IMM PF HPF 0.7 IMM PF IMM PF PF 0.6 2

0.5 IMM 1.5 0.4 IMM PF

speed RMSerror speed 0.3 RMSerror speed 1 HPF 0.2 IMM 0.5 0.1

0 0 IMM & 0 20 40 60 80 100 0 20 40 60 IMM PF 80 100 PF's time (seconds) time (seconds) b. 103 particles b. 10 3 particles

Figure7. Scenario 3. The target accelerates with 0.1 g after 40 s. Figure 8. Scenario 4. The target accelerates with 0.1 g after 40 s. σ = τ = σ = The particle filter parameters are a 0.1g , 1 50s The particle filter parameters are a 0.1 g , τ = τ = τ = and 2 5s . 1 5000s and 2 500s .

0.5 0.7 IMM IMM 0.45 PF 0.6 PF PF 0.4 HPF HPF IMM PF IMM PF 0.35 0.5 HPF PF's 0.3 0.4 0.25 0.3 0.2

0.15 0.2 accelerationRMSerror IMM accelerationRMSerror PF IMM PF 0.1 HPF & PF's PF's 0.1 IMM IMM PF 0.05 PF's

0 0 0 20 40 60 80 100 0 20 40 60 80 100 IMM time (seconds) IMM IMM time (seconds) IMM a. 10 4 particles a. 10 4 particles

0.5 0.7 IMM IMM 0.45 PF 0.6 PF 0.4 HPF HPF IMM PF IMM PF PF 0.35 0.5 PF 0.3 0.4 HPF 0.25 0.3 0.2 HPF IMM PF 0.15 0.2 accelerationRMSerror accelerationRMSerror IMM HPF 0.1 IMM PF HPF & PF's IMM PF 0.1 PF's IMM PF 0.05 PF IMM 0 0 0 20 40 60 80 100 0 20 40 60 80 100 PF IMM time (seconds) IMM IMM time (seconds) IMM b. 10 3 particles b. 10 3 particles

Figure 9. Scenario 1. The target accelerates with 5g between 40 s Figure 10. Scenario 2. The target accelerates with 5g between 40 s σ = σ = and 60 s. The particle filter parameters are a 5g , and 60 s. The particle filter parameters are a 5g , τ = τ = τ = τ = 1 50s and 2 5s . 1 5000s and 2 5s .

0.01 0.012 IMM IMM 0.009 PF PF 0.01 0.008 HPF HPF PF IMM PF IMM PF 0.007 PF's PF 0.008 IMM 0.006

0.005 0.006

0.004 HPF & 0.004 IMM & 0.003 IMM PF

accelerationRMSerror accelerationRMSerror IMM HPF & 0.002 IMM PF 0.002 0.001 PF's PF's

0 0 0 20 40 60 80 100 0 20 40 60 80 100 IMM time (seconds) IMM time (seconds) a. 10 4 particles a. 10 4 particles

0.012 0.02 HPF & IMM IMM HPF & IMM PF 0.018 PF IMM PF HPF PF 0.01 HPF HPF 0.016 PF IMM PF IMM PF 0.014 0.008 PF PF PF 0.012 IMM & IMM & 0.006 IMM IMM PF PF 0.01 HPF 0.008 0.004 0.006 accelerationRMSerror accelerationRMSerror HPF & 0.004 IMM & 0.002 IMM PF IMM IMM PF PF's 0.002 PF's

0 0 0 20 40 60 80 100 0 20 40 60 80 100 IMM time (seconds) IMM time (seconds) b. 10 3 particles b. 10 3 particles

Figure 11. Scenario 3. The target accelerates with 0.1 g after 40 s. Figure 12. Scenario 4. The target accelerates with 0.1 g after 40 s. σ = τ = σ = τ = The particle filter parameters are a 0.1g , 1 50s The particle filter parameters are a 0.1g , 1 5000s τ = τ = and 2 5s . and 2 500s .