
A Poisson Process Whose Rate Is a Hidden Markov Process Author(s): D. S. Freed and L. A. Shepp Source: Advances in Applied Probability, Vol. 14, No. 1 (Mar., 1982), pp. 21-36 Published by: Applied Probability Trust Stable URL: http://www.jstor.org/stable/1426731 Accessed: 19/11/2010 16:51 Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in the JSTOR archive only for your personal, non-commercial use. Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at http://www.jstor.org/action/showPublisher?publisherCode=apt. Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission. JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected]. Applied Probability Trust is collaborating with JSTOR to digitize, preserve and extend access to Advances in Applied Probability. http://www.jstor.org Adv. Appl. Prob. 14, 21-36 (1982) Printed in N. Ireland 0001-8678/82/010021-17$01.95 @ Applied Probability Trust 1982 A POISSON PROCESS WHOSE RATE IS A HIDDEN MARKOV PROCESS D. S. FREED,* Harvard University L. A. SHEPP,** Bell Laboratories Abstract Let a Poisson process be observed whose output rate is one of two levels given by the state of an unseen Markov process. If one of the levels is 0, a simple formula is given for the best guess of the state at any instant based on the stream of past Poisson events. In other cases bounds are given for the likelihood ratio of the state probabilities given the event stream. POISSON; COX; DOUBLY STOCHASTIC; HIDDEN RATE; LIKELIHOOD RATIO 1. Introduction Let S,, 0, be a Markov process with two states 1, 2, and let 3i be the rate t- at which S leaves state i. While S, = i a source generates events with exponen- tial delay at rate ai. Given only the stream YSof generated events, we want to guess the state So of the Markov process S at time t =0. Variants of the problem have been widely studied and in general the likelihood ratio L = P(S = 1 I f)/P(So = 21 1), needed to compute the best guess of So, involves complicated infinite products of non-commuting matrices. However, when the low rate a2 is 0 (this means no events are generated while S, = 2), we give the simple formula L = (A + B-cl)/(1 - e-,1) where xl is the time of the first event and A, B, c are constants involving the parameters of S and a,. We also: and lower bounds on L which show that if (a) give upper a2<<a1, 1,32,, then L depends essentially only on x1; (b) prove (the intuitive conjecture) that earlier event times have more ... influence on L, i.e., JaL/8x1ll jL/ax21I where x, are the interevent times; (c) prove(the intuitiveconjecture) that if a, > a2, then L decreasesas each interevent time increases, i.e. 8L/8x,?0; (d) give a simulationwhich shows clearly that when ao anda2 are not small Received 3 December 1980; revision received 23 April 1981. * Present address: 970 Evans Hall, Mathematics Department, University of California, Berk- eley, CA 94720, U.S.A. Research carried out while this author was a summer employee at Bell Laboratories. ** Postal address: Bell Laboratories, 600 Mountain Ave, Murray Hill, NJ 07924, U.S.A. 21 22 D. S. FREED AND L. A. SHEPP relativeto 01 and t2, and a1 and a2 are nearlyequal, that very late event times can still significantlyinfluence L. Variable-ratePoisson processes where the rate itself is a random process have been called 'doubly stochastic Poisson processes' by Cox [2]. Many references are given in [8], pp. 403-5, (see also [1], [4]-[7] to various approachesto the problem of inferringthe underlyingrandom rate, different approachesbeing based on slightly different models. Each model refers to a real-worldsituation where some sort of steady signal process (say a beam of X-rays or light) causes a counter (of photons or electrons) to register a sequence of events. The rate of the events at any time is a measure of the strengthof the steady signal at that same time. Errorsin the determinationof the actual hidden state are due to confusing the fluctuationsof the signal process with the fluctuationsof the counting process. We study the simplest non-trivialcase: the hiddensignal process takes only two values and is Markov. Thus let t be a Markov on states i = 2 with S,, _ 0, process 1, Ti= P(So = i), S=lim -P(S+h, # i IS,= i), (1.1) a = lim -P(event in (t, t + h) I S,= i), h so that y, is the prior or unconditioneddistribution of So,,Pi is the rate that S leaves state i, and a, is the rate of the generatingevents while St = i. Note that for some real-world applications it is more natural to have t 0, or -oo<t < 0, but these modificationsare easily made since S_,, t > 0 is also a Markovchain, and t > 0, and t > 0 are independentgiven So. Thus the use of Bayes's St, S-,, rule in (1.4) below allows one to treat the streamsY5,, 9_ for t > 0 and t <0 independently, P(So= 11S+, 9) 1P(+ ISo= 1) P(_I So= 1) P(So = 21f+, _) y2 P(+ So = 2) P(_ I So= 2)" Let the actual stream of events observed be at times iT, and let Xi be the interarrivaltimes, i = 1, 2, --, so that 0 < < 72 < ... (1.2) (1.2) T1 - Xi = 7i 7Ti-1, i1,2,-- where To=0, but there is no event observed at time 0. Define the likelihood ratio LK, K = 0, 1, 2, , by ? ? P(So = 1 x, , xK) (1.3) L xx, P(So=s 2 , xK) Lo = 1. A Poisson processwhose rate is a hidden Markovprocess 23 Then the best of x1, xK is 1 or 2 according as LK> 1 or guess So given , LK <1. From Bayes's rule, . 1) P(So = 1) ' P(xXl, , xK So= (1.4) LK = 2) , xK 2) -P(So So= P(x1, We shall obtain an explicit formula for the second ratio on the right of (1.4), which of course may be thought of as a Radon-Nikodym derivative, P(xl, , XK ISo= 1)= Y2 (1.5) RK = = . ' LK RK(Xl,..., XK) P(xl, , xK So= 2) Yi as a function of a product of K matrices. A general martingale theorem ([3], p. 348) then gives the likelihood ratio for the full stream as = (1.6) L = L = P(So x1, * * = lim LK. P(SO= 2 x1, x2,x29 ) K-*oo )* When a2 = 0 (but only for this case) the explicit formula to be derived for L becomes very simple, namely 1 - 2)e-(84+-8-)x(- (1.7) LK (1.7) (022 + K 1 Y2 2 1 -( where 01+ 2)? + 92)2 44192 (1.8) 6S = (a•1+ "V(a+12 -1-- Thus for a2 =0, L involves only the first event time, x1. This is intuitive because there is no information left in the process after the first event occurs; we then know for sure that Sx,= 1 (cf. [5]). When 0< a2<<a1, the expression obtained in Section 2 for L is complicated, but we shall obtain bounds for L which show that L is essentially still determined by xl, provided 13 and 02 are in Section 3 show that L does not too small. When a2 -1 simulations really depend on all of the x's in general. Nevertheless, we show in Section 2 that earlier events influence L more than later ones in the sense that LK (1.9) aLK i < K, I8axi axi and that LK decreases in each xi; for a2 <a,1 (1.10) L O<, i?K. axi Each of (1.9) and (1.10) holds for L replacing LK by taking K -+ oo. 24 D. S. FREED AND L. A. SHEPP 2. Explicit formulasand inequalitiesfor LK At first we allow S to have n states. Thus let a,, Pi, be as in (1.1) where - - - y• now i=1, , n. Let Aii be the probability of transition from state i to state j # i given that a transition takes place, and define A,,= -1. Note that the rows of A = sum to 0 and that for n =2, A (Ai) =(-1 1 -1/ ) automatically.----~---' Let Pij(t) be defined by (2.1) Pj(t) = P(S, = ij So = i), so that, as is easily derived, Pi(t) = + s) ds, (2.2) e-",'8ii ~ Ae-"13 AkPkJ(t- and P(t)=(Pij(t)) is given by 01 0 (2.3) P(t)= exp (BAt), B= 2 0 O, Let Fii(t) be the probability density of the first event occurring at t while S, = j given that So = i, i.e., (2.4) Fij(t) dt = P(X E dt, S, = j So = i).
Details
-
File Typepdf
-
Upload Time-
-
Content LanguagesEnglish
-
Upload UserAnonymous/Not logged-in
-
File Pages17 Page
-
File Size-