A Poisson Process Whose Rate Is a Hidden Markov Process Author(s): D. S. Freed and L. A. Shepp Source: Advances in Applied Probability, Vol. 14, No. 1 (Mar., 1982), pp. 21-36 Published by: Applied Probability Trust Stable URL: http://www.jstor.org/stable/1426731 Accessed: 19/11/2010 16:51
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in the JSTOR archive only for your personal, non-commercial use.
Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at http://www.jstor.org/action/showPublisher?publisherCode=apt.
Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected].
Applied Probability Trust is collaborating with JSTOR to digitize, preserve and extend access to Advances in Applied Probability.
http://www.jstor.org Adv. Appl. Prob. 14, 21-36 (1982) Printed in N. Ireland 0001-8678/82/010021-17$01.95 @ Applied Probability Trust 1982 A POISSON PROCESS WHOSE RATE IS A HIDDEN MARKOV PROCESS
D. S. FREED,* Harvard University L. A. SHEPP,** Bell Laboratories
Abstract
Let a Poisson process be observed whose output rate is one of two levels given by the state of an unseen Markov process. If one of the levels is 0, a simple formula is given for the best guess of the state at any instant based on the stream of past Poisson events. In other cases bounds are given for the likelihood ratio of the state probabilities given the event stream.
POISSON; COX; DOUBLY STOCHASTIC; HIDDEN RATE; LIKELIHOOD RATIO
1. Introduction
Let S,, 0, be a Markov process with two states 1, 2, and let 3i be the rate t- at which S leaves state i. While S, = i a source generates events with exponen- tial delay at rate ai. Given only the stream YSof generated events, we want to guess the state So of the Markov process S at time t =0. Variants of the problem have been widely studied and in general the likelihood ratio L = P(S = 1 I f)/P(So = 21 1), needed to compute the best guess of So, involves complicated infinite products of non-commuting matrices. However, when the low rate a2 is 0 (this means no events are generated while S, = 2), we give the simple formula L = (A + B-cl)/(1 - e-,1) where xl is the time of the first event and A, B, c are constants involving the parameters of S and a,. We also: and lower bounds on L which show that if (a) give upper a2<
Received 3 December 1980; revision received 23 April 1981. * Present address: 970 Evans Hall, Mathematics Department, University of California, Berk- eley, CA 94720, U.S.A. Research carried out while this author was a summer employee at Bell Laboratories. ** Postal address: Bell Laboratories, 600 Mountain Ave, Murray Hill, NJ 07924, U.S.A. 21 22 D. S. FREED AND L. A. SHEPP relativeto 01 and t2, and a1 and a2 are nearlyequal, that very late event times can still significantlyinfluence L. Variable-ratePoisson processes where the rate itself is a random process have been called 'doubly stochastic Poisson processes' by Cox [2]. Many references are given in [8], pp. 403-5, (see also [1], [4]-[7] to various approachesto the problem of inferringthe underlyingrandom rate, different approachesbeing based on slightly different models. Each model refers to a real-worldsituation where some sort of steady signal process (say a beam of X-rays or light) causes a counter (of photons or electrons) to register a sequence of events. The rate of the events at any time is a measure of the strengthof the steady signal at that same time. Errorsin the determinationof the actual hidden state are due to confusing the fluctuationsof the signal process with the fluctuationsof the counting process. We study the simplest non-trivialcase: the hiddensignal process takes only two values and is Markov. Thus let t be a Markov on states i = 2 with S,, _ 0, process 1, Ti= P(So = i),
S=lim -P(S+h, # i IS,= i), (1.1) a = lim -P(event in (t, t + h) I S,= i), h so that y, is the prior or unconditioneddistribution of So,,Pi is the rate that S leaves state i, and a, is the rate of the generatingevents while St = i. Note that for some real-world applications it is more natural to have t 0, or -oo
Then the best of x1, xK is 1 or 2 according as LK> 1 or guess So given , LK <1. From Bayes's rule,
. . 1) P(So = 1) ' P(xXl, , xK So= (1.4) LK = 2) , xK 2) -P(So So= P(x1, We shall obtain an explicit formula for the second ratio on the right of (1.4), which of course may be thought of as a Radon-Nikodym derivative,
P(xl, , XK ISo= 1)= Y2 (1.5) RK = = . ' LK RK(Xl,..., XK) P(xl, , xK So= 2) Yi as a function of a product of K matrices. A general martingale theorem ([3], p. 348) then gives the likelihood ratio for the full stream as
= (1.6) L = L = P(So x1, * * = lim LK. P(SO= 2 x1, x2,x29 ) K-*oo )* When a2 = 0 (but only for this case) the explicit formula to be derived for L becomes very simple, namely
1 - 2)e-(84+-8-)x(- (1.7) LK (1.7) (022 + K 1 Y2 2 1 -( where
01+ 2)? + 92)2 44192 (1.8) 6S = (a•1+ "V(a+12 -1--
Thus for a2 =0, L involves only the first event time, x1. This is intuitive because there is no information left in the process after the first event occurs; we then know for sure that Sx,= 1 (cf. [5]). When 0< a2< LK (1.9) aLK i < K, I8axi axi and that LK decreases in each xi; for a2 (1.10) L O<, i?K. axi Each of (1.9) and (1.10) holds for L replacing LK by taking K -+ oo. 24 D. S. FREED AND L. A. SHEPP 2. Explicit formulasand inequalitiesfor LK At first we allow S to have n states. Thus let a,, Pi, be as in (1.1) where - - - y• now i=1, , n. Let Aii be the probability of transition from state i to state j # i given that a transition takes place, and define A,,= -1. Note that the rows of A = sum to 0 and that for n =2, A (Ai) =(-1 1 -1/ ) automatically.----~---' Let Pij(t) be defined by (2.1) Pj(t) = P(S, = ij So = i), so that, as is easily derived, Pi(t) = + s) ds, (2.2) e-",'8ii ~ Ae-"13 AkPkJ(t- and P(t)=(Pij(t)) is given by 01 0 (2.3) P(t)= exp (BAt), B= 2 0 O, Let Fii(t) be the probability density of the first event occurring at t while S, = j given that So = i, i.e., (2.4) Fij(t) dt = P(X E dt, S, = j So = i). Then, as is easy to see, = + A - ds. (2.5) F1j(t) e-"aisae-3i XkFki(t s) e-,'aie-a"'sti As with (2.2), differentiating (2.5) we find that F(t)=(F1j(t)) is a 0 (2.6) F(t) = exp (Ct)A, A = 2 C= BA-A. O an, It is clear from the definition of matrix multiplication that when n = 2, which we assume from now on, P(xl, , xK I So= 1)= (1, -- . F(xK)(1, 1), (2.7) O)F(xl) P(xl, , I = 2)= (0, F(xK)(1, 1), xK So 1)F(xl) ? ? ? where (1, 1)= (1, 1)T. Letting 9?{(vl, v2)} denote the ratio of the components of A Poissonprocess whose rate is a hiddenMarkov process 25 a column 2-vector, (v, v2) = V21), (2.8) 1 R{(V1,, )2= V2 we have that RK in (1.5) is given by (2.9) RK F(xK)(1, =R{F(x') ' 1)}. Although this gives an explicit formula for RK and hence for LK, it is unfortunate that the matrix product in (2.9) is awkward, even for n = 2, since the matrices F(xi) are non-commuting. If a, = a2, then of course RK 1 since the events have no information about So. Indeed, in this case (1, 1) is a right eigenvector for C, hence for exp (Cx), and A is a multiple of the identity so ... that the components of F(xl) F(xK)(1, 1) remain equal. Thus we assume without loss of generality that a, > a2 0. We think of 1 as the high state and 2 as the low state. We next derive simple upper and lower bounds on L. Let the joint density of ... x1, , xK when STK= j be denoted (2.10) P,j(x, , xK)dx .. dxK= P(X, dxl,I = 1, ..., K, STK= jISo = i), so that (2.11) (Pi(xl, - , xK))= F(xl) ... F(xK). def Clearly for K > M, if x (x,... ,), * Pi(x1, *, xM) (x1) + P12(()2P2K(x M+1) (2.12)K P11(x)Pl P2i (1 XM) 21(xK)PliiX •+1)+ 22(\ )P2i(x"+1) Setting * * , xK) Ok= max Pi(xl, i=1,2 1 9XK) (2.13) P2i(Xl, . . XK) = P1i(X1.. qK i=1,2min P2i(X,'...XK)1 it followseasily from (2.13) that for K < M, (2.14) QM qM - QK, qK. Since (2.15) RK(Xl,"", XK)= 11 12 P21(xK -P22(xK' 26 D. S. FREED AND L. A. SHEPP we find immediately that for K M, (2.16) qK- RM• OQK Since qK and QOKare easy to compute for small K, (2.15) gives simple upper and lower bounds on R (let M-+ oo) and hence on L = Ryl/y2 from (1.5). We next show that a more careful analysis using the explicit form of F in (2.6) will allow us to improve the bounds (2.16) by showing that RK is monotonic in each x,. Since C in (2.6) is diagonalizable, we find after a calculation that (2.17) Fi(t) = lie- + mie- where = + a2 (2.18) 8, [(a1 + 1+ 2) J 1--12- 2)2 +44 2] and the matrices I and m are given by (2.19)a l(a21 a (a + 2 -b) a21 5+-S-( a192 a2(a1+1P,5-) - m1 (l(a + a2- 2) -a21 ) 5 \ -a902 2( + -a- 91) +--- Define aK, bK,CK, dK, functions of x1, ... , xK, by aK (2.20) bK) = F(x) cK dK ...F(xK). Note the simplification that occurs when a2 = 0; the second column of F(t) is then 0, so in this case 1 (2.21) (aK bK) (F1(X1)F11(X2) 0O' cK dK ?F21(X1)F11(X2)... F1(xk)F11(Xk) 0 Thus from (2.9), (2.22) RK Fll(xj) F21(X1)'F21(x1) and now (1.7) follows from (2.17)-(2.19) and (1.5). As a preliminarystep in provingthat RK is monotonicdecreasing in each x, we note that the function ae"' +be"' (2.23) tP(t)= ce** + de" ' A Poisson processwhose rate is a hiddenMarkov process 27 is monotonic since its derivative (ad - bc)(jL- v)e(+V)' (2.24) f'(t)= (ce*' + de"')2 is of one sign. Since F,(t) is given by (2.17) we see that R1(t) is of the form (2.23) and so is monotonic. Now R1(O)= (111+ m1)/(121 + m21) a /a2 from (2.19) and R(oo)= 111/121 since 8+>8_ by (2.18) when 13I32#O0(which we assume). An easy calculation shows that a2 _ if a2 a1,. Hence a2 2 (2.25) =1 - = R(0) Rl() 121 02 a2 so that R1(t) must be monotonically decreasing. Now from (2.9) and (2.20) we see that for K -5M, aKr+bK (2.26) R(xl, , x) = Kr + bK cKr+ dK where aK, bK, CK, dK are given by (2.20) and (2.27) r = RI{FxK+) F(xm)(l, 1)}. The ratio on the right of (2.26), + bK (2.28)(2.28)(r)aKr 4K(r) cKr + dK is increasing in r since by (2.20), aKdK- bKCK (cKr+ dK)2 det (F(xl) ... F(xK)) (2.29) (cKr+dK)2 det F(xl) ... det F(xK) (cKr+ dK)2 The last inequality follows from (2.6) since (2.30) det F(x)= etr(c)x det A > 0. To prove that RM is decreasing in xi, we take K = i -1 in (2.26)-(2.29) and reduce the problem to the case i= 1. Now an induction on M completes the proof of (1.10) and also shows that ? (2.31) q* R (Q K M, _ _*, 28 D. S. FREEDAND L. A. SHEPP where Q*= Mlim o lim RM(xl, , XK+1,'",XM1 0O " " xM), = sup ) (2.32) R(x,' XK+1.... q* = lim lim Rm(xl, , xm). M t o0 XK+ Ti ,"',XJM " = inf XK +1.'' R(x1,'). The upper bound is the same as in (2.16); the lower bound is strictly sharper than in (2.16), since Rn bK KMlim OO {(aic dK \ M-K Q*= )(c-M•K aK (2.33) CK = max (aK bK CK K• \ = QK, since bKIdK:a-K/cIK for a, a2, whereas R q*= lim lim aK bKF(xK+) .F(x)1 Mto CK dK XK+1TO• aKR (oo)+ bK + cKR1(00) dK bK (2.34) dKb = minminaK bK)K\ = qK* Explicit formulas for QO and q* are given later in (3.5). We remark that the improved bounds (2.31) are actually the best possible - (i.e. are attained for xK+1, XK+2,"" 0, o). Furthermore, they are as easy to compute as the bounds (2.16). Also, in the limit a2 t 0, Q 1 (while */qfC 1) which justifies our earlier statement that for small a2 and fixed QI/ql -• a1l, /1, /2, L depends essentially only on We will use the bounds (2.31) for xl. A Poisson process whose rate is a hidden Markov process 29 the simulation in Section 3. It seems likely that whenever + x2 +.... =0 (which happens with probability 1) that Q*/q-- 1, but we have not proved this. Finally, we prove (1.9), or equivalently (see (1.5)) that IRK 3RK (2.35) aK> >- I ax, I= aOxi i (2.37) RK *-Kf4( Hence K K2 (2.38) aRK ax a a', c(4,K)2 We compare the expression on the right of (2.35) for a = i, i+ 1; clearly it suffices to consider just the numerator. Now 4xot F(x) [y C exp i+1 F(xj)I), K 1i=1 O 1 If=a+1(Cx,)A O( Kc+l _ I F(xl ) (2.39) =[HlIF(x)]exp (Cx)CA[ from (2.6), (2.9), (2.37), and - = C (2.40) dt (exp (Ct)) exp (Ct)= exp (Ct)C. It follows that the expression aax = UOV(1, 1) U= [HF(xj) exp (Cxi), (2.41) K j=i+l =CA, a= i, = C, a = i+l, O•tA 30 D. S. FREED AND L. A. SHEPP is valid for a = i, i + 1. Hence (2.42) K)9 UQV(1, 1). - = (,- 1 Define (2.43) so that (2.44) p, r, s>0, q 5 0. Some calculation now shows that the statement K K (2.45) x, -aX - =-- <-- +i 3x01K 1 is equivalent to (2.46) qr - ps, which is trivial in view of (2.44). Finally, (2.45), (2.38), and the fact that RK/aox, <0 establish (2.35). We remark that in the general n-state case the function corresponding to (2.23), t a,aie (2.47) q(t) = b e 1 " i Cit is not monotonicfor generala's and b's since q'(t) can in generalhave both positiveand negative values. It seemslikely that for the specialcase where A is given by 1 (2.48) = , i#j; =-1 n, n-l1 Ai and so states can be lumped, that functions corresponding to R and L are monotonic in each x1, but this is probably not so for the general case. 3. Simulations Figures 1-6 display data from simulations of the point process generated as follows. We choose yi = y2 = so that the initial state is determined by a 0"5 So fair coin toss. Next, Poisson processes of rate j3i,i = 1, 2, are generated using A Poisson process whose rate is a hidden Markov process 31 1.0 1...... , ,il ", , ,, , .. , . , to .. -0.11. I'l a ,aP2,/,/ 2= 1,.7,.1.1 S0 =2 X1= 1.8374 0.8 0.8 a 1 I)/fSo=, ,,a2A,,/, SO=Xj=.5667711,.7,.01,D0 0.6 0.6 0Q4 04 02 02 0 20 40 60 80 0O 200 0 8 100 K K Figure 1 Figure 2 1.0 g aoaa, So=1 1 1 0.8 X1=17729 ,a1 ,p 5,.01,.01 a8• So= 1 1j=o117,.01,01 X, .42447 0.6 ,b 04 004 0.2 02- 0 20 40 60 80 100 S 20 40 80 100 K K Figure 3 Figure 4 0.1 1 1.0I1. ISo-, aa213 A?2-1,.5,_1, 01 f 1 "' - 0.8- V X .85926 08- Xj-2-.18445I lag?,agfSoo1 ,.999,01,01 Q6 0.6- 04 04- 02 02 0 20 40 80 20 40 6 80 100 O0 K 100 6O K Figure 5 Figure 6 32 D. S. FREED AND L. A. SHEPP the usual construction of an exponential random variable as the logarithm of a uniform random variable. These events determine the switch points of the S process and are illustrated schematically by black circles at the top of each figure. The tick marks shown directly above the circles are the events Ti, T2,... of a Poisson process whose rate ai depends on the state S,; namely, the rate at time t is In all of our simulations 013= Hence the values of = as,. 32. y• 0.5 chosen give the stationary mode for the S process. The horizontal axis plots the number of points observed and the vertical axis plots the probability that So = 1. (Notice that the scale unit for the plot of the circles and tick marks at the top is 'time', the scale for each figure being determined by the value of x, shown, whereas the scale unit in the horizontal axis at the bottom is 'number of points', and the scale is uniform over all figures.) Denoting this probability by x and letting y be the ratio P(So = 1)IP(So = 2), we have the simple relation (3.1) x = (y)= l+y The curves C1-C4 on each figure are: C21: C2: (3.2) xi C3: 0 RK)=0(LK), C4:C4 t1*0(YQ2)K= T 1Ti Y22QK). and satisfy C1 - -C3-: C4 in each figure. Thus C1 gives the weak lower C2- bound (2.16) of O(L) given xl, X,xK2 gives the strong lower bound (2.31), "", the value = 1 and the bound C3 gives P(So 1 xK), C4 gives upper (2.31). We emphasize that the boundsxl,' C2', and C4 (as well as Cl) hold uniformlyin * * * future X's, i.e. for any values of xK+1, K+2, . The parameters ai and Pi are indicated on each plot as is the value of x, (which determines the scale at the top of each figure). The actual value of So is also displayed in each figure. The figures bear out our conjecture in Section 2 that the upper and lower bounds converge to the same value (i.e. thatQI/q*i--* 1). We will give some qualitative observations about this convergence. We note first that the common value to which the curves converge (i.e. will deviate significantly from O(L)) 0.5 (assuming yi = Y2= 05) only if the following two conditions are met. (A) a1 is not very close to a2. (B) The 3's are small relative to the a's. A Poisson processwhose rate is a hiddenMarkov process 33 Note that a limiting value of 0-5 (or, for general yi, of 0(^y1/y^2)=-^1/ Y2))+ indicates that the data from the point process had no effect on the final guess of So since 0(Lo) = This occurs when (A) fails since in that case the high state 0.5. = and the low state are essentially indistinguishable (see Figure 6 where a1 1, a2= When (B) fails the S process is switching states too often to 0.999). allow enough observations of the point process to determine So accurately (see Figure 1 where •1 = t2 = 0-1, a1 = 1, a2 = 0-7). In both of these cases the point process carries little information. Notice that in Figures 2 and 4, where (A) and (B) hold. 0(L) is not close to 0-5. (However, see Figures 3 and 5 which show that O(L) is not necessarily extreme, i.e. close to 0 or 1, with these values of the parameters. Indeed, 0(L) is still possible, depending on the x's.) 0.5 Closely related is the rate of convergence of the curves. We noted earlier than when a2 is close to 0, 0(L) is essentially determined by the first event. (This is confirmed by simulations not shown here.) We now observe that convergence is also fast if (B) is not satisfied. The intuition is as before; for O's which are close to the a's the S process interferes with the counting process to such a degree that more than a few observations carry no information. (Compare Figure 1 with Figures 2 and 3.) By the same token if the O's are very small relative to the a's, the law of large numbers applies to show that late observations still carry information. In fact, in the limit 31= 32=0 the S process is constant, say at So = io, and so 1 (3.3) = lim Xl-+"'-+"XK 10ioK--* K since the expected value of an exponential random variable Xi with parameter a is 1/a. Setting B =0 in (2.6), we find easily from (2.9) and (3.1) that, taking io= 1 for definiteness, P(So= 1 x1, , x 1 K)= ^Y2 RK) (3.4) 71a~: exp -a, xi 1i=1 K K (-a, K K Y1al exp (-aI + y2aK exp -a2 i=1 i=1 x which, using (3.3) and assuming E'= xx, =oo, approaches 1 as K - oo. In this : case, then, So is completely determined if a1 a2. This fact, in addition to providing more support for our claim that L is more likely to converge to extreme values if the 3's are small, shows that convergence will be relatively slow since K must be large for the ratio in (3.3) to stabilize. These remarks are illustrated in Figures 4 and 5. There were no switches of 34 D. S. FREED AND L. A. SHEPP the S process in Figure 4, at least over the time period shown, and so there was enough information present to determine So with probability But in Figure 0.9. 5 a switch occurs early, and the value of So cannot be determined with confidence. Indeed, the final guess is actually wrong. Notice that the 20th-25th observed points are most significant here; this could not be true if the 3's were not so small. In both of these plots it is interesting to observe the correlation between the oscillations of C3 and the position of the corresponding observed point. As we have already seen, there is little information in the point process if Condition (A) fails. So by the same reasoning as above, we expect quick convergence of the bounds in this case. However, Figure 6 shows clearly that our expectation is not fulfilled. (Note here the improvement of qK over qK; this is important in what follows.) To study this behavior we give the explicit formulas - (a2 - a2 + 32 - &S)e8-_ + (8+ -2)e-8+x1 + (_P2)e-8+x1 (3.5) (P32)e-8x1 a2021 [•1(a2 +0&2-F_)2+ ]e-a-8x +[a (8+ -2 32)(a2+( (2- a2931 e-8+Xx -)- + 2 - 2 -)-1 Ie-8_x [a0(a02 - + ( 1+301 81 )(a2+2 - ) + 2( - -l31)]1 e-+x1 which are derived from (2.17)-(2.19), (2.20), (2.25), (2.33), and (2.34). Now if a2 = a1 then RK 1 so that QO and q*, by definition the best bounds on RK, are also identically equal to 1. Fix a, and let a2 T a. Then we observe that q* is continuous whereas Of is discontinuous. (If a2 ? a the reverse occurs.) This -- is most easily seen by observing first that in (2.25), R1(o) 1 as a2 1a1, and that (3.6) q1 = S1l(1),(R )) a2a2< = a1,Ol, where 41 is as in (2.28), while (3.7) =f1(00), 2 I* <•1. (Note that the old lower bound, q1, is also discontinuous. Also, the results of this paragraph are valid for Q• and q .) A Poisson processwhose rate is a hiddenMarkov process 35 The discontinuity of QI* has the following ramifications for Figure 6. If we observe the first 20 points, say, then we guess that So = 1 with probability 0.5. No matter what we subsequently observe, this probability will not decrease substantially; C2 and C3 are essentially equal. But if we do observe many close points after the first 20 (i.e. if x21, x22, - are all close to 0), then this probability can increase to a value greater than '. The asymmetry of the situation is puzzling and we do not understand it completely. We do feel, however, that C3 will almost certainly stay constant at 0-5 and only with very small probability will it increase. To conclude this section we formulate a general (but somewhat vague) conjecture based on viewing a lot of simulations. Note that in Figures 2, 3, 4, 6 one of Q*(C4) and q*(C2) 'stabilizes' first, i.e. as K increases the bound becomes horizontal. This does not happen in Figure 5. However when it does happen, as for example in Figure 2 where C4 stabilizes at K 5 with Q*- we conjecture that with high probability the limiting value L of LK is close0.97, to the value of the bound at which stabilization occurred first. Thus in Figure 2, Q*T 097 and Lo10 Figure 5 is a counterexample to show that 0"95. the limit value does not always agree with the first stabilized value but such counterexamples seem rare. Further, in almost all cases observed, C3 meets the first stabilized bound before it meets the other bound. Thus, for example, in Figure 6 (and also Figure 3) we should believe that with high probability - - L - L1oo. Of course for some xo10,x102, , LK will come up to C4 in Figure 6, but this will have low probability. It would be nice to have some sort of precise mathematical confirmation of this empirical observation. 4. Open questions We state briefly some questions which merit further study, some of which were mentioned earlier. 1. What is the qualitative effect of (1 (32? 2. Is 1 if -> oo (See discussion after (2.34).) limK-woo QI/q= K- 3. What can be said about the n-state case (i.e. S takes n values instead of two)? What if A is given by (2.48)? 4. Explain the discontinuity of Q* at a, = a2. (See (3.7).) Acknowledgement We thank Dr Jan Grandell for his very careful reading of the paper and for supplying references [2] and [4]-[7] and several comments, in particular on the experiment in Figure 2 which points out an interesting aspect of our algorithm. Indeed, the prediction that So = 1 with very large probability is at first (>0.95) 36 D. S. FREED AND L. A. SHEPP surprisingin view of the fact that most events occur while S = 2. But the algorithm for computing L depends only on the actual stream of observed events from the point process, not on the hidden state S. In this experiment, the first several interarrival times x, are extremely small, even after the switch to state 2. Note The plot makes clear that the next few x' are even x1 0"57. smaller. But in state 1 the expected value of x, is 1, in state 2 it is ? 1/0.7 1.4. So that the fact that the interarrival times were much less than the expected value led our algorithm to predict the high state So= 1, even though the information on which this estimate is based came mostly from the low state S = 2. (Comparethis to Figure 5 where informationafter the transitionof the hidden state leads to an incorrectestimate of So.) References [1] BOEL, R. ANDBENES, V. E. (1981) Recursivenonlinear estimation of a diffusionacting as the rate of an observedPoisson process. IEEE Trans.Information Theory. [2] Cox, D. R. (1955) Some statisticalmethods related with series of events. J. R. Statist.Soc. B 17, 129-164. [3] DooB, J. L. (1953) StochasticProcesses. Wiley, New York. [4] GRANDELL,J. (1976) Doubly StochasticPoisson Processes. Lecture Notes in Mathematics 529, Springer-Verlag,Berlin. [5] KINGMAN,J. F. C. (1964) On doubly stochasticPoisson processes. Proc. Camb. Phil. Soc. 60, 923-930. [6] LIPSTER,R. SH. ANDSHIRYAYEV, A. N. (1978) Statistics of Random Processes II. Springer- Verlag, Berlin. [7] RUDEMO,M. E. (1975) Predictionand smoothingfor partiallyobserved Markov chains. J. Math. Anal. Appl. 49, 1-23. [8] SNYDER,D. L. (1975) RandomPoint Processes.Wiley, New York.