DOCSLIB.ORG

A Poisson Process Whose Rate Is a Hidden Markov Process Author(S): D

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
A Poisson Process Whose Rate Is a Hidden Markov Process Author(S): D A Poisson Process Whose Rate Is a Hidden Markov Process Author(s): D. S. Freed and L. A. Shepp Source: Advances in Applied Probability, Vol. 14, No. 1 (Mar., 1982), pp. 21-36 Published by: Applied Probability Trust Stable URL: http://www.jstor.org/stable/1426731 Accessed: 19/11/2010 16:51 Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in the JSTOR archive only for your personal, non-commercial use. Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at http://www.jstor.org/action/showPublisher?publisherCode=apt. Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission. JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected]. Applied Probability Trust is collaborating with JSTOR to digitize, preserve and extend access to Advances in Applied Probability. http://www.jstor.org Adv. Appl. Prob. 14, 21-36 (1982) Printed in N. Ireland 0001-8678/82/010021-17$01.95 @ Applied Probability Trust 1982 A POISSON PROCESS WHOSE RATE IS A HIDDEN MARKOV PROCESS D. S. FREED,* Harvard University L. A. SHEPP,** Bell Laboratories Abstract Let a Poisson process be observed whose output rate is one of two levels given by the state of an unseen Markov process. If one of the levels is 0, a simple formula is given for the best guess of the state at any instant based on the stream of past Poisson events. In other cases bounds are given for the likelihood ratio of the state probabilities given the event stream. POISSON; COX; DOUBLY STOCHASTIC; HIDDEN RATE; LIKELIHOOD RATIO 1. Introduction Let S,, 0, be a Markov process with two states 1, 2, and let 3i be the rate t- at which S leaves state i. While S, = i a source generates events with exponen- tial delay at rate ai. Given only the stream YSof generated events, we want to guess the state So of the Markov process S at time t =0. Variants of the problem have been widely studied and in general the likelihood ratio L = P(S = 1 I f)/P(So = 21 1), needed to compute the best guess of So, involves complicated infinite products of non-commuting matrices. However, when the low rate a2 is 0 (this means no events are generated while S, = 2), we give the simple formula L = (A + B-cl)/(1 - e-,1) where xl is the time of the first event and A, B, c are constants involving the parameters of S and a,. We also: and lower bounds on L which show that if (a) give upper a2<<a1, 1,32,, then L depends essentially only on x1; (b) prove (the intuitive conjecture) that earlier event times have more ... influence on L, i.e., JaL/8x1ll jL/ax21I where x, are the interevent times; (c) prove(the intuitiveconjecture) that if a, > a2, then L decreasesas each interevent time increases, i.e. 8L/8x,?0; (d) give a simulationwhich shows clearly that when ao anda2 are not small Received 3 December 1980; revision received 23 April 1981. * Present address: 970 Evans Hall, Mathematics Department, University of California, Berk- eley, CA 94720, U.S.A. Research carried out while this author was a summer employee at Bell Laboratories. ** Postal address: Bell Laboratories, 600 Mountain Ave, Murray Hill, NJ 07924, U.S.A. 21 22 D. S. FREED AND L. A. SHEPP relativeto 01 and t2, and a1 and a2 are nearlyequal, that very late event times can still significantlyinfluence L. Variable-ratePoisson processes where the rate itself is a random process have been called 'doubly stochastic Poisson processes' by Cox [2]. Many references are given in [8], pp. 403-5, (see also [1], [4]-[7] to various approachesto the problem of inferringthe underlyingrandom rate, different approachesbeing based on slightly different models. Each model refers to a real-worldsituation where some sort of steady signal process (say a beam of X-rays or light) causes a counter (of photons or electrons) to register a sequence of events. The rate of the events at any time is a measure of the strengthof the steady signal at that same time. Errorsin the determinationof the actual hidden state are due to confusing the fluctuationsof the signal process with the fluctuationsof the counting process. We study the simplest non-trivialcase: the hiddensignal process takes only two values and is Markov. Thus let t be a Markov on states i = 2 with S,, _ 0, process 1, Ti= P(So = i), S=lim -P(S+h, # i IS,= i), (1.1) a = lim -P(event in (t, t + h) I S,= i), h so that y, is the prior or unconditioneddistribution of So,,Pi is the rate that S leaves state i, and a, is the rate of the generatingevents while St = i. Note that for some real-world applications it is more natural to have t 0, or -oo<t < 0, but these modificationsare easily made since S_,, t > 0 is also a Markovchain, and t > 0, and t > 0 are independentgiven So. Thus the use of Bayes's St, S-,, rule in (1.4) below allows one to treat the streamsY5,, 9_ for t > 0 and t <0 independently, P(So= 11S+, 9) 1P(+ ISo= 1) P(_I So= 1) P(So = 21f+, _) y2 P(+ So = 2) P(_ I So= 2)" Let the actual stream of events observed be at times iT, and let Xi be the interarrivaltimes, i = 1, 2, --, so that 0 < < 72 < ... (1.2) (1.2) T1 - Xi = 7i 7Ti-1, i1,2,-- where To=0, but there is no event observed at time 0. Define the likelihood ratio LK, K = 0, 1, 2, , by ? ? P(So = 1 x, , xK) (1.3) L xx, P(So=s 2 , xK) Lo = 1. A Poisson processwhose rate is a hidden Markovprocess 23 Then the best of x1, xK is 1 or 2 according as LK> 1 or guess So given , LK <1. From Bayes's rule, . 1) P(So = 1) ' P(xXl, , xK So= (1.4) LK = 2) , xK 2) -P(So So= P(x1, We shall obtain an explicit formula for the second ratio on the right of (1.4), which of course may be thought of as a Radon-Nikodym derivative, P(xl, , XK ISo= 1)= Y2 (1.5) RK = = . ' LK RK(Xl,..., XK) P(xl, , xK So= 2) Yi as a function of a product of K matrices. A general martingale theorem ([3], p. 348) then gives the likelihood ratio for the full stream as = (1.6) L = L = P(So x1, * * = lim LK. P(SO= 2 x1, x2,x29 ) K-*oo )* When a2 = 0 (but only for this case) the explicit formula to be derived for L becomes very simple, namely 1 - 2)e-(84+-8-)x(- (1.7) LK (1.7) (022 + K 1 Y2 2 1 -( where 01+ 2)? + 92)2 44192 (1.8) 6S = (a•1+ "V(a+12 -1-- Thus for a2 =0, L involves only the first event time, x1. This is intuitive because there is no information left in the process after the first event occurs; we then know for sure that Sx,= 1 (cf. [5]). When 0< a2<<a1, the expression obtained in Section 2 for L is complicated, but we shall obtain bounds for L which show that L is essentially still determined by xl, provided 13 and 02 are in Section 3 show that L does not too small. When a2 -1 simulations really depend on all of the x's in general. Nevertheless, we show in Section 2 that earlier events influence L more than later ones in the sense that LK (1.9) aLK i < K, I8axi axi and that LK decreases in each xi; for a2 <a,1 (1.10) L O<, i?K. axi Each of (1.9) and (1.10) holds for L replacing LK by taking K -+ oo. 24 D. S. FREED AND L. A. SHEPP 2. Explicit formulasand inequalitiesfor LK At first we allow S to have n states. Thus let a,, Pi, be as in (1.1) where - - - y• now i=1, , n. Let Aii be the probability of transition from state i to state j # i given that a transition takes place, and define A,,= -1. Note that the rows of A = sum to 0 and that for n =2, A (Ai) =(-1 1 -1/ ) automatically.----~---' Let Pij(t) be defined by (2.1) Pj(t) = P(S, = ij So = i), so that, as is easily derived, Pi(t) = + s) ds, (2.2) e-",'8ii ~ Ae-"13 AkPkJ(t- and P(t)=(Pij(t)) is given by 01 0 (2.3) P(t)= exp (BAt), B= 2 0 O, Let Fii(t) be the probability density of the first event occurring at t while S, = j given that So = i, i.e., (2.4) Fij(t) dt = P(X E dt, S, = j So = i).
Load more

Recommended publications
  • A Class of Measure-Valued Markov Chains and Bayesian Nonparametrics
    Bernoulli 18(3), 2012, 1002–1030 DOI: 10.3150/11-BEJ356 A class of measure-valued Markov chains and Bayesian nonparametrics STEFANO FAVARO1, ALESSANDRA GUGLIELMI2 and STEPHEN G. WALKER3 1Universit`adi Torino and Collegio Carlo Alberto, Dipartimento di Statistica e Matematica Ap- plicata, Corso Unione Sovietica 218/bis, 10134 Torino, Italy. E-mail: [email protected] 2Politecnico di Milano, Dipartimento di Matematica, P.zza Leonardo da Vinci 32, 20133 Milano, Italy. E-mail: [email protected] 3University of Kent, Institute of Mathematics, Statistics and Actuarial Science, Canterbury CT27NZ, UK. E-mail: [email protected] Measure-valued Markov chains have raised interest in Bayesian nonparametrics since the seminal paper by (Math. Proc. Cambridge Philos. Soc. 105 (1989) 579–585) where a Markov chain having the law of the Dirichlet process as unique invariant measure has been introduced. In the present paper, we propose and investigate a new class of measure-valued Markov chains defined via exchangeable sequences of random variables. Asymptotic properties for this new class are derived and applications related to Bayesian nonparametric mixture modeling, and to a generalization of the Markov chain proposed by (Math. Proc. Cambridge Philos. Soc. 105 (1989) 579–585), are discussed. These results and their applications highlight once again the interplay between Bayesian nonparametrics and the theory of measure-valued Markov chains. Keywords: Bayesian nonparametrics; Dirichlet process; exchangeable sequences; linear functionals
    [Show full text]
  • Markov Process Duality
    Markov Process Duality Jan M. Swart Luminy, October 23 and 24, 2013 Jan M. Swart Markov Process Duality Markov Chains S finite set. RS space of functions f : S R. ! S For probability kernel P = (P(x; y))x;y2S and f R define left and right multiplication as 2 X X Pf (x) := P(x; y)f (y) and fP(x) := f (y)P(y; x): y y (I do not distinguish row and column vectors.) Def Chain X = (Xk )k≥0 of S-valued r.v.'s is Markov chain with transition kernel P and state space S if S E f (Xk+1) (X0;:::; Xk ) = Pf (Xk ) a:s: (f R ) 2 P (X0;:::; Xk ) = (x0;:::; xk ) , = P[X0 = x0]P(x0; x1) P(xk−1; xk ): ··· µ µ µ Write P ; E for process with initial law µ = P [X0 ]. x δx x 2 · P := P with δx (y) := 1fx=yg. E similar. Jan M. Swart Markov Process Duality Markov Chains Set k µ k x µk := µP (x) = P [Xk = x] and fk := P f (x) = E [f (Xk )]: Then the forward and backward equations read µk+1 = µk P and fk+1 = Pfk : In particular π invariant law iff πP = π. Function h harmonic iff Ph = h h(Xk ) martingale. , Jan M. Swart Markov Process Duality Random mapping representation (Zk )k≥1 i.i.d. with common law ν, take values in (E; ). φ : S E S measurable E × ! P(x; y) = P[φ(x; Z1) = y]: Random mapping representation (E; ; ν; φ) always exists, highly non-unique.
    [Show full text]
  • 1 Introduction Branching Mechanism in a Superprocess from a Branching
    数理解析研究所講究録 1157 巻 2000 年 1-16 1 An example of random snakes by Le Gall and its applications 渡辺信三 Shinzo Watanabe, Kyoto University 1 Introduction The notion of random snakes has been introduced by Le Gall ([Le 1], [Le 2]) to construct a class of measure-valued branching processes, called superprocesses or continuous state branching processes ([Da], [Dy]). A main idea is to produce the branching mechanism in a superprocess from a branching tree embedded in excur- sions at each different level of a Brownian sample path. There is no clear notion of particles in a superprocess; it is something like a cloud or mist. Nevertheless, a random snake could provide us with a clear picture of historical or genealogical developments of”particles” in a superprocess. ” : In this note, we give a sample pathwise construction of a random snake in the case when the underlying Markov process is a Markov chain on a tree. A simplest case has been discussed in [War 1] and [Wat 2]. The construction can be reduced to this case locally and we need to consider a recurrence family of stochastic differential equations for reflecting Brownian motions with sticky boundaries. A special case has been already discussed by J. Warren [War 2] with an application to a coalescing stochastic flow of piece-wise linear transformations in connection with a non-white or non-Gaussian predictable noise in the sense of B. Tsirelson. 2 Brownian snakes Throughout this section, let $\xi=\{\xi(t), P_{x}\}$ be a Hunt Markov process on a locally compact separable metric space $S$ endowed with a metric $d_{S}(\cdot, *)$ .
    [Show full text]
  • 1 Markov Chain Notation for a Continuous State Space
    The Metropolis-Hastings Algorithm 1 Markov Chain Notation for a Continuous State Space A sequence of random variables X0;X1;X2;:::, is a Markov chain on a continuous state space if... ... where it goes depends on where is is but not where is was. I'd really just prefer that you have the “flavor" here. The previous Markov property equation that we had P (Xn+1 = jjXn = i; Xn−1 = in−1;:::;X0 = i0) = P (Xn+1 = jjXn = i) is uninteresting now since both of those probabilities are zero when the state space is con- tinuous. It would be better to say that, for any A ⊆ S, where S is the state space, we have P (Xn+1 2 AjXn = i; Xn−1 = in−1;:::;X0 = i0) = P (Xn+1 2 AjXn = i): Note that it is okay to have equalities on the right side of the conditional line. Once these continuous random variables have been observed, they are fixed and nailed down to discrete values. 1.1 Transition Densities The continuous state analog of the one-step transition probability pij is the one-step tran- sition density. We will denote this as p(x; y): This is not the probability that the chain makes a move from state x to state y. Instead, it is a probability density function in y which describes a curve under which area represents probability. x can be thought of as a parameter of this density. For example, given a Markov chain is currently in state x, the next value y might be drawn from a normal distribution centered at x.
    [Show full text]
  • Simulation of Markov Chains
    Copyright c 2007 by Karl Sigman 1 Simulating Markov chains Many stochastic processes used for the modeling of financial assets and other systems in engi- neering are Markovian, and this makes it relatively easy to simulate from them. Here we present a brief introduction to the simulation of Markov chains. Our emphasis is on discrete-state chains both in discrete and continuous time, but some examples with a general state space will be discussed too. 1.1 Definition of a Markov chain We shall assume that the state space S of our Markov chain is S = ZZ= f:::; −2; −1; 0; 1; 2;:::g, the integers, or a proper subset of the integers. Typical examples are S = IN = f0; 1; 2 :::g, the non-negative integers, or S = f0; 1; 2 : : : ; ag, or S = {−b; : : : ; 0; 1; 2 : : : ; ag for some integers a; b > 0, in which case the state space is finite. Definition 1.1 A stochastic process fXn : n ≥ 0g is called a Markov chain if for all times n ≥ 0 and all states i0; : : : ; i; j 2 S, P (Xn+1 = jjXn = i; Xn−1 = in−1;:::;X0 = i0) = P (Xn+1 = jjXn = i) (1) = Pij: Pij denotes the probability that the chain, whenever in state i, moves next (one unit of time later) into state j, and is referred to as a one-step transition probability. The square matrix P = (Pij); i; j 2 S; is called the one-step transition matrix, and since when leaving state i the chain must move to one of the states j 2 S, each row sums to one (e.g., forms a probability distribution): For each i X Pij = 1: j2S We are assuming that the transition probabilities do not depend on the time n, and so, in particular, using n = 0 in (1) yields Pij = P (X1 = jjX0 = i): (Formally we are considering only time homogenous MC's meaning that their transition prob- abilities are time-homogenous (time stationary).) The defining property (1) can be described in words as the future is independent of the past given the present state.
    [Show full text]
  • Chapter 8: Markov Chains
    149 Chapter 8: Markov Chains 8.1 Introduction So far, we have examined several stochastic processes using transition diagrams and First-Step Analysis. The processes can be written as {X0, X1, X2,...}, where Xt is the state at time t. A.A.Markov On the transition diagram, Xt corresponds to 1856-1922 which box we are in at step t. In the Gambler’s Ruin (Section 2.7), Xt is the amount of money the gambler possesses after toss t. In the model for gene spread (Section 3.7), Xt is the number of animals possessing the harmful allele A in generation t. The processes that we have looked at via the transition diagram have a crucial property in common: Xt+1 depends only on Xt. It does not depend upon X0, X1,...,Xt−1. Processes like this are called Markov Chains. Example: Random Walk (see Chapter 4) none of these steps matter for time t+1 ? time t+1 time t ? In a Markov chain, the future depends only upon the present: NOT upon the past. 150 .. ...... .. .. ...... .. ....... ....... ....... ....... ... .... .... .... .. .. .. .. ... ... ... ... ....... ..... ..... ....... ........4................. ......7.................. The text-book image ...... ... ... ...... .... ... ... .... ... ... ... ... ... ... ... ... ... ... ... .... ... ... ... ... of a Markov chain has ... ... ... ... 1 ... .... ... ... ... ... ... ... ... .1.. 1... 1.. 3... ... ... ... ... ... ... ... ... 2 ... ... ... a flea hopping about at ... ..... ..... ... ...... ... ....... ...... ...... ....... ... ...... ......... ......... .. ........... ........ ......... ........... ........
    [Show full text]
  • Markov Random Fields and Stochastic Image Models
    Markov Random Fields and Stochastic Image Models Charles A. Bouman School of Electrical and Computer Engineering Purdue University Phone: (317) 494-0340 Fax: (317) 494-3358 email [email protected] Available from: http://dynamo.ecn.purdue.edu/»bouman/ Tutorial Presented at: 1995 IEEE International Conference on Image Processing 23-26 October 1995 Washington, D.C. Special thanks to: Ken Sauer Suhail Saquib Department of Electrical School of Electrical and Computer Engineering Engineering University of Notre Dame Purdue University 1 Overview of Topics 1. Introduction (b) Non-Gaussian MRF's 2. The Bayesian Approach i. Quadratic functions ii. Non-Convex functions 3. Discrete Models iii. Continuous MAP estimation (a) Markov Chains iv. Convex functions (b) Markov Random Fields (MRF) (c) Parameter Estimation (c) Simulation i. Estimation of σ (d) Parameter estimation ii. Estimation of T and p parameters 4. Application of MRF's to Segmentation 6. Application to Tomography (a) The Model (a) Tomographic system and data models (b) Bayesian Estimation (b) MAP Optimization (c) MAP Optimization (c) Parameter estimation (d) Parameter Estimation 7. Multiscale Stochastic Models (e) Other Approaches (a) Continuous models 5. Continuous Models (b) Discrete models (a) Gaussian Random Process Models 8. High Level Image Models i. Autoregressive (AR) models ii. Simultaneous AR (SAR) models iii. Gaussian MRF's iv. Generalization to 2-D 2 References in Statistical Image Modeling 1. Overview references [100, 89, 50, 54, 162, 4, 44] 4. Simulation and Stochastic Optimization Methods [118, 80, 129, 100, 68, 141, 61, 76, 62, 63] 2. Type of Random Field Model 5. Computational Methods used with MRF Models (a) Discrete Models i.
    [Show full text]
  • Temporally-Reweighted Chinese Restaurant Process Mixtures for Clustering, Imputing, and Forecasting Multivariate Time Series
    Temporally-Reweighted Chinese Restaurant Process Mixtures for Clustering, Imputing, and Forecasting Multivariate Time Series Feras A. Saad Vikash K. Mansinghka Probabilistic Computing Project Probabilistic Computing Project Massachusetts Institute of Technology Massachusetts Institute of Technology Abstract there are tens or hundreds of time series. One chal- lenge in these settings is that the data may reflect un- derlying processes with widely varying, non-stationary This article proposes a Bayesian nonparamet- dynamics [13]. Another challenge is that standard ric method for forecasting, imputation, and parametric approaches such as state-space models and clustering in sparsely observed, multivariate vector autoregression often become statistically and time series data. The method is appropri- numerically unstable in high dimensions [20]. Models ate for jointly modeling hundreds of time se- from these families further require users to perform ries with widely varying, non-stationary dy- significant custom modeling on a per-dataset basis, or namics. Given a collection of N time se- to search over a large set of possible parameter set- ries, the Bayesian model first partitions them tings and model configurations. In econometrics and into independent clusters using a Chinese finance, there is an increasing need for multivariate restaurant process prior. Within a clus- methods that exploit sparsity, are computationally ef- ter, all time series are modeled jointly us- ficient, and can accurately model hundreds of time se- ing a novel \temporally-reweighted" exten- ries (see introduction of [15], and references therein). sion of the Chinese restaurant process mix- ture. Markov chain Monte Carlo techniques This paper presents a nonparametric Bayesian method are used to obtain samples from the poste- for multivariate time series that aims to address some rior distribution, which are then used to form of the above challenges.
    [Show full text]
  • A Review on Statistical Inference Methods for Discrete Markov Random Fields Julien Stoehr, Richard Everitt, Matthew T
    A review on statistical inference methods for discrete Markov random fields Julien Stoehr, Richard Everitt, Matthew T. Moores To cite this version: Julien Stoehr, Richard Everitt, Matthew T. Moores. A review on statistical inference methods for discrete Markov random fields. 2017. hal-01462078v2 HAL Id: hal-01462078 https://hal.archives-ouvertes.fr/hal-01462078v2 Preprint submitted on 11 Apr 2017 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. A review on statistical inference methods for discrete Markov random fields Julien Stoehr1 1School of Mathematical Sciences & Insight Centre for Data Analytics, University College Dublin, Ireland Abstract Developing satisfactory methodology for the analysis of Markov random field is a very challenging task. Indeed, due to the Markovian dependence structure, the normalizing constant of the fields cannot be computed using standard analytical or numerical methods. This forms a central issue for any statistical approach as the likelihood is an integral part of the procedure. Furthermore, such unobserved fields cannot be integrated out and the likelihood evaluation becomes a doubly intractable problem. This report gives an overview of some of the methods used in the literature to analyse such observed or unobserved random fields.
    [Show full text]
  • Monte Carlo Based Distance Dependent Chinese Restaurant Process for Segmentation of 3D LIDAR Data Using Motion and Spatial Features
    18th International Conference on Information Fusion Washington, DC - July 6-9, 2015 Monte Carlo Based Distance Dependent Chinese Restaurant Process for Segmentation of 3D LIDAR Data Using Motion and Spatial Features Mehmet Ali Cagri Tuncer Dirk Schulz Cognitive Mobile Systems Cognitive Mobile Systems Fraunhofer FKIE Fraunhofer FKIE Wachtberg, Germany 53343 Wachtberg, Germany 53343 Email: [email protected] Email: [email protected] Abstract—This paper proposes a novel method to obtain robust tends to unavoidable segmentation errors. These errors in turn and accurate object segmentations from 3D Light Detection cause erroneous track estimates. So improving the segmen- and Ranging (LIDAR) data points. The method exploits motion tation process is important to achieve progress on the whole information simultaneously estimated by a tracking algorithm in order to resolve ambiguities in complex dynamic scenes. recognition and tracking process. Typical approaches for tracking multiple objects in LIDAR data In urban scenarios, traffic participants are assumed well follow three steps; point cloud segmentation, object tracking, and segmented from each other. Therefore, current point cloud track classification. A large number of errors is due to failures segmentation approaches rely on geometrical features only. in the segmentation component, mainly because segmentation However the assumption may not hold, especially for bicyclists and tracking are performed consecutively and the segmentation step solely relies on geometrical features. This article presents and pedestrians. They often get close to other objects such as a 3D LIDAR based object segmentation method that exploits parking cars. Under these circumstances, segmentation gets the motion information provided by a tracking algorithm and difficult and under-segmentation of objects can occur.
    [Show full text]
  • Introduction to Hidden Markov Models
    Introduction to Hidden Markov Models Slides Borrowed From Venu Govindaraju Markov Models • Set of states: • Process moves from one state to another generating a sequence of states : • Markov chain property: probability of each subsequent state depends only on what was the previous state: • To define Markov model, the following probabilities have to be specified: transition probabilities and initial probabilities Example of Markov Model 0.3 0.7 Rain Dry 0.2 0.8 • Two states : ‘Rain’ and ‘Dry’. • Transition probabilities: P(‘Rain’|‘Rain’)=0.3 , P(‘Dry’|‘Rain’)=0.7 , P(‘Rain’|‘Dry’)=0.2, P(‘Dry’|‘Dry’)=0.8 • Initial probabilities: say P(‘Rain’)=0.4 , P(‘Dry’)=0.6 . Calculation of sequence probability • By Markov chain property, probability of state sequence can be found by the formula: • Suppose we want to calculate a probability of a sequence of states in our example, {‘Dry’,’Dry’,’Rain’,Rain’}. P({‘Dry’,’Dry’,’Rain’,Rain’} ) = P(‘Rain’|’Rain’) P(‘Rain’|’Dry’) P(‘Dry’|’Dry’) P(‘Dry’)= = 0.3*0.2*0.8*0.6 Hidden Markov models. • Set of states: • Process moves from one state to another generating a sequence of states : • Markov chain property: probability of each subsequent state depends only on what was the previous state: P (sik | si1,si2,…,sik−1) = P(sik | sik−1) • States are not visible, but each state randomly generates one of M observations (or visible states) • To€ define hidden Markov model, the following probabilities have to be specified: matrix of transition probabilities A=(aij), aij= P(si | sj) , matrix of observation probabilities B=(bi (vm )), bi(vm ) = P(vm | si) and a vector of initial probabilities π=(πi), πi = P(si) .
    [Show full text]
  • Chapter 2 Some Basic Tools
    Chapter 2 Some basic tools 2.1 Time series: Theory 2.1.1 Stochastic processes A stochastic process is a sequence of random variables . , x0, x1, x2,.... In this class, the subscript always means time. As always, x can be a vector. Associated with this stochastic process is its history: Ht = {xt, xt−1, xt−2,...} (2.1) The history is just the set of past values of xt. The expected value at time t of a particular random variable is just: Et(xs) = E(xs|Ht) (2.2) or its expected value conditional on all information available at t. Notice that if s ≤ t, then Et(xs) = xs. A few results from basic probability theory should be noted here. 1. E(.) is a linear operator, i.e., if X, Y , and Z are random variables then E(a(Z)X + b(Z)Y |Z = z) = a(z)E(x|Z = z) + b(z)E(y|Z = z). 2. The law of iterated expectations. Let Ω0 ⊂ Ω1 be sets of conditioning variables (or equivalently let Ω0 and Ω1 be events and let Ω1 ⊂ Ω0), and let X be a random variable. Then E(E(X|Ω1)|Ω0) = E(X|Ω0). 1 2 CHAPTER 2. SOME BASIC TOOLS 3. If E(Y |X) = 0, then E(f(x)Y ) = 0 for any function f(.). Next we consider two special classes of stochastic process that macroeconomists find particularly useful. 2.1.2 Markov processes Definition 2.1.1 (Markov process) A Markov process is a stochastic pro- cess with the property that: Pr(xt+1|Ht) = Pr(xt+1|xt) (2.3) where Pr(x) is the probability distribution of x.
    [Show full text]