DOCSLIB.ORG

Markov Chains and Hidden Markov Models

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Markov Chains and Hidden Markov Models COMP 182 Algorithmic Thinking Luay Nakhleh Markov Chains and Computer Science Hidden Markov Models Rice University ❖ What is p(01110000)? ❖ Assume: ❖ 8 independent Bernoulli trials with success probability of α? ❖ Answer: ❖ (1-α)5α3 ❖ However, what if the assumption of independence doesn’t hold? ❖ That is, what if the outcome in a Bernoulli trial depends on the outcomes of the trials the preceded it? Markov Chains ❖ Given a sequence of observations X1,X2,…,XT ❖ The basic idea behind a Markov chain (or, Markov model) is to assume that Xt captures all the relevant information for predicting the future. ❖ In this case: T p(X1X2 ...XT )=p(X1)p(X2 X1)p(X3 X2) p(XT XT 1)=p(X1) p(Xt Xt 1) | | ··· | − | − t=2 Y Markov Chains ❖ When Xt is discrete, so Xt∈{1,…,K}, the conditional distribution p(Xt|Xt-1) can be written as a K×K matrix, known as the transition matrix A, where Aij=p(Xt=j|Xt-1=i) is the probability of going from state i to state j. ❖ Each row of the matrix sums to one, so this is called a stochastic matrix. 590 Chapter 17. Markov and hidden Markov models Markov Chains 1 α 1 β − α − A11 A22 A33 1 2 A12 A23 1 2 3 β ❖ A finite-state Markov chain is equivalent(a) (b) Figure 17.1 State transition diagrams for some simple Markov chains. Left: a 2-state chain. Right: a to a stochastic automaton3-state left-to-right. chain. ❖ One way to represent aAstationary,finite-stateMarkovchainisequivalenttoa Markov chain is stochastic automaton.Itiscommon 590 to visualizeChapter such 17. automataMarkov and by hidden drawing Markov a directed models graph, where nodes represent states and arrows through a state transitionrepresent legaldiagram transitions, i.e., non-zero elements of A.Thisisknownasastate transition diagram.Theweightsassociatedwiththearcsaretheprobabilities.Forexample,thefollowing 2-state chain 1 α 1 β − α A11 A22 A33 − 1 αα A = A12 A23 (17.2) 1 2 −β 1 β ! 1 − 2 " 3 β is illustrated in Figure 17.1(left). The following 3-state chain (a) A11 A12 (b) 0 A = 0 A22 A23 (17.3) Figure 17.1 State transition diagrams for some simple⎛ Markov001 chains. Left: a⎞ 2-state chain. Right: a 3-state left-to-right chain. is illustrated⎝ in Figure 17.1(right).⎠ This is called a left-to-right transition matrix,andiscom- monly used in speech recognition (Section 17.6.2). Astationary,finite-stateMarkovchainisequivalenttoastochastic automaton.Itiscommon The A element of the transition matrix specifies the probability of getting from i to j in to visualize such automata by drawing a directed graph,ij where nodes represent states and arrows represent legal transitions, i.e., non-zero elementsone step. of TheA.Thisisknownasan-step transition matrixstate transitionA(n) is defined as diagram.Theweightsassociatedwiththearcsaretheprobabilities.Forexample,thefollowing Aij(n) ! p(Xt+n = j Xt = i) (17.4) 2-state chain | 1 αα which is the probability of getting from i to j in exactly n steps. Obviously A(1) = A.The A = (17.2) −β 1 β Chapman-Kolmogorov equations state that ! − " is illustrated in Figure 17.1(left). The following 3-state chain K Aij(m + n)= Aik(m)Akj(n) (17.5) A11 A12 0 k=1 A = 0 A22 A23 ' (17.3) ⎛ 001⎞ In words, the probability of getting from i to j in m + n steps is just the probability of getting is illustrated⎝ in Figure 17.1(right).⎠ This is calledfrom ai left-to-rightto k in m steps, transition and then matrix from,andiscom-k to j in n steps, summed up over all k.Wecanwrite monly used in speech recognition (Sectionthe 17.6.2). above as a matrix multiplication The Aij element of the transition matrix specifiesA(m + then)= probabilityA(m)A of(n getting) from i to j in (17.6) one step. The n-step transition matrix A(n) is defined as Hence A (n) p(X = j X = i) (17.4) ij ! t+n | t A(n)=AA(n 1) = AAA(n 2) = = An (17.7) which is the probability of getting from i to j in exactly n steps.− Obviously A(1) =−A.The··· Chapman-Kolmogorov equations state thatThus we can simulate multiple steps of a Markov chain by “powering up” the transition matrix. K Aij(m + n)= Aik(m)Akj(n) (17.5) k'=1 In words, the probability of getting from i to j in m + n steps is just the probability of getting from i to k in m steps, and then from k to j in n steps, summed up over all k.Wecanwrite the above as a matrix multiplication A(m + n)=A(m)A(n) (17.6) Hence A(n)=AA(n 1) = AAA(n 2) = = An (17.7) − − ··· Thus we can simulate multiple steps of a Markov chain by “powering up” the transition matrix. Markov Chains ❖ The Aij element of the transition matrix specifies the probability of getting from i to j in one step. ❖ The n-step transition matrix A(n) is defined as A (n)=p(X = j X = i) ij t+n | t Markov Chains ❖ Obviously, A(1)=A. ❖ The Chapman-Kolmogorov equations state that K Aij(m + n)= Aik(m)Akj(n) kX=1 Markov Chains ❖ Therefore, we have A(m+n)=A(m)A(n). ❖ Hence, A(n)=AA(n-1)=AAA(n-2)=…=An. ❖ Thus, we can simulate multiple steps of a Markov chain by “powering up” the transition matrix. Language Modeling ❖ One important application of Markov models is to make statistical language models, which are probability distributions over sequences of words. ❖ We define the state space to be all the words in English (or, the language of interest). ❖ The probabilities p(Xt=k) are called unigram statistics. ❖ If we use a first-order Markov model, then p(Xt=k|Xt-1=j) is called a bigram model. 592 Language ModelingChapter 17. Markov and hidden Markov models Bigrams Unigrams _ a b c d e f g h i j k l m n o p q r s t u v w x y z 1 0.16098 _ _ 2 0.06687 a a 3 0.01414 b b 4 0.02938 c c 5 0.03107 d d 6 0.11055 e e 7 0.02325 f f 8 0.01530 g g 9 0.04174 h h 10 0.06233 i i 11 0.00060 j j 12 0.00309 k k 13 0.03515 l l 14 0.02107 m m 15 0.06007 n n 16 0.06066 o o 17 0.01594 p p 18 0.00077 q q 19 0.05265 r r 20 0.05761 s s 21 0.07566 t t 22 0.02149 u u 23 0.00993 v v 24 0.01341 w w 25 0.00208 x x 26 0.01381 y y 27 0.00039 z z Unigram and bigram counts for Darwin’s On the Origin of Species Figure 17.2 Unigram and bigram counts from Darwin’s On The Origin Of Species.The2Dpictureonthe right is a Hinton diagram of the joint distribution. The size of the white squares is proportional to the value of the entry in the corresponding vector/ matrix. Based on (MacKay 2003, p22). Figure generated by ngramPlot. 17.2.2.1 MLE for Markov language models We now discuss a simple way to estimate the transition matrix from training data. The proba- bility of any particular sequence of length T is given by p(x1:T θ)=π(x1)A(x1,x2 ) ...A(xT 1,xT ) (17.8) | − K T K K I(x1=j) I(xt=k,xt 1=j) = (πj ) (Ajk) − (17.9) j=1 t=2 j=1 ! ! ! k!=1 Hence the log-likelihood of a set of sequences =(x ,...,x ),wherex =(x ,...,x ) D 1 N i i1 i,Ti is a sequence of length Ti,isgivenby N logp( θ)= logp(x θ)= N 1 logπ + N logA (17.10) D| i| j j jk jk i=1 j j " " " "k where we define the following counts: N N T 1 i− 1 Nj ! I(xi1 = j),Njk ! I(xi,t = j, xi,t+1 = k) (17.11) i=1 i=1 t=1 " " " Parameter Estimation for Markov Chains ❖ The parameters of a Markov chain, denoted by θ, consist of the transition matrix (A) and the distribution on the initial states (#). ❖ We want to estimate these parameters from a training data set. ❖ Such a data set consists of sequences X1,X2,…,Xm. ❖ Sequence Xk has length Lk. Parameter Estimation for Markov Chains ❖ The maximum likelihood estimate (MLE) of the parameters is easy to obtain from the data: m m L 1 i− 1 i i i Nj = I(X1 = j) Njk = I(Xt = j, Xt+1 = k) i=1 i=1 t=1 X X X N 1 N j Aˆ = jk ⇡ˆj = 1 jk Njk i Ni k0 0 P P Parameter Estimation for Markov Chains ❖ The maximum likelihood estimate (MLE) of the parameters is easy to obtain from the data: m m L 1 i− 1 i i i Nj = I(X1 = j) Njk = I(Xt = j, Xt+1 = k) i=1 i=1 t=1 X X X N 1 N j Aˆ = jk ⇡ˆj = 1 jk Njk i Ni k0 0 1ife is true Indicator function:P (e)= P I 0ife is false ⇢ Parameter Estimation for Markov Chains ❖ It is very important to handle zero counts properly (this is called smoothing). Hidden Markov Models ❖ A hidden Markov model, or HMM, consists of a discrete-time, discrete state Markov chain, with hidden states Zt∈{1,…,K}, plus an observation model p(Xt|Zt) (emission probabilities).
Load more

Recommended publications
  • Multivariate Poisson Hidden Markov Models for Analysis of Spatial Counts
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by University of Saskatchewan's Research Archive MULTIVARIATE POISSON HIDDEN MARKOV MODELS FOR ANALYSIS OF SPATIAL COUNTS A Thesis Submitted to the Faculty of Graduate Studies and Research in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in the Department of Mathematics and Statistics University of Saskatchewan, Saskatoon, SK, Canada by Chandima Piyadharshani Karunanayake @Copyright Chandima Piyadharshani Karunanayake, June 2007. All rights Reserved. PERMISSION TO USE The author has agreed that the libraries of this University may provide the thesis freely available for inspection. Moreover, the author has agreed that permission for copying of the thesis in any manner, entirely or in part, for scholarly purposes may be granted by the Professor or Professors who supervised my thesis work or in their absence, by the Head of the Department of Mathematics and Statistics or the Dean of the College in which the thesis work was done. It is understood that any copying or publication or use of the thesis or parts thereof for finanancial gain shall not be allowed without my written permission. It is also understood that due recognition shall be given to the author and to the University of Saskatchewan in any scholarly use which may be made of any material in this thesis. Requests for permission to copy or to make other use of any material in the thesis should be addressed to: Head Department of Mathematics and Statistics University of Saskatchewan 106, Wiggins Road Saskatoon, Saskatchewan Canada, S7N 5E6 i ABSTRACT Multivariate count data are found in a variety of fields.
    [Show full text]
  • Regime Heteroskedasticity in Bitcoin: a Comparison of Markov Switching Models
    Munich Personal RePEc Archive Regime heteroskedasticity in Bitcoin: A comparison of Markov switching models Chappell, Daniel Birkbeck College, University of London 28 September 2018 Online at https://mpra.ub.uni-muenchen.de/90682/ MPRA Paper No. 90682, posted 24 Dec 2018 06:38 UTC Regime heteroskedasticity in Bitcoin: A comparison of Markov switching models Daniel R. Chappell Department of Economics, Mathematics and Statistics Birkbeck College, University of London [email protected] 28th September 2018 Abstract Markov regime-switching (MRS) models, also known as hidden Markov models (HMM), are used extensively to account for regime heteroskedasticity within the returns of financial assets. However, we believe this paper to be one of the first to apply such methodology to the time series of cryptocurrencies. In light of Moln´arand Thies (2018) demonstrating that the price data of Bitcoin contained seven distinct volatility regimes, we will fit a sample of Bitcoin returns with six m-state MRS estimations, with m ∈ {2,..., 7}. Our aim is to identify the optimal number of states for modelling the regime heteroskedasticity in the price data of Bitcoin. Goodness-of-fit will be judged using three information criteria, namely: Bayesian (BIC); Hannan-Quinn (HQ); and Akaike (AIC). We determined that the restricted 5-state model generated the optimal estima- tion for the sample. In addition, we found evidence of volatility clustering, volatility jumps and asymmetric volatility transitions whilst also inferring the persistence of shocks in the price data of Bitcoin. Keywords Bitcoin; Markov regime-switching; regime heteroskedasticity; volatility transitions. 1 2 List of Tables Table 1. Summary statistics for Bitcoin (23rd April 2014 to 31st May 2018) .
    [Show full text]
  • 12 : Conditional Random Fields 1 Hidden Markov Model
    10-708: Probabilistic Graphical Models 10-708, Spring 2014 12 : Conditional Random Fields Lecturer: Eric P. Xing Scribes: Qin Gao, Siheng Chen 1 Hidden Markov Model 1.1 General parametric form In hidden Markov model (HMM), we have three sets of parameters, j i transition probability matrix A : p(yt = 1jyt−1 = 1) = ai;j; initialprobabilities : p(y1) ∼ Multinomial(π1; π2; :::; πM ); i emission probabilities : p(xtjyt) ∼ Multinomial(bi;1; bi;2; :::; bi;K ): 1.2 Inference k k The inference can be done with forward algorithm which computes αt ≡ µt−1!t(k) = P (x1; :::; xt−1; xt; yt = 1) recursively by k k X i αt = p(xtjyt = 1) αt−1ai;k; (1) i k k and the backward algorithm which computes βt ≡ µt t+1(k) = P (xt+1; :::; xT jyt = 1) recursively by k X i i βt = ak;ip(xt+1jyt+1 = 1)βt+1: (2) i Another key quantity is the conditional probability of any hidden state given the entire sequence, which can be computed by the dot product of forward message and backward message by, i i i i X i;j γt = p(yt = 1jx1:T ) / αtβt = ξt ; (3) j where we define, i;j i j ξt = p(yt = 1; yt−1 = 1; x1:T ); i j / µt−1!t(yt = 1)µt t+1(yt+1 = 1)p(xt+1jyt+1)p(yt+1jyt); i j i = αtβt+1ai;jp(xt+1jyt+1 = 1): The implementation in Matlab can be vectorized by using, i Bt(i) = p(xtjyt = 1); j i A(i; j) = p(yt+1 = 1jyt = 1): 1 2 12 : Conditional Random Fields The relation of those quantities can be simply written in pseudocode as, T αt = (A αt−1): ∗ Bt; βt = A(βt+1: ∗ Bt+1); T ξt = (αt(βt+1: ∗ Bt+1) ): ∗ A; γt = αt: ∗ βt: 1.3 Learning 1.3.1 Supervised Learning The supervised learning is trivial if only we know the true state path.
    [Show full text]
  • Modeling Dependence in Data: Options Pricing and Random Walks
    UNIVERSITY OF CALIFORNIA, MERCED PH.D. DISSERTATION Modeling Dependence in Data: Options Pricing and Random Walks Nitesh Kumar A dissertation submitted in partial fulfillment of the requirements for the degree Doctor of Philosophy in Applied Mathematics March, 2013 UNIVERSITY OF CALIFORNIA, MERCED Graduate Division This is to certify that I have examined a copy of a dissertation by Nitesh Kumar and found it satisfactory in all respects, and that any and all revisions required by the examining committee have been made. Faculty Advisor: Harish S. Bhat Committee Members: Arnold D. Kim Roummel F. Marcia Applied Mathematics Graduate Studies Chair: Boaz Ilan Arnold D. Kim Date Contents 1 Introduction 2 1.1 Brief Review of the Option Pricing Problem and Models . ......... 2 2 Markov Tree: Discrete Model 6 2.1 Introduction.................................... 6 2.2 Motivation...................................... 7 2.3 PastWork....................................... 8 2.4 Order Estimation: Methodology . ...... 9 2.5 OrderEstimation:Results. ..... 13 2.6 MarkovTreeModel:Theory . 14 2.6.1 NoArbitrage.................................. 17 2.6.2 Implementation Notes. 18 2.7 TreeModel:Results................................ 18 2.7.1 Comparison of Model and Market Prices. 19 2.7.2 Comparison of Volatilities. 20 2.8 Conclusion ...................................... 21 3 Markov Tree: Continuous Model 25 3.1 Introduction.................................... 25 3.2 Markov Tree Generation and Computational Tractability . ............. 26 3.2.1 Persistentrandomwalk. 27 3.2.2 Number of states in a tree of fixed depth . ..... 28 3.2.3 Markov tree probability mass function . ....... 28 3.3 Continuous Approximation of the Markov Tree . ........ 30 3.3.1 Recursion................................... 30 3.3.2 Exact solution in Fourier space .
    [Show full text]
  • Estimation of Viterbi Path in Bayesian Hidden Markov Models
    Estimation of Viterbi path in Bayesian hidden Markov models Jüri Lember1,∗ Dario Gasbarra2, Alexey Koloydenko3 and Kristi Kuljus1 May 14, 2019 1University of Tartu, Estonia; 2University of Helsinki, Finland; 3Royal Holloway, University of London, UK Abstract The article studies different methods for estimating the Viterbi path in the Bayesian framework. The Viterbi path is an estimate of the underlying state path in hidden Markov models (HMMs), which has a maximum joint posterior probability. Hence it is also called the maximum a posteriori (MAP) path. For an HMM with given parameters, the Viterbi path can be easily found with the Viterbi algorithm. In the Bayesian framework the Viterbi algorithm is not applicable and several iterative methods can be used instead. We introduce a new EM-type algorithm for finding the MAP path and compare it with various other methods for finding the MAP path, including the variational Bayes approach and MCMC methods. Examples with simulated data are used to compare the performance of the methods. The main focus is on non-stochastic iterative methods and our results show that the best of those methods work as well or better than the best MCMC methods. Our results demonstrate that when the primary goal is segmentation, then it is more reasonable to perform segmentation directly by considering the transition and emission parameters as nuisance parameters. Keywords: HMM, Bayes inference, MAP path, Viterbi algorithm, segmentation, EM, variational Bayes, simulated annealing 1 Introduction and preliminaries arXiv:1802.01630v2 [stat.CO] 11 May 2019 Hidden Markov models (HMMs) are widely used in application areas including speech recognition, com- putational linguistics, computational molecular biology, and many more.
    [Show full text]
  • Ergodicity, Decisions, and Partial Information
    Ergodicity, Decisions, and Partial Information Ramon van Handel Abstract In the simplest sequential decision problem for an ergodic stochastic pro- cess X, at each time n a decision un is made as a function of past observations X0,...,Xn 1, and a loss l(un,Xn) is incurred. In this setting, it is known that one may choose− (under a mild integrability assumption) a decision strategy whose path- wise time-average loss is asymptotically smaller than that of any other strategy. The corresponding problem in the case of partial information proves to be much more delicate, however: if the process X is not observable, but decisions must be based on the observation of a different process Y, the existence of pathwise optimal strategies is not guaranteed. The aim of this paper is to exhibit connections between pathwise optimal strategies and notions from ergodic theory. The sequential decision problem is developed in the general setting of an ergodic dynamical system (Ω,B,P,T) with partial information Y B. The existence of pathwise optimal strategies grounded in ⊆ two basic properties: the conditional ergodic theory of the dynamical system, and the complexity of the loss function. When the loss function is not too complex, a gen- eral sufficient condition for the existence of pathwise optimal strategies is that the dynamical system is a conditional K-automorphism relative to the past observations n n 0 T Y. If the conditional ergodicity assumption is strengthened, the complexity assumption≥ can be weakened. Several examples demonstrate the interplay between complexity and ergodicity, which does not arise in the case of full information.
    [Show full text]
  • A Class of Measure-Valued Markov Chains and Bayesian Nonparametrics
    Bernoulli 18(3), 2012, 1002–1030 DOI: 10.3150/11-BEJ356 A class of measure-valued Markov chains and Bayesian nonparametrics STEFANO FAVARO1, ALESSANDRA GUGLIELMI2 and STEPHEN G. WALKER3 1Universit`adi Torino and Collegio Carlo Alberto, Dipartimento di Statistica e Matematica Ap- plicata, Corso Unione Sovietica 218/bis, 10134 Torino, Italy. E-mail: [email protected] 2Politecnico di Milano, Dipartimento di Matematica, P.zza Leonardo da Vinci 32, 20133 Milano, Italy. E-mail: [email protected] 3University of Kent, Institute of Mathematics, Statistics and Actuarial Science, Canterbury CT27NZ, UK. E-mail: [email protected] Measure-valued Markov chains have raised interest in Bayesian nonparametrics since the seminal paper by (Math. Proc. Cambridge Philos. Soc. 105 (1989) 579–585) where a Markov chain having the law of the Dirichlet process as unique invariant measure has been introduced. In the present paper, we propose and investigate a new class of measure-valued Markov chains defined via exchangeable sequences of random variables. Asymptotic properties for this new class are derived and applications related to Bayesian nonparametric mixture modeling, and to a generalization of the Markov chain proposed by (Math. Proc. Cambridge Philos. Soc. 105 (1989) 579–585), are discussed. These results and their applications highlight once again the interplay between Bayesian nonparametrics and the theory of measure-valued Markov chains. Keywords: Bayesian nonparametrics; Dirichlet process; exchangeable sequences; linear functionals
    [Show full text]
  • A Study of Hidden Markov Model
    University of Tennessee, Knoxville TRACE: Tennessee Research and Creative Exchange Masters Theses Graduate School 8-2004 A Study of Hidden Markov Model Yang Liu University of Tennessee - Knoxville Follow this and additional works at: https://trace.tennessee.edu/utk_gradthes Part of the Mathematics Commons Recommended Citation Liu, Yang, "A Study of Hidden Markov Model. " Master's Thesis, University of Tennessee, 2004. https://trace.tennessee.edu/utk_gradthes/2326 This Thesis is brought to you for free and open access by the Graduate School at TRACE: Tennessee Research and Creative Exchange. It has been accepted for inclusion in Masters Theses by an authorized administrator of TRACE: Tennessee Research and Creative Exchange. For more information, please contact [email protected]. To the Graduate Council: I am submitting herewith a thesis written by Yang Liu entitled "A Study of Hidden Markov Model." I have examined the final electronic copy of this thesis for form and content and recommend that it be accepted in partial fulfillment of the equirr ements for the degree of Master of Science, with a major in Mathematics. Jan Rosinski, Major Professor We have read this thesis and recommend its acceptance: Xia Chen, Balram Rajput Accepted for the Council: Carolyn R. Hodges Vice Provost and Dean of the Graduate School (Original signatures are on file with official studentecor r ds.) To the Graduate Council: I am submitting herewith a thesis written by Yang Liu entitled “A Study of Hidden Markov Model.” I have examined the final electronic copy of this thesis for form and content and recommend that it be accepted in partial fulfillment of the requirements for the degree of Master of Science, with a major in Mathematics.
    [Show full text]
  • Markov Process Duality
    Markov Process Duality Jan M. Swart Luminy, October 23 and 24, 2013 Jan M. Swart Markov Process Duality Markov Chains S finite set. RS space of functions f : S R. ! S For probability kernel P = (P(x; y))x;y2S and f R define left and right multiplication as 2 X X Pf (x) := P(x; y)f (y) and fP(x) := f (y)P(y; x): y y (I do not distinguish row and column vectors.) Def Chain X = (Xk )k≥0 of S-valued r.v.'s is Markov chain with transition kernel P and state space S if S E f (Xk+1) (X0;:::; Xk ) = Pf (Xk ) a:s: (f R ) 2 P (X0;:::; Xk ) = (x0;:::; xk ) , = P[X0 = x0]P(x0; x1) P(xk−1; xk ): ··· µ µ µ Write P ; E for process with initial law µ = P [X0 ]. x δx x 2 · P := P with δx (y) := 1fx=yg. E similar. Jan M. Swart Markov Process Duality Markov Chains Set k µ k x µk := µP (x) = P [Xk = x] and fk := P f (x) = E [f (Xk )]: Then the forward and backward equations read µk+1 = µk P and fk+1 = Pfk : In particular π invariant law iff πP = π. Function h harmonic iff Ph = h h(Xk ) martingale. , Jan M. Swart Markov Process Duality Random mapping representation (Zk )k≥1 i.i.d. with common law ν, take values in (E; ). φ : S E S measurable E × ! P(x; y) = P[φ(x; Z1) = y]: Random mapping representation (E; ; ν; φ) always exists, highly non-unique.
    [Show full text]
  • Particle Gibbs for Infinite Hidden Markov Models
    Particle Gibbs for Infinite Hidden Markov Models Nilesh Tripuraneni* Shixiang Gu* Hong Ge University of Cambridge University of Cambridge University of Cambridge [email protected] MPI for Intelligent Systems [email protected] [email protected] Zoubin Ghahramani University of Cambridge [email protected] Abstract Infinite Hidden Markov Models (iHMM’s) are an attractive, nonparametric gener- alization of the classical Hidden Markov Model which can automatically infer the number of hidden states in the system. However, due to the infinite-dimensional nature of the transition dynamics, performing inference in the iHMM is difficult. In this paper, we present an infinite-state Particle Gibbs (PG) algorithm to re- sample state trajectories for the iHMM. The proposed algorithm uses an efficient proposal optimized for iHMMs and leverages ancestor sampling to improve the mixing of the standard PG algorithm. Our algorithm demonstrates significant con- vergence improvements on synthetic and real world data sets. 1 Introduction Hidden Markov Models (HMM’s) are among the most widely adopted latent-variable models used to model time-series datasets in the statistics and machine learning communities. They have also been successfully applied in a variety of domains including genomics, language, and finance where sequential data naturally arises [Rabiner, 1989; Bishop, 2006]. One possible disadvantage of the finite-state space HMM framework is that one must a-priori spec- ify the number of latent states K. Standard model selection techniques can be applied to the fi- nite state-space HMM but bear a high computational overhead since they require the repetitive trainingexploration of many HMM’s of different sizes.
    [Show full text]
  • Hidden Markov Models (Particle Filtering)
    CSE 473: Artificial Intelligence Spring 2014 Hidden Markov Models & Particle Filtering Hanna Hajishirzi Many slides adapted from Dan Weld, Pieter Abbeel, Dan Klein, Stuart Russell, Andrew Moore & Luke Zettlemoyer 1 Outline § Probabilistic sequence models (and inference) § Probability and Uncertainty – Preview § Markov Chains § Hidden Markov Models § Exact Inference § Particle Filters § Applications Example § A robot move in a discrete grid § May fail to move in the desired direction with some probability § Observation from noisy sensor at each time § Is a function of robot position § Goal: Find the robot position (probability that a robot is at a specific position) § Cannot always compute this probability exactly è Approximation methods Here: Approximate a distribution by sampling 3 Hidden Markov Model § State Space Model § Hidden states: Modeled as a Markov Process P(x0), P(xk | xk-1) § Observations: ek P(ek | xk) Position of the robot P(x1|x0) x x x 0 1 … n Observed position from P(e |x ) 0 0 the sensor y0 y1 yn 4 Exact Solution: Forward Algorithm § Filtering is the inference process of finding a distribution over XT given e1 through eT : P( XT | e1:t ) § We first compute P( X1 | e1 ): § For each t from 2 to T, we have P( Xt-1 | e1:t-1 ) § Elapse time: compute P( Xt | e1:t-1 ) § Observe: compute P(Xt | e1:t-1 , et) = P( Xt | e1:t ) Approximate Inference: § Sometimes |X| is too big for exact inference § |X| may be too big to even store B(X) § E.g. when X is continuous § |X|2 may be too big to do updates § Solution: approximate inference by sampling § How robot localization works in practice 6 What is Sampling? § Goal: Approximate the original distribution: § Approximate with Gaussian distribution § Draw samples from a distribution close enough to the original distribution § Here: A general framework for a sampling method 7 Approximate Solution: Perfect Sampling Robot path till time n 1 Time 1 Time n Assume we can sample Particle 1 x0:n from the original distribution .
    [Show full text]
  • 1 Introduction Branching Mechanism in a Superprocess from a Branching
    数理解析研究所講究録 1157 巻 2000 年 1-16 1 An example of random snakes by Le Gall and its applications 渡辺信三 Shinzo Watanabe, Kyoto University 1 Introduction The notion of random snakes has been introduced by Le Gall ([Le 1], [Le 2]) to construct a class of measure-valued branching processes, called superprocesses or continuous state branching processes ([Da], [Dy]). A main idea is to produce the branching mechanism in a superprocess from a branching tree embedded in excur- sions at each different level of a Brownian sample path. There is no clear notion of particles in a superprocess; it is something like a cloud or mist. Nevertheless, a random snake could provide us with a clear picture of historical or genealogical developments of”particles” in a superprocess. ” : In this note, we give a sample pathwise construction of a random snake in the case when the underlying Markov process is a Markov chain on a tree. A simplest case has been discussed in [War 1] and [Wat 2]. The construction can be reduced to this case locally and we need to consider a recurrence family of stochastic differential equations for reflecting Brownian motions with sticky boundaries. A special case has been already discussed by J. Warren [War 2] with an application to a coalescing stochastic flow of piece-wise linear transformations in connection with a non-white or non-Gaussian predictable noise in the sense of B. Tsirelson. 2 Brownian snakes Throughout this section, let $\xi=\{\xi(t), P_{x}\}$ be a Hunt Markov process on a locally compact separable metric space $S$ endowed with a metric $d_{S}(\cdot, *)$ .
    [Show full text]