DOCSLIB.ORG

Part VIII Markov Chains

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Part VIII Markov Chains Part VIII Markov chains - random walks Chapter 8. RANDOM WALKS - MARKOV CHAINS/MODELS Random walks on graphs are a very simple, interesting and fundamental tool with surprisingly many applications in informatics and also in mathematics and natural sciences. Design and analysis of randomized algorithms is one of them. The concept of a random walk is closely related with that of Markov chain/model { one of the key concepts of discrete stochastic processes. prof. Jozef Gruska IV054 8. Markov chains - random walks 2/110 APPLICATIONS In physical sciences, Markov chains provide a fundamental model for the emergence of global properties from local interactions. In informatics, random walks provide a general paradigm for random exploration of an exponentially large combinatorial structures (for example graphs), by a sequence of simple and local transitions. prof. Jozef Gruska IV054 8. Markov chains - random walks 3/110 HIDDEN MARKOV MODELS Hidden Markov Model (HMM) is a Markov model, with random transitions among states, extended by random outputs of the states. HMM works in such a way that an observer can see only a sequence of states' outputs and not the internal structure (states, transition and emission probabilities) of the underlying Markov model. Hidden Markov Model (HMM), has a lot of applications, especially in artificial intelligence. For example, almost all fast speech and patterns recognition systems use HMM. prof. Jozef Gruska IV054 8. Markov chains - random walks 4/110 DEVELOPMENT of MARKOV CHAIN The concept of Markov chain introduced Andreei Andreevich Markov in 1906, in order to consider the case of having a sequence of random experiments in which result of any experiment depends also on the result of the previous experiment. Before that only such sequences of random experiments were considered where the result of each random experiment was fully independent from all previous experiments. prof. Jozef Gruska IV054 8. Markov chains - random walks 5/110 UNIVERSALITY of QUANTUM RANDOM WALKS It can be also shown that any quantum evolution can be seen as so-called continuous quantum walk. prof. Jozef Gruska IV054 8. Markov chains - random walks 6/110 BASIC CONCEPTS I Notation Let G = (V ; E) be a connected, undirected, graph with n nodes (in V ) and m edges (in E). For a node v 2 V , let ΓG (v) denote the set of neighbors of v in G. A random walk on G is the following sequence of moves of the process that starts in some node v0 and then: a neighbor v1 of v0 is chosen, randomly and independently, and then the process moves (walks) from v0 to v1; afterwards a neighbor v2 of v1 is chosen randomly and independently and the process walks from v1 to v2; the process continues to walk this way :::for ever. Typical problems to explore for a given graph G: What is the expected number of steps to get from a given node u to a given node v. What is the expected number of steps needed to visit all nodes of G when starting in a given node u. prof. Jozef Gruska IV054 8. Markov chains - random walks 7/110 DRUNKARD's WALK Let a drunken seaman walk on a linear, both sides infinite graph, each time choosing a step right or left with the same probability. What are the probabilities for such a drunken man to be in a particular position after some steps in case he starts in some other position. prof. Jozef Gruska IV054 8. Markov chains - random walks 8/110 EXAMPLE Let G = Kn be the complete graph of n nodes u 6= v be any two vertices of G. It holds: The expected number of steps of a random walk that begins in u and ends when first reaching v is ??????? prof. Jozef Gruska IV054 8. Markov chains - random walks 9/110 Simple hypercubes prof. Jozef Gruska IV054 8. Markov chains - random walks 10/110 6-d hypercube prof. Jozef Gruska IV054 8. Markov chains - random walks 11/110 EXAMPLE - to finish Let G = Kn be the complete graph of n nodes It holds: The expected number of steps of a random walk in Kn that begins in a fixed node u and ends when first reaching a fixed node v is 1 n − 2 p = 1 · + (1 + p) · n − 1 n − 1 p = n − 1 prof. Jozef Gruska IV054 8. Markov chains - random walks 12/110 The expected number of steps to visit all nodes in G starting from any node u is (n − 1)Hn; where Hn is so called Harmonic number prof. Jozef Gruska IV054 8. Markov chains - random walks 13/110 A RELATED PROBLEM - COUPON SELECTION Coupon selector problem: There are n types of coupons and at each time a coupon is chosen randomly and returned. It has been shown that the average number of trials needed to have a coupon of each type is nHn: prof. Jozef Gruska IV054 8. Markov chains - random walks 14/110 EXAMPLE Let us consider graph K3 with the initial probabilities of its three nodes, say A0, A1, A2, being p0; p1; p2 and let 1=2 be the probability of the transmission through through any edge. If p is the initial probability of a node X, then probability of being in the same node after one step is 1 1 p P = (1 − p) = − ; 1 2 2 2 after two steps is 1 1 1 1 p P = (1 − (1 − p) ) = − + 2 2 2 2 4 4 and after i steps the probability is i X j+1 1 i p P = (−1) + (−1) : i 2j 2i j=1 Therefore 1 lim Pi = i!1 3 prof. Jozef Gruska IV054 8. Markov chains - random walks 15/110 EXAMPLE { 2-SATISFIABILITY A simple polynomial-time (Monte Carlo) algorithm will be given to solve 2-SAT problem (for a formula in conjunctive normal form in which each clause having two literals) satisfiable with a single assignment. Algorithm 1 Start with an arbitrary assignment. 2 while there is an unsatisfied clause C, choose randomly one of two literals of C and complement its value in the current assignment. Theorem The expected number of steps of the above algorithm at finding a satisfying assignment is O(n2) (where n is the number of variables). prof. Jozef Gruska IV054 8. Markov chains - random walks 16/110 RELATION TO A RANDOM WALK ON THE LINE Let A be a particular satisfying assignment. The progress of the above algorithm can be represented by a particle moving between integers f0; 1;:::; ng on the real line. The position of the particle will always indicate how many variables in the current assignment have the correct value (as in A). Crucial observation. In an unsatisfied clause at least one of two literals has an incorrect 1 value. Therefore at each step of the algorithm with probability 2 we increase by one the 1 number of variables having their correct value; and with probability 2 the number of variables having correct value is decreased by one. The motion of the particle therefore resembles a random walk on the line (that is on the linear graph). prof. Jozef Gruska IV054 8. Markov chains - random walks 17/110 STOCHASTIC PROCESSES in RANDOMIZED ALGORITHMS A stochastic process is a sequence of random variables fXt gt≥1, where we think of values of Xt as values (states) of the process at time t. Two types of stochastic processes come up over and over again in the analysis of randomized algorithms: 1 Martingale, where values of each next variable may depend, even in a complicated way, on the past history, but its expectation is 0. 2 Markov chain where next state depends always only on the current state and not on ways to get there - not on the past history. The most useful algorithmic property of Markov chains, to be explored in the next, is their convergence to a fixed (probability) distributions on states. prof. Jozef Gruska IV054 8. Markov chains - random walks 18/110 MARKOV CHAINS - 1st DEFINITION A Markov chain is a discrete-time stochastic process defined over a set of states S in terms of a matrix P of transition probabilities P(i; j) = pij = the probability that the next state will be j if the current state is i: Probability conditions: For any i; j it has to hold X 0 ≤ pij ≤ 1 and pij = 1: j Denote Xt the state of the Markov chain at time t. 1 The stochastic process fXt gt=0, specifying the history of the evolution of the Markov chain at time t, has the following memoryless property: The future behaviors of a Markov chain depends on its current state, and not how the chain arrived at the current state. That is, it holds, for any t ≥ 1: Pr[Xt+1 = jjX0 = i0; X1 = i1;:::; Xt = it = i] = Pr[Xt+1 = jjXt = i] = pij : prof. Jozef Gruska IV054 8. Markov chains - random walks 19/110 NOTE Note: Markov chains do not need to have prespecified initial states. In general, initial states are chosen according to some probability distribution X0 over S. Such a distribution is called the initial (probability) distribution. prof. Jozef Gruska IV054 8. Markov chains - random walks 20/110 MARKOV CHAIN {2nd DEFINITION A Markov chain is a discrete time stochastic process fXk gk≥0 of random variables with values in a countable set I such that for every k ≥ 1 and every i0; i1;:::; ik from I we have Pr[Xk = ik j Xk−1 = ik−1;:::; X0 = i0] = Pr[Xk = ik j Xk−1 = ik−1] = pik−1ik : The matrix P(i; j) = pij is called the transition matrix of one-step transition probabilities.
Load more

Recommended publications
  • Multivariate Poisson Hidden Markov Models for Analysis of Spatial Counts
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by University of Saskatchewan's Research Archive MULTIVARIATE POISSON HIDDEN MARKOV MODELS FOR ANALYSIS OF SPATIAL COUNTS A Thesis Submitted to the Faculty of Graduate Studies and Research in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in the Department of Mathematics and Statistics University of Saskatchewan, Saskatoon, SK, Canada by Chandima Piyadharshani Karunanayake @Copyright Chandima Piyadharshani Karunanayake, June 2007. All rights Reserved. PERMISSION TO USE The author has agreed that the libraries of this University may provide the thesis freely available for inspection. Moreover, the author has agreed that permission for copying of the thesis in any manner, entirely or in part, for scholarly purposes may be granted by the Professor or Professors who supervised my thesis work or in their absence, by the Head of the Department of Mathematics and Statistics or the Dean of the College in which the thesis work was done. It is understood that any copying or publication or use of the thesis or parts thereof for finanancial gain shall not be allowed without my written permission. It is also understood that due recognition shall be given to the author and to the University of Saskatchewan in any scholarly use which may be made of any material in this thesis. Requests for permission to copy or to make other use of any material in the thesis should be addressed to: Head Department of Mathematics and Statistics University of Saskatchewan 106, Wiggins Road Saskatoon, Saskatchewan Canada, S7N 5E6 i ABSTRACT Multivariate count data are found in a variety of fields.
    [Show full text]
  • Regime Heteroskedasticity in Bitcoin: a Comparison of Markov Switching Models
    Munich Personal RePEc Archive Regime heteroskedasticity in Bitcoin: A comparison of Markov switching models Chappell, Daniel Birkbeck College, University of London 28 September 2018 Online at https://mpra.ub.uni-muenchen.de/90682/ MPRA Paper No. 90682, posted 24 Dec 2018 06:38 UTC Regime heteroskedasticity in Bitcoin: A comparison of Markov switching models Daniel R. Chappell Department of Economics, Mathematics and Statistics Birkbeck College, University of London [email protected] 28th September 2018 Abstract Markov regime-switching (MRS) models, also known as hidden Markov models (HMM), are used extensively to account for regime heteroskedasticity within the returns of financial assets. However, we believe this paper to be one of the first to apply such methodology to the time series of cryptocurrencies. In light of Moln´arand Thies (2018) demonstrating that the price data of Bitcoin contained seven distinct volatility regimes, we will fit a sample of Bitcoin returns with six m-state MRS estimations, with m ∈ {2,..., 7}. Our aim is to identify the optimal number of states for modelling the regime heteroskedasticity in the price data of Bitcoin. Goodness-of-fit will be judged using three information criteria, namely: Bayesian (BIC); Hannan-Quinn (HQ); and Akaike (AIC). We determined that the restricted 5-state model generated the optimal estima- tion for the sample. In addition, we found evidence of volatility clustering, volatility jumps and asymmetric volatility transitions whilst also inferring the persistence of shocks in the price data of Bitcoin. Keywords Bitcoin; Markov regime-switching; regime heteroskedasticity; volatility transitions. 1 2 List of Tables Table 1. Summary statistics for Bitcoin (23rd April 2014 to 31st May 2018) .
    [Show full text]
  • Superprocesses and Mckean-Vlasov Equations with Creation of Mass
    Sup erpro cesses and McKean-Vlasov equations with creation of mass L. Overb eck Department of Statistics, University of California, Berkeley, 367, Evans Hall Berkeley, CA 94720, y U.S.A. Abstract Weak solutions of McKean-Vlasov equations with creation of mass are given in terms of sup erpro cesses. The solutions can b e approxi- mated by a sequence of non-interacting sup erpro cesses or by the mean- eld of multityp e sup erpro cesses with mean- eld interaction. The lat- ter approximation is asso ciated with a propagation of chaos statement for weakly interacting multityp e sup erpro cesses. Running title: Sup erpro cesses and McKean-Vlasov equations . 1 Intro duction Sup erpro cesses are useful in solving nonlinear partial di erential equation of 1+ the typ e f = f , 2 0; 1], cf. [Dy]. Wenowchange the p oint of view and showhowtheyprovide sto chastic solutions of nonlinear partial di erential Supp orted byanFellowship of the Deutsche Forschungsgemeinschaft. y On leave from the Universitat Bonn, Institut fur Angewandte Mathematik, Wegelerstr. 6, 53115 Bonn, Germany. 1 equation of McKean-Vlasovtyp e, i.e. wewant to nd weak solutions of d d 2 X X @ @ @ + d x; + bx; : 1.1 = a x; t i t t t t t ij t @t @x @x @x i j i i=1 i;j =1 d Aweak solution = 2 C [0;T];MIR satis es s Z 2 t X X @ @ a f = f + f + d f + b f ds: s ij s t 0 i s s @x @x @x 0 i j i Equation 1.1 generalizes McKean-Vlasov equations of twodi erenttyp es.
    [Show full text]
  • 12 : Conditional Random Fields 1 Hidden Markov Model
    10-708: Probabilistic Graphical Models 10-708, Spring 2014 12 : Conditional Random Fields Lecturer: Eric P. Xing Scribes: Qin Gao, Siheng Chen 1 Hidden Markov Model 1.1 General parametric form In hidden Markov model (HMM), we have three sets of parameters, j i transition probability matrix A : p(yt = 1jyt−1 = 1) = ai;j; initialprobabilities : p(y1) ∼ Multinomial(π1; π2; :::; πM ); i emission probabilities : p(xtjyt) ∼ Multinomial(bi;1; bi;2; :::; bi;K ): 1.2 Inference k k The inference can be done with forward algorithm which computes αt ≡ µt−1!t(k) = P (x1; :::; xt−1; xt; yt = 1) recursively by k k X i αt = p(xtjyt = 1) αt−1ai;k; (1) i k k and the backward algorithm which computes βt ≡ µt t+1(k) = P (xt+1; :::; xT jyt = 1) recursively by k X i i βt = ak;ip(xt+1jyt+1 = 1)βt+1: (2) i Another key quantity is the conditional probability of any hidden state given the entire sequence, which can be computed by the dot product of forward message and backward message by, i i i i X i;j γt = p(yt = 1jx1:T ) / αtβt = ξt ; (3) j where we define, i;j i j ξt = p(yt = 1; yt−1 = 1; x1:T ); i j / µt−1!t(yt = 1)µt t+1(yt+1 = 1)p(xt+1jyt+1)p(yt+1jyt); i j i = αtβt+1ai;jp(xt+1jyt+1 = 1): The implementation in Matlab can be vectorized by using, i Bt(i) = p(xtjyt = 1); j i A(i; j) = p(yt+1 = 1jyt = 1): 1 2 12 : Conditional Random Fields The relation of those quantities can be simply written in pseudocode as, T αt = (A αt−1): ∗ Bt; βt = A(βt+1: ∗ Bt+1); T ξt = (αt(βt+1: ∗ Bt+1) ): ∗ A; γt = αt: ∗ βt: 1.3 Learning 1.3.1 Supervised Learning The supervised learning is trivial if only we know the true state path.
    [Show full text]
  • Modeling Dependence in Data: Options Pricing and Random Walks
    UNIVERSITY OF CALIFORNIA, MERCED PH.D. DISSERTATION Modeling Dependence in Data: Options Pricing and Random Walks Nitesh Kumar A dissertation submitted in partial fulfillment of the requirements for the degree Doctor of Philosophy in Applied Mathematics March, 2013 UNIVERSITY OF CALIFORNIA, MERCED Graduate Division This is to certify that I have examined a copy of a dissertation by Nitesh Kumar and found it satisfactory in all respects, and that any and all revisions required by the examining committee have been made. Faculty Advisor: Harish S. Bhat Committee Members: Arnold D. Kim Roummel F. Marcia Applied Mathematics Graduate Studies Chair: Boaz Ilan Arnold D. Kim Date Contents 1 Introduction 2 1.1 Brief Review of the Option Pricing Problem and Models . ......... 2 2 Markov Tree: Discrete Model 6 2.1 Introduction.................................... 6 2.2 Motivation...................................... 7 2.3 PastWork....................................... 8 2.4 Order Estimation: Methodology . ...... 9 2.5 OrderEstimation:Results. ..... 13 2.6 MarkovTreeModel:Theory . 14 2.6.1 NoArbitrage.................................. 17 2.6.2 Implementation Notes. 18 2.7 TreeModel:Results................................ 18 2.7.1 Comparison of Model and Market Prices. 19 2.7.2 Comparison of Volatilities. 20 2.8 Conclusion ...................................... 21 3 Markov Tree: Continuous Model 25 3.1 Introduction.................................... 25 3.2 Markov Tree Generation and Computational Tractability . ............. 26 3.2.1 Persistentrandomwalk. 27 3.2.2 Number of states in a tree of fixed depth . ..... 28 3.2.3 Markov tree probability mass function . ....... 28 3.3 Continuous Approximation of the Markov Tree . ........ 30 3.3.1 Recursion................................... 30 3.3.2 Exact solution in Fourier space .
    [Show full text]
  • Ergodicity, Decisions, and Partial Information
    Ergodicity, Decisions, and Partial Information Ramon van Handel Abstract In the simplest sequential decision problem for an ergodic stochastic pro- cess X, at each time n a decision un is made as a function of past observations X0,...,Xn 1, and a loss l(un,Xn) is incurred. In this setting, it is known that one may choose− (under a mild integrability assumption) a decision strategy whose path- wise time-average loss is asymptotically smaller than that of any other strategy. The corresponding problem in the case of partial information proves to be much more delicate, however: if the process X is not observable, but decisions must be based on the observation of a different process Y, the existence of pathwise optimal strategies is not guaranteed. The aim of this paper is to exhibit connections between pathwise optimal strategies and notions from ergodic theory. The sequential decision problem is developed in the general setting of an ergodic dynamical system (Ω,B,P,T) with partial information Y B. The existence of pathwise optimal strategies grounded in ⊆ two basic properties: the conditional ergodic theory of the dynamical system, and the complexity of the loss function. When the loss function is not too complex, a gen- eral sufficient condition for the existence of pathwise optimal strategies is that the dynamical system is a conditional K-automorphism relative to the past observations n n 0 T Y. If the conditional ergodicity assumption is strengthened, the complexity assumption≥ can be weakened. Several examples demonstrate the interplay between complexity and ergodicity, which does not arise in the case of full information.
    [Show full text]
  • A Study of Hidden Markov Model
    University of Tennessee, Knoxville TRACE: Tennessee Research and Creative Exchange Masters Theses Graduate School 8-2004 A Study of Hidden Markov Model Yang Liu University of Tennessee - Knoxville Follow this and additional works at: https://trace.tennessee.edu/utk_gradthes Part of the Mathematics Commons Recommended Citation Liu, Yang, "A Study of Hidden Markov Model. " Master's Thesis, University of Tennessee, 2004. https://trace.tennessee.edu/utk_gradthes/2326 This Thesis is brought to you for free and open access by the Graduate School at TRACE: Tennessee Research and Creative Exchange. It has been accepted for inclusion in Masters Theses by an authorized administrator of TRACE: Tennessee Research and Creative Exchange. For more information, please contact [email protected]. To the Graduate Council: I am submitting herewith a thesis written by Yang Liu entitled "A Study of Hidden Markov Model." I have examined the final electronic copy of this thesis for form and content and recommend that it be accepted in partial fulfillment of the equirr ements for the degree of Master of Science, with a major in Mathematics. Jan Rosinski, Major Professor We have read this thesis and recommend its acceptance: Xia Chen, Balram Rajput Accepted for the Council: Carolyn R. Hodges Vice Provost and Dean of the Graduate School (Original signatures are on file with official studentecor r ds.) To the Graduate Council: I am submitting herewith a thesis written by Yang Liu entitled “A Study of Hidden Markov Model.” I have examined the final electronic copy of this thesis for form and content and recommend that it be accepted in partial fulfillment of the requirements for the degree of Master of Science, with a major in Mathematics.
    [Show full text]
  • Lecture 19: Polymer Models: Persistence and Self-Avoidance
    Lecture 19: Polymer Models: Persistence and Self-Avoidance Scribe: Allison Ferguson (and Martin Z. Bazant) Martin Fisher School of Physics, Brandeis University April 14, 2005 Polymers are high molecular weight molecules formed by combining a large number of smaller molecules (monomers) in a regular pattern. Polymers in solution (i.e. uncrystallized) can be characterized as long flexible chains, and thus may be well described by random walk models of various complexity. 1 Simple Random Walk (IID Steps) Consider a Rayleigh random walk (i.e. a fixed step size a with a random angle θ). Recall (from Lecture 1) that the probability distribution function (PDF) of finding the walker at position R~ after N steps is given by −3R2 2 ~ e 2a N PN (R) ∼ d (1) (2πa2N/d) 2 Define RN = |R~| as the end-to-end distance of the polymer. Then from previous results we q √ 2 2 ¯ 2 know hRN i = Na and RN = hRN i = a N. We can define an additional quantity, the radius of gyration GN as the radius from the centre of mass (assuming the polymer has an approximately 1 2 spherical shape ). Hughes [1] gives an expression for this quantity in terms of hRN i as follows: 1 1 hG2 i = 1 − hR2 i (2) N 6 N 2 N As an example of the type of prediction which can be made using this type of simple model, consider the following experiment: Pull on both ends of a polymer and measure the force f required to stretch it out2. Note that the force will be given by f = −(∂F/∂R) where F = U − TS is the Helmholtz free energy (T is the temperature, U is the internal energy, and S is the entropy).
    [Show full text]
  • Particle Gibbs for Infinite Hidden Markov Models
    Particle Gibbs for Infinite Hidden Markov Models Nilesh Tripuraneni* Shixiang Gu* Hong Ge University of Cambridge University of Cambridge University of Cambridge [email protected] MPI for Intelligent Systems [email protected] [email protected] Zoubin Ghahramani University of Cambridge [email protected] Abstract Infinite Hidden Markov Models (iHMM’s) are an attractive, nonparametric gener- alization of the classical Hidden Markov Model which can automatically infer the number of hidden states in the system. However, due to the infinite-dimensional nature of the transition dynamics, performing inference in the iHMM is difficult. In this paper, we present an infinite-state Particle Gibbs (PG) algorithm to re- sample state trajectories for the iHMM. The proposed algorithm uses an efficient proposal optimized for iHMMs and leverages ancestor sampling to improve the mixing of the standard PG algorithm. Our algorithm demonstrates significant con- vergence improvements on synthetic and real world data sets. 1 Introduction Hidden Markov Models (HMM’s) are among the most widely adopted latent-variable models used to model time-series datasets in the statistics and machine learning communities. They have also been successfully applied in a variety of domains including genomics, language, and finance where sequential data naturally arises [Rabiner, 1989; Bishop, 2006]. One possible disadvantage of the finite-state space HMM framework is that one must a-priori spec- ify the number of latent states K. Standard model selection techniques can be applied to the fi- nite state-space HMM but bear a high computational overhead since they require the repetitive trainingexploration of many HMM’s of different sizes.
    [Show full text]
  • Hidden Markov Models (Particle Filtering)
    CSE 473: Artificial Intelligence Spring 2014 Hidden Markov Models & Particle Filtering Hanna Hajishirzi Many slides adapted from Dan Weld, Pieter Abbeel, Dan Klein, Stuart Russell, Andrew Moore & Luke Zettlemoyer 1 Outline § Probabilistic sequence models (and inference) § Probability and Uncertainty – Preview § Markov Chains § Hidden Markov Models § Exact Inference § Particle Filters § Applications Example § A robot move in a discrete grid § May fail to move in the desired direction with some probability § Observation from noisy sensor at each time § Is a function of robot position § Goal: Find the robot position (probability that a robot is at a specific position) § Cannot always compute this probability exactly è Approximation methods Here: Approximate a distribution by sampling 3 Hidden Markov Model § State Space Model § Hidden states: Modeled as a Markov Process P(x0), P(xk | xk-1) § Observations: ek P(ek | xk) Position of the robot P(x1|x0) x x x 0 1 … n Observed position from P(e |x ) 0 0 the sensor y0 y1 yn 4 Exact Solution: Forward Algorithm § Filtering is the inference process of finding a distribution over XT given e1 through eT : P( XT | e1:t ) § We first compute P( X1 | e1 ): § For each t from 2 to T, we have P( Xt-1 | e1:t-1 ) § Elapse time: compute P( Xt | e1:t-1 ) § Observe: compute P(Xt | e1:t-1 , et) = P( Xt | e1:t ) Approximate Inference: § Sometimes |X| is too big for exact inference § |X| may be too big to even store B(X) § E.g. when X is continuous § |X|2 may be too big to do updates § Solution: approximate inference by sampling § How robot localization works in practice 6 What is Sampling? § Goal: Approximate the original distribution: § Approximate with Gaussian distribution § Draw samples from a distribution close enough to the original distribution § Here: A general framework for a sampling method 7 Approximate Solution: Perfect Sampling Robot path till time n 1 Time 1 Time n Assume we can sample Particle 1 x0:n from the original distribution .
    [Show full text]
  • 8 Convergence of Loop-Erased Random Walk to SLE2 8.1 Comments on the Paper
    8 Convergence of loop-erased random walk to SLE2 8.1 Comments on the paper The proof that the loop-erased random walk converges to SLE2 is in the paper \Con- formal invariance of planar loop-erased random walks and uniform spanning trees" by Lawler, Schramm and Werner. (arxiv.org: math.PR/0112234). This section contains some comments to give you some hope of being able to actually make sense of this paper. In the first sections of the paper, δ is used to denote the lattice spacing. The main theorem is that as δ 0, the loop erased random walk converges to SLE2. HOWEVER, beginning in section !2.2 the lattice spacing is 1, and in section 3.2 the notation δ reappears but means something completely different. (δ also appears in the proof of lemma 2.1 in section 2.1, but its meaning here is restricted to this proof. The δ in this proof has nothing to do with any δ's outside the proof.) After the statement of the main theorem, there is no mention of the lattice spacing going to zero. This is hidden in conditions on the \inner radius." For a domain D containing 0 the inner radius is rad0(D) = inf z : z = D (364) fj j 2 g Now fix a domain D containing the origin. Let be the conformal map of D onto U. We want to show that the image under of a LERW on a lattice with spacing δ converges to SLE2 in the unit disc. In the paper they do this in a slightly different way.
    [Show full text]
  • Hierarchical Dirichlet Process Hidden Markov Model for Unsupervised Bioacoustic Analysis
    Hierarchical Dirichlet Process Hidden Markov Model for Unsupervised Bioacoustic Analysis Marius Bartcus, Faicel Chamroukhi, Herve Glotin Abstract-Hidden Markov Models (HMMs) are one of the the standard HDP-HMM Gibbs sampling has the limitation most popular and successful models in statistics and machine of an inadequate modeling of the temporal persistence of learning for modeling sequential data. However, one main issue states [9]. This problem has been addressed in [9] by relying in HMMs is the one of choosing the number of hidden states. on a sticky extension which allows a more robust learning. The Hierarchical Dirichlet Process (HDP)-HMM is a Bayesian Other solutions for the inference of the hidden Markov non-parametric alternative for standard HMMs that offers a model in this infinite state space models are using the Beam principled way to tackle this challenging problem by relying sampling [lO] rather than Gibbs sampling. on a Hierarchical Dirichlet Process (HDP) prior. We investigate the HDP-HMM in a challenging problem of unsupervised We investigate the BNP formulation for the HMM, that is learning from bioacoustic data by using Markov-Chain Monte the HDP-HMM into a challenging problem of unsupervised Carlo (MCMC) sampling techniques, namely the Gibbs sampler. learning from bioacoustic data. The problem consists of We consider a real problem of fully unsupervised humpback extracting and classifying, in a fully unsupervised way, whale song decomposition. It consists in simultaneously finding the structure of hidden whale song units, and automatically unknown number of whale song units. We use the Gibbs inferring the unknown number of the hidden units from the Mel sampler to infer the HDP-HMM from the bioacoustic data.
    [Show full text]