DOCSLIB.ORG

Lecture 5: Random Walks and Markov Chain

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

Spectral Graph Theory and Applications WS 2011/2012 Lecture 5: Random Walks and Markov Chain Lecturer: Thomas Sauerwald & He Sun 1 Introduction to Markov Chains Definition 5.1. A sequence of random variables X0;X1;::: ⊆ Ω is a Markov chain with state space Ω if for all possible states xt+1; xt; : : : ; x1 2 Ω we have Pr [ Xt+1 = xt+1 j Xt = xt ^ Xt−1 = xt−1 ^ · · · ^ X0 = x0 ] = Pr [ Xt+1 = xt+1 j Xt = xt ] : The Markov chain is called time-homogenous if the latter probability is independent of t. In this course, we will only consider finite Markov chains which are also time-homogenous. In this case, it is convenient to store all information about the transition in a Markov chain in the so-called transition matrix P which is a jΩj × jΩj matrix defined by: Px;y := Pr [ X1 = y j X0 = x ] : Properties of the transition matrix P: • all entries in P are in [0; 1], • every row sum equals 1. 0 0 1=2 1=2 0 0 1 B1=3 0 1=3 1=3 0 C B C P = B1=4 1=4 0 1=4 1=4C B C @ 0 1=2 0 0 1=2A 0 0 0 1 0 1 4 2 3 5 Figure 1: Example of a Markov chain corresponding to a random walk on a graph G with 5 vertices. A very important special case is the Markov chain that corresponds to a random walk on an undirected, unweighted graph. Here, the random walk picks each step a neighbor chosen uniformly at random and moves to that neighbor. Hence, the transition matrix is P = D−1 · A, 1 where D is the diagonal matrix with Di;i = deg(i) . The next lemma shows how to compute the probability for multiple steps. 1 Lecture 5: Random Walks and Markov Chain 2 Lemma 5.2. For any two states x; y 2 Ω and any round t 2 N, t Pr [ Xt = y j X0 = x ] = Px;y; where P0 is the identity matrix. Proof. Let µ0 be the (row-)unit-vector which has 1 at component x and 0 otherwise (the starting distribution). Then let µt be the distribution of the random walk at step t. Clearly, 1 X 0 µy = 1 · Px;y = µz · Pz;y; z2Ω and therefore, µ1 = µ0 · P; and more generally, µt = µt−1 · P = µ0 · Pt: Let us now return to the above example and see how the powers of P look like. 0 0 1=2 1=2 0 0 1 B1=3 0 1=3 1=3 0 C B C P = B1=4 1=4 0 1=4 1=4C B C @ 0 1=2 0 0 1=2A 0 0 0 1 0 00:17 0:24 0:32 0:17 0:091 B0:16 0:25 0:34 0:16 0:08C 10 B C P = B0:16 0:26 0:35 0:16 0:08C B C @0:17 0:24 0:32 0:17 0:09A 0:18 0:24 0:31 0:18 0:09 01=6 1=4 1=3 1=6 1=121 B1=6 1=4 1=3 1=6 1=12C 1 B C P = B1=6 1=4 1=3 1=6 1=12C B C @1=6 1=4 1=3 1=6 1=12A 1=6 1=4 1=3 1=6 1=12 Another important special case of a Markov chain is the random walk on edge-weighted graphs. Here, every (undirected) edge fx; yg 2 E has a non-negative weight wfx;yg. The transition matrix P is defined as wfx;yg Pfx;yg = ; wx where X wx = wfx;zg: z2N(x) Definition 5.3. A distribution π is called stationary, if πP = π. A distribution π is (time)-reversible w.r.t. P, if it satisfies for all x; y 2 Ω, πxPx;y = πyPy;x: Lemma 5.4. Let π be a probability distribution that is time-reversible w.r.t. P. Then π is a stationary distribution for P. Lecture 5: Random Walks and Markov Chain 3 Proof. Summing over all states y 2 Ω yields X X πxPx;y = πyPy;x = πy: x2Ω x2Ω Hence, if π is time-reversible w.r.t. P, then once the distribution π is attained, the chain moves with the same frequency from x to y than from y to x. Random walks on graphs and random walks on edge-weighted graphs are always reversible. (A simple example for a non-reversible Markov chain is a Markov chain for which there are two states with Px;y > 0 but Px;y = 0.) Let us now compute the stationary distribution for three important examples of Markov chains: deg(x) • For a random walk on an (unweighted) graph, πx = 2jEj : X deg(y) X deg(y) 1 deg(x) π = · P = · = : x 2jEj y;x 2jEj deg(y) 2jEj y2N(x) y2N(x) An even more easier way to verify this is to use Lemma 5.4. • If the transition matrix P is symmetric, then π = (1=n; : : : ; 1=n) is uniform. • For a random walk with non-negative edge-weights wfu;vg; fu; vg 2 E, let X wG = wx: x2V Then the Markov chain is reversible with πx = wx=wG (and hence π is the stationary distribution): wx wfx;yg wfx;yg wfy;xg wy wfy;xg πxPx;y = · = = = · = πyPy;x: wG wx wG wG wG wy Lecture 5: Random Walks and Markov Chain 4 1=8 5=8 rainy sunny 1 1=4 5 rainy sunny 2=3 1=4 1=2 3=4 1=3 2 3 foggy cloudy 1=3 foggy cloudy 2 1 1=6 Figure 2: The Markov chain representing the weather in Saarbr¨ucken. The transition matrix for the weather chain (the states are ordered R; F; C; S). 05=8 1=4 0 1=81 B2=3 0 1=3 0 C P = B C @ 0 1=6 1=3 1=2A 1=4 0 3=4 0 By using the above formula for edge-weighted random walks, we can compute the stationary distribution: π = (8=21; 3=21; 6=21; 4=21) (note that 21 is the sum of all weights of the vertices, where every edge weight is counted twice, but every loop weight is counted once) t The naive approach to compute P numerically involves ≈ log2(t) matrix multiplications (fastest algorithm O(n2:376)). Using the reversibility, we now show how to do it using only a constant number of matrix multiplications (assuming that we can efficiently compute the eigenvectors { for very small t it might be actually more efficient to compute Pt directly). t Moreover, the approach will give us a precise formula for any entry Px;y as a function of t. Since the matrix P is reversible w.r.t. to π, it follows that the matrix 1=2 −1=2 Nx;y := πx Px;yπy ; is symmetric. Let us rewrite this using diagonal matrices: 1=2 −1=2 N = Dπ PDπ : Hence the matrix N can be diagonalized, i.e., there is an orthogonal matrix R consisting of the eigenvalues of N so that RNR−1 = D, where D is the diagonal matrix with eigenvalues λ1 > λ2 > ··· > λn of N. Therefore, we can compute powers of the matrix P in the following way: t t −1=2 +1=2 P = Dπ NDπ t −1=2 −1 +1=2 = Dπ R DRDπ −1=2 −1 t +1=2 1 = Dπ R · D · Dπ R Definition 5.5. A Markov chain is called aperiodic if for all states x 2 Ω: t gcd( t 2 N: Px;x > 0 ) = 1: Lecture 5: Random Walks and Markov Chain 5 0 0 1 0 0 1 B 0 0 1 0 C P = B C @1=2 0 0 1=2A 0 1 0 0 0 0 0 1 0 1 2 B1=2 0 0 1=2C P = B C @ 0 1 0 0 A 0 0 1 0 01=2 0 0 1=21 3 B 0 1 0 0 C P = B C @ 0 0 1 0 A 1=2 0 0 1=2 0 0 1 0 0 1 4 1 B 0 0 1 0 C P = P = B C @1=2 0 0 1=2A 0 1 0 0 1 4 1=2 1=2 2 3 t Figure 3: Example of a Markov chain with period 3. The set ft 2 N: P1;1 > 0g contains only multiples of 3. Definition 5.6. A Markov chain is irreducible if for any two states x; y 2 Ω there exists an t integer t such that Px;y > 0. Intuitively, a Markov chain is called irreducible if it is connected viewed as a graph. The following Markov chain corresponds to a random walk on the (unconnected) graph which consists of two disjoint cycles of length 2. Hence the transition matrix P below is not irreducible. 00 1 0 01 B1 0 0 0C P = B C @0 0 0 1A 0 0 1 0 Note that P has two stationary distributions: π = (1=2; 1=2; 0; 0) and π = (0; 0; 1=2; 1=2). Theorem 5.7 (Fundamental Theorem for Markov Chains). If a Markov chain is aperiodic and ireducible, then it holds for all states x; y 2 Ω that: t • there exists an integer T = T (x; y) 2 N so that for all t > T , Px;y > 0.
Recommended publications
  • Superprocesses and Mckean-Vlasov Equations with Creation of Mass

    Superprocesses and Mckean-Vlasov Equations with Creation of Mass

    Sup erpro cesses and McKean-Vlasov equations with creation of mass L. Overb eck Department of Statistics, University of California, Berkeley, 367, Evans Hall Berkeley, CA 94720, y U.S.A. Abstract Weak solutions of McKean-Vlasov equations with creation of mass are given in terms of sup erpro cesses. The solutions can b e approxi- mated by a sequence of non-interacting sup erpro cesses or by the mean- eld of multityp e sup erpro cesses with mean- eld interaction. The lat- ter approximation is asso ciated with a propagation of chaos statement for weakly interacting multityp e sup erpro cesses. Running title: Sup erpro cesses and McKean-Vlasov equations . 1 Intro duction Sup erpro cesses are useful in solving nonlinear partial di erential equation of 1+ the typ e f = f , 2 0; 1], cf. [Dy]. Wenowchange the p oint of view and showhowtheyprovide sto chastic solutions of nonlinear partial di erential Supp orted byanFellowship of the Deutsche Forschungsgemeinschaft. y On leave from the Universitat Bonn, Institut fur Angewandte Mathematik, Wegelerstr. 6, 53115 Bonn, Germany. 1 equation of McKean-Vlasovtyp e, i.e. wewant to nd weak solutions of d d 2 X X @ @ @ + d x; + bx; : 1.1 = a x; t i t t t t t ij t @t @x @x @x i j i i=1 i;j =1 d Aweak solution = 2 C [0;T];MIR satis es s Z 2 t X X @ @ a f = f + f + d f + b f ds: s ij s t 0 i s s @x @x @x 0 i j i Equation 1.1 generalizes McKean-Vlasov equations of twodi erenttyp es.
  • A Class of Measure-Valued Markov Chains and Bayesian Nonparametrics

    A Class of Measure-Valued Markov Chains and Bayesian Nonparametrics

    Bernoulli 18(3), 2012, 1002–1030 DOI: 10.3150/11-BEJ356 A class of measure-valued Markov chains and Bayesian nonparametrics STEFANO FAVARO1, ALESSANDRA GUGLIELMI2 and STEPHEN G. WALKER3 1Universit`adi Torino and Collegio Carlo Alberto, Dipartimento di Statistica e Matematica Ap- plicata, Corso Unione Sovietica 218/bis, 10134 Torino, Italy. E-mail: [email protected] 2Politecnico di Milano, Dipartimento di Matematica, P.zza Leonardo da Vinci 32, 20133 Milano, Italy. E-mail: [email protected] 3University of Kent, Institute of Mathematics, Statistics and Actuarial Science, Canterbury CT27NZ, UK. E-mail: [email protected] Measure-valued Markov chains have raised interest in Bayesian nonparametrics since the seminal paper by (Math. Proc. Cambridge Philos. Soc. 105 (1989) 579–585) where a Markov chain having the law of the Dirichlet process as unique invariant measure has been introduced. In the present paper, we propose and investigate a new class of measure-valued Markov chains defined via exchangeable sequences of random variables. Asymptotic properties for this new class are derived and applications related to Bayesian nonparametric mixture modeling, and to a generalization of the Markov chain proposed by (Math. Proc. Cambridge Philos. Soc. 105 (1989) 579–585), are discussed. These results and their applications highlight once again the interplay between Bayesian nonparametrics and the theory of measure-valued Markov chains. Keywords: Bayesian nonparametrics; Dirichlet process; exchangeable sequences; linear functionals
  • Markov Process Duality

    Markov Process Duality

    Markov Process Duality Jan M. Swart Luminy, October 23 and 24, 2013 Jan M. Swart Markov Process Duality Markov Chains S finite set. RS space of functions f : S R. ! S For probability kernel P = (P(x; y))x;y2S and f R define left and right multiplication as 2 X X Pf (x) := P(x; y)f (y) and fP(x) := f (y)P(y; x): y y (I do not distinguish row and column vectors.) Def Chain X = (Xk )k≥0 of S-valued r.v.'s is Markov chain with transition kernel P and state space S if S E f (Xk+1) (X0;:::; Xk ) = Pf (Xk ) a:s: (f R ) 2 P (X0;:::; Xk ) = (x0;:::; xk ) , = P[X0 = x0]P(x0; x1) P(xk−1; xk ): ··· µ µ µ Write P ; E for process with initial law µ = P [X0 ]. x δx x 2 · P := P with δx (y) := 1fx=yg. E similar. Jan M. Swart Markov Process Duality Markov Chains Set k µ k x µk := µP (x) = P [Xk = x] and fk := P f (x) = E [f (Xk )]: Then the forward and backward equations read µk+1 = µk P and fk+1 = Pfk : In particular π invariant law iff πP = π. Function h harmonic iff Ph = h h(Xk ) martingale. , Jan M. Swart Markov Process Duality Random mapping representation (Zk )k≥1 i.i.d. with common law ν, take values in (E; ). φ : S E S measurable E × ! P(x; y) = P[φ(x; Z1) = y]: Random mapping representation (E; ; ν; φ) always exists, highly non-unique.
  • Lecture 19: Polymer Models: Persistence and Self-Avoidance

    Lecture 19: Polymer Models: Persistence and Self-Avoidance

    Lecture 19: Polymer Models: Persistence and Self-Avoidance Scribe: Allison Ferguson (and Martin Z. Bazant) Martin Fisher School of Physics, Brandeis University April 14, 2005 Polymers are high molecular weight molecules formed by combining a large number of smaller molecules (monomers) in a regular pattern. Polymers in solution (i.e. uncrystallized) can be characterized as long flexible chains, and thus may be well described by random walk models of various complexity. 1 Simple Random Walk (IID Steps) Consider a Rayleigh random walk (i.e. a fixed step size a with a random angle θ). Recall (from Lecture 1) that the probability distribution function (PDF) of finding the walker at position R~ after N steps is given by −3R2 2 ~ e 2a N PN (R) ∼ d (1) (2πa2N/d) 2 Define RN = |R~| as the end-to-end distance of the polymer. Then from previous results we q √ 2 2 ¯ 2 know hRN i = Na and RN = hRN i = a N. We can define an additional quantity, the radius of gyration GN as the radius from the centre of mass (assuming the polymer has an approximately 1 2 spherical shape ). Hughes [1] gives an expression for this quantity in terms of hRN i as follows: 1 1 hG2 i = 1 − hR2 i (2) N 6 N 2 N As an example of the type of prediction which can be made using this type of simple model, consider the following experiment: Pull on both ends of a polymer and measure the force f required to stretch it out2. Note that the force will be given by f = −(∂F/∂R) where F = U − TS is the Helmholtz free energy (T is the temperature, U is the internal energy, and S is the entropy).
  • 1 Introduction Branching Mechanism in a Superprocess from a Branching

    1 Introduction Branching Mechanism in a Superprocess from a Branching

    数理解析研究所講究録 1157 巻 2000 年 1-16 1 An example of random snakes by Le Gall and its applications 渡辺信三 Shinzo Watanabe, Kyoto University 1 Introduction The notion of random snakes has been introduced by Le Gall ([Le 1], [Le 2]) to construct a class of measure-valued branching processes, called superprocesses or continuous state branching processes ([Da], [Dy]). A main idea is to produce the branching mechanism in a superprocess from a branching tree embedded in excur- sions at each different level of a Brownian sample path. There is no clear notion of particles in a superprocess; it is something like a cloud or mist. Nevertheless, a random snake could provide us with a clear picture of historical or genealogical developments of”particles” in a superprocess. ” : In this note, we give a sample pathwise construction of a random snake in the case when the underlying Markov process is a Markov chain on a tree. A simplest case has been discussed in [War 1] and [Wat 2]. The construction can be reduced to this case locally and we need to consider a recurrence family of stochastic differential equations for reflecting Brownian motions with sticky boundaries. A special case has been already discussed by J. Warren [War 2] with an application to a coalescing stochastic flow of piece-wise linear transformations in connection with a non-white or non-Gaussian predictable noise in the sense of B. Tsirelson. 2 Brownian snakes Throughout this section, let $\xi=\{\xi(t), P_{x}\}$ be a Hunt Markov process on a locally compact separable metric space $S$ endowed with a metric $d_{S}(\cdot, *)$ .
  • 8 Convergence of Loop-Erased Random Walk to SLE2 8.1 Comments on the Paper

    8 Convergence of Loop-Erased Random Walk to SLE2 8.1 Comments on the Paper

    8 Convergence of loop-erased random walk to SLE2 8.1 Comments on the paper The proof that the loop-erased random walk converges to SLE2 is in the paper \Con- formal invariance of planar loop-erased random walks and uniform spanning trees" by Lawler, Schramm and Werner. (arxiv.org: math.PR/0112234). This section contains some comments to give you some hope of being able to actually make sense of this paper. In the first sections of the paper, δ is used to denote the lattice spacing. The main theorem is that as δ 0, the loop erased random walk converges to SLE2. HOWEVER, beginning in section !2.2 the lattice spacing is 1, and in section 3.2 the notation δ reappears but means something completely different. (δ also appears in the proof of lemma 2.1 in section 2.1, but its meaning here is restricted to this proof. The δ in this proof has nothing to do with any δ's outside the proof.) After the statement of the main theorem, there is no mention of the lattice spacing going to zero. This is hidden in conditions on the \inner radius." For a domain D containing 0 the inner radius is rad0(D) = inf z : z = D (364) fj j 2 g Now fix a domain D containing the origin. Let be the conformal map of D onto U. We want to show that the image under of a LERW on a lattice with spacing δ converges to SLE2 in the unit disc. In the paper they do this in a slightly different way.
  • 1 Markov Chain Notation for a Continuous State Space

    1 Markov Chain Notation for a Continuous State Space

    The Metropolis-Hastings Algorithm 1 Markov Chain Notation for a Continuous State Space A sequence of random variables X0;X1;X2;:::, is a Markov chain on a continuous state space if... ... where it goes depends on where is is but not where is was. I'd really just prefer that you have the “flavor" here. The previous Markov property equation that we had P (Xn+1 = jjXn = i; Xn−1 = in−1;:::;X0 = i0) = P (Xn+1 = jjXn = i) is uninteresting now since both of those probabilities are zero when the state space is con- tinuous. It would be better to say that, for any A ⊆ S, where S is the state space, we have P (Xn+1 2 AjXn = i; Xn−1 = in−1;:::;X0 = i0) = P (Xn+1 2 AjXn = i): Note that it is okay to have equalities on the right side of the conditional line. Once these continuous random variables have been observed, they are fixed and nailed down to discrete values. 1.1 Transition Densities The continuous state analog of the one-step transition probability pij is the one-step tran- sition density. We will denote this as p(x; y): This is not the probability that the chain makes a move from state x to state y. Instead, it is a probability density function in y which describes a curve under which area represents probability. x can be thought of as a parameter of this density. For example, given a Markov chain is currently in state x, the next value y might be drawn from a normal distribution centered at x.
  • Simulation of Markov Chains

    Simulation of Markov Chains

    Copyright c 2007 by Karl Sigman 1 Simulating Markov chains Many stochastic processes used for the modeling of financial assets and other systems in engi- neering are Markovian, and this makes it relatively easy to simulate from them. Here we present a brief introduction to the simulation of Markov chains. Our emphasis is on discrete-state chains both in discrete and continuous time, but some examples with a general state space will be discussed too. 1.1 Definition of a Markov chain We shall assume that the state space S of our Markov chain is S = ZZ= f:::; −2; −1; 0; 1; 2;:::g, the integers, or a proper subset of the integers. Typical examples are S = IN = f0; 1; 2 :::g, the non-negative integers, or S = f0; 1; 2 : : : ; ag, or S = {−b; : : : ; 0; 1; 2 : : : ; ag for some integers a; b > 0, in which case the state space is finite. Definition 1.1 A stochastic process fXn : n ≥ 0g is called a Markov chain if for all times n ≥ 0 and all states i0; : : : ; i; j 2 S, P (Xn+1 = jjXn = i; Xn−1 = in−1;:::;X0 = i0) = P (Xn+1 = jjXn = i) (1) = Pij: Pij denotes the probability that the chain, whenever in state i, moves next (one unit of time later) into state j, and is referred to as a one-step transition probability. The square matrix P = (Pij); i; j 2 S; is called the one-step transition matrix, and since when leaving state i the chain must move to one of the states j 2 S, each row sums to one (e.g., forms a probability distribution): For each i X Pij = 1: j2S We are assuming that the transition probabilities do not depend on the time n, and so, in particular, using n = 0 in (1) yields Pij = P (X1 = jjX0 = i): (Formally we are considering only time homogenous MC's meaning that their transition prob- abilities are time-homogenous (time stationary).) The defining property (1) can be described in words as the future is independent of the past given the present state.
  • Chapter 8: Markov Chains

    Chapter 8: Markov Chains

    149 Chapter 8: Markov Chains 8.1 Introduction So far, we have examined several stochastic processes using transition diagrams and First-Step Analysis. The processes can be written as {X0, X1, X2,...}, where Xt is the state at time t. A.A.Markov On the transition diagram, Xt corresponds to 1856-1922 which box we are in at step t. In the Gambler’s Ruin (Section 2.7), Xt is the amount of money the gambler possesses after toss t. In the model for gene spread (Section 3.7), Xt is the number of animals possessing the harmful allele A in generation t. The processes that we have looked at via the transition diagram have a crucial property in common: Xt+1 depends only on Xt. It does not depend upon X0, X1,...,Xt−1. Processes like this are called Markov Chains. Example: Random Walk (see Chapter 4) none of these steps matter for time t+1 ? time t+1 time t ? In a Markov chain, the future depends only upon the present: NOT upon the past. 150 .. ...... .. .. ...... .. ....... ....... ....... ....... ... .... .... .... .. .. .. .. ... ... ... ... ....... ..... ..... ....... ........4................. ......7.................. The text-book image ...... ... ... ...... .... ... ... .... ... ... ... ... ... ... ... ... ... ... ... .... ... ... ... ... of a Markov chain has ... ... ... ... 1 ... .... ... ... ... ... ... ... ... .1.. 1... 1.. 3... ... ... ... ... ... ... ... ... 2 ... ... ... a flea hopping about at ... ..... ..... ... ...... ... ....... ...... ...... ....... ... ...... ......... ......... .. ........... ........ ......... ........... ........
  • Markov Random Fields and Stochastic Image Models

    Markov Random Fields and Stochastic Image Models

    Markov Random Fields and Stochastic Image Models Charles A. Bouman School of Electrical and Computer Engineering Purdue University Phone: (317) 494-0340 Fax: (317) 494-3358 email [email protected] Available from: http://dynamo.ecn.purdue.edu/»bouman/ Tutorial Presented at: 1995 IEEE International Conference on Image Processing 23-26 October 1995 Washington, D.C. Special thanks to: Ken Sauer Suhail Saquib Department of Electrical School of Electrical and Computer Engineering Engineering University of Notre Dame Purdue University 1 Overview of Topics 1. Introduction (b) Non-Gaussian MRF's 2. The Bayesian Approach i. Quadratic functions ii. Non-Convex functions 3. Discrete Models iii. Continuous MAP estimation (a) Markov Chains iv. Convex functions (b) Markov Random Fields (MRF) (c) Parameter Estimation (c) Simulation i. Estimation of σ (d) Parameter estimation ii. Estimation of T and p parameters 4. Application of MRF's to Segmentation 6. Application to Tomography (a) The Model (a) Tomographic system and data models (b) Bayesian Estimation (b) MAP Optimization (c) MAP Optimization (c) Parameter estimation (d) Parameter Estimation 7. Multiscale Stochastic Models (e) Other Approaches (a) Continuous models 5. Continuous Models (b) Discrete models (a) Gaussian Random Process Models 8. High Level Image Models i. Autoregressive (AR) models ii. Simultaneous AR (SAR) models iii. Gaussian MRF's iv. Generalization to 2-D 2 References in Statistical Image Modeling 1. Overview references [100, 89, 50, 54, 162, 4, 44] 4. Simulation and Stochastic Optimization Methods [118, 80, 129, 100, 68, 141, 61, 76, 62, 63] 2. Type of Random Field Model 5. Computational Methods used with MRF Models (a) Discrete Models i.
  • Random Walk in Random Scenery (RWRS)

    Random Walk in Random Scenery (RWRS)

    IMS Lecture Notes–Monograph Series Dynamics & Stochastics Vol. 48 (2006) 53–65 c Institute of Mathematical Statistics, 2006 DOI: 10.1214/074921706000000077 Random walk in random scenery: A survey of some recent results Frank den Hollander1,2,* and Jeffrey E. Steif 3,† Leiden University & EURANDOM and Chalmers University of Technology Abstract. In this paper we give a survey of some recent results for random walk in random scenery (RWRS). On Zd, d ≥ 1, we are given a random walk with i.i.d. increments and a random scenery with i.i.d. components. The walk and the scenery are assumed to be independent. RWRS is the random process where time is indexed by Z, and at each unit of time both the step taken by the walk and the scenery value at the site that is visited are registered. We collect various results that classify the ergodic behavior of RWRS in terms of the characteristics of the underlying random walk (and discuss extensions to stationary walk increments and stationary scenery components as well). We describe a number of results for scenery reconstruction and close by listing some open questions. 1. Introduction Random walk in random scenery is a family of stationary random processes ex- hibiting amazingly rich behavior. We will survey some of the results that have been obtained in recent years and list some open questions. Mike Keane has made funda- mental contributions to this topic. As close colleagues it has been a great pleasure to work with him. We begin by defining the object of our study. Fix an integer d ≥ 1.
  • Particle Filtering Within Adaptive Metropolis Hastings Sampling

    Particle Filtering Within Adaptive Metropolis Hastings Sampling

    Particle filtering within adaptive Metropolis-Hastings sampling Ralph S. Silva Paolo Giordani School of Economics Research Department University of New South Wales Sveriges Riksbank [email protected] [email protected] Robert Kohn∗ Michael K. Pitt School of Economics Economics Department University of New South Wales University of Warwick [email protected] [email protected] October 29, 2018 Abstract We show that it is feasible to carry out exact Bayesian inference for non-Gaussian state space models using an adaptive Metropolis-Hastings sampling scheme with the likelihood approximated by the particle filter. Furthermore, an adaptive independent Metropolis Hastings sampler based on a mixture of normals proposal is computation- ally much more efficient than an adaptive random walk Metropolis proposal because the cost of constructing a good adaptive proposal is negligible compared to the cost of approximating the likelihood. Independent Metropolis-Hastings proposals are also arXiv:0911.0230v1 [stat.CO] 2 Nov 2009 attractive because they are easy to run in parallel on multiple processors. We also show that when the particle filter is used, the marginal likelihood of any model is obtained in an efficient and unbiased manner, making model comparison straightforward. Keywords: Auxiliary variables; Bayesian inference; Bridge sampling; Marginal likeli- hood. ∗Corresponding author. 1 Introduction We show that it is feasible to carry out exact Bayesian inference on the parameters of a general state space model by using the particle filter to approximate the likelihood and adaptive Metropolis-Hastings sampling to generate unknown parameters. The state space model can be quite general, but we assume that the observation equation can be evaluated analytically and that it is possible to generate from the state transition equation.