Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications Random Walks: Basic Concepts and Applications Laura Ricci Dipartimento di Informatica 25 luglio 2012 PhD in Computer Science Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications Outline 1 Basic Concepts 2 Natural Random Walk 3 Random Walks Characterization 4 Metropolis Hastings 5 Applications Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications Random Walk: Basic Concepts A Random Walk in synthesis: given an indirected graph and a starting point, select a neighbour at random move to the selected neighbour and repeat the same process till a termination condition is verified the random sequence of points selected in this way is a random walk of the graph Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications A Random Walk Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications A Random Walk Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications A Random Walk Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications A Random Walk Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications A Random Walk Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications A Random Walk Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications A Random Walk Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications A Random Walk Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications The Natural Random Walk Natural Random Walk Given an undirected graph G = (V; E), with n =j V j and m =j E j, a natural random walk is a stochastic process that starts from a given vertex, and then selects one of its neighbors uniformly at random to visit. The natural random walk is defined by the following transition matrix P : ( 1 ; y is a neighbour of x P (x; y) = degree(x) 0; otherwise where x is the out degree of the node x Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications Natural Random Walk note that we assume undirected graph: i.e. if the walker can go from i to j, it can also go from j to i this does not imply that the probability of the transition ij is the same of the transition ji it depends on the degree distribution of the nodes Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications Natural Random Walk: Stationary Distribution Stationary Distribution Given an irreducible and aperiodic graph with a set of nodes N and a set E of edges, the probability of being at a particular node v converges to the stationary distribution RW deg(v) π (v) = 2×|Ej = / degree(v) if we run the random walk for sufficiently long, then we get arbitrarily close to achieving the distribution πRW (v) put in a different way: the fraction of time spent at a node is directly proportional to the degree of node.... the probability of sampling a node depends on its degree the natural random walk is inherently biased towards node with higher degree Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications Natural Random Walk Stationary Distribution Consider a random walk on the house graph below: d1; d2; d3; d4; d5) = (2; 3; 3; 2; 2) so stationary distribution is 2 3 3 2 2 π1; π2; π3; π4; π5 = ( 12 ; 12 ; 12 ; 12 ; 12 ) Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications Natural Random Walk Stationary Distribution A king moves at random on an 8 × 8 chessboard. The number of moves in various locations are as follows: interior tiles: 8 moves edge tiles: 5 moves corner tiles: 3 moves The number of all possible edges(moves on the chessboard) is 420 8 3 5 Therefore, the stationary distribution is ( 420 ; 420 ; 420 ) This gives an idea of the time spent by the king on each kind of tile during the random walk Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications Random Walks and Markov Chains Time Reversible Markov Chain: the probability of the occurrence of a state sequence occurring is equal to the probability of the reversal of that state sequence running time backward does not affect the distribution at all Consider a Markov Chain with state space fs1; : : : ; skg and probability transition matrix P . A stationary distribution π on S is said to be time reversible for the chain if 8i; j 2 f1; : : : kg, we have: π(i) × P (i; j) = π(j) × P (j; i) Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications Time Reversible Markov Chain Time Reversibility interpretation think of π(i) × P (i; j) as the limiting long run fraction of transitions made by the Markov chain that go from state i to state j. time reversibility requires that the long run fraction of i to j transitions is the same as that of the j to i transitions, 8i; j. note that this is a more stringent requirement than stationarity, which equates the long run fraction of transitions that go out of state i to the long run fraction of transitions that go into state i. Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications Natural Random Walk and Markov Chains the natural random walk on a graph is a time reversible Markov chain with respect to its stationary distribution as a matter of fact, for all vi and vj neighbours, d 1 1 π(i) × P (i; j) = i × = d di d d 1 1 π(j) × P (j; i) = j × = d dj d otherwise, if vi and vj are not neighbours π(i) × P (i; j) = π(j) × P (j; i) (1) Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications Random Walks and Markov Chains not all walks on a graph are reversible Markov Chains consider the following walker: at each time step, the walker 3 moves one step clockwise with probability 4 and one step 1 counter clockwise with probability 4 1 1 1 1 π = ( 4 ; 4 ; 4 ; 4 ) is the only stationary distribution Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications Random Walks and Markov Chains The transition graph is the following one: is it sufficient to show that the stationary distribution 1 1 1 1 π = ( 4 ; 4 ; 4 ; 4 ) is not reversible, to conclude that the chain is not reversible 1 3 3 π(1) × P (1; 2) = × = 4 4 16 1 1 1 π(2) × P (2; 1) = × = 4 4 16 Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications Random Walk Metrics Important measures of Random Walk Access or Hitting Time, Hij: expected number of steps before node j is visited, starting from node i. Commute Time: expected number of steps in the random walk starting at i, before node j is visited and then node i is reached again. Cover time expected number of steps to reach every node, starting from a given initial distribution. Graph Cover Time Maximum Cover Times over all Vertexes Mixing Rate measures how fast the random walk converges to the Stationary Distribution (Steady State). Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications Computing Random Walk Metrics: A Warm Up Example the values of the metrics depends on the graph topology the probability fluxes on the graph let us consider the (simple and not realistic) case of complete graph with nodes f0; : : : ; n − 1g each pair of vertices is connected by an edge consider a natural random walk on this graph and compute the access time for a pair of nodes the cover time of the random walk Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications A Warm Up Example: Hitting Time each node has the same number of connections to other nodes so we can consider a generic pair of nodes, for instance node 0 and node 1 and compute H(0; 1), without loss of generality. the probability that, staring from the node 0, we reach node 1 in the t-h step is t−1 n−2 1 n−1 × n−1 so the expected hitting time is: t−1 P1 n−2 1 H(0; 1) = t=1 t × n−1 × n−1 = n − 1 Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications A Warm Up Example: Cover Time The problem of the coverage of a complete graph is closely related to the Coupon Collector Problem since you are eager of cereals you often buy them each box of cereal contains one of n different coupons each coupon is chosen independently and uniformly at random from n ones you cannot collaborate with other people to collect the coupons! when you have collected one of every type of coupon, you win a prize! under these conditions, what is the expected number of box of cereals you have to buy before you win the prize? Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications A Warm Up Example: Cover Time Coupon collection problem is modelled through a geometric distribution Geometric Random Distribution: a sequence of independent trials repeated until the first success each trial succeeds with probability p P(X = n) = (1 − p)n−1p 1 E[X] = p Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications A Warm Example: Cover Time Modelling the coverage problem as a coupon collection problem: the coupons are the vertexes of the graph collecting a coupon corresponds to visiting a new node Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications A Warm Up Example: Cover Time Cover time may be modelled by a sequence of Geometric Variables Let us define a vertex as collected when it has been visited at least a time by the random walk X number of vertexes visited by the random walk before collecting all the vertexes in the graph Xi number of vertexes visited after having collected i − 1 vertexes and before collecting a new vertex.