Markov Chains and Mixing Times Second Edition David A. Levin Yuval Peres With contributions by Elizabeth L. Wilmer A MERICAN MATHEMATICAL SOCIETY 10.1090/mbk/107 Markov Chains and Mixing Times Second Edition David A. Levin University of Oregon Yuval Peres Microsoft Research With contributions by Elizabeth L. Wilmer With a chapter on “Coupling from the Past” by James G. Propp and David B. Wilson A MERICAN MATHEMATICAL SOCIETY Providence, Rhode Island 2010 Mathematics Subject Classification. Primary 60J10, 60J27, 60B15, 60C05, 65C05, 60K35, 68W20, 68U20, 82C22. FRONT COVER: The figure on the bottom left of the front cover, courtesy of David B. Wilson, is a uniformly random lozenge tiling of a hexagon (see Section 25.2). The figure on the bottom center, also from David B. Wilson, is a random sample of an Ising model at its critical temperature (see Sections 3.3.5 and 25.2) with mixed boundary conditions. The figure on the bottom right, courtesy of Eyal Lubetzky, is a portion of an expander graph (see Section 13.6). For additional information and updates on this book, visit www.ams.org/bookpages/mbk-107 Library of Congress Cataloging-in-Publication Data Names: Levin, David Asher, 1971- | Peres, Y. (Yuval) | Wilmer, Elizabeth L. (Elizabeth Lee), 1970- | Propp, James, 1960- | Wilson, David B. (David Bruce) Title: Markov chains and mixing times / David A. Levin, Yuval Peres ; with contributions by Elizabeth L. Wilmer. Description: Second edition. | Providence, Rhode Island : American Mathematical Society, [2017] | “With a chapter on Coupling from the past, by James G. Propp and David B. Wilson.” | Includes bibliographical references and indexes. Identifiers: LCCN 2017017451 | ISBN 9781470429621 (alk. paper) Subjects: LCSH: Markov processes–Textbooks. | Distribution (Probability theory)–Textbooks. | AMS: Probability theory and stochastic processes – Markov processes – Markov chains (discrete-time Markov processes on discrete state spaces). msc | Probability theory and sto- chastic processes – Markov processes – Continuous-time Markov processes on discrete state spaces. msc | Probability theory and stochastic processes – Probability theory on algebraic and topological structures – Probability measures on groups or semigroups, Fourier transforms, factorization. msc | Probability theory and stochastic processes – Combinatorial probability – Combinatorial probability. msc | Numerical analysis – Probabilistic methods, simulation and stochastic differential equations – Monte Carlo methods. msc | Probability theory and stochastic processes – Special processes – Interacting random processes; statistical mechan- ics type models; percolation theory. msc | Computer science – Algorithms – Randomized algorithms. msc | Computer science – Computing methodologies and applications – Simula- tion. msc | Statistical mechanics, structure of matter – Time-dependent statistical mechanics (dynamic and nonequilibrium) – Interacting particle systems. msc Classification: LCC QA274.7 .L48 2017 | DDC 519.2/33–dc23 LC record available at https://lccn. loc.gov/2017017451 Copying and reprinting. Individual readers of this publication, and nonprofit libraries acting for them, are permitted to make fair use of the material, such as to copy select pages for use in teaching or research. Permission is granted to quote brief passages from this publication in reviews, provided the customary acknowledgment of the source is given. Republication, systematic copying, or multiple reproduction of any material in this publication is permitted only under license from the American Mathematical Society. Permissions to reuse portions of AMS publication content are handled by Copyright Clearance Center’s RightsLink service. For more information, please visit: http://www.ams.org/rightslink. Send requests for translation rights and licensed reprints to [email protected]. Excluded from these provisions is material for which the author holds copyright. In such cases, requests for permission to reuse or reprint material should be addressed directly to the author(s). Copyright ownership is indicated on the copyright page, or on the lower right-hand corner of the first page of each article within proceedings volumes. Second edition c 2017 by the authors. All rights reserved. First edition c 2009 by the authors. All rights reserved. Printed in the United States of America. ∞ The paper used in this book is acid-free and falls within the guidelines established to ensure permanence and durability. Visit the AMS home page at http://www.ams.org/ 10987654321 171615141312 Contents Preface ix Preface to the Second Edition ix Preface to the First Edition ix Overview xi For the Reader xii For the Instructor xiii For the Expert xiv Acknowledgements xvi Part I: Basic Methods and Examples 1 Chapter 1. Introduction to Finite Markov Chains 2 1.1. Markov Chains 2 1.2. Random Mapping Representation 5 1.3. Irreducibility and Aperiodicity 7 1.4. Random Walks on Graphs 8 1.5. Stationary Distributions 9 1.6. Reversibility and Time Reversals 13 1.7. Classifying the States of a Markov Chain* 15 Exercises 17 Notes 19 Chapter 2. Classical (and Useful) Markov Chains 21 2.1. Gambler’s Ruin 21 2.2. Coupon Collecting 22 2.3. The Hypercube and the Ehrenfest Urn Model 23 2.4. The P´olya Urn Model 25 2.5. Birth-and-Death Chains 26 2.6. Random Walks on Groups 27 2.7. Random Walks on Z and Reflection Principles 30 Exercises 34 Notes 35 Chapter 3. Markov Chain Monte Carlo: Metropolis and Glauber Chains 38 3.1. Introduction 38 3.2. Metropolis Chains 38 3.3. Glauber Dynamics 41 Exercises 45 Notes 45 iii iv CONTENTS Chapter 4. Introduction to Markov Chain Mixing 47 4.1. Total Variation Distance 47 4.2. Coupling and Total Variation Distance 49 4.3. The Convergence Theorem 52 4.4. Standardizing Distance from Stationarity 53 4.5. Mixing Time 54 4.6. Mixing and Time Reversal 55 4.7. p Distance and Mixing 56 Exercises 57 Notes 58 Chapter 5. Coupling 60 5.1. Definition 60 5.2. Bounding Total Variation Distance 61 5.3. Examples 62 5.4. Grand Couplings 69 Exercises 73 Notes 73 Chapter 6. Strong Stationary Times 75 6.1. Top-to-Random Shuffle 75 6.2. Markov Chains with Filtrations 76 6.3. Stationary Times 77 6.4. Strong Stationary Times and Bounding Distance 78 6.5. Examples 81 6.6. Stationary Times and Ces`aro Mixing Time 83 6.7. Optimal Strong Stationary Times* 84 Exercises 85 Notes 86 Chapter 7. Lower Bounds on Mixing Times 87 7.1. Counting and Diameter Bounds 87 7.2. Bottleneck Ratio 88 7.3. Distinguishing Statistics 91 7.4. Examples 95 Exercises 97 Notes 98 Chapter 8. The Symmetric Group and Shuffling Cards 99 8.1. The Symmetric Group 99 8.2. Random Transpositions 101 8.3. Riffle Shuffles 106 Exercises 109 Notes 112 Chapter 9. Random Walks on Networks 115 9.1. Networks and Reversible Markov Chains 115 9.2. Harmonic Functions 116 9.3. Voltages and Current Flows 117 CONTENTS v 9.4. Effective Resistance 118 9.5. Escape Probabilities on a Square 123 Exercises 125 Notes 126 Chapter 10. Hitting Times 127 10.1. Definition 127 10.2. Random Target Times 128 10.3. Commute Time 130 10.4. Hitting Times on Trees 133 10.5. Hitting Times for Eulerian Graphs 135 10.6. Hitting Times for the Torus 136 10.7. Bounding Mixing Times via Hitting Times 139 10.8. Mixing for the Walk on Two Glued Graphs 143 Exercises 145 Notes 147 Chapter 11. Cover Times 149 11.1. Definitions 149 11.2. The Matthews Method 149 11.3. Applications of the Matthews Method 151 11.4. Spanning Tree Bound for Cover Time 153 11.5. Waiting for All Patterns in Coin Tossing 155 Exercises 157 Notes 157 Chapter 12. Eigenvalues 160 12.1. The Spectral Representation of a Reversible Transition Matrix 160 12.2. The Relaxation Time 162 12.3. Eigenvalues and Eigenfunctions of Some Simple Random Walks 164 12.4. Product Chains 168 12.5. Spectral Formula for the Target Time 171 12.6. An 2 Bound 171 12.7. Time Averages 172 Exercises 176 Notes 177 Part II: The Plot Thickens 179 Chapter 13. Eigenfunctions and Comparison of Chains 180 13.1. Bounds on Spectral Gap via Contractions 180 13.2. The Dirichlet Form and the Bottleneck Ratio 181 13.3. Simple Comparison of Markov Chains 185 13.4. The Path Method 188 13.5. Wilson’s Method for Lower Bounds 193 13.6. Expander Graphs* 196 Exercises 198 Notes 199 vi CONTENTS Chapter 14. The Transportation Metric and Path Coupling 201 14.1. The Transportation Metric 201 14.2. Path Coupling 203 14.3. Rapid Mixing for Colorings 207 14.4. Approximate Counting 209 Exercises 213 Notes 214 Chapter 15. The Ising Model 216 15.1. Fast Mixing at High Temperature 216 15.2. The Complete Graph 219 15.3. The Cycle 220 15.4. The Tree 221 15.5. Block Dynamics 224 15.6. Lower Bound for the Ising Model on Square* 227 Exercises 229 Notes 230 Chapter 16. From Shuffling Cards to Shuffling Genes 233 16.1. Random Adjacent Transpositions 233 16.2. Shuffling Genes 237 Exercises 241 Notes 242 Chapter 17. Martingales and Evolving Sets 244 17.1. Definition and Examples 244 17.2. Optional Stopping Theorem 245 17.3. Applications 247 17.4. Evolving Sets 250 17.5. A General Bound on Return Probabilities 255 17.6. Harmonic Functions and the Doob h-Transform 257 17.7. Strong Stationary Times from Evolving Sets 258 Exercises 260 Notes 260 Chapter 18. The Cutoff Phenomenon 262 18.1. Definition 262 18.2. Examples of Cutoff 263 18.3. A Necessary Condition for Cutoff 268 18.4. Separation Cutoff 269 Exercises 270 Notes 270 Chapter 19. Lamplighter Walks 273 19.1. Introduction 273 19.2. Relaxation Time Bounds 274 19.3. Mixing Time Bounds 276 19.4. Examples 278 Exercises 278 Notes 279 CONTENTS vii Chapter 20. Continuous-Time Chains* 281 20.1. Definitions 281 20.2. Continuous-Time Mixing 283 20.3.