Modern Discrete Probability An Essential Toolkit Sebastien´ Roch November 20, 2014 Modern Discrete Probability: An Essential Toolkit Manuscript version of November 20, 2014 Copyright c 2014 Sebastien´ Roch Contents Preface vi I Introduction 1 1 Models 2 1.1 Preliminaries ............................ 2 1.1.1 Review of graph theory ................... 2 1.1.2 Review of Markov chain theory .............. 2 1.2 Percolation ............................. 2 1.3 Random graphs ........................... 2 1.4 Markov random fields ........................ 2 1.5 Random walks on networks ..................... 3 1.6 Interacting particles on finite graphs ................ 3 1.7 Other random structures ...................... 3 II Basic tools: moments, martingales, couplings 4 2 Moments and tails 5 2.1 First moment method ........................ 6 2.1.1 The probabilistic method .................. 6 2.1.2 Markov’s inequality .................... 8 2.1.3 . Random permutations: longest increasing subsequence . 9 2.1.4 . Constraint satisfaction: bound on random k-SAT threshold 11 2.1.5 . Percolation on Zd: existence of a phase transition .... 12 2.2 Second moment method ...................... 16 2.2.1 Chebyshev’s and Paley-Zygmund inequalities ....... 17 2.2.2 . Erdos-R¨ enyi´ graphs: small subgraph containment .... 21 2.2.3 . Erdos-R¨ enyi´ graphs: connectivity threshold ....... 25 i 2.2.4 . Percolation on trees: branching number, weighted sec- ond moment method, and application to the critical value . 28 2.3 Chernoff-Cramer´ method ...................... 32 2.3.1 Tail bounds via the moment-generating function ..... 33 2.3.2 . Randomized algorithms: Johnson-Lindenstrauss, "-nets, and application to compressed sensing .......... 37 2.3.3 . Information theory: Shannon’s theorem ......... 44 2.3.4 . Markov chains: Varopoulos-Carne, and a bound on mixing 44 2.3.5 Hoeffding’s and Bennett’s inequalities ........... 50 2.3.6 . Knapsack: probabilistic analysis ............. 53 2.4 Matrix tail bounds .......................... 53 2.4.1 Ahlswede-Winter inequality ................ 54 2.4.2 . Randomized algorithms: low-rank approximations ... 54 3 Stopping times, martingales, and harmonic functions 58 3.1 Background ............................. 58 3.1.1 Stopping times ....................... 58 3.1.2 . Markov chains: exponential tail of hitting times, and some cover time bounds .................. 58 3.1.3 Martingales ......................... 58 3.1.4 . Percolation on trees: critical regime ........... 58 3.2 Concentration for martingales ................... 59 3.2.1 Azuma-Hoeffding inequality ................ 59 3.2.2 Method of bounded differences .............. 60 3.2.3 . Erdos-R¨ enyi´ graphs: exposure martingales, and applica- tion to the chromatic number ................ 64 3.2.4 . Hypercube: concentration of measure .......... 68 3.2.5 . Preferential attachment graphs: degree sequence .... 71 3.3 Electrical networks ......................... 78 3.3.1 Martingales and the Dirichlet problem ........... 79 3.3.2 Basic electrical network theory ............... 82 3.3.3 Bounding the effective resistance ............. 91 3.3.4 . Random walk in a random environment: supercritical percolation clusters ..................... 106 3.3.5 . Uniform spanning trees: Wilson’s method ........ 106 3.3.6 . Ising model on trees: the reconstruction problem .... 113 4 Coupling 118 4.1 Background ............................. 118 4.1.1 . Erdos-R¨ enyi´ graphs: degree sequence .......... 118 ii 4.1.2 . Harmonic functions: lattices and trees .......... 118 4.2 Stochastic domination and correlation inequalities ......... 118 4.2.1 Definitions ......................... 119 4.2.2 . Ising model on Zd: extremal measures .......... 128 4.2.3 . Random walk on trees: speed .............. 131 4.2.4 FKG and Holley’s inequalities ............... 132 4.2.5 . Erdos-R¨ enyi´ graphs: Janson’s inequality, and applica- tion to the containment problem .............. 137 4.2.6 . Percolation on Z2: RSW theory, and a proof of Harris’ theorem ........................... 140 4.3 Couplings of Markov chains .................... 147 4.3.1 Bounding the mixing time via coupling .......... 148 4.3.2 . Markov chains: mixing on cycles, hypercubes, and trees 149 4.3.3 Path coupling ........................ 154 4.3.4 . Ising model: Glauber dynamics at high temperature .. 158 4.3.5 . Colorings: from approximate sampling to approximate counting .......................... 160 4.4 Duality ............................... 161 4.4.1 Graphical representations and coupling of initial configu- rations ............................ 161 4.4.2 . Interacting particles: voter model on the complete graph and on the line ....................... 161 III Further techniques 165 5 Branching processes 166 5.1 Background ............................. 166 5.1.1 . Random walk in a random environment: Galton-Watson trees ............................. 166 5.2 Comparison to branching processes ................ 166 5.2.1 . Percolation on trees: branching process approach ... 166 5.2.2 . Randomized algorithms: height of random binary search tree ............................. 170 5.2.3 . Erdos-R¨ enyi´ graphs: the phase transition ........ 170 5.3 Further applications ......................... 187 5.3.1 . Random trees: local limit of uniform random trees ... 187 iii 6 Spectral techniques 190 6.1 Bounding the mixing time via the spectral gap ........... 191 6.1.1 . Markov chains: random walk on cycles and hypercubes 191 6.2 Eigenvalues and isoperimetry .................... 191 6.2.1 Edge and vertex expansion ................. 191 6.2.2 . Percolation on Zd: uniqueness of the infinite cluster, and extension to transitive amenable graphs .......... 191 6.2.3 . Random regular graphs: existence of expander graphs . 191 6.2.4 Bounding the spectral gap via the expansion constant ... 191 6.2.5 . Markov chains: random walk on cycles, hypercubes, and trees revisited ........................ 191 6.2.6 . Ising model: Glauber dynamics on the complete graph and expander graphs .................... 191 6.3 Further applications ......................... 191 6.3.1 . Markov random fields: more on the reconstruction problem191 6.3.2 . Stochastic blockmodel: spectral partitioning ...... 191 7 Correlation 192 7.1 Correlation decay .......................... 193 7.1.1 Definitions ......................... 193 7.1.2 . Percolation on Zd: BK inequality, and exponential decay of the correlation length .................. 193 7.1.3 . Ising model on Z2: uniqueness .............. 193 7.1.4 . Ising model on trees: non-reconstruction ........ 193 7.1.5 Weitz’s computation tree .................. 193 7.2 Local dependence .......................... 193 7.2.1 Lovasz´ local lemma .................... 193 7.2.2 Chen-Stein method ..................... 193 7.2.3 . Occupancy problems: birthday paradox, and variants . 193 7.3 Poissonization ............................ 193 7.3.1 . Occupancy problems: species sampling ......... 193 7.4 Exchangeability ........................... 193 7.4.1 . Preferential attachment graphs: connectivity via Polya´ urns ............................. 193 7.4.2 . Random partitions: size-biasing and stick-breaking, and application to cycles of random permutations ....... 193 8 More on concentration and isoperimetry 194 8.1 Variance bounds ........................... 195 8.1.1 Efron-Stein and Poincare´ inequalities ........... 195 iv 8.1.2 . Random permutations: variance of longest increasing subsequence ........................ 195 8.1.3 Canonical paths and comparison .............. 195 8.2 Talagrand’s inequality ........................ 195 8.2.1 Proof via a log Sobolev inequality ............. 195 8.2.2 . Random permutations: concentration of longest increas- ing subsequence ...................... 195 8.2.3 . Random structures: stochastic traveling salesman .... 195 8.3 Influence .............................. 195 8.3.1 Definitions ......................... 195 8.3.2 Russo’s formula ...................... 195 8.3.3 . Ising model on trees: recursive majority ......... 195 8.3.4 . Percolation on Z2: Kesten’s theorem ........... 195 8.3.5 Hypercontractivity ..................... 195 8.3.6 . First-passage percolation: anomalous fluctuations ... 195 8.4 Beyond the union bound ...................... 195 8.4.1 "-nets and chaining ..................... 195 8.4.2 VC-dimension ....................... 195 v Preface These lecture notes form the basis for a one-semester introduction to modern dis- crete probability, with an emphasis on essential techniques used throughout the field. The material is borrowed mostly from the following excellent texts. (Consult the bibliography for complete citations.) [AF] David Aldous and James Allen Fill. Reversible Markov chains and random walks on graphs. [AS11] N. Alon and J.H. Spencer. The Probabilistic Method. [BLM13] S. Boucheron, G. Lugosi, and P. Massart. Concentration Inequalities: A Nonasymptotic Theory of Independence. [Gri10b] G.R. Grimmett. Percolation. [JLR11] S. Janson, T. Luczak, and A. Rucinski. Random Graphs. [LP] R. Lyons with Y. Peres. Probability on trees and networks. [LPW06] David A. Levin, Yuval Peres, and Elizabeth L. Wilmer. Markov chains and mixing times. [MU05] Michael Mitzenmacher and Eli Upfal. Probability and Computing: Ran- domized Algorithms and Probabilistic Analysis. [RAS] Firas Rassoul-Agha and Timo Seppal¨ ainen.¨ A course on large deviations with an introduction to Gibbs measures. [Ste] J. E. Steif. A mini course on percolation theory. [vdH10] Remco van der Hofstad. Percolation and random graphs.