Preface If you place a large number of points randomly in the unit square, what is the distribution of the radius of the largest circle containing no points? Of the smallest circle containing 4 points? Why do Brownian sample paths have local maxima but not points of increase, and how nearly do they have points of increase? Given two long strings of letters drawn i.i.d. from a finite alphabet, how long is the longest consecutive (resp. non-consecutive) substring appearing in both strings? If an imaginary particle performs a simple random walk on the vertices of a high-dimensional cube, how long does it take to visit every vertex? If a particle moves under the influence of a potential field and random perturbations of velocity, how long does it take to escape from a deep potential well? If cars on a freeway move with constant speed (random from car to car), what is the longest stretch of empty road you will see during a long journey? If you take a large i.i.d. sample from a 2-dimensional rotationally-invariant distribution, what is the maximum over all half-spaces of the deviation between the empirical and true distributions? These questions cover a wide cross-section of theoretical and applied probability. The common theme is that they all deal with maxima or min- ima, in some sense. The purpose of this book is to explain a simple idea which enables one to write down, with little effort, approximate solutions to such questions. Let us try to say this idea in one paragraph. (a) Problems about random extrema can often be translated into prob- lems about sparse random sets in d 1 dimensions. ≥ (b) Sparse random sets often resemble i.i.d. random clumps thrown down randomly (i.e., centered at points of a Poisson process). (c) The problem of interest reduces to estimating mean clump size. (d) This mean clump size can be estimated by approximating the under- lying random process locally by a simpler, known process for which explicit calculations are possible. (Part (b) explains the name Poisson clumping heuristic). ii This idea is known, though rarely explicitly stated, in several specific settings, but its power and range seems not to be appreciated. I assert that this idea provides the correct way to look at extrema and rare events in a wide range of probabilistic settings: to demonstrate this assertion, this book treats over 100 examples. Our arguments are informal, and we are not going to prove anything. This is a rather eccentric format for a mathematics book | some reasons for this format are indicated later. The opening list of problems was intended to persuade every probabilist to read the book! I hope it will appeal to graduate students as well as experts, to the applied workers as well as theoreticians. Much of it should be comprehensible to the reader with a knowledge of stochastic processes at the non-measure-theoretic level (Markov chains, Poisson process, renewal theory, introduction to Brownian motion), as provided by the books of Karlin and Taylor (1975; 1982) and Ross (1983). Different chapters are somewhat independent, and the level of sophistication varies. Although the book ranges over many fields of probability theory, in each field we focus narrowly on examples where the heuristic is applicable, so this work does not constitute a complete account of any field. I have tried to maintain an honest \lecture notes" style through the main part of each chapter. At the end of each chapter is a \Commentary" giving references to background material and rigorous results. In giving references I try to give a book or survey article on the field in question, supplemented by recent research papers where appropriate: I do not attempt to attribute results to their discoverers. Almost all the examples are natural (rather than invented to show off the technique), though I haven't always given a thorough explanation of how they arise. The arguments in examples are sometimes deliberately concise. Most results depend on one key calculation, and it is easier to see this in a half-page argument than in a three-page argument. In rigorous treatments it is often necessary to spend much effort in showing that certain effects are ultimately negligible; we simply omit this effort, relying on intuition to see what the dominant effect is. No doubt one or two of our heuristic conclusions are wrong: if heuristic arguments always gave the right answer, then there wouldn't be any point in ever doing rigorous arguments, would there? Various problems which seem interesting and unsolved are noted as \thesis projects", though actually some are too easy, and others too hard, for an average Ph.D. thesis. The most-studied field of application of the heuristic is to extremes of 1- parameter stationary processes. The standard reference work on this field, Leadbetter et al. (1983), gives theoretical results covering perhaps 10 of our examples. One could write ten similar books, each covering 10 examples from another field. But I don't have the energy or inclination to do so, which is one reason why this book gives only heuristics. Another reason is that connections between examples in different fields are much clearer in the heuristic treatment than in a complete technical treatment, and I hope iii this book will make these connections more visible. At the risk of boring some readers and annoying others, here is a para- graph on the philosophy of approximations, heuristic and limit theorems. The proper business of probabilists is calculating probabilities. Often exact calculations are tedious or impossible, so we resort to approximations. A limit theorem is an assertion of the form: \the error in a certain approxi- mation tends to 0 as (say) N ". Call such limit theorem naive if there is no explicit error bound in terms! 1 of N and the parameters of the under- lying process. Such theorems are so prevalent in theoretical and applied probability that people seldom stop to ask their purpose. Given a serious applied problem involving specific parameters, the natural first steps are to seek rough analytic approximations and to run computer simulations; the next step is to do careful numerical analysis. It is hard to give any argument for the relevance of a proof of a naive limit theorem, except as a vague reassurance that your approximation is sensible, and a good heuris- tic argument seems equally reassuring. For the theoretician, the defense of naive limit theorems is \I want to prove something, and that's all I can prove". There are fields which are sufficiently hard that this is a reasonable attitude (some of the areas in Chapters G, I, J for example), but in most of the fields in this book the theoretical tools for proving naive limit the- orems have been sufficiently developed that such theorems are no longer of serious theoretical research interest (although a few books consolidating the techniques would be useful). Most of our approximations in particular examples correspond to known naive limit theorems, mentioned in the Commentaries. I deliberately de- emphasize this aspect, since as argued above I regard the naive limit theory as irrelevant for applications and mostly trite as theory. On the other hand, explicit error bounds are plainly relevant for applications and interesting as theory (because they are difficult, for a start!). In most of our examples, explicit error bounds are not know: I regard this as an important area for future research. Stein's method is a powerful modern tool for getting explicit bounds in \combinatorial" type examples, whose potential is not widely realized. Hopefully other tools will become available in the future. Acknowledgements: As someone unable to recollect what I had for dinner last night, I am even more unable to recollect the many people who (con- sciously or unconsciously) provided sources of the examples; but I thank them. Course based on partial early drafts of the book were given in Berke- ley in 1984 and Cornell in 1986, and I thank the audiences for their feed- back. In particular, I thank Persi Diaconis, Rick Durrett, Harry Kesten, V. Anantharam and Jim Pitman for helpful comments. I also thank Pilar Fresnedo for drawing the diagrams, and Ed Sznyter for a great job con- a verting my haphazard two-fingered typing into this elegant LTEX book. Contents A The Heuristic 1 A1 The M/M/1 queue. : : : : : : : : : : : : : : : : : : : : : : : 1 A2 Mosaic processes on R2. : : : : : : : : : : : : : : : : : : : : 2 A3 Mosaic processes on other spaces. : : : : : : : : : : : : : : : 5 A4 The heuristic. : : : : : : : : : : : : : : : : : : : : : : : : : : 5 A5 Estimating clump sizes. : : : : : : : : : : : : : : : : : : : : 7 A6 The harmonic mean formula. : : : : : : : : : : : : : : : : : 8 A7 Conditioning on semi-local maxima. : : : : : : : : : : : : : 10 A8 The renewal-sojourn method. : : : : : : : : : : : : : : : : : 11 A9 The ergodic-exit method. : : : : : : : : : : : : : : : : : : : 12 A10 Limit assertions. : : : : : : : : : : : : : : : : : : : : : : : : 14 A11{A21 Commentary : : : : : : : : : : : : : : : : : : : : : : : 15 B Markov Chain Hitting Times 23 B1 Introduction. : : : : : : : : : : : : : : : : : : : : : : : : : : 23 B2 The heuristic for Markov hitting times. : : : : : : : : : : : : 24 B3 Example: Basic single server queue. : : : : : : : : : : : : : : 25 B4 Example: Birth-and-death processes. : : : : : : : : : : : : : 25 B5 Example: Patterns in coin-tossing. : : : : : : : : : : : : : : 26 B6 Example: Card-shuffling. : : : : : : : : : : : : : : : : : : : : 27 B7 Example: Random walk on Zd mod N. : : : : : : : : : : : : 28 B8 Example: Random