Randomized Algorithms RAJEEV MOTWANI Department of Computer Science, Stanford University, Stanford, California PRABHAKAR RAGHAVAN IBM Almaden Research Center, San Jose, California Randomized algorithms, once viewed as HISTORY AND SOURCES a tool in computational number theory, The roots of randomized algorithms can have by now found widespread applica- be traced back to Monte Carlo methods tion. Growth has been fueled by the two used in numerical analysis, statistical major benefits of randomization: sim- physics, and simulation. In complexity plicity and speed. For many applica- theory, the notion of a probabilistic Tur- tions a randomized algorithm is the ing machine was proposed by de Leeuw fastest algorithm available, or the sim- et al. [1955] and was further explored in plest, or both. the pioneering work of Rabin [1963] and A randomized algorithm is an algo- Gill [1977]. The earliest examples of rithm that uses random numbers to in- concrete randomized algorithms ap- fluence the choices it makes in the peared in the work of Berlekamp [1970], course of its computation. Thus its be- Rabin [1976], and Solovay and Strassen havior (typically quantified as running [1977]. Rabin [1976] explicitly proposed time or quality of output) varies from randomization as an algorithmic tool one execution to another even with a using as examples problems in compu- fixed input. In the analysis of a random- tational geometry and in number the- ized algorithm we establish bounds on ory. At about the same time, Solovay the expected value of a performance and Strassen [1977] presented a ran- measure (e.g., the running time of the domized primality-testing algorithm; algorithm) that are valid for every in- these were predated by the randomized put; the distribution of the performance polynomial-factoring algorithm pre- measure is on the random choices made sented by Berlekamp [1970]. by the algorithm based on the random Since then, an impressive array of bits provided to it. techniques for devising and analyzing randomized algorithms have been devel- oped. The reader may refer to Karp Caveat: The analysis of randomized [1991], Maffioli et al. [1985], and Welsh algorithms should be distinguished [1983] for recent surveys of the research from the probabilistic or average-case into randomized algorithms. The proba- analysis of an algorithm, in which it is bilistic (or “average-case”) analysis of assumed that the input is chosen from a algorithms (sometimes also called “dis- probability distribution. In the latter tributional complexity”) is surveyed by case, the analysis would typically imply Johnson [1984a], and this is contrasted only that the algorithm is good for most with randomized algorithms in his fol- inputs but not for all. lowing bulletin [1984b]. The work of R. Motwani was supported by an Alfred P. Sloan Research Fellowship, an IBM Faculty Partnership Award, and NSF Young Investigator Award CCR-9357849, with matching funds from IBM, Schlumberger Foundation, Shell Foundation, and Xerox Corporation. Copyright © 1996, CRC Press. ACM Computing Surveys, Vol. 28, No. 1, March 1996 34 • Rajeev Motwani and Prabhakar Raghavan The recent book by the authors [Mot- population is representative of the pop- wani and Raghavan 1995] gives a com- ulation as a whole. Because computa- prehensive introduction to randomized tions involving small samples are inex- algorithms. pensive, their properties can be used to guide the computations of an algorithm PARADIGMS FOR RANDOMIZED attempting to determine some feature of ALGORITHMS the entire population. For instance, a simple randomized algorithm [Floyd In spite of the multitude of areas in and Rivest 1975] based on sampling which randomized algorithms find ap- finds the kth largest of n elements in plication, a handful of general princi- 1.5n 1 o(n) comparison steps, with ples underlie almost all of them. Using high probability. In contrast, it is the summary in Karp [1991], we known that any deterministic algorithm present these principles in the follow- must make at least 2n comparisons in ing. the worst case. Foiling an Adversary. In the classical worst-case analysis of deterministic al- Abundance of Witnesses. Often a gorithms, a lower bound is established computational problem can be reduced on the running time of algorithms by to finding a witness or a certificate that postulating an “adversary” that con- would efficiently verify an hypothesis. structs an input on which the algorithm (For example, to show that a number is fares poorly. The input thus constructed not prime, it suffices to exhibit any non- may be different for each deterministic trivial factor of that number.) For many algorithm. With a game-theoretic inter- problems, the witness lies in a search pretation of the relationship between an space that is too large to be searched algorithm and an adversary, we can exhaustively. However, if the search view a randomized algorithm as a prob- space were to contain a relatively large ability distribution on a set of determin- number of witnesses, a randomly chosen istic algorithms. (This observation un- element is likely to be a witness. Fur- derlies Yao’s [1977] adaptation of von ther, independent repetitions of the Neumann’s Mini-Max Theorem in game sampling reduce the probability that a theory into a technique for establishing witness is not found on any of the repe- limits on the performance improve- titions. The most striking examples of ments possible via the use of a random- this phenomenon occur in number the- ized algorithm.) Although the adversary ory. Indeed, the problem of testing a may be able to construct an input that given integer for primality has no foils one (or a small fraction) of the known deterministic polynomial-time deterministic algorithms in the set, it algorithm. There are, however, several may be impossible to devise a single randomized polynomial-time algorithms input that is likely to defeat a randomly [Solovay and Strassen 1978; Rabin chosen algorithm. For example, consider 1980] that will, on any input, correctly a uniform binary AND-OR tree with n perform this test with high probability. leaves. Any deterministic algorithm that evaluates such a tree can be forced Fingerprinting and Hashing. A fin- to read the Boolean values at every one gerprint is the image of an element from of the n leaves. However, there is a a (large) universe under a mapping into simple randomized algorithm [Snir another (smaller) universe. Finger- 1985] for which the expected number of prints obtained via random mappings leaves read on any input is O(n0.794). have many useful properties. For exam- ple, in pattern-matching applications Random Sampling. A pervasive theme [Karp and Rabin 1987] it can be shown in randomized algorithms is the idea that two strings are likely to be identi- that a small random sample from a cal if their fingerprints are identical; ACM Computing Surveys, Vol. 28, No. 1, March 1996 Randomized Algorithms •35 comparing the short fingerprints is con- the resources. This paradigm has found siderably faster than comparing the many interesting applications in paral- strings themselves. Another example is lel and distributed computing where re- hashing [Carter and Wegman 1979], source utilization decisions have to be where the elements of a set S (drawn made locally, without global knowledge. from a universe U) are stored in a table Consider packet routing in an n-node of size linear in uSu (even though uUu .. butterfly network. It is known that uSu) with the guarantee that the ex- V(=n) steps are required by any deter- pected number of elements in S mapped ministic oblivious algorithm (a class of to a given location in the table is O(1). simple routing algorithms in which the This leads to efficient schemes for decid- route followed by a packet is indepen- ing membership in S. Random finger- dent of the routes of other packets). In prints have found a variety of applica- contrast, based on ideas of Valiant tions in generating pseudorandom [1982], it has been shown [Aleliunas numbers and complexity theory (for in- 1982; Upfal 1984] that there is a ran- stance, the verification of algebraic domized algorithm that terminates in identities [Freivalds 1977]). O(log n) steps with high probability. Random Reordering. A large class of Rapidly Mixing Markov Chains. In problems has the property that a rela- counting problems, the goal is to deter- tively naive algorithm A can be shown mine the number of combinatorial ob- to perform extremely well provided the jects with a specified property. When input data is presented in a random the space of objects is large, an appeal- order. Although A may have poor worst- ing solution is the use of the Monte case performance, randomly reordering Carlo approach of determining the num- the input data ensures that the input is ber of desired objects in a random sam- unlikely to be in one of the orderings ple of the entire space. In a number of that is pathological for A. The earliest cases, it can be shown that picking a instance of this phenomenon can be uniform random sample is as difficult as found in the behavior of the Quicksort the counting problem itself. A particu- algorithm [Hoare 1962]. The random re- larly successful technique for dealing ordering approach has been particularly with such problems is to generate near- successful in tackling problems in data uniform random samples by defining a structures and computational geometry. Markov chain on the elements of the For instance, there are simple algo- population, and showing that a short rithms for computing the convex hull of random walk using this Markov chain is n points in the plane that process the likely to sample the population uni- input one point at a time. Such algo- formly. This method is at the core of a rithms are doomed to take V(n2) steps if number of algorithms used in statistical the input points are presented in an physics [Sinclair 1992].