DOCSLIB.ORG

Randomized Algorithms

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Randomized Algorithms Randomized Algorithms 1. Concepts of randomized algorithms 2. Randomize algorithm analysis 3. Randomized complexity classes 4. Pseudorandom number generation 5. Global min-cut problem 6. Maximum 3-SAT 7. Primality test CP412 ADAII Introduction # 1 1. Concepts of randomized algorithms • A randomized algorithm is an algorithm whose working not only depends on the input but also on certain random choices made by the algorithm during the execution – A randomized algorithm depends on random numbers for its operation – Assume that we have a random number generator Random(a, b) that generates for two integers a, b with a < b an integer r with a <=r <= b uniformly at random. In other words. We assume that Random(a, b) runs in O(1) time. Examples randomized quick sort: randomly choose a pivoting element CP412 ADAII Introduction # 2 Deterministic vs Randomized algorithms Boolean equivalence problem: given two boolean functions f1(x1, .., xn) and f2(x1, …, xn), question: f1=f2 ? Deterministic algorithm: for each X in the enumeration of {0, 1}n if f1(X) != f2(X) return false; return true Randomized algorithm (probabilistic algorithms): for k from 1 to N (or while true), randomly choose X from {0, 1}n if f1(X) != f2(X) return false; return true Note that “true” output of the above probabilistic algorithm may not be correct. But the “false” output is an correct answer. CP412 ADAII Introduction # 3 Two types of randomized algorithms • Las Vegas algorithms – use the randomness to reduce the expected running time or memory usage of computing process, but always terminate with a correct result in a bounded amount of time. e.g. randomized quicksort • Monte Carlo algorithms (probabilistic algorithms) – depend on the random input, have a chance of producing an incorrect result or fail to produce a result either by signalling a failure or failing to terminate e.g. Randomized Boolean equivalence algorithm CP412 ADAII Introduction # 4 Why randomized algorithms? • Use randomized algorithm to find a solution • Using randomness to improve algorithm performance • Using randomness to improve a solution to a problem • Generating random data for testing • Simulation CP412 ADAII Introduction # 5 3. Analysis • Resource usage for randomized algorithms – expected worst-case performance, which is the average amount of time it takes on the worst input of a given size. This average is computed over all the possible outcomes of the random numbers during the execution of the algorithm. – Generally for Las Vegas algorithms For randomized algorithm A, when input x and random bits r are given, it becomes deteminstic algorithm. Let TA (x, r) denote the computing time of the algorithem. TA (x, r) is a random function of random variable r. The expected running time is E[TAA (x, r) ] = T (x, r) P[r] r 6 Randomized QuickSort • Pick a pivot p uniformly at random from the elements of the array Let T(n) be the expected number of comparisons done on an array of n elements. We have T(0) = T(1) = 0 and for larger n. Because there are n equally-likely choices for our pivot (hence the 1/n), and for each choice the expected cost is T(k) + T(n − 1 − k), where k is the number of elements less than the pivot. 7 1 n1 Tn() (() Tk Tn ( 1 k )) n k0 222nnn111 Tk() Tk () ak log k nnnkkk011 22 22log1aannnn kkdklog ( ) nn1 244 On(log) n more precise: Tn( ) 2 n log n 8 Expectation Expectation. Given a discrete random variables X, its expectation E[X] is defined by: Waiting for a first success. Coin is heads with probability p and tails with probability 1– p. How many independent flips X until first heads? Linearity of expectation. Given two random variables X and Y defined over the same probability space, E[X + Y] = E[X] + E[Y]. CP412 ADAII Introduction # 9 Guessing cards Game. Shuffle a deck of n cards; turn them over one at a time; try to guess each card. Guessing with memory. Guess a card uniformly at random from cards not yet seen. Claim. The expected number of correct guesses is Θ(log n). Coupon collector Coupon collector. Each box of cereal contains a coupon. There are n different types of coupons. Assuming all boxes are equally likely to contain each coupon, how many boxes before you have ≥ 1 coupon of each type? Claim. The expected number of steps is Θ(n log n). Monte Carlo algorithm analysis For Monte Carlo algorithms, when an random input is given, the execution is deterministic, but may get incorrect answer. We are interested in the probability of getting correct answer. To get high probability of correctness, we can do many runs. The overall running time is the number of runs times the time of each run. E.g. Boolean equivalence problem, we are interested in probability of getting a correct answer of the probabilistic algorithm. • For decision problems, these algorithms are generally classified as either false-biased or true-biased. A false-biased Monte Carlo algorithm is always correct when it returns false; . a true-biased algorithm is always correct when it returns true 12 Randomized complexity classes The class R or RP (Randomized Polynomial time) consists of languages L for which a polynomial-time Turing machine M exists such that P[M(x, r) = 1] ≥ 1/2 if x is in L P[M(x, r) = 1] = 0 if x is not in L In other words, we can find a witness that x is in L with constant probability. With n runs: P[Get yes answer] ≥ 1 − 2−n . The class co-R (co-RP): P[M(x, r) = 1] ≥ 1/2 if x is not L P[M(x, r) = 1] = 0 if x is in L CP412 ADAII Introduction # 13 Randomized complexity classes The class ZPP (Zero-error Probabilistic Polynomial time) is defined as RP union co-RP. The class BPP (Bounded-error Probabilistic Polynomial time): P[M(x, r) = 1] ≥ 2/3 if x is in L P[M(x, r) = 1] ≤ 1/3 if x is not in L The class PP (Probabilistic Polynomial time): P[M(x, r) = 1] > 1/2 if x is in L P[M(x, r) = 1] ≤ 1/2 if x is not in L CP412 ADAII Introduction # 14 4. Random number generation “Anyone who attempts to generate random numbers by deterministic means is, of course, living in a state of sin.” —John von Neuman • The computer is not capable of generating truly random numbers – The computer can only generate pseudorandom numbers-- numbers that are generated by a formula – Pseudorandom numbers look random, but are perfectly predictable if you know the formula • Pseudorandom numbers are good enough for most purposes, but not all--for example, not for serious security applications – Devices for generating truly random numbers do exist • They are based on physics properties CP412 ADAII Properties of good pseudorandom number generator 1. The numbers must have a correct distribution. In simulations, it is important that the sequence of random numbers is uncorrelated (i.e. numbers in the sequence are not related in any way). In numerical integration, it is important that the distribution is flat. 2. The sequence of random numbers must have a long period. All random number generators will repeat the same sequence of numbers eventually, but it is important that the sequence is sufficiently long. 3. The sequences should be reproducible. Often it is necessary to test the effect of certain simulation parameters and the exact same sequence of random numbers should be used to run many different simulation runs. It must also be possible to stop a simulation run and then re-continue the same run which means that the state of the RNG must be stored in memory. 4. The RNG should be easy to export from one computing environment to another. 5. The RNG must be fast. Large amounts of random numbers are needed in simulations. 6. The algorithm must be parallelizable. 16 Random Bit/Number Generator RBG: a device or algorithm which outputs a sequence of statistically independent and unbiased binary digits • Hardware-based – elapsed time between emission of particle during radioactive decay – thermal noise from a semiconductor diode or resistor; – the frequency instability of a free running oscillator; – air turbulence within disk drive which causes random fluctuations – drive sector read latency times – sound from a microphone or video input from a camera. • Software-based – the system clock – elapsed time between keystrokes or mouse movement – content of input/output buffers – user input – operating system values such as system load and network statistics Adopted from slides of Introduction # 17 Pseudo Random Bit/Number Generator PRBG – Input: a seed i.e. a truly random input sequence of length k (the seed) – Output: a deterministic sequence of length l >> k that “seems random” An adversary cannot efficiently distinguish between sequences of PRBG and truly RBG of length l. Introduction # 18 CP412 ADAII Introduction # 19 CP412 ADAII Introduction # 20 CP412 ADAII Introduction # 21 PRBG test CP412 ADAII Introduction # 22 CP412 ADAII Introduction # 23 CP412 ADAII Introduction # 24 CP412 ADAII Introduction # 25 CP412 ADAII Linear congruential generator • The simple way of generating pseudo-random numbers is by the linear congruential generator (LCG): xi+1 = (a * xi + b) % m, i=0, 1, 2, …., ; where a, b an m are constant numbers, say a, b are large prime numbers m is 232 or 264 – The initial value of x0 is called the seed LCG(a, b, m) or with mixed b>0, MLCG(a, m) GGL: 31 xi+1 = (16807xi) mod (2 −1) RAND: 32 32 xi+1 = ((69069xi+1) mod (2 ) or LCG(69069, 1, 2 ) 27 RAND implementation #include <stdio.h> double lcg(long int *xi) /* m=2**32 */ #include <stdlib.h> { long int getseed(); static long int a=69069, double lcg(long int *xi); b=1, m=4294967296; main(int argc, char **argv) { static double rm=4294967296.0; long int seed; *xi = (*xi*a+b)%m; int i; return (double)*xi/rm; double r1, r2; } /* Seed calculated from time */ seed = getseed(); #include <sys/time.h> /* Generate random number pairs */ long int getseed() { for(i=1; i<=20000; i++) { long int i; r1 = lcg(&seed); r2 = lcg(&seed); struct timeval tp; fprintf(stdout,"%g %g\n",r1,r2); if (gettimeofday(&tp,(struct timezone *) NULL)==0) { } i=tp.tv_sec+tp.tv_usec; return(0); i=i%1000000|1; } return i; } else { return -1; } } Introduction # 28 Visual test • 20000 random number pairs in two dimensions on a unit square CP412 ADAII Introduction # 29 Lagged Fibonacci generators • A lagged Fibonacci generator requires an initial set of elements x1, x2, ….
Load more

Recommended publications
  • Primality Testing for Beginners
    STUDENT MATHEMATICAL LIBRARY Volume 70 Primality Testing for Beginners Lasse Rempe-Gillen Rebecca Waldecker http://dx.doi.org/10.1090/stml/070 Primality Testing for Beginners STUDENT MATHEMATICAL LIBRARY Volume 70 Primality Testing for Beginners Lasse Rempe-Gillen Rebecca Waldecker American Mathematical Society Providence, Rhode Island Editorial Board Satyan L. Devadoss John Stillwell Gerald B. Folland (Chair) Serge Tabachnikov The cover illustration is a variant of the Sieve of Eratosthenes (Sec- tion 1.5), showing the integers from 1 to 2704 colored by the number of their prime factors, including repeats. The illustration was created us- ing MATLAB. The back cover shows a phase plot of the Riemann zeta function (see Appendix A), which appears courtesy of Elias Wegert (www.visual.wegert.com). 2010 Mathematics Subject Classification. Primary 11-01, 11-02, 11Axx, 11Y11, 11Y16. For additional information and updates on this book, visit www.ams.org/bookpages/stml-70 Library of Congress Cataloging-in-Publication Data Rempe-Gillen, Lasse, 1978– author. [Primzahltests f¨ur Einsteiger. English] Primality testing for beginners / Lasse Rempe-Gillen, Rebecca Waldecker. pages cm. — (Student mathematical library ; volume 70) Translation of: Primzahltests f¨ur Einsteiger : Zahlentheorie - Algorithmik - Kryptographie. Includes bibliographical references and index. ISBN 978-0-8218-9883-3 (alk. paper) 1. Number theory. I. Waldecker, Rebecca, 1979– author. II. Title. QA241.R45813 2014 512.72—dc23 2013032423 Copying and reprinting. Individual readers of this publication, and nonprofit libraries acting for them, are permitted to make fair use of the material, such as to copy a chapter for use in teaching or research. Permission is granted to quote brief passages from this publication in reviews, provided the customary acknowledgment of the source is given.
    [Show full text]
  • CS265/CME309: Randomized Algorithms and Probabilistic Analysis Lecture #1:Computational Models, and the Schwartz-Zippel Randomized Polynomial Identity Test
    CS265/CME309: Randomized Algorithms and Probabilistic Analysis Lecture #1:Computational Models, and the Schwartz-Zippel Randomized Polynomial Identity Test Gregory Valiant,∗ updated by Mary Wootters September 7, 2020 1 Introduction Welcome to CS265/CME309!! This course will revolve around two intertwined themes: • Analyzing random structures, including a variety of models of natural randomized processes, including random walks, and random graphs/networks. • Designing random structures and algorithms that solve problems we care about. By the end of this course, you should be able to think precisely about randomness, and also appreciate that it can be a powerful tool. As we will see, there are a number of basic problems for which extremely simple randomized algorithms outperform (both in theory and in practice) the best deterministic algorithms that we currently know. 2 Computational Model During this course, we will discuss algorithms at a high level of abstraction. Nonetheless, it's helpful to begin with a (somewhat) formal model of randomized computation just to make sure we're all on the same page. Definition 2.1 A randomized algorithm is an algorithm that can be computed by a Turing machine (or random access machine), which has access to an infinite string of uniformly random bits. Equivalently, it is an algorithm that can be performed by a Turing machine that has a special instruction “flip-a-coin”, which returns the outcome of an independent flip of a fair coin. ∗©2019, Gregory Valiant. Not to be sold, published, or distributed without the authors' consent. 1 We will never describe algorithms at the level of Turing machine instructions, though if you are ever uncertain whether or not an algorithm you have in mind is \allowed", you can return to this definition.
    [Show full text]
  • Lecture 1: January 21 Instructor: Alistair Sinclair
    CS271 Randomness & Computation Spring 2020 Lecture 1: January 21 Instructor: Alistair Sinclair Disclaimer: These notes have not been subjected to the usual scrutiny accorded to formal publications. They may be distributed outside this class only with the permission of the Instructor. 1.1 Introduction There are two main flavors of research in randomness and computation. The first looks at randomness as a computational resource, like space and time. The second looks at the use of randomness to design algorithms. This course focuses on the second area; for the first area, see other graduate classes such as CS278. The starting point for the course is the observation that, by giving a computer access to a string of random bits, one can frequently obtain algorithms that are far simpler and more elegant than their deterministic counterparts. Sometimes these algorithms can be “derandomized,” although the resulting deterministic algorithms are often rather obscure. In certain restricted models of computation, one can even prove that randomized algorithms are more efficient than the best possible deterministic algorithm. In this course we will look at both randomized algorithms and random structures (random graphs, random boolean formulas etc.). Random structures are of interest both as a means of understanding the behavior of (randomized or deterministic) algorithms on “typical” inputs, and as a mathematically clean framework in which to investigate various probabilistic techniques that are also of use in analyzing randomized algorithms. 1.1.1 Model of Computation We first need to formalize the model of computation we will use. Our machine extends a standard model (e.g., a Turing Machine or random-access machine), with an additional input consisting of a stream of perfectly random bits (fair coin flips).
    [Show full text]
  • Randomized Algorithms 2018/9A Lecture 1 ∗ Min Cut Algorithm, Closest Pairs, (Multi)-Set Equality
    Randomized Algorithms 2018/9A Lecture 1 ∗ Min Cut Algorithm, Closest Pairs, (Multi)-Set Equality Moni Naor The lecture introduces randomized algorithms. Why are they interesting? They may solve problems faster than deterministic ones, they may be essential in some settings, especially when we want to go to the sublinear time complexity realm1, they are essential in distributed algorithms e.g. for breaking symmetry, they yield construction of desirable objects that we do not know how to build explicitly and are essential for cryptography2 and privacy3. Another type of study is to analyze algorithms when assuming some distribution on the input, or some mixture of worst case and then a perturbation of the input (known as smoothed analysis). But our emphasis would be worst case data where the randomness is created independently of it. That is we assume the algorithm or computing device in addition to the inputs gets also a random `tape' (like the other tapes of the Turing Machine, but this one with truly random symbols). One nice feature that some randomized algorithms have is that they are simple. We demonstrated this in three algorithms in three different scenarios. Randomized algorithms existed for a long time, since the dawn of computing (for instance the numerical \Monte Carlo Method"4 from the 1940's or Shannon's work [9], also from that time. Later, when people started arguing rigorously about the running time of programs, the idea of complexity classes of probabilistic machines emerged. We mentioned three such classes: RP , Co − RP and BP P . The common conjecture is that P = BP P and it is known to hold under the assumption that there are sufficiently good pseudo-random generators, that is a function G : f0; 1g∗ 7! f0; 1g∗ that stretches its input significantly and the output cannot be distinguished from random.
    [Show full text]
  • Advanced Algorithm
    Advanced Algorithm Jialin Zhang [email protected] Institute of Computing Technology, Chinese Academy of Sciences March 21, 2019 Jialin Advanced Algorithm A Randomized Algorithm for Min-Cut Problem Contrast Algorithm: 1 Pick an edge uniformly at random; 2 Merge the endpoints of this edge; 3 Remove self-loops; 4 Repeat steps 1-3 until there are only two vertices remain. 5 The remaining edges form a candidate cut. Jialin Advanced Algorithm Min Cut 1 Successful probability: Ω( n2 ). Time complexity: O(n2). Improvement? FastCut algorithm Ref: Randomized Algorithm - Chapter 10:2. Algorithm FastCut runs in O(n2 log n) time and uses O(n2) space. 1 Successful Probability is Ω( log n ). Jialin Advanced Algorithm Homework 1 Prove the successful probability for FastCut algorithm is 1 Ω( log n ). 2 Randomized Algorithm - Exercise 10:9, Page 293. Jialin Advanced Algorithm Homework (optional) We define a k-way cut-set in an undirected graph as a set of edges whose removal breaks the graph into k or more connected components. Show that the randomized min-cut algorithm can be modified to find a minimum k-way cut-set in O(n2(k−1)) time. Hints: 1 When the graph has become small enough, say less than 2k vertices, you can apply a deterministic k-way min-cut algorithm to find the minimum k-way cut-set without any cost. 2 To lower bound the number of edges in the graph, one possible way is to sum over all possible trivial k-way, i.e. k − 1 singletons and the complement, and count how many times an edge is (over)-counted.
    [Show full text]
  • Notes on Primality Testing and Public Key Cryptography Part 1: Randomized Algorithms Miller–Rabin and Solovay–Strassen Tests
    Notes on Primality Testing And Public Key Cryptography Part 1: Randomized Algorithms Miller{Rabin and Solovay{Strassen Tests Jean Gallier and Jocelyn Quaintance Department of Computer and Information Science University of Pennsylvania Philadelphia, PA 19104, USA e-mail: [email protected] c Jean Gallier February 27, 2019 Contents Contents 2 1 Introduction 5 1.1 Prime Numbers and Composite Numbers . 5 1.2 Methods for Primality Testing . 6 1.3 Some Tests for Compositeness . 9 2 Public Key Cryptography 13 2.1 Public Key Cryptography; The RSA System . 13 2.2 Correctness of The RSA System . 18 2.3 Algorithms for Computing Powers and Inverses Modulo m . 22 2.4 Finding Large Primes; Signatures; Safety of RSA . 26 3 Primality Testing Using Randomized Algorithms 33 4 Basic Facts About Groups, and Number Theory 37 4.1 Groups, Subgroups, Cosets . 37 4.2 Cyclic Groups . 50 4.3 Rings and Fields . 60 4.4 Primitive Roots . 67 4.5 Which Groups (Z=nZ)∗ Have Primitive Roots . 75 4.6 The Lucas Theorem, PRIMES is in NP .................... 80 4.7 The Structure of Finite Fields . 90 5 The Miller{Rabin Test 93 5.1 Square Roots of Unity . 94 5.2 The Fermat Test; F -Witnesses and F -Liars . 96 5.3 Carmichael Numbers . 99 5.4 The Miller{Rabin Test; MR-Witnesses and MR-Liars . 103 5.5 The Monier{Rabin Bound on the Size of the Set of MR-Liars . 116 5.6 The Least MR-Witness for n . 121 6 The Solovay{Strassen Test 125 6.1 Quadratic Residues .
    [Show full text]
  • Randomized Algorithms We Already Learned Quite a Few Randomized Algorithms in the Online Algorithm Lectures
    Randomized Algorithms We already learned quite a few randomized algorithms in the online algorithm lectures. For example, the MARKING algorithm for paging was a randomized algorithm; as well as the Randomized Weighted Majority. In designing online algorithms, randomization provides much power against an oblivious adversary. Thus, the worst case analysis (the worst case of the expectation) may become better by using randomization. We also noticed that randomization does not provide as much power against the adaptive adversaries. We’ll study some examples and concepts in randomized algorithms. Much of this section is based on (Motwani and Raghavan, Randomized Algorithm, Chapters 1, 5, 6). Random QuickSort Suppose is an array of numbers. Sorting puts the numbers in ascending order. The quick sort is one of the fastest sorting algorithm. QuickSort: 1. If is short, sort using insertion sort algorithm. 2. Select a “pivot” arbitrarily. 3. Put all members in and others in . 4. QuickSort and , respectively. | | Suppose the choice of is such that | | | | , then the time complexity is ( ) | | | | ( ). By using Master’s theorem, ( ) ( ). This is even true for | | . For worst case analysis, QuickSort may perform badly because may be such that and are awfully imbalanced. So the time complexity becomes ( ). If we don’t like average analysis, which is venerable to a malicious adversary, we can do randomization. RandomQS: In QuickSort, select the pivot uniformly at random from . Theorem 1: The expected number of comparisons in an execution of RandomQS is at most , where is the -th harmonic number. Proof: Clearly any pair of elements are compared at most once during any execution of the algorithm.
    [Show full text]
  • Monte Carlo Algorithm (10C)
    Monte Carlo Algorithm (10C) Young Won Lim 5/31/17 Copyright (c) 2017 Young W. Lim. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License". Please send corrections (or suggestions) to [email protected]. This document was produced by using OpenOffice and Octave. Young Won Lim 5/31/17 Monte Carlo ● Monte Carlo Method (Monte Carlo Simulation) ● Monte Carlo Algorithm Young Won Lim Monte Carlo Algorithm (10C) 3 5/31/17 Monte Carlo Method a broad class of computational algorithms that rely on repeated random sampling to obtain numerical results. Their essential idea is using randomness to solve problems that might be deterministic in principle. They are often used in physical and mathematical problems and are most useful when it is difficult or impossible to use other approaches. used in three distinct problem classes: ● optimization, ● numerical integration ● generating draws from a probability distribution. https://en.wikipedia.org/wiki/Monte_Carlo_method Young Won Lim Monte Carlo Algorithm (10C) 4 5/31/17 Patterns of Monte Carlo Methods Monte Carlo methods vary, but tend to follow a particular pattern: ● Define a domain of possible inputs. ● Generate inputs randomly from a probability distribution over the domain. ● Perform a deterministic computation on the inputs. ● Aggregate the results. https://en.wikipedia.org/wiki/Monte_Carlo_method Young Won Lim Monte Carlo Algorithm (10C) 5 5/31/17 Examples of Monte Carlo Methods For example, consider a circle inscribed in a unit square.
    [Show full text]
  • Randomized Algorithms 2
    15-251 Great Theoretical Ideas in Computer Science Lecture 20: Randomized Algorithms November 5th, 2015 So far Formalization of computation/algorithm Computability / Uncomputability Computational complexity Graph theory and graph algorithms NP-completeness. Identifying intractable problems. Making use of intractable problems (in Social choice). Dealing with intractable problems: Approximation algs. Online algs. Next Randomness and the universe Does the universe have true randomness? Newtonian physics suggests that the universe evolves deterministically. Quantum physics says otherwise. Randomness and the universe Does the universe have true randomness? God does not play dice with the world. - Albert Einstein Einstein, don’t tell God what to do. - Niels Bohr Randomness and the universe Does the universe have true randomness? Even if it doesn’t, we can still model our uncertainty about things using probability. Randomness is an essential tool in modeling and analyzing nature. It also plays a key role in computer science. Randomness in computer science Randomized algorithms Does randomness speed up computation? Statistics via sampling e.g. election polls Nash equilibrium in Game Theory Nash equilibrium always exists if players can have probabilistic strategies. Cryptography A secret is only as good as the entropy/uncertainty in it. Randomness in computer science Randomized models for deterministic objects e.g. the www graph Quantum computing Randomness is inherent in quantum mechanics. Machine learning theory Data is generated by some probability
    [Show full text]
  • Lecture 2 — 31 August 2014 1 Overview 2 Food for Thought 3 Probability Background
    CS 388R: Randomized Algorithms Fall 2015 Lecture 2 | 31 August 2014 Prof. Eric Price Scribe: Sid Kapur, Neil Vyas 1 Overview In this lecture we will first introduce some probability machinery: linearity of expectation, the central limit theorem, and the discrete-setting Chernoff bound. We will then apply these tools to some simple problems: repeated Bernoulli trials and the coupon collecting problem. Finally, we will discuss how randomized algorithms fit in to the landscape of complexity theory. 2 Food for Thought 1. Suppose you flip an unbiased coin 1000 times. How surprised would you be if we observed • 500 heads? • 510 heads? • 600 heads? • 1000 heads? 2. Suppose you have a biased coin that lands heads with an unknown probability p. How many flips will it take to learn p to an error of ± with probability 1 − δ? 3 Probability Background Theorem 1. Linearity of expectation. If X1;:::;Xn are random variables, then " n # n X X E Xi = E[Xi] i=1 i=1 Pn Example: Suppose X1;:::;Xn are Bernoulli trials with p = 0:5. Let X = i=1 Ei. Then n X n [X] = [X ] = E E i 2 i=1 Definition 2. The variance of a random variable X, denoted Var[X], is defined as 2 Var[X] = E (X − E[X]) 1 Observation 3. Note that if Y1 and Y2 are independent random variables with E[Y1] = E[Y2] = 0, then 2 Var[Y1 + Y2] = E (Y1 + Y2) 2 2 = E Y1 + Y2 + 2Y1Y2 2 2 = E Y1 + E Y2 + 2 E[Y1] E[Y2] (since Y1 and Y2 are independent) 2 2 = E Y1 + E Y2 = Var[Y1] + Var[Y2] Claim 4.
    [Show full text]
  • Primality Testing∗
    Primality Testing∗ Richard P. Brent MSI & CECS, ANU 24 August 2010 ∗ Copyright c 2010, R. P. Brent. Abstract For many years mathematicians and computer scientists have searched for a fast and reliable primality test. This is especially relevant nowadays, because the popular RSA public-key cryptosystem requires very large primes in order to generate secure keys. I will describe some efficient randomised algorithms that are useful, but have the defect of occasionally giving the wrong answer, or taking a very long time to give an answer. In 2002, Agrawal, Kayal and Saxena (AKS) found a deterministic polynomial-time algorithm1 for primality testing. I will describe the original AKS algorithm and some improvements by Bernstein and Lenstra. As far as theory is concerned, we now know that “PRIMES is in P”, and this appears to be the end of the story. However, I will explain why it is preferable to use randomised algorithms in practice. 1Not mentioned on page 983 of the textbook! 2 First, some notation As usual, we say that f(n) = O(nk) if, for some c and n , for all n n , 0 ≥ 0 f(n) cnk . | | ≤ We say that f(n) = O(nk) if, for some c 0, e ≥ f(n) = O(nk(log n)c) . The “O” notation is useful to avoid terms like log n and loglog n. For example, when referring e to the Sch¨onhage-Strassen algorithm for n-bit integer multiplication, it is easier to write O(n) than the (more precise)e O(n log n log log n) . 3 Complexity Classes (informal) P is the class of decision (“yes”/“no”) problems that have a deterministic polynomial time algorithm, i.e.
    [Show full text]
  • Model Counting for Logical Theories Wednesday
    Model Counting for Logical Theories Wednesday Dmitry Chistikov Rayna Dimitrova Department of Computer Science University of Oxford, UK Max Planck Institute for Software Systems (MPI-SWS) Kaiserslautern and Saarbr¨ucken, Germany ESSLLI 2016 SAT SMT satisfiability satisfiability modulo theories #SMT #SAT model counting model counting modulo theories 2/54 Agenda Tuesday computational complexity, probability theory Wednesday randomized algorithms, Monte Carlo methods Thursday hashing-based approach to model counting Friday from discrete to continuous model counting 3/54 Outline 1. Randomized algorithms Complexity classes RP and BPP 2. Monte Carlo methods 3. Markov chain Monte Carlo 4/54 Decision problems and algorithms Decision problem: L ⊆ f0; 1g∗ (encodings of yes-instances) Algorithm for L: says \yes" on every x 2 L, \no" on every x 2 f0; 1g∗ n L 5/54 Complexity classes: brief summary P: polynomial time (efficiently solvable) NP: nondeterministic polynomial time (with efficiently verifiable solutions) #P: counting polynomial time 6/54 Examples of randomized algorithms I Primality testing I Polynomial identity testing I Undirected reachability I Volume estimation 7/54 Randomized algorithms: our model The algorithm can toss fair coins: 1. Syntactically, a randomized algorithm is an algorithm that has access to a source of randomness, but acts deterministically if the random input is fixed. 2. It has on-demand access to arbitrary many independent 1 random variables that have Bernoulli( 2 ) distribution. 3. Each request takes 1 computational step. 8/54 Deterministic and randomized time complexity Recall deterministic time complexity: I of algorithm A on input x Randomized time complexity: Maximum (worst-case) over all possible sequences of random bits Then take maximum (worst-case) over all inputs of length n.
    [Show full text]