Computability and computational complexity Lecture 10: Randomized computation Ion Petre Computer Science, Åbo Akademi University Fall 2015 http://users.abo.fi/ipetre/computability/ December 2, 2015 http://users.abo.fi/ipetre/computability/ 1 Content n Monte Carlo algorithms: examples n Random walk n Randomized complexity classes q RP q ZPP q PP q BPP n Sources of randomness December 2, 2015 http://users.abo.fi/ipetre/computability/ 2 MONTE CARLO ALGORITHMS: EXAMPLES December 2, 2015 http://users.abo.fi/ipetre/computability/ 3 PRIMES n Very important to be able to generate (very) large primes – e.g., in cryptography (RSA) n No practical techniques to yield large prime numbers RSA scheme n Procedure: generate random odd numbers and test –Key generation whether that integer is prime •Choose primes p,q n Testing whether or not an integer n is a prime is a •Compute n=pq difficult problem (“primality is difficult”) •Choose e, 1<e<f(n) with gcd(f(n),e)=1 •Compute dºe-1 mod f(n) q There has been a long standing question in math whether or not primality can be tested in polynomial •Private key is {d,n} deterministic time •Public key is {e,n} q Answer (2002): YES! –Encryption e q Manindra Agrawal, Neeraj Kayal and Nitin Saxena, •C=M mod n “PRIMES is in P”, Ann. of Math. (2), 160:2 (2004) 781-- –Decryption: 793. •Cd mod n = Mde mod n = M 12 q Drawback: high complexity – O((log n) f(log log n)), where f is a polynomial n RSA typically uses primes on 1024-2048 bits 6 q Subsequently improved to O((log n) f(log log n)) December 2, 2015 http://users.abo.fi/ipetre/computability/ 4 Miller-Rabin primality test n Faster methods of testing primality exist – they are all probabilistic q Such an algorithm can give two answers to the question “Is n prime?” n No, it is not n n is probably prime n The probability can be made arbitrarily large q Other types of randomized algorithms may give a precise answer but with low probability they may take a long time to finish n Most popular primality test: Miller- Rabin, based on Fermat’s little theorem December 2, 2015 http://users.abo.fi/ipetre/computability/ 5 Fermat’s little theorem n Fermat’s little theorem: if p is prime and a is positive integer not divisible by p, then ap-1 º 1 mod p n Corollary: For any positive integer a and prime p, ap º a mod p n Comments: q Fermat’s little theorem provides a necessary condition for an integer p to be prime – the condition is not sufficient n We will turn this theorem into a (probabilistic) test for primality December 2, 2015 http://users.abo.fi/ipetre/computability/ 6 Miller-Rabin primality test Fermat’s little theorem: if p is prime and a is positive integer not divisible by p, then ap-1 º 1 mod p •Question: for how many integers a does the test fail? TEST(n) •Failure: n is not prime but the algorithm return 1. n-1=2kq: compute k and q “probably prime” •Answer: for at most (n-1)/4 integers a with 1£a£n-1 2. Select a random integer a, •Thus, the probability of failure is at most ¼ 1<a<n-1 •Practical implementation: 3. If aqmod n=1 then return •Repeatedly invoke TEST(n) using random “probably prime” choices for a •If TEST(n) return at least once “not a prime”, 4. For j=0 to k-1 do then n is not a prime 2jq 5. If a mod n = n-1, then return •If t executions of TEST(n) return “probably “probably prime” prime”, then the probability that n is indeed a 6. Return “not a prime” prime is larger than 1-4-t •t=10 gives probability larger than 0.999999 December 2, 2015 http://users.abo.fi/ipetre/computability/ 7 Miller-Rabin primality test n Fermat’s little theorem: if p is prime and a is positive integer not divisible by p, then ap-1 º 1 mod p n Idea of the Miller-Rabin test: q We need to test if the odd integer n is prime: test the equality in Fermat’s little theorem for n and a random a n A speedup may be done so that we do not have to compute all powers of a – details bellow k q n-1 is even, i.e., of the form n-1=2 q, with k>0, q odd: k and q easy to find q Choose an integer a such that 1<a<n-1 2jq q 2q 2k-1q 2kq q Compute modulo n the values a , 0≤j≤q: a , a ,…, a , a q By Fermat’s theorem, if n is prime, then the last value in the sequence is 1 – the sequence may have some other 1s, consider the first 1 in the sequence n Case 1: the first number in the sequence is 1 – then all other powers are also 1 j j-1 n Case 2: some number a2 q in the sequence is 1 – in this case a2 q = n-1 mod n j j-1 j-1 j-1 j-1 q 0 = (a2 q -1) mod n = (a2 q – 1) (a2 q + 1) mod n, i.e., n divides (a2 q – 1) or (a2 q + 1) j-1 j-1 q Since we took the first 1 in the sequence, it follows that n divides (a2 q + 1): a2 q = n-1 mod n q The test: if either the first element in the sequence is 1, or some element before the last is n-1, then n could be prime. Otherwise n is certainly not prime December 2, 2015 http://users.abo.fi/ipetre/computability/ 8 Monte-Carlo algorithms n Definition: Monte-Carlo algorithms q Polynomial-time randomized algorithms: it finishes in polynomial time for all random choices q No false positives: if the algorithm says “yes”, then the result is certain q There might be false negatives: the algorithm does not answer “no” but rather “probably not” q Probability of false negatives is less than 1 n For random choices we assume the existence of a fair coin that generates perfectly random bits (re-discuss sources of randomness later in this lecture) n Important note: by repeating the algorithm in case of a negative answer we can make the probability of a false negative arbitrarily small q p<1 is the probability of a false negative k q getting a false negative in k consecutive runs of the algorithm is p , which decreases to 0 as k increases q running time remains polynomial even if we repeat the algorithm k times December 2, 2015 http://users.abo.fi/ipetre/computability/ 9 Random walks n A randomized algorithm for SAT q start with an arbitrary truth assignment T and repeat the following r times n if all clauses are satisfied, then reply “satisfiable” and halt n otherwise, take any unsatisfied clause: all of its literals are false under T n pick any of these literals and flip its truth value, updating T q after r iterations, reply “probably unsatisfiable” and halt n This is called a random walk algorithm: random path through the possible truth assignments n If the formula is unsatisfiable, then the algorithm answers well: “probably unsatisfiable” n If the formula is satisfiable: q easy to show that if we allow exponentially many repetitions, then a satisfying truth assignment can be found with high probability q Question: how about if we only try a number of times r that is polynomial in the number of variables? n main problem: the (very) large probability of false negatives q Answer: It does not work well for (some instances of) 3SAT! But it works for 2SAT! December 2, 2015 http://users.abo.fi/ipetre/computability/ 10 Random walks and 2SAT n Theorem. Suppose that the random walk algorithm with r=2n2 is applied to any satisfiable instance of 2SAT with n variables. Then the probability that a satisfying truth assignment will be discovered is at least ½. n Important technical result for the proof of the theorem (even though we skip it here): n Lemma. If x is a random variable taking nonnegative integer values, then for any k>0, P(x³k×E(x))£1/k, where E(x) is the expected value of x. December 2, 2015 http://users.abo.fi/ipetre/computability/ 11 RANDOMIZED COMPLEXITY CLASSES December 2, 2015 http://users.abo.fi/ipetre/computability/ 12 Randomized complexity classes n Several types of randomized algorithms exist q Monte Carlo q Las Vegas q Decision reached by simple majority q Decision reached by strong majority q … n Discuss here several of these concepts and the relationships among them n Also discuss how to implement randomized algorithms in practice December 2, 2015 http://users.abo.fi/ipetre/computability/ 13 Randomized complexity classes: RP n We only consider here polynomial-time bounded nondeterministic TM N standardized as follows: q N is precise (all nondeterministic computations halt after exactly the same number of steps) q there are exactly two nondeterministic choices in each step of N n Definition. Let L be a language. A polynomial Monte Carlo TM for L is a nondeterministic TM standardized as above such that: q each computation on an input of size n halts in exactly p(n) steps p(|x|) q if xÎL, then at least half of the 2 computations halt with “yes” q if xÏL, then all computations halt with “no” q In other words, no false positives and false negatives with a probability not more than ½ n The class of all languages with polynomial Monte Carlo TM is denoted RP (randomized polynomial time) December 2, 2015 http://users.abo.fi/ipetre/computability/ 14 RP n RP q N is precise (all nondeterministic computations halt after exactly the same number of steps) q there are exactly two nondeterministic choices in each step of N q each computation on an input of size n halts in exactly p(n) steps p(|x|) q if xÎL, then at least half of the 2 computations halt with “yes” q if xÏL, then all computations halt with “no” December 2, 2015 http://users.abo.fi/ipetre/computability/ 15 Discussion n The class RP remains the same if we replaced the condition that the probability of acceptance is at least half with a number strictly between 0 and 1 q Argument based on repeating the TM computation enough many times n What is the relationship between RP and P, NP? q clearly somewhere between P and NP n any polynomial Monte Carlo algorithm is by definition a nondeterministic polynomial-time algorithm