Randomized Algorithms Lecture :Kavitha Telikepalli Topic :(2-SAT & MAX-3-SAT) Naveen Garg Scribe : Sandhya S. Pillai(2007MCS3120) Suita Sharma(2007MCS2927) Date : 14/02/2008 Agenda 1. Introduction 2. 2-SAT Problem 3. Monte Carlo vs. Las Vegas methods 4. Analysis of 2-SAT 5. Random Walks and Markov inequality 6. 3-SAT Problem 7. Why 3-SAT is NP-Hard? 8. Max 3-SAT Problem randomization & de-randomization 9. References Introduction Randomized Algorithm In addition to input, algorithm takes a source of random numbers and makes random choices during execution. Behavior can vary even on a fixed input. SAT Satisfiability (SAT) is the problem of deciding whether a boolean formula in propositional logic has an assignment that evaluates to true. SAT occurs as a problem and is a tool in applications (e.g. Artificial Intelligence and circuit design) and it is considered a fundamental problem in theory, since many problems can be naturally reduced to it and it is the ©mother© of NP-complete problems. 2-SAT Problem φ = (x1 V~x2) ^ (x1 V ~x3) ^ (~x1 V x3) ^ (~x1 V ~x2) The k-SAT problem is the variant of SAT, in which, each clause consists of exactly k distinct literals. For k >= 3, k-SAT is NP hard, but for k = 1 and 2, there are polynomial time solutions. For k = 1, this solution is trivial. But for k = 2, it is slightly tricky. There is a much easier random algorithm. Example : AND of clauses (A V A V A V A ) Λ (Ā V Ā V Ā ) Λ (A V Ā V Ā ) 1 2 3 4 1 3 4 1 2 3 Each clause has at least one literal set to true. If Ā = true, A = true then Clause 1 and 2 1 2 are satisfiable but the Clause 3 requires A3 literal set to be false for it to be a satisfiable one. For a system with literals Xi, 0 <= i < m, and clauses Cj, 0 <= j < n, it goes as follows: Algorithm : Input : A Boolean formula φ in conjunctive normal form with exactly two distinct literals in every clause. E.g., φ = (x1 V~x2) ^ (x1 V ~x3) ^ (~x1 V x3) ^ (~x1 V ~x2) 1)Start with an arbitrary initial assignment to the literals 2)The function number_satisfied() returns the number of satisfied clauses for the current assignment 1) for (i = 0; i < m; i++) xi = true; // Let us call this assignment A 1. for (t = 0; t < T && number_satisfied(x, n) < n; t++) { select an arbitrary non-satisfied clause Cj; randomly and uniformly pick one of the literals xi in Cj; xi = (xi + 1) mod 2; //toggle the truth value of xi } 3) if (number_satisfied(x, n) == n) then report that the set of clauses is satisfiable else report that the set of clauses is not satisfiable Function number_satisfied() can be computed in linear time, some savings might be achieved by only keeping track of the changes between rounds. However, computation time is not so much the issue here, the main point is answering the question how large T must be taken to be reasonably sure that this Monte-Carlo algorithm gives the correct answer for satisfiable systems. There is a chance of error and we need to bound that. Monte Carlo vs. Las Vegas methods A Las Vegas algorithm is a randomized algorithms that always return the correct result. The only variant is that it's running time might change between executions. The QuickSort algorithm is an example for a Las Vegas algorithm A Monte Carlo algorithm is a randomized algorithm that might output an incorrect result. However, the probability of error can be diminished by repeated executions of the algorithm. The MinCut algorithm is an example of a Monte Carlo algorithm. This algorithm will always give correct non-satisfiable instances of ф. But if ф is satisfiable then we need to fix value of T such that this algorithm says ªnot satisfiable ª with a probability <=1/4. If ф is satisfiable, there is a satisfiable assignment to the xi. Let us call it S. Let A be the current assignment of our algorithm which differs from S in at least one variable. Let i = the number of variables having truth values same as in A and S. Now by flipping the variable in A, we progress by making i = i+1 with probability atleast 1/2. The above process can be modeled as a random walk on a line graph with n nodes: node y is connected to node y - 1 and node y + 1, as far as these indices are at least 1 and at most n. Random Walk Let G = (V,E) be a connected, undirected graph. A random walk on G, starting from vertex s ∈ V , is the random process defined as follows which is obviously trivial to implement. 1. u := s 2. repeat for T steps choose a neighbor v of u uniformly at random. u := v Consider a particle moving in an one-dimensional line. At each point in time, the particle will move either1 step to the right with probability p or 1 step to the left with probability 1 − p. Let A 2 {0,1}n be any satisfying assignment With probability at least ½ distance to A is reduced With probability at most ½ distance to A is increased Randomized 2-SAT Analysis Distance can never be larger than n if it starts at some 0 < i < n. Dominated by a walk where With probability exactly ½ distance to A is reduced With probability exactly ½ distance to A is increased Let us d0efine a random variable n Xi=# of steps to reach state n starting from state i. Xi=1 + # of steps to reach state n starting from state i+1 with probably ½ (Xi+1) Xi=1 + # of steps to reach state n starting from state i-1 with probably ½ ( Xi-1) Since this is a memory less property, start from the current state is a fresh start. E[Xi ]=1/2 E[1+Xi+1 ]+1/2E[1+ Xi-1 ] = 1+(E[ Xi-1 ]+E[Xi+1])/2 Let Si=E[ Xi ] then Si=1+( Si-1+Si+1)/2 Sn-1=1+( Sn-2 + Sn)/2 S0=S1+1 ............(1) 2S1 =S0 +S2 +2...............(2) . 2Sn-1 =Sn-2 +2..............(n) adding equations 1 to n we get Sn-1 =2(n-1)+1=2n-1 Sn-2 =(2n-1)+(2n-3) . 2 S0 =(2n-1)+(2n-3)+..............+3+1=n Let us define another random variable Y Y= # of variables whose truth value is same in current assignment A & satisfiable assignment S Movement of Y is similar to i-1 <-------- i -------> i+1 Now we have to fix T. For this we make use of Markov Inequality. Markov Inequality Markov©s inequality gives an upper bound for the probability that a non-negative function of a random variable is greater than or equal to some positive constant Proposition :For any non-negative random variable Y and any real number k >0, Pr [Y>=k] <= E [Y] / k As an example let k = 2*E[ Y ]. Then the above says Pr [Y>= 2 *E[ Y ] ] <= 1/2. Namely, if you move out to twice the expectation, you can have only half the area under the curve to your right. This is quite intuitive. Let Y be any positive random variable, then Pr[Y>=k*E[Y]] <= 1/k Proof for Markov Inequality E[Y]=∑ Pr[Y=y]*y = ∑ Pr[Y=y]*y+∑ Pr[Y=y]*y y<k y>=k >= 0 + k* Pr [ Y>=k ] =Pr [ Y>=k ]<=E [ Y ] / k Therefore T=4n2 Pr[φ is satisfiable but algorithm returns unsatisfiable] =Pr[our algorithm does not reach state n in 4n2 steps] = Pr[ Z>=4E[Z]] <= ¼ Z=# of steps to reach state n from our start state E[Z]<= n2 3 CNF SAT A special case of SAT that is incredibly useful in proving NP-hardness results is 3SAT (or 3-CNF-SAT). A boolean formula is in conjunctive normal form (CNF) if it is a conjunction (and) of several clauses, each of which is the disjunction (or) of several literals, each of which is either a variable or its negation. For example: (A V A V A )Λ (Ā V Ā V A ) Λ (A V Ā V A ) 1 2 3 1 3 4 1 2 3 Given such a boolean formula, can we come up with an algorithm, that is polynomial in time ? The answer to this question is NO!!! Hence this is NP hard problem. (Proof is given in the Appendix) Max 3-CNF This problem is to find an assignment which maximizes the number of satisfiable clauses. Let us take for example (A V A V Ā ) ^ (A V Ā VA ) ^ ( A V Ā V A ) 1 2 3 1 2 4 2 3 4 Randomized Algorithm for Max 3-CNF Set each variable to true with probability 1/2 independently. (for instance, toss the coin and if head, set the variable to true and if tail, set the variable to false). If a clause is not satisfied, this means all the 3 variables in the clause are false. Prob. that a clause is not satisfied = 1/2 * 1/2 * 1/2 (since each variable is independently set to false) Prob. that a clause is satisfied = 1 - Prob. that a clause is not satisfied = 1 - 1/8 = 7/8 There may be dependent clauses too. Eg. (A V A V A ) Λ ( Ā V A V A ) 1 2 3 1 2 3 In the above two clauses the first clause is true and hence the second clause is bound to be true if x2 = true or x3 = true or both are true.