NP-Completeness and Boolean Satisfiability Mridul Aanjaneya Stanford University August 14, 2012 Mridul Aanjaneya Automata Theory 1/ 49 Time-Bounded Turing Machines • A Turing Machine that, given an input of lengthn, always halts within T(n) moves is said to be T(n)-time bounded. The TM can be multitape. Sometimes, it can be nondeterministic. • The deterministic, multitape case corresponds roughly to an O(T(n)) running-time algorithm. Mridul Aanjaneya Automata Theory 2/ 49 The class P • If a DTMM is T(n)-time bounded for some polynomial T(n), then we sayM is polynomial-time(polytime) bounded. • AndL(M) is said to be in the classP. • Important Point: When we talk ofP, it doesn't matter whether we mean by a computer or by a TM. Mridul Aanjaneya Automata Theory 3/ 49 Polynomial Equivalence of Computers and TM's • A multitape TM can simulate a computer that runs for time O(T(n)) in at most O(T2(n)) of its own steps. • If T(n) is a polynomial, so isT 2(n). Mridul Aanjaneya Automata Theory 4/ 49 Examples of Problems in P • Isw in L(G), for a given CFGG? Inputw. Use CYK algorithm, which is O(n3). • Is there a path from nodex to nodey in graphG? Input =x,y, andG. Use Dijkstra's algorithm, which is O(n log n) on a graph ofn nodes and arcs. Mridul Aanjaneya Automata Theory 5/ 49 Running Times Between Polynomials • You might worry that something like O(n log n) is not a polynomial. • However, to be inP, a problem only needs an algorithm that runs in time less than some polynomial. • Surely O(n log n) is less than the polynomial O(n2). Mridul Aanjaneya Automata Theory 6/ 49 A Tricky Case: Knapsack • The Knapsack problem is: given positive integersi 1, i2,:::, in, can we divide them in two sets with equal sums? • Perhaps we can solve this problem in polytime by a dynamic-programming algorithm: Maintain a table of all the differences we can achieve by partitioning the firstj integers. Mridul Aanjaneya Automata Theory 7/ 49 Knapsack • Basis: j=0. Initially, the table has true for0 and false for all other differences. • Induction: To consideri j , start with a new table initially all false. • Then setk to true if, in the old table, there is a valuem that was true, andk is either m+i j or m-ij . Mridul Aanjaneya Automata Theory 8/ 49 Knapsack • Suppose we measure running time in terms of the sum of the integers, saym. • Each table only needs space O(m) to represent all the positive and negative differences we could achieve. • Each table can be constructed in time O(n). Mridul Aanjaneya Automata Theory 9/ 49 Knapsack • Sincen ≤ m, we can build the final table in O(m2) time. • From that table, we can see if0 is achievable and solve the problem. Mridul Aanjaneya Automata Theory 10/ 49 Subtlety: Measuring Input Size • Input size has a specific meaning: the length of the representation of the problem as it is input to a TM. • For the Knapsack problem, you cannot always write the input in a number of characters that is polynomial in either the number of or sum of the integers. Mridul Aanjaneya Automata Theory 11/ 49 Knapsack - Bad Case • Suppose we haven integers, each of which is around2 n. • We can write integers in binary, so the input takes O(n2) space to write down. • But the tables require space O(n2n). • They therefore require at least that order of time to construct. Mridul Aanjaneya Automata Theory 12/ 49 Bad Case • Thus, the proposed polynomial algorithm actually takes time O(n22n) on an input of length O(n2). • Or, since we like to usen as the input size, it takes time O(n2sqrt(n)) on an input of lengthn. • In fact, it appears no algorithm solves Knapsack in polynomial time. Mridul Aanjaneya Automata Theory 13/ 49 Redefining Knapsack • We are free to describe another problem, call it Pseudo-Knapsack, where integers are represented unary. • Pseudo-Knapsack is inP. Mridul Aanjaneya Automata Theory 14/ 49 The class NP • The running time of a nondeterministic TM is the maximum number of steps taken along any branch. • If that time bound is polynomial, the NTM is said to be polynomial-time bounded. • And its language/problem is said to be in the classNP. Mridul Aanjaneya Automata Theory 15/ 49 Example: NP • The Knapsack problem is definitely inNP, even using the conventional binary representation of integers. • Use nondeterminism to guess one of the subsets. • Sum the two subsets and compare. Mridul Aanjaneya Automata Theory 16/ 49 P versus NP • Originally a curiosity of Computer Science, mathematicians now recognize as one of the most important open problems the questionP=NP? • There are thousands of problems that are inNP but appear not to be inP. • But no proof that they aren't really inP. Mridul Aanjaneya Automata Theory 17/ 49 Complete Problems • One way to address theP=NP question is to identify complete problems forNP. • An NP-complete problem has the property that if it is inP, then every problem inNP is also inP. • Defined formally via polytime reductions. Mridul Aanjaneya Automata Theory 18/ 49 Complete Problems: Intuition • A complete problem for a class embodies every problem in the class, even if it does not appear so. • Strange but true: Knapsack embodies every polytime NTM computation. Mridul Aanjaneya Automata Theory 19/ 49 Polytime Reductions • Goal: Find a way to show problem X to be NP-complete by reducing every language/problem inNP to X in such a way that if we had a deterministic polynomial time algorithm for X , then we could construct a deterministic polynomial time algorithm for any problem inNP. Mridul Aanjaneya Automata Theory 20/ 49 Polytime Reductions • We need the notion of a polytime transducer - a TM that: 1 Takes an input of lengthn. 2 Operates deterministically for some polynomial time p(n). 3 Produces an output on a separate output tape. • Note: output length is at most p(n). Mridul Aanjaneya Automata Theory 21/ 49 Polytime Reductions State input n scratch tapes output <= p(n) Mridul Aanjaneya Automata Theory 22/ 49 Polytime Reductions • LetL andM be languages. • SayL is polytime reducible toM if there is a polytime transducerT such that for every inputw toT, the outputx= T(w) 2 M if and only ifw 2 L. Mridul Aanjaneya Automata Theory 23/ 49 Picture of Polytime Reductions in L in M not T in L not in M Mridul Aanjaneya Automata Theory 24/ 49 NP-complete Problems • A problem/languageM is said to be NP-complete if for every languageL 2 NP, there is a polytime reduction fromL toM. • Fundamental Property: IfM has a polytime algorithm, then L also has a polytime algorithm. i.e., ifM 2 P, then everyL 2 NP is also inP, orP=NP. Mridul Aanjaneya Automata Theory 25/ 49 Proof that Polytime Reductions Work • SupposeM has an algorithm of polynomial time q(n). • LetL have a polytime transducerT toM, taking polynomial time p(n). • The output ofT, given an input of lengthn, is at most of length p(n). • The algorithm forM on the output ofT takes time at most q(p(n)). Mridul Aanjaneya Automata Theory 26/ 49 Proof that Polytime Reductions Work • We now have a polytime algorithm forL: 1 Givenw of lengthn, useT to producex of length ≤ p(n), taking time ≤ p(n). 2 Use the algorithm forM to tell ifx 2 M in time ≤ q(p(n)). 3 Answer forw is whatever the answer forx is. • Total time ≤ p(n)+ q(p(n))= a polynomial. Mridul Aanjaneya Automata Theory 27/ 49 Boolean Expressions • Boolean, or propositional-logic expressions are built from variables and constants using the operators AND,OR, and NOT. Constants are true and false, represented by1 and0, respectively. We'll use concatenation for AND, + forOR, - for NOT. Mridul Aanjaneya Automata Theory 28/ 49 Example: Boolean Expressions • (x+y)(-x+-y) is true only when variablesx andy have opposite truth values. • Note: parentheses can be used at will, and are needed to modify the precendence order NOT (highest), AND,OR. Mridul Aanjaneya Automata Theory 29/ 49 The Satisfiability Problem (SAT) • Study of boolean functions generally is concerned with the set of truth assignments (assignments of0 or1 to each of the variables) that make the function true. • NP-completeness needs only a simpler question (SAT): does there exist a truth assignment making the function true? Mridul Aanjaneya Automata Theory 30/ 49 Example: SAT • (x+y)(-x+-y) is satisfiable. • There are, in fact, two satisfying truth assignments: 1 x=0;y=1. 2 x=1;y=0. • x(-x) is not satisfiable. Mridul Aanjaneya Automata Theory 31/ 49 SAT is in NP • There is a multitape NTM that can decide if a Boolean formula of lengthn is satisfiable. • The NTM takes O(n2) time along any path. • Use nondeterminism to guess a truth assignment on a second tape. • Replace all variables by guessed truth values. • Evaluate the formula for this assignment. • Accept if true. Cook's Theorem SAT is NP-complete. Mridul Aanjaneya Automata Theory 32/ 49 Picture so far • We have one NP-complete problem: SAT. • In the future, we shall do polytime reductions of SAT to other problems, thereby showing them to be NP-complete. • Why? If we polytime reduce SAT to X , and X 2 P, then so is SAT, and therefore so is all ofNP. Mridul Aanjaneya Automata Theory 33/ 49 Conjunctive Normal Form • A Boolean formula is in Conjunctive Normal Form(CNF) if it is the AND of clauses.