6 - 1 Computational Complexity P. Parrilo and S. Lall, CDC 2003 2003.12.07.06 6. Computational Complexity Computational models ² Turing Machines ² Time complexity ² Non-determinism, witnesses, and short proofs. ² Complexity classes: P, NP, coNP ² Polynomial-time reductions ² NP-hardness, NP-completeness ² Relativization, quanti¯ers, and the polynomial hierarchy ² The role of relaxations ² 6 - 2 Computational Complexity P. Parrilo and S. Lall, CDC 2003 2003.12.07.06 Computation Want to study and understand The power and limitations of computational methods. This requires a ² formalization of the notion of algorithm. What can and cannot be computed, and the resources needed to do ² so. Why polynomial problems do not have easy solutions in general. ² In our context, understand relaxations from a complexity perspective. 6 - 3 Computational Complexity P. Parrilo and S. Lall, CDC 2003 2003.12.07.06 Computational models Finite automata ² Pushdown automata ² Turing machines ² Church's ¸-calculus ² Any modern computer programming language ² Exact choice of model is not crucial. Last three have similar computational power. The Church-Turing thesis: The intuitive notion of an algorithm is captured by a Turing machine. ² Any conceivable computational process can be performed in a Turing ² machine. 6 - 4 Computational Complexity P. Parrilo and S. Lall, CDC 2003 2003.12.07.06 Turing machines An in¯nite tape, with a head that moves forwards and backwards. M 1 1 0 0 0 1 0 1 0 A ¯nite list of rules, that describes the behavior. Machine reads symbol in tape, moves forward or backwards, and optionally writes. A very idealized abstraction, of purely theoretical use ² This was before computers even existed! (1936) ² 6 - 5 Computational Complexity P. Parrilo and S. Lall, CDC 2003 2003.12.07.06 A formal notion of computation allows for results on: Computability: what can be decided algorithmically? ² A qualitative viewpoint ² Existence of non-algorithmically decidable questions. ² Undecidability of the Turing machine halting problem. ² Application: quadratic integer programming is undecidable. ² Connections with GÄodel's incompleteness theorem. ² Complexity: what resources (time, memory, communication) are needed? ² Emphasis on quantitative properties ² Can we solve the problem e±ciently? ² Much more recent theory (started in 1970s) ² 6 - 6 Computational Complexity P. Parrilo and S. Lall, CDC 2003 2003.12.07.06 Decision problems Complexity classes are de¯ned for decision problems, i.e., those with a yes/no answer. Examples: Is this graph connected? Is this matrix singular? Is this polynomial nonnegative? Is this proposition satis¯able? Is this optimization problem feasible? Given a problem, is the answer \Yes" or \No"? A language is the set of inputs with the desired property. So we can equivalent ask \Is this instance in the language?" We use \problem" informally, to denote \language". 6 - 7 Computational Complexity P. Parrilo and S. Lall, CDC 2003 2003.12.07.06 Time complexity We are interested in the number of operations, as a function of the length of the input. Standard O( ) notation: f(n) is O(g(n)) if there exists a n0 and c > 0 such that ¢ n n f(n) cg(n) ¸ 0 ) · . n+5 3 Example: 3 is O(n ). ¡ ¢ For instance, we can say: a Turing machine requires O(n2) units of time, where n is the length of the input. 6 - 8 Computational Complexity P. Parrilo and S. Lall, CDC 2003 2003.12.07.06 Complexity classes: P P is the class of problems that can be decided in polynomial time, i.e., those for which the running time is O(nk), for k N. 2 Some examples in P: Connectedness: Is the graph G connected? ² Nonsingularity: Is the rational matrix M singular? ² Positive semide¯niteness: Is this rational matrix PSD? ² Stability: Is this rational matrix Hurwitz stable? ² Controllability: Is this system controllable? ² For all of these, there are algorithms requiring only a polynomial number of steps. 6 - 9 Computational Complexity P. Parrilo and S. Lall, CDC 2003 2003.12.07.06 Complexity classes: P Unambiguous and correct decision (yes or no), in polynomial time. \E±ciently" or \quickly" are synonyms for \in polynomial time". If a language is in P, so is its complement. Ex: If I can e±ciently decide \Is this matrix singular?", then I can also solve e±ciently \Is this matrix nonsingular?" Formally, we have that P=coP. 6 - 10 Computational Complexity P. Parrilo and S. Lall, CDC 2003 2003.12.07.06 Complexity classes: NP NP stands for nondeterministic polynomial time. What does this mean? Given a \hint", \guess", or \certi¯cate", can verify in polynomial time. Examples: Is this logical proposition satis¯able? ² If you give me the assignment, it's easy. Compositeness: Is this number composite (not prime)? ² If you give me the factors, I can multiply them. Does this graph have a Hamiltonian path? ² Given the path, I can easily verify if it is Hamiltonian. Is this matrix singular? ² Give me a vector in the kernel, then I can quickly check. A \Yes" answer is easy to verify. Can use the certi¯cate as a proof, to quickly convince someone else. 6 - 11 Computational Complexity P. Parrilo and S. Lall, CDC 2003 2003.12.07.06 Complexity classes: NP NP is that class of languages that: Have polynomial-time veri¯ers ² Can be decided by nondeterministic polynomial time Turing machines ² \If you help me, I can e±ciently check whether it's correct" Verifying is easy. Deciding may be easy, or hard. Notice that, in particular, P NP Unlike P, it is not known whether NP is closed under complement. 6 - 12 Computational Complexity P. Parrilo and S. Lall, CDC 2003 2003.12.07.06 Complementation Consider a logical proposition Q = ((p q) (q r)). : ^ ) ^ : p q r p q q r Q ^ ^ : T T T T F T T T F T T F T F T F F F For a proposition in n variables, T F F F F F the truth table has 2n rows. F T T F F F F T F F T F F F T F F F F F F F F F To verify that the proposition is satis¯able, it is enough to look at one row. If you tell me which row, I can do that quickly. However, if it is unsatis¯able, no easy shortcuts in general. For instance, need to look at all 2n rows. 6 - 13 Computational Complexity P. Parrilo and S. Lall, CDC 2003 2003.12.07.06 Complexity classes: coNP coNP is the set of languages for which non-membership can be e±ciently veri¯ed, with ² a guess. problems that have polynomial-time refutations ² problems for which a \No" answer is easy to verify ² languages whose complement is in NP ² Examples: Tautologies (propositions that are true for every possible assignment). ² Graphs without Hamiltonian circuits ² Nonnegative functions in [0; 1]n. ² 6 - 14 Computational Complexity P. Parrilo and S. Lall, CDC 2003 2003.12.07.06 Relationships Clearly, P NP, and P coNP. Therefore: co-NP NP P Unless they're all the same! In fact... 6 - 15 Computational Complexity P. Parrilo and S. Lall, CDC 2003 2003.12.07.06 The big questions There is a lot that we don't know: Is P=NP? ² Does the existence of a short proof guarantee that we can ¯nd it ² e±ciently? Is NP=coNP? ² Do all tautologies have short proofs? ² Is it equally easy to certify \yes" and \no" answers? ² Have been open for more than 30 years. They're certainly the most important open questions in theoretical com- puter science, and perhaps in mathematics. A 1M USD reward from the Clay Institute for a solution. 6 - 16 Computational Complexity P. Parrilo and S. Lall, CDC 2003 2003.12.07.06 Polynomial-time reductions An important concept: the reducibility of a problem to another. We transform a problem (language) into another one, not changing signif- icantly the running time. For instance, we can reduce MAXCUT to a QCQP, e±ciently. If problem A is reducible to B, we write A B ¹P The reduction implies that B is \as hard" as A. Since we could use an algorithm for problem B to solve A, it cannot be easier. 6 - 17 Computational Complexity P. Parrilo and S. Lall, CDC 2003 2003.12.07.06 NP-hardness A problem B is NP-hard if every problem in NP can be reduced to B. A NP = A B 2 ) ¹P NP-hard problems are as hard as any problem in NP The traveling salesperson problem ² Does this graph have a Hamiltonian path? ² Boolean quadratic optimization ² Satis¯ability ² Is the structured singular value ¹ less than 1? ² NP-hard problems are not necessarily in NP. 6 - 18 Computational Complexity P. Parrilo and S. Lall, CDC 2003 2003.12.07.06 NP-completeness NP-complete = NP NP-hard \ A problem is NP-complete if it is in NP ² every other problem in NP is polynomial-time reducible to it. ² A nontrivial fact: NP-complete problems exist. Satis¯ability is NP-complete (Cook-Levin). ² The proof, essentially writing down the equations for a Turing machine. ² NP-complete problems are the hardest problems in NP. 6 - 19 Computational Complexity P. Parrilo and S. Lall, CDC 2003 2003.12.07.06 Remarks The updated picture: co-NP NP P Co-NPC NPC A statement \Problem X is in NP" means that it is easy in some sense. Problems can be much harder than NP-complete. No e±cient way of verifying either positive or negative solutions! 6 - 20 Computational Complexity P. Parrilo and S. Lall, CDC 2003 2003.12.07.06 The famous cartoon From Garey & Johnson (1979).