(Design And) Analysis of Algorithms NP and Complexity Classes
Total Page:16
File Type:pdf, Size:1020Kb
Intro P and NP Hard problems CSE 548: (Design and) Analysis of Algorithms NP and Complexity Classes R. Sekar 1 / 38 Intro P and NP Hard problems Search and Optimization Problems Many problems of our interest are search problems with exponentially (or even infinitely) many solutions Shortest of the paths between two vertices Spanning tree with minimal cost Combination of variable values that minimize an objective We should be surprised we find ecient (i.e., polynomial-time) solutions to these problems It seems like these should be the exceptions rather than the norm! What do we do when we hit upon other search problems? 2 / 38 Intro P and NP Hard problems Hard Problems: Where you find yourself ... I can’t find an ecient algorithm, I guess I’m just too dumb. Images from “Computers and Intractability” by Garey and Johnson 3 / 38 Intro P and NP Hard problems Search and Optimization Problems What do we do when we hit upon hard search problems? Can we prove they can’t be solved eciently? 4 / 38 Intro P and NP Hard problems Hard Problems: Where you would like to be ... I can’t find an ecient algorithm, because no such algorithm is possible. Images from “Computers and Intractability” by Garey and Johnson 5 / 38 Intro P and NP Hard problems Search and Optimization Problems Unfortunately, it is very hard to prove that ecient algorithms are impossible Second best alternative: Show that the problem is as hard as many other problems that have been worked on by a host of brilliant scientists over a very long time Much of complexity theory is concerned with categorizing hard problems into such equivalence classes 6 / 38 Intro P and NP Hard problems P, NP, Co-NP, NP-hard and NP-complete 7 / 38 Intro P and NP Hard problems Nondeterminism and Search Problems Nondeterminism is an oft-used abstraction in language theory Non-deterministic FSA Non-deterministic PDA So, why not non-deterministic Turing machines? Acceptance criteria is analogous to NFA and NPDA if there is a sequence of transitions to an accepting state, an NDTM will take that path. What does nondeterminism, a theoretical construct, mean in practice? You can think of it as a boundless potential to search for and identify the correct path that leads to a solution So, it does not change the class of problems that can be solved, just the time/space needed to solve. 8 / 38 Intro P and NP Hard problems Class NP: Non-deterministic Polynomial Time How they operate: Guess a solution verify correctness in polynomial time Polynomial time verifiability is the key property of NP. This is how you build a path from P to NP. Ideal formulation for search problems, where correct solutions are hard to find but easy to recognize. Example: Boolean formula satisfiability (SAT) Given a boolean formula in CNF, find an assignment of {true, false} to variables that makes it true. Why not DNF? 9 / 38 Intro P and NP Hard problems What are the bounds of NP? Only Decision problems: Problems with an “yes” or “no” answer Optimization problems are generally not in NP But we can often find optimal solutions using “binary search” “No” answers are usually not verifiable in P-time So, complement of NP problems are often not NP. UNSAT — show that a CNF formula is false for all truth assignments1 Key point: You cannot negate nondeterministic automata. So, we are unable to convert an NDTM for SAT to solve UNSAT in NP-time. 1Whether UNSAT 2 NP is unknown! 10 / 38 Intro P and NP Hard problems What are the bounds of NP? Existentially quantified vs Universally quantified formulas NP is good for 9x P(x): guess a value for x and check if P(x) holds. NP is not good for 8x P(x): Guessing does not seem to help if you need to check all values of x. Negation of existential formula yields a universal formula. No surprise that complement of NP problems are typically not in NP. UNSAT: 8x:P(x) where P is in CNF VALID: 8xP(x), where P is in DNF NP seems to be a good way to separate hard problems from even harder ones! 11 / 38 Algorithms Lecture 30: NP-Hard Problems[Fa’13] is harder than just checking that a solution is correct. But nobody knows how to prove it! The Clay Mathematics Institute lists P versus NP as thefirst of its seven Millennium Prize Problems, offering a $1,000,000 reward for its solution. And yes, in fact, several people have lost their souls attempting to solve this problem. Intro P and NP Hard problems A more subtle but still openCo-NP: question Problems is whether whose the complexity complement classes is NP in and NP co-NP are different. Even if we can verify every YES answer quickly, there’s no reason to believe we can also verify NO answers quickly. For example, as far as we know, there is no short proof that a boolean circuit is not satisfiable. It is generally believed that NP= co-NP,Decision but problems nobody that knows have a polynomially how to prove checkable it. proof when � the answer is “no” ������ � What we think the world looks like. Biggest open problem: Is P = NP? Will also imply co-NP = P 30.3 NP-hard, NP-easy, and NP-complete A problemΠ is NP-hard if a polynomial-time algorithm forΠ would imply a polynomial-time12 / 38 algorithm for every problem in NP. In other words: Π is NP-hard IfΠ can be solved in polynomial time, then P=NP ⇐⇒ Intuitively, if we could solve one particular NP-hard problem quickly, then we could quickly solve any problem whose solution is easy to understand, using the solution to that one special problem as a subroutine. NP-hard problems are at least as hard as any problem in NP. Calling a problem NP-hard is like saying ‘If I own a dog, then it can speakfluent English.’ You probably don’t know whether or not I own a dog, but I bet you’repretty sure that I don’t own a talking dog. Nobody has a mathematical proof that dogs can’t speak English—the fact that no one has ever heard a dog speak English is evidence, as are the hundreds of examinations of dogs that lacked the proper mouth shape and brainpower, but mere evidence is not a mathematical proof. Nevertheless, no sane person would believe me if I said I owned a dog that spokefluent English. So the statement ‘If I own a dog, then it can speakfluent English’ has a natural corollary: No one in their right mind should believe that I own a dog! Likewise, if a problem is NP-hard, no one in their right mind should believe it can be solved in polynomial time. Finally, a problem is NP-complete if it is both NP-hard and an element of NP (or ‘NP-easy’). NP- complete problems are the hardest problems in NP. If anyonefinds a polynomial-time algorithm for even one NP-complete problem, then that would imply a polynomial-time algorithm for every NP-complete problem. Literally thousands of problems have been shown to be NP-complete, so a polynomial-time algorithm for one (and therefore all) of them seems incredibly unlikely. ������� ���� �� ����������� � More of what we think the world looks like. 3 Intro P and NP Hard problems The class Co-NP \ NP Often, problems that are in NP \ co-NP are in P It requires considerable insight and/or structure in the problem to show that something is both NP and co-NP This can often be turned into a P-time algorithm Examples Linear programming [1979] Obviously in NP. To see why it is in co-NP, we can derive a lower bound by multiplying the constraints by a suitable (guessed) number and adding. Primality testing [2002] Obviously in co−NP; See “primality certificate” for proof it is NP Integer factorization? 13 / 38 Algorithms Lecture 30: NP-Hard Problems[Fa’13] is harder than just checking that a solution is correct. But nobody knows how to prove it! The Clay Mathematics Institute lists P versus NP as thefirst of its seven Millennium Prize Problems, offering a $1,000,000 reward for its solution. And yes, in fact, several people have lost their souls attempting to solve this problem. Intro P and NP Hard problems A more subtle but still openNP-hard question and is whether NP-complete the complexity classes NP and co-NP are different. Even if we can verify every YES answer quickly, there’s no reason to believe we can also verify NO answers quickly. For example, as far as weA problem know, thereΠ is NP is-hard no short if the proofavailability that of a a boolean polynomial circuit solution is not satisfiable. It is generally believed that NP= co-NP,to Π will but allow nobodyNP-problems knows to howbe solved to prove in polynomial it. time. � Π is NP-hard , if Π can be solved in P-time, P = NP NP-complete = NP-hard \ NP ������ � What we think the world looks like. 30.3 NP-hard, NP-easy, and NP-complete A problemΠ is NP-hard if a polynomial-time algorithm forΠ would imply a polynomial-time14 / 38 algorithm for every problem in NP. In other words: Π is NP-hard IfΠ can be solved in polynomial time, then P=NP ⇐⇒ Intuitively, if we could solve one particular NP-hard problem quickly, then we could quickly solve any problem whose solution is easy to understand, using the solution to that one special problem as a subroutine.