Computational Complexity n Classify problems according to the amount of CSE 421: Introduction to computational resources used by the best Algorithms algorithms that solve them n Recall: n worst-case running time of an algorithm NP-completeness n max # steps algorithm takes on any input of size n n Winter 2003 Define: Paul Beame n TIME(f(n)) to be the set of all decision problems solved by algorithms having worst-case running time O(f(n)) 1 2 Decision problems Polynomial time n Computational complexity usually analyzed n Define P (polynomial-time) to be using decision problems n the set of all decision problems solvable by n answer is just 1 or 0 (yes or no). algorithms whose worst-case running time is bounded by some polynomial in the input n Why? size. n much simpler to deal with n deciding whether G has a path from s to t, is certainly no harder than finding a path from s to t k in G, so a lower bound on deciding is also a lower n P = Uk³0TIME(n ) bound on finding n Less important, but if you have a good decider, you can often use it to get a good finder. 3 4 Beyond P? Relative Complexity of Problems n There are many natural, practical n Want a notion that allows us to compare problems for which we don’t know any the complexity of problems polynomial-time algorithms n Want to be able to make statements of the form “If we could solve problem R in n e.g. decisionTSP: polynomial time then we can solve problem L in polynomial time” n Given a weighted graph G and an integer k, does there exist a tour that visits all vertices in G having total weight at most k? “Problem R is at least as hard as problem L” 5 6 1 A Special kind of Polynomial-Time Polynomial Time Reduction Reduction n L £P R if there is an algorithm for L using a ‘black box’ n We will always use a restricted form of (subroutine) that solves R that n Uses only a polynomial number of steps polynomial-time reduction often called Karp n Makes only a polynomial number of calls to a subroutine for or many-one reduction R n Thus, poly time algorithm for R implies poly time 1 algorithm for L n L£P R if and only if there is an algorithm for L n Not only is the number of calls polynomial but the size of the given a black box solving R that on input x inputs on which the calls are made is polynomial! n Runs for polynomial time computing an input T(x) n If you can prove there is no fast algorithm for L, then n Makes one call to the black box for R that proves there is no fast algorithm for R n Returns the answer that the black box gave We say that the function T is the reduction 7 8 Why the name reduction? A geek joke n An engineer n Weird: it maps an easier problem into a n is placed in a kitchen with an empty kettle on the table and told harder one to boil water; she fills the kettle with water, puts it on the stove, turns on the gas and boils water. n she is next confronted with a kettle full of water sitting on the n Same sense as saying Maxwell reduced counter and told to boil water; she puts it on the stove, turns on the problem of analyzing electricity & the gas and boils water. magnetism to solving partial differential n A mathematician n is placed in a kitchen with an empty kettle on the table and told equations to boil water; he fills the kettle with water, puts it on the stove, turns on the gas and boils water. n solving partial differential equations in n he is next confronted with a kettle full of water sitting on the general is a much harder problem than counter and told to boil water: he empties the kettle in the sink, places the empty kettle on the table and says, “I’ve reduced this solving E&M problems to an already solved problem”. 9 10 Reductions from a Special Case to a General Case The Simple Reduction n Show: Vertex-Cover £P Set-Cover n Transformation T maps n Vertex-Cover: (G=(V,E),k) to (U,S1,…,Sm,k’) n Given an undirected graph G=(V,E) and an integer n U¬E k is there a subset W of V of size at most k such n For each vertex vÎV create a set Sv that every edge of G has at least one endpoint in containing all edges that touch v W? (i.e. W covers all edges of G). n k’¬k n Set-Cover: n Reduction T is clearly polynomial-time to compute n Given a set U of n elements, a collection S1,…,Sm of subsets of U, and an integer k, does there exist n We need to prove that the resulting a collection of at most k sets whose union is equal to U? algorithm gives the right answer! 11 12 2 Proof of Correctness Reductions by Simple Equivalence n Two directions: n Show: Independent-Set £P Clique n If the answer to Vertex-Cover on (G,k) is YES then the answer for Set-Cover on T(G,k) is YES n Independent-Set: n If a set W of k vertices covers all edges then n Given a graph G=(V,E) and an integer k, is the collection {Sv | vÎ W} of k sets covers all of there a subset U of V with |U| ³ k such that U no two vertices in U are joined by an edge. n If the answer to Set-Cover on T(G,k) is YES then n Clique: the answer for Vertex-Cover on (G,k) is YES n Given a graph G=(V,E) and an integer k, is n If a subcollection S ,…,S covers all of U then v1 vk there a subset U of V with |U| ³ k such that the set {v1,…,vk} is a vertex cover in G. every pair of vertices in U is joined by an edge. 13 14 Independent-Set £P Clique More Reductions n Given (G,k) as input to Independent-Set n Show: Independent Set £P Vertex-Cover where G=(V,E) n Vertex-Cover: n Given an undirected graph G=(V,E) and an integer n Transform to (G’,k) where G’=(V,E’) k is there a subset W of V of size at most k such has the same vertices as G but E’ that every edge of G has at least one endpoint in consists of precisely those edges that W? (i.e. W covers all edges of G). are not edges of G n Independent-Set: n U is an independent set in G n Given a graph G=(V,E) and an integer k, is there a Û U is a clique in G’ subset U of V with |U| ³ k such that no two vertices in U are joined by an edge. 15 16 Reduction Idea Reduction n Claim: In a graph G=(V,E), S is an n Map (G,k) to (G,n-k) independent set iff V-S is a vertex cover n Previous lemma proves correctness n Proof: n Þ Let S be an independent set in G n Then S contains at most one endpoint of each n Clearly polynomial time edge of G n At least one endpoint must be in V-S n V-S is a vertex cover n We also get that n ÜLet W=V-S be a vertex cover of G n Then S does not contain both endpoints of any n Vertex-Cover £P Independent Set edge (else W would miss that edge) n S is an independent set 17 18 3 Satisfiability Satisfiability n Boolean variables x1,...,xn n CNF formula example n taking values in {0,1}. 0=false, 1=true n (x1 Ú Øx3 Ú x7 Ú x12) Ù ( x2 Ú Øx4 Ú x7 Ú x5) n Literals n If there is some assignment of 0’s and n xi or Øxi for i=1,...,n 1’s to the variables that makes it true n Clause then we say the formula is satisfiable n a logical OR of one or more literals n the one above is, the following isn’t n e.g. (x1 Ú Øx3 Ú x7 Ú x12) n x1 Ù (Øx1 Ú x2) Ù (Øx2 Ú x3) Ù Øx3 n CNF formula n Satisfiability: Given a CNF formula F, is n a logical AND of a bunch of clauses it satisfiable? 19 20 Common property of these problems The complexity class NP n There is a special piece of information, a NP consists of all decision problems where short certificate or proof, that allows you to efficiently verify (in polynomial-time) that the n You can verify the YES answers efficiently YES answer is correct. This certificate might (in polynomial time) given a short be very hard to find (polynomial-size) certificate n e.g. And n DecisionTSP: the tour itself, n Independent-Set, Clique: the set U n No certificate can fool your polynomial time n Satisfiability: an assignment that makes F verifier into saying YES for a NO instance true.