NP-Completeness : Concepts ••• Why Studying NP-Completeness ? ♣ Pursuing your Ph.D. ♣ Keeping your job Before studying NP-completeness : “I can’t find an efficient algorithm, I guess I’m just too dumb.” 1 After studying NP-completeness : “I can’t find an efficient algorithm, because no such algorithm is possible !” “I can’t find an efficient algorithm, but neither can all these famous people.” 2 ••• Measure to Time Complexity l : measure to the time complexity of an algorithm The discussion of NP-completeness considers l the input size, i.e., the total length of all inputs to the algorithm. Two assumptions : (1) all inputs are integers (a rational number can be represented by a pair of integers); (2) each integer has a binary representation. Ex. Sorting a1, a1, …, an. n l = (loga + 1 ) . ∑ 2 i i=1 3 Ex. Consider the following procedure. input (n); s ←←← 0; for i ←←← 1 to n do s ←←← s + i; output (s). l = log 2 n + 1. l The procedure takes O(n) = O(2 ) time. ⇒ an exponential-time algorithm ! 4 ••• Polynomial-Time Algorithms vs. Exponential-Time Algorithms Suppose that your computer takes 1 second to perform 10 6 operations. The following is the time requirement for your computer to perform f(n) operations, where 2 3 5 n n f(n) = n, n , n , n , 2 , 3 and n = 10, 20, 30, 40, 50, 60. 5 The following shows the largest value of n such that f(n) operations can be performed in 1 hour on a faster computer. 6 An algorithm is referred to as a polynomial-time algorithm, if its time complexity can be bounded above by a polynomial function of input size. An algorithm is referred to as an exponential-time algorithm , if its time complexity cannot be thus bounded (even if the function is not normally log n regarded as an exponential one, like n ). Usually, a problem is referred to as tractable if it can be solved with a polynomial-time algorithm, and intractable otherwise. The two tables above give us a reason why polynomial-time algorithms are much more desirable than exponential-time algorithms. They also motive us to study the theory of NP-completeness. 7 ••• Maximal vs. Maximum Ex. maximal cliques : {1, 2, 3}, {2, 3, 4, 5}, {4, 6} maximum cliques : {2, 3, 4, 5} 8 ••• Decision Problems vs. Optimization Problems A decision problem asks a solution of “yes ” or “no ”. An optimization problem asks a solution of an optimal value (a maximum or a minimum). Ex. The maximum clique problem can be expressed as a decision problem as follows. Instance : An undirected graph G = (V, E) and a positive integer k ≤≤≤ |V|. Question : Does G contain a clique of size ≥≥≥ k ? It can be also expressed as an optimization problem as follows. Instance : An undirected graph G = (V, E). Question : What is the size of a maximum clique of G ? 9 Ex. The traveling salesman problem can be expressed as a decision problem as follows. Instance : A set C of m cities, distances di,j > 0 for all pairs of cities i, j ∈∈∈ C, and a positive integer k. Question : Is there a tour of length ≤≤≤ k that starts at any city, visits each of the other m −−− 1 cities exactly once, and returns to the initial city ? It can be also expressed as an optimization problem as follows. Instance : A set C of m cities and distances di,j > 0 for all pairs of cities i, j ∈∈∈ C. Question : What is the length of a shortest tour that starts at any city, visits each of the other m −−− 1 cities exactly once, and returns to the initial city ? 10 Ex. The problem of sorting a1, a1, …, an can be expressed as a decision problem as follows. Instance : Given a1, a2, …, an and a positive integer k. Question : Is there a permutation of a1, a2, …, an, denoted by a’1, a’2, …, a’n, such that |a’2 −−− a’1| + |a’3 −−− a’2| + … + |a’n −−− a’n−−−1| ≤≤≤ k ? An optimization problem is “harder” than its corresponding decision problem. Since the NP-completeness concerns whether or not a problem can be solved in polynomial time, the discussion of NP-completeness considers only decision problems. (If a decision problem is not polynomial-time solvable, then its corresponding optimization problem is not polynomial-time solvable either.) 11 ••• Problem Reduction A problem P1 reduces to another problem P2, denoted by P1 ∝∝∝ P2, if any instance of P1 can be transformed into an instance of P2 such that the solution for P1 can be obtained from the solution for P2. T∝∝∝ : the reduction time. T : the time required to obtain the solution for P1 from the solution for P2. Since the NP-completeness concerns whether or not a problem can be solved in polynomial time, we consider only the reductions with both T∝∝∝ and T polynomial. (Thus, P2 ∈∈∈ P ⇒⇒⇒ P1 ∈∈∈ P or P1 ∉∉∉ P ⇒⇒⇒ P2 ∉∉∉ P.) If P1 ∝∝∝ P2 and P2 ∝∝∝ P3, then P1 ∝∝∝ P3. 12 ••• P, NP, and NP-Complete Three classes of decision problems : P, NP, and NP-complete. P : the set of decision problems that can be solved in polynomial time by deterministic algorithms. NP : the set of decision problems that can be solved in polynomial time by non-deterministic algorithms. Any non-deterministic algorithm consists of two phases : guessing and checking . 13 For the maximum clique problem, the guessing phase will return a clique, and the checking phase will decide whether or not the clique size is greater than or equal to k. For the traveling salesman problem, the guessing phase will return a tour, and the checking phase will decide whether or not the tour length is greater than or equal to k. A decision problem has an AFFIRMATIVE answer. ⇔ The guessing is SUCCESSFUL. Notice that non-deterministic algorithms are imaginary. A more detailed description of non- deterministic algorithms and more illustrative examples can be found in Ref. (2). 14 Every decision problem in P is also in NP, i.e., P ⊆⊆⊆ NP. An NP problem is NP-complete if every NP problem can reduce to it in polynomial time. ⇒⇒⇒ If any NP-complete problem can be solved in polynomial time, then every NP problem can be solved in polynomial time (i.e., P = NP). (Intuitively, NP-complete problems are the “hardest” problems in NP.) It is one of the most famous open problems in computer science whether P ≠≠≠ NP or P = NP. 15 When P ≠≠≠ NP, NP NP-Complete P (There exist problems in NP that are neither in P, nor in NP-complete (see Chap. 7 in Ref. (1).) When P = NP, P = NP = NP-Complete Almost all people believe P ≠≠≠ NP. 16 A problem is NP-hard if an NP-complete problem can be reduced to it in polynomial time. (Equivalently, a problem is NP-hard if every NP problem can be reduced to it in polynomial time.) ⇒⇒⇒ If any NP-hard problem can be solved in polynomial time, then all NP-complete problems can be solved in polynomial time. (Intuitively, NP-hard problems are “harder” than NP-complete problems.) NP NP-hard NP-complete The class of NP-hard problems contains both decision problems and optimization problems. 17 If an NP-hard problem is in NP, then it is an NP-complete problem. (Intuitively, NP-complete problems are an “easier” subclass of NP-hard problems.) The corresponding optimization problems of NP-complete problems are NP-hard. The well-known halting problem (a decision problem), which is to determine whether or not an algorithm will terminate with a given input, is NP-hard, but not NP-complete. 18 ••• Pseudo-Polynomial Time Algorithms Ex. Given a set S = {a1, a1, …, an} of integers and an integer M > 0, the sum -of -subset problem is to determine whether or not there exists a subset of S whose sum is equal to M. This problem can be solved in O(nM ) time by dynamic programming as follows. Let t(i, j) = true , if there exists a subset of {a1, a2, …, ai} whose sum is equal to j, and false else. Then, t(i, j) = t(i −−− 1, j) + t(i −−− 1, j −−− ai), where i > 1. Initially, t(1, j) = true , if j = 0 or j = a1, and false else. The answer is t(n, M). 19 Although the time complexity is exponential with respect to M, the problem is considered polynomial-time solvable, if M is bounded. An algorithm like this is usually referred to as a pseudo-polynomial time algorithm . An NP-complete problem is in the strong sense if and only if there exists no pseudo-polynomial time algorithm for solving it (unless P = NP). Intuitively, NP-complete problems in the strong sense are “harder” NP-complete problems (refer to Ref. (1)). 20 ••• The Satisfiability Problem and Cook’s Theorem The satisfiability problem, which is the first NP-complete problem, is defined as follows. Instance : A set U of Boolean variables and a collection C of clauses over U. Question : Is there an assignment of U that can satisfy C ? Ex. When U = {x1, x2, x3} and C = {x1 ∨∨∨ x2 ∨∨∨ x3, x , x }, the assignment of U : x ←←← F, 1 2 1 x2 ←←← F and x3 ←←← T, can satisfy C (i.e., (x ∨∨∨ x ∨∨∨ x ) ∧∧∧ ( x ) ∧∧∧ ( x ) = T). 1 2 3 1 2 21 Ex. When U = {x , x } and C = {x ∨∨∨ x , x ∨∨∨ x , 1 2 1 2 1 2 x ∨∨∨ x , x ∨∨∨ x }, no assignment of U can 1 2 1 2 satisfy C. Cook’s Theorem : The satisfiability problem is NP-complete.