© from notes written mostly by Dr. Matt Stallmann: All Rights Reserved CSC 505, Fall 2000: Week 10 Objectives: • understand problem complexity and classes defined by it • understand relationships among decision, witness, evaluation, and optimization problems • understand what it means to reduce one problem to another • understand the meaning of membership in the class NP, and NP-completeness Page 1 of 21 © from notes written mostly by Dr. Matt Stallmann: All Rights Reserved A complexity class (a class of problems that are related based on the efficiency -- time or space -- of the best possible algorithms for them) Class P is the class of all decision problems that can be solved in time polynomial in the input size. Questions: 1. Why polynomial? (i.e. why is P an important class?) 2. Why decision problems? 3. How do we count time and input size? Page 2 of 21 © from notes written mostly by Dr. Matt Stallmann: All Rights Reserved MST as a decision problem/language MST decision problem: Given a weighted,undirected graph G = (V,E,w) and an integer k, does G have a spanning tree whose total weight is k? LMST = { strings z | z encodes a graph G = (V,E,w) and an integer k and G has a spanning tree T with w(T) k } 1s @ 45 @ @ G = s @s 238 z = 1#10#100##1#11#101##10#11#1000###111 Do the details of the encoding matter? Page 3 of 21 © from notes written mostly by Dr. Matt Stallmann: All Rights Reserved Rules for this particular encoding alphabet = {0,1,#} xed, nite hinstancei ::= hgraphi###hinti G and k hgraphi ::= hedgei{##hedgei} list of edges hedgei ::= hvertexi#vertexi#hinti 2 ends and weight hvertexi ::= hinti vertex names are ints hinti ::= hbiti{hbiti} ints are in binary hbiti ::= 0 | 1 12s 8 s ¨ ¨¨ 659¨ ¨¨ Another example: G = s¨ s k =18 347 1#10#1000##1#11#110##11#10#1001##11#100#111##10#100#101###10010 Page 4 of 21 © from notes written mostly by Dr. Matt Stallmann: All Rights Reserved Types of combinatorial problems Optimization a combinatorial object (set, sequence, .. ) of optimum (min/max) weight Example: Output a minimum spanning tree T of G = (V,E,w). Evaluation the weight of an optimum object Example: Output w(T ), the weight of a minimum spanning tree T of G = (V,E,w). Decision yes/no (does an object of a certain weight or better exist) Example: Given G = (V,E,w) and an integer k, does there exist a spanning tree T such that w(T) k? Witness a certicate for the decision problem, i.e. proof of a yes answer if such proof exists Example: Given G = (V,E,w) and an integer k, output a spanning tree T with w(T) k or report that none exists. Page 5 of 21 © from notes written mostly by Dr. Matt Stallmann: All Rights Reserved Are the problem types related? If you can solve the optimization problem (in polynomial time), you can solve the evaluation, decision, and witness problems (in polynomial time). (Obvious. Why?) If you can solve the decision problem, you can solve the evaluation problem. (Binary search on the interval of pos- sible solution values) If you can solve both the evaluation and the witness prob- lem, you can solve the optimization problem. (Easy. How?) The missing link is how to solve the witness problem using the decision problem. Page 6 of 21 © from notes written mostly by Dr. Matt Stallmann: All Rights Reserved Self-reducibility w Graph G has a spanning tree u v with weight k . Can you determine if edge (u,v) is in such a spanning tree? Your only resource is a “black box” that allows you to input a graph and a weight and will output “yes” if there is a spanning tree with that weight or less, no otherwise. You can perform modifications on G but you can’t “look” at it (i.e. you can put a modified form of G into the box). Page 7 of 21 © from notes written mostly by Dr. Matt Stallmann: All Rights Reserved Using the decision problem to solve the witness problem function MST-Witness(G, k) is . returns a spanning tree of G with weight k, . if one exists. Pick an arbitrary u ∈ V [G] for each v ∈ Adj[u] do if MST-Decision(G/uv, k w(uv)) then return MST-Witness(G/uv, k w(uv)) . G/uv is G with u and v joined into one vertex. ERROR: no such tree end MST-Witness Time bound, i.e. number of times MST-Decision is called? Page 8 of 21 © from notes written mostly by Dr. Matt Stallmann: All Rights Reserved Non-deterministic “algorithm” for MST-Decision function MST-Decision(G, k) is . returns true i a spanning tree of G with weight k exists Choose n 1 edges to form a subset T E[G] Use DFS (for example) to check that T is a spanning tree (quit when a cycle is found or there’s more than one tree). Add up edge costs and check that the total is k. if T passes both tests then return true else return false end MST-Witness Guess a certicate and then check the validity of the certicate. Page 9 of 21 © from notes written mostly by Dr. Matt Stallmann: All Rights Reserved What is nondeterminism and what is NP? How nondeterminism works. If there exists a choice that leads to a ”yes” answer, the algorithm answers ”yes” (true) for that instance. “magic” — no computational device works this way. Why nondeterminisim is useful. Algorithms are simple. Algorithms can be simulated by deterministic ones. It is a useful way to classify problems. N P is the class of all decision problems that can be solved by a nondeterministic algorithm (guess a certicate and check) in polynomial time (certicate has polynomial size; checking takes polynomial time). Compare with text! Page 10 of 21 © from notes written mostly by Dr. Matt Stallmann: All Rights Reserved P and NP: what’s all the fuss about? Traveling salesperson problem NP Polynomial algorithm? ?? Non-polynomial lower bound? At least as hard as... P Minimum spanning-tree problem TSP: Given an n x n distance matrix D (where D[i,j] = distance from city i to city j) and an integer K, does there exist a permutation p of {1,...,n} so that D[p(n),p(1)] + sum i from 1 to n–1 of D[p(i),p(i+1)] is at most K? Page 11 of 21 © from notes written mostly by Dr. Matt Stallmann: All Rights Reserved Reduction: a confusing concept A reduces to B means A can besolvedin terms of B, or A is at least as easy as B. In divide-and-conquer, greedy, dynamic and programming: re- duce (an instance of) a problem to (a smaller instance of) itself. Witness problem can be solved in terms of decision problem. An instance of transitive closure is reduced to several instances of matrix multiplication and 3 smaller instances of transitive closure. So an O(n)-time algorithm for matrix multiplication with 2 implies an O(n)-time algorithm for transitive closure. But A reduces to B can also mean “B is at least as hard as A”. If transitive closure can’t be solved in O(n) time for some 2, neither can matrix multiplication! If A is known to be “hard” (NP-complete) and A reduces (in polynomial time) to B, then B is also hard (NP-complete). Page 12 of 21 © from notes written mostly by Dr. Matt Stallmann: All Rights Reserved Reduction: technical details (Decision) problem P1 reduces in polynomial time to prob- lem P2, notation P1 P P2, if there exists a polynomial- time reduction algorithm F that transforms an arbitrary instance of P1 to an instance of P2 so that F (x) has a yes answer in P2 if and only if x has a yes answer in P1. Simple example: PATH P SR. An instance of PATH is a directed, unweighted graph G = (V,E) and two vertices s and t — answer yes if there is a path from s to t in G. An instance of SR (shortest route) is an nn distance ma- trix D = {D[i, j]}, two “cities” s and t, and an integer B — answerP yes if there is a sequence s = x0, x1, ...,xk = t k1 so that i=0 D[xi,xi+1] B. How does the reduction algorithm work? Page 13 of 21 © from notes written mostly by Dr. Matt Stallmann: All Rights Reserved Proof of correctness for the reduction Let F (G, s, t) be dened by the following: 1. Number V [G] from 1 to n, where n = |V [G]|. 2. For each pair i, j, if ij ∈ E[G], let D[i, j] =0, otherwise let D[i, j] = 1. Let B = 0 and output D,s,t,B. Claim: There is a path from s to t in G i there is a path with sum of all edges = 0 from s to t using the distance matrix D. Page 14 of 21 © from notes written mostly by Dr. Matt Stallmann: All Rights Reserved Another reduction: TSP to Longest simple path TSP P LSP. An instance of LSP (longest simple path) is a weighted, directed graph G = (V,E,w), two vertices s and t, and an integer bound B — answer yes if G has a path P from s to t with w(P ) B.