Strong NP Hardness Approximation Primal-Dual Discrete Optimization 2010 Lecture 10 More about -complete Problems NP Marc Uetz University of Twente [email protected] Lecture 10: sheet 1 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Outline 1 Pseudo-Polynomial Time Algorithms and Strong -hardness NP 2 Approximation Algorithms 3 Primal-Dual Algorithms Lecture 10: sheet 2 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Knapsack Problem Given n items with weights wj 0 and value vj 0 ≥ ≥ a knapsack with total (weight) capacity W Problem find subset S of items with weight W and maximal value ≤ Theorem The Knapsack problem is -hard (Reduction from Partition). NP Lecture 10: sheet 3 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Solving the Knapsack Problem weights wj: 1 2 3 values vj: v1 = 2 v2 = 3 v3 = 4 capacity W = 4 Construct directed graph G = (V ; A): nodes ujk , k = wj ;:::; W , edges( uik ; uj`) with lengths vj i < j and ` k = wj , − size item 1 2 3 4 = W 1 u11 u12 u13 u14 v1 v2 v2 v2 2 s u22 u23 u24 t v3 v3 3 u33 u34 length 0 Lecture 10: sheet 4 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation!"#$%&'%()*%"'(+,%-.*/-%0(1#23-%-#2 Primal-Dual Solving the Knapsack Problem !Observations!"#$%&'(!"#$%&'%(%$)(*+,#-'. )*(&!"%(+#$ ! /+-(0123(12any (s;#t)-path43(5+(.67+( traverses%$5" some-$%$8(,60+(*%5+* `rows' of the. digraph,.9)'(-'%-(0123(".(%,( thereby 9&&+$( 569,0collecting:6$(-'+(.'6$-+.-(&%-'(:$67(.(-6(2(12 the corresponding items #4;.3< ! ='+(,60+(*%5+*.(0123(%$+(.'6$-+.-(&%-'(*+,#-'.(":(%P,0(6,*8(": by definition, all these items fit in the knapsack, wi W 01>3(! 01,3?) :6$(%**(12@>3#A ≤ length of path = total value,%- of items collected along the path longest (s; t)-path = optimal solution Knapsack ) ! " > $ )2> How to compute! longest! paths" in digraphs?" Same* as shortest paths only with -inequality+ 2 4 reversed:(&)d(s; w)) d(s; v) + c)vw %&' ≥ $ (digraph is acyclic,! so we even have# a linear time algorithm) $ # ! # " Lecture 10: sheet 5 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Solving the Knapsack Problem Knapsack is -hard NP Longest Path is polynomially solvable, so 2 P And we have a transformation T : Knapsack Longest Path ! Hence, can we conclude that = and claim 1.000.000 $ from theP ClayNP Mathematics Institute? Lecture 10: sheet 6 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual No 1.000.000 $ (yet). Encoding length of Knapsack O( n log U ), where 2 U := max W ; n; wj ; vj ; j = 1;:::; n = maximal input number f g Transformation, in general, yields digraph withΩ( W ) nodes So in general, not even the size of the digraph is polynomial, and transformation T : Knapsack Longest Path is not poly-time ! n Example: Say, knapsack size W = 2 , and all wj ; vj W , then Knapsack O( n log W ) = O( n2 ) ≤ j j 2 but (as long as there is one item with wj O( 1 )), we have that Digraph Ω( 2n ) 2 j j 2 (and the longest path computation takes Ω( 2n ) time, too) Lecture 10: sheet 7 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Solving the Knapsack Problem Yet, we have an algorithm to (optimally) solve Knapsack, what is it's computation time? Again, let U be the largest number appearing in the input Longest path computation O( n0 + m0 ) (with n0 and m0 the number of nodes & arcs in2 the digraph) n0 O( nU ), and m0 O( (nU)2 ) 2 2 hence computation time O( nU + (nU)2 ) O( (nU)2 ) 2 2 and encoding length Knapsack is Ω( n + log U ) 2 As long as U poly( n ), this is poly-time algorithm for Knapsack 2 In other words: Knapsack no longer -hard if U poly( n ) (unless = ) NP 2 P NP Lecture 10: sheet 8 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Pseudo Polynomial Time Algorithms Let I be instance of problem P (either decision or optimization), I its (binary) encoding length, and number(I ) := the largest numberj j appearing in instance I Definition We say that algorithm A is a pseudo polynomial time algorithm for P if there exists a polynomial function q : R R R such that × ! A solves P tA(I ) q( I ; number(I )) for all instances I P ≤ j j 2 Example: Our previous algorithm for the Knapsack problem Note: Pseudo poly-time is equivalent to poly-time w.r.t. unary encoding of instances of the problem ( I unary number(I )) j j ≥ Lecture 10: sheet 9 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Strongly / Unary -complete Problems NP Let P be a problem, I the (binary) encoding length of any j j instance I P, and f : R R be some nonnegative function, 2 ! denote by Pf the restricted version of problem P Pf = I P number(I ) f ( I ) f 2 j ≤ j j g Definition Problem P is strongly / unary -complete( -hard) if there NP NP exists a polynomial function p such that Pp is -complete ( -hard) NP NP In words: P remains -hard even if `all numbers are restricted to be small', i.e. polynomialNP in I j j Knapsack is -complete, but not strongly -complete NP NP Lecture 10: sheet 10 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Strongly -complete problems: Examples NP (1) Problems that have `no large numbers' anyhow, so where number(I ) I , for example. ≤ j j Clique encoding length Clique is Ω( n + m ) 2 and number(I ) = m (assuming a connected graph) hence Cliquep=Clique is -complete, e.g. for p(x) = x (as number(I ) I for anyNPI Clique) ≤ j j 2 So, Clique is strongly -complete NP Lecture 10: sheet 11 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Strongly -complete problems: Examples NP (2) Problems where -completeness does not depend on the presence of large numbers,NP for example. 0 1 Integer Programming Feasibility (IPF) − m×n m n given A Z , b Z , x 0; 1 with Ax b? 2 2 9 2 f g ≤ encoding length A; b is certainly Ω( n + m ) 2 transformation SAT 0 1 IPF, only requires coefficients 0 and 1 in A and largest! coefficient− in b is m 1 ± − Hence, restricted version 0 1 IPF, where all coefficients in A; b are at most m, is still − -complete NP Hence, 0 1 IPFp is -complete, e.g. for p(x) = x (number(−I ) m INPsuffices for transformation from SAT) ≤ ≤ j j So, 0 1 IPF is strongly -complete − NP Lecture 10: sheet 12 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Strong -completeness NP Theorem A strongly -complete problem P cannot have a pseudo polynomialNP time algorithm, unless = . P NP Proof: Assume the contrary, so P has pseudo poly-time algorithm A, we know polynomial q with 9 tA(I ) q( I ; number(I )) I P ≤ j j 8 2 P is strongly -complete, have polynomial p, such that Pp is NP still -complete (Pp = instances with number(I ) p( I )). NP ≤ j j Now for all instances I Pp, computation time of A is 2 tA(I ) q( I ; p( I )) ; ≤ j j j j so polynomial in I . Hence = . j j P NP Lecture 10: sheet 13 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Outline 1 Pseudo-Polynomial Time Algorithms and Strong -hardness NP 2 Approximation Algorithms 3 Primal-Dual Algorithms Lecture 10: sheet 14 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Dealing with -hard problems NP So your favourite problem P turns out to be -hard. NP Algorithmic Choices 1 Exact Algorithms compute optimum solution, no bound on computation time (Branch & Bound, Cutting Planes, Branch & Cut, . ) 2 Heuristic & Local Search compute some solution, no bound on computation time (Constructive heuristics, and then: Simulated Annealing, Tabu Search, Genetic Algorithms, Ant Colony Optimization, . ) 3 Approximation Algorithms compute solution with (worst-case) performance guarantee do it quickly, i.e., polynomial bound on computation time Lecture 10: sheet 15 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Performance Guarantees !"#$%#&'()"*+,'#'(-"". Definition Given a combinatorial minimization problem P, and α 1, then !"#$%$&$'%()*$+"%),%)'-&$.$/,&$'%)0.$%$.$/,&$'%1)-2'34".)56)≥,%),47'2$&8.)9) algorithm8,:),)0.A;4&$-4$<,&$+"1has performance)-"2#'2.,%<")7;,2,%&"") guarantee α for! #'2)-2'34".)5P if $#) 1 A returns!9)2"&;2%:),):'4;&$'%)90=1) a solution for all instances>))#'2),44)$%:&,%<":I P =) 5 ! 2 ! 2 if A!(I$#)90=1)$:)&8"):'4;&$'%)+,4) = value of As solution;")'#)&8"):'4;&$'%)2"&;2%"?)3@)96 for I , and OPT (I ) = optimal solution,%?)A5B0=1)$:)&8"):'4;&$'%)+, value, then 4;")'#),%)'-&$.,4):'4;&$'%6 A(I ) 90=1)α"OPT! ! A5B0=1(I ) for#'2),44)$%:&,%<":)=) all instances I ! 5P ≤ 2 OPT(I) !#! OPT(I) A(I) Note: For0C".,2D(),??$&$+")-"2#'2.,%<")7;,2,%&"") maximization problems 0 α!())90=1)1 and" A5B0=1)E)A(I ) ! "α=)OPT! 51 (I ) ≤ ≤ ≥ Lecture 10: sheet 16 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Approximation Algorithms Definition Algorithm A is an α-approximation algorithm for problem P if A has a performance guarantee α for P A is a polynomial time algorithm Lecture 10: sheet 17 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Worst-case Ratios Definition We say α is the worst case ratio for algorithm A on minimization (maximization) problem P if A has performance guarantee α for all β < α (β > α), there exist instances I such that A(I ) > βOPT (I ) A(I ) < βOPT (I ) In words: The performance guarantee of A on P is indeed not better than α, or `analysis of A is tight' Lecture 10: sheet 18 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation