Discrete Optimization 2010 Lecture 10 More About NP-Complete Problems

Home , Knapsack problem

Strong NP Hardness Approximation Primal-Dual

Discrete Optimization 2010 Lecture 10 More about -complete Problems NP

Marc Uetz University of Twente

[email protected]

Lecture 10: sheet 1 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Outline

1 Pseudo-Polynomial Time Algorithms and Strong -hardness NP

2 Approximation Algorithms

3 Primal-Dual Algorithms

Lecture 10: sheet 2 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Knapsack Problem

Given

n items with weights wj 0 and value vj 0 ≥ ≥ a knapsack with total (weight) capacity W

Problem ﬁnd subset S of items with weight W and maximal value ≤

Theorem The Knapsack problem is -hard (Reduction from Partition). NP

Lecture 10: sheet 3 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Solving the Knapsack Problem

weights wj: 1 2 3

values vj: v1 = 2 v2 = 3 v3 = 4 capacity W = 4

Construct directed graph G = (V , A): nodes ujk , k = wj ,..., W , edges( uik , uj`) with lengths vj i < j and ` k = wj ⇔ − size item 1 2 3 4 = W 1 u11 u12 u13 u14 v1 v2 v2 v2 2 s u22 u23 u24 t

v3 v3 3 u33 u34 length 0

Lecture 10: sheet 4 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation!"#$%&'%()*%"'(+,%-.*/-%0(1#23-%-#2 Primal-Dual Solving the Knapsack Problem

!Observations!"#$%&'(!"#$%&'%(%$)(*+,#-'. )*(&!"%(+#$ ! /+-(0123(12any (s,#t)-path43(5+(.67+( traverses%$5" some-$%$8(,60+(*%5+* ‘rows’ of the. digraph,.9)'(-'%-(0123(".(%,( thereby 9&&+$( 569,0collecting:6$(-'+(.'6$-+.-(&%-'(:$67(.(-6(2(12 the corresponding items #4;.3< ! ='+(,60+(*%5+*.(0123(%$+(.'6$-+.-(&%-'(*+,#-'.(":(%P,0(6,*8(": by deﬁnition, all these items ﬁt in the knapsack, wi W 01>3(! 01,3?) :6$(%**(12@>3#A ≤ length of path = total value,%- of items collected along the path longest (s, t)-path = optimal solution Knapsack ⇒ ! " > $ )2> How to compute! longest! paths" in digraphs?" Same* as shortest paths only with -inequality+ 2 4 reversed:(&)d(s, w)) d(s, v) + c)vw %&' ≥ $ (digraph is acyclic,! so we even have# a linear time algorithm) $ # ! # "

Lecture 10: sheet 5 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Solving the Knapsack Problem

Knapsack is -hard NP Longest Path is polynomially solvable, so ∈ P And we have a transformation

T : Knapsack Longest Path →

Hence, can we conclude that = and claim 1.000.000 $ from theP ClayNP Mathematics Institute?

Lecture 10: sheet 6 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual No 1.000.000 $ (yet). . .

Encoding length of Knapsack O( n log U ), where ∈ U := max W , n, wj , vj , j = 1,..., n = maximal input number { }

Transformation, in general, yields digraph withΩ( W ) nodes

So in general, not even the size of the digraph is polynomial, and transformation T : Knapsack Longest Path is not poly-time → n Example: Say, knapsack size W = 2 , and all wj , vj W , then Knapsack O( n log W ) = O( n2 ) ≤ | | ∈ but (as long as there is one item with wj O( 1 )), we have that Digraph Ω( 2n ) ∈ | | ∈ (and the longest path computation takes Ω( 2n ) time, too)

Lecture 10: sheet 7 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Solving the Knapsack Problem

Yet, we have an algorithm to (optimally) solve Knapsack, what is it’s computation time?

Again, let U be the largest number appearing in the input Longest path computation O( n0 + m0 ) (with n0 and m0 the number of nodes & arcs in∈ the digraph) n0 O( nU ), and m0 O( (nU)2 ) ∈ ∈ hence computation time O( nU + (nU)2 ) O( (nU)2 ) ∈ ∈ and encoding length Knapsack is Ω( n + log U ) ∈ As long as U poly( n ), this is poly-time algorithm for Knapsack ∈ In other words: Knapsack no longer -hard if U poly( n ) (unless = ) NP ∈ P NP

Lecture 10: sheet 8 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Pseudo Polynomial Time Algorithms

Let I be instance of problem P (either decision or optimization), I its (binary) encoding length, and number(I ) := the largest number| | appearing in instance I Deﬁnition We say that algorithm A is a pseudo polynomial time algorithm for P if there exists a polynomial function q : R R R such that × → A solves P

tA(I ) q( I , number(I )) for all instances I P ≤ | | ∈

Example: Our previous algorithm for the Knapsack problem

Note: Pseudo poly-time is equivalent to poly-time w.r.t. unary encoding of instances of the problem ( I unary number(I )) | | ≥

Lecture 10: sheet 9 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Strongly / Unary -complete Problems NP Let P be a problem, I the (binary) encoding length of any | | instance I P, and f : R R be some nonnegative function, ∈ → denote by Pf the restricted version of problem P

Pf = I P number(I ) f ( I ) { ∈ | ≤ | | } Deﬁnition Problem P is strongly / unary -complete( -hard) if there NP NP exists a polynomial function p such that Pp is -complete ( -hard) NP NP In words: P remains -hard even if ‘all numbers are restricted to be small’, i.e. polynomialNP in I | | Knapsack is -complete, but not strongly -complete NP NP

Lecture 10: sheet 10 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Strongly -complete problems: Examples NP

(1) Problems that have ‘no large numbers’ anyhow, so where number(I ) I , for example. . . ≤ | | Clique encoding length Clique is Ω( n + m ) ∈ and number(I ) = m (assuming a connected graph)

hence Cliquep=Clique is -complete, e.g. for p(x) = x (as number(I ) I for anyNPI Clique) ≤ | | ∈

So, Clique is strongly -complete NP

Lecture 10: sheet 11 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Strongly -complete problems: Examples NP (2) Problems where -completeness does not depend on the presence of large numbers,NP for example. . . 0 1 Integer Programming Feasibility (IPF) − m×n m n given A Z , b Z , x 0, 1 with Ax b? ∈ ∈ ∃ ∈ { } ≤ encoding length A, b is certainly Ω( n + m ) ∈ transformation SAT 0 1 IPF, only requires coefficients 0 and 1 in A and largest→ coefficient− in b is m 1 ± − Hence, restricted version 0 1 IPF, where all coefficients in A, b are at most m, is still − -complete NP Hence, 0 1 IPFp is -complete, e.g. for p(x) = x (number(−I ) m INPsuffices for transformation from SAT) ≤ ≤ | |

So, 0 1 IPF is strongly -complete − NP

Lecture 10: sheet 12 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Strong -completeness NP

Theorem A strongly -complete problem P cannot have a pseudo polynomialNP time algorithm, unless = . P NP Proof: Assume the contrary, so P has pseudo poly-time algorithm A, we know polynomial q with ∃ tA(I ) q( I , number(I )) I P ≤ | | ∀ ∈

P is strongly -complete, have polynomial p, such that Pp is NP still -complete (Pp = instances with number(I ) p( I )). NP ≤ | | Now for all instances I Pp, computation time of A is ∈

Lecture 10: sheet 13 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Outline

1 Pseudo-Polynomial Time Algorithms and Strong -hardness NP

2 Approximation Algorithms

3 Primal-Dual Algorithms

Lecture 10: sheet 14 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Dealing with -hard problems NP

So your favourite problem P turns out to be -hard. . . NP Algorithmic Choices 1 Exact Algorithms compute optimum solution, no bound on computation time (Branch & Bound, Cutting Planes, Branch & Cut, . . . ) 2 Heuristic & Local Search compute some solution, no bound on computation time (Constructive heuristics, and then: Simulated Annealing, Tabu Search, Genetic Algorithms, Ant Colony Optimization, . . . ) 3 Approximation Algorithms compute solution with (worst-case) performance guarantee do it quickly, i.e., polynomial bound on computation time

Lecture 10: sheet 15 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Performance Guarantees !"#$%#&'()"*+,'#'(-"". Deﬁnition Given a combinatorial minimization problem P, and α 1, then !"#$%$&$'%()*$+"%),%)'-&$.$/,&$'%)0.$%$.$/,&$'%1)-2'34".)56)≥,%),47'2$&8.)9) algorithm8,:),)0.A;4&$-4$<,&$+"1has performance)-"2#'2.,%<")7;,2,%&"") guarantee α for! #'2)-2'34".)5P if $#) 1 A returns!9)2"&;2%:),):'4;&$'%)90=1) a solution for all instances>))#'2),44)$%:&,%<":I P =) 5 ! ∈ ! 2 if A!(I$#)90=1)$:)&8"):'4;&$'%)+,4) = value of As solution;")'#)&8"):'4;&$'%)2"&;2%"?)3@)96 for I , and OPT (I ) = optimal solution,%?)A5B0=1)$:)&8"):'4;&$'%)+, value, then 4;")'#),%)'-&$.,4):'4;&$'%6 A(I ) 90=1)α"OPT! ! A5B0=1(I ) for#'2),44)$%:&,%<":)=) all instances I ! 5P ≤ ∈

OPT(I) !#! OPT(I)

A(I)

Note: For0C".,2D(),??$&$+")-"2#'2.,%<")7;,2,%&"") maximization problems 0 α!())90=1)1 and" A5B0=1)E)A(I ) ! "α=)OPT! 51 (I ) ≤ ≤ ≥

Lecture 10: sheet 16 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Approximation Algorithms

Deﬁnition Algorithm A is an α-approximation algorithm for problem P if A has a performance guarantee α for P A is a polynomial time algorithm

Lecture 10: sheet 17 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Worst-case Ratios

Deﬁnition We say α is the worst case ratio for algorithm A on minimization (maximization) problem P if A has performance guarantee α for all β < α (β > α), there exist instances I such that A(I ) > βOPT (I ) A(I ) < βOPT (I )

In words: The performance guarantee of A on P is indeed not better than α, or ‘analysis of A is tight’

Lecture 10: sheet 18 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual The Vertex Cover Problem (Again)

Vertex Cover Given undirected graph G = (V , E) ﬁnd subset C V of nodes of minimal cardinality such that e C = for all e ⊆E ∩ 6 ∅ ∈ Vertex Cover is (strongly) -hard ( transformation from SAT) NP ∃ Applications: telecommunication!""#$%&'$()*(+*,(-.*/0.1'.23*4(5.1 (when placing antennae), or for example: In street network, place minimum # tollbooths such that any path!"#$#%&'((&#"(&)*'+,#-.$/(#01"1020#$0*2"&#*3#&*..#4**&5%#%2/5#&5$&# can be charged correctly $"6#-$&5#&'$7(..(8 /$"#4(#/5$'9(8#/*''(/&.6:

Lecture 10: sheet 19 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Greedy Algorithm for Vertex Cover

Given G = (V , E), the following seems perfectly reasonable. . .

Algorithm Greedy let W = ∅ while (there are edges left in G) pick v V with largest degree d(v) := δ(v) , W W v eliminate∈ v and all incident edges from G| | ← ∪ return vertex cover W Computation time is O( n2 ), what about the (guaranteed) performance?

Lecture 10: sheet 20 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Analysis Greedy Algorithm

Pm 1 Let Hm = be the mth harmonic number (Hm log m) i=1 i ≈ Theorem

G = (V , E), E = m, Greedy is Hm-approximation algorithm | |

Proof: Say in iteration k we pick vk covering d(vk ) uncovered edges, we 1 say each of these edges e has coste := . Then the solution value of d(vk ) Greedy can be written Xm coste e=1 Let us assume the edges are covered in the order e1,..., em. Then, before the iteration (say iteration k) where edge ej was covered, there are at least m j + 1 uncovered edges left. OPT can cover these edges with − m−j+1 OPT nodes, thus there is a node v left with degree d(v) |OPT | , ≤ | | |OPT | ≥ so edge ej , covered in iteration k, has coste . Therefore, j ≤ m−j+1 m m X X 1 Greedy = coste OPT = Hm OPT ≤ | | m e + 1 | | e=1 e=1 −

Lecture 10: sheet 21 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation!"#$%&'&(!$)*+',-.(/ Primal-Dual Greedy Algorithm for Vertex Cover

Greedy may compute vertex cover containing A, of size 8, where OPT = 5 | | !"#$%&'()*+,*-($$./.*0*1*!*! 2*$%*0*1*!*! 3*" 404*1*56 Lecture 10: sheet 22 / 33 Marc Uetz Discrete Optimization 78'*$9'&)8)*&.*0*1*2*:&'(*404*1*; Strong NP Hardness Approximation Primal-Dual Greedy Algorithm for Vertex Cover !"#$%&'&(!$)*+',-.(/(01*",'"2345

Greedy may compute vertex cover containing A, of size 14, where OPT = 6 | !"#$%&'()*+,*-($$.| /.*0*1*!*! 2**" 303*1+45 67'*$8'&)7)*&.*0*1*2*9&'(*303*1*:*;

Lecture 10: sheet 23 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Greedy Algorithm for Vertex Cover !"#$%&'&(!$)*+',-.(/(01*",'"2345

More generally:!"#$%&'()*+,*-($$. Layers B /.*0*1*!*and C! 2**=" 303*1+45 perfect matching of 2k nodes Layer A: 67'*$8'&)7)*&.*0*1*2*9&'(*303*1*:*; k nodes, each connected to 2 nodes of B b 2 c k nodes, each connected to 3 nodes of B, etc. . . b 3 c . . . 1 node connected to all nodes of B So Greedy may pick vertex cover containing A, total size k k 1 1 1 k + 2 + 3 + + 1 k( 2 + 3 + + k ) = k(Hk 1),where OPTb c= kb c ··· ≥ ··· − | | Lecture 10: sheet 24 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Greedy Algorithm for Vertex Cover

Theorem The worst-case ratio of Greedy is Θ( log n ) for Vertex Cover.

Proof: Have seen that Greedy is an Hm-approximation, and have seen instances where Greedy (Hk 1)OPT , for k = k(n) as in the ≥ − example before, so if α is the worst-case ratio for Greedy,

Hk 1 α Hm − ≤ ≤ 2 Finally, Hm Θ( log m ) O( log(n ) ) = O( log n ) ∈ ∈ 2 and Hk 1 Ω( log k ) = Ω( log(k )) Ω( log n ). (in previous− ∈ example, n 2k + k log k ∈ k2) ≤ ≤

Lecture 10: sheet 25 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Design of Approximation Algorithms

Let P be a (combinatorial) minimization problem, α 1 ≥ The Main Challenge what we want is poly-time algorithm A with

A(I ) αOPT (I ) for all instances I P ≤ ∈ but the trouble is: mostly we don’t know OPT (I )

So what we typically need: some lower bound LB(I ) with

LB(I ) OPT (I ), and A(I ) αLB(I ) ≤ ≤

Lecture 10: sheet 26 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Another Algorithm for Vertex Cover

Now we want to make use of a clever lower bound on OPT ... | | Algorithm A let W = ∅ while (there are edges left in G) pick an arbitrary edge e = v, w let W W v, w { } eliminate← v and∪ {w and} all incident edges from G

Computation time is O( n + m ), what about A’s performance?

Lecture 10: sheet 27 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Analysis Algorithm A

Theorem On any G, algorithm A is a 2-approximation for Vertex Cover

Proof: Observe that OPT M for any matching M in G (size of any matching≥ | is a| lower bound on OPT )

The edges selected by algorithm A are a matching, say MA

A selects both endpoints of any edge, so

A = 2 MA 2OPT | | ≤

Lecture 10: sheet 28 / 33 Marc Uetz Discrete Optimization !"#$%&'&(!$)*+',-.(/(01*",'"2345

!"#$%#&'()*+$%,-"&(.("/0(/(1$%0-(2/0#(%/-,$($3(.(3$%(4$5#(2$6#%73$%(4$5#(2$6#% Strong NP Hardness Approximation Primal-Dual Analysis Algorithm A 8%$$3'( )*%#/59("/6#'().(,0(/(.:/;;%$<,&/-,$4(/*+$%,-"&7 =##5#5'(>40-/42#?0@(>(1"#%#(;#%3$%&/42#(,0(/%A,-%/%,*9Theorem The worst-case ratio of algorithm A is 2.

2*$0#(-$(.(! B8!?>@7Proof:

C$$D(/-(-",0(,40-/42#'

Algorithm E?).@A selects(F(.4(F(. all nodes, half!B8!7 of them would suﬃce

Lecture 10: sheet 29 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Outline

1 Pseudo-Polynomial Time Algorithms and Strong -hardness NP

2 Approximation Algorithms

3 Primal-Dual Algorithms

Lecture 10: sheet 30 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Crucial Idea Behind Algorithm A

Recall, the dual of Vertex Cover is the Matching problem Vertex Cover X min xv v∈V s.t. xv + xw 1 v, w E ≥ ∀ { } ∈ x 0, 1 ∈ { } Matching = Dual of Vertex Cover X max ye e∈E X s.t. ye 1 v V e∈δ(v) ≤ ∀ ∈ x 0, 1 ∈ { }

Lecture 10: sheet 31 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Crucial Idea Behind Algorithm A !"#$%&'()"*+,- ./,01*("2$*3$'$45'/$.6610')"

!"#$!%$! & '% OPT ()*)$$$'% +$', " -$$$# ./0%1,2$! 3

'% ! 041-2$$$$$$# %$! &

LP Relaxation (LP) LP Dual (D)

!"#$!%$! & '% !5'$!.$! 3 6. LP D z ()*)$$$'% +$', " -$$$# ./0%1,2$! 3 z ()*)$$$$!.$! "7%8 6. ! -$$$# %$! &

'% " 4$$$$$$$$$$$$$# %$! & 6. " 4$$$$$$$$$$$$$$$$$# .$! 3

zLP = zD OPT

AnyAny solution(!) feasible of solut theion LP of (D) dual is a islow lowerer bound bound on OPT on OPT ! Aand matching any matching is just M is some nothing solution but a feasible of the solut LPion dualfor (D)

Lecture 10: sheet 32 / 33 Marc Uetz Discrete Optimization Strong NP Hardness Approximation Primal-Dual Crucial Idea Behind Algorithm A

For any minimization problem P, a feasible solution for the dual is a valid lower bound on OPT

So if we can compute a feasible solution x for P, and a feasible solution y for P’s dual, and prove x and y are ‘close’, we’re done

Here, algorithm A computes feasible solutions x = VC for primal and y = M for dual, respectively, such that VC 2 M | | ≤ | | OPT (LP )

Lecture 10: sheet 33 / 33 Marc Uetz Discrete Optimization