Problem Analysis and Complexity Theory SS16

Institute for Software Technology Graz University of Technology

Welcome to ... Problem Analysis and Complexity Theory 716.054, 3 VU Birgit Vogtenhuber Institute for Software Technology Graz University of Technology email: [email protected] office: Inffeldgasse 16B/II, room IC02044 office hour: Wednesday 10:30-11:30 slides: http://www.ist.tugraz.at/pact16.html Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 1 Institute for Software Technology Graz University of Technology Last Time finished Space Complexity • showed that TQBF is PSPACE-complete • made some space-summary discussed Questions

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 2 Institute for Software Technology Graz University of Technology Last Time started Optimization and Approximation • Discussed Decision vs. Optimization • Formalized the notion of Approximation: n v(x,y) opt(x) o ◦ approximation ratio R(x, y) = max opt(x) , v(x,y) ◦ defined r(n)-approximation algorithm • Defined complexity classes for optimization problems: PO, NPO, APX(r(n)), APX, APX∗ Question:• ConsideredHow the are following these classes optimization defined? problems: Question:MIN VERTEXWhat COVER, are their containmentMAX (k-DEG.) relations? INDEP. SET

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 3-6 Institute for Software Technology Graz University of Technology Last Time started Optimization and Approximation • Discussed Decision vs. Optimization • Formalized the notion of Approximation: n v(x,y) opt(x) o ◦ approximation ratio R(x, y) = max opt(x) , v(x,y) ◦ defined r(n)-approximation algorithm • Defined complexity classes for optimization problems: PO, NPO, APX(r(n)), APX, APX∗ • ConsideredShowed that the for following some NP optimization-hard problems, problems: the MINoptimization VERTEX problems COVER, can MAX be efficiently (k-DEG.) approximated INDEP. SET • For some factors, approximation may still be NP-hard

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 3-7 Institute for Software Technology Graz University of Technology Last Time started Optimization and Approximation • Discussed Decision vs. Optimization • Formalized the notion of Approximation: n v(x,y) opt(x) o ◦ approximation ratio R(x, y) = max opt(x) , v(x,y) ◦ deﬁned r(n)-approximation algorithm • Deﬁned complexity classes for optimization problems: PO, NPO, APX(r(n)), APX, APX∗ • Considered the following optimization problems: MIN VERTEX COVER, MAX (k-DEG.) INDEP. SET Question: What did we show how for these problems? Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 3-10 Institute for Software Technology Graz University of Technology This Time / Next Time continue Optimization and Approximation • Repeat some essentials • Consider MIN TSP, MIN TSP∆ and MAX KNAPSACK • Introduce problems with small solution values • Reductions for approximation problems • New concept / class PCP: Probabilistically Checkable Proofs • Topics for Presentations

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 4 Institute for Software Technology Graz University of Technology Optimization Problems Definition: An optimization problem A can be defined as a 4-tuple A = (I, S, v, d) where • I is the set of instances of A. • S(x): x∈I is the set of solutions for instance x. • v(x, s): s∈S(x) is the value of soution s for instance x. • d∈{min, max} is the optimization direction. Definition: The class of all optimization problems whose decision variant is in NP is called NPO. ⇒ x∈I decidable in polynomial time. ⇒ y ∈S(x) decidable in polynomial time and |y| ≤ p(|x|). ⇒ v(x, s) computable in polynomial time.

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 5 Institute for Software Technology Graz University of Technology Measuring Approximation Definition: For an optimization problem A = (I, S, v, d), an instance x∈I, and a solution y ∈S(x), the approximation ratio of y with respect to x is defined as n v(x, y) opt(x) o R(x, y) := max , ≥ 1 opt(x) v(x, y) | {z } | {z } d=min d=max Definition: Let r : lN → [1, ∞) with r(n + 1) ≥ r(n). M is a r(n)-approximation algorithm for A if ∀x ∈ I : M(x) ∈ S(x) and R(x, M(x)) ≤ r(|x|).

The worst case approximation ratio rM (n) of M is deﬁned as rM (n) := sup{R(x, M(x)) : |x| ≤ n}. Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 6 Institute for Software Technology Graz University of Technology The Class APX Deﬁnition: Let r : lN → [1, ∞) with r(n + 1) ≥ r(n). • The complexity class APX(r(n)) contains all optimization problems which can be approximated by a polynomial-time algorithm M with worst-case approximation ratio rM (n) ≤ r(n). S • APX = c≥1 APX(c) is the class of optimization problems admitting a polynomial-time constant factor approximation algorithm for some constant c ≥ 1. ∗ T • APX = c>1 APX(c) is the class of optimization problems admitting a polynomial-time constant factor approximation algorithm for any constant c > 1.

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 7 Institute for Software Technology Graz University of Technology Optimization Problems • Considered several optimization problems: Problem Instance Parameter APX MIN V.C. graph |vertex cover| MAX CLIQUE graph |clique| APX?MAX IND. SET graph |indep. set| MAX2SAT 2CNF formula |sat. clauses| APX MIN TSP weighted graph cost of tour MAX KNAPSACK objects, t. weight total value

Question: Whatif P is6= theNP relation between APX and APX∗? Question: Which of the problems are in APX, APX∗? Make your guess list!

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 8 Institute for Software Technology Graz University of Technology MIN TSP TSP / MIN TSP: the Travelling Salesman Problem • Problem: How eﬃcient can a salesman make a round trip visiting given locations? • Decision problem: Is it possible to visit all locations of L within time t?

• Problem instance: A set L = {l1, . . . ln} of locations, with travel costs / times (Ti,j) between them, plus a maximum total time t. Theorem: TSP is (strongly) NP-complete.

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 9-2 Institute for Software Technology Graz University of Technology MIN TSP TSP / MIN TSP: the Travelling Salesman Problem • Problem: How eﬃcient can a salesman make a round trip visiting given locations? • Optimization problem: What is the minimum time t needed for a round trip that visits all locations of L?

• Problem instance: A set L = {l1, . . . ln} of locations, with travel costs / times (Ti,j) between them, plus a maximum total time t. Theorem: TSP is (strongly) NP-complete. Corollary: MIN TSP is in NPO.

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 9-4 Institute for Software Technology Graz University of Technology MIN TSP and APX Theorem: If P 6= NP then MIN TSP is not in APX. Proof: X • Assume there exists a polynomial time c-approximation algorithm A for MIN TSP for some constant c > 1. • Given a directed graph G = (V,E), we construct an input for A by assigning weights to the edges of the complete graph on V : 1 if (i, j) ∈ E T = i,j c|V | if (i, j) 6∈ E ⇒ If G contains a Hamiltonian cycle then A(G)≤c|V |. Otherwise A(G)≥|V |−1+c|V |>c|V |. ⇒ Algorithm A solves HAMCYCLE in polynomial time.

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 10 Institute for Software Technology Graz University of Technology The Gap Technique Theorem: Given an optimization problem such that 1. for every input x ∈ I and every solution s ∈ S(x), s does not lie in the interval (a, b), i.e., either v(s, x) ≤ a or v(s, x) ≥ b, and 2. it is NP-hard to determine whether opt(x) ≤ a or opt(x) ≥ b. then it is NP-hard to obtain a worst-case approximation ratio that is smaller than b/a. Proof: Exercise. Question: What was the gap in our proof for MIN TSP 6∈ APX if P 6= NP?

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 11-3 Institute for Software Technology Graz University of Technology The Gap Technique Theorem: Given an optimization problem such that 1. for every input x ∈ I and every solution s ∈ S(x), s does not lie in the interval (a, b), i.e., either v(s, x) ≤ a or v(s, x) ≥ b, and 2. it is NP-hard to determine whether opt(x) ≤ a or opt(x) ≥ b. then it is NP-hard to obtain a worst-case approximation ratio that is smaller than b/a. Proof: Exercise. Remark: The proof for MIN TSP 6∈ APX if P 6= NP used a gap of (|V |, c|V | + |V | − 1).

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 11-4 Institute for Software Technology Graz University of Technology MIN TSP with ∆-Inequality Just proved: MIN TSP does not have polynomial time constant factor approximation for any constant factor c. Question: Does this imply that route planning is completely intractible? ∆ MIN TSP : Given set L = {l1, . . . ln} of locations, with travel times (Ti,j) between them, such that

∀li, lj, lk ∈ L : Ti,j + Tj,k ≥ Ti,k, ljwhat is the minimum time t needed for a Tj,k Ti,j round trip that visits all locations of L? lk T li i,k Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 12-4 Institute for Software Technology Graz University of Technology MIN TSP with ∆-Inequality Just proved: MIN TSP does not have polynomial time constant factor approximation for any constant factor c. Question: Does this imply that route planning is completely intractible? ∆ MIN TSP : Given set L = {l1, . . . ln} of locations, with travel times (Ti,j) between them, such that

∀li, lj, lk ∈ L : Ti,j + Tj,k ≥ Ti,k, what is the minimum time t needed for a round trip that visits all locations of L?

Problem instance: A set L = {l1, . . . ln} of locations and a matrix (Ti,j) of travel times. Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 12-6 Institute for Software Technology Graz University of Technology MIN TSP with ∆-Inequality Question: What about MIN TSP∆ and APX? Idea: Use algorithm for minimum spanning tree (in PO) tour(L, Ti,j):

1. Find a minimum spanning tree S for L, Ti,j 2. Traverse S with depth ﬁrst search, jumping over already visited vertices (via direct edges)

Claim 1: tour(L, Ti,j) really returns a tour. Claim 2: |minimum spanning tree|≤| cheapest tour|

Claim 3: |tour(L, Ti,j)| ≤ 2·| minimum spanning tree| Corollary: MIN TSP∆ has a polynomial-time 2-approximation and hence is in APX. Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 X 12-14 Institute for Software Technology Graz University of Technology PTAS and FPTAS Definition: An algorithm M for an optimization problem A = (I, S, v, d) is an approximation scheme for A if ∀x ∈ I, ∀ε > 1: R(x, M(x, ε)) ≤ ε. Definition: An optimization problem is in the class PTAS if it has a polynomial-time approximation scheme. Question: What’s the relation APX∗ versus PTAS? Question: What’s the running-time of a PTAS algorithm? Definition: An optimization problem is in the class FPTAS (= fully PTAS) if it has an approximation scheme M for which the running time for M(x, ε) is bounded by some polynomial p(|x|, 1/(1 − ε)). Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 13 Institute for Software Technology Graz University of Technology MAX KNAPSACK • Problem: What should be packed into hand luggage as there is a restriction on the total weight? • Problem instance: A maximum weight W ∈ lN and

a (multi)-set P of n objects pi = (vpi , wpi ) where

◦ vpi ∈ lN is the value of pi and

◦ wpi ∈ lN is the weight of pi. • Optimization problem: What is the maximum value V such that P contains a subset Y ⊆ P with P P y∈Y vy = V and y∈Y wy ≤ W ?

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 14 Institute for Software Technology Graz University of Technology MAX KNAPSACK ∈ FPTAS Theorem: MAX KNAPSACK is in FPTAS. Proof: 0. Recall the pseudopolynomial algorithm which we had for the decision problem: V (w, i) = maximum possible value of a subset of the ﬁrst i objects of P which has weight exactly w:

V (w, i + 1) = max{V (w, i), vi+1 + V (w−wi+1, i)} with V (w, 0)=0 ∀ w ≥ 0 and V (w, i)=−∞ ∀ w < 0. For a packing of value at least V with objects of P , it suﬃces to compute at most the ﬁrst n × W entries of the table V (w, i) (stopping if V (w, i) ≥ V ). O(nW ) with dynamic programming. Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 15-3 Institute for Software Technology Graz University of Technology MAX KNAPSACK ∈ FPTAS Theorem: MAX KNAPSACK is in FPTAS. Proof: 1. We mimic this algorithm, inverting roles of V and W :

Let V := max{v1, . . . , vn}. For i ∈ {1, . . . , n} and v ∈ {0, . . . , nV }, let W (i, v) be the minimal weight when taking exactly value v from the ﬁrst i objects:

W (i + 1, v) = min{W (i, v),W (i, v − vi+1) + wi+1} | {z } | {z } pi+16∈ pi+1∈ with W (i, 0) := 0 ∀ i and W (0, v) := ∞ ∀ v 6= 0 ⇒ maximum value via dynamic prog. in O(n2V ) ⇒ again a pesudopolynomial algorithm

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 15-7 Institute for Software Technology Graz University of Technology MAX KNAPSACK ∈ FPTAS Theorem: MAX KNAPSACK is in FPTAS. Proof: 2. We buy higher speed for smaller precision. 0 b b We make a new instance with vi := 2 bvi/2 c, i.e. we disregard the b least signiﬁcant bits of each value. By this we obtain • an approximation for the original instance • in timeO( n2V/2b). Question: Why does this give a valid solution?

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 15-11 Institute for Software Technology Graz University of Technology MAX KNAPSACK ∈ FPTAS Theorem: MAX KNAPSACK is in FPTAS. Proof: 3. We estimate the worst case approximation factor: Let Y = opt. solution, Y 0 = approx. solution. Then P P 0 0 i∈Y 0 vi ≥ i∈Y 0 vi ⇐ vi ≥ vi ≥ P v0 ⇐ Y 0 is opt for x0 0 i∈Y i val(Y ) P b 0 b ≥ i∈Y (vi − 2 ) ⇐ vi ≥ vi − 2 P b opt(x) ≥ i∈Y vi − n2 ⇐ |Y | ≤ n P With wi ≤ W ∀ i we obtain i∈Y vi = opt(x) ≥ V > 0 and thus R(x, Y 0) = opt(x) ≤ opt(x) ≤ V val(Y 0) opt(x)−n2b V −n2b

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 15-19 Institute for Software Technology Graz University of Technology MAX KNAPSACK ∈ FPTAS Theorem: MAX KNAPSACK is in FPTAS. Proof: V ⇒ we have an ε-approximation algorithm with ε := V −n2b .

l V (ε−1) m 4. Choosing b := log( nε ) , we obtain a time bound of 2 b n3ε O(n V/2 ) = O( (ε−1) ), which for 1 < ε ≤ const. is 1 polynomial in n and (ε−1) . ⇒ the designed algorithm is a fully polynomial time approximation scheme. X

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 15-23 Institute for Software Technology Graz University of Technology Problems with Small Solutions • We have just seen that MAX KNAPSACK ∈ FPTAS. • We have also seen that MAX KNAPSACK is pseudopolynomial: If the maximum value V is polynomial, then so is the optimum solution, and the problem can be solved in polynomial time. ⇒ What about problems that are also complicated for small solution values? Especially, optimization problems whose decision variants are strongly NP-hard? Deﬁnition: An optimization problem is a problem with small solution values if all solutions are positive integers with values polynomially bounded in the input length.

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 16-5 Institute for Software Technology Graz University of Technology Problems with Small Solutions Theorem: If P6= NP, then no NP-hard problem with small solution values has an FPTAS. Proof: Exercise. • Assume existence of an FPTAS A. • Let |x| = n and p(n) be a polynomial bound for the solution values for x, and let ε(n) := 1/p(n) + 1 ⇒ A(x, ε(n)) runs in q(n, 1/(ε(n) − 1)) = q(n, p(n)) time, which is polynomial in n. ⇒ A(x, ε(n)) can’t be the optimal solution for all inputs x. ⇒ As all solutions ∈ {1, . . . , p(n)}, every non-optimal solution gives a worst case approximation ratio of at least p(n)/(p(n) − 1) = 1 + 1/(p(n) − 1) > ε(n). Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 16-10 Institute for Software Technology Graz University of Technology Approximation Results Approximation results we have seen by now: • MAX KNAPSACK ∈ FPTAS • MIN TSP 6∈ APX if P6= NP: gap-argument with HAMCYCLE • MIN TSP∆ ∈ APX: 2-approximation • MIN VERTEX COVER ∈ APX: 2-approximation • k-DEGREE INDEP. SET ∈ APX for k constant: (k + 1)-approximation. 6∈ FPTAS if P6= NP: small solution values

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 17 Institute for Software Technology Graz University of Technology Next: PTAS Reductions

Deﬁnition: A PTAS reduction A ≤P T AS B of an opt. problem A = (IA,SA, vA, dA) to B = (IB,SB, vB, dB) is a function-triple (f, g, α) with the following properties:

• f : IA → IB maps instances of A to instances of B and can be computed in polynomial time.

• g maps triples (x, s, ε), where x ∈ IA, s ∈ SB(f(x)), + and ε ∈ Q to solutions g(x, s, ε) ∈ SA(x) and can be computed in polynomial time • α : Q+ → Q+ is a surjective polynomial-time computable function. And

• if rB(f(x), s) ≤ 1 + α(ε), then rA(x, g(x, s, ε)) ≤ 1 + ε.

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 18-1 Institute for Software Technology Graz University of Technology Next: PTAS Reductions

Deﬁnition: A PTAS reduction A ≤P T AS B of an opt. problem A = (IA,SA, vA, dA) to B = (IB,SB, vB, dB) is a function-triple (f, g, α) with the following properties:

• f : IA → IB maps instances of A to instances of B and can be computed in polynomial time.

• if rB(f(x), s) ≤ 1 + α(ε), then rA(x, g(x, s, ε)) ≤ 1 + ε.

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 18-2 Institute for Software Technology Graz University of Technology Next: PTAS Reductions

Deﬁnition: A PTAS reduction A ≤P T AS B of an opt. problem A = (IA,SA, vA, dA) to B = (IB,SB, vB, dB) is a function-triple (f, g, α) with the following properties:

• f : IA → IB maps instances off A to instances of B IB and can be computedIA in polynomial time. f•, g,maps and α triples (x, s, ε), where x ∈ IA, s ∈ SB(f(x)), + poly-timeand ε ∈ Q to solutions g(x, s, ε) ∈ SA(x) and can be computablecomputed in polynomial time SA(x) + + SB(f(x)) • α : Q → Q is a surjectiveg polynomial-time computable function. And

• ifz rB(f(x), s}|) ≤ 1 + α(ε{), thenz rA(x, g(x,}| s, ε)) ≤ 1 +{ ε. rA(x, g(x, s, ε)) ≤ 1 + ε ⇐ rB(f(x), s) ≤ 1 + α(ε)

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 18-3 Institute for Software Technology Graz University of Technology Next: PTAS Reductions

Lemma: If A ≤P T AS B and B ∈ PTAS, then A ∈ PTAS. Proof: • PTAS for A: must get input (x, ε) • Compute (f(x), α(ε)) in polynomial time • Use PTAS for B on (f(x), α(ε)) to obtain a 1+α(ε)-approximation s of B

• Compute g(x, s, ε) ∈ SA(x) in polynomial time. • By the last property, g(x, s, ε) is a 1+ε-approximation of x for B. (1 + ε)-approximation = ε-optimal solution X

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 18-11 Institute for Software Technology Graz University of Technology Next: PTAS Reductions

Lemma: If A ≤P T AS B and B ∈ PTAS, then A ∈ PTAS.

Lemma: If A ≤P T AS B and B ∈ APX, then A ∈ APXX. Proof: • B ∈ APX ⇒ ∃ poly-time (1+δ)-approximation algorithm for B for some δ ∈ Q+. • α surjective ⇒ ∃ ε ∈ Q+ : α(ε) = δ. • Combining f, g, and (1+δ)-approximation algorithm for B gives a (1+ε)-approximation algorithm for A.X

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 18-16 Institute for Software Technology Graz University of Technology Next: PTAS Reductions

Lemma: If A ≤P T AS B and B ∈ PTAS, then A ∈ PTAS.

Lemma: If A ≤P T AS B and B ∈ APX, then A ∈ APXX.

Theorem: MAX 2SAT ≤P T AS MAX INDEP. SET andX MAX INDEP. SET ≡P T AS MAX CLIQUE Proof: Exercise. Hint: consider ≤P -reductions we made. consider the deﬁnition of PTAS-reductions.

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 18-17 Institute for Software Technology Graz University of Technology Next: PTAS Reductions

Lemma: If A ≤P T AS B and B ∈ PTAS, then A ∈ PTAS.

Lemma: If A ≤P T AS B and B ∈ APX, then A ∈ APXX.

Theorem: MAX 2SAT ≤P T AS MAX INDEP. SET andX MAX INDEP. SET ≡P T AS MAX CLIQUE Deﬁnition: An optimization problem A is X • NPO-complete if it is in NPO and all problems from NPO can be PTAS-reduced to A. • APX-complete if it is in APX and all problems from APX can be PTAS-reduced to A. • PTAS-complete if it is in PTAS and all problems from PTAS can be PTAS-reduced to A.

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 18-21 Institute for Software Technology Graz University of Technology Next: PTAS Reductions

Lemma: If A ≤P T AS B and B ∈ PTAS, then A ∈ PTAS.

Lemma: If A ≤P T AS B and B ∈ APX, then A ∈ APXX.

Theorem: MAX 2SAT ≤P T AS MAX INDEP. SET andX MAX INDEP. SET ≡P T AS MAX CLIQUE Deﬁnition: An optimization problem A is X • NPO-completehard if it is in NPO and all problems from NPO can be PTAS-reduced to A. • APX-completehard if it is in APX and all problems from APX can be PTAS-reduced to A. • PTAS-completehard if it is in PTAS and all problems from PTAS can be PTAS-reduced to A.

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 18-22