Welcome to ... Problem Analysis and Complexity Theory 716.054, 3 VU Birgit Vogtenhuber Institute for Software Technology Graz University of Technology email: [email protected] office: Inffeldgasse 16B/II, room IC02116 office hour: by appointment, in webex meeting room office hour: Wednesday 10:30-11:30 slides: http://www.ist.tugraz.at/pact20.html

Birgit Vogtenhuber Problem Analysis and Complexity Theory 196 Last Time Optimization and Approximation • Decision vs. Optimization • Notion of Approximation: n v(x,y) opt(x) o ◦ approximation ratio (x, y) = max opt(x) , v(x,y) ◦ defined r(n)-approximation • First complexity classes for optimization problems: PO, NPO, APX(r(n)), APX, APX∗ • First optimization problems: MIN , MAX (k-DEG.) INDEP. SET

Birgit Vogtenhuber Problem Analysis and Complexity Theory 197 - 7 Last Time Presentations: Organizational Issues • Format of the presentations: pdf (optionally: same as lecture) • Length of each presentation: 30-40 minutes • For each presentation: Discussion, Questions • Presentations will take place on Tuesday, 30.06.2020, via Webex, during the lecture time. • Timing: send presentation and needed time at latest until Friday, 26.06.2020

Birgit Vogtenhuber Problem Analysis and Complexity Theory 198 - 2 Last Time Presentations: Organizational Issues More Optimization and Approximation: • Consider MIN TSP • Prove the Gap Theorem • GivenConsider an MIN optimization TSP∆ problem such that 1. ∀ inputs x ∈ I and ∀ solutions s ∈ S(x), s 6∈ (a, b), • Introducethat is,approximation either v(s) ≤ a schemeor v(s) ≥, PTASb, and, and FPTAS • 2.Consider it is NP MAX-hard KNAPSACK to determine if opt(x) ≤ a or opt(x) ≥ b. Then it is NP-hard to obtain a worst-case approximation ratio that is smaller than b/a.

Birgit Vogtenhuber Problem Analysis and Complexity Theory 198 - 7 Last Time Presentations: Organizational Issues More Optimization and Approximation: • Consider MIN TSP • Prove the Gap Theorem • Consider MIN TSP∆ • Introduce approximation scheme, PTAS, and FPTAS • Consider MAX KNAPSACK

Birgit Vogtenhuber Problem Analysis and Complexity Theory 198 - 10 Last Time Definition: Algorithm M is approximation scheme for optimization problem A = (I, S, v, d): ∀x ∈ I, ∀ε > 1: R(x, M(x, ε)) ≤ ε.

Definition: Class PTAS: Optimization problems with polynomial time approximation scheme M: M runs in polynomial time.

Definition: Class FPTAS: Optimization problems with fully polynomial time approximation scheme M: M(x, ε) runs in time polynomial in |x| and 1/(ε − 1)).

Birgit Vogtenhuber Problem Analysis and Complexity Theory 198 - 13 Last Time Presentations: Organizational Issues More Optimization and Approximation: • Consider MIN TSP • Prove the Gap Theorem • Consider MIN TSP∆ • Introduce approximation scheme, PTAS, and FPTAS • Consider MAX KNAPSACK

Birgit Vogtenhuber Problem Analysis and Complexity Theory 198 - 14 This Time / Next Time Optimization and Approximation: • Learn about problems with small solution values • Consider an intermediate overview of approximation results with respect to small solution values • Learn to make reductions for approximation problems • New concept / class PCP: Probabilistically Checkable Proofs • Maybe some bonus material • Summary Presentations

Birgit Vogtenhuber Problem Analysis and Complexity Theory 199 - 8 Problems with Small Solutions • We have just seen that MAX KNAPSACK ∈ FPTAS. • We have also seen that MAX KNAPSACK is pseudopolynomial: If the maximum value V is polynomial, then so is the optimum solution, and the problem can be solved in polynomial time. ⇒ What about problems that are also complicated for small solution values? Especially, optimization problems whose decision variants are strongly NP-hard? Definition: An optimization problem is a problem with small solution values if solutions are positive integers with values polynomially bounded in the input length.

Birgit Vogtenhuber Problem Analysis and Complexity Theory 200 - 5 Problems with Small Solutions Theorem: If P6= NP, then no NP-hard problem with small solution values has an FPTAS. Proof: Exercise. • Assume(solutions existence≤ ofp(n an); FPTASε(n) = 1A/p.(n) + 1;) • Let |x| = n and p(n) be a polynomial bound for the solution values for x, and Let ε(n) := 1/p(n) + 1 ⇒ A(x, ε(n)) runs in q(n, 1/(ε(n) − 1)) = q(n, p(n)) time, which is polynomial in n. ⇒ A(x, ε(n)) can’t be the optimal solution for all inputs x. ⇒ As all solutions ∈ {1, . . . , p(n)}, every non-optimal solution gives a worst case approximation ratio of at least p(n)/(p(n) − 1) = 1 + 1/(p(n) − 1) > ε(n).

Birgit Vogtenhuber Problem Analysis and Complexity Theory 200 - 8 Problems with Small Solutions Theorem: If P6= NP, then no NP-hard problem with small solution values has an FPTAS. Proof: Exercise. • Assume existence of an FPTAS A. • Let |x| = n and p(n) be a polynomial bound for the solution values for x, and Let ε(n) := 1/p(n) + 1 ⇒ A(x, ε(n)) runs in q(n, 1/(ε(n) − 1)) = q(n, p(n)) time, which is polynomial in n. ⇒ A(x, ε(n)) can’t be the optimal solution for all inputs x. ⇒ As all solutions ∈ {1, . . . , p(n)}, every non-optimal solution gives a worst case approximation ratio of at least p(n)/(p(n) − 1) = 1 + 1/(p(n) − 1) > ε(n).

Birgit Vogtenhuber Problem Analysis and Complexity Theory 200 - 10 Approximation Results Approximation results we have seen by now: • MAX KNAPSACK ∈ FPTAS • MIN TSP 6∈ APX if P6= NP: Question: gap-argument with HAMCYCLE What do the insights • MIN TSP∆ ∈ APX: about small solution 2-approximation values imply for these • MIN VERTEX COVER ∈ APX: problems? 2-approximation • k-DEGREE INDEP. SET ∈ APX for k constant: (k + 1)-approximation.

Birgit Vogtenhuber Problem Analysis and Complexity Theory 201 - 7 Approximation Results Approximation results we have seen by now: • MAX KNAPSACK ∈ FPTAS • MIN TSP 6∈ APX if P6= NP: gap-argument with HAMCYCLE • MIN TSP∆ ∈ APX: 2-approximation • MIN VERTEX COVER ∈ APX: 2-approximation • k-DEGREE INDEP. SET ∈ APX for k constant: (k + 1)-approximation. 6∈ FPTAS if P6= NP: small solution values

Birgit Vogtenhuber Problem Analysis and Complexity Theory 201 - 8 PTAS Reductions

Definition: A PTAS reduction A ≤P T AS B of an opt. problem A = (IA,SA, vA, dA) to B = (IB,SB, vB, dB) is a function-triple (f, g, α) with the following properties:

• f : IA → IB maps instances of A to instances of B and can be computed in polynomial time.

• g maps triples (x, s, ε), where x ∈ IA, s ∈ SB(f(x)), + and ε ∈ Q to solutions g(x, s, ε) ∈ SA(x) and can be computed in polynomial time • α : Q+ → Q+ is a surjective polynomial-time . And

• if rB(f(x), s) ≤ 1 + α(ε), then rA(x, g(x, s, ε)) ≤ 1 + ε.

Birgit Vogtenhuber Problem Analysis and Complexity Theory 202 - 5 PTAS Reductions

Definition: A PTAS reduction A ≤P T AS B of an opt. problem A = (IA,SA, vA, dA) to B = (IB,SB, vB, dB) is a function-triple (f, g, α) with the following properties:

• f : IA → IB maps instances of A to instances of B and can be computed in polynomial time.

• g maps triples (x, s, ε), where x ∈ IA, s ∈ SB(f(x)), + and ε ∈ Q to solutions g(x, s, ε) ∈ SA(x) and can be computed in polynomial time • α : Q+ → Q+ is a surjective polynomial-time computable function. And

• if rB(f(x), s) ≤ 1 + α(ε), then rA(x, g(x, s, ε)) ≤ 1 + ε.

Birgit Vogtenhuber Problem Analysis and Complexity Theory 202 - 6 PTAS Reductions

Definition: A PTAS reduction A ≤P T AS B of an opt. problem A = (IA,SA, vA, dA) to B = (IB,SB, vB, dB) is a function-triple (f, g, α) with the following properties:

• f : IA → IB maps instances off A to instances of B IB and can be computedIA in polynomial time. f•, g,maps and α triples (x, s, ε), where x ∈ IA, s ∈ SB(f(x)), + poly-timeand ε ∈ Q to solutions g(x, s, ε) ∈ SA(x) and can be computablecomputed in polynomial time SA(x) + + SB(f(x)) • α : Q → Q is a surjectiveg polynomial-time computable function. And

• ifz rB(f(x), s}|) ≤ 1 + α(ε{), thenz rA(x, g(x,}| s, ε)) ≤ 1 +{ ε. rA(x, g(x, s, ε)) ≤ 1 + ε ⇐ rB(f(x), s) ≤ 1 + α(ε)

Birgit Vogtenhuber Problem Analysis and Complexity Theory 202 - 7 PTAS Reductions

Lemma: If A ≤P T AS B and B ∈ PTAS, then A ∈ PTAS. Proof: • PTAS for A: must get input (x, ε) • Compute (f(x), α(ε)) in polynomial time • Use PTAS for B on (f(x), 1 + α(ε)) to obtain a (1 + α(ε))-approximation s of B

• Compute g(x, s, ε) ∈ SA(x) in polynomial time. • By the last property, g(x, s, ε) is a (1+ε)-approximation of x for B. Remark: some literature uses the term ε-optimal solution.X ε-optimal solution == (1 + ε)-approximation Birgit Vogtenhuber Problem Analysis and Complexity Theory 202 - 15 PTAS Reductions

Lemma: If A ≤P T AS B and B ∈ PTAS, then A ∈ PTAS.

Lemma: If A ≤P T AS B and B ∈ APX, then A ∈ APXX. Proof: • B ∈ APX ⇒ ∃ poly-time (1+δ)- for B for some δ ∈ Q+. • α surjective ⇒ ∃ ε ∈ Q+ : α(ε) = δ. • Combining f, g, and (1+δ)-approximation algorithm for B gives a (1+ε)-approximation algorithm for A.X

Birgit Vogtenhuber Problem Analysis and Complexity Theory 202 - 20 PTAS Reductions

Lemma: If A ≤P T AS B and B ∈ PTAS, then A ∈ PTAS.

Lemma: If A ≤P T AS B and B ∈ APX, then A ∈ APXX.

Theorem: MAX 2SAT ≤P T AS MAX INDEP. SET andX MAX INDEP. SET ≡P T AS MAX CLIQUE Proof: Exercise. Hint: consider ≤p-reductions we made. consider the definition of PTAS-reductions.

Birgit Vogtenhuber Problem Analysis and Complexity Theory 202 - 21 PTAS Reductions

Lemma: If A ≤P T AS B and B ∈ PTAS, then A ∈ PTAS.

Lemma: If A ≤P T AS B and B ∈ APX, then A ∈ APXX.

Theorem: MAX 2SAT ≤P T AS MAX INDEP. SET andX MAX INDEP. SET ≡P T AS MAX CLIQUE Definition: An optimization problem A is X • NPO-complete if it is in NPO and all problems from NPO can be PTAS-reduced to A. • APX-complete if it is in APX and all problems from APX can be PTAS-reduced to A. • PTAS-complete if it is in PTAS and all problems from PTAS can be PTAS-reduced to A.

Birgit Vogtenhuber Problem Analysis and Complexity Theory 202 - 25 PTAS Reductions

Lemma: If A ≤P T AS B and B ∈ PTAS, then A ∈ PTAS.

Lemma: If A ≤P T AS B and B ∈ APX, then A ∈ APXX.

Theorem: MAX 2SAT ≤P T AS MAX INDEP. SET andX MAX INDEP. SET ≡P T AS MAX CLIQUE Definition: An optimization problem A is X • NPO-completehard if it is in NPO and all problems from NPO can be PTAS-reduced to A. • APX-completehard if it is in APX and all problems from APX can be PTAS-reduced to A. • PTAS-completehard if it is in PTAS and all problems from PTAS can be PTAS-reduced to A.

Birgit Vogtenhuber Problem Analysis and Complexity Theory 202 - 26 Next: PCP Next we consider a class for decision problems which will help us to prove completeness results for approximation classes: PCP=ProbabilisticallyCheckableProofs. Definition: For r, q : lN → lN, an (r(n), q(n))-bounded probabilistic proof checker is a poly-time algorithm V . For an input x, |x| = n, and a proof P , the algorithm V has access to x and to a random vector r ∈ {0, 1}O(r(n)). First V computes to O(q(n)) positions from x and r. Then V can use x, r, and as additional information the values of the bits of the proof P in the before-computed positions to compute the decision V (x, r, P ) ∈ {0, 1}.

Birgit Vogtenhuber Problem Analysis and Complexity Theory 203 - 5 Next: PCP Intuition of TM interpretation for pobabilistic proof checker:

Input tape . 1 0 1 1 t t t ···

Work tape . 1 0 0 0 0 1 t ···

Proof tape . t t t t t t t ···

Read only O(q(n)) positions. Choose them by using x and a random vector r : {0, 1}O(r(n)).

Birgit Vogtenhuber Problem Analysis and Complexity Theory 203 - 6 PCP: Definition PCP=ProbabilisticallyCheckableProofs Definition: A decision problem belongs to the class PCP(r(n),q(n)) (probabilistically checkable proofs with O(r(n)) random bits and O(q(n)) query bits), if there is an (r(n), q(n))-bounded probabilistic proof checker V with the following properties: • If x ∈ L, then there is a proof P (x) such that Prob(V (x, r, P (x)) = 1) = 1. ⇐ always accept! • If x 6∈ L, then for all proofs P it holds that that 1 1 Prob(V (x, r, P ) = 0) ≥ 2 . ⇐ reject with prob. ≥ 2 . 1 Remark: 2 can be replaced by any other const. 0<δ <1. Birgit Vogtenhuber Problem Analysis and Complexity Theory 204 - 5 PCP: Results PCP=ProbabilisticallyCheckableProofs Theorem: PCP(0,0) = P, PCP(0,poly) = NP. Proof PCP(0,0) = P: X • PCP(0,0) ⊆ P: • q(n) = 0 ⇔ no proof. • r(n) = 0 ⇔ no randomness. ⇒ Prob(V (x, r, P ) = 0) = 1 for x 6∈ L. ⇒ V decides L in poly-time. • P ⊆ PCP(0,0): Use poly-time algorithm as V . Proof PCP(0,poly) = NP: Exercise

Birgit Vogtenhuber Problem Analysis and Complexity Theory 205 - 8 PCP: Results PCP=ProbabilisticallyCheckableProofs Theorem: PCP(0,0) = P, PCP(0,poly) = NP. Theorem: L ∈ PCP(r(n),q(n)) ⇒ ∃ nondeterministicX Turing machine that decides L in time 2O(r(n)+log n). Proof: Assume PCP A for L runs in p(n) time. • For each of the 2O(r(n)) possible bit vectors r: generate at most min{p(n), q(n)} ≤ p(n) positions in time O(p(n)). • Generate all proof bits (for all r’s) nondeterministically. • For each bit vector r: run the proof checker V . ⇒ Total time (all r’s): ≤ p(n) · 2O(r(n)) = 2O(r(n)+log n). ⇒ x ∈ L: ∃P s.t. accepted for all random bit vectors. x 6∈ L: ∀P rejected for ≥ half of the random bit vectors. Birgit Vogtenhuber Problem Analysis and Complexity Theory 205 - 13 PCP: Results PCP=ProbabilisticallyCheckableProofs Theorem: PCP(0,0) = P, PCP(0,poly) = NP. Theorem: L ∈ PCP(r(n),q(n)) ⇒ ∃ nondeterministicX Turing machine that decides L in time 2O(r(n)+log n). Proof: Assume PCP A for L runs in p(n) time. X • For each of the 2O(r(n)) possible bit vectors r: generate at most min{p(n), q(n)} ≤ p(n) positions in time O(p(n)). • Generate all proof bits (for all r’s) nondeterministically. • For each bit vector r: run the proof checker V . ⇒ Total time (all r’s): ≤ p(n) · 2O(r(n)) = 2O(r(n)+log n). ⇒ x ∈ L: ∃P s.t. accepted for all random bit vectors. x 6∈ L: ∀P rejected for ≥ half of the random bit vectors. Birgit Vogtenhuber Problem Analysis and Complexity Theory 205 - 17 PCP: Results PCP=ProbabilisticallyCheckableProofs Theorem: PCP(0,0) = P, PCP(0,poly) = NP. Theorem: L ∈ PCP(r(n),q(n)) ⇒ ∃ nondeterministicX Turing machine that decides L in time 2O(r(n)+log n). Corollary: NP = PCP(log n, poly). X Proof: Each direction one of the above theorems. X Theorem [PCP Theorem]: NP = PCP(log n, 1).

NP ⊆ PCP(log n, 1): 3SAT ∈ PCP(log n, 1): > 50 pages 3 3SAT ∈ PCP(n , 1): ≈ 10 pages [Wegener: Complexity Theory]

Birgit Vogtenhuber Problem Analysis and Complexity Theory 205 - 24 PCP: Results PCP=ProbabilisticallyCheckableProofs Theorem: PCP(0,0) = P, PCP(0,poly) = NP. Theorem: L ∈ PCP(r(n),q(n)) ⇒ ∃ nondeterministicX Turing machine that decides L in time 2O(r(n)+log n). Corollary: NP = PCP(log n, poly). X Proof: Each direction one of the above theorems. X Theorem [PCP Theorem]: NP = PCP(log n, 1).

G¨odelPrize 2001: PCP Theorem & its appli- cations to the hardness of approximation.

Birgit Vogtenhuber Problem Analysis and Complexity Theory 205 - 25 Hardness of Approximation Principle Ingredient 1: given L ∈ NP •∃ c, k constant, ∃ probabilitstcally checkable proof for L that, for any input of size n and proof P , • uses at most c log n random bits and • reads at most k bits of the proof P . ⇒ at most N := 2c log n = O() possible assignments for bit vector r. • at most k proof bit positions to be read per r. ⇒≤ k · N = O(nc) proof bits have prob. 6= 0 to be read. ⇒ We can assume P to have length ≤ k · N = O(nc). Principle Ingredient 2: Gap Theorem

Birgit Vogtenhuber Problem Analysis and Complexity Theory 206 - 10 Hardness of Approximation Theorem: If P 6= NP, then MAX CLIQUE does not have a polynomial-time < 2-approximation algorithm. Proof: • We use 3SAT and a PCP V as just described: ≤ k proof bits, ≤ c log n random bits for n variables. • For each of the ≤ N := 2c log n random vectors r: Consider all 2k assignments of the Consider at most k read proof bits of proof P . • Construct graph G in poly-time: ≤ N · 2k vertices • vertex for every (r, P ) that is accepted by V . • edge between vertices iff different r’s and non-contradicting P ’s.

Birgit Vogtenhuber Problem Analysis and Complexity Theory 207 - 9 Hardness of Approximation Theorem: If P 6= NP, then MAX CLIQUE does not have a polynomial-time < 2-approximation algorithm. Proof: • Graph G: • vertex for every (r, P ) that is accepted by V . • edge for different r’s and non-contradicting P ’s. ⇒ Clique in G: no two vertices with same r. • Formula satisfiable: ⇒∃ P that is accepted for all r’s. ⇒ clique of sizeN.

Birgit Vogtenhuber Problem Analysis and Complexity Theory 207 - 14 Hardness of Approximation Theorem: If P 6= NP, then MAX CLIQUE does not have a polynomial-time < 2-approximation algorithm. Proof: • Graph G: • vertex for every (r, P ) that is accepted by V • edge for different r’s and non-contradicting P ’s ⇒ Clique in G: no two vertices with same r. • Formula not satisfiable: • every P accepted for at most half of the r’s. • assume clique of size N 0 > N/2 in G ⇒ ∃ P com- patible with all used partial assigments of proof bits. ⇒ P accepted for N 0 > N/2 of the r’s. Birgit Vogtenhuber Problem Analysis and Complexity Theory 207 - 19 Hardness of Approximation Theorem: If P 6= NP, then MAX CLIQUE does not have a polynomial-time < 2-approximation algorithm. Proof: • Graph G: • vertex for every (r, P ) that is accepted by V • edge for different r’s and non-contradicting P ’s ⇒ Clique in G: no two vertices with same r. • Formula satisfiable ⇒ clique of size N. • Formula not satisfiable ⇒ no clique of size > N/2. • Decision of satisfiability NP-hard. ⇒ < 2-approximation NP-hard (by the Gap Theorem).X Birgit Vogtenhuber Problem Analysis and Complexity Theory 207 - 23 Hardness of Approximation Theorem: If P 6= NP, then MAX CLIQUE does not have a polynomial-time < 2-approximation algorithm. Corollary: If P 6= NP, then MAX INDEP. SET doesX not have a polynomial-time < 2-approximation algorithm. Corollary: If P 6= NP, then MAX CLIQUE and MAX INDEP. SET are not in PTAS. Corollary: If P 6= NP, then MAX CLIQUE and MAX INDEP. SET are not in APX.

Missing ingredient - Bonus Material: MAX IND. SET ∈ APX ⇒ MAX IND. SET ∈ PTAS.

Birgit Vogtenhuber Problem Analysis and Complexity Theory 207 - 26 Hardness of Approximation Theorem: If P 6= NP, then MAX CLIQUE does not have a polynomial-time < 2-approximation algorithm. Corollary: If P 6= NP, then MAX INDEP. SET doesX not have a polynomial-time < 2-approximation algorithm. Corollary: If P 6= NP, then MAX CLIQUE and MAX INDEP. SET are not in PTAS. Corollary: If P 6= NP, then MAX CLIQUE and MAX INDEP. SET are not in APX.

Missing ingredient - Bonus Material: MAX IND. SET 6∈ APX ⇐ MAX IND. SET 6∈ PTAS.

Birgit Vogtenhuber Problem Analysis and Complexity Theory 207 - 27