Introduction to Approximation algorithms
Algorithms and Networks 2016/2017 Johan M. M. van Rooij Hans L. Bodlaender
1 Solution Optimal Bound on Good quality quality solution, no quality Computation guarantee time
Polynomial Approximation Construction Polynomial solution algorithms heuristics algorithms
Exact algorithms: Hybrid Meta • Tree search algorithms heuristics: Super • Dynamic • Column • Local polynomial programming generation search and/or no • Integer linear without • Genetic guarantee programming complete algorithms • …… branch-and- price
2 Solution Optimal Bound on Good quality quality solution, no quality Computation guarantee time
Polynomial ApproximationAlgorithms Construction Polynomial solution algorithmsand networks heuristics algorithms
ExactAlgorithms algorithms : Hybrid Meta a• ndTree networks search algorithms heuristics: Super • Dynamic • Column • Local polynomial programming generation search and/or no • Integer linear without • Genetic guarantee programming complete algorithms • …… branch-and- price
3 What to do if a problem is NP-complete?
We have already seen many options to deal with NP- complete problems. Earlier in this course: Special cases (planar graphs), exact exponential-time algorithms. Later in this course: FPT and treewidth. In other courses: local search, ILP, constraint programming, … Approximation algorithms are one of these options. An approximation algorithm is a heuristic with a performance guarantee. We consider polynomial-time approximation algorithms. Non-optimal solutions, but with some performance guarantee compared to the optimal solution. Also useful as a starting point for other approaches: Local search, branch and bound.
4 What is a polynomial-time approximation algorithm?
An algorithm that... 1. ‘Solves’ (applies to) an optimisation problem. A minimisation problem or a maximisation problem. E.g., independent set, vertex cover, knapsack, max-SAT. Not: SAT, 3-SAT, etc. 2. Runs in polynomial time. 3. It has a bound on the quality of the solution. This is called the approximation ratio. E.g.: value of algorithm result / optimal solution value · c. We call such an algorithm a c-approximation.
5 Approximation ratio
For an instance I of an optimisation problem, let OPT(I) be the value of the optimal solution to I. ALG(I) be the value computed by the approximation algorithm.
An algorithm for a minimisation problem has an approximation ratio c, if for all instances I:
An algorithm for a maximisation problem has an approximation ratio c, if for all instances I:
6 Class of approximation algorithms
Class of problems with approximation algorithms whose approximation ratio is a constant: APX. Subclass of NP. We only consider optimisation problems in NP.
Other notions of approximability also exist. We see some of them the next lecture. And one at the end of this lecture.
Notions of APX-completeness also exist. As NP-completeness. We will see this at the end of the lectures on approximation algorithms.
7 This lecture
Approximation algorithms for a series of problems: Travelling Salesman Problem. Minimum (Weight) Vertex Cover. Max Satisfiability.
8 Approximation algorithms - Algorithms and Networks MAXIMUM SATISFIABILITY
9 Max satisfiability (decision version)
Maximum Satisfiability (Max SAT). Instance: A set of clauses C containing literals of variables from a set X and an integer k. Question: Does there exist a truth assignment to the variables in X such that at least k clauses in C are satisfied?
k-Satisfiability (k-SAT): Instance: set of clauses C (logical formula in CNF format) where each clause contains at most k literals. Question: does there exist a satisfying assignment to the variables in C, i.e., a truth assignment such that each clause has at least one literal set to true.
Defined similarity: Maximum k-Satisfiability (Max k-SAT)
10 Approximating Max Satisfiability
Algorithm: 1. Take any truth assignment a to the variables X. 2. Let :a be the assignment a with all variables negated. 3. Return form {a,:a} the assignment that satisfies most clauses.
This is a 2-approximation. ALG ¸ |C|/2, OPT · |C|. Why?
Can you also give approximation algorithms for max k- satisfiability?
11 Approximation algorithms - Algorithms and Networks TRAVELLING SALESMAN PROBLEM
12 The travelling salesman problem
Instance: n vertices 4 (cities), distance 1 2 between every pair of 2 3 vertices. 5 Question: Find shortest 2 (simple) cycle that visits 3 2 4 every city? 4 4 Restriction on distances: 1 2 1 2 distances are non- 2 3 2 3 5 negative (or positive). 5 Triangle inequality: 2 2 for all x, y, z: 3 4 3 4 2 2 w(x,y) + w(y,z) w(x,z) 13 11
13 A simple algorithm
Consider the Travelling Salesman Problem with triangle inequality.
Algorithm: 1. Find a minimum spanning tree. 2. Output vertices of tree in preorder. Preorder: visit a node before its children. Take a tour following the MST
This algorithm has approximation ratio 2: OPT ≥ MST. 2 MST ≥ ALG. ALG / OPT ≤ 2MST / MST = 2.
14 Can we do better? Yes: Christofides’ algorithm
Christofides’ Algorithm: 1. Construct a Minimum Spanning Tree T. 2. Set W = {v | v has odd degree in tree T}. 3. Compute a minimum weight matching M in the graph G[W]. 4. Look at the graph T+M. Note that T+M is Eulerian! 5. Compute an Euler tour C’ in T+M. 6. Add shortcuts to C’ to get a TSP-tour.
This is a 3/2-approximation. Proof on the blackboard.
15 Ratio 1.5
Total length edges in T: at most OPT Total length edges in matching M: at most OPT/2. T+M has length at most 3/2 OPT. Use triangle inequality.
16 Proving approximation ratios
We just saw two algorithms for TSP with triangle inequality.
We proved approximation ratios in the following way: 1. Use a quantity x that relates to both the optimal solution and the result of the algorithm A minimum spanning tree in both cases.
2. Prove that OPT ¸ c1 x.
3. Prove that ALG · c2 x 4. Combine both inequalities for a bound on ALG / OPT.
Next, we consider some more problems.
17 Approximation algorithms - Algorithms and Networks MINIMUM (WEIGHT) VERTEX COVER
18 Approximation for minimum vertex cover
Algorithm: 1. Let E’ = E, C = ;. 2. While E’ ;. 1. Let {u,v} be any edge from E’. 2. C := C [ {u,v}. 3. Remove every edge incident to u or v from E’. 3. Return C.
Approximation algorithm? Runs in polynomial time. Returns a vertex cover. How good is this vertex cover?
19 2-approximation for vertex cover
Theorem: The algorithm on the previous slide is a 2-approximation.
Proof: Let A be the set of edges which endpoints we picked. OPT ¸ |A|, because every edge in A must be covered. ALG = 2|A| · 2OPT, hence ALG/OPT · 2.
20 Minimum weight vertex cover
Minimum weight vertex cover: Vertex cover where each vertex has a weight. We look for the minimum weight vertex cover.
2-approximation for vertex cover no longer works. We may select very heavy vertices using that algorithm.
Next, we will give a 2-approximation for minimum weights vertex cover using LP rounding.
21 An ILP formulation of the problem
Consider the following ILP:
minimise: w(v)x(v) v V X2 subject to:x(u) + x(v) 1 for each (u; v) E ¸ 2 x(v) 0; 1 for each v V 2 f g 2 It’s LP relaxation is ILP with the last constraint replaced by: 0 · x(v) · 1.
Linear programming can be solved in polynomial time. Not by the simplex algorithm!! Ellipsoid method / interior point methods.
22 2-approximation algorithm for minimum weight vertex cover
Algorithm: 1. Compute the optimal solution to the LP relaxation. 2. Output all v with x(v) ¸ ½.
This algorithm returns a vertex cover. For every edge, sum of incident vertices at least 1. Hence at least one of the vertex variables at least ½.
Approximation algorithm? Runs in polynomial time and produces a vertex cover. How good is this vertex cover?
Proof of 2-approximation on the blackboard!
23 Proof of 2-approximation algorithm for minimum weight vertex cover
Let z* be the solution to the LP. Because any vertex cover is a solution to the LP we have:
Also, we can bound ALG in terms of z*:
Hence:
QED
24 Approximation algorithms - Algorithms and Networks CONCLUSION
25 Conclusion
We have seen several different approximation algorithms for different problems: 2-approximations for Max Satisfiability (and Max k-SAT) 1.5- and 2-approximations for TSP. 2-approximations for vertex cover and weighted vertex cover.
c-approximations, for a constant c, are called constant factor approximation algorithms.
There are more different types of approximations. These we will see after the break.
26 The Landscape of Approximation Algorithms
Algorithms and Networks 2016/2017 Johan M. M. van Rooij Hans L. Bodlaender
27 What is a polynomial-time approximation algorithm?
An algorithm that... 1. Solves an optimisation problem. 2. Runs in polynomial time. 3. It has a bound on the quality of the solution.
Approximation ratio c: ALG/OPT · c for minimisation problems. OPT/ALG · c for maximisation problems. Ratio always bigger or equal to 1.
28 Different forms of approximation algorithms (outline of two lectures)
Qualities of polynomial-time approximation algorithms: 1. Absolute constant difference. |OPT – ALG| · c 2. FPTAS: Fully polynomial-time approximation scheme. Approximation ratio 1+² for any ² > 0, while the algorithm runs in time polynomial in n and 1/². 3. PTAS: Polynomial-time approximation scheme. Approximation ratio 1+² for any ² > 0, while the algorithm runs in polynomial time for any fixed ². 4. APX: Constant-factor approximation. Approximation ratio: ALG/OPT · c for minimisation problems. Approximation ratio: OPT/ALG · c for maximisation problems. 5. f(n)-APX: Approximation by a factor of f(n). f(n) depends only on the size of the input.
29 Qualities of poly-time approximation algorithms: 1. Absolute constant difference. 2.FPTAS: Fully polynomial-time approximation scheme. 3.PTAS: Polynomial-time approximation scheme. 4.APX: Constant-factor approximation. 5.f(n)-APX: Approximation by a factor of f(n).
Approximation algorithms - Algorithms and Networks ABSOLUTE CONSTANT DIFFERENCE
30 Absolute constant difference
Algorithms that run in worst-case polynomial time, with: |OPT – ALG| · c
Example: Planar graph colouring. Algorithm that tests 1 and 2-colourability and if no solution is found outputs 4. Algorithm has an error of at most 1, because of the four colour theorem.
Four colour theorem: Any planar graph G can be coloured with at most four colours.
31 Absolute constant difference for edge colouring
Vizings Theorem: Any graph G can be edge coloured with at most ¢(G)+1 colours, where ¢(G) is the maximum degree of G. Note that any edge colouring of a graph G requires at least ¢(G) colours.
Corollary: Edge colouring is approximable with an absolute constant error of one. Algorithm: Return ¢(G)+1.
Still, these examples of approximation with additive constant difference are very rare!
32 Absolute constant difference: very rare
Suppose we could approximate maximum independent set within an additive integer constant c > 0. Then, we can solve an NP-complete problem in polynomial time (proof on the blackboard).
Similar construction exist for many more problems.
33 Absolute constant difference: very rare
Suppose we could approximate maximum independent set within an additive integer constant c > 0. 1. Take an instance of maximum independent set: a graph G, where we look for an independent set of size at least k. 2. Make c + 1 disjoint copies of G. 3. Run the approximation algorithm on the new graph. 4. If an independent set of size at least k exists in G, then the algorithm returns a solution of size ¸ (c+1)k - c. 5. This means that it must be of size ¸ k on at least one disjoint copy. 6. If the independent set does not exits, then the algorithm returns a solution of size · (c+1)(k-1). 7. This way, we can solve an NP-complete problem using this polynomial-time algorithm. That should not be possible. Similar construction exist for many more problems.
34 Qualities of poly-time approximation algorithms: 1. Absolute constant difference. 2.FPTAS: Fully polynomial-time approximation scheme. 3.PTAS: Polynomial-time approximation scheme. 4.APX: Constant-factor approximation. 5.f(n)-APX: Approximation by a factor of f(n).
Approximation algorithms - Algorithms and Networks FULLY POLYNOMIAL-TIME APPROXIMATION SCHEME
35 Knapsack
We will introduce the FPTAS (Fully Polynomial-Time Approximation Scheme) using an example first.
Knapsack: Given: n items each with a positive integer weight w(i) and integer value v(i) (1 ≤ i ≤ n), and an integer W. Question: select items with total weight at most W such that the total value is as large as possible.
36 Dynamic programming for knapsack
Let P be the maximum value of any item. We can solve the problem in O(n2P) time with dynamic programming: Tabulate M(i,Q): minimum total weight of a subset of items 1,…,i with total value Q for Q at most nP. M(0,0) = 0 M(0,Q) = ∞, if Q>0 M(i+1,Q) = min{ M(i,Q), M(i,Q-v(i+1)) + w(i+1) }
This algorithm is clearly correct and runs in the given time. This is a pseudo-polynomial-time algorithm. Pseudo polynomial: polynomial in n and the weights P (or W). Super-polynomial running time if P is (exponentially) large compared to n.
37 Scaling for knapsack
We will turn the pseudo-polynomial time algorithm into an approximation algorithm.
Algorithm for knapsack: 1. Throw away all items that have weight larger than W (they are never used). 2. Build a new instance: 1. Let c be some positive constant. 2. Do not change weights, but set new values v’(i) = v(i) / c. 3. Solve scaled instance with DP optimally. 4. Output this solution: approximate solution to the original instance.
38 The question is….
As we scale values, not weights, the output is always feasible.
The question is: how do we set c, such that… Approximation ratio good enough? Polynomial time?
Approximation quality (blackboard): ALG ¸ OPT – nc. Running time: O(n2P/c).
39 Approximation ratio
Consider optimal solution Y for original problem. Let Y have value OPT.
Solution value scaled instance: At most n items, for each v(i)/c - v(i)/c <1 So, DP finds a solution of value at least OPT/c – n for scaled problem instance.
Value of approximate solution for original instance is at least c*(OPT/c –n) = OPT – nc.
40 A different kind of approximation algorithm
For any constant ² > 0, we use c = eP/(2n). Largest value of an item is at most P/c = P / (eP/(2n)) = 2n/e . Running time is O(n2 * 2n/e) = O(n3/e). Approximation ratio (on blackboard): 1+e.
Arbitrary good approximation. For fixed epsilon, these run in polynomial time. Running time grows exponentially in number of bits precision.
This is a Polynomial Time Approximation Scheme (PTAS). That is, an algorithm, that gets two inputs: the “usual” input X, and a real value e>0. And, for each fixed e>0, the algorithm is a (1+e)-approximation algorithm.
41 (1+e)-approximation
OPT ≥ P. Each single-item set is a solution in itself, as we removed items with weight more than B.
Algorithm gives solution of value at least: OPT – nc = OPT – n(e P / (2n) ) = OPT – eP/2
OPT / (OPT – eP/2) ≤ OPT / (OPT – e/2 OPT) = 1 / (1 - e/2) ≤ 1+e.
Where the last inequality is true if e ≤ 1. QED.
42 PTAS and FPTAS
For any constant ² > 0, we have an algorithm with: Running time is O(n3/e). Approximation ratio: 1+e.
Actually, this is a Fully Polynomial Time Approximation Scheme (FPTAS). That is, an algorithm, that gets two inputs: the “usual” input X, and a real value e>0. And, for each fixed e>0, the algorithm is a (1+e)-approximation algorithm. And, the algorithm uses polynomial time in both n and 1/e.
43 Qualities of poly-time approximation algorithms: 1. Absolute constant difference. 2.FPTAS: Fully polynomial-time approximation scheme. 3.PTAS: Polynomial-time approximation scheme. 4.APX: Constant-factor approximation. 5.f(n)-APX: Approximation by a factor of f(n).
Approximation algorithms - Algorithms and Networks POLYNOMIAL-TIME APPROXIMATION SCHEME
44 PTAS and FPTAS
A Polynomial Time Approximation Scheme is an algorithm, that gets two inputs: the “usual” input X, and a positive real value e>0. For each fixed e>0, the algorithm: Is an (1+e)-approximation algorithm. Uses polynomial time.
A Fully Polynomial Time Approximation Scheme is an algorithm, that gets two inputs: the “usual” input X, and a positve real value e>0. For each fixed e>0, the algorithm: Is an (1+e)-approximation algorithm. The algorithm uses polynomial time in the size of X and in 1/e.
45 PTAS that are not an FPTAS
It is not always possible to construct an FPTAS (even if we have a PTAS).
Example: maximum independent set has no FPTAS. 1. Suppose that it has, for each fixed e>0, some algorithm: Is an (1+e)-approximation algorithm. Uses polynomial time in the size of X and in 1/e. 2. Let ² = 1/n and run the algorithm. The algorithm uses polynomial time. The algorithm solves the problem optimally as:
3. Hence, no such algorithm exists unless P=NP.
46
PTAS that are not FPTAS are often found in a geometric setting
Proof that no FPTAS exists also works if we restrict the class of graphs. The only property of max independent set that we have used is that the optimum is bounded by n.
There exists a PTAS for maximum independent set restricted to: Planar graphs (next!). Unit-disk graphs (we will not consider these in the lectures).
47 Planar graphs and k-outerplanar graphs
Given a planar embedding of a graph G, we define the level of a vertex as follows: All nodes that lie on the exterior face have level 1. For any i, if we remove from the embedding all nodes of level smaller than i, then all nodes that lie on the border of the exterior face have level i.
A k-outerplanar graph is a graph that allows a planar embedding consisting of at most k levels.
Lemma (that we will not prove in this lecture): For any k, maximum independent set restricted to k- outerplanar graphs can be solved optimally in O(8kn) time.
48 PTAS for maximum independent set on planar graphs
Algorithm (input: graph G, approximation ratio r>1) 1. Let k = d1/(r-1)e. 2. Compute a planar embedding of G. 3. Compute the level numbers for all vertices in G. 4. For i = 0 to k:
1. Let Vi be V with all nodes that modulo k+1 have level i removed.
2. Let Gi be the graph induced by Vi (note: it is k-outerplanar).
3. Compute a maximal independent set in Gi using the lemma. 5. Return the largest independent set found over all k tries.
Consider the maximum independent set I in G.
At least one Vi has |Vi Å I| · |I|/(k+1). Thus the resulting sets is of size at least: |I| k/(k+1). Approximation ratio r.
49 PTAS for maximum independent set on planar graphs
Algorithm (input: graph G, approximation ratio r>1) 1. Let k = d1/(r-1)e. 2. Compute a planar embedding of G. 3. Compute the level numbers for all vertices in G. 4. For i = 0 to k:
1. Let Vi be V with all nodes that modulo k+1 have level i removed.
2. Let Gi be the graph induced by Vi (note: it is k-outerplanar).
3. Compute a maximal independent set in Gi using the lemma. 5. Return the largest independent set found over all k tries.
Running time: O(8kkn). Since k = d1/(r-1)e, this is polynomial for a fixed approximation ratio r.
50 Approximation algorithms - Algorithms and Networks CONCLUSION
51 Conclusion
1. Absolute constant difference. |OPT – ALG| · c Examples: Planar graph colouring, edge colouring. Negative example: maximum independent set. 2. FPTAS: Fully polynomial-time approximation scheme. Approximation ratio 1+² for any ² > 0, while the algorithm runs in time polynomial in n and 1/². Example: knapsack (scaling). Negative example: maximum independent set. 3. PTAS: Polynomial-time approximation scheme. Approximation ratio 1+² for any ² > 0, while the algorithm runs in polynomial time for any fixed ². Example: maximum independent set on planar graphs.
52 The Landscape of Approximation Algorithms Part 2
Algorithms and Networks 2016/2017 Johan M. M. van Rooij Hans L. Bodlaender
53 Different forms of approximation algorithms (outline of two lectures)
Qualities of polynomial-time approximation algorithms: 1. Absolute constant difference. |OPT – ALG| · c 2. FPTAS: Fully polynomial-time approximation scheme. Approximation ratio 1+² for any ² > 0, while the algorithm runs in time polynomial in n and 1/². 3. PTAS: Polynomial-time approximation scheme. Approximation ratio 1+² for any ² > 0, while the algorithm runs in polynomial time for any fixed ². 4. APX: Constant-factor approximation. Approximation ratio: ALG/OPT · c for minimisation problems. Approximation ratio: OPT/ALG · c for maximisation problems. 5. f(n)-APX: Approximation by a factor of f(n). f(n) depends only on the size of the input.
54 What we have seen already
1. Absolute constant difference. |OPT – ALG| · c Examples: Planar graph colouring, edge colouring. Negative example: maximum independent set. 2. FPTAS: Fully polynomial-time approximation scheme. Approximation ratio 1+² for any ² > 0, while the algorithm runs in time polynomial in n and 1/². Example: knapsack (scaling). Negative example: maximum independent set. 3. PTAS: Polynomial-time approximation scheme. Approximation ratio 1+² for any ² > 0, while the algorithm runs in polynomial time for any fixed ². Example: maximum independent set on planar graphs.
55 Qualities of poly-time approximation algorithms: 1. Absolute constant difference. 2.FPTAS: Fully polynomial-time approximation scheme. 3.PTAS: Polynomial-time approximation scheme. 4.APX: Constant-factor approximation. 5.f(n)-APX: Approximation by a factor of f(n).
Approximation algorithms - Algorithms and Networks APX
56 APX: constant factor approximation
APX: class of problems for which approximation algorithms exists with a constant approximation ratio. This is different from the PTAS / FPTAS setting where we can get arbitrarily good approximations. Problems with an (F)PTAS are in APX.
We have seen enough examples of this last week. 2-approximation for Max Satisfiability (and Max k-SAT) 1.5- and 2-approximations for TSP. 2-approximations for vertex cover and weighted vertex cover.
Negative examples, after first considering the last class of approximation algorithms.
57 Qualities of poly-time approximation algorithms: 1. Absolute constant difference. 2.FPTAS: Fully polynomial-time approximation scheme. 3.PTAS: Polynomial-time approximation scheme. 4.APX: Constant-factor approximation. 5.f(n)-APX: Approximation by a factor of f(n).
Approximation algorithms - Algorithms and Networks F(N)-APX
58 Approximation by a factor of f(n)
Approximation ratio of f(n). Approximation ratio depends on the size of the input (and can be very bad)
Set Cover Given: finite set U and a family F of subsets of U. Question: Does there exists a subset C of F of size at most
k such that [S2C S = U?
Greedy algorithm: While U ;: Select S 2 F that maximises |S Å U|. Add S to C and let U := U \ S.
59 An (ln(n)+1)-approximation algorithm for set cover
Theorem: The greedy set cover algorithm is an (ln(n)+1)-approximation.
Proof: 1. The algorithm runs in polynomial time and returns a set cover. th 2. Let Si be the i set from F picked by the algorithm.
3. We assign cost cx to elements from U:
Let x be firstly covered by set Si while running the algorithm.
And, let Xi = Si \ (S1 [ S2 [ ... [ Si-1)
Define: cx = 1 / |Xi|. Now we have:
4. ...
60 Proof of the (ln(n)+1)-approximation algorithm for set cover
The nth Harmonic number H(n) is defined as: From mathematics (bounding using integrals):
On the blackboard, we will prove that for any S 2 F:
From this we derive our approximation ratio: Where C* is the optimal set cover.
61 Bounding the cost of any set S
What remains is to prove that for any S 2 F: Remind:
If x firstly covered by Si, then cx = 1/|Xi|
Where: Xi = Si \ (S1 [ S2 [ ... [ Si-1)
For any S 2 F, let ui be the number of uncovered items in S when the algorithm selects Si.
Take the smallest value k s.t. uk = 0.
With the last equality due to the selection by the algorithm. Then...
62 Bounding the cost of any set S
To prove:
Last slide:
Thus:
Remind:
Q.E.D.
63 Approximation algorithms - Algorithms and Networks CONCLUSION
64 Conclusion
Qualities of polynomial-time approximation algorithms: 1. Absolute constant difference. Planar graph colouring. 2. FPTAS: Fully polynomial-time approximation scheme. Scaling for Knapsack. 3. PTAS: Polynomial-time approximation scheme. 4. APX: Constant-factor approximation. TSP with triangle inequality. (Weighted) vertex cover. 5. f(n)-APX: Approximation by a factor of f(n). Set cover.
65 Approximation complexity
Algorithms and Networks 2016/2017 Johan M. M. van Rooij Hans L. Bodlaender
66 Approximation complexity
We know how to prove that a problem can be solved in polynomial time. Give a polynomial-time algorithm and prove correctness. We also know how to prove that a problem ‘most likely’ cannot be solved in polynomial time. Prove NP-hardness: give a polynomial-time reduction from a known NP-compete / NP-hard problem. NP-complete: in NP and NP-hard.
How does this work for approximation algorithms? Positive results: give algorithm and prove that it is the appropriate type of approximation algorithm. Negative results?? Well... we have seen some of them already.
67 How to prove that an algorithm from one of the categories does not exist?
Qualities of polynomial-time approximation algorithms: 1. Absolute constant difference. 2. FPTAS: Fully polynomial-time approximation scheme. 3. PTAS: Polynomial-time approximation scheme. 4. APX: Constant-factor approximation. 5. f(n)-APX: Approximation by a factor of f(n).
We have already proved that ‘most likely’: Absolute constant difference approximation is not possible for maximum independent set. There is no FPTAS for maximum independent set. But can we do more?
68 How to prove that an algorithm from one of the categories does not exist?
Qualities of polynomial-time approximation algorithms: 1. Absolute constant difference. 2. FPTAS: Fully polynomial-time approximation scheme. 3. PTAS: Polynomial-time approximation scheme. 4. APX: Constant-factor approximation. 5. f(n)-APX: Approximation by a factor of f(n).
This lecture: Proving that no PTAS exists for some problems. APX-completeness. Proving that a problem does not belong to APX.
69 This lecture
The gap technique. Recap on NP-completeness. The complexity class APX and APX-completeness. Inapproximability of maximum independent set.
Note: All cases proving that some algorithm ‘most likely’ does not exists are based on the ‘likely’ assumption PNP.
70 Approximation algorithms - Algorithms and Networks THE GAP TECHNIQUE
71 The gap technique
The gap technique: An NP-hardness proof that creates a (wide) gap between YES-instance and NO-instances.
Example: There is no polynomial time algorithm A for TSP without triangle inequality that approximates within a ratio c, for any constant c>1. Reduction: 1. Take instance G=(V,E) of Hamiltonian Circuit. 2. Build instance of TSP: A city for each v 2 V. If (v,w) 2 E, then d(v,w) = 1, otherwise d(v,w) = nc+1. 3. A finds a tour with distance at most nc, if and only if, G has a Hamiltonian circuit.
72 Using the gap technique
The gap technique can (sometimes) be used to prove: Absolute constant difference approximation is not possible. Constant-factor approximation is not possible. PTAS is not possible.
We have seen two examples of the gap technique: 1. One to prove non-existence of absolute constant difference approximation of independent set (second lecture) 2. One to prove non-existence of constant-factor approximation of TSP without triangle inequality.
73 Another example: knapsack
We know there is an FPTAS for knapsack, if we can rule out an absolute constant difference approximation, then we know how difficult knapsack is from the viewpoint of approximation.
Suppose there is an absolute constant difference approximation for knapsack with absolute integer error c¸1: 1. Take an instance of knapsack. 2. Multiply the value of all items by c+1. 3. This does not change the set of feasible solutions. 4. However, the only solution with a difference in value of at most c from the optimal solution, is the optimal solution itself. 5. An absolute constant difference approximation will thus solve this knapsack instance optimally, contradicting the assumption P NP.
74 Approximation algorithms - Algorithms and Networks A RECAP ON NP-COMPLETENESS
75 P and NP
A decision problem belongs to the class P if: There is an algorithm solving the problem with a running time that is polynomial in the input size.
A decision problem belongs to the class NP if: Any solution y (called a certificate) can be encoded in polynomial space with respect to the size of the input x. Checking whether a given certificate y resembles a `yes’- instance can be done in polynomial time with respect to the size of x and y.
Clearly P is contained in NP. We can ignore the certificate y. When checking y, just solve x in polynomial time.
76 Reducibility
Definition:
Problem P1 is polynomial-time reducible to problem P2 (or: P1 P P2 ), if there exists a polynomial time computable function f mapping instances I1 of P1 to instances I2 of P2 such that:
I1 is a ‘yes’-instance, if and only if, I2 is a ‘yes’-instance.
Lemma:
If P1 P P2 then: if P2 P, then P1 P.
Run an algorithm for P2 on f(I) for instance I for problem P1.
Lemma:
If P1 P P2 and P2 P P3 then: P1 P P3. Reductions can be composed together.
77 NP-completeness
A problem P is NP-complete if: The problem belongs to NP. All problems in NP are polynomial-time reducible to P.
From a viewpoint of polynomial-time computability, NP- complete problems are the hardest problems in NP. If one of them can be solved in polynomial time, all problems in NP can be solved in polynomial time.
A problem P is NP-hard if: All problems in NP are polynomial-time reducible to P.
78 Proving that ‘most likely’ no polynomial-time algorithms exist
NP-complete/NP-hard problems ‘most likely’ have no polynomial-time algorithms. We often assume PNP.
Proving NP-completeness for problem P: Take a known NP-complete problem Q. Show that their exists a polynomial-time reduction from Q to P. This implies that all problems in NP can be reduced to P.
Cook-Levin theorem: Satisfiability is NP-complete. This was the ‘first known’ NP-complete problem.
79 Approximation algorithms - Algorithms and Networks APX AND APX-COMPLETENESS
80 APX and APX-completeness
APX: class of problems for which approximation algorithms exists with a constant approximation ratio. Thus, problems with an (F)PTAS are also in APX.
APX-completeness: Hardest problems in APX with respect to having a PTAS. If one APX-complete has a PTAS, then all APX-complete problems have a PTAS.
Max-3SAT is APX-complete. Consequence of the famous PCP theorem, that we will not discuss further. Corollary: any APX-complete problem has no PTAS unless P=NP.
81 L-reduction
P1 is L-reducible to P2 (P1 L P2) if there exist:
a function f mapping instances x of P1 to instances f(x) of P2, a function g mapping solutions of f(x) to solutions of x, two constants a and b such that: 1. f and g are computable in polynomial-time computable.
2. For any instance x of P1: OPT( f(x) ) · a ¢ OPT(x).
3. For any instance x of P1 and any solution y of f(x): |OPT(x) – VAL(g(y))| · b ¢ |OPT(f(x)) – VAL(y)|.
Lemma:
If P1 L P2 then: if P2 has a PTAS, then P1 has a PTAS.
Run an algorithm for P2 on f(x) for instance x for problem P1. L-Reduction preserves approximation.
82 L-reduction
P1 is L-reducible to P2 (P1 L P2) if there exist:
a function f mapping instances x of P1 to instances f(x) of P2, a function g mapping solutions of f(x) to solutions of x, two constants a and b such that: 1. f and g are computable in polynomial-time computable.
2. For any instance x of P1: OPT( f(x) ) · a ¢ OPT(x).
3. For any instance x of P1 and any solution y of f(x): |OPT(x) – VAL(g(y))| · b ¢ |OPT(f(x)) – VAL(y)|.
Lemma:
If P1 L P2 and P2 L P3 then: P1 L P3. Reductions can be composed together.
83 Proving APX-completeness
Proving APX-completeness of problem P: Give a c-approximation (prove that P is in APX). Give an L-reduction from an APX-complete problem to P.
Proving APX-hardness of problem P: Give an L-reduction from an APX-hard problem to P.
Similar to NP-completeness. PNP implies PTASAPX.
84 NP-completeness and APX-completeness
NP-completeness APX-completeness 1. NP-complete: if one 1. APX-complete: if one problem is problem has a PTAS, polynomial-time then all have. solvable, then all are. 2. Polynomial-time 2. L-reductions. reductions. 3. Cook’s theorem: SAT 3. PCP theorem implies is NP-complete. Max 3-SAT is APX- complete.
85 Example L-reduction
Theorem: Max 3-SAT L Max 2-SAT. Proof: Transformation f: replace any clause (x,y,z) by: (x), (y), (z), (v), (:x,:y), (:x,:z), (:y,:z), (x,:v), (y,:v), (z, :v). Here v is a new variable (each clause gets a different one). Transformation g: restrict the truth assignment to the original variables. a=13, b=1. To prove:
1. For any instance I of P1: OPT( f(I) ) · a ¢ OPT(I).
2. For any instance x of P1 and any solution y of f(x): |OPT(x) – VAL(g(y))| · b ¢ |OPT(f(x)) – VAL(y)|.
86 Example L-reduction
Transformation f: replace any clause (x,y,z) by: (x), (y), (z), (v), (:x,:y), (:x,:z), (:y,:z), (x,:v), (y,:v), (z, :v). Transformation g: restrict the truth assignment to the original variables. a=13, b=1.
1. For any instance I of P1: OPT( f(I) ) · a ¢ OPT(I). OPT(f(I)) = 6|C| + OPT(I) · 12 OPT(I) + OPT(I) = 13 OPT(I) 1. OPT(f(I)) = 6|C| + OPT(I) follows from truth tables. 2. 6|C| · 12 OPT(I) because the optimum always satisfies half the clauses.
2. For any instance x of P1 and any solution y of f(x): |OPT(x) – VAL(g(y))| · b ¢ |OPT(f(x)) – VAL(y)|. |OPT(x) – VAL(g(y))| = |OPT(f(x)) – VAL(y)|, the transformation goes from 1 true clause to 7 or from 0 to 6 (truth tables)
87 APX-completeness of Max 2-SAT
Corollary: Max 2-SAT is APX-complete. APX-hard by the L-reduction from Max 3-SAT. In APX as there is a 2-approximation for MaxSat (hence also for Max 2-SAT).
88 Approximation algorithms - Algorithms and Networks INAPPROXIMABILITY OF MAX INDEPENDENT SET
89 Inapproximability of maximum independent set
We have seen that max independent set does not allow absolute constant difference approximation, nor an FPTAS.
Next, we show that there also is no PTAS, by proving APX- hardness. L-reduction from Max 2-SAT:
No, we will not prove APX-completeness. Max independent set also not in APX as we will conclude thereafter.
90 L-reduction from max 2-SAT to maximum independent set
L-Reduction: Max 2-SAT L Maximum Independent Set. Transformation f: Replace any clause (x,y) by two vertices x and y connected by an edge. For all variables x: connect all pairs (x,:x) by an edge. Transformation g: Set x to true if any x vertex in the solution, to false if any :x vertex in the solution. Choose arbitrarily otherwise. a=1, b=1. To prove:
1. For any instance I of P1: OPT( f(I) ) · a ¢ OPT(I).
2. For any instance x of P1 and any solution y of f(x): |OPT(x) – VAL(g(y))| · b ¢ |OPT(f(x)) – VAL(y)|.
91 L-reduction from max 2-SAT to maximum independent set
Transformation f: Replace clause (x,y) by vertices x and y connected by an edge. For all variables x: connect all pairs (x,:x) by an edge. Transformation g: Set x to true if any x vertex in the solution, to false if any :x vertex in the solution. Choose arbitrarily otherwise. a=1, b=1.
1. For any instance I of P1: OPT( f(I) ) · a ¢ OPT(I). In f(I) we can choose vertices corresponding to a truth assignment, or choose less vertices, never more.
2. For any instance x of P1 and any solution y of f(x): |OPT(x) – VAL(g(y))| · b ¢ |OPT(f(x)) – VAL(y)|. OPT(x) = OPT(f(x)). VAL(g(y)) ¸ VAL(y), as we may have some unassigned variables.
92 No PTAS for maximum independent set
Corollary: There is no approximation algorithm for maximum independent set that, for each fixed e>0: Is an (1+e)-approximation algorithm. Uses polynomial time.
Next step is to prove that we cannot find a c- approximation for maximum independent set. We will use a self reduction.
93 Self reduction for maximum independent set
Lemma: If independent set can be approximated with approximation ratio c, then it can be approximated with ratio √c. Proof: 1. Given a graph G, we construct G x G: 1. It has nodes (u,v) corresponding to tuples of vertices from G. 2. It has edge (u,v)-(u’,v’) iff: • u = u’, and (v,v’) an edge in G. • (u,u’) an edge in G. 2. Claim G has an independent set of size k, if and only if, G x G has an independent set of size k2. 3. Run the c-approximation on G x G with result J: 1. The result in G x G has OPT2 · c |J|. 2. Thus in G: √OPT · √c √|J|
94 Proof of claim
Claim: G has an independent set of size ¸ k, if and only if, G x G has an independent set of size ¸ k2. Proof: 1. Given an independent set I in G of size k, we construct one in G x G of size k2 by taking all (v,u) for v 2 I and u 2 I. 2. Let J be an independent set in G x G of size ¸ k2. 1. Consider all u with (u,v) 2 J. 2. If these vertices form an independent set of size ¸ k in G then we are done. 3. Otherwise, there are less than k different such u. 4. So for at lease one fixed u, there must be more than k different (u,v) 2 J. 5. These vertices v form an independent set in G of size ¸ k.
95 No PTAS for maximum independent set
If independent set can be approximated with approximation ratio c, then it can be approximated with ratio √c.
Corollary: If independent set can be approximated with approximation ratio c, then it can be approximated with ratio 1 + ² (it has a PTAS). For fixed ², we can repeat the square root trick often enough (but a fixed number of times) until the ratio is good enough.
Corollary: Maximum independent set is not in APX. It has no PTAS assuming PNP, hence also no c- approximation.
96 Concluding remarks on independent set
Maximum independent set is in f(n)-APX, where:
Things can be even worse: TSP without triangle inequality cannot be approximated by any polynomial-time approximation algorithm with an approximation ratio p(n) for any polynomial p(n) unless P=NP.
97 Approximation algorithms - Algorithms and Networks CONCLUSION
98 Conclusion on approximation
Qualities of polynomial-time approximation algorithms: 1. Absolute constant difference. 2. FPTAS: Fully polynomial-time approximation scheme. 3. PTAS: Polynomial-time approximation scheme. 4. APX: Constant-factor approximation. 5. f(n)-APX: Approximation by a factor of f(n).
Proving that a quality of approximation algorithm ‘most likely’ does not exist: Gap technique. APX-completeness / APX-hardness. Self reduction. Polynomial bound on solution size, then there is no FPTAS.
99