Introduction to Approximation algorithms Algorithms and Networks 2016/2017 Johan M. M. van Rooij Hans L. Bodlaender 1 Solution Optimal Bound on Good quality quality solution, no quality Computation guarantee time Polynomial Approximation Construction Polynomial solution algorithms heuristics algorithms Exact algorithms: Hybrid Meta • Tree search algorithms heuristics: Super • Dynamic • Column • Local polynomial programming generation search and/or no • Integer linear without • Genetic guarantee programming complete algorithms • …… branch-and- price 2 Solution Optimal Bound on Good quality quality solution, no quality Computation guarantee time Polynomial ApproximationAlgorithms Construction Polynomial solution algorithmsand networks heuristics algorithms ExactAlgorithms algorithms : Hybrid Meta a• ndTree networks search algorithms heuristics: Super • Dynamic • Column • Local polynomial programming generation search and/or no • Integer linear without • Genetic guarantee programming complete algorithms • …… branch-and- price 3 What to do if a problem is NP-complete? We have already seen many options to deal with NP- complete problems. Earlier in this course: Special cases (planar graphs), exact exponential-time algorithms. Later in this course: FPT and treewidth. In other courses: local search, ILP, constraint programming, … Approximation algorithms are one of these options. An approximation algorithm is a heuristic with a performance guarantee. We consider polynomial-time approximation algorithms. Non-optimal solutions, but with some performance guarantee compared to the optimal solution. Also useful as a starting point for other approaches: Local search, branch and bound. 4 What is a polynomial-time approximation algorithm? An algorithm that... 1. ‘Solves’ (applies to) an optimisation problem. A minimisation problem or a maximisation problem. E.g., independent set, vertex cover, knapsack, max-SAT. Not: SAT, 3-SAT, etc. 2. Runs in polynomial time. 3. It has a bound on the quality of the solution. This is called the approximation ratio. E.g.: value of algorithm result / optimal solution value · c. We call such an algorithm a c-approximation. 5 Approximation ratio For an instance I of an optimisation problem, let OPT(I) be the value of the optimal solution to I. ALG(I) be the value computed by the approximation algorithm. An algorithm for a minimisation problem has an approximation ratio c, if for all instances I: An algorithm for a maximisation problem has an approximation ratio c, if for all instances I: 6 Class of approximation algorithms Class of problems with approximation algorithms whose approximation ratio is a constant: APX. Subclass of NP. We only consider optimisation problems in NP. Other notions of approximability also exist. We see some of them the next lecture. And one at the end of this lecture. Notions of APX-completeness also exist. As NP-completeness. We will see this at the end of the lectures on approximation algorithms. 7 This lecture Approximation algorithms for a series of problems: Travelling Salesman Problem. Minimum (Weight) Vertex Cover. Max Satisfiability. 8 Approximation algorithms - Algorithms and Networks MAXIMUM SATISFIABILITY 9 Max satisfiability (decision version) Maximum Satisfiability (Max SAT). Instance: A set of clauses C containing literals of variables from a set X and an integer k. Question: Does there exist a truth assignment to the variables in X such that at least k clauses in C are satisfied? k-Satisfiability (k-SAT): Instance: set of clauses C (logical formula in CNF format) where each clause contains at most k literals. Question: does there exist a satisfying assignment to the variables in C, i.e., a truth assignment such that each clause has at least one literal set to true. Defined similarity: Maximum k-Satisfiability (Max k-SAT) 10 Approximating Max Satisfiability Algorithm: 1. Take any truth assignment a to the variables X. 2. Let :a be the assignment a with all variables negated. 3. Return form {a,:a} the assignment that satisfies most clauses. This is a 2-approximation. ALG ¸ |C|/2, OPT · |C|. Why? Can you also give approximation algorithms for max k- satisfiability? 11 Approximation algorithms - Algorithms and Networks TRAVELLING SALESMAN PROBLEM 12 The travelling salesman problem Instance: n vertices 4 (cities), distance 1 2 between every pair of 2 3 vertices. 5 Question: Find shortest 2 (simple) cycle that visits 3 2 4 every city? 4 4 Restriction on distances: 1 2 1 2 distances are non- 2 3 2 3 5 negative (or positive). 5 Triangle inequality: 2 2 for all x, y, z: 3 4 3 4 2 2 w(x,y) + w(y,z) w(x,z) 13 11 13 A simple algorithm Consider the Travelling Salesman Problem with triangle inequality. Algorithm: 1. Find a minimum spanning tree. 2. Output vertices of tree in preorder. Preorder: visit a node before its children. Take a tour following the MST This algorithm has approximation ratio 2: OPT ≥ MST. 2 MST ≥ ALG. ALG / OPT ≤ 2MST / MST = 2. 14 Can we do better? Yes: Christofides’ algorithm Christofides’ Algorithm: 1. Construct a Minimum Spanning Tree T. 2. Set W = {v | v has odd degree in tree T}. 3. Compute a minimum weight matching M in the graph G[W]. 4. Look at the graph T+M. Note that T+M is Eulerian! 5. Compute an Euler tour C’ in T+M. 6. Add shortcuts to C’ to get a TSP-tour. This is a 3/2-approximation. Proof on the blackboard. 15 Ratio 1.5 Total length edges in T: at most OPT Total length edges in matching M: at most OPT/2. T+M has length at most 3/2 OPT. Use triangle inequality. 16 Proving approximation ratios We just saw two algorithms for TSP with triangle inequality. We proved approximation ratios in the following way: 1. Use a quantity x that relates to both the optimal solution and the result of the algorithm A minimum spanning tree in both cases. 2. Prove that OPT ¸ c1 x. 3. Prove that ALG · c2 x 4. Combine both inequalities for a bound on ALG / OPT. Next, we consider some more problems. 17 Approximation algorithms - Algorithms and Networks MINIMUM (WEIGHT) VERTEX COVER 18 Approximation for minimum vertex cover Algorithm: 1. Let E’ = E, C = ;. 2. While E’ ;. 1. Let {u,v} be any edge from E’. 2. C := C [ {u,v}. 3. Remove every edge incident to u or v from E’. 3. Return C. Approximation algorithm? Runs in polynomial time. Returns a vertex cover. How good is this vertex cover? 19 2-approximation for vertex cover Theorem: The algorithm on the previous slide is a 2-approximation. Proof: Let A be the set of edges which endpoints we picked. OPT ¸ |A|, because every edge in A must be covered. ALG = 2|A| · 2OPT, hence ALG/OPT · 2. 20 Minimum weight vertex cover Minimum weight vertex cover: Vertex cover where each vertex has a weight. We look for the minimum weight vertex cover. 2-approximation for vertex cover no longer works. We may select very heavy vertices using that algorithm. Next, we will give a 2-approximation for minimum weights vertex cover using LP rounding. 21 An ILP formulation of the problem Consider the following ILP: minimise: w(v)x(v) v V X2 subject to:x(u) + x(v) 1 for each (u; v) E ¸ 2 x(v) 0; 1 for each v V 2 f g 2 It’s LP relaxation is ILP with the last constraint replaced by: 0 · x(v) · 1. Linear programming can be solved in polynomial time. Not by the simplex algorithm!! Ellipsoid method / interior point methods. 22 2-approximation algorithm for minimum weight vertex cover Algorithm: 1. Compute the optimal solution to the LP relaxation. 2. Output all v with x(v) ¸ ½. This algorithm returns a vertex cover. For every edge, sum of incident vertices at least 1. Hence at least one of the vertex variables at least ½. Approximation algorithm? Runs in polynomial time and produces a vertex cover. How good is this vertex cover? Proof of 2-approximation on the blackboard! 23 Proof of 2-approximation algorithm for minimum weight vertex cover Let z* be the solution to the LP. Because any vertex cover is a solution to the LP we have: Also, we can bound ALG in terms of z*: Hence: QED 24 Approximation algorithms - Algorithms and Networks CONCLUSION 25 Conclusion We have seen several different approximation algorithms for different problems: 2-approximations for Max Satisfiability (and Max k-SAT) 1.5- and 2-approximations for TSP. 2-approximations for vertex cover and weighted vertex cover. c-approximations, for a constant c, are called constant factor approximation algorithms. There are more different types of approximations. These we will see after the break. 26 The Landscape of Approximation Algorithms Algorithms and Networks 2016/2017 Johan M. M. van Rooij Hans L. Bodlaender 27 What is a polynomial-time approximation algorithm? An algorithm that... 1. Solves an optimisation problem. 2. Runs in polynomial time. 3. It has a bound on the quality of the solution. Approximation ratio c: ALG/OPT · c for minimisation problems. OPT/ALG · c for maximisation problems. Ratio always bigger or equal to 1. 28 Different forms of approximation algorithms (outline of two lectures) Qualities of polynomial-time approximation algorithms: 1. Absolute constant difference. |OPT – ALG| · c 2. FPTAS: Fully polynomial-time approximation scheme. Approximation ratio 1+² for any ² > 0, while the algorithm runs in time polynomial in n and 1/². 3. PTAS: Polynomial-time approximation scheme. Approximation ratio 1+² for any ² > 0, while the algorithm runs in polynomial time for any fixed ². 4. APX: Constant-factor approximation. Approximation ratio: ALG/OPT · c for minimisation problems. Approximation ratio: OPT/ALG · c for maximisation problems. 5. f(n)-APX: Approximation by a factor of f(n). f(n) depends only on the size of the input. 29 Qualities of poly-time approximation algorithms: 1. Absolute constant difference.