Graphs 12/12/2019 11:37 AM

Presentation for use with the textbook, Algorithm Design and Applications, by M. T. Goodrich and R. Tamassia, Wiley, 2015

Approximation Algorithms

1

1

Bike Tour

Suppose you decide to ride a bicycle around Ireland

 you will start in Dublin

 the goal is to visit Cork, Galway, Limerick, and Belfast before returning to Dublin

What is the best itinerary?

 how can you minimize the number of kilometers yet make sure you visit all the cities?

2

1 Graphs 12/12/2019 11:37 AM

Optimal Tour If there are only 5 cities it’s not too hard to figure out the optimal tour

 the shortest path is most likely a “loop”

 any path that crosses over itself will be longer than a path that travels in a big circle

3

Exhaustive Search: Too Many Tours

There is a problem with the exhaustive search strategy  the number of possible tours of a map with n cities is (n − 1)! / 2  n! is the product n × (n − 1) × (n − 2) ... × 2 × 1 The number of tours grows incredibly quickly as we add cities to the map

#cities #tours 512 660 7 360 8 2,520

9 20,160 The number of tours for 25 cities: 10 181,440 310,224,200,866,619,719,680,000

4

2 Graphs 12/12/2019 11:37 AM

The Traveling Salesman

Computer scientists call the problem of finding an optimal path between n points the traveling salesman problem (TSP) The TSP is a famous problem

 first posed by Irish mathematician W. R. Hamilton in the 19th century

 intensely studied in operations research and other areas since 1930

This tour of 13,500 US cities was generated by an advanced algorithm that used several “tricks” to limit the number of possible tours

Required 5 “CPU-years” http://www.tsp.gatech.edu/

5

Dealing with Hard Combinatorial Problems (CP) 1. For many Combinatorial Problems (CP), not all instances are hard to solve. Some special cases can be solved easily, e.g, graph coloring is hard, but trees can be two-colored easily. 2. For small problems, it may be possible to solve them using backtracking search or branch-and- bound algorithms. 3. Look for good-enough approximate solutions. 4. Use random algorithms to find solutions with high probability.

6

3 Graphs 12/12/2019 11:37 AM

Bin Packing

Bin Packing Problem The bins (capacity 1)

1 …

.5 .7 .5 .2 .4 .2 .5 .1 .6

.2 .4 .2 .1 .5 .5 .5 .6 .7

Items to be packed

7

Bin Packing

Bin Packing Problem

1 …

.5 .7 .5 .2 .4 .2 .5 .1 .6

Optimal Packing

.1 .5 .2 .5 .6 Optimal solution: .7 .5 .4 C* = 4 .2

8

4 Graphs 12/12/2019 11:37 AM

First Fit Packing Algorithm

Bin Packing Problem .5 .7 .5 .2 .4 .2 .5 .1 .6

.1 .5 .2 .5 Optimal solution: .6 C* = 4 .7 .5 .4 .2

First Fit Packing Algorithm (FF): For each item si, find the first bin (from left to right) which can hold si.

.1 FF solution: C = 5 .5 .2 .2 C .7 .6  2 .5 .4 .5 C *

9

Best Fit Packing Algorithm

Bin Packing Problem .5 .7 .5 .2 .4 .2 .5 .1 .6

Best Fit Packing Algorithm (BF): For each item si, find the most-full bin which can hold si.

.1 BF solution: C = 5 .5 .2 .2 C .7 .6  2 .5 .4 .5 C *

10

5 Graphs 12/12/2019 11:37 AM

First Fit Decreasing Packing Algorithm

Bin Packing Problem .5 .7 .5 .2 .4 .2 .5 .1 .6

First Fit Decreasing (FFD): First sort the items in decreasing order, then apply FF. .7 .6 .5 .5 .5 .4 .2 .2 .1 FFD solution: C = C* = 4 .1 .2 .4 .5 .2 C .7 .6  2 .5 .5 C *

11

Best Fit Decreasing Packing Algorithm

Bin Packing Problem .5 .7 .5 .2 .4 .2 .5 .1 .6

Best Fit Decreasing (BFD): First sort the items in decreasing order, then apply BF. .7 .6 .5 .5 .5 .4 .2 .2 .1

BFD solution: C = C* = 4 .1 .2 .4 .5 .2 C .7 .6  2 .5 .5 C *

12

6 Graphs 12/12/2019 11:37 AM

Next Fit Packing Algorithm

Bin Packing Problem .5 .7 .5 .2 .4 .2 .5 .1 .6

.1 .5 .2 .5 .6 C* = 4 .7 .5 .4 .2

Next Fit Packing Algorithm: If the current bin cannot

hold si, start a new bin (good for online application).

.2 .2 .1 .7 .6 .5 .5 .4 .5

C= 6

13

Approximation Terminology

Given an instance I of an optimization problem, let C* = OPT(I) be the cost of an optimal solution, and let C be the cost of the solution of an . The algorithm has an approximation ratio of ρ(n) if, for all solutions max(C/C*,C*/C) ≤ ρ(n). We say that an approximation algorithm with an approximation ration of ρ(n) is a ρ(n)- approximation algorithm.

14

7 Graphs 12/12/2019 11:37 AM

Approximation Terminology

An approximation scheme is an approximation algorithm that takes an instance and an ε > 0, and produces a (1+ε) approximation (ρ(n) = 1+ε). If an approximation scheme runs in polynomial time in the size of its input when given any ε, we say it is a polynomial-time approximation scheme. Note that the running time could still increase rapidly as ε decreases.

15

Approximation Algorithms

Not always optimal solution, but with some performance guarantee (eg, no worst than twice the optimal)

Even though we don’t know what the optimal solution is!!!

16

8 Graphs 12/12/2019 11:37 AM

Approximate Bin Packing Algorithms: C* = OPT(I), C is the approximate cost

B … 1 1 B2 B3 BC-1 BC

s(B1)+s(B2) > 1

s(B2)+s(B3) > 1

si = s(Bi): total size of items in Bi … … … … …

s(BC-1)+s(BC) > 1 Let s1+ s2 + … + sm= S s(B1) > 0, s(BC) > 0 Then C*  S 2(s(B )+s(B )+…+ s(B )) > C– 1 From 2S > C – 1 1 2 C Or 2S > C – 1 We have 2C*  2S  C

17

Approximate Bin Packing Algorithms

Theorem • First-Fit: C ≤ 17/10C* + 2 • First-Fit Decreasing: C ≤ 11/9C* + 4

18

9 Graphs 12/12/2019 11:37 AM

Vertex Cover Problem

Let G=(V, E). The subset S of V that meets every edge of E is called the cover. The Vertex Cover problem is to find a vertex cover of the minimum size. It is a hard combinatorial optimization problem (NP- complete).

19

19

Examples of vertex cover

Red points are in vertex cover.

20

20

10 Graphs 12/12/2019 11:37 AM

Approximating Vertex Cover

The following algorithm is a 2-approximation algorithm for the vertex-cover optimization problem:

 Choose an edge e from G.

 Add both e’s endpoints (say u and v) to the vertex cover.

 Remove all edges incident to u and v.

 Repeat while there are edges left.

21

APPROX_VERTEX_COVER(G) 1 C   2 E'  E(G) 3 while E'   4 do let (u,v) be an arbitrary edge of E' 5 C  C {u,v} 6 remove from E' every edge incident on either u or v 7 return C

22

11 Graphs 12/12/2019 11:37 AM

b c d b c d

a e f g a e f g

b c d b c d

a e f g a e f g

b c d b c d

a e f g a e f g

Approx. solution Optimal solution

C = 6, C* = 3 Complexity: O(E)

23

C*: the size of optimal solution C: the size of approximate solution Theorem A: the set of edges selected in step 4 APPROX_VERTEX_COVER has ratio bound of 2. Proof. Let A be the set of selected edges. C  2 | A | When one edge is selected, 2 vertices are added into C.

| A | C * No two edges in A share a common endpoint due to step 6.

 C  2C *

24

12 Graphs 12/12/2019 11:37 AM

Vertex Cover: Greedy Algorithm 2

Idea: Keep finding a vertex which covers the maximum number of edges.

Greedy Algorithm 2: 1. Find a vertex v with maximum degree. 2. Add v to the solution and remove v and all its incident edges from the graph. 3. Repeat until all the edges are covered.

How good is this algorithm?

25

25

Vertex Cover: Greedy Algorithm 2

OPT = 6, all red vertices.

SOL = 11, if we are unlucky in breaking ties. First we might choose all the green vertices. Then we might choose all the blue vertices. And then we might choose all the orange vertices. 26

26

13 Graphs 12/12/2019 11:37 AM

Vertex Cover: Greedy Algorithm 2

Not a constant factor k! vertices of degree k approximation algorithm!

… Generalizing the example!

… ………

k!/k vertices of degree k k!/(k-1) vertices of degree k-1 … k! vertices of degree 1

OPT = k!, all top vertices.

SOL = k! (1/k + 1/(k-1) + 1/(k-2) + … + 1) ≈ k! log(k), all bottom vertices. 27

27

Euclidian Traveling Salesman Problem

The costs in the traveling salesman problem do not need to represent distances (e.g., they may be things like airline fares). If the costs do represent geometric distances, they obey the triangle inequality for all vertices u, v, and w: c(u,w) ≤ c(u,v) + c(v,w) The problem remains very hard. b 5 4

a c 7

28

14 Graphs 12/12/2019 11:37 AM

Euclidian Traveling Salesman Problem

Euclidian Traveling Salesman Problem is the Traveling Salesman Problem with the Triangle Inequality. The following is a 2-approximation of TSP with the triangle inequality:

 compute a minimum spanning T of the weighted clique

 order the vertices according to a preorder walk on T

 return this order as the TSP tour

29

Euclidian Traveling Salesman Problem MST root a e a e f f b g h b g h

c d c d

Pre-Order walk Hamiltonian Cycle a e a e f f b g h b g h

c d c d

30

30

15 Graphs 12/12/2019 11:37 AM

Euclidian Traveling Salesman Problem

APPROX-TSP-TOUR(G, W) 1 select a vertex r Є V[G] to be root. 2 compute a MST for G from root r using Prim Alg. 3 L = list of vertices in preorder walk of that MST. 4 return L as the Hamiltonian cycle.

31

31

Traveling salesman problem This is polynomial-time 2-approximation algorithm. (Why?)

 Because:  APPROX-TSP-TOUR is O(V2)  C(MST) ≤ C(H*) H*: optimal soln Optimal C(W) = 2C(MST) W: Preorder walk Pre-order C(W) ≤ 2C(H*) C(H) ≤ C(W) H: approx soln & C(H) ≤ 2C(H*) triangle inequality Solution

32

32

16 Graphs 12/12/2019 11:37 AM

Solving a maze Given a maze, find a path from start to finish At each intersection, you have to decide between three or fewer choices:  Go straight  Go left  Go right You don’t have enough information to choose correctly Each choice leads to another set of choices One or more sequences of choices may (or may not) lead to a solution Many types of maze problem can be solved with backtracking

33

33

Backtracking Suppose you have to make a series of decisions, among various choices, where

 You don’t have enough information to know what to choose.

 Each decision leads to a new set of choices

 Some sequence of choices (possibly more than one) may be a solution to your problem Backtracking is a methodical way of trying out various sequences of decisions, until you find one that “works”.

34

34

17 Graphs 12/12/2019 11:37 AM

Coloring a map

You wish to color a map with not more than four colors

 red, yellow, green, blue Adjacent countries must be in different colors You don’t have enough information to choose colors Each choice leads to another set of choices One or more sequences of choices may (or may not) lead to a solution Many coloring problems can be solved with backtracking 35

35

Longest Path Problem

Given two nodes A and B, find the longest (simple) path from A to B.

Solution: Starting from A, using backtrack search. 0 8 A 4 2 8 712 4 B C D 39 25 E F

36

18 Graphs 12/12/2019 11:37 AM

Real and virtual graphs There is a type of data structure called a graph

 But we are not using it here If we diagram the sequence of choices we make, the diagram looks like a tree

 In fact, we did just this in DFS algorithm.

 Backtracking algorithm works in general in virtual graph (implicitly defined by the starting state and the consequence relation induced by decisions.

 Our backtracking algorithm “sweeps out a tree” in “problem state space”.

37

37

Knight Puzzle

In this puzzle, move the knight legally in the board. The object is to go to each position exactly once.

You don’t have enough information to jump correctly Each choice leads to another set of choices One or more sequences of choices may (or may not) lead to a solution Many kinds of puzzle can be solved with backtracking search. 38

38

19 Graphs 12/12/2019 11:37 AM

Backtracking dead end ? dead end dead end ? start ? ? dead end dead end ? success!

A solution for 5x5 board.

39

Backtracking dead end ? dead end dead end ? start ? ? dead end dead end ? success!

A solution for 5x5 board.

40

20 Graphs 12/12/2019 11:37 AM

The backtracking algorithm

Backtracking is really quite simple--we “explore” each node, as follows: To “explore” node N: proc Explore(N) 1. If N is a goal node, return “success” 2. If N is a leaf node, return “failure” 3. For each child C of N, 3.1. result = Explore(C) 3.2. If result was “success”, return “success” 4. Return “failure”

41

41

DFS vs Backtracking Search

DFS visits each node once and marks the node once it’s visited. Hence the linear . Backtracking Search in general does not mark nodes, and may visit a node many times. Example: What algorithm to use?

 Find a path in a graph: DFS

 Find the shortest path in a graph: Dijkstra’s

 Find the longest simple path in a graph: Backtracking

42

42

21 Graphs 12/12/2019 11:37 AM

Full example: Map coloring

The Four Color Theorem states that any map on a plane can be colored with no more than four colors, so that no two countries with a common border are the same color For most maps, finding a legal coloring is easy For some maps, it can be fairly difficult to find a legal coloring We will develop a complete Java program to solve this problem

43

43

Data structures We need a data structure that is easy to work with, and supports:

 Setting a color for each country

 For each country, finding all adjacent countries We can do this with two arrays

 An array of “colors”, where countryColor[i] is the color of the ith country

 A ragged array of adjacent countries, where map[i][j] is the jth country adjacent to country i  Example: map[5][3]==8 means the 3th country adjacent to country 5 is country 8

44

44

22 Graphs 12/12/2019 11:37 AM

Creating the map

01 4 int map[][]; 2 3 6 void createMap() { 5 map = new int[7][]; map[0] = new int[] { 1, 4, 2, 5 }; map[1] = new int[] { 0, 4, 6, 5 }; map[2] = new int[] { 0, 4, 3, 6, 5 }; map[3] = new int[] { 2, 4, 6 }; map[4] = new int[] { 0, 1, 6, 3, 2 }; map[5] = new int[] { 2, 6, 1, 0 }; map[6] = new int[] { 2, 3, 4, 1, 5 }; } 45

45

Setting the initial colors

static final int NONE = 0; static final int RED = 1; static final int YELLOW = 2; static final int GREEN = 3; static final int BLUE = 4;

int mapColors[] = { NONE, NONE, NONE, NONE, NONE, NONE, NONE };

46

46

23 Graphs 12/12/2019 11:37 AM

The main program

(The name of the enclosing class is ColoredMap)

public static void main(String args[]) { ColoredMap m = new ColoredMap(); m.createMap(); boolean result = m.explore(0, RED); System.out.println(result); m.printMap(); }

47

47

The backtracking method

boolean explore(int country, int color) { if (country >= map.length) return true; if (okToColor(country, color)) { mapColors[country] = color; for (int i = RED; i <= BLUE; i++) { if (explore(country + 1, i)) return true; } } return false; }

48

48

24 Graphs 12/12/2019 11:37 AM

Checking if a color can be used

boolean okToColor(int country, int color) { for (int i = 0; i < map[country].length; i++) { int ithAdjCountry = map[country][i]; if (mapColors[ithAdjCountry] == color) { return false; } } return true; }

49

49

Printing the results

void printMap() { for (int i = 0; i < mapColors.length; i++) { System.out.print("map[" + i + "] is "); switch (mapColors[i]) { case NONE: System.out.println("none"); break; case RED: System.out.println("red"); break; case YELLOW: System.out.println("yellow"); break; case GREEN: System.out.println("green"); break; case BLUE: System.out.println("blue"); break; } } }

50

50

25 Graphs 12/12/2019 11:37 AM

Recap

We went through all the countries recursively, starting with country zero At each country we had to decide a color

 It had to be different from all adjacent countries

 If we could not find a legal color, we reported failure

 If we could find a color, we used it and recurred with the next country

 If we ran out of countries (colored them all), we reported success When we returned from the topmost call, we were done 51

51

26