Lecture #9:

0.0.1 Graph Algorithms: Minimum Spanning (Chapter 23)

Problem 1 Given a connected undirected simple graph G = [V,E], and a func- tion w : E −→ R, (i.e. a weight w(u, v) for each edge (u, v) ∈ E, find a spanning tree T ∗ which minimizes w(T ) =  w(u, v) where the sum is over the edges in T . This tree represents the minimum weight connected subgraph in G.

First recall some properties of a spanning tree:

• Any two nodes in a tree are connected by a unique path • If any edge of a tree is removed, then the resulting graph is a pair of trees that are not connected. • |E| = |V | − 1 • If an edge not in the tree is added to it, we get precisely one .

Consider the graph below:

B 8E 7 G 4 2 9 11 A D 4 14 I 7 6 8 10 C F H 1 2

The following is a spanning tree (not one of minimum cost):

B E G

11 A D 4 14 I 7 8 10 C F H 1 2

If we add an edge (D,E) not in the above tree, you would get some thing

1 like:

B E G 2 11 A D 4 14 I 7 8 10 C F H 1 2

The red edges indicate the cycle formed by adding the new edge to the tree. If we now delete the edge (C,D) whose cost is the highest among the edges in the cycle, we get an ”improved” spanning tree that looks like:

B E G 2 11 A D 4 14 I

8 10 C F H 1 2

By doing this repeatedly until no such exchange is possible, yields an algo- rithm for finding the . There are two things with this statement that need proving/checking: (i)that the final result (the one where no such exchange improves the solution) is optimal; (ii)the number of such exchanges required. Algorithms of this sort which ”move” from one feasible so- lution to an improved feasible solution are called improvement type algorithms. In this particular case, we can show that an edge that is ”removed” is never again part of the subsequent spanning trees. This means that the number of such exchanges is no more than |E| − |V | + 1. But there is work to be done in each time in identifying the cycle and checking if the exchange improves the solution. Faster algorithms are available for this problem all of which are greedy methods. There are several of these and we present some of them.

0.0.2 General Greedy Method: This process consists of coloring edges all of which are initially uncolored. We use two colors — blue (to indicate that this edge is included in the final tree) and red (to indicate that this edge is not included in the final tree). To color the edges we maintain the following color invariant: There is a minimum spanning tree that includes all blue edges and none of the red edges. At the , the blue edges form the required spanning tree. To formulate the coloring rules, we need some definition: Definition 2 A cut in a graph G = [V,E] is a partition of V into two parts, X and X = V − X. An edge crosses the above cut if one end is in X and the other is in X. The remaining edges ”respect” the cut.

2 B 8E 7 G 4 2 9 11 A D 4 14 I 7 6 8 10 C F H 1 2

is the cut corresponding to X = {A, G, H, I} and X = {B,C,D,E,F}. Edges (A, B), (A, C), (E,G), (E,H), (F,H) cross the cut and others ”respect” the cut. All Greedy methods use two coloring rules: Blue Rule: Select a cut that blue edges respect (do not cross). Among uncolored edges crossing the cut, select one of minimum cost and color it blue. Red Rule: Select a simple cycle containing no red edges. Among uncolored edges on the cycle, select one of maximum cost and color it red. We are free to apply either rule at any time, in arbitrary order, until all edges are colored. BORUVKA’s Algorithm (1926): Begin with n blue trees each consisting of a node in G. Coloring Step: For every blue tree T , select a minimum cost edge incident from some node of T to some node not in T . Color all selected edges blue. Since every edge thas two ends, an edge may be selected twice in the same execution of the coloring step, once for each end. This algorithm is guaranteed to work correctly only if all edge weights are distinct (or they are totally ordered according to some criterion that preserves the ordering as per cost. One way to do this is to number edges arbitrarily from 1 to |E| and break cost "ties" by using this number). The evolution of this algorithm on the above graph is

3 shown below:

B E G

A D I

C F H

B E 7 G 4 2 9

A D I

C F H 1 2

B 8E 7 G 4 2 9

A D 4 I

C F H 1 2

An alternate solution for the last step is given by:

B E 7 G 4 2 9

A D 4 I

8 C F H 1 2

Kruskal’s Algorithms: (1956) Algorithm A:Apply the coloring step in nondecreasing order of costs. Coloring Step: Begin with n trees each consisting of a node. If the current edge e has both ends in the same tree, color it red; otherwise color it blue.

4 The evolution of this algorithm on the above graph is shown below:

8 7 4 B E G 8 7 2 9 B E G 11 4 A D 4 14 I 2 9 7 6 8 10 11 C F H A D 4 14 I 1 2 7 6 8 7 4 B E G 8 10 2 9 C F H 11 A D 4 14 I 1 2 7 6 8 10 C F H 8 7 1 2 B E G 4 2 9

8 7 11 14 4 B E G A D 4 I 2 9 7 11 6 A D 4 14 I 7 6 8 10 8 10 C F H C F H 1 2 1 2

8 7 4 B E G 2 9 B 8E 7 G 11 4 A D 4 14 I 2 9 7 6 8 10 C F H 11 14 1 2 A D 4 I 7 6 8 10 C F H B 8E 7 G 4 2 9 1 2 11 A D 4 14 I 7 6 8 10 C F H 8 7 1 2 4 B E G 8 7 2 9 4 B E G 2 9 11 11 A D 14 I A D 4 14 I 4 7 6 7 6 8 10 C F H 8 10 1 2 C F H 1 2 B 8E 7 G 4 2 9 11 A D 4 14 I 7 6 8 7 8 10 B E G C F H 4 1 2 2 9

8 7 11 B E G A D 4 14 I 4 2 9 7 11 6 A D 4 14 I 7 6 8 10 8 10 C F H C F H 1 2 1 2

Algorithm B:Apply the coloring step in nonincreasing order of costs. Coloring Step: Begin with the original graph with all edges present. If the removal of the current edge would disconnect the graph, color it blue; else color it red.

5 The evolution of this algorithm on the above graph is shown below:

8 7 8 7 4 B E G B E G 2 9 4 2 9 11 14 A D 4 I 11 14 7 6 A D 4 I 8 10 7 6 C F H 8 10 1 2 C F H 1 2 B 8E 7 G 4 2 9 8 7 11 14 B E G A D 4 I 4 2 9 7 6 8 10 11 C F H A D 4 14 I 1 2 7 6 8 10 C F H 1 2 B 8E 7 G 4 2 9 B 8E 7 G 11 4 A D 4 14 I 2 9 7 6 8 10 A 11 D 14 I C F H 4 1 2 7 6 8 10 C F H 8 7 B E G 1 2 4 2 9 11 14 A D 4 I B 8E 7 G 7 6 4 8 10 2 9 C F H 11 1 2 A D 4 14 I 7 6 8 7 8 10 4 B E G C F H 2 9 1 2 11 A D 4 14 I 7 6 8 7 8 10 B E G C F H 4 2 9 1 2 A 11 D 4 14 I 8 7 7 4 B E G 6 2 9 8 10 11 C F H A D 4 14 I 1 2 7 6 8 10 C F H 8 7 1 2 B E G 4 2 9 8 7 11 B E G A D 4 14 I 4 2 9 7 6 11 A D 4 14 I 8 10 7 6 C F H 8 10 1 2 C F H 1 2

8 7 4 B E G B 8E 7 G 2 9 4 2 9 11 11 A D 4 14 I A D 4 14 I 7 7 6 6 8 10 8 10 C F H C F H 1 2 1 2

Please note that this is one of two possible cases since there was a tie at one point in the algorithm.

Jarnik (1930)-Prim (1957)-Dijkstra (1959) Algorithm: This is commonly known as Prim’s algorithm. The algorithm starts with an arbitrarily selected node of the graph say s. It repeatedly uses the following coloring step: Coloring Step: Let T be the blue tree containing s. In the beginning, T has only the node s. Select a minimum cost edge one end of which is in T and the other not in T and color it blue.

6 The evolution of this algorithm on the above graph is shown below:

B 8E 7 G 4 2 9 11 A D 4 14 I 7 6 8 10 C F H 1 2

Start with node D

E E E 2 2 2

D D 4 D 4

H F H 2

E E 7 G 2 2 D 4 D 4

C F H C F H 1 2 1 2

B 8E 7 G 2

D 4

C F H 1 2

B 8E 7 G 4 2

A D 4

C F H 1 2

B 8E 7 G 4 2 9

A D 4 I

C F H 1 2

0.0.3 Proofs of validity: We have already defined cuts, ”crossing a cut” or ”respecting” a cut. An edge is a light edge crossing a cut if its weight is the minimum of any edge crossing the cut. There may be ties in selecting this minimum cost edge. Given a set A of edges which is contained in some minimum cost spanning tree, an edge (u, v) is a ”safe edge” for A if A∪ (u, v) is also in some minimum cost spanning tree. All these algorithms (with the exception of the improvement algorithm) add safe edges each time starting with the empty set. This is what needs proving in each algorithm.

Theorem 3 Let A be a subset of some minimum cost spanning tree edges in the graph G = [V,E]. Let [S,V − S] be any cut that respects A and let (u, v) be a light edge crossing the above cut. Then, edge (u, v) is safe for A.

Proof. See page 501-502 in the book.

7 Corollary 4 Let A be a subset of some minimum cost spanning tree edges in the graph G = [V,E] and let C be a connected (tree) in the forest GA = [V,A]. If (u, v) is a light edge connecting C with some other component of GA, then it is safe for A.

0.0.4 Implementation: Kruskal Algorithm A: Here, the main question is: Does the current edge e have both ends in the same tree? For this purpose, we use the UNION-FIND data structure (found in Chapter 22 of your book)˙ This data structure also called disjoint-set data structure maintains a collection of disjoint dynamic sets. Each set in the collection is identified by a representative, which is usually a member of the set. The main operations are:

MAKE-SET(x): creates a new set whose only member (and thus its rep- resentative) is pointed to by x. For this x must not be already in any existing set. UNION(x, y): unites sets containing x and y say Sx and Sy, into a new set that is the union of these two sets. It is assumed that these were disjoint before. The representative of this new set is usually chosen from among the representatives of the sets Sx and Sy. After doing this we destroy the sets Sx and Sy and now we have only the set Sx ∪ Sy in the collection. FIND-SET(x): returns a pointer to the representative of the (unique) set containing x. Analysis of this data structure is based on two parameters: the number, n, of MAKE-SET operations and the total number, m, of MAKE-SET, UNION, and FIND-SET operations. There can be at most n−1 UNION operations since there are only n elements and these sets are disjoint. Also, m ≥ n. Applications: CONNECTED-COMPONENTS(G): for each vertex v ∈ V [G] do MAKE-SET(v) for each edge (u, v) ∈ E[G] do if FIND-SET(u) = FIND-SET(v) then UNION(u, v)

SAME-COMPONENT(u, v) if FIND-SET(u) = FIND-SET(v) then return TRUE else return FALSE

Kruskal Algorithm A:

MST-KRUSKAL(G, w) A ←− φ

8 for each vertex v ∈ V [G] do MAKE-SET(v) sort the edges of E by nondecreasing weight w for each edge (u, v) ∈ E, in order of nondecreasing weight do if FIND-SET(u) = FIND-SET(v) then A ←− A ∪ {(u, v)} UNION(u, v) return A

The simplest implementation of UNION-FIND is shown in the diagrams below corresponding to UNION steps.

A B D G A B D G

I F I E H C E F H C

A B D H A B E H B I E H I H B F D F A D F I I G G E F E G A G C C C

D C

I H H B I H B I G B G F F E E G A A F E A D C D C

C D

In this simple implementation, the tree may have a large depth. This is not good for FIND-SET operations. Two strategies: always join the smaller tree to the larger one by making the parent of the root of the smaller tree as the root of the larger tree as shown below: first tree is BEFORE and the second is AFTER performing UNION(A,D)

I H I H B B F F A E E G G A

D C D C

9 For this we need to keep either the information on depth of the trees or the number of nodes. Usually, the latter is kept since it is easier to do. The second strategy is known as PATH-COMPRESSION. Each time we do a FIND-SET operation, we traverse the tree twice once to find the root and the second time to make pointers of all nodes on the way to point directly to the root. This keeps the tree shallow: see figures below [Let us say we are processing edge (A,D) and the structure BEFORE looks like the first of these figures and we do a FIND-SET(D) then the diagram should look like the second one:

I H I H B B

E F F G A A E

C G D C

D

10 These are shown below for our case:

A B D G A B D G

I F I E H C E F H C

A B D F A B F E A G E F

H C I D H C G B I E G I D H C

I F F A I F A I G F G I G H C A H C A B B H C G E H C E B E E B

D D D D

======

11 A B D G A B D G A B D F

I F C I H E H I E G C E F H C

A B E F A A I G F G E F I D H C G B H E C B I D H C D

A I G F A I G F A B I F H C E D B H C E D B H C G E D

I F F A A H C E D I G G C E D H F B B B A I G H C E D

The second one is with path compression. Prim Algorithm: The key to implementing this algorithm is to find the minimum cost edge connecting the current tree to a node outside the tree efficiently. All nodes not in the tree reside in a priority queue Q based on a key field. For each node v, key[v] is the minimum weight of any edge connecting v to a node in the tree; key[v] = ∞, if no such edge exists. The field π(v) names the parent of v in the tree. The set A of edges in the tree is kept implicitly by the collection of {(v, π(v)}; v ∈ V − {r} − Q where r is the starting node and hence the root of the tree. At termination, Q is empty and A gives the minimum spanning tree.

MST-PRIM(G, w, r) Q ←− V [G] for each u ∈ Q do key[u] ←− ∞ key[r] ←− 0 π[r] ←−NIL while Q = φ do u ←−EXTRACT-MIN(Q) for each v ∈ Adj[u] do if v ∈ Q and w(u, v) < key[v] then π[v] ←− u

12 key[v] ←− w(u, v)

The priority queue as it occurs in our example is shown below:

E, H, 2 4

C, F, C, F, 7 6 7 6

B, G, A I A B G H 8 7

I

F, C, G, 2 1 7

C, G, B, G, B, I, 7 7 8 7 8 10

B, I, I, A, A A 8 10 10 8

B, A, I, 8 4 9

A, I, I, 8 9 9

0.0.5 Analysis: Kruskal Algorithm A: Time to sort edges by weight takes O(|E| lg |E|). Initialization takes O(|V |). The time required for UNION and FIND-SET operations involved with both strategies — joining the smaller tree to the larger one and path compression is given by O(|E| α(|E| , |V |) where the function α(m, n) is very slowly growing function and can for all practical purposes be taken to be a constant. Thus, the overall complexity is O(|E| lg |E|) = O(|E| lg |V |). Prim Algorithm: The performance of this algorithm depends on how the priority queue is implemented. We assume that binary heap is used. Initialization is to use BUILD-HEAP and takes O |V |) time. The loop is performed |V | times and each EXTRACT-MIN takes O(lg(|V |)) time so that the total effort so far is

13 O(|V | lg(|V |)). Test for membership in Q is implemented in constant time. DECREASE-KEY operation takes O(lg |V |) time each time (once for each edge) it is called. Thus, the overall time for Prim’s algorithm with this implementation is O(|V | lg(|V |) + |E| lg(|V |)) = O(|E| lg(|V |)). This is the same as in Kruskal’s algorithm. If we use other types of heaps, Prim’s algorithm’s bound can be improved to O(|E| + |V | lg(|V |)). This completes the discussion on this problem.

14