Minimum Spanning Tree

Minimum Spanning Tree • Each edge has a cost. • Find a minimal-cost subset of edges that CSE 326 w ill keep the graph connected. (must be a D ata Structures ST). 21 2 75 Unit 14 1 17 3 25 19 7 10 64 Reading: Chapter 9.5 6 4 4 18 5 80

1 2

Example of a Spanning Tree Minimum Spanning Tree Problem • Input:Undirected connected graph G = 21 2 75 (V,E) and a cost function C from E to the 1 17 3 reals. C(e) is the cost of edge e. 25 7 64 19 10 • O utput:A spanning tree T w ith minimum total cost. That is: T that minimizes 6 4 4 = 18 5 80 C(T) ƒC(e) e∈T

Price of this tree = 18+19+4+10+17+64 • A nother formulation:Remove from G edges w ith maximal total cost, but keep G connected. 3 4 Minimum Spanning Tree Minimum Spanning Tree Problem • D efinition: For a given partition of V into • Boruvka 1926 U and V-U, the cut defined by U is the • Kruskal 1956 set of edges w ith one end in U and one end in V-U. • Prim 1957 also by J arnik 1930 • Karger, Klein, Tarjan 1995 – R andomized linear time algorithm U V-U – Probably not practical, but very interesting

5 The cut defined by U 6

A n A lgorithm for MST The B lue/R ed Edge-coloring Rules

• The algorithm colors the edges of the • The blue rule:Find a cut w ith no blue graph. Initially, all edges are black. edge. Color blue the cheapest black edge • A blue edge – belongs to T. in the cut. • A red edge – does not belong to T. • The red rule: Find a cycle w ith no red • W e continue to color edges until w e have edge. Color red the most expensive black n-1 blue edges. edge in the cycle. • H ow do w e select w hich edge to color next? H ow do w e select its color? These rules can be applied in any order. W e w ill see tw o specific algorithms.

7 8 Example of Blue/Red rules (1) Example of B lue/R ed rules (2)

1 1 2 3 2 3 1 1 3 3 3 3 3 7 3 3 7 4 0 4 0 3 6 1 4 6 1 4 2 5 2 2 5 2

Consider the cut defined by {2,3} Consider the cycle (7-5-4) - color (1,2) blue - color (4,5) red

9 10

Example of Blue/Red rules (3) Example of B lue/R ed rules (4)

1 1 2 3 2 3 1 1 3 3 3 3 3 7 3 3 7 3 4 0 4 0 6 1 4 6 1 4 2 5 2 2 5 2

Consider the cut defined by {3,5,6} Consider the cycle (1-2-7-5) - color (5,7) blue - color (1,5) red

11 12 Example of Blue/Red rules (5) Example of B lue/R ed rules (6)

1 1 2 3 2 3 1 1 3 3 3 3 3 7 3 3 7 4 0 4 0 3 6 1 4 6 1 4 2 5 2 2 5 2

Consider the cycle (1-2-7-5-6) Consider the cut defined by {4} - color (2,7) red. - color (4,7) blue

13 14

Example of Blue/Red rules (7) Example of B lue/R ed rules (8)

1 1 2 3 2 3 1 1 3 3 3 3 3 7 3 3 7 3 4 0 4 0 6 1 4 6 1 4 2 5 2 2 5 2

Consider the cut defined by {6} Consider the cut defined by {3} - color (5,6) blue - color (2,3) blue

15 16 Example of Blue/Red rules (9) Example of B lue/R ed rules (10)

1 1 2 3 2 3 1 1 3 3 3 3 3 7 3 3 7 4 0 4 0 3 6 1 4 6 1 4 2 5 2 2 5 2

Consider the cut defined by {1,2,3} Final MST - color (1,6) blue

17 18

Proof of Blue/Red Rules Case 1: A pplying the B lue R ule • Claim: for any k ≥ 0, after w e color k edges there exists an MST that includes C(u,v) is minimal all the blue edges and none of the red edges. U • Proof:B y induction on k. u • B ase:k=0 trivially holds. ek V-U • Step:A ssume this is true after w e color v k-1 edges e1,e2,…,ek-1. Consider the coloring of ek.

19 20 Case 1: A pplying the Blue Rule Case 1: A pplying the B lue R ule

T C(u,v) is minimal Consider T’ = T ∪ (u,v) -(x,y) C(u,v) < C(x,y) T’ U U

u u x v V-U x v V-U

y y

T’ is also a minimum ∉ If (u,v) T, then T must includes some other edge C(T’) = C(T) + C(u,v) - C(x,y) spanning tree, and it (x,y) in the cut defined by U (T is connected, so C(T’) < C(T) includes ek there is a path u—v). 21 22

Case 2: A pplying the Red Rule Case 2: A pplying the R ed R ule

C(u,v) is maximal C(u,v) is maximal in some cycle in some cycle u u ek v v

∈ The cycle that causes us to color (u-v) red includes an A ssume (u,v) T. edge connecting the tw o component (w hose cost is at By removing (u-v) from T w e get tw o components. most c(u,v). è 23 There is an alternative MST, that does not include e24k O ne more point: W e can alw ays proceed Kruskal’s Greedy A lgorithm Select an edge e. Sort the edges by increasing cost; •If e connects tw o blue •O therw ise, e closes Initialize T to be empty; sub-trees, then there is a a cycle in w hich e is For each edge e chosen in increasing order do cut w ithout any blue edge the most expensive if adding e does not form a cycle then and w e can run the blue edge (w hy?) so w e add e to T rule on this cut. can color e red.

Proof:The algorithm follow s the blue/red rules: •If e closes a cycle – apply the red rule (by the sorting, e is the most expensive in this cycle). e e •O therw ise – apply the blue rule (e connects tw o components, consider the cut defined by any of them. e is the cheapest edge in this cut) 25 26

Example of Kruskal 1 Example of Kruskal 2

1 1 2 3 2 3 1 1 3 3 3 3 3 7 3 3 7 3 4 0 4 0 6 1 4 6 1 4 2 5 2 2 5 2

{7,4} {2,1} {7,5} {5,6} {5,4} {1,6} {2,7} {2,3} {3,4} {1,5} {7,4} {2,1} {7,5} {5,6} {5,4} {1,6} {2,7} {2,3} {3,4} {1,5} 0 1 1 2 2 3 3 3 3 4 0 1 1 2 2 3 3 3 3 4

27 28 Example of Kruskal 2 Example of Kruskal 3

1 1 2 3 2 3 1 1 3 3 3 3 3 7 3 3 7 4 0 4 0 3 6 1 4 6 1 4 2 5 2 2 5 2

{7,4} {2,1} {7,5} {5,6} {5,4} {1,6} {2,7} {2,3} {3,4} {1,5} {7,4} {2,1} {7,5} {5,6} {5,4} {1,6} {2,7} {2,3} {3,4} {1,5} 0 1 1 2 2 3 3 3 3 4 0 1 1 2 2 3 3 3 3 4

29 30

Example of Kruskal 4 Example of Kruskal 5

1 1 2 3 2 3 1 1 3 3 3 3 3 7 3 3 7 3 4 0 4 0 6 1 4 6 1 4 2 5 2 2 5 2

{7,4} {2,1} {7,5} {5,6} {5,4} {1,6} {2,7} {2,3} {3,4} {1,5} {7,4} {2,1} {7,5} {5,6} {5,4} {1,6} {2,7} {2,3} {3,4} {1,5} 0 1 1 2 2 3 3 3 3 4 0 1 1 2 2 3 3 3 3 4

31 32 Example of Kruskal 6 Example of Kruskal 7

1 1 2 3 2 3 1 1 3 3 3 3 3 7 3 3 7 4 0 4 0 3 6 1 4 6 1 4 2 5 2 2 5 2

{7,4} {2,1} {7,5} {5,6} {5,4} {1,6} {2,7} {2,3} {3,4} {1,5} {7,4} {2,1} {7,5} {5,6} {5,4} {1,6} {2,7} {2,3} {3,4} {1,5} 0 1 1 2 2 3 3 3 3 4 0 1 1 2 2 3 3 3 3 4

33 34

Example of Kruskal 8 Example of Kruskal 9

1 1 2 3 2 3 1 1 3 3 3 3 3 7 3 3 7 3 4 0 4 0 6 1 4 6 1 4 2 5 2 2 5 2

{7,4} {2,1} {7,5} {5,6} {5,4} {1,6} {2,7} {2,3} {3,4} {1,5} {7,4} {2,1} {7,5} {5,6} {5,4} {1,6} {2,7} {2,3} {3,4} {1,5} 0 1 1 2 2 3 3 3 3 4 0 1 1 2 2 3 3 3 3 4

35 36 D ata Structures for Kruskal Example of D U/F (1)

1 1 Find(5) = 7 • Sorted edge list 2 3 Find(4) = 7 1 3 {7,4} {2,1} {7,5} {5,6} {5,4} {1,6} {2,7} {2,3} {3,4} {1,5} 3 3 0 1 1 2 2 3 3 3 3 4 3 7 4 0 3 • D isjoint Union / Find 7 6 1 4 – Union(a,b) - union the disjoint sets 2 5 2 named by a and b – Find(a) returns the name of the set containing a {7,4} {2,1} {7,5} {5,6} {5,4} {1,6} {2,7} {2,3} {3,4} {1,5} 0 1 1 2 2 3 3 3 3 4

Remark: The set name is one of its members 37 u,v in the same set (u,v) is not added to T 38

Example of D U/F (2) Example of D U/F (3) Union(1,7) Find(1) = 1 1 1 1 2 3 Find(6) = 7 2 3 3 1 3 1 3 3 3 3 3 7 0 3 3 7 0 3 4 7 4 7 6 1 4 6 1 4 2 5 2 2 5 2

{7,4} {2,1} {7,5} {5,6} {5,4} {1,6} {2,7} {2,3} {3,4} {1,5} {7,4} {2,1} {7,5} {5,6} {5,4} {1,6} {2,7} {2,3} {3,4} {1,5} 0 1 1 2 2 3 3 3 3 4 0 1 1 2 2 3 3 3 3 4 u,v in different sets add (u,v) to T, union the sets. 39 40 Kruskal’s A lgorithm w ith D U / F A mortized Complexity

Sort the edges by increasing cost; Initialize T to be empty; • D isjoint union/find can be implemented for each edge {i,j} chosen in increasing order do such that the average time per operation is u := Find(i); essentially a constant. v := Find(j); • A n individual operation can be costly, but if (u ≠ v) then add {i,j} to T; over time the average cost per operation is Union(u,v); not. • O n average, each U/F operation takes O (m α(m,n)) time. Ekerman function. Practically, this is 41 a constant. 42

Prim’s A lgorithm Evaluation of Kruskal • W e maintain a single tree. • Initially, the tree consists of one vertex. • G has n vertices and m edges. • For each vertex not in the tree maintain the • Sort the edges - O (m log m). cheapest edge to a vertex in the tree (if exists). • Traverse the sorted edge list using 1 2 3 efficient UF - O (m α(m,n)). 1 3 3 • Total time is O (m log m). 3 7 4 0 3 6 1 4 2 5 2

43 44 Prim’s A lgorithm 2 Prim’s A lgorithm 3

1 1 2 3 2 3 1 1 3 3 3 3 3 7 3 3 7 3 4 0 4 0 6 1 4 6 1 4 2 5 2 2 5 2

45 46

Prim’s A lgorithm 4 Prim’s A lgorithm 5

1 1 2 3 2 3 1 1 3 3 3 3 3 7 3 3 7 4 0 4 0 3 6 1 4 6 1 4 2 5 2 2 5 2

47 48 Prim’s A lgorithm 6 Prim’s A lgorithm 7

1 1 2 3 2 3 1 1 3 3 3 3 3 7 3 3 7 3 4 0 4 0 6 1 4 6 1 4 2 5 2 2 5 2

49 50

Correctness Proof for Prim D ata Structures for Prim

• Repeatedly executes the blue rule (n-1 • A djacency Lists - w e need to look at all the times). edges from a new ly added vertex. • In each step w e consider the cut defined • A rray for the best edges to the tree. by the vertices that are already in T. 1 2 3 1 2 3 4 5 6 7 1 3 3 to 2 1 1 2 cost 3 4 3 3 3 7 4 0 3 6 1 4 2 5 2

51 52 D ata Structures for Prim • Priority queue for all edges to the tree (orange Evaluation of Prim edges). – Insert, delete-min, delete (e.g. binary heap). • n vertices and m edges. 1 1 2 3 2 3 • Priority queue O (log n) per operation. 1 1 3 3 3 3 • O (m) priority queue operations. 3 7 3 3 7 3 4 0 4 0 – A n edge is visited w hen a vertex incident 6 1 4 6 1 4 to it joins the tree. 2 5 2 2 5 2 • Time complexity is O (m log n). 1 2 3 4 5 6 7 1 2 3 4 5 6 7 • Storage complexity is O (m). to 2 1 1 2 to 2 7 7 1 cost 3 4 3 3 cost 3 0 1 3 53 54

Kruskal vs Prim

• Kruskal – Simple – Good w ith sparse graphs - O (m log m) • Prim – More complicated – Perhaps better w ith dense graphs - O (m log n)

N ote:O (log n)= O (log m) (since m