Outline MST Prim Kruskal Optimisation Minimum Spanning Trees Prim Kruskal NP-complete problems Lecturer: Georgy Gimel'farb COMPSCI 220 Algorithms and Data Structures 1 / 60 Outline MST Prim Kruskal Optimisation 1 Minimum spanning tree problem 2 Prim's MST Algorithm 3 Kruskal's MST algorithm 4 Other graph/network optimisation problems 2 / 60 Outline MST Prim Kruskal Optimisation Minimum Spanning Tree Minimum spanning tree (MST) of a weighted graph G: A spanning tree, i.e., a subgraph, being a tree and containing all vertices, having minimum total weight (sum of all edge weights). 0 1 0 1 4 4 1 2 1 2 2 4 3 4 3 2 4 3 4 3 3 1 3 1 2 2 4 5 4 5 The MST with the total weight 9 3 / 60 Outline MST Prim Kruskal Optimisation Finding Minimum Spanning Tree Many applications: • Electrical, communication, road etc network design. • Data coding and clustering. • Approximate NP-complete graph optimisation. • Travelling salesman problem: the MST is within a factor of two of the optimal path. • Image analysis. http://www.geeksforgeeks.org/applications-of-minimum-spanning-tree/ 4 / 60 Outline MST Prim Kruskal Optimisation Finding Minimum Spanning Tree Two efficient greedy Prim's and Kruskal's MST algorithms: • Each algorithm selects edges in order of their increasing weight, but avoids creating a cycle. • The Prim's algorithm maintains a tree at each stage that grows to span. • The Kruskal's algorithm maintains a forest whose trees coalesce into one spanning tree. • The Prim's algorithm implemented with a priority queue is very similar to the Dijkstra's algorithm. • This implementation of the Prim's algorithm runs in time O(m + n log n). • The Kruskal's algorithm uses disjoint sets ADT and can be implemented to run in time O(m log n). 5 / 60 Outline MST Prim Kruskal Optimisation Finding Minimum Spanning Tree Two efficient greedy Prim's and Kruskal's MST algorithms: • Each algorithm selects edges in order of their increasing weight, but avoids creating a cycle. • The Prim's algorithm maintains a tree at each stage that grows to span. • The Kruskal's algorithm maintains a forest whose trees coalesce into one spanning tree. • The Prim's algorithm implemented with a priority queue is very similar to the Dijkstra's algorithm. • This implementation of the Prim's algorithm runs in time O(m + n log n). • The Kruskal's algorithm uses disjoint sets ADT and can be implemented to run in time O(m log n). 5 / 60 Outline MST Prim Kruskal Optimisation Finding Minimum Spanning Tree Two efficient greedy Prim's and Kruskal's MST algorithms: • Each algorithm selects edges in order of their increasing weight, but avoids creating a cycle. • The Prim's algorithm maintains a tree at each stage that grows to span. • The Kruskal's algorithm maintains a forest whose trees coalesce into one spanning tree. • The Prim's algorithm implemented with a priority queue is very similar to the Dijkstra's algorithm. • This implementation of the Prim's algorithm runs in time O(m + n log n). • The Kruskal's algorithm uses disjoint sets ADT and can be implemented to run in time O(m log n). 5 / 60 Outline MST Prim Kruskal Optimisation Prim's MST Algorithm algorithm Prim( weighted graph (G; c), vertex s ) array w[n] = fc[s; 0]; c[s; 1]; : : : ; c[s; n − 1]g S fsg first vertex added to MST while S =6 V (G) do find u 2 V (G) n S so that w[u] is minimum S S [ fug adding an edge adjacent to u to MST for x 2 V (G) n S do w[x] minfw[x]; c[u; x]g end for end while Very similar to the Dijkstra's algorithm: • Priority queue should be used for selecting the lowest edge weights w[:::]. • In the priority queue implementation, most time is taken by EXTRACT-MIN and DECREASE-KEY operations. 6 / 60 Outline MST Prim Kruskal Optimisation Illustrating the Prim's Algorithm 1 13 a b c 10 8 5 4 7 2 d 13 g 5 8 f h 1 6 15 2 9 12 i j a b c d f g h i j S = a w = 0 1 1 8 1 1 2 1 1 7 / 60 Outline MST Prim Kruskal Optimisation Illustrating the Prim's Algorithm 1 13 a b c 10 8 5 4 7 2 d 13 g 5 8 f h 1 6 15 2 9 12 i j a b c d f g h i j S = a,b w = 0 1a 13b 8a 5b 1 2a 1 1 7 / 60 Outline MST Prim Kruskal Optimisation Illustrating the Prim's Algorithm 1 13 a b c 10 8 5 4 7 2 d 13 g 5 8 f h 1 6 15 2 9 12 i j a b c d f g h i j S = a,b,h w = 0 1a 13b 5h 5b 1 2a 9h 1 7 / 60 Outline MST Prim Kruskal Optimisation Illustrating the Prim's Algorithm 1 13 a b c 10 8 5 4 7 2 d 13 g 5 8 f h 1 6 15 2 9 12 i j a b c d f g h i j S = a,b,d,h w = 0 1a 13b 5h 5b 1 2a 1d 1 7 / 60 Outline MST Prim Kruskal Optimisation Illustrating the Prim's Algorithm 1 13 a b c 10 8 5 4 7 2 d 13 g 5 8 f h 1 6 15 2 9 12 i j a b c d f g h i j S = a,b,d,h,i w = 0 1a 13b 5h 5b 1 2a 1d 12i 7 / 60 Outline MST Prim Kruskal Optimisation Illustrating the Prim's Algorithm 1 13 a b c 10 8 5 4 7 2 d 13 g 5 8 f h 1 6 15 2 9 12 i j a b c d f g h i j S = a,b,d,f,h,i w = 0 1a 7f 5h 5b 13f 2a 1d 12i 7 / 60 Outline MST Prim Kruskal Optimisation Illustrating the Prim's Algorithm 1 13 a b c 10 8 5 4 7 2 d 13 g 5 8 f h 1 6 15 2 9 12 i j a b c d f g h i j S = a,b,c,d,f,h,i w = 0 1a 7f 5h 5b 4c 2a 1d 12i 7 / 60 Outline MST Prim Kruskal Optimisation Illustrating the Prim's Algorithm 1 13 a b c 10 8 5 4 7 2 d 13 g 5 8 f h 1 6 15 2 9 12 i j a b c d f g h i j S = a,b,c,d,f,g,h,i w = 0 1a 7f 5h 5b 4c 2a 1d 2g 7 / 60 Outline MST Prim Kruskal Optimisation Illustrating the Prim's Algorithm 1 13 a b c 10 8 5 4 7 2 d 13 g 5 8 f h 1 6 15 2 9 12 i j a b c d f g h i j S = a,b,c,d,f,g,h,i,j w = 0 1a 7f 5h 5b 4c 2a 1d 2g 7 / 60 Outline MST Prim Kruskal Optimisation Prim's Algorithm (Priority Queue Implementation) algorithm Prim (weighted graph (G; c), vertex s 2 V (G) ) priority queue Q, arrays colour[n], pred[n] for u 2 V (G) do pred[u] NULL; colour[u] WHITE end for colour[s] GREY; Q:insert(s; 0) while not Q:isEmpty() do u Q:peek() for each x adjacent to u do t c(u; x); if colour[x] = WHITE then colour[x] GREY; pred[x] u; Q:insert(x; t) else if colour[x] = GREY and Q:getKey(x) > t then Q:decreaseKey(x; t); pred[x] u end if end for Q:delete(); colour[u] BLACK end while return pred 8 / 60 Outline MST Prim Kruskal Optimisation MST: Prim's Algorithm { Starting Vertex a a d 3 7 3 5 2 c f g 6 9 8 4 1 b e 3 Initialisation: Priority queue Q: fakey=0g v 2 V a b c d e f g pred[v] − − − − − − − keyv 0 9 / 60 Outline MST Prim Kruskal Optimisation MST: Prim's Algorithm: 1 { 2 u a = Q:peek(); for-loop a d 3 7 3 5 2 c f g 6 9 8 4 1 b e 3 u = a; adjacent x 2 fb; c; dg; x b; keyb cost(a; b) = 2 Priority queue Q: fa0; b2g v 2 V a b c d e f g pred[v] − a − − − − − keyv 0 2 10 / 60 Outline MST Prim Kruskal Optimisation MST: Prim's Algorithm: 3 u = a; for-loop a d 3 7 3 5 2 c f g 6 9 8 4 1 b e 3 adjacent x 2 fb; c; dg; x c; keyc cost(a; c) = 3 Priority queue Q: fa0; b2; c3g v 2 V a b c d e f g pred[v] − a a − − − − keyv 0 2 3 11 / 60 Outline MST Prim Kruskal Optimisation MST: Prim's Algorithm: 4 u = a; for-loop a d 3 7 3 5 2 c f g 6 9 8 4 1 b e 3 adjacent x 2 fb; c; dg; x d; keyd cost(a; d) = 3 Priority queue Q: fa0; b2; c3; d3g v 2 V a b c d e f g pred[v] − a a a − − − keyv 0 2 3 3 12 / 60 Outline MST Prim Kruskal Optimisation MST: Prim's Algorithm: 5 { 6 Q:delete() a d 3 7 3 5 2 c f g 6 9 8 4 1 b e 3 Q:delete() { excluding the vertex a Priority queue Q: fb2; c3; d3g v 2 V a b c d e f g pred[v] − a a a − − − keyv 0 2 3 3 13 / 60 Outline MST Prim Kruskal Optimisation MST: Prim's Algorithm: 7 u b = Q:peek(); for-loop a d 3 7 3 5 2 c f g 6 9 8 4 1 b e 3 adjacent x 2 fc; eg; x c; keyc = 3 < cost(b; c) = 4 Priority queue Q: fb2; c3; d3g v 2 V a b c d e f g pred[v] − a a a − − − keyv 0 2 3 3 14 / 60 Outline MST Prim Kruskal Optimisation MST: Prim's Algorithm: 8 u = b; for-loop a d 3 7 3 5 2 c f g 6 9 8 4 1 b e 3 adjacent x 2 fc; eg; x e; keye cost(b; e) = 3 Priority queue Q: fb2; c3; d3; e3g v 2 V a b c d e f g pred[v] − a a a b − − keyv 0 2 3 3 3 15 / 60 Outline MST Prim Kruskal Optimisation MST: Prim's Algorithm: 9 { 10 Q:delete() a d 3 7 3 5 2 c f g 6 9 8 4 1 b e 3 Q:delete() { excluding the vertex b Priority queue Q: fc3; d3; e3g v 2 V a b c d e f g pred[v] − a a a b − − keyv 0 2 3 3 3 16 / 60 Outline MST Prim Kruskal Optimisation MST: Prim's Algorithm: 11 u c = Q:peek(); for-loop a d 3 7 3 5 2 c f g 6 9 8 4 1 b e 3 adjacent x 2 fd; e; fg; x d; keyd = 3 < cost(c; d) = 5 Priority queue Q: fc3; d3; e3g v 2 V a b c d e f g pred[v] − a a a b − − keyv 0 2 3 3 3 17 / 60 Outline MST Prim Kruskal Optimisation MST: Prim's Algorithm: 12 u = c; for-loop a d 3 7 3 5 2 c f g 6 9 8 4 1 b e 3 adjacent x 2 fd; e; fg; x e; keye = 3 > cost(c; e) = 1; keye 1 Priority queue Q: fc3; d3; e1g v 2 V a b c d e f g pred[v] − a a a c − − keyv 0 2 3 3 1 18 / 60 Outline MST Prim Kruskal Optimisation MST: Prim's Algorithm: 13 u = c; for-loop a d 3 7 3 5 2 c f g 6 9 8 4 1 b e 3 adjacent x 2 fd; e; fg; x f; keyf 6 Priority queue Q: fc3; d3; e1; f6g v 2 V a b c d e f g pred[v] − a a a c c − keyv 0 2 3 3 1 6 19 / 60 Outline MST Prim Kruskal Optimisation MST: Prim's Algorithm: 14 { 15 Q:delete() a d 3 7 3 5 2 c f g 6 9 8 4 1 b e 3 Q:delete() { excluding the vertex c Priority queue Q: fe1; d3; f6g v 2 V a b c d e f g pred[v] − a a a c c − keyv 0 2 3 3 1 6 20 / 60 Outline MST Prim Kruskal Optimisation MST: Prim's Algorithm: 16 u e = Q:peek(); for-loop a d 3 7 3 5 2 c f g 6 9 8 4 1 b e 3 adjacent x 2 ffg; x f; keyf 6 < cost(e; f) = 8 Priority queue Q: fe1; d3; f6g v 2 V a b c d e f g pred[v] − a a a c c − keyv 0 2 3 3 1 6 21 / 60 Outline MST Prim Kruskal Optimisation MST: Prim's Algorithm: 17 { 18 Q:delete() a d 3 7 3 5 2 c f g 6 9 8 4 1 b e 3 Q:delete() { excluding the vertex e Priority queue Q: fd3; f6g v 2 V a b c d e f g pred[v] − a a a c c − keyv 0