Minimum Spanning Trees Algorithmique Fall semester 2011/12 Acknowledgment: Slides modeled after the course CO226 at Princeton Given: Undirected+connected graph G with positive edge weights Definition: A spanning tree of G is a connected acyclic subgraph T of G with the same set of vertices as G. Goal: find a minimum weight spanning tree of G. (Minimum spanning tree, or MST) 12 4 10 23 14 7 7 19 11 4 2 9 5 Graph G Given: Undirected+connected graph G with positive edge weights Definition: A spanning tree of G is a connected acyclic subgraph T of G with the same set of vertices as G. Goal: find a minimum weight spanning tree of G. (Minimum spanning tree, or MST) 12 4 10 23 14 7 7 19 11 4 2 9 5 Acyclic, but not spanning Given: Undirected+connected graph G with positive edge weights Definition: A spanning tree of G is a connected acyclic subgraph T of G with the same set of vertices as G. Goal: find a minimum weight spanning tree of G. (Minimum spanning tree, or MST) 12 4 10 23 14 7 7 19 11 4 2 9 5 Spanning, acyclic, but not connected Given: Undirected+connected graph G with positive edge weights Definition: A spanning tree of G is a connected acyclic subgraph T of G with the same set of vertices as G. Goal: find a minimum weight spanning tree of G. (Minimum spanning tree, or MST) 12 4 10 23 14 7 7 19 11 4 2 9 5 Spanning, connected, but not acyclic Given: Undirected+connected graph G with positive edge weights Definition: A spanning tree of G is a connected acyclic subgraph T of G with the same set of vertices as G. Goal: find a minimum weight spanning tree of G. (Minimum spanning tree, or MST) 12 4 10 23 14 7 7 19 11 4 2 9 5 Spanning tree of cost 4+7+11+10+4+7+5 = 48 Examples Example 1: Communication Networks A multinational company wants to lease communication lines between its various locations. Example 1: Communication Networks A multinational company wants to lease communication lines between its various locations. Example 1: Communication Networks A multinational company wants to lease communication lines between its various locations. Example 1: Communication Networks 120 400 100 19 90 10 20 10 12 122 10 10 19 5 19 10 10 18 200 20 10 90 80 100 20 50 18 20 540 120 90 90 20 21 200 200 30 43 10 20 25 152 Each communication line comes with its own price tag. Company wants to spend the least amount of money, and have all its branches connected. Solution given by a MST on the graph. (Why?) Example 2: Clustering Edge values equal to distances of nodes Find “clusters” of nodes. Example 2: Clustering Possible solution: Find MST. Eliminate “fat” edges. Example 2: Clustering Possible solution: Find MST. Eliminate “fat” edges. Note: this is a “heuristic” algorithm. Needs analysis. Example 3: Dendritic Structures in the Brain Problem: Reconstruct shape of neurons from noisy microscopy data via automatic tools http://cvlab.epfl.ch/research/medical/neurons/ Example 3: Dendritic Structures in the Brain Dendrite tracking in microscopic images using minimum spanning trees and localized EM -- by Fleuret and Fua Example 4: Phylogenetic Trees Genetic variability and population structure of endangered Panax ginseng in the Russian Primorye -- by Zhuravlev et al Algorithms Cuts Cut: A cut (A,B) in a graph G=(V,E) is a partition of V into two nonempty sets A and B. Crossing edge: Any edge connecting a vertex in A to a vertex in B. Crossing edges A B Cut Property Cut C=(A,B), tree T on A which is part of MST, e crossing edge of minimum weight. Then there is MST M containing e and T. Crossing edge of minimum weight A MST Cut Property Cut C=(A,B), tree T on A which is part of MST, e crossing edge of minimum weight. Then there is MST M containing e and T. Proof: • Take MST containing T • Add crossing edge e of min weight to MST e • This creates a cycle e • Cycle has one other crossing edge f • Weight of f is at least equal to that of e f e Replace f by e in the MST • f • This gives new MST which contains e • This means that the weights of e and f have been equal Prim’s Algorithm http://inserv.math.muni.cz/biografie/vojtech_jarnik.html http://www.ithistory.org/honor_roll/fame-detail.php?recordID=882 http://en.wikipedia.org/wiki/Edsger_W._Dijkstra Voijtech Jarnik Robert Prim Edsger Dijkstra 1897-1970 1921 - 1930-2002 Prim’s Algorithm Start with any vertex v, set tree T to singleton v. Greedily grow tree T: at each step add to T a minimum weight edge with exactly one endpoint in T. Edges connected to exactly one tree node Tree edges Tree nodes v etc Prim’s Algorithm Why does it work? T is always a subtree of a MST Induction on number of nodes in T. Final T is MST by this result. Start: trivial Singleton v is part of a MST v Step: use cut property Crossing edge of minimum weight In MST by hypothesis In MST by cut property Prim’s Algorithm 10 2 3 5 4 8 12 10 2 23 19 18 22 12 17 Prim’s Algorithm 10 2 3 5 4 8 12 10 2 23 19 18 22 12 17 Prim’s Algorithm 10 2 3 5 4 8 12 10 2 23 19 18 22 12 17 Prim’s Algorithm 10 2 3 5 4 8 12 10 2 23 16 19 18 22 12 17 Prim’s Algorithm 10 2 3 5 4 8 12 10 2 23 16 19 18 22 12 17 Prim’s Algorithm 10 2 3 5 4 8 12 10 2 23 16 19 18 22 12 17 Prim’s Algorithm 10 2 3 5 4 8 12 10 2 23 16 19 18 22 12 17 Prim’s Algorithm 10 2 3 5 4 8 12 10 2 23 16 19 18 22 12 17 Prim’s Algorithm 10 2 3 5 4 8 12 10 2 23 16 19 18 22 12 17 Prim’s Algorithm 10 2 3 5 4 8 12 10 2 23 16 19 18 22 12 17 Prim’s Algorithm 10 2 3 5 4 8 12 10 2 23 16 19 18 22 12 17 Implementation Challenge How do we find minimum crossing edge at every iteration? Check all the outgoing edges: O(|E|) comparisons at every iteration O(|E| |V|) running time in total More clever data structure: • For every node w, keep value dist(w) that measures the “distance” of w from current tree • At the start, dist(v) = 0, and dist(w) = infinity for all other w • When a new node u is added to tree, check whether the neighbors of u decrease their distance to tree; if so, decrease distance. Maintain a min-priority queue for the nodes and their distances. Implementation (1) dist(v) = 0, dist(w) = for w = v, pred(v) = NULL ∞ (2) Create min-priority queue Q for V with respect to dist (3) While Q is not empty do Q contains all nodes that are not yet covered by the MST (a) u = deleteMin(Q) u is node with smallest distance to the current tree (b) if ( u is not marked) then (i) Mark u u is now covered (ii) For all neighbors w of u do a. if ( dist(w) > weight of edge (u,w) and w not marked) then i. dist(w) = weight of edge (u,w) Update distance of neighbors of u ii. pred(w) = u Predecessor of w in the tree iii. Sift up w in Q Multiple copies of w may be present in Q, but only one gets marked (4) Output tree {(pred(v),v) | v in V} Analysis (1) dist(v) = 0, dist(w) = for w = v, pred(v) = NULL ∞ (2) Create min-priority queue Q for V with respect to dist O(|V|) (3) While Q is not empty do < |E| times (each vertex gets added to Q at most its degree times) (a) u = deleteMin(Q) O(log(|Q|)) = O(log(|E|)) (b) if ( u is not marked) then (i) Mark u (ii) For all neighbors w of u do a. if ( dist(w) > weight of edge (u,w) and w not marked) then i. dist(w) = weight of edge (u,w) at most |E| times in total ii. pred(w) = u iii. Sift up w in Q O(log(|Q|)) = O(log(|E|)) (4) Output tree {(pred(v),v) | v in V} O(|E| log(|E|)) for connected graphs. Kruskal’s Algorithm http://www.voteview.com/ideal_point_Non_Metric_MDS.htm Joseph B. Kruskal 1928-2010 Kruskal’s Algorithm Maintains forest which will become a MST at the end. (1)Start from empty tree T (2)Consider edges in ascending order of cost. Add next edge in list to T if it doesn’t create a cycle. Several components 2 2 2 2 0 6 0 6 0 6 0 6 7 7 7 7 u v weight 1 1 1 1 3 5 0.18 1 7 0.21 3 3 3 3 6 7 0.25 5 4 5 4 5 4 5 4 0 2 0.29 3-5 1-7 6-7 0-2 0 7 0.31 0 1 0.32 2 2 2 3 4 0.34 0 6 0 6 0 6 4 5 0.40 4 7 0.46 7 7 7 0 6 0.51 1 1 1 4 6 0.51 0 5 0.6 3 3 3 5 4 5 4 5 4 0-7 3-4 4-7 Kruskal’s Algorithm Why does it work? T is always a sub-forest of a MST Induction on number of nodes in T.