Lecture 15 Minimum Spanning Trees Announcements • HW5 due Friday • HW6 released Friday Last time • Greedy algorithms • Make a series of choices. • Choose this activity, then that one, .. • Never backtrack. • Show that, at each step, your choice does not rule out success. • At every step, there exists an optimal solution consistent with the choices we’ve made so far. • At the end of the day: • you’ve built only one solution, • never having ruled out success, • so your solution must be correct. Today • Greedy algorithms for Minimum Spanning Tree. • Agenda: 1. What is a Minimum Spanning Tree? 2. Short break to introduce some graph theory tools 3. Prim’s algorithm 4. Kruskal’s algorithm Minimum Spanning Tree Say we have an undirected weighted graph 8 7 B C D 4 9 2 11 4 A I 14 E 7 6 8 10 1 2 A tree is a H G F connected graph with no cycles! A spanning tree is a tree that connects all of the vertices. Minimum Spanning Tree Say we have an undirected weighted graph The cost of a This is a spanning tree is 8 7 spanning tree. the sum of the B C D weights on the edges. 4 9 2 11 4 A I 14 E 7 6 8 10 A tree is a This tree 1 2 H G F connected graph has cost 67 with no cycles! A spanning tree is a tree that connects all of the vertices. Minimum Spanning Tree Say we have an undirected weighted graph This is also a 8 7 spanning tree. B C D It has cost 37 4 9 2 11 4 A I 14 E 7 6 8 10 1 2 A tree is a H G F connected graph with no cycles! A spanning tree is a tree that connects all of the vertices. Minimum Spanning Tree Say we have an undirected weighted graph 8 7 B C D 4 9 2 11 4 A I 14 E 7 6 8 10 1 2 H G F minimum of minimal cost A spanning tree is a tree that connects all of the vertices. Minimum Spanning Tree Say we have an undirected weighted graph This is a minimum 8 7 spanning tree. B C D It has cost 37 4 9 2 11 4 A I 14 E 7 6 8 10 1 2 H G F minimum of minimal cost A spanning tree is a tree that connects all of the vertices. Why MSTs? • Network design • Connecting cities with roads/electricity/telephone/… • cluster analysis • eg, genetic distance • image processing • eg, image segmentation • Useful primitive • for other graph algs Figure 2: Fully parsimonious minimal spanning tree of 933 SNPs for 282 isolates of Y. pestis colored by location. Morelli et al. Nature genetics 2010 How to find an MST? • Today we’ll see two greedy algorithms. • In order to prove that these greedy algorithms work, we’ll need to show something like: Suppose that our choices so far haven’t ruled out success. Then the next greedy choice that we make also won’t rule out success. • Here, success means finding an MST. Let’s brainstorm • How would we design a greedy algorithm? 8 7 B C D 4 9 2 11 4 A I 14 E 7 6 8 10 1 2 H G F Brief aside for a discussion of cuts in graphs! Cuts in graphs • A cut is a partition of the vertices into two parts: 8 7 B C D 4 9 2 11 4 A I 14 E 7 6 8 10 1 2 H G F This is the cut “{A,B,D,E} and {C,I,H,G,F}” Let A be a set of edges in G • We say a cut respects A if no edges in A cross the cut. • An edge crossing a cut is called light if it has the smallest weight of any edge crossing the cut. 8 B C D 7 4 9 2 11 4 A I 14 E 7 6 8 10 1 2 H G F A is the thick orange edges Let A be a set of edges in G • We say a cut respects A if no edges in A cross the cut. • An edge crossing a cut is called light if it has the smallest weight of any edge crossing the cut. This edge is light 8 B C D 7 4 9 2 11 4 A I 14 E 7 6 8 10 1 2 H G F A is the thick orange edges Lemma • Let A be a set of edges, and consider a cut that respects A. • Suppose there is an MST containing A. • Let (u,v) be a light edge. • Then there is an MST containing A ∪ {(u,v)} This edge is light B 8 C D 7 9 4 2 11 4 A I 14 E 6 8 7 10 H 1 G 2 F A is the thick orange edges Lemma • Let A be a set of edges, and consider a cut that respects A. • Suppose there is an MST containing A. • Let (u,v) be a light edge. • Then there is an MST containing A ∪ {(u,v)} We can safely add This is precisely the this edge to the tree sort of statement we need for a greedy B 8 C D algorithm: 7 9 4 2 If we haven’t ruled 11 4 out the possibility of A I 14 E success so far, then adding a light edge 7 6 still won’t rule it out. 8 10 H 1 G 2 F A is the thick orange edges Proof of Lemma • Assume that we have: • a cut that respects A x y u a v b Proof of Lemma • Assume that we have: • a cut that respects A • A is part of some MST T. • Say that (u,v) is light. • lowest cost crossing the cut x y u a v b Claim: Adding any additional edge to a spanning tree will create a cycle. Proof: Both endpoints are already in Proof of Lemma the tree and connected to each other. • Assume that we have: • a cut that respects A • A is part of some MST T. • Say that (u,v) is light. • lowest cost crossing the cut x y • But (u,v) is not in T. • So adding (u,v) to T will make a cycle. u a v b Claim: Adding any additional edge to a spanning tree will create a cycle. Proof: Both endpoints are already in Proof of Lemma the tree and connected to each other. • Assume that we have: • a cut that respects A • A is part of some MST T. • Say that (u,v) is light. • lowest cost crossing the cut x y • But (u,v) is not in T. • So adding (u,v) to T will make a cycle. u a • So there is at least one other edge in this cycle crossing the cut. v b • call it (x,y) Proof of Lemma ctd. • Consider swapping (u,v) for (x,y) in T. • Call the resulting tree T’. x y u a v b Proof of Lemma ctd. • Consider swapping (u,v) for (x,y) in T. • Call the resulting tree T’. • Claim: T’ is still an MST. • It is still a tree: x y • we deleted (x,y) • It has cost at most that of T • because (u,v) was light. • T had minimal cost. • So T’ does too. u a • So T’ is an MST containing (u,v). • This is what we wanted. v b Lemma • Let A be a set of edges, and consider a cut that respects A. • Suppose there is an MST containing A. • Let (u,v) be a light edge. • Then there is an MST containing A ∪ {(u,v)} This edge is light B 8 C D 7 9 4 2 11 4 A I 14 E 6 8 7 10 H 1 G 2 F A is the thick orange edges End aside Back to MSTs! Back to MSTs • How do we find one? • Today we’ll see two greedy algorithms. • The strategy: • Make a series of choices, adding edges to the tree. • Show that each edge we add is safe to add: • we do not rule out the possibility of success • we will choose light edges crossing cuts and use the Lemma. • Keep going until we have an MST. Idea 1 Start growing a tree, greedily add the shortest edge we can to grow the tree. 8 7 B C D 4 9 2 11 4 A I 14 E 7 6 8 10 1 2 H G F Idea 1 Start growing a tree, greedily add the shortest edge we can to grow the tree. 8 7 B C D 4 9 2 11 4 A I 14 E 7 6 8 10 1 2 H G F Idea 1 Start growing a tree, greedily add the shortest edge we can to grow the tree. 8 7 B C D 4 9 2 11 4 A I 14 E 7 6 8 10 1 2 H G F Idea 1 Start growing a tree, greedily add the shortest edge we can to grow the tree. 8 7 B C D 4 9 2 11 4 A I 14 E 7 6 8 10 1 2 H G F Idea 1 Start growing a tree, greedily add the shortest edge we can to grow the tree. 8 7 B C D 4 9 2 11 4 A I 14 E 7 6 8 10 1 2 H G F Idea 1 Start growing a tree, greedily add the shortest edge we can to grow the tree.