Submodular Functions, Optimization, and Applications to Machine Learning | Spring Quarter, Lecture 8 | http://www.ee.washington.edu/people/faculty/bilmes/classes/ee596b_spring_2016/ Prof. Jeff Bilmes University of Washington, Seattle Department of Electrical Engineering http://melodi.ee.washington.edu/~bilmes Apr 25th, 2016 f (A) + f (B) f (A B) + f (A B) Clockwise from top left:v Lásló Lovász Jack Edmonds ∪ ∩ Satoru Fujishige George Nemhauser ≥ Laurence Wolsey = f (A ) + 2f (C) + f (B ) = f (A ) +f (C) + f (B ) = f (A B) András Frank r r r r Lloyd Shapley ∩ H. Narayanan Robert Bixby William Cunningham William Tutte Richard Rado Alexander Schrijver Garrett Birkho Hassler Whitney Richard Dedekind Prof. Jeff Bilmes EE596b/Spring 2016/Submodularity - Lecture 8 - Apr 25th, 2016 F1/40 (pg.1/162) Logistics Review Cumulative Outstanding Reading Read chapters 2 and 3 from Fujishige's book. Read chapter 1 from Fujishige's book. Prof. Jeff Bilmes EE596b/Spring 2016/Submodularity - Lecture 8 - Apr 25th, 2016 F2/40 (pg.2/162) Logistics Review Announcements, Assignments, and Reminders Homework 3, available at our assignment dropbox (https://canvas.uw.edu/courses/1039754/assignments), due (electronically) Monday (5/2) at 11:55pm. Homework 2, available at our assignment dropbox (https://canvas.uw.edu/courses/1039754/assignments), due (electronically) Monday (4/18) at 11:55pm. Homework 1, available at our assignment dropbox (https://canvas.uw.edu/courses/1039754/assignments), due (electronically) Friday (4/8) at 11:55pm. Weekly Office Hours: Mondays, 3:30-4:30, or by skype or google hangout (set up meeting via our our discussion board (https: //canvas.uw.edu/courses/1039754/discussion_topics)). Prof. Jeff Bilmes EE596b/Spring 2016/Submodularity - Lecture 8 - Apr 25th, 2016 F3/40 (pg.3/162) Logistics Review Class Road Map - IT-I L1(3/28): Motivation, Applications, & L11(5/2): Basic Definitions L12(5/4): L2(3/30): Machine Learning Apps L13(5/9): (diversity, complexity, parameter, learning L14(5/11): target, surrogate). L15(5/16): L3(4/4): Info theory exs, more apps, L16(5/18): definitions, graph/combinatorial examples, matrix rank example, visualization L17(5/23): L4(4/6): Graph and Combinatorial L18(5/25): Examples, matrix rank, Venn diagrams, L19(6/1): examples of proofs of submodularity, some L20(6/6): Final Presentations useful properties maximization. L5(4/11): Examples & Properties, Other Defs., Independence L6(4/13): Independence, Matroids, Matroid Examples, matroid rank is submodular L7(4/18): Matroid Rank, More on Partition Matroid, System of Distinct Reps, Transversals, Transversal Matroid, L8(4/20): Transversals, Matroid and representation, Dual Matroids, Geometries L9(4/25): L10(4/27): Finals Week: June 6th-10th, 2016. Prof. Jeff Bilmes EE596b/Spring 2016/Submodularity - Lecture 8 - Apr 25th, 2016 F4/40 (pg.4/162) Logistics Review System of Distinct Representatives Let (V; ) be a set system (i.e., = (Vk : i I) where Vi V for all V V 2 ⊆ i), and I is an index set. Hence, I = . j j jVj A family (vi : i I) with vi V is said to be a system of distinct 2 2 representatives of if a bijection π : I I such that vi V and V 9 $ 2 π(i) vi = vj for all i = j. 6 6 In a system of distinct representatives, there is a requirement for the representatives to be distinct. We can re-state (and rename) this as a: Definition 8.2.2 (transversal) Given a set system (V; ) and index set I for as defined above, a set V V T V is a transversal of if there is a bijection π : T I such that ⊆ V $ x V for all x T (8.19) 2 π(x) 2 Note that due to π : T I being a bijection, all of I and T are $ \covered" (so this makes things distinct automatically). Prof. Jeff Bilmes EE596b/Spring 2016/Submodularity - Lecture 8 - Apr 25th, 2016 F5/40 (pg.5/162) Logistics Review When do transversals exist? As we saw, a transversal might not always exist. How to tell? Given a set system (V; ) with = (Vi : i I), and Vi V for all i. V V 2 ⊆ Then, for any J I, let ⊆ V (J) = j2J Vj (8.19) [ I so V (J) : 2 Z+ is the set cover func. (we know is submodular). Wej have j ! Theorem 8.2.2 (Hall's theorem) Given a set system (V; ), the family of subsets = (Vi : i I) has a V V 2 transversal (vi : i I) iff for all J I 2 ⊆ V (J) J (8.20) j j ≥ j j Prof. Jeff Bilmes EE596b/Spring 2016/Submodularity - Lecture 8 - Apr 25th, 2016 F6/40 (pg.6/162) Logistics Review When do transversals exist? As we saw, a transversal might not always exist. How to tell? Given a set system (V; ) with = (Vi : i I), and Vi V for all i. V V 2 ⊆ Then, for any J I, let ⊆ V (J) = j2J Vj (8.19) [ I so V (J) : 2 Z+ is the set cover func. (we know is submodular). Hall'sj theoremj ! ( J I; V (J) J ) as a bipartite graph. 8 ⊆ j j ≥ j j V I 4 3 2 1 Prof. Jeff Bilmes EE596b/Spring 2016/Submodularity - Lecture 8 - Apr 25th, 2016 F6/40 (pg.7/162) Logistics Review When do transversals exist? As we saw, a transversal might not always exist. How to tell? Given a set system (V; ) with = (Vi : i I), and Vi V for all i. V V 2 ⊆ Then, for any J I, let ⊆ V (J) = j2J Vj (8.19) [ I so V (J) : 2 Z+ is the set cover func. (we know is submodular). Hall'sj theoremj ! ( J I; V (J) J ) as a bipartite graph. 8 ⊆ j j ≥ j j V I 4 3 2 1 Prof. Jeff Bilmes EE596b/Spring 2016/Submodularity - Lecture 8 - Apr 25th, 2016 F6/40 (pg.8/162) Logistics Review When do transversals exist? As we saw, a transversal might not always exist. How to tell? Given a set system (V; ) with = (Vi : i I), and Vi V for all i. V V 2 ⊆ Then, for any J I, let ⊆ V (J) = j2J Vj (8.19) [ I so V (J) : 2 Z+ is the set cover func. (we know is submodular). Moreover,j j we have! Theorem 8.2.3 (Rado's theorem (1942)) If M = (V; r) is a matroid on V with rank function r, then the family of subsets (Vi : i I) of V has a transversal (vi : i I) that is independent in 2 2 M iff for all J I ⊆ r(V (J)) J (8.21) ≥ j j Note, a transversal T independent in M means that r(T ) = T . j j Prof. Jeff Bilmes EE596b/Spring 2016/Submodularity - Lecture 8 - Apr 25th, 2016 F6/40 (pg.9/162) We wish all jobs to be filled, and hence Hall's condition ( J I; V (J) J ) is a necessary and sufficient condition for this 8 ⊆ j j ≥ j j to be possible. Note if V = I , then Hall's theorem is the Marriage Theorem j j j j (Frobenious 1917), where an edge (v; i) in the graph indicate compatibility between two individuals v V and i I coming from 2 2 two separate groups V and I. If J I; V (J) J , then all individuals in each group can be 8 ⊆ j j ≥ j j matched with a compatible mate. Logistics Review Application's of Hall's theorem Consider a set of jobs I and a set of applicants V to the jobs. If an applicant v V is qualified for job i I, we add edge (v; i) to the 2 2 bipartite graph G = (V; I; E). Prof. Jeff Bilmes EE596b/Spring 2016/Submodularity - Lecture 8 - Apr 25th, 2016 F7/40 (pg.10/162) Note if V = I , then Hall's theorem is the Marriage Theorem j j j j (Frobenious 1917), where an edge (v; i) in the graph indicate compatibility between two individuals v V and i I coming from 2 2 two separate groups V and I. If J I; V (J) J , then all individuals in each group can be 8 ⊆ j j ≥ j j matched with a compatible mate. Logistics Review Application's of Hall's theorem Consider a set of jobs I and a set of applicants V to the jobs. If an applicant v V is qualified for job i I, we add edge (v; i) to the 2 2 bipartite graph G = (V; I; E). We wish all jobs to be filled, and hence Hall's condition ( J I; V (J) J ) is a necessary and sufficient condition for this 8 ⊆ j j ≥ j j to be possible. Prof. Jeff Bilmes EE596b/Spring 2016/Submodularity - Lecture 8 - Apr 25th, 2016 F7/40 (pg.11/162) If J I; V (J) J , then all individuals in each group can be 8 ⊆ j j ≥ j j matched with a compatible mate. Logistics Review Application's of Hall's theorem Consider a set of jobs I and a set of applicants V to the jobs. If an applicant v V is qualified for job i I, we add edge (v; i) to the 2 2 bipartite graph G = (V; I; E). We wish all jobs to be filled, and hence Hall's condition ( J I; V (J) J ) is a necessary and sufficient condition for this 8 ⊆ j j ≥ j j to be possible. Note if V = I , then Hall's theorem is the Marriage Theorem j j j j (Frobenious 1917), where an edge (v; i) in the graph indicate compatibility between two individuals v V and i I coming from 2 2 two separate groups V and I.