Submodular Functions, Optimization, and Applications to Machine Learning — Spring Quarter, Lecture 6 —

http://www.ee.washington.edu/people/faculty/bilmes/classes/ee563_spring_2018/

Prof. Jeﬀ Bilmes

University of Washington, Seattle Department of Electrical Engineering http://melodi.ee.washington.edu/~bilmes

April 11th, 2018

f (A) + f (B) f (A B) + f (A B) Clockwise from top left:v Lásló Lovász Jack Edmonds ∪ ∩ Satoru Fujishige George Nemhauser ≥ Laurence Wolsey = f (A ) + 2f (C) + f (B ) = f (A ) +f (C) + f (B ) = f (A B) András Frank r r r r Lloyd Shapley ∩ H. Narayanan Robert Bixby William Cunningham William Tutte Richard Rado Alexander Schrijver Garrett Birkho Hassler Whitney Richard Dedekind

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F1/46 (pg.1/163) Logistics Review Cumulative Outstanding Reading

Read chapter 1 from Fujishige’s book. Read chapter 2 from Fujishige’s book.

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F2/46 (pg.2/163) Logistics Review Announcements, Assignments, and Reminders

If you have any questions about anything, please ask then via our discussion board (https://canvas.uw.edu/courses/1216339/discussion_topics).

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F3/46 (pg.3/163) Logistics Review Class Road Map - EE563

L1(3/26): Motivation, Applications, & L11(4/30): Basic Definitions, L12(5/2): L2(3/28): Machine Learning Apps L13(5/7): (diversity, complexity, parameter, learning L14(5/9): target, surrogate). L15(5/14): L3(4/2): Info theory exs, more apps, L16(5/16): definitions, graph/combinatorial examples L17(5/21): L4(4/4): Graph and Combinatorial Examples, Matrix Rank, Examples and L18(5/23): Properties, visualizations L–(5/28): Memorial Day (holiday) L5(4/9): More Examples/Properties/ L19(5/30): Other Submodular Defs., Independence, L21(6/4): Final Presentations L6(4/11): Matroids, Matroid Examples, maximization. Matroid Rank, Partition/Laminar Matroids L7(4/16): Laminar Matroids, System of Distinct Reps, Transversals, Transversal Matroid, Matroid Representation, Dual Matroids L8(4/18): L9(4/23): L10(4/25): Last day of instruction, June 1st. Finals Week: June 2-8, 2018. Prof. Jeff Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F4/46 (pg.4/163) Logistics Review Composition of non-decreasing submodular and non-decreasing concave

Theorem 6.2.1 Given two functions, one deﬁned on sets

V f : 2 R (6.1) → and another continuous valued one:

φ : R R (6.2) → V the composition formed as h = φ f : 2 R (deﬁned as ◦ → h(S) = φ(f(S))) is nondecreasing submodular, if φ is non-decreasing concave and f is nondecreasing submodular.

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F5/46 (pg.5/163) Logistics Review Monotone diﬀerence of two functions

Let f and g both be submodular functions on subsets of V and let (f g)( ) be either monotone non-decreasing or monotone non-increasing − · Then h : 2V R deﬁned by → h(A) = min(f(A), g(A)) (6.1) is submodular. Proof. If h(A) agrees with f on both X and Y (or g on both X and Y ), and since h(X) + h(Y ) = f(X) + f(Y ) f(X Y ) + f(X Y ) (6.2) ≥ ∪ ∩ or h(X) + h(Y ) = g(X) + g(Y ) g(X Y ) + g(X Y ), (6.3) ≥ ∪ ∩ the result (Equation ?? being submodular) follows since f(X) + f(Y ) min(f(X Y ), g(X Y )) + min(f(X Y ), g(X Y )) g(X) + g(Y ) ≥ ∪ ∪ ∩ ∩ (6.4) ...

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F6/46 (pg.6/163) Logistics Review Arbitrary functions: diﬀerence between submodular funcs.

Theorem 6.2.1 Given an arbitrary set function h, it can be expressed as a difference V between two submodular functions (i.e., h 2 R, ∀ ∈ → f, g s.t. A, h(A) = f(A) g(A) where both f and g are submodular). ∃ ∀ − Proof. Let h be given and arbitrary, and define: ∆ α = min h(X) + h(Y ) h(X Y ) h(X Y ) (6.4) X,Y :X6⊆Y,Y 6⊆X − ∪ − ∩ If α 0 then h is submodular, so by assumption α < 0. Now let f be an arbitrary≥ strict submodular function and define ∆ β = min f(X) + f(Y ) f(X Y ) f(X Y ) . (6.5) X,Y :X6⊆Y,Y 6⊆X − ∪ − ∩ Strict means that β > 0. ...

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F7/46 (pg.7/163) Logistics Review Many (Equivalent) Deﬁnitions of Submodularity

f(A) + f(B) f(A B) + f(A B), A, B V (6.16) ≥ ∪ ∩ ∀ ⊆ f(j S) f(j T ), S T V, with j V T (6.17) | ≥ | ∀ ⊆ ⊆ ∈ \ f(C S) f(C T ), S T V, with C V T (6.18) | ≥ | ∀ ⊆ ⊆ ⊆ \ f(j S) f(j S k ), S V with j V (S k ) (6.19) | ≥ | ∪ { } ∀ ⊆ ∈ \ ∪ { } f(A B A B) f(A A B) + f(B A B), A, B V (6.20) ∪ | ∩ ≤ X| ∩ X| ∩ ∀ ⊆ f(T ) f(S) + f(j S) f(j S T j ), S, T V ≤ | − | ∪ − { } ∀ ⊆ j∈T \S j∈S\T (6.21) X f(T ) f(S) + f(j S), S T V (6.22) ≤ | ∀ ⊆ ⊆ j∈T \S X X f(T ) f(S) f(j S j ) + f(j S T ) S, T V ≤ − | \{ } | ∩ ∀ ⊆ j∈S\T j∈T \S (6.23) X f(T ) f(S) f(j S j ), T S V (6.24) ≤ − | \{ } ∀ ⊆ ⊆ j∈S\T

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F8/46 (pg.8/163) Logistics Review On Rank

V Let rank : 2 Z+ be the rank function. → In general, rank(A) A , and vectors in A are linearly independent if ≤ | | and only if rank(A) = A . | | If A, B are such that rank(A) = A and rank(B) = B , with | | | | A < B , then the space spanned by B is greater, and we can find a | | | | vector in B that is linearly independent of the space spanned by vectors in A. To stress this point, note that the above condition is A < B , not | | | | A B which is sufficient (to be able to find an independent vector) but⊆ not required. In other words, given A, B with rank(A) = A & rank(B) = B , then | | | | A < B an b B such that rank(A b ) = A + 1. | | | | ⇔ ∃ ∈ ∪ { } | |

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F9/46 (pg.9/163) Logistics Review Spanning trees/forests & incidence matrices

5 A directed version of the graph 2 7 1 12 (right) and its adjacency matrix 3 9 8 8 (below). 1 4 6 11 Orientation can be arbitrary. 4 7 2 10 Note, rank of this matrix is 7. 3 6 5

1 2 3 4 5 6 7 8 9 10 11 12 1  110000000000  − 2 1 0 1 0 1 0 0 0 0 0 0 0   −  3 0 1 0 1 0 1 0 0 0 0 0 0   − −  4 0 0 1 1 0 0 1 1 0 0 0 0   − −  5 0 0 0 0 0 1 1 0 0 1 0 0   −  6 0 0 0 0 0 0 0 1 1 0 1 0   − −  7 0 0 0 0 1 0 0 0 1 0 0 1  8 0 0 0 0− 0 0 0 0 0 1 1 1 − − Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F10/46 (pg.10/163) Hence, is down-closed or “subclusive”, under subsets. In other words, I A B and B A (6.1) ⊆ ∈ I ⇒ ∈ I

Let = I1,I2,... be a set of all subsets of V such that for any I , I { } ∈ I the vectors indexed by I are linearly independent. Given a set B of linearly independent vectors, then any subset A B is also linearly∈ independent. I ⊆

maxInd: Inclusionwise maximal independent subsets (i.e., the set of bases of) of any set B V deﬁned as: ⊆ maxInd(B) A B : A and v B A, A v / (6.2) , { ⊆ ∈ I ∀ ∈ \ ∪ { } ∈ I} Given any set B V of vectors, all maximal (by set inclusion) subsets of ⊂ linearly independent vectors are the same size. That is, for all B V , ⊆ A1,A2 maxInd(B), A1 = A2 = rank(B) (6.3) ∀ ∈ | | | |

Matroids Matroid Examples Matroid Rank More on Partition Matroid From Matrix Rank Matroid → So V is set of column vector indices of a matrix.

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F11/46 (pg.11/163) Hence, is down-closed or “subclusive”, under subsets. In other words, I A B and B A (6.1) ⊆ ∈ I ⇒ ∈ I

Given a set B of linearly independent vectors, then any subset A B is also linearly∈ independent. I ⊆

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F11/46 (pg.12/163) Hence, is down-closed or “subclusive”, under subsets. In other words, I A B and B A (6.1) ⊆ ∈ I ⇒ ∈ I maxInd: Inclusionwise maximal independent subsets (i.e., the set of bases of) of any set B V deﬁned as: ⊆ maxInd(B) A B : A and v B A, A v / (6.2) , { ⊆ ∈ I ∀ ∈ \ ∪ { } ∈ I} Given any set B V of vectors, all maximal (by set inclusion) subsets of ⊂ linearly independent vectors are the same size. That is, for all B V , ⊆ A1,A2 maxInd(B), A1 = A2 = rank(B) (6.3) ∀ ∈ | | | |

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F11/46 (pg.13/163) In other words, A B and B A (6.1) ⊆ ∈ I ⇒ ∈ I maxInd: Inclusionwise maximal independent subsets (i.e., the set of bases of) of any set B V deﬁned as: ⊆ maxInd(B) A B : A and v B A, A v / (6.2) , { ⊆ ∈ I ∀ ∈ \ ∪ { } ∈ I} Given any set B V of vectors, all maximal (by set inclusion) subsets of ⊂ linearly independent vectors are the same size. That is, for all B V , ⊆ A1,A2 maxInd(B), A1 = A2 = rank(B) (6.3) ∀ ∈ | | | |

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F11/46 (pg.14/163) maxInd: Inclusionwise maximal independent subsets (i.e., the set of bases of) of any set B V deﬁned as: ⊆ maxInd(B) A B : A and v B A, A v / (6.2) , { ⊆ ∈ I ∀ ∈ \ ∪ { } ∈ I} Given any set B V of vectors, all maximal (by set inclusion) subsets of ⊂ linearly independent vectors are the same size. That is, for all B V , ⊆ A1,A2 maxInd(B), A1 = A2 = rank(B) (6.3) ∀ ∈ | | | |

Matroids Matroid Examples Matroid Rank More on Partition Matroid From Matrix Rank Matroid → So V is set of column vector indices of a matrix. Let = I1,I2,... be a set of all subsets of V such that for any I , I { } ∈ I the vectors indexed by I are linearly independent. Given a set B of linearly independent vectors, then any subset A B is also linearly∈ independent. I Hence, is down-closed or “subclusive”, ⊆ under subsets. In other words, I A B and B A (6.1) ⊆ ∈ I ⇒ ∈ I

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F11/46 (pg.15/163) Given any set B V of vectors, all maximal (by set inclusion) subsets of ⊂ linearly independent vectors are the same size. That is, for all B V , ⊆ A1,A2 maxInd(B), A1 = A2 = rank(B) (6.3) ∀ ∈ | | | |

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F11/46 (pg.16/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid From Matrix Rank Matroid → So V is set of column vector indices of a matrix. Let = I1,I2,... be a set of all subsets of V such that for any I , I { } ∈ I the vectors indexed by I are linearly independent. Given a set B of linearly independent vectors, then any subset A B is also linearly∈ independent. I Hence, is down-closed or “subclusive”, ⊆ under subsets. In other words, I A B and B A (6.1) ⊆ ∈ I ⇒ ∈ I maxInd: Inclusionwise maximal independent subsets (i.e., the set of bases of) of any set B V deﬁned as: ⊆ maxInd(B) A B : A and v B A, A v / (6.2) , { ⊆ ∈ I ∀ ∈ \ ∪ { } ∈ I} Given any set B V of vectors, all maximal (by set inclusion) subsets of ⊂ linearly independent vectors are the same size. That is, for all B V , ⊆ A1,A2 maxInd(B), A1 = A2 = rank(B) (6.3) ∀ ∈ | | | |

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F11/46 (pg.17/163) Thus, for all I , the matrix rank function has the property ∈ I r(I) = I (6.4) | | and for any B/ , ∈ I r(B) = max A : A B and A < B (6.5) {| | ⊆ ∈ I} | | Since all maximally independent subsets of a set are the same size, the rank function is well deﬁned.

Matroids Matroid Examples Matroid Rank More on Partition Matroid From Matrix Rank Matroid →

Let = I1,I2,... be the set of sets as described above. I { }

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F12/46 (pg.18/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid From Matrix Rank Matroid →

Let = I1,I2,... be the set of sets as described above. I { } Thus, for all I , the matrix rank function has the property ∈ I r(I) = I (6.4) | | and for any B/ , ∈ I r(B) = max A : A B and A < B (6.5) {| | ⊆ ∈ I} | | Since all maximally independent subsets of a set are the same size, the rank function is well deﬁned.

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F12/46 (pg.19/163) In a matroid, there is an underlying ground set, say E (or V ), and a collection of subsets = I1,I2,... of E that correspond to independent elements.I { } There are many deﬁnitions of matroids that are mathematically equivalent, we’ll see some of them here.

Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroid

Matroids abstract the notion of linear independence of a set of vectors to general algebraic properties.

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F13/46 (pg.20/163) There are many deﬁnitions of matroids that are mathematically equivalent, we’ll see some of them here.

Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroid

Matroids abstract the notion of linear independence of a set of vectors to general algebraic properties. In a matroid, there is an underlying ground set, say E (or V ), and a collection of subsets = I1,I2,... of E that correspond to independent elements.I { }

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F13/46 (pg.21/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroid

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F13/46 (pg.22/163) One useful property is “heredity.” Namely, a set system is a hereditary set system if for any A B , we have that A . ⊂ ∈ I ∈ I

Matroids Matroid Examples Matroid Rank More on Partition Matroid Independence System

Deﬁnition 6.3.1 (set system) A (ﬁnite) ground set E and a set of subsets of E, = 2E is called a set ∅ 6 I ⊆ system, notated (E, ). I Set systems can be arbitrarily complex since, as stated, there is no systematic method (besides exponential-cost exhaustive search) to determine if a given set S E has S . ⊆ ∈ I

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F14/46 (pg.23/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Independence System

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F14/46 (pg.24/163) Example: E = 1, 2, 3, 4 . With = , 1 , 1, 2 , 1, 2, 4 . { } I {∅ { } { } { }} Then (E, ) is a set system, but not an independence system since it is I not down closed (e.g., we have 1, 2 but not 2 ). { } ∈ I { } ∈ I With = , 1 , 2 , 1, 2 , then (E, ) is now an independence (hereditary)I {∅ system.{ } { } { }} I

Matroids Matroid Examples Matroid Rank More on Partition Matroid Independence System

Deﬁnition 6.3.2 (independence (or hereditary) system) A set system (V, ) is an independence system if I (emptyset containing) (I1) ∅ ∈ I and

I ,J I J (subclusive) (I2) ∀ ∈ I ⊂ ⇒ ∈ I

Property (I2) called “down monotone,” “down closed,” or “subclusive”

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F15/46 (pg.25/163) Then (E, ) is a set system, but not an independence system since it is I not down closed (e.g., we have 1, 2 but not 2 ). { } ∈ I { } ∈ I With = , 1 , 2 , 1, 2 , then (E, ) is now an independence (hereditary)I {∅ system.{ } { } { }} I

Matroids Matroid Examples Matroid Rank More on Partition Matroid Independence System

Deﬁnition 6.3.2 (independence (or hereditary) system) A set system (V, ) is an independence system if I (emptyset containing) (I1) ∅ ∈ I and

I ,J I J (subclusive) (I2) ∀ ∈ I ⊂ ⇒ ∈ I

Property (I2) called “down monotone,” “down closed,” or “subclusive” Example: E = 1, 2, 3, 4 . With = , 1 , 1, 2 , 1, 2, 4 . { } I {∅ { } { } { }}

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F15/46 (pg.26/163) With = , 1 , 2 , 1, 2 , then (E, ) is now an independence (hereditary)I {∅ system.{ } { } { }} I

Matroids Matroid Examples Matroid Rank More on Partition Matroid Independence System

Deﬁnition 6.3.2 (independence (or hereditary) system) A set system (V, ) is an independence system if I (emptyset containing) (I1) ∅ ∈ I and

I ,J I J (subclusive) (I2) ∀ ∈ I ⊂ ⇒ ∈ I

Property (I2) called “down monotone,” “down closed,” or “subclusive” Example: E = 1, 2, 3, 4 . With = , 1 , 1, 2 , 1, 2, 4 . { } I {∅ { } { } { }} Then (E, ) is a set system, but not an independence system since it is I not down closed (e.g., we have 1, 2 but not 2 ). { } ∈ I { } ∈ I

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F15/46 (pg.27/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Independence System

Deﬁnition 6.3.2 (independence (or hereditary) system) A set system (V, ) is an independence system if I (emptyset containing) (I1) ∅ ∈ I and

I ,J I J (subclusive) (I2) ∀ ∈ I ⊂ ⇒ ∈ I

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F15/46 (pg.28/163) Given any forest Gf that is an edge-induced sub-graph of a graph G, any sub-graph of Gf is also a forest. So these both constitute independence systems.

Matroids Matroid Examples Matroid Rank More on Partition Matroid Independence System

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8     1 0 0 1 1 2 1 3 1 | | | | | | | |     (6.6) 2 0 1 1 0 2 0 2 4  =  x1 x2 x3 x4 x5 x6 x7 x8  3 1 1 1 0 0 3 1 5 | | | | | | | |

Given any set of linearly independent vectors A, any subset B A will also be linearly independent. ⊂

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F16/46 (pg.29/163) So these both constitute independence systems.

Matroids Matroid Examples Matroid Rank More on Partition Matroid Independence System

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8     1 0 0 1 1 2 1 3 1 | | | | | | | |     (6.6) 2 0 1 1 0 2 0 2 4  =  x1 x2 x3 x4 x5 x6 x7 x8  3 1 1 1 0 0 3 1 5 | | | | | | | |

Given any set of linearly independent vectors A, any subset B A will also be linearly independent. ⊂

Given any forest Gf that is an edge-induced sub-graph of a graph G, any sub-graph of Gf is also a forest.

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F16/46 (pg.30/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Independence System

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8     1 0 0 1 1 2 1 3 1 | | | | | | | |     (6.6) 2 0 1 1 0 2 0 2 4  =  x1 x2 x3 x4 x5 x6 x7 x8  3 1 1 1 0 0 3 1 5 | | | | | | | |

Given any set of linearly independent vectors A, any subset B A will also be linearly independent. ⊂

Given any forest Gf that is an edge-induced sub-graph of a graph G, any sub-graph of Gf is also a forest. So these both constitute independence systems.

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F16/46 (pg.31/163) Because without (I1) could have a non-matroid where = . I {}

Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroid

Independent set deﬁnition of a matroid is perhaps most natural. Note, if J , then J is said to be an independent set. ∈ I Deﬁnition 6.3.3 (Matroid)

A set system (E, ) is a Matroid if I (I1) ∅ ∈ I (I2) I ,J I J (down-closed or subclusive) ∀ ∈ I ⊂ ⇒ ∈ I (I3) I,J , with I = J + 1, then there exists x I J such that ∀ ∈ I | | | | ∈ \ J x . ∪ { } ∈ I Why is (I1) is not redundant given (I2)?

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F17/46 (pg.32/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroid

Independent set deﬁnition of a matroid is perhaps most natural. Note, if J , then J is said to be an independent set. ∈ I Deﬁnition 6.3.3 (Matroid)

Prof. Jeff Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F17/46 (pg.33/163) Takeo Nakasawa, 1935, also early work. Forgotten for 20 years until mid 1950s. Matroids are powerful and flexible combinatorial objects. The rank function of a matroid is already a very powerful submodular function (perhaps all we need for many problems). Understanding matroids crucial for understanding submodularity. Matroid independent sets (i.e., A s.t. r(A) = A ) are useful constraint set, and fast algorithms for submodular optimization| | subject to one (or more) matroid independence constraints exist. Crapo & Rota preferred the term “combinatorial geometry”, or more specifically a “pregeometry” and said that pregeometries are “often described by the ineffably cacaphonic [sic] term ’matroid’, which we prefer to avoid in favor of the term ’pregeometry’.”

Matroids Matroid Examples Matroid Rank More on Partition Matroid On Matroids

Abstract properties of linear dependence (Hassler Whitney, 1935), but already then found instances of objects with those properties not based on a matrix.

Prof. Jeff Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F18/46 (pg.34/163) Forgotten for 20 years until mid 1950s. Matroids are powerful and flexible combinatorial objects. The rank function of a matroid is already a very powerful submodular function (perhaps all we need for many problems). Understanding matroids crucial for understanding submodularity. Matroid independent sets (i.e., A s.t. r(A) = A ) are useful constraint set, and fast algorithms for submodular optimization| | subject to one (or more) matroid independence constraints exist. Crapo & Rota preferred the term “combinatorial geometry”, or more specifically a “pregeometry” and said that pregeometries are “often described by the ineffably cacaphonic [sic] term ’matroid’, which we prefer to avoid in favor of the term ’pregeometry’.”

Matroids Matroid Examples Matroid Rank More on Partition Matroid On Matroids

Abstract properties of linear dependence (Hassler Whitney, 1935), but already then found instances of objects with those properties not based on a matrix. Takeo Nakasawa, 1935, also early work.

Prof. Jeff Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F18/46 (pg.35/163) Matroids are powerful and flexible combinatorial objects. The rank function of a matroid is already a very powerful submodular function (perhaps all we need for many problems). Understanding matroids crucial for understanding submodularity. Matroid independent sets (i.e., A s.t. r(A) = A ) are useful constraint set, and fast algorithms for submodular optimization| | subject to one (or more) matroid independence constraints exist. Crapo & Rota preferred the term “combinatorial geometry”, or more specifically a “pregeometry” and said that pregeometries are “often described by the ineffably cacaphonic [sic] term ’matroid’, which we prefer to avoid in favor of the term ’pregeometry’.”

Matroids Matroid Examples Matroid Rank More on Partition Matroid On Matroids

Prof. Jeff Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F18/46 (pg.36/163) The rank function of a matroid is already a very powerful submodular function (perhaps all we need for many problems). Understanding matroids crucial for understanding submodularity. Matroid independent sets (i.e., A s.t. r(A) = A ) are useful constraint set, and fast algorithms for submodular optimization| | subject to one (or more) matroid independence constraints exist. Crapo & Rota preferred the term “combinatorial geometry”, or more specifically a “pregeometry” and said that pregeometries are “often described by the ineffably cacaphonic [sic] term ’matroid’, which we prefer to avoid in favor of the term ’pregeometry’.”

Matroids Matroid Examples Matroid Rank More on Partition Matroid On Matroids

Prof. Jeff Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F18/46 (pg.37/163) Understanding matroids crucial for understanding submodularity. Matroid independent sets (i.e., A s.t. r(A) = A ) are useful constraint set, and fast algorithms for submodular optimization| | subject to one (or more) matroid independence constraints exist. Crapo & Rota preferred the term “combinatorial geometry”, or more specifically a “pregeometry” and said that pregeometries are “often described by the ineffably cacaphonic [sic] term ’matroid’, which we prefer to avoid in favor of the term ’pregeometry’.”

Matroids Matroid Examples Matroid Rank More on Partition Matroid On Matroids

Prof. Jeff Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F18/46 (pg.38/163) Matroid independent sets (i.e., A s.t. r(A) = A ) are useful constraint set, and fast algorithms for submodular optimization| | subject to one (or more) matroid independence constraints exist. Crapo & Rota preferred the term “combinatorial geometry”, or more specifically a “pregeometry” and said that pregeometries are “often described by the ineffably cacaphonic [sic] term ’matroid’, which we prefer to avoid in favor of the term ’pregeometry’.”

Matroids Matroid Examples Matroid Rank More on Partition Matroid On Matroids

Prof. Jeff Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F18/46 (pg.39/163) Crapo & Rota preferred the term “combinatorial geometry”, or more specifically a “pregeometry” and said that pregeometries are “often described by the ineffably cacaphonic [sic] term ’matroid’, which we prefer to avoid in favor of the term ’pregeometry’.”

Matroids Matroid Examples Matroid Rank More on Partition Matroid On Matroids

Abstract properties of linear dependence (Hassler Whitney, 1935), but already then found instances of objects with those properties not based on a matrix. Takeo Nakasawa, 1935, also early work. Forgotten for 20 years until mid 1950s. Matroids are powerful and ﬂexible combinatorial objects. The rank function of a matroid is already a very powerful submodular function (perhaps all we need for many problems). Understanding matroids crucial for understanding submodularity. Matroid independent sets (i.e., A s.t. r(A) = A ) are useful constraint set, and fast algorithms for submodular optimization| | subject to one (or more) matroid independence constraints exist.

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F18/46 (pg.40/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid On Matroids

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F18/46 (pg.41/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroid

Slight modiﬁcation (non unit increment) that is equivalent.

Deﬁnition 6.3.4 (Matroid-II)

A set system (E, ) is a Matroid if I (I1’) ∅ ∈ I (I2’) I ,J I J (down-closed or subclusive) ∀ ∈ I ⊂ ⇒ ∈ I (I3’) I,J , with I > J , then there exists x I J such that ∀ ∈ I | | | | ∈ \ J x ∪ { } ∈ I Note (I1)=(I1’), (I2)=(I2’), and we get (I3) (I3’) using induction. ≡

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F19/46 (pg.42/163) A base of U E: For U E, a subset B U is called a base of U if ⊆ ⊆ ⊆ B is inclusionwise maximally independent subset of U. That is, B ∈ I and there is no Z with B Z U. ∈ I ⊂ ⊆ A base of a matroid: If U = E, then a “base of E” is just called a base of the matroid M (this corresponds to a basis in a linear space, or a spanning forest in a graph, or a spanning tree in a connected graph).

Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids, independent sets, and bases

Independent sets: Given a matroid M = (E, ), a subset A E is I ⊆ called independent if A and otherwise A is called dependent. ∈ I

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F20/46 (pg.43/163) A base of a matroid: If U = E, then a “base of E” is just called a base of the matroid M (this corresponds to a basis in a linear space, or a spanning forest in a graph, or a spanning tree in a connected graph).

Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids, independent sets, and bases

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F20/46 (pg.44/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids, independent sets, and bases

Independent sets: Given a matroid M = (E, ), a subset A E is I ⊆ called independent if A and otherwise A is called dependent. ∈ I A base of U E: For U E, a subset B U is called a base of U if ⊆ ⊆ ⊆ B is inclusionwise maximally independent subset of U. That is, B ∈ I and there is no Z with B Z U. ∈ I ⊂ ⊆ A base of a matroid: If U = E, then a “base of E” is just called a base of the matroid M (this corresponds to a basis in a linear space, or a spanning forest in a graph, or a spanning tree in a connected graph).

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F20/46 (pg.45/163) (I1’) (emptyset containing) ∅ ∈ I (I2’) I ,J I J (down-closed or subclusive) ∀ ∈ I ⊂ ⇒ ∈ I (I3’) X V , and I1,I2 maxInd(X), we have I1 = I2 (all maximally independent∀ ⊆ subsets∈ of X have the same size).| | | |

Deﬁnition 6.3.6 (Matroid) A set system (V, ) is a Matroid if I

Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids - important property

Proposition 6.3.5 In a matroid M = (E, ), for any U E(M), any two bases of U have the same size. I ⊆

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F21/46 (pg.46/163) (I1’) (emptyset containing) ∅ ∈ I (I2’) I ,J I J (down-closed or subclusive) ∀ ∈ I ⊂ ⇒ ∈ I (I3’) X V , and I1,I2 maxInd(X), we have I1 = I2 (all maximally independent∀ ⊆ subsets∈ of X have the same size).| | | |

Deﬁnition 6.3.6 (Matroid) A set system (V, ) is a Matroid if I

In fact, under (I1),(I2), this condition is equivalent to (I3). Exercise: show the following is equivalent to the above.

Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids - important property

Proposition 6.3.5 In a matroid M = (E, ), for any U E(M), any two bases of U have the same size. I ⊆

In matrix terms, given a set of vectors U, all sets of independent vectors that span the space spanned by U have the same size.

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F21/46 (pg.47/163) (I1’) (emptyset containing) ∅ ∈ I (I2’) I ,J I J (down-closed or subclusive) ∀ ∈ I ⊂ ⇒ ∈ I (I3’) X V , and I1,I2 maxInd(X), we have I1 = I2 (all maximally independent∀ ⊆ subsets∈ of X have the same size).| | | |

Deﬁnition 6.3.6 (Matroid) A set system (V, ) is a Matroid if I

Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids - important property

Proposition 6.3.5 In a matroid M = (E, ), for any U E(M), any two bases of U have the same size. I ⊆

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F21/46 (pg.48/163) (I1’) (emptyset containing) ∅ ∈ I (I2’) I ,J I J (down-closed or subclusive) ∀ ∈ I ⊂ ⇒ ∈ I (I3’) X V , and I1,I2 maxInd(X), we have I1 = I2 (all maximally independent∀ ⊆ subsets∈ of X have the same size).| | | |

Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids - important property

Proposition 6.3.5 In a matroid M = (E, ), for any U E(M), any two bases of U have the same size. I ⊆

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F21/46 (pg.49/163) (I2’) I ,J I J (down-closed or subclusive) ∀ ∈ I ⊂ ⇒ ∈ I (I3’) X V , and I1,I2 maxInd(X), we have I1 = I2 (all maximally independent∀ ⊆ subsets∈ of X have the same size).| | | |

Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids - important property

Proposition 6.3.5 In a matroid M = (E, ), for any U E(M), any two bases of U have the same size. I ⊆

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F21/46 (pg.50/163) (I3’) X V , and I1,I2 maxInd(X), we have I1 = I2 (all maximally independent∀ ⊆ subsets∈ of X have the same size).| | | |

Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids - important property

Proposition 6.3.5 In a matroid M = (E, ), for any U E(M), any two bases of U have the same size. I ⊆

In matrix terms, given a set of vectors U, all sets of independent vectors that span the space spanned by U have the same size. In fact, under (I1),(I2), this condition is equivalent to (I3). Exercise: show the following is equivalent to the above. Deﬁnition 6.3.6 (Matroid) A set system (V, ) is a Matroid if I (I1’) (emptyset containing) ∅ ∈ I (I2’) I ,J I J (down-closed or subclusive) ∀ ∈ I ⊂ ⇒ ∈ I

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F21/46 (pg.51/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids - important property

Proposition 6.3.5 In a matroid M = (E, ), for any U E(M), any two bases of U have the same size. I ⊆

Prof. Jeff Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F21/46 (pg.52/163) Definition 6.3.7 (matroid rank function) E The rank function of a matroid is a function r : 2 Z+ defined by → r(A) = max X : X A, X = max A X (6.7) {| | ⊆ ∈ I} X∈I | ∩ |

The common size of all the bases of U is called the rank of U, denoted rM (U) or just r(U) when the matroid in equation is unambiguous. r(E) = r(E,I) is the rank of the matroid, and is the common size of all the bases of the matroid. We can a bit more formally deﬁne the rank function this way.

From the above, we immediately see that r(A) A . ≤ | | Moreover, if r(A) = A , then A , meaning A is independent (in | | ∈ I this case, A is a self base).

Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids - rank Thus, in any matroid M = (E, ), U E(M), any two bases of U have the same size. I ∀ ⊆

Prof. Jeff Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F22/46 (pg.53/163) Definition 6.3.7 (matroid rank function) E The rank function of a matroid is a function r : 2 Z+ defined by → r(A) = max X : X A, X = max A X (6.7) {| | ⊆ ∈ I} X∈I | ∩ |

r(E) = r(E,I) is the rank of the matroid, and is the common size of all the bases of the matroid. We can a bit more formally deﬁne the rank function this way.

From the above, we immediately see that r(A) A . ≤ | | Moreover, if r(A) = A , then A , meaning A is independent (in | | ∈ I this case, A is a self base).

Prof. Jeff Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F22/46 (pg.54/163) Definition 6.3.7 (matroid rank function) E The rank function of a matroid is a function r : 2 Z+ defined by → r(A) = max X : X A, X = max A X (6.7) {| | ⊆ ∈ I} X∈I | ∩ |

We can a bit more formally deﬁne the rank function this way.

From the above, we immediately see that r(A) A . ≤ | | Moreover, if r(A) = A , then A , meaning A is independent (in | | ∈ I this case, A is a self base).

Prof. Jeff Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F22/46 (pg.55/163) Definition 6.3.7 (matroid rank function) E The rank function of a matroid is a function r : 2 Z+ defined by → r(A) = max X : X A, X = max A X (6.7) {| | ⊆ ∈ I} X∈I | ∩ |

From the above, we immediately see that r(A) A . ≤ | | Moreover, if r(A) = A , then A , meaning A is independent (in | | ∈ I this case, A is a self base).

Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids - rank Thus, in any matroid M = (E, ), U E(M), any two bases of U have the same size. I ∀ ⊆ The common size of all the bases of U is called the rank of U, denoted rM (U) or just r(U) when the matroid in equation is unambiguous. r(E) = r(E,I) is the rank of the matroid, and is the common size of all the bases of the matroid. We can a bit more formally deﬁne the rank function this way.

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F22/46 (pg.56/163) From the above, we immediately see that r(A) A . ≤ | | Moreover, if r(A) = A , then A , meaning A is independent (in | | ∈ I this case, A is a self base).

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F22/46 (pg.57/163) Moreover, if r(A) = A , then A , meaning A is independent (in | | ∈ I this case, A is a self base).

From the above, we immediately see that r(A) A . ≤ | |

Prof. Jeff Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F22/46 (pg.58/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids - rank Thus, in any matroid M = (E, ), U E(M), any two bases of U have the same size. I ∀ ⊆ The common size of all the bases of U is called the rank of U, denoted rM (U) or just r(U) when the matroid in equation is unambiguous. r(E) = r(E,I) is the rank of the matroid, and is the common size of all the bases of the matroid. We can a bit more formally define the rank function this way. Definition 6.3.7 (matroid rank function) E The rank function of a matroid is a function r : 2 Z+ defined by → r(A) = max X : X A, X = max A X (6.7) {| | ⊆ ∈ I} X∈I | ∩ |

From the above, we immediately see that r(A) A . ≤ | | Moreover, if r(A) = A , then A , meaning A is independent (in | | ∈ I this case, A is a self base).

Prof. Jeff Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F22/46 (pg.59/163) Definition 6.3.9 (closure) Given A E, the closure (or span) of A, is defined by ⊆ span(A) = b E : r(A b ) = r(A) . { ∈ ∪ { } } Therefore, a closed set A has span(A) = A. Definition 6.3.10 (circuit)

A subset A E is circuit or a cycle if it is an inclusionwise-minimal ⊆ dependent set (i.e., if r(A) < A and for any a A, r(A a ) = A 1). | | ∈ \{ } | | −

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F23/46 (pg.60/163) Therefore, a closed set A has span(A) = A. Deﬁnition 6.3.10 (circuit)

A subset A E is circuit or a cycle if it is an inclusionwise-minimal ⊆ dependent set (i.e., if r(A) < A and for any a A, r(A a ) = A 1). | | ∈ \{ } | | −

Matroids Matroid Examples Matroid Rank More on Partition Matroid V Matroids, other definitions using matroid rank r : 2 Z+ → Definition 6.3.8 (closed/flat/subspace) A subset A E is closed (equivalently, a flat or a subspace) of matroid M ⊆ if for all x E A, r(A x ) = r(A) + 1. ∈ \ ∪ { } Definition: A hyperplane is a flat of rank r(M) 1. − Definition 6.3.9 (closure) Given A E, the closure (or span) of A, is defined by ⊆ span(A) = b E : r(A b ) = r(A) . { ∈ ∪ { } }

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F23/46 (pg.61/163) Deﬁnition 6.3.10 (circuit)

A subset A E is circuit or a cycle if it is an inclusionwise-minimal ⊆ dependent set (i.e., if r(A) < A and for any a A, r(A a ) = A 1). | | ∈ \{ } | | −

Prof. Jeff Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F23/46 (pg.62/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid V Matroids, other definitions using matroid rank r : 2 Z+ → Definition 6.3.8 (closed/flat/subspace) A subset A E is closed (equivalently, a flat or a subspace) of matroid M ⊆ if for all x E A, r(A x ) = r(A) + 1. ∈ \ ∪ { } Definition: A hyperplane is a flat of rank r(M) 1. − Definition 6.3.9 (closure) Given A E, the closure (or span) of A, is defined by ⊆ span(A) = b E : r(A b ) = r(A) . { ∈ ∪ { } } Therefore, a closed set A has span(A) = A. Definition 6.3.10 (circuit)

A subset A E is circuit or a cycle if it is an inclusionwise-minimal ⊆ dependent set (i.e., if r(A) < A and for any a A, r(A a ) = A 1). | | ∈ \{ } | | −

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F23/46 (pg.63/163) Proof here is omitted but think about this for a moment in terms of linear spaces and matrices, and (alternatively) spanning trees.

Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids by bases

In general, besides independent sets and rank functions, there are other equivalent ways to characterize matroids. Theorem 6.3.11 (Matroid (by bases)) Let E be a set and be a nonempty collection of subsets of E. Then the following are equivalent.B 1 is the collection of bases of a matroid; B 2 if B,B0 , and x B0 B, then B0 x + y for some y B B0. ∈ B ∈ \ − ∈ B ∈ \ 3 If B,B0 , and x B0 B, then B y + x for some y B B0. ∈ B ∈ \ − ∈ B ∈ \ Properties 2 and 3 are called “exchange properties.”

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F24/46 (pg.64/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids by bases

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F24/46 (pg.65/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids by circuits

A set is independent if and only if it contains no circuit. Therefore, it is not surprising that circuits can also characterize a matroid. Theorem 6.3.12 (Matroid by circuits) Let E be a set and be a collection of subsets of E that satisfy the following three properties:C 1 (C1): / ∅ ∈ C 2 (C2): if C1,C2 and C1 C2, then C1 = C2. ∈ C ⊆ 3 (C3): if C1,C2 with C1 = C2, and e C1 C2, then there exists a ∈ C 6 ∈ ∩ C3 such that C3 (C1 C2) e . ∈ C ⊆ ∪ \{ }

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F25/46 (pg.66/163) Again, think about this for a moment in terms of linear spaces and matrices, and spanning trees.

Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids by circuits

Several circuit deﬁnitions for matroids. Theorem 6.3.13 (Matroid by circuits) Let E be a set and be a collection of nonempty subsets of E, such that no two sets in areC contained in each other. Then the following are equivalent. C 1 is the collection of circuits of a matroid; C 2 if C,C0 , and x C C0, then (C C0) x contains a set in ; ∈ C ∈ ∩ ∪ \{ } C 3 if C,C0 , and x C C0, and y C C0, then (C C0) x ∈ C ∈ ∩ ∈ \ ∪ \{ } contains a set in containing y; C

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F26/46 (pg.67/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids by circuits

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F26/46 (pg.68/163) Then (E, ) is a matroid called a k-uniform matroid. I Note, if I,J , and I < J k, and j J such that j I, then j ∈ I | | | | ≤ ∈ 6∈ is such that I + j k and so I + j . Rank function| | ≤ ∈ I ( A if A k r(A) = | | | | ≤ (6.8) k if A > k | | Note, this function is submodular. Not surprising since r(A) = min( A , k) which is a non-decreasing concave function applied to a modular| function.| Closure function ( A if A < k, span(A) = | | (6.9) E if A k, | | ≥ A “free” matroid sets k = E , so everything is independent. | |

Matroids Matroid Examples Matroid Rank More on Partition Matroid Uniform Matroid Given E, consider to be all subsets of E that are at most size k. I That is = A E : A k . I { ⊆ | | ≤ }

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F27/46 (pg.69/163) Note, if I,J , and I < J k, and j J such that j I, then j ∈ I | | | | ≤ ∈ 6∈ is such that I + j k and so I + j . Rank function| | ≤ ∈ I ( A if A k r(A) = | | | | ≤ (6.8) k if A > k | | Note, this function is submodular. Not surprising since r(A) = min( A , k) which is a non-decreasing concave function applied to a modular| function.| Closure function ( A if A < k, span(A) = | | (6.9) E if A k, | | ≥ A “free” matroid sets k = E , so everything is independent. | |

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F27/46 (pg.70/163) Rank function ( A if A k r(A) = | | | | ≤ (6.8) k if A > k | | Note, this function is submodular. Not surprising since r(A) = min( A , k) which is a non-decreasing concave function applied to a modular| function.| Closure function ( A if A < k, span(A) = | | (6.9) E if A k, | | ≥ A “free” matroid sets k = E , so everything is independent. | |

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F27/46 (pg.71/163) Note, this function is submodular. Not surprising since r(A) = min( A , k) which is a non-decreasing concave function applied to a modular| function.| Closure function ( A if A < k, span(A) = | | (6.9) E if A k, | | ≥ A “free” matroid sets k = E , so everything is independent. | |

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F27/46 (pg.72/163) Closure function ( A if A < k, span(A) = | | (6.9) E if A k, | | ≥ A “free” matroid sets k = E , so everything is independent. | |

Matroids Matroid Examples Matroid Rank More on Partition Matroid Uniform Matroid Given E, consider to be all subsets of E that are at most size k. I That is = A E : A k . I { ⊆ | | ≤ } Then (E, ) is a matroid called a k-uniform matroid. I Note, if I,J , and I < J k, and j J such that j I, then j ∈ I | | | | ≤ ∈ 6∈ is such that I + j k and so I + j . Rank function| | ≤ ∈ I ( A if A k r(A) = | | | | ≤ (6.8) k if A > k | | Note, this function is submodular. Not surprising since r(A) = min( A , k) which is a non-decreasing concave function applied to a modular| function.|

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F27/46 (pg.73/163) A “free” matroid sets k = E , so everything is independent. | |

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F27/46 (pg.74/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Uniform Matroid Given E, consider to be all subsets of E that are at most size k. I That is = A E : A k . I { ⊆ | | ≤ } Then (E, ) is a matroid called a k-uniform matroid. I Note, if I,J , and I < J k, and j J such that j I, then j ∈ I | | | | ≤ ∈ 6∈ is such that I + j k and so I + j . Rank function| | ≤ ∈ I ( A if A k r(A) = | | | | ≤ (6.8) k if A > k | | Note, this function is submodular. Not surprising since r(A) = min( A , k) which is a non-decreasing concave function applied to a modular| function.| Closure function ( A if A < k, span(A) = | | (6.9) E if A k, | | ≥ A “free” matroid sets k = E , so everything is independent. | | Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F27/46 (pg.75/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Linear (or Matric) Matroid

Let X be an n m matrix and E = 1, . . . , m × { } Let consists of subsets of E such that if A , and I ∈ I A = a1, a2, . . . , ak then the vectors xa1 , xa2 , . . . , xak are linearly independent.{ } the rank function is just the rank of the space spanned by the corresponding set of vectors. rank is submodular, it is intuitive that it satisﬁes the diminishing returns property (a given vector can only become linearly dependent in a greater context, thereby no longer contributing to rank). Called both linear matroids and matric matroids.

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F28/46 (pg.76/163) Then M = (E, ) is a matroid. I contains all forests. I Bases are spanning forests (spanning trees if G is connected). Rank function r(A) is the size of the largest spanning forest contained in G(V,A).

Recall from earlier, r(A) = V (G) kG(A), where for A E(G), we | | − ⊆ deﬁne kG(A) as the number of connected components of the edge-induced spanning subgraph (V (G),A), and that kG(A) is supermodular, so V (G) kG(A) is submodular. | | − Closure function adds all edges between the vertices adjacent to any edge in A. Closure of a spanning forest is G.

Matroids Matroid Examples Matroid Rank More on Partition Matroid Cycle Matroid of a graph: Graphic Matroids

Let G = (V,E) be a graph. Consider (E, ) where the edges of the I graph E are the ground set and A if the edge-induced graph ∈ I G(V,A) by A does not contain any cycle.

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F29/46 (pg.77/163) contains all forests. I Bases are spanning forests (spanning trees if G is connected). Rank function r(A) is the size of the largest spanning forest contained in G(V,A).

Matroids Matroid Examples Matroid Rank More on Partition Matroid Cycle Matroid of a graph: Graphic Matroids

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F29/46 (pg.78/163) Bases are spanning forests (spanning trees if G is connected). Rank function r(A) is the size of the largest spanning forest contained in G(V,A).

Matroids Matroid Examples Matroid Rank More on Partition Matroid Cycle Matroid of a graph: Graphic Matroids

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F29/46 (pg.79/163) Rank function r(A) is the size of the largest spanning forest contained in G(V,A).

Matroids Matroid Examples Matroid Rank More on Partition Matroid Cycle Matroid of a graph: Graphic Matroids

Let G = (V,E) be a graph. Consider (E, ) where the edges of the I graph E are the ground set and A if the edge-induced graph ∈ I G(V,A) by A does not contain any cycle. Then M = (E, ) is a matroid. I contains all forests. I Bases are spanning forests (spanning trees if G is connected).

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F29/46 (pg.80/163) Recall from earlier, r(A) = V (G) kG(A), where for A E(G), we | | − ⊆ deﬁne kG(A) as the number of connected components of the edge-induced spanning subgraph (V (G),A), and that kG(A) is supermodular, so V (G) kG(A) is submodular. | | − Closure function adds all edges between the vertices adjacent to any edge in A. Closure of a spanning forest is G.

Matroids Matroid Examples Matroid Rank More on Partition Matroid Cycle Matroid of a graph: Graphic Matroids

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F29/46 (pg.81/163) Closure function adds all edges between the vertices adjacent to any edge in A. Closure of a spanning forest is G.

Matroids Matroid Examples Matroid Rank More on Partition Matroid Cycle Matroid of a graph: Graphic Matroids

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F29/46 (pg.82/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Cycle Matroid of a graph: Graphic Matroids

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F29/46 (pg.83/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Example: graphic matroid

A graph deﬁnes a matroid on edge sets, independent sets are those without a cycle. 5 2 7 1 12 3 9 8 8 1 4 6 11 4 7 2 10

3 6 5

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F30/46 (pg.84/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Example: graphic matroid

A graph deﬁnes a matroid on edge sets, independent sets are those without a cycle. 5 2 7 1 12 3 9 8 8 1 4 6 11 4 7 2 10

3 6 5

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F30/46 (pg.85/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Example: graphic matroid

A graph deﬁnes a matroid on edge sets, independent sets are those without a cycle. 5 2 7 1 12 3 9 8 8 1 4 6 11 4 7 2 10

3 6 5

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F30/46 (pg.86/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Example: graphic matroid

A graph deﬁnes a matroid on edge sets, independent sets are those without a cycle. 5 2 7 1 12 3 9 8 8 1 4 6 11 4 7 2 10

3 6 5

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F30/46 (pg.87/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Example: graphic matroid

A graph deﬁnes a matroid on edge sets, independent sets are those without a cycle. 5 2 7 1 12 3 9 8 8 1 4 6 11 4 7 2 10

3 6 5

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F30/46 (pg.88/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Example: graphic matroid

A graph deﬁnes a matroid on edge sets, independent sets are those without a cycle. 5 2 7 1 12 3 9 8 8 1 4 6 11 4 7 2 10

3 6 5

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F30/46 (pg.89/163) Let V = V1 V2 V` be a partition of V into ` blocks (i.e., ∪ ∪ · · · ∪ disjoint sets). Deﬁne a set of subsets of V as

= X V : X Vi ki for all i = 1, . . . , ` . (6.10) I { ⊆ | ∩ | ≤ } where k1, . . . , k` are ﬁxed “limit” parameters, ki 0. Then M = (V, ) is a matroid. ≥ I Note that a k-uniform matroid is a trivial example of a partition matroid with ` = 1, V1 = V , and k1 = k. Parameters associated with a partition matroid: ` and k1, k2, . . . , k` although often the ki’s are all the same. We’ll show that property (I3’) in Def 6.3.4 holds. First note, for any P` X V , X = X Vi since V1,V2,...,V` is a partition. ⊆ | | i=1 | ∩ | { } If X,Y with Y > X , then there must be at least one i with ∈ I | | | | Y Vi > X Vi . Therefore, adding one element e Vi (Y X) | ∩ | | ∩ | ∈ ∩ \ to X won’t break independence.

Matroids Matroid Examples Matroid Rank More on Partition Matroid Partition Matroid

Let V be our ground set.

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F31/46 (pg.90/163) Note that a k-uniform matroid is a trivial example of a partition matroid with ` = 1, V1 = V , and k1 = k. Parameters associated with a partition matroid: ` and k1, k2, . . . , k` although often the ki’s are all the same. We’ll show that property (I3’) in Def 6.3.4 holds. First note, for any P` X V , X = X Vi since V1,V2,...,V` is a partition. ⊆ | | i=1 | ∩ | { } If X,Y with Y > X , then there must be at least one i with ∈ I | | | | Y Vi > X Vi . Therefore, adding one element e Vi (Y X) | ∩ | | ∩ | ∈ ∩ \ to X won’t break independence.

Matroids Matroid Examples Matroid Rank More on Partition Matroid Partition Matroid

Let V be our ground set. Let V = V1 V2 V` be a partition of V into ` blocks (i.e., ∪ ∪ · · · ∪ disjoint sets). Deﬁne a set of subsets of V as

= X V : X Vi ki for all i = 1, . . . , ` . (6.10) I { ⊆ | ∩ | ≤ } where k1, . . . , k` are ﬁxed “limit” parameters, ki 0. Then M = (V, ) is a matroid. ≥ I

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F31/46 (pg.91/163) Parameters associated with a partition matroid: ` and k1, k2, . . . , k` although often the ki’s are all the same. We’ll show that property (I3’) in Def 6.3.4 holds. First note, for any P` X V , X = X Vi since V1,V2,...,V` is a partition. ⊆ | | i=1 | ∩ | { } If X,Y with Y > X , then there must be at least one i with ∈ I | | | | Y Vi > X Vi . Therefore, adding one element e Vi (Y X) | ∩ | | ∩ | ∈ ∩ \ to X won’t break independence.

Matroids Matroid Examples Matroid Rank More on Partition Matroid Partition Matroid

Let V be our ground set. Let V = V1 V2 V` be a partition of V into ` blocks (i.e., ∪ ∪ · · · ∪ disjoint sets). Deﬁne a set of subsets of V as

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F31/46 (pg.92/163) We’ll show that property (I3’) in Def 6.3.4 holds. First note, for any P` X V , X = X Vi since V1,V2,...,V` is a partition. ⊆ | | i=1 | ∩ | { } If X,Y with Y > X , then there must be at least one i with ∈ I | | | | Y Vi > X Vi . Therefore, adding one element e Vi (Y X) | ∩ | | ∩ | ∈ ∩ \ to X won’t break independence.

Matroids Matroid Examples Matroid Rank More on Partition Matroid Partition Matroid

Let V be our ground set. Let V = V1 V2 V` be a partition of V into ` blocks (i.e., ∪ ∪ · · · ∪ disjoint sets). Deﬁne a set of subsets of V as

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F31/46 (pg.93/163) If X,Y with Y > X , then there must be at least one i with ∈ I | | | | Y Vi > X Vi . Therefore, adding one element e Vi (Y X) | ∩ | | ∩ | ∈ ∩ \ to X won’t break independence.

Matroids Matroid Examples Matroid Rank More on Partition Matroid Partition Matroid

Let V be our ground set. Let V = V1 V2 V` be a partition of V into ` blocks (i.e., ∪ ∪ · · · ∪ disjoint sets). Deﬁne a set of subsets of V as

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F31/46 (pg.94/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Partition Matroid

Let V be our ground set. Let V = V1 V2 V` be a partition of V into ` blocks (i.e., ∪ ∪ · · · ∪ disjoint sets). Deﬁne a set of subsets of V as

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F31/46 (pg.95/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Partition Matroid Ground set of objects, V =

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F32/46 (pg.96/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Partition Matroid

Partition of V into six blocks, V1,V2,...,V6

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F32/46 (pg.97/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Partition Matroid

Limit associated with each block, k1, k2, . . . , k6 { }

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F32/46 (pg.98/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Partition Matroid Independent subset but not maximally independent.

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F32/46 (pg.99/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Partition Matroid Maximally independent subset, what is called a base.

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F32/46 (pg.100/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Partition Matroid Not independent since over limit in set six.

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F32/46 (pg.101/163) Y A + Y B ≥ | ∩ | | ∩ | = Y (A B) + Y (A B) (6.12) | ∩ ∩ | | ∩ ∪ | X + Y = r(A B) + r(A B) (6.13) ≥ | | | | ∩ ∪

1 Let X be an inclusionwise maximal set with X A B ∈ I ⊆ ∩ 2 Let Y be inclusionwise maximal set with X Y A B. ∈ I ⊆ ⊆ ∪ 3 Since M is a matroid, we know that r(A B) = r(X) = X , and ∩ | | r(A B) = r(Y ) = Y . Also, for any U , r(A) A U . ∪ | | ∈ I ≥ | ∩ | 4 Then we have (since X A B, X Y , and Y A B), ⊆ ∩ ⊆ ⊆ ∪ r(A) + r(B) (6.11)

Proof.

Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids - rank function is submodular Lemma 6.5.1

E The rank function r : 2 Z+ of a matroid is submodular, that is → r(A) + r(B) r(A B) + r(A B) ≥ ∪ ∩

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F33/46 (pg.102/163) Y A + Y B ≥ | ∩ | | ∩ | = Y (A B) + Y (A B) (6.12) | ∩ ∩ | | ∩ ∪ | X + Y = r(A B) + r(A B) (6.13) ≥ | | | | ∩ ∪

2 Let Y be inclusionwise maximal set with X Y A B. ∈ I ⊆ ⊆ ∪ 3 Since M is a matroid, we know that r(A B) = r(X) = X , and ∩ | | r(A B) = r(Y ) = Y . Also, for any U , r(A) A U . ∪ | | ∈ I ≥ | ∩ | 4 Then we have (since X A B, X Y , and Y A B), ⊆ ∩ ⊆ ⊆ ∪ r(A) + r(B) (6.11)

Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids - rank function is submodular Lemma 6.5.1

E The rank function r : 2 Z+ of a matroid is submodular, that is → r(A) + r(B) r(A B) + r(A B) ≥ ∪ ∩ Proof. 1 Let X be an inclusionwise maximal set with X A B ∈ I ⊆ ∩

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F33/46 (pg.103/163) Y A + Y B ≥ | ∩ | | ∩ | = Y (A B) + Y (A B) (6.12) | ∩ ∩ | | ∩ ∪ | X + Y = r(A B) + r(A B) (6.13) ≥ | | | | ∩ ∪

Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids - rank function is submodular Lemma 6.5.1

E The rank function r : 2 Z+ of a matroid is submodular, that is → r(A) + r(B) r(A B) + r(A B) ≥ ∪ ∩ Proof. 1 Let X be an inclusionwise maximal set with X A B ∈ I ⊆ ∩ 2 Let Y be inclusionwise maximal set with X Y A B. We ∈ I ⊆ ⊆ ∪ can ﬁnd such a Y X because the following. Let Y 0 be any inclusionwise ⊇ ∈ I maximal set with Y 0 A B, which might not have X Y 0. Starting from ⊆ ∪ ⊆ Y X A B, since Y 0 X , there exists a y Y 0 X A B such that ← ⊆ ∪ | | ≥ | | ∈ \ ⊆ ∪ X + y but since y A B, also X + y A B — we then add y to Y . We ∈ I ∈ ∪ ∈ ∪ can keep doing this while Y 0 > X since this is a matroid. We end up with an | | | | inclusionwise maximal set Y with Y and X Y . ∈ I ⊆

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F33/46 (pg.104/163) Y A + Y B ≥ | ∩ | | ∩ | = Y (A B) + Y (A B) (6.12) | ∩ ∩ | | ∩ ∪ | X + Y = r(A B) + r(A B) (6.13) ≥ | | | | ∩ ∪

4 Then we have (since X A B, X Y , and Y A B), ⊆ ∩ ⊆ ⊆ ∪ r(A) + r(B) (6.11)

Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids - rank function is submodular Lemma 6.5.1

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F33/46 (pg.105/163) Y A + Y B ≥ | ∩ | | ∩ | = Y (A B) + Y (A B) (6.12) | ∩ ∩ | | ∩ ∪ | X + Y = r(A B) + r(A B) (6.13) ≥ | | | | ∩ ∪

Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids - rank function is submodular Lemma 6.5.1

E The rank function r : 2 Z+ of a matroid is submodular, that is → r(A) + r(B) r(A B) + r(A B) ≥ ∪ ∩ Proof. 1 Let X be an inclusionwise maximal set with X A B ∈ I ⊆ ∩ 2 Let Y be inclusionwise maximal set with X Y A B. ∈ I ⊆ ⊆ ∪ 3 Since M is a matroid, we know that r(A B) = r(X) = X , and ∩ | | r(A B) = r(Y ) = Y . Also, for any U , r(A) A U . ∪ | | ∈ I ≥ | ∩ | 4 Then we have (since X A B, X Y , and Y A B), ⊆ ∩ ⊆ ⊆ ∪ r(A) + r(B) (6.11)

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F33/46 (pg.106/163) = Y (A B) + Y (A B) (6.12) | ∩ ∩ | | ∩ ∪ | X + Y = r(A B) + r(A B) (6.13) ≥ | | | | ∩ ∪

Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids - rank function is submodular Lemma 6.5.1

E The rank function r : 2 Z+ of a matroid is submodular, that is → r(A) + r(B) r(A B) + r(A B) ≥ ∪ ∩ Proof. 1 Let X be an inclusionwise maximal set with X A B ∈ I ⊆ ∩ 2 Let Y be inclusionwise maximal set with X Y A B. ∈ I ⊆ ⊆ ∪ 3 Since M is a matroid, we know that r(A B) = r(X) = X , and ∩ | | r(A B) = r(Y ) = Y . Also, for any U , r(A) A U . ∪ | | ∈ I ≥ | ∩ | 4 Then we have (since X A B, X Y , and Y A B), ⊆ ∩ ⊆ ⊆ ∪ r(A) + r(B) Y A + Y B (6.11) ≥ | ∩ | | ∩ |

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F33/46 (pg.107/163) X + Y = r(A B) + r(A B) (6.13) ≥ | | | | ∩ ∪

Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids - rank function is submodular Lemma 6.5.1

E The rank function r : 2 Z+ of a matroid is submodular, that is → r(A) + r(B) r(A B) + r(A B) ≥ ∪ ∩ Proof. 1 Let X be an inclusionwise maximal set with X A B ∈ I ⊆ ∩ 2 Let Y be inclusionwise maximal set with X Y A B. ∈ I ⊆ ⊆ ∪ 3 Since M is a matroid, we know that r(A B) = r(X) = X , and ∩ | | r(A B) = r(Y ) = Y . Also, for any U , r(A) A U . ∪ | | ∈ I ≥ | ∩ | 4 Then we have (since X A B, X Y , and Y A B), ⊆ ∩ ⊆ ⊆ ∪ r(A) + r(B) Y A + Y B (6.11) ≥ | ∩ | | ∩ | = Y (A B) + Y (A B) (6.12) | ∩ ∩ | | ∩ ∪ |

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F33/46 (pg.108/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid Matroids - rank function is submodular Lemma 6.5.1

E The rank function r : 2 Z+ of a matroid is submodular, that is → r(A) + r(B) r(A B) + r(A B) ≥ ∪ ∩ Proof. 1 Let X be an inclusionwise maximal set with X A B ∈ I ⊆ ∩ 2 Let Y be inclusionwise maximal set with X Y A B. ∈ I ⊆ ⊆ ∪ 3 Since M is a matroid, we know that r(A B) = r(X) = X , and ∩ | | r(A B) = r(Y ) = Y . Also, for any U , r(A) A U . ∪ | | ∈ I ≥ | ∩ | 4 Then we have (since X A B, X Y , and Y A B), ⊆ ∩ ⊆ ⊆ ∪ r(A) + r(B) Y A + Y B (6.11) ≥ | ∩ | | ∩ | = Y (A B) + Y (A B) (6.12) | ∩ ∩ | | ∩ ∪ | X + Y = r(A B) + r(A B) (6.13) ≥ | | | | ∩ ∪

Prof. Jeff Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F33/46 (pg.109/163) Hence, unit increment (if r(A) = k, then either r(A v ) = k or ∪ { } r(A v ) = k + 1) holds. Thus,∪ submodularity { } , non-negative monotone non-decreasing, and unit increment of rank is necessary and sufficient to define a matroid. Can refer to matroid as (E, r), E is ground set, r is rank function.

Matroids Matroid Examples Matroid Rank More on Partition Matroid A matroid is deﬁned from its rank function Theorem 6.5.2 (Matroid from rank)

Prof. Jeff Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F34/46 (pg.110/163) Thus, submodularity, non-negative monotone non-decreasing, and unit increment of rank is necessary and sufficient to define a matroid. Can refer to matroid as (E, r), E is ground set, r is rank function.

Matroids Matroid Examples Matroid Rank More on Partition Matroid A matroid is deﬁned from its rank function Theorem 6.5.2 (Matroid from rank)

E Let E be a set and let r : 2 Z+ be a function. Then r( ) deﬁnes a → · matroid with r being its rank function if and only if for all A, B E: ⊆ (R1) A E 0 r(A) A (non-negative cardinality bounded) ∀ ⊆ ≤ ≤ | | (R2) r(A) r(B) whenever A B E (monotone non-decreasing) ≤ ⊆ ⊆ (R3) r(A B) + r(A B) r(A) + r(B) for all A, B E (submodular) ∪ ∩ ≤ ⊆ From above, r( ) = 0. Let v / A, then by monotonicity and ∅ ∈ submodularity, r(A) r(A v ) r(A) + r( v ) which gives only ≤ ∪ { } ≤ { } two possible values to r(A v ), namely r(A) or r(A) + 1. ∪ { } Hence, unit increment (if r(A) = k, then either r(A v ) = k or ∪ { } r(A v ) = k + 1) holds. ∪ { }

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F34/46 (pg.111/163) Can refer to matroid as (E, r), E is ground set, r is rank function.

Matroids Matroid Examples Matroid Rank More on Partition Matroid A matroid is deﬁned from its rank function Theorem 6.5.2 (Matroid from rank)

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F34/46 (pg.112/163) Matroids Matroid Examples Matroid Rank More on Partition Matroid A matroid is deﬁned from its rank function Theorem 6.5.2 (Matroid from rank)

Prof. Jeﬀ Bilmes EE563/Spring 2018/Submodularity - Lecture 6 - April 11th, 2018 F34/46 (pg.113/163) + r( ) ∅

r(X) r(Y ) r(Y X) (6.14) ≥ − \ Y Y X (6.15) ≥ | | − | \ | = X (6.16) | | implying r(X) = X , and thus X . | | ∈ I

Next, assume we have (R1), (R2), and (R3). Deﬁne = X E : r(X) = X . We will show that (E, ) is a matroid. I { ⊆ | |} I First, . ∅ ∈ I Also, if Y and X Y then by submodularity, ∈ I ⊆