CS675: Convex and Combinatorial Optimization Fall 2019 Introduction to Matroid Theory

Home , Convex hull, Matroid intersection

Instructor: Shaddin Dughmi Set system: Pair (X , I) where X is a ﬁnite ground set and I ⊆ 2X are the feasible sets Objective: often “linear”, referred to as modular Analogues of concave and convex: submodular and supermodular (in no particular order!) Today, we will look only at optimizing modular objectives over an extremely proliﬁc family of set systems Related, directly or indirectly, to a large fraction of optimization problems in P Also pops up in submodular/supermodular optimization problems

Optimization over Sets

Most combinatorial optimization problems can be thought of as choosing the best set from a family of allowable sets Shortest paths Max-weight matching Independent set ...

1/30 Objective: often “linear”, referred to as modular Analogues of concave and convex: submodular and supermodular (in no particular order!) Today, we will look only at optimizing modular objectives over an extremely proliﬁc family of set systems Related, directly or indirectly, to a large fraction of optimization problems in P Also pops up in submodular/supermodular optimization problems

Optimization over Sets

1/30 Analogues of concave and convex: submodular and supermodular (in no particular order!) Today, we will look only at optimizing modular objectives over an extremely proliﬁc family of set systems Related, directly or indirectly, to a large fraction of optimization problems in P Also pops up in submodular/supermodular optimization problems

Optimization over Sets

1/30 Today, we will look only at optimizing modular objectives over an extremely proliﬁc family of set systems Related, directly or indirectly, to a large fraction of optimization problems in P Also pops up in submodular/supermodular optimization problems

Optimization over Sets

1/30 Optimization over Sets

Most combinatorial optimization problems can be thought of as choosing the best set from a family of allowable sets Shortest paths Max-weight matching Independent set ... Set system: Pair (X , I) where X is a ﬁnite ground set and I ⊆ 2X are the feasible sets Objective: often “linear”, referred to as modular Analogues of concave and convex: submodular and supermodular (in no particular order!) Today, we will look only at optimizing modular objectives over an extremely proliﬁc family of set systems Related, directly or indirectly, to a large fraction of optimization problems in P Also pops up in submodular/supermodular optimization problems

1/30 Outline

1 Matroids and The Greedy Algorithm

2 Basic Terminology and Properties

3 The Matroid Polytope

4 Matroid Intersection Maximum Weight Forest Problem

Given an undirected graph G = (V,E), and weights we ∈ R on edges e, ﬁnd a maximum weight acyclic subgraph (aka forest) of G.

Slight generalization of minimum weight spanning tree We use n and m to denote |V | and |E|, respectively.

Matroids and The Greedy Algorithm 2/30 Theorem The greedy algorithm outputs a maximum-weight forest.

The Greedy Algorithm

1 B ← ∅ 2 Sort non-negative weight edges in decreasing order of weight

e1, . . . , em, with w1 ≥ w2 ≥ ... ≥ wm ≥ 0 3 For i = 1 to m: S if B {ei} is acyclic, add ei to B.

Matroids and The Greedy Algorithm 3/30 The Greedy Algorithm

1 B ← ∅ 2 Sort non-negative weight edges in decreasing order of weight

e1, . . . , em, with w1 ≥ w2 ≥ ... ≥ wm ≥ 0 3 For i = 1 to m: S if B {ei} is acyclic, add ei to B.

Theorem The greedy algorithm outputs a maximum-weight forest.

Matroids and The Greedy Algorithm 3/30 Contrapositive: if B cyclic then so is A

Inductively: can extend A by adding |B| − |A| elements from B \ A

Lemma 1 The empty set is acyclic 2 If A is an acyclic set of edges, and B ⊆ A, then B is also acyclic.

3 If A, B are acyclic, and |B| > |A|, then there is e ∈ B \ A such that A S {e} is acyclic

Matroids and The Greedy Algorithm 4/30 Contrapositive: if B cyclic then so is A

Inductively: can extend A by adding |B| − |A| elements from B \ A

Lemma 1 The empty set is acyclic 2 If A is an acyclic set of edges, and B ⊆ A, then B is also acyclic.

3 If A, B are acyclic, and |B| > |A|, then there is e ∈ B \ A such that A S {e} is acyclic

(1) and (2) are trivial, so let’s prove (3)

Matroids and The Greedy Algorithm 4/30 Contrapositive: if B cyclic then so is A

Inductively: can extend A by adding |B| − |A| elements from B \ A

When |B| > |A|, this means #components(B) < #components(A) Can’t be that all e ∈ B are “inside” connected components of A Some e ∈ B must “go between” connected components of A.

Lemma 1 The empty set is acyclic 2 If A is an acyclic set of edges, and B ⊆ A, then B is also acyclic.

3 If A, B are acyclic, and |B| > |A|, then there is e ∈ B \ A such that A S {e} is acyclic

Sub-lemma: if C is acyclic, then |C| = n − #components(C). Induction

Matroids and The Greedy Algorithm 4/30 Contrapositive: if B cyclic then so is A

Inductively: can extend A by adding |B| − |A| elements from B \ A

Can’t be that all e ∈ B are “inside” connected components of A Some e ∈ B must “go between” connected components of A.

Lemma 1 The empty set is acyclic 2 If A is an acyclic set of edges, and B ⊆ A, then B is also acyclic.

3 If A, B are acyclic, and |B| > |A|, then there is e ∈ B \ A such that A S {e} is acyclic

Sub-lemma: if C is acyclic, then |C| = n − #components(C). Induction When |B| > |A|, this means #components(B) < #components(A)

Matroids and The Greedy Algorithm 4/30 Contrapositive: if B cyclic then so is A

Inductively: can extend A by adding |B| − |A| elements from B \ A

Some e ∈ B must “go between” connected components of A.

Lemma 1 The empty set is acyclic 2 If A is an acyclic set of edges, and B ⊆ A, then B is also acyclic.

3 If A, B are acyclic, and |B| > |A|, then there is e ∈ B \ A such that A S {e} is acyclic

Matroids and The Greedy Algorithm 4/30 Contrapositive: if B cyclic then so is A

Inductively: can extend A by adding |B| − |A| elements from B \ A

Lemma 1 The empty set is acyclic 2 If A is an acyclic set of edges, and B ⊆ A, then B is also acyclic.

3 If A, B are acyclic, and |B| > |A|, then there is e ∈ B \ A such that A S {e} is acyclic

Sub-lemma: if C is acyclic, then |C| = n − #components(C). Induction When |B| > |A|, this means #components(B) < #components(A) Can’t be that all e ∈ B are “inside” connected components of A Some e ∈ B must “go between” connected components of A. Matroids and The Greedy Algorithm 4/30 Lemma 1 The empty set is acyclic 2 If A is an acyclic set of edges, and B ⊆ A, then B is also acyclic. Contrapositive: if B cyclic then so is A 3 If A, B are acyclic, and |B| > |A|, then there is e ∈ B \ A such that A S {e} is acyclic Inductively: can extend A by adding |B| − |A| elements from B \ A

Sub-lemma: if C is acyclic, then |C| = n − #components(C). Induction When |B| > |A|, this means #components(B) < #components(A) Can’t be that all e ∈ B are “inside” connected components of A Some e ∈ B must “go between” connected components of A. Matroids and The Greedy Algorithm 4/30 Assume true for step i − 1, and consider step i S ∗ If ei 6∈ Bi, then Bi−1 {ei} is cyclic. Since Bi−1 ⊆ Bi−1, then ∗ ∗ ∗ ei 6∈ Bi−1 (Property 2). So take Bi = Bi−1. ∗ ∗ If ei ∈ Bi and ei 6∈ Bi−1, build Bi by repeatedly extending Bi using ∗ Bi−1 (property 3) S ∗ Recall that Bi = Bi−1 {ei} agrees with Bi−1 on e1, . . . , ei−1. ∗ ∗ S Bi = Bi−1 {ei}\{ek} for some k > i ∗ ∗ Bi has weight no less than Bi−1, so optimal.

Proof

Going back to proving the algorithm correct. Inductive Hypothesis (i) ∗ There is a maximum-weight acyclic forest Bi which “agrees” with the algorithm’s choices on edges e1, . . . , ei.

i.e. if Bi denotes the algorithm’s choice up to iteration i, then ∗ T Bi = Bi {e1, . . . , ei}

Matroids and The Greedy Algorithm 5/30 S ∗ If ei 6∈ Bi, then Bi−1 {ei} is cyclic. Since Bi−1 ⊆ Bi−1, then ∗ ∗ ∗ ei 6∈ Bi−1 (Property 2). So take Bi = Bi−1. ∗ ∗ If ei ∈ Bi and ei 6∈ Bi−1, build Bi by repeatedly extending Bi using ∗ Bi−1 (property 3) S ∗ Recall that Bi = Bi−1 {ei} agrees with Bi−1 on e1, . . . , ei−1. ∗ ∗ S Bi = Bi−1 {ei}\{ek} for some k > i ∗ ∗ Bi has weight no less than Bi−1, so optimal.

Proof

Going back to proving the algorithm correct. Inductive Hypothesis (i) ∗ There is a maximum-weight acyclic forest Bi which “agrees” with the algorithm’s choices on edges e1, . . . , ei.

i.e. if Bi denotes the algorithm’s choice up to iteration i, then ∗ T Bi = Bi {e1, . . . , ei}

Assume true for step i − 1, and consider step i

Matroids and The Greedy Algorithm 5/30 ∗ ∗ If ei ∈ Bi and ei 6∈ Bi−1, build Bi by repeatedly extending Bi using ∗ Bi−1 (property 3) S ∗ Recall that Bi = Bi−1 {ei} agrees with Bi−1 on e1, . . . , ei−1. ∗ ∗ S Bi = Bi−1 {ei}\{ek} for some k > i ∗ ∗ Bi has weight no less than Bi−1, so optimal.

Proof

Going back to proving the algorithm correct. Inductive Hypothesis (i) ∗ There is a maximum-weight acyclic forest Bi which “agrees” with the algorithm’s choices on edges e1, . . . , ei.

i.e. if Bi denotes the algorithm’s choice up to iteration i, then ∗ T Bi = Bi {e1, . . . , ei}

Assume true for step i − 1, and consider step i S ∗ If ei 6∈ Bi, then Bi−1 {ei} is cyclic. Since Bi−1 ⊆ Bi−1, then ∗ ∗ ∗ ei 6∈ Bi−1 (Property 2). So take Bi = Bi−1.

Matroids and The Greedy Algorithm 5/30 Proof

Going back to proving the algorithm correct. Inductive Hypothesis (i) ∗ There is a maximum-weight acyclic forest Bi which “agrees” with the algorithm’s choices on edges e1, . . . , ei.

i.e. if Bi denotes the algorithm’s choice up to iteration i, then ∗ T Bi = Bi {e1, . . . , ei}

Assume true for step i − 1, and consider step i S ∗ If ei 6∈ Bi, then Bi−1 {ei} is cyclic. Since Bi−1 ⊆ Bi−1, then ∗ ∗ ∗ ei 6∈ Bi−1 (Property 2). So take Bi = Bi−1. ∗ ∗ If ei ∈ Bi and ei 6∈ Bi−1, build Bi by repeatedly extending Bi using ∗ Bi−1 (property 3) S ∗ Recall that Bi = Bi−1 {ei} agrees with Bi−1 on e1, . . . , ei−1. ∗ ∗ S Bi = Bi−1 {ei}\{ek} for some k > i ∗ ∗ Bi has weight no less than Bi−1, so optimal.

Matroids and The Greedy Algorithm 5/30 Three conditions above are called the matroid axioms A ∈ I is called an independent set of the matroid. The matroid whose independent sets are acyclic subgraphs is called a graphic matroid Other examples abound!

To prove optimality of the greedy algorithm, all we needed was the following. Matroids A set system M = (X , I) is a matroid if 1 ∅ ∈ I 2 If A ∈ I and B ⊆ A, then B ∈ I (Downward Closure) 3 If A, B ∈ I and |B| > |A|, then ∃ x ∈ B \ A such that A S {x} ∈ I (Exchange Property)

Matroids and The Greedy Algorithm 6/30 The matroid whose independent sets are acyclic subgraphs is called a graphic matroid Other examples abound!

Three conditions above are called the matroid axioms A ∈ I is called an independent set of the matroid.

Matroids and The Greedy Algorithm 6/30 To prove optimality of the greedy algorithm, all we needed was the following. Matroids A set system M = (X , I) is a matroid if 1 ∅ ∈ I 2 If A ∈ I and B ⊆ A, then B ∈ I (Downward Closure) 3 If A, B ∈ I and |B| > |A|, then ∃ x ∈ B \ A such that A S {x} ∈ I (Exchange Property)

Three conditions above are called the matroid axioms A ∈ I is called an independent set of the matroid. The matroid whose independent sets are acyclic subgraphs is called a graphic matroid Other examples abound!

Matroids and The Greedy Algorithm 6/30 Example: Linear Matroid n X is a ﬁnite set of vectors {v1, . . . , vm} ⊆ R S ∈ I iff the vectors in S are linearly independent

Downward closure: If a set of vectors is linearly independent, then every subset of it is also Exchange property: Can always extend a low-dimension independent set S by adding vectors from a higher dimension independent set T

Matroids and The Greedy Algorithm 7/30 Example: Uniform Matroid X is an arbitrary ﬁnite set {1, . . . , n}. S ∈ I iff |S| ≤ k.

Downward closure: If a set S has |S| ≤ k then the same holds for T ⊆ S. Exchange property: If |S| < |T | ≤ k, then there is an element in T \ S, and we can add it to S while preserving independence.

Matroids and The Greedy Algorithm 8/30 Example: Partition Matroid

X is the disjoint union of classes X1,...,Xm

Each class Xj has an upperbound kj. T S ∈ I iff |S Xj| ≤ kj for all j

This is the “disjoint union” of a number of uniform matroids

Matroids and The Greedy Algorithm 9/30 Example: Transversal Matroid Described by a bipartite graph E ⊆ L × R X = L S ∈ I iff there is a bipartite matching which matches S

Downward closure: If we can match S, then we can match T ⊆ S. Exchange property: If |T | > |S| is matchable, then an augmenting path/alternating path extends the matching of S to some x ∈ T \ S.

Matroids and The Greedy Algorithm 10/30 The Greedy Algorithm on Matroids

The Greedy Algorithm

1 B ← ∅ 2 Sort nonnegative elements of X in decreasing order of weight

{1, . . . , n} with w1 ≥ w2, ≥ ... ≥ wn ≥ 0. 3 For i = 1 to n: if B S {i} ∈ I, add i to B.

Theorem The greedy algorithm returns the maximum weight feasible set for every choice of weights if and only if the set system (X , I) is a matroid.

We already saw the “if” direction. We will skip “only if”.

Matroids and The Greedy Algorithm 11/30 To implement this, we need an independence oracle for step 3 A subroutine which checks whether S ∈ I or not. Runs in time O(n log n) + nT , where T is runtime of the independence oracle. For most “natural” matroids, independence oracle is easy to implement efﬁciently Graphic matroid Linear matroid Uniform/partition matroid Transversal matroid

The Greedy Algorithm on Matroids

The Greedy Algorithm

1 B ← ∅ 2 Sort nonnegative elements of X in decreasing order of weight

{1, . . . , n} with w1 ≥ w2, ≥ ... ≥ wn ≥ 0. 3 For i = 1 to n: if B S {i} ∈ I, add i to B.

Matroids and The Greedy Algorithm 12/30 For most “natural” matroids, independence oracle is easy to implement efﬁciently Graphic matroid Linear matroid Uniform/partition matroid Transversal matroid

The Greedy Algorithm on Matroids

The Greedy Algorithm

1 B ← ∅ 2 Sort nonnegative elements of X in decreasing order of weight

{1, . . . , n} with w1 ≥ w2, ≥ ... ≥ wn ≥ 0. 3 For i = 1 to n: if B S {i} ∈ I, add i to B.

To implement this, we need an independence oracle for step 3 A subroutine which checks whether S ∈ I or not. Runs in time O(n log n) + nT , where T is runtime of the independence oracle.

Matroids and The Greedy Algorithm 12/30 The Greedy Algorithm on Matroids

The Greedy Algorithm

1 B ← ∅ 2 Sort nonnegative elements of X in decreasing order of weight

{1, . . . , n} with w1 ≥ w2, ≥ ... ≥ wn ≥ 0. 3 For i = 1 to n: if B S {i} ∈ I, add i to B.

To implement this, we need an independence oracle for step 3 A subroutine which checks whether S ∈ I or not. Runs in time O(n log n) + nT , where T is runtime of the independence oracle. For most “natural” matroids, independence oracle is easy to implement efﬁciently Graphic matroid Linear matroid Uniform/partition matroid Transversal matroid Matroids and The Greedy Algorithm 12/30 Outline

1 Matroids and The Greedy Algorithm

2 Basic Terminology and Properties

3 The Matroid Polytope

4 Matroid Intersection A base of M is a maximal independent set A base of S ⊆ X in M is maximal independent subset of S I.e. a base of the matroid after deleting S. A circuit S ⊆ X is a minimal dependent subset of X

What are these for: Graphic matroid Linear matroid Uniform matroid Partition matroid Transversal matroid

Independent Sets, Bases, and Circuits

Consider a matroid M = (X , I). An independent set is a set A ∈ I.

Basic Terminology and Properties 13/30 A base of S ⊆ X in M is maximal independent subset of S I.e. a base of the matroid after deleting S. A circuit S ⊆ X is a minimal dependent subset of X

What are these for: Graphic matroid Linear matroid Uniform matroid Partition matroid Transversal matroid

Independent Sets, Bases, and Circuits

Consider a matroid M = (X , I). An independent set is a set A ∈ I. A base of M is a maximal independent set

Basic Terminology and Properties 13/30 A circuit S ⊆ X is a minimal dependent subset of X

What are these for: Graphic matroid Linear matroid Uniform matroid Partition matroid Transversal matroid

Independent Sets, Bases, and Circuits

Consider a matroid M = (X , I). An independent set is a set A ∈ I. A base of M is a maximal independent set A base of S ⊆ X in M is maximal independent subset of S I.e. a base of the matroid after deleting S.

Basic Terminology and Properties 13/30 What are these for: Graphic matroid Linear matroid Uniform matroid Partition matroid Transversal matroid

Independent Sets, Bases, and Circuits

Basic Terminology and Properties 13/30 Independent Sets, Bases, and Circuits

What are these for: Graphic matroid Linear matroid Uniform matroid Partition matroid Transversal matroid

Basic Terminology and Properties 13/30 Follows directly from the exchange property.

Rank

Lemma For every S ⊆ X , all bases of S in M have the same cardinality.

Special case of S = X : all bases of M have the same cardinality. Should remind you of vector space dimension

Basic Terminology and Properties 14/30 The following analogue of vector space dimension is well-deﬁned. Rank The Rank of S ⊆ X in M is the size of the maximal independent subsets (i.e. bases) of S. The rank of M is the size of the bases of M. X The function rankM(S) : 2 → N is called the rank function of M. E.g.: Graphic matroid, linear matroid, transversal matroid

Rank

Lemma For every S ⊆ X , all bases of S in M have the same cardinality.

Special case of S = X : all bases of M have the same cardinality. Should remind you of vector space dimension

Follows directly from the exchange property.

Basic Terminology and Properties 14/30 E.g.: Graphic matroid, linear matroid, transversal matroid

Rank

Lemma For every S ⊆ X , all bases of S in M have the same cardinality.

Special case of S = X : all bases of M have the same cardinality. Should remind you of vector space dimension

Follows directly from the exchange property.

Basic Terminology and Properties 14/30 Rank

Lemma For every S ⊆ X , all bases of S in M have the same cardinality.

Special case of S = X : all bases of M have the same cardinality. Should remind you of vector space dimension

Follows directly from the exchange property.

The following analogue of vector space dimension is well-deﬁned. Rank The Rank of S ⊆ X in M is the size of the maximal independent subsets (i.e. bases) of S. The rank of M is the size of the bases of M. X The function rankM(S) : 2 → N is called the rank function of M. E.g.: Graphic matroid, linear matroid, transversal matroid Basic Terminology and Properties 14/30 Observation i ∈ {1, . . . , n} is selected by the greedy algorithm iff i 6∈ span({1, . . . , i − 1})

Span

Span Given S ⊆ X , span(S) = {i ∈ X : rank(S) = rank(S S {i})}

i.e. S itself, plus the elements which would form a circuit if added to a base of S e.g.: Linear matroid, graphic matroid, uniform matroid.

Basic Terminology and Properties 15/30 Span

Span Given S ⊆ X , span(S) = {i ∈ X : rank(S) = rank(S S {i})}

i.e. S itself, plus the elements which would form a circuit if added to a base of S e.g.: Linear matroid, graphic matroid, uniform matroid.

Observation i ∈ {1, . . . , n} is selected by the greedy algorithm iff i 6∈ span({1, . . . , i − 1})

Basic Terminology and Properties 15/30 Restriction: M|B = M\ B, where B is shorthand for X\ B.

Disjoint union: Given M1 = (X1, I2) and M2 = (X2, I2) with T X1 X2 = ∅, we deﬁne [ n [ o M1 ⊕ M2 = (X1 X2, A1 A2 : A1 ∈ I1,A2 ∈ I2 ) Graphic: combining two node-disjoint graphs Contraction: Given B ⊆ X , let M/B = (X 0, I0) with X 0 = X\ B, n [ o I0 = S ⊆ X0 : B S ∈ I i.e. Think of B as always being included Graphic: contract the connected components of B Others: truncation, dual, union...

Operations preserving Matroidness

Given M = (X , I), consider the following operations: Deletion: For B ⊆ X , we deﬁne M\ B = (X 0, I0) with X 0 = X\ B, I0 = S ⊆ X 0 : S ∈ I Graphic: deleting edges from the graph

Basic Terminology and Properties 16/30 Disjoint union: Given M1 = (X1, I2) and M2 = (X2, I2) with T X1 X2 = ∅, we deﬁne [ n [ o M1 ⊕ M2 = (X1 X2, A1 A2 : A1 ∈ I1,A2 ∈ I2 ) Graphic: combining two node-disjoint graphs Contraction: Given B ⊆ X , let M/B = (X 0, I0) with X 0 = X\ B, n [ o I0 = S ⊆ X0 : B S ∈ I i.e. Think of B as always being included Graphic: contract the connected components of B Others: truncation, dual, union...

Operations preserving Matroidness

Given M = (X , I), consider the following operations: Deletion: For B ⊆ X , we deﬁne M\ B = (X 0, I0) with X 0 = X\ B, I0 = S ⊆ X 0 : S ∈ I Graphic: deleting edges from the graph Restriction: M|B = M\ B, where B is shorthand for X\ B.

Basic Terminology and Properties 16/30 Contraction: Given B ⊆ X , let M/B = (X 0, I0) with X 0 = X\ B, n [ o I0 = S ⊆ X0 : B S ∈ I i.e. Think of B as always being included Graphic: contract the connected components of B Others: truncation, dual, union...

Operations preserving Matroidness

Disjoint union: Given M1 = (X1, I2) and M2 = (X2, I2) with T X1 X2 = ∅, we deﬁne [ n [ o M1 ⊕ M2 = (X1 X2, A1 A2 : A1 ∈ I1,A2 ∈ I2 ) Graphic: combining two node-disjoint graphs

Basic Terminology and Properties 16/30 Others: truncation, dual, union...

Operations preserving Matroidness

Basic Terminology and Properties 16/30 Operations preserving Matroidness

Basic Terminology and Properties 16/30 Operations preserving set convexity are analogous to operations preserving matroid structure Arguably, matroids and submodular functions are discrete analogues of convex sets and convex functions, respectively. Less exhaustive

Matroids as an Algebra of Tractable Discrete Problems

Optimization over matroids is “easy”, in the same way that optimization over convex sets is “easy”

Basic Terminology and Properties 17/30 Arguably, matroids and submodular functions are discrete analogues of convex sets and convex functions, respectively. Less exhaustive

Matroids as an Algebra of Tractable Discrete Problems

Optimization over matroids is “easy”, in the same way that optimization over convex sets is “easy” Operations preserving set convexity are analogous to operations preserving matroid structure

Basic Terminology and Properties 17/30 Matroids as an Algebra of Tractable Discrete Problems

Optimization over matroids is “easy”, in the same way that optimization over convex sets is “easy” Operations preserving set convexity are analogous to operations preserving matroid structure Arguably, matroids and submodular functions are discrete analogues of convex sets and convex functions, respectively. Less exhaustive

Basic Terminology and Properties 17/30 Outline

1 Matroids and The Greedy Algorithm

2 Basic Terminology and Properties

3 The Matroid Polytope

4 Matroid Intersection For M = (X , I), the convex hull of independent sets can be written as a polytope in a natural way The polytope is “solvable”, and admits a polytime separation oracle This perspective will be crucial for more advanced applications of matroids Optimization of linear functions over matroid intersections Optimization of submodular functions over matroids ...

Viewing Matroids Polyhedrally

As is often the case with tractable discrete problems, we can view their feasible set as a polyhedron

The Matroid Polytope 18/30 This perspective will be crucial for more advanced applications of matroids Optimization of linear functions over matroid intersections Optimization of submodular functions over matroids ...

Viewing Matroids Polyhedrally

The Matroid Polytope 18/30 Viewing Matroids Polyhedrally

As is often the case with tractable discrete problems, we can view their feasible set as a polyhedron For M = (X , I), the convex hull of independent sets can be written as a polytope in a natural way The polytope is “solvable”, and admits a polytime separation oracle This perspective will be crucial for more advanced applications of matroids Optimization of linear functions over matroid intersections Optimization of submodular functions over matroids ...

The Matroid Polytope 18/30 The 0-1 indicator vector xI for independent set I ∈ I is in the above polytope In fact, we will show that P(M) is precisely the convex hull of independent sets I Note: polytope has 2|X | constraints.

The Matroid Polytope

Polytope P(M) for M = (X , I) P xi ≤ rankM(S), for S ⊆ X . i∈S xi ≥ 0, for i ∈ X .

Assigns a variable xi to every element i of the ground set Each feasible x is a fractional subset of X

0 ≤ xi ≤ 1 since the rank of a singleton is at most 1.

The Matroid Polytope 19/30 In fact, we will show that P(M) is precisely the convex hull of independent sets I Note: polytope has 2|X | constraints.

The Matroid Polytope

Polytope P(M) for M = (X , I) P xi ≤ rankM(S), for S ⊆ X . i∈S xi ≥ 0, for i ∈ X .

Assigns a variable xi to every element i of the ground set Each feasible x is a fractional subset of X

0 ≤ xi ≤ 1 since the rank of a singleton is at most 1.

The 0-1 indicator vector xI for independent set I ∈ I is in the above polytope

The Matroid Polytope 19/30 Note: polytope has 2|X | constraints.

The Matroid Polytope

Polytope P(M) for M = (X , I) P xi ≤ rankM(S), for S ⊆ X . i∈S xi ≥ 0, for i ∈ X .

Assigns a variable xi to every element i of the ground set Each feasible x is a fractional subset of X

0 ≤ xi ≤ 1 since the rank of a singleton is at most 1.

The 0-1 indicator vector xI for independent set I ∈ I is in the above polytope In fact, we will show that P(M) is precisely the convex hull of independent sets I

The Matroid Polytope 19/30 The Matroid Polytope

Polytope P(M) for M = (X , I) P xi ≤ rankM(S), for S ⊆ X . i∈S xi ≥ 0, for i ∈ X .

Assigns a variable xi to every element i of the ground set Each feasible x is a fractional subset of X

0 ≤ xi ≤ 1 since the rank of a singleton is at most 1.

The 0-1 indicator vector xI for independent set I ∈ I is in the above polytope In fact, we will show that P(M) is precisely the convex hull of independent sets I Note: polytope has 2|X | constraints.

The Matroid Polytope 19/30 It is clear that P(M) ⊇ convexhull {xI : I ∈ I}

To show that P(M) ⊆ convexhull {xI : I ∈ I}, we will show that every vertex of P(M) equals xI for some I ∈ I. Recall: sufﬁces to show that every linear function wT x is maximized over P(M) at some xI for I ∈ I.

Integrality of the Matroid Polytope

Polytope P(M) for M = (X , I) P xi ≤ rankM(S), for S ⊆ X . i∈S xi ≥ 0, for i ∈ X .

Theorem

P(M) = convexhull {xI : I ∈ I}

The Matroid Polytope 20/30 To show that P(M) ⊆ convexhull {xI : I ∈ I}, we will show that every vertex of P(M) equals xI for some I ∈ I. Recall: sufﬁces to show that every linear function wT x is maximized over P(M) at some xI for I ∈ I.

Integrality of the Matroid Polytope

Polytope P(M) for M = (X , I) P xi ≤ rankM(S), for S ⊆ X . i∈S xi ≥ 0, for i ∈ X .

Theorem

P(M) = convexhull {xI : I ∈ I}

It is clear that P(M) ⊇ convexhull {xI : I ∈ I}

The Matroid Polytope 20/30 Recall: sufﬁces to show that every linear function wT x is maximized over P(M) at some xI for I ∈ I.

Integrality of the Matroid Polytope

Polytope P(M) for M = (X , I) P xi ≤ rankM(S), for S ⊆ X . i∈S xi ≥ 0, for i ∈ X .

Theorem

P(M) = convexhull {xI : I ∈ I}

It is clear that P(M) ⊇ convexhull {xI : I ∈ I}

To show that P(M) ⊆ convexhull {xI : I ∈ I}, we will show that every vertex of P(M) equals xI for some I ∈ I.

The Matroid Polytope 20/30 Integrality of the Matroid Polytope

Polytope P(M) for M = (X , I) P xi ≤ rankM(S), for S ⊆ X . i∈S xi ≥ 0, for i ∈ X .

Theorem

P(M) = convexhull {xI : I ∈ I}

It is clear that P(M) ⊇ convexhull {xI : I ∈ I}

The Matroid Polytope 20/30 We can think of the greedy algorithm as computing the indicator ∗ vector x = xB ∈ P(M) We will show that x∗ maximizes w|x over x ∈ P(M).

Recall:The Greedy Algorithm

1 B ← ∅ 2 Sort nonnegative elements of X in decreasing order of weight

{1, . . . , n} with w1 ≥ w2, ≥ ... ≥ wn ≥ 0. 3 For i = 1 to n: if B S {i} ∈ I, add i to B.

Theorem The greedy algorithm returns the maximum weight set for every choice of weights if and only if the set system (X , I) is a matroid.

The Matroid Polytope 21/30 Recall:The Greedy Algorithm

1 B ← ∅ 2 Sort nonnegative elements of X in decreasing order of weight

{1, . . . , n} with w1 ≥ w2, ≥ ... ≥ wn ≥ 0. 3 For i = 1 to n: if B S {i} ∈ I, add i to B.

Theorem The greedy algorithm returns the maximum weight set for every choice of weights if and only if the set system (X , I) is a matroid.

We can think of the greedy algorithm as computing the indicator ∗ vector x = xB ∈ P(M) We will show that x∗ maximizes w|x over x ∈ P(M).

The Matroid Polytope 21/30 n X (wi − wi+1)x(1 : i) i=1 n X ≤ (wi − wi+1)rank(1 : i) i=1 n X = wi(rank[1 : i] − rank[1 : i − 1]) i=1

Recall: Observation i ∈ {1, . . . , n} selected by greedy algorithm iff i 6∈ span({1, . . . , i − 1}) i.e. if rank[1 : i] − rank[1 : i − 1] = 1.

The Matroid Polytope 22/30 n X (wi − wi+1)x(1 : i) i=1 n X ≤ (wi − wi+1)rank(1 : i) i=1 n X = wi(rank[1 : i] − rank[1 : i − 1]) i=1

Consider an arbitrary x ∈ P(M) n X X wixi ≤ wixi = i∈X i=1

Recall: Observation i ∈ {1, . . . , n} selected by greedy algorithm iff i 6∈ span({1, . . . , i − 1}) i.e. if rank[1 : i] − rank[1 : i − 1] = 1.

∗ Therefore, xi = rank[1 : i] − rank[1 : i − 1] for nonneg-weight i, ∗ and xi = 0 for negative-weight i. n n X ∗ X ∗ X wixi = wixi = wi(rank[1 : i] − rank[1 : i − 1]) i∈X i=1 i=1

The Matroid Polytope 22/30 n X (wi − wi+1)x(1 : i) i=1 n X ≤ (wi − wi+1)rank(1 : i) i=1 n X = wi(rank[1 : i] − rank[1 : i − 1]) i=1

Recall: Observation i ∈ {1, . . . , n} selected by greedy algorithm iff i 6∈ span({1, . . . , i − 1}) i.e. if rank[1 : i] − rank[1 : i − 1] = 1.

The Matroid Polytope 22/30 n X ≤ (wi − wi+1)rank(1 : i) i=1 n X = wi(rank[1 : i] − rank[1 : i − 1]) i=1

Recall: Observation i ∈ {1, . . . , n} selected by greedy algorithm iff i 6∈ span({1, . . . , i − 1}) i.e. if rank[1 : i] − rank[1 : i − 1] = 1.

The Matroid Polytope 22/30 n X = wi(rank[1 : i] − rank[1 : i − 1]) i=1

Recall: Observation i ∈ {1, . . . , n} selected by greedy algorithm iff i 6∈ span({1, . . . , i − 1}) i.e. if rank[1 : i] − rank[1 : i − 1] = 1.

The Matroid Polytope 22/30 Recall: Observation i ∈ {1, . . . , n} selected by greedy algorithm iff i 6∈ span({1, . . . , i − 1}) i.e. if rank[1 : i] − rank[1 : i − 1] = 1.

∗ Therefore, xi = rank[1 : i] − rank[1 : i − 1] for nonneg-weight i, ∗ and xi = 0 for negative-weight i. n n X ∗ X ∗ X wixi = wixi = wi(rank[1 : i] − rank[1 : i − 1]) i∈X i=1 i=1 Consider an arbitrary x ∈ P(M) n n X X X wixi ≤ wixi = (wi − wi+1)x(1 : i) i∈X i=1 i=1 n X ≤ (wi − wi+1)rank(1 : i) i=1 n X = wi(rank[1 : i] − rank[1 : i − 1]) i=1 The Matroid Polytope 22/30 Polytope Pbase(M) for M = (X , I) P xi ≤ rankM(S), for S ⊆ X . i∈S P xi = rank(M) i∈X xi ≥ 0, for i ∈ X .

The 0-1 indicator vector xB for a base B of M is in above polytope

In fact, we will show that Pbase(M) is precisely the convex hull of bases of M

The Matroid Base Polytope

The matroid polytope is the convex hull of independent sets Graphic: convex hull of forests What if we wish to consider only “full-rank” sets? Graphic: spanning trees

The Matroid Polytope 23/30 In fact, we will show that Pbase(M) is precisely the convex hull of bases of M

The Matroid Base Polytope

The matroid polytope is the convex hull of independent sets Graphic: convex hull of forests What if we wish to consider only “full-rank” sets? Graphic: spanning trees

Polytope Pbase(M) for M = (X , I) P xi ≤ rankM(S), for S ⊆ X . i∈S P xi = rank(M) i∈X xi ≥ 0, for i ∈ X .

The 0-1 indicator vector xB for a base B of M is in above polytope

The Matroid Polytope 23/30 The Matroid Base Polytope

The matroid polytope is the convex hull of independent sets Graphic: convex hull of forests What if we wish to consider only “full-rank” sets? Graphic: spanning trees

Polytope Pbase(M) for M = (X , I) P xi ≤ rankM(S), for S ⊆ X . i∈S P xi = rank(M) i∈X xi ≥ 0, for i ∈ X .

The 0-1 indicator vector xB for a base B of M is in above polytope

In fact, we will show that Pbase(M) is precisely the convex hull of bases of M

The Matroid Polytope 23/30 As before, one direction is obvious: Pbase(M) ⊇ convexhull {xB : B is a base of M}

For the other direction, take x ∈ Pbase(M) Since x ∈ P(M), x is a convex combination of independent sets I1,...,Ik of M.

Since ||x||1 = rank(M), and ||xI` ||1 ≤ rank(M) for all `, it must

be that ||xI1 ||1 = ||xI2 ||1 = ... = ||xIk ||1 = rank(M)

Polytope Pbase(M) for M = (X , I) P xi ≤ rankM(S), for S ⊆ X . i∈S P xi = rank(M) i∈X xi ≥ 0, for i ∈ X .

Theorem

Pbase(M) = convexhull {xB : B is a base of M}

The Matroid Polytope 24/30 For the other direction, take x ∈ Pbase(M) Since x ∈ P(M), x is a convex combination of independent sets I1,...,Ik of M.

Since ||x||1 = rank(M), and ||xI` ||1 ≤ rank(M) for all `, it must

be that ||xI1 ||1 = ||xI2 ||1 = ... = ||xIk ||1 = rank(M)

Polytope Pbase(M) for M = (X , I) P xi ≤ rankM(S), for S ⊆ X . i∈S P xi = rank(M) i∈X xi ≥ 0, for i ∈ X .

Theorem

Pbase(M) = convexhull {xB : B is a base of M}

As before, one direction is obvious: Pbase(M) ⊇ convexhull {xB : B is a base of M}

The Matroid Polytope 24/30 Since x ∈ P(M), x is a convex combination of independent sets I1,...,Ik of M.

Since ||x||1 = rank(M), and ||xI` ||1 ≤ rank(M) for all `, it must

be that ||xI1 ||1 = ||xI2 ||1 = ... = ||xIk ||1 = rank(M)

Polytope Pbase(M) for M = (X , I) P xi ≤ rankM(S), for S ⊆ X . i∈S P xi = rank(M) i∈X xi ≥ 0, for i ∈ X .

Theorem

Pbase(M) = convexhull {xB : B is a base of M}

As before, one direction is obvious: Pbase(M) ⊇ convexhull {xB : B is a base of M}

For the other direction, take x ∈ Pbase(M)

The Matroid Polytope 24/30 Since ||x||1 = rank(M), and ||xI` ||1 ≤ rank(M) for all `, it must

be that ||xI1 ||1 = ||xI2 ||1 = ... = ||xIk ||1 = rank(M)

Polytope Pbase(M) for M = (X , I) P xi ≤ rankM(S), for S ⊆ X . i∈S P xi = rank(M) i∈X xi ≥ 0, for i ∈ X .

Theorem

Pbase(M) = convexhull {xB : B is a base of M}

As before, one direction is obvious: Pbase(M) ⊇ convexhull {xB : B is a base of M}

For the other direction, take x ∈ Pbase(M) Since x ∈ P(M), x is a convex combination of independent sets I1,...,Ik of M.

The Matroid Polytope 24/30 Polytope Pbase(M) for M = (X , I) P xi ≤ rankM(S), for S ⊆ X . i∈S P xi = rank(M) i∈X xi ≥ 0, for i ∈ X .

Theorem

Pbase(M) = convexhull {xB : B is a base of M}

As before, one direction is obvious: Pbase(M) ⊇ convexhull {xB : B is a base of M}

For the other direction, take x ∈ Pbase(M) Since x ∈ P(M), x is a convex combination of independent sets I1,...,Ik of M.

Since ||x||1 = rank(M), and ||xI` ||1 ≤ rank(M) for all `, it must

be that ||xI1 ||1 = ||xI2 ||1 = ... = ||xIk ||1 = rank(M)

The Matroid Polytope 24/30 Therefore, by equivalence of separation and optimization, can also implement a separation oracle for P(M) in poly(n, T ) time. A more direct proof: reduces to submodular function minimization

rankM is a submodular set function.

Solvability of Matroid Polytopes

Polytope P(M) for M = (X , I) P xi ≤ rankM(S), for S ⊆ X . i∈S xi ≥ 0, for i ∈ X .

The Matroid Polytope 25/30 A more direct proof: reduces to submodular function minimization

rankM is a submodular set function.

Solvability of Matroid Polytopes

Polytope P(M) for M = (X , I) P xi ≤ rankM(S), for S ⊆ X . i∈S xi ≥ 0, for i ∈ X .

When given an independence oracle for M, we can maximize linear functions over P(M) in O(n log n + nT ) time, where T is runtime of independence oracle By integrality, same as ﬁnding max-weight independent set of M. Therefore, by equivalence of separation and optimization, can also implement a separation oracle for P(M) in poly(n, T ) time.

The Matroid Polytope 25/30 Solvability of Matroid Polytopes

Polytope P(M) for M = (X , I) P xi ≤ rankM(S), for S ⊆ X . i∈S xi ≥ 0, for i ∈ X .

rankM is a submodular set function.

The Matroid Polytope 25/30 Outline

1 Matroids and The Greedy Algorithm

2 Basic Terminology and Properties

3 The Matroid Polytope

4 Matroid Intersection Matroid Intersection

Given matroids M1 = (X , I1) and M2 = (X , I2) on the same ground T T set, we deﬁne the set system M1 M2 = (X , I1 I2).

i.e. a set is feasible if it is independent in both matroids In general, does not produce a matroid Nevertheless, it will turn out that maximizing linear functions over a matroid intersection is tractable However, maximizing linear functions over the intersection of 3 or more matroids is NP-hard

Matroid Intersection

Optimization of linear functions over matroids is tractable Matroid operations provide an algebra for constructing new matroids from old We will look at one operation on matroids which does not produce a matroid, but nevertheless produces a solvable problem.

Matroid Intersection 26/30 i.e. a set is feasible if it is independent in both matroids In general, does not produce a matroid Nevertheless, it will turn out that maximizing linear functions over a matroid intersection is tractable However, maximizing linear functions over the intersection of 3 or more matroids is NP-hard

Matroid Intersection

Given matroids M1 = (X , I1) and M2 = (X , I2) on the same ground T T set, we deﬁne the set system M1 M2 = (X , I1 I2).

Matroid Intersection 26/30 In general, does not produce a matroid Nevertheless, it will turn out that maximizing linear functions over a matroid intersection is tractable However, maximizing linear functions over the intersection of 3 or more matroids is NP-hard

Matroid Intersection

Given matroids M1 = (X , I1) and M2 = (X , I2) on the same ground T T set, we deﬁne the set system M1 M2 = (X , I1 I2).

i.e. a set is feasible if it is independent in both matroids

Matroid Intersection 26/30 Nevertheless, it will turn out that maximizing linear functions over a matroid intersection is tractable However, maximizing linear functions over the intersection of 3 or more matroids is NP-hard

Matroid Intersection

Given matroids M1 = (X , I1) and M2 = (X , I2) on the same ground T T set, we deﬁne the set system M1 M2 = (X , I1 I2).

i.e. a set is feasible if it is independent in both matroids In general, does not produce a matroid

Matroid Intersection 26/30 However, maximizing linear functions over the intersection of 3 or more matroids is NP-hard

Matroid Intersection

Given matroids M1 = (X , I1) and M2 = (X , I2) on the same ground T T set, we deﬁne the set system M1 M2 = (X , I1 I2).

Matroid Intersection 26/30 Matroid Intersection

Matroid Intersection

Given matroids M1 = (X , I1) and M2 = (X , I2) on the same ground T T set, we deﬁne the set system M1 M2 = (X , I1 I2).

Matroid Intersection 26/30 Arborescence Given a directed graph G, a set of edges is an r-arborescence if it is a tree directed away from the root r.

Others: colorful spanning trees, orientations, . . .

Examples

Bipartite Matching Given a bipartite graph G, a set of edges F is a bipartite matching if and only if each node is incident on at most one edge in F .

Matroid Intersection 27/30 Others: colorful spanning trees, orientations, . . .

Examples

Bipartite Matching Given a bipartite graph G, a set of edges F is a bipartite matching if and only if each node is incident on at most one edge in F .

Arborescence Given a directed graph G, a set of edges is an r-arborescence if it is a tree directed away from the root r.

Matroid Intersection 27/30 Examples

Bipartite Matching Given a bipartite graph G, a set of edges F is a bipartite matching if and only if each node is incident on at most one edge in F .

Arborescence Given a directed graph G, a set of edges is an r-arborescence if it is a tree directed away from the root r.

Others: colorful spanning trees, orientations, . . .

Matroid Intersection 27/30 One direction is obvious: T T P(M1) P(M2) ⊇ convexhull {xI : I ∈ I1 I2} The other direction is not so obvious T It is conceivable that P(M1) P(M2) has fractional vertices Nevertheless, it is true but hard to prove . . .

The Matroid Intersection Polytope

Matroid Intersection

Given matroids M1 = (X , I1) and M2 = (X , I2) on the same ground T T set, we deﬁne the set system M1 M2 = (X , I1 I2). T Optimizing a modular function over M1 M2 is equivalent to T optimizing a linear function over convexhull {xI : I ∈ I1 I2}. As it turns out, this is a solvable polytope. Theorem T T P(M1) P(M2) = convexhull {xI : I ∈ I1 I2}

Matroid Intersection 28/30 The other direction is not so obvious T It is conceivable that P(M1) P(M2) has fractional vertices Nevertheless, it is true but hard to prove . . .

The Matroid Intersection Polytope

Matroid Intersection

One direction is obvious: T T P(M1) P(M2) ⊇ convexhull {xI : I ∈ I1 I2}

Matroid Intersection 28/30 T It is conceivable that P(M1) P(M2) has fractional vertices Nevertheless, it is true but hard to prove . . .

The Matroid Intersection Polytope

Matroid Intersection

One direction is obvious: T T P(M1) P(M2) ⊇ convexhull {xI : I ∈ I1 I2} The other direction is not so obvious

Matroid Intersection 28/30 Nevertheless, it is true but hard to prove . . .

The Matroid Intersection Polytope

Matroid Intersection

One direction is obvious: T T P(M1) P(M2) ⊇ convexhull {xI : I ∈ I1 I2} The other direction is not so obvious T It is conceivable that P(M1) P(M2) has fractional vertices

Matroid Intersection 28/30 The Matroid Intersection Polytope

Matroid Intersection

One direction is obvious: T T P(M1) P(M2) ⊇ convexhull {xI : I ∈ I1 I2} The other direction is not so obvious T It is conceivable that P(M1) P(M2) has fractional vertices Nevertheless, it is true but hard to prove . . .

Matroid Intersection 28/30 Theorem

Given independence oracles to both matroids M1 and M2, there is an T algorithm for ﬁnding the maximum weight set in M1 M2 which runs in poly(n) time.

Proof: Using equivalence of separation and optimization, and the fact that all coefﬁcients in the LP have poly(n) bits.

Optimization over Matroid Intersections

T Optimization over Matroid Intersection M1 M2 P maximize i∈X wixi subject to P xi ≤ rankM1 (S), for S ⊆ X . i∈S P xi ≤ rankM2 (S), for S ⊆ X . i∈S xi ≥ 0, for i ∈ X .

Matroid Intersection 29/30 Proof: Using equivalence of separation and optimization, and the fact that all coefﬁcients in the LP have poly(n) bits.

Optimization over Matroid Intersections

T Optimization over Matroid Intersection M1 M2 P maximize i∈X wixi subject to P xi ≤ rankM1 (S), for S ⊆ X . i∈S P xi ≤ rankM2 (S), for S ⊆ X . i∈S xi ≥ 0, for i ∈ X .

Theorem

Given independence oracles to both matroids M1 and M2, there is an T algorithm for ﬁnding the maximum weight set in M1 M2 which runs in poly(n) time.

Matroid Intersection 29/30 Optimization over Matroid Intersections

T Optimization over Matroid Intersection M1 M2 P maximize i∈X wixi subject to P xi ≤ rankM1 (S), for S ⊆ X . i∈S P xi ≤ rankM2 (S), for S ⊆ X . i∈S xi ≥ 0, for i ∈ X .

Theorem

Given independence oracles to both matroids M1 and M2, there is an T algorithm for ﬁnding the maximum weight set in M1 M2 which runs in poly(n) time.

Proof: Using equivalence of separation and optimization, and the fact that all coefﬁcients in the LP have poly(n) bits.

Matroid Intersection 29/30 NP-hardness of 3-way Matroid Intersection

By a reduction from Hamiltonian Path in directed graphs

Matroid Intersection 30/30