Calculating Rate Regions for Network Coding and Distributed Storage Via Polyhedral Projection and Representable Matroid Enumeration

Calculating rate regions for network coding and distributed storage via polyhedral projection and representable matroid enumeration

John MacLaren Walsh Department of Electrical and Computer Engineering Drexel University Philadelphia, PA [email protected]

Thanks to NSF CCF-1016588, NSF CCF-1053702, & AFOSR FA9550-12-1-0086.

1 Entropic Vectors and Polyhedral Computation (Development Team)

Jayant Apte Congduan Li Daniel Venutolo efﬁcient parallel rate regions & computing non-Shannon inﬁnite precision rate delay tradeoffs inequalities polyhedral computation

Prof. Steven Weber Drexel University 2 Outline 1. Bounding Rate Regions in Principle/Theory (a) Distributed Storage/Network Coding Regions & Entropic Vectors (b) Outer Bounds: Shannon/Non-Shannon (c) Inner Bounds: Linear Polymatroids/ Representable Matroids 2. Calculating Polyhedral Rate Regions: Why is na¨ıve? " (a) Di↵erence Between Descriptions (b) Projecting Polyhedra (c) Exploiting Symmetries 3. Linear Polymatroid Enumeration for Rate Regions (a) Review of Non-isomorphic Matroid Enumeration (b) Non-isomorphic Representable Matroid Enumeration (c) Non-isomorphic Extremal Representable Matroid Enumeration (d) Network Constrained Linear Polymatroid Enumeration 4. Related Current & Future Work Outline 1. Bounding Rate Regions in Principle/Theory (a) Distributed Storage/Network Coding Regions & Entropic Vectors (b) Outer Bounds: Shannon/Non-Shannon (c) Inner Bounds: Linear Polymatroids/ Representable Matroids 2. Calculating Polyhedral Rate Regions: Why is na¨ıve? " (a) Di↵erence Between Descriptions (b) Projecting Polyhedra (c) Exploiting Symmetries 3. Linear Polymatroid Enumeration for Rate Regions (a) Review of Non-isomorphic Matroid Enumeration (b) Non-isomorphic Representable Matroid Enumeration (c) Non-isomorphic Extremal Representable Matroid Enumeration (d) Network Constrained Linear Polymatroid Enumeration 4. Related Current & Future Work Networking Coding and Distributed Storage Rate Regions Network Coding Distributed Storage (MDCS DSCSC)

. Out(i) . . . . . In(i) . . . . Ys i Yj H(Xj) Zl Rl s t Sj El Dm Fm s (t) . . . . e Ue . . Ul . Vm ...... S Re A B S E E D D S ¯ T S E¯ (Roughly) intersect N⇤ with (Roughly) intersect N⇤ with

and project onto !s,Re and project onto H(Xi),Rl { } ¯ Substitute inner/outer bounds for N⇤ to get inner/outer bounds for rate region

5 ¯ Region of Entropic Vectors N⇤ – What is it?

1. X =(X1,...,XN ) N discrete RVs

2. every subset X =(Xi,i ) H(XY ) A 2A A✓ 1,...,N [N] has joint entropy { }⌘ h(X ). A H(Y ) 2N 1 3. h =(h(X ) [N]) R en- A |A ✓ 2 tropic vector Example: for N =3, h = • H(X) (h1,h2,h3,h12,h13,h23,h123). 2N 1 REV for N =2: 4. a ho R is entropic if joint PMF 2 9 pX s.t. h(pX )=ho. H(X) H(XY )  5. Region of entropic vectors =⇤ H(Y ) H(XY ) N  6. Closure ¯⇤ is a convex cone [1]. H(XY ) H(X)+H(Y ) N 

¯⇤ is an unknown non-polyhedral convex cone for N 4. N determining capacity regions of all networks under network coding complete ¯ , characterization of N⇤ [2] Outline 1. Bounding Rate Regions in Principle/Theory (a) Distributed Storage/Network Coding Regions & Entropic Vectors (b) Outer Bounds: Shannon/Non-Shannon (c) Inner Bounds: Linear Polymatroids/ Representable Matroids 2. Calculating Polyhedral Rate Regions: Why is na¨ıve? " (a) Di↵erence Between Descriptions (b) Projecting Polyhedra (c) Exploiting Symmetries 3. Linear Polymatroid Enumeration for Rate Regions (a) Review of Non-isomorphic Matroid Enumeration (b) Non-isomorphic Representable Matroid Enumeration (c) Non-isomorphic Extremal Representable Matroid Enumeration (d) Network Constrained Linear Polymatroid Enumeration 4. Related Current & Future Work ¯ Bounding the Region of Entropic Vectors N⇤ from the Outside Shannon Outer Bound: . = region of poly- • N matroid rank functions = entropy is submodular: I(X ; X X ) 0 , , A B| C 8A B C ¯ ¯ 2 = 2⇤, 3 = 3⇤. = ¯⇤ ,N 4 ¯⇤ non-polyhedral convex cone N 6 N N Non-Shannon Outer Bounds:[3, 4, 5, 6, 7, 8, 9] • Yeung & Zhang, Dougherty & Freiling & Zeger, Matus Start with 4 unconstr. r.v.s add rv. obeying distr. match & Markov. cond. Intersect for N 5 w/ Markov & distr. match N Project back to orig. 4 unconstr. vars.

N Shannon Outer Bound obtain new information inequalities this way! N Non-Shannon Outer Bound Z¯ Region of Entropic Vectors overall: Shannon linear eq./ineq. project N⇤ ! ! H(XY ) Subspace Ranks Bound H(XY ) SN T H(Y ) q GF(q)-Representable Matroid Constraints Projection MN Bound H(Y ) H(Y ) 4 binary entropic vectors H(X) conv(4) convex hull H(X) H(X) 8 Outline 1. Bounding Rate Regions in Principle/Theory (a) Distributed Storage/Network Coding Regions & Entropic Vectors (b) Outer Bounds: Shannon/Non-Shannon (c) Inner Bounds: Linear Polymatroids/ Representable Matroids 2. Calculating Polyhedral Rate Regions: Why is na¨ıve? " (a) Di↵erence Between Descriptions (b) Projecting Polyhedra (c) Exploiting Symmetries 3. Linear Polymatroid Enumeration for Rate Regions (a) Review of Non-isomorphic Matroid Enumeration (b) Non-isomorphic Representable Matroid Enumeration (c) Non-isomorphic Extremal Representable Matroid Enumeration (d) Network Constrained Linear Polymatroid Enumeration 4. Related Current & Future Work ¯ Bounding N⇤ from the Inside, 1: Representable Matroids Conceptual Inner Bound Creation Process 2N 1 Matroids: r N Z ,r( ) all non-isomorphic matroids 2 \ A |A| w/ ground set size N All non-isomorphic matroids for N 9 [10] ’08 •  2N 1 Minor Exclusion: GF (q)-Representable Matroid: r N Z Purge any w/ U 2 , 4 . M N 2 \ s.t. A GF (q) ⇥ s.t. r( ) = rank(A:, ) 9 2 A A Add isomorphs repr. matroid = scaled EV!: u (GF (q)M ) • ⇠U (permutations) to list by mapping elements in ground X = uA h = r( ) log2 q set to network variables ) A A Key: representability no forbidden minors: Constraint Exclusion: • , Purge vectors not obeying – complete small list known for q 2, 3, 4 2{ } network constraints from list [11, 12, 13, 14, 15] eg.:GF (2) repr. no , Projection: Project rays onto U(2, 4) minor (Tutte 1958) capacity/rate variables, & q remove non-extremal rays ¯⇤ bound: conic hull of GF (q)-repr. ma- • N N Result: extreme ray troids. representation of rate region polyhedral inner bound

10 ¯ Bounding N⇤ from the Inside, 2: Inner Bounds from Subspace Ranks 2N 1 Subspace Bounds: r N Z projec- 2 \ tions of representable matroids, N 0 N,parti- N tion 1,...,N0 = , = n = k { } n=1 Gn Gn \Gk ; 6 Build inner bound for ¯ : S N ( N⇤ r( ) = rank([A:, n n ]) (1) S A G | 2A 1. Obtain q using, e.g., N 0 r( ) =scaledEV!=linear polymatroid method from previous slide, · 2. project (remove all but en- Xn = uA:, n h = r( ) log2 q G ) A A tropies where each element in N : conic hull of all subspace ranks S Xn appears together) : Ingleton’s [16, 17, 18] S4 4\

H(XY ) H(XY ) H(Y ) I(X ; X )+I(X ; X X )+I(X ; X X ) I(X ; X ) 0 1 2 3 4| 1 3 4| 2 3 4 Constraints Projection H(Y ) H(Y )

H(X) recently characterized by DFZ + Kinser [19, H(X) H(X) S5 20] unknown for N 6, but can inner bounded sound familiar??? SN by projecting q (see right) Shannon lin. project N ! ! 11 T Outline 1. Bounding Rate Regions in Principle/Theory (a) Distributed Storage/Network Coding Regions & Entropic Vectors (b) Outer Bounds: Shannon/Non-Shannon (c) Inner Bounds: Linear Polymatroids/ Representable Matroids 2. Calculating Polyhedral Rate Regions: Why is na¨ıve? " (a) Di↵erence Between Descriptions (b) Projecting Polyhedra (c) Exploiting Symmetries 3. Linear Polymatroid Enumeration for Rate Regions (a) Review of Non-isomorphic Matroid Enumeration (b) Non-isomorphic Representable Matroid Enumeration (c) Non-isomorphic Extremal Representable Matroid Enumeration (d) Network Constrained Linear Polymatroid Enumeration 4. Related Current & Future Work Why this is na¨ıve : Di↵erence Between Descriptions

Inner Bounds Outer Bounds 6 Shannon Outer Bound 5 Binary Inner Bound 10 10 4 5 4 >3.3*10 Inequalities 10 10 Inequalities 4 Extreme rays 3 10 10 Extreme rays 3 2 10 10 2 1 10

10 # in log scale 1 # in log scale 10 0 10 0 3 4 5 10 # of RVs 3 4 5 # of RVs Small # of Extreme Rays Small # of Inequalities HUGE # of Inequalities HUGE # of Extreme Rays

13 Outline 1. Bounding Rate Regions in Principle/Theory (a) Distributed Storage/Network Coding Regions & Entropic Vectors (b) Outer Bounds: Shannon/Non-Shannon (c) Inner Bounds: Linear Polymatroids/ Representable Matroids 2. Calculating Polyhedral Rate Regions: Why is na¨ıve? " (a) Di↵erence Between Descriptions (b) Projecting Polyhedra (c) Exploiting Symmetries 3. Linear Polymatroid Enumeration for Rate Regions (a) Review of Non-isomorphic Matroid Enumeration (b) Non-isomorphic Representable Matroid Enumeration (c) Non-isomorphic Extremal Representable Matroid Enumeration (d) Network Constrained Linear Polymatroid Enumeration 4. Related Current & Future Work Why this is na¨ıve : Projecting Polyhedra – Options[7, 21, 22, 23, 24, 25, 26]

Extreme Ray/Point Fourier Motzkin Benson's Outer Convex Hull Method Projection Elimination Approximation Algorithm Successively only deals with Project Extreme Eliminates works directly Pareto frontier in Rays & Points Variables (slow) in projected projected space inequalities space of N dimen. Redundancy polyhedron Removal (convex hull) N-1 dim. proj. extremal repr. of (elim. 1 var) projected polyhedron N-2 dim. proj. (elim. 1 var) Representation conversion

inequality repr. of projected polyhedron

15 Why this is na¨ıve : Projecting Polyhedra – Signiﬁcance Runtime Comparison of Polyhedral Projection Methods >2hrs Benson 1000 CHM Non−Shannon w/ 3 aux. rvs 100 FM Non−Shannon VE(cdd) w/ 2 aux. rvs 10

1 Non−Shannon .1 w/ 3 aux. rvs

time in seconds Zhang−Yeung Inequality .01 0 1 2 3 4 5 log2(OriginalDimension/ProjectionDimension) Which projection method you use makes a gigantic diﬀerence. Projection Algorithms should work in projected space, use a description in which the polyhedron is small, and focus on the Pareto frontier

16 Outline 1. Bounding Rate Regions in Principle/Theory (a) Distributed Storage/Network Coding Regions & Entropic Vectors (b) Outer Bounds: Shannon/Non-Shannon (c) Inner Bounds: Linear Polymatroids/ Representable Matroids 2. Calculating Polyhedral Rate Regions: Why is na¨ıve? " (a) Di↵erence Between Descriptions (b) Projecting Polyhedra (c) Exploiting Symmetries 3. Linear Polymatroid Enumeration for Rate Regions (a) Review of Non-isomorphic Matroid Enumeration (b) Non-isomorphic Representable Matroid Enumeration (c) Non-isomorphic Extremal Representable Matroid Enumeration (d) Network Constrained Linear Polymatroid Enumeration 4. Related Current & Future Work Why this is na¨ıve : Symmetries

# non−isomorphic / # with isomorphs 0 10 −1 10 −2 10 −3 10 Binary inner bound extreme rays −4 10 Shannon outer bound inequalities

fraction in log scale 3 4 5 6 7 8 Number of RVs A Signiﬁcant Amount of the Combinatorial Explosion == isomorphs: Practical Algorithms MUST EXPLOIT SYMMETRIES

Today: for inner bounds, we can do this by enumerating non-isomorphic matroids together with a list of permutations for which they obey the network constraints 18 Why this is na¨ıve : Network Constrained Enumeration – Motivation Conceptual Inner Bound Creation Process all non-isomorphic matroids Comparison of numbers of various non−isomorphic matroids w/ ground set size N 6 10 All matroids Minor Exclusion: All Matroids r<= 3 Purge any w/ U . 5 2,4 10 Binary matorids Binary r<=3 Add isomorphs 4 Ternary matroids (permutations) to list by 10 Ternary r<=3 mapping elements in ground Scalar Binary w/cons. set to network variables 3 Vec Binary w/cons 10 Constraint Exclusion: Purge vectors not obeying 2 network constraints from list 10

Projection: Project rays onto Number in log scale 1 capacity/rate variables, & 10 remove non-extremal rays

Result: extreme ray 0 representation of rate region 10 polyhedral inner bound 3 4 5 6 7 8 9 10 11 12 Ground set size N rethink what’s possible! 19 Outline 1. Bounding Rate Regions in Principle/Theory (a) Distributed Storage/Network Coding Regions & Entropic Vectors (b) Outer Bounds: Shannon/Non-Shannon (c) Inner Bounds: Linear Polymatroids/ Representable Matroids 2. Calculating Polyhedral Rate Regions: Why is na¨ıve? " (a) Di↵erence Between Descriptions (b) Projecting Polyhedra (c) Exploiting Symmetries 3. Linear Polymatroid Enumeration for Rate Regions (a) Review of Non-isomorphic Matroid Enumeration (b) Non-isomorphic Representable Matroid Enumeration (c) Non-isomorphic Extremal Representable Matroid Enumeration (d) Network Constrained Linear Polymatroid Enumeration 4. Related Current & Future Work Non-isomorphic Matroid Enumeration (Review) For matroids/representable matroids/linear polymatroids • Ever since Crapo & Higgs [27], non-isomorphic matroid enumeration based on • 1,...,N single element extensions:startw/rN :2{ } Z 0, and from it form 1,...,N+1 ! rN+1 :2{ } Z 0 w/ rN+1( )=rN ( ) 1,...,N . ! A A 8A✓{ } r( 1 ) r( 2 ) r( 1, 2 ) r( 3 ) r( 1, 3 ) r( 2, 3 ) r( 1, 2, 3 ) { } { } { } { } { } { } { } r ( )1 1 2 2 · r ( )1 1 2 1 2 2 3 3 · Original [28]: Perform all SEEs of current list, remove all isomorphs, repeat • Now: Make use of modular cuts (collections of ﬂats) and taboo ﬂats to directly list • only non-isomorphic SEEs rather than having to remove some isomorphs at each stage. Royle & Mayhew[10], Matsumoto [29]

21 Outline 1. Bounding Rate Regions in Principle/Theory (a) Distributed Storage/Network Coding Regions & Entropic Vectors (b) Outer Bounds: Shannon/Non-Shannon (c) Inner Bounds: Linear Polymatroids/ Representable Matroids 2. Calculating Polyhedral Rate Regions: Why is na¨ıve? " (a) Di↵erence Between Descriptions (b) Projecting Polyhedra (c) Exploiting Symmetries 3. Linear Polymatroid Enumeration for Rate Regions (a) Review of Non-isomorphic Matroid Enumeration (b) Non-isomorphic Representable Matroid Enumeration (c) Non-isomorphic Extremal Representable Matroid Enumeration (d) Network Constrained Linear Polymatroid Enumeration 4. Related Current & Future Work Directly Enumerating Non-isomorphic F-Representable Matroids

Conceptual Inner Bound Creation Process all non-isomorphic matroids Comparison of numbers of various non−isomorphic matroids w/ ground set size N 6 10 All matroids Minor Exclusion: All Matroids r<= 3 Purge any w/ U . 5 2,4 10 Binary matorids Binary r<=3 Add isomorphs 4 Ternary matroids (permutations) to list by 10 Ternary r<=3 mapping elements in ground Scalar Binary w/cons. set to network variables 3 Vec Binary w/cons 10 Constraint Exclusion: Purge vectors not obeying 2 network constraints from list 10

Projection: Project rays onto Number in log scale 1 capacity/rate variables, & 10 remove non-extremal rays

Result: extreme ray 0 representation of rate region 10 polyhedral inner bound 3 4 5 6 7 8 9 10 11 12 Ground set size N

23 Directly Enumerating Non-isomorphic F-Representable Matroids

Conceptual Inner Bound Creation Process allINTEGRATE non-isomorphic matroids Comparison of numbers of various non−isomorphic matroids 6 w/ ground set size N 10 All matroids Minor Exclusion: 5 All Matroids r<= 3 Purge any w/ U 2 , 4 . 10 Binary matorids Binary r<=3 Add isomorphs 4 Ternary matroids (permutations) to list by 10 Ternary r<=3 mapping elements in ground Scalar Binary w/cons. set to network variables 3 Vec Binary w/cons 10 Constraint Exclusion: Purge vectors not obeying 2 network constraints from list 10

Projection: Project rays onto Number in log scale 1 capacity/rate variables, & 10 remove non-extremal rays Result: extreme ray 0 10 representation of rate region 3 4 5 6 7 8 9 10 11 12 polyhedral inner bound Ground set size N

24 Directly Enumerating Non-isomorphic F-Representable Matroids

list of nonisomorphic binary Lack of binary representability matroids on ground set size k is an inherited characteristic, so no need to extend along for each matroid form single non-binary matroids. element extensions by enumerating modular cuts k= For Binary Matroid Enumeration, w/o taboo flats k+1 ground prune non-binary (red) branches in rank vector set size ordinary nonisomorphic extension N-2 ...... extended rank vector ? ... ? ? N-1 ......

exclude any with U 2 ,4 minors ...... binary representable list of nonisomorphic binary not binary representable extension of binary representable matroids on ground set size k+1 extension of not binary representable

25 Outline 1. Bounding Rate Regions in Principle/Theory (a) Distributed Storage/Network Coding Regions & Entropic Vectors (b) Outer Bounds: Shannon/Non-Shannon (c) Inner Bounds: Linear Polymatroids/ Representable Matroids 2. Calculating Polyhedral Rate Regions: Why is na¨ıve? " (a) Di↵erence Between Descriptions (b) Projecting Polyhedra (c) Exploiting Symmetries 3. Linear Polymatroid Enumeration for Rate Regions (a) Review of Non-isomorphic Matroid Enumeration (b) Non-isomorphic Representable Matroid Enumeration (c) Non-isomorphic Extremal Representable Matroid Enumeration (d) Network Constrained Linear Polymatroid Enumeration 4. Related Current & Future Work Directly Enumerating Extremal Non-isomorphic F-Representable Matroids

27 Directly Enumerating Extremal Non-isomorphic F-Representable Matroids

Conceptual Inner Bound Creation Process allINTEGRATE non-isomorphic matroids Comparison of numbers of various non−isomorphic matroids 6 w/ ground set size N 10 All matroids Minor Exclusion: 5 All Matroids r<= 3 Purge any w/ U 2 , 4 . 10 Binary matorids Binary r<=3 Add isomorphs 4 Ternary matroids (permutations) to list by 10 Ternary r<=3 mapping elements in ground Scalar Binary w/cons. set to network variables 3 Vec Binary w/cons 10 LOOKConstraint AHEAD Exclusion: Purge vectors not obeying 2 network constraints from list 10

28 Directly Enumerating Extremal Non-isomorphic F-Representable Matroids No matroid here! No q-rep. matroid here! Polymatroid Extr. Rays N Matroid Extr. Rays mat N q-rep. matroid q extr. rays N

Problem: some connected all matroids convex extremal [30] ... matroids are not • reachable via a matroid conic extremal connected [31] SEE of a • , connected matroid! UCM F-representable matroid conic extremal F- • , ground representable & connected [30] set size ...... N-2 enumerate connected F-rep. matroids • ) N-1 ...... connected matroids s.t. EVERY S.E.D. is • 9 ...... not connected == class UCM [32] connected (i.e. lack of connectedness is not an inherited not connected • extension of connected characteristic in S.E.E. process) extension of not connected 29 Directly Enumerating Extremal Non-isomorphic F-Representable Matroids

Problem: matroid Solution: ... duality ...

However, M in UCM, dual M ⇤ has UCM UCM* • 8 ) at least one S.E.D. that is connected ground set size [33, 32] ...... N-2 UCM solution: S.E.E. of connected • matroids & their duals, followed by N-1 ...... connectedness check [32] ...... Also M is F repr. M ⇤ F-repr. connected • , Connected req. cuts # of non-iso. not connected • extension of connected matroids by at most 1 asympt. [34] extension of not connected 2

30 Outline 1. Bounding Rate Regions in Principle/Theory (a) Distributed Storage/Network Coding Regions & Entropic Vectors (b) Outer Bounds: Shannon/Non-Shannon (c) Inner Bounds: Linear Polymatroids/ Representable Matroids 2. Calculating Polyhedral Rate Regions: Why is na¨ıve? " (a) Di↵erence Between Descriptions (b) Projecting Polyhedra (c) Exploiting Symmetries 3. Linear Polymatroid Enumeration for Rate Regions (a) Review of Non-isomorphic Matroid Enumeration (b) Non-isomorphic Representable Matroid Enumeration (c) Non-isomorphic Extremal Representable Matroid Enumeration (d) Network Constrained Linear Polymatroid Enumeration 4. Related Current & Future Work Applying Network Constraints & Adding Network Obeying Isomorphs: Constraint Permutation [32]

Projection: Project rays onto Number in log scale 1 capacity/rate variables, & 10 remove non-extremal rays

Result: extreme ray 0 representation of rate region 10 polyhedral inner bound 3 4 5 6 7 8 9 10 11 12 Ground set size N

32 Applying Network Constraints & Adding Network Obeying Isomorphs: Constraint Permutation [32]

Conceptual Inner Bound Creation Process all non-isomorphic matroids Comparison of numbers of various non−isomorphic matroids w/ ground set size N 6 10 All matroids Minor Exclusion: All Matroids r<= 3 Purge any w/ U . 5 2,4 10 Binary matorids Binary r<=3 Add isomorphs INTEGRATE 4 Ternary matroids (permutations) to list by 10 Ternary r<=3 mapping elements in ground Scalar Binary w/cons. set to network variables 3 Vec Binary w/cons 10 Constraint Exclusion: Purge vectors not obeying 2 network constraints from list 10

Projection: Project rays onto Number in log scale 1 capacity/rate variables, & 10 remove non-extremal rays

Result: extreme ray 0 representation of rate region 10 polyhedral inner bound 3 4 5 6 7 8 9 10 11 12 Ground set size N

33 Applying Network Constraints & Adding Network Obeying Isomorphs: Constraint Permutation [32] Adding all N! isomorphs, then ap- h1 = h13 h34 = h1234 h12 = h124 • 1 4 3 2 plying all constraints is inecient! 1 2 3 4 1 2 3 4 Build permutation incrementally: 1 ? 3 ? • 3 4 1 2 (partial permutation) 3 ? 1 ? 3 2 1 4 3 2 1 4 1 ? 2 ? 1 4 2 3 Apply constraints incr. as new 2 ? 1 ? • 1 3 2 4 1 3 2 4 Fail var.s are deﬁned in partial perm. 2 ? 3 ? 2 4 1 3 3 ? 2 ? No need to extend partial permuta- 2 3 1 4 2 3 1 4 • 1 ? 4 ? 2 4 3 1 tions that do not obey a constraint 4 ? 1 ? 2 1 3 4 2 1 3 4 If no extensions, then this non-iso. 2 ? 4 ? • X3 matroid can’t obey cons. 4 ? 2 ? enc dec X1 3 ? 4 ? X1,X2 X4 all var.s deﬁned surviving perm.s enc dec X1,X2 • ! 4 ? 3 ? are cons. obeying isomorphs h1 h2 h12 h3 h13 h23 h123 h4 h14 h24 h124 h34 h134 h234 h1234 11 1 1 1 1 1 1 2 2 2 2 2 2 2 easily generalized to vector codes • Even better: only include matroids which can be extended to obey constraints in the extension process 34 Directly Enumerating Network Constrained Linear Polymatroids [32]

Projection: Project rays onto Number in log scale 1 capacity/rate variables, & 10 remove non-extremal rays

Result: extreme ray 0 representation of rate region 10 polyhedral inner bound 3 4 5 6 7 8 9 10 11 12 Ground set size N

35 Directly Enumerating Network Constrained Linear Polymatroids [32]

Conceptual Inner Bound Creation Process all non-isomorphic matroids Comparison of numbers of various non−isomorphic matroids 6 w/ ground set size N 10 INTEGRATE All matroids Minor Exclusion: 5 All Matroids r<= 3 Purge any w/ U 2 , 4 . 10 Binary matorids Binary r<=3 Add isomorphs 4 Ternary matroids (permutations) to list by 10 Ternary r<=3 mapping elements in ground Scalar Binary w/cons. set to network variables 3 Vec Binary w/cons 10 Constraint Exclusion: Purge vectors not obeying 2 network constraints from list 10

36 Directly Enumerating Network Constrained Linear Polymatroids [32] Purpose: Embed incremental constraint checking & compliant isomorph building • process into non-isomorphic F-representable matroid enumeration process Benefit: Do not extend along those repr. matroids whose extension’s linear • polymatroids could never obey the network constraints Together with each non-isomorphic matroid (on intermediate ground set sizes), • maintain a list of partial matroid to network mappings for that matroid, defining a subset of the RVs in the network, and obeying the constraints set out by the network. Those partial mappings (extensions of previous partial mappings by defining at • most one new variable) that do not obey constraints are discarded. Any non-isomorphic matroid with no partial mappings left is discarded (not • extended along). When a non-isomorphic matroid is SEE’ed, consider every extension of each of its • previous partial maps, by defining at most one new variable to be a subset of unmapped ground set elements including the new element. 37 Conclusions / Related Current & Future Work

Identiﬁed a need for special focus on the polyhedral computations for rate region • calculation. – Exploit symmetries, and the sparser polyhedral description – utilize the best option among projection methods Showed how to construct inner bounds for rate regions by enumerating matroids • – Can extend non-isomorphic matroid enumeration techniques to representable, extremal, and network constrained – Multiple orders of magnitude improvement in complexity over direct, but na¨ıve obvious method Currently using the software developed to calculate capacity regions of all small • MDCS networks. More software under development will be shared. •

38 References

[1] Raymond W. Yeung, Information Theory and Network Coding.Springer,2008.

[2] T. Chan and A. Grant, “Entropy Vectors and Network Codes,” in IEEE International Symposium on Information Theory,June2007.

[3] Raymond W. Yeung, “A Framework for Linear Information Inequalities,” IEEE Transactions on Information Theory,vol.43,no.6,Nov.1997.

[4] Zhen Zhang and Raymond W. Yeung, “On Characterization of Entropy Function via Information Inequalities,” IEEE Transactions on Information Theory,vol.44,no.4,July 1998.

[5] ——, “A Non-Shannon-Type Conditional Inequality of Information Quantities,” IEEE Transactions on Information Theory,vol.43,no.6,Nov.1997.

[6] K. Makarychev, Y. Makarychev, A. Romashchenko, and N. Vereshchagin, “A new class of non-Shannon-type inequalities for entropies,” Communication in Information and Systems, vol. 2, no. 2, pp. 147–166, December 2002.

[7] Weidong Xu, Jia Wang, Jun Sun, “A projection method for derivation of non-Shannon-type information inequalities,” in IEEE International Symposium on Information Theory (ISIT),2008,pp.2116–2120. 39 [8] Frantiˇsek Matúˇs, “Infinitely Many Information Inequalities,” in IEEE International Symposium on Information Theory (ISIT),June2007,pp.41–44. [9] Randall Dougherty, Chris Freiling, Kenneth Zeger, “Non-Shannon Information Inequalities in Four Random Variables,” Apr. 2011, arXiv:1104.3602v1. [Online]. Available: http://arxiv.org/pdf/1104.3602.pdf [10] Dillon Mayhew, Gordon F. Royle, “Matroids with nine elements,” Journal of Combinatorial Theory, Series B,vol.98,no.2,pp.415–431,2008. [11] James Oxley, Matroid Theory, 2nd. Ed. Oxford University Press, 2011. [12] W. T. Tutte, “A homotopy theorem for matroids, I, II.” Trans. American Mathematical Society,vol.88,pp.144–174,1958. [13] R. E. Bixby, “On Reid’s Characterization of the Ternary Matroids,” J. Combin. Theory Ser. B,no.26,pp.174–204,1979. [14] P. D. Seymour, “Matroid Representation over GF(3),” J. Combin. Theory Ser. B,no.26, pp. 159–173, 1979. [15] A. M. H. Gerards, “A short proof of Tutte’s characterization of totally unimodular matroids,” Linear Algebra Appl.,no.114/115,pp.207–212,1989. [16] A. W. Ingleton, “Representation of Matroids,” in Combinatorial Mathematics and its Applications, D. J. A. Welsh, Ed. San Diego: Academic Press, 1971, pp. 149–167. 40 [17] D. Hammer, A. Romashschenko, A. Shen, N. Vereshchagin, “Inequalities for Shannon Entropy and Kolmogorov Complexity,” Journal of Computer and System Sciences, vol. 60, pp. 442–464, 2000. [18] F. Matúˇsand M. Studený, “Conditional Independences among Four Random Variables I,” Combinatorics, Probability and Computing,no.4,pp.269–278,1995. [19] Randall Dougherty, Chris Freiling, Kenneth Zeger, “Linear rank inequalities on five or more variables,” submitted to SIAM J. Discrete Math. arXiv:0910.0284. [20] Ryan Kinser, “New Inequalities for Subspace Arrangements,” New Inequalities for Subspace Arrangements,vol.188,no.1,pp.152–161,Jan.2011. [21] C. Lassez and J.-L. Lassez, “Quantifier elimination for conjunctions of linear constraints via a convex hull algorithm,” in Symbolic and Numerical Computation for Artificial Intelligence, K. Donald and Mundy, Eds. Academic Press, 1993. [22] LászlóCsirmaz, “Using multiobjective optimization to map the entropy region of four random variables,” 2013, submitted to Journal of Global Optimization.[Online]. Available: http://arxiv.org/pdf/1310.4638 [23] H. P. Benson, “An outer approximation algorithm for generating all ecient extreme points in the outcome set of a multiple objective linear program,” J. Global Optim, vol. 13, no. 1, p. 124, 1998. [Online]. Available: http://link.springer.com/article/10.1023/A:1008215702611 41 [24] B. A. Burton, M. Ozlen, “Projective geometry and the outer approximation algorithm for multiobjective linear programming,” June 2010, arxiv 1006.3085. [Online]. Available: http://arxiv.org/pdf/1006.3085v1 [25] M. Ehrgott, A. Löhne, L. Shao, “A dual variant of Bensons outer approximation algorithm for multiple objective linear programming,” J. Glob. Optim,vol.52,pp.757–778,2012. [26] F. Heyde and A. Löhne, “Geometric duality in multiple objective linear programming,” SIAM Journal on Optimization,vol.19,no.2,pp.836–845,2008. [27] H. H. Crapo, “Single-element extensions of matroids,” J. Res. Natl. Bur. Standards, Sec. B: Math. & Math. Phys.,vol.69B,no.1-2,pp.55–65,1965. [28] J. E. Blackburn, H. H. Crapo, and D. A. Higgs, “A catalogue of combinatorial geometries,” Mathematics of Computation,vol.27,no.121,pp.pp.155–166+62+64, 1973. [29] Y. Matsumoto, S. Moriyama, H. Imai, and D. Bremner, “Matroid enumeration for incidence geometry,” Discrete Comput. Geom.,vol.47,no.1,pp.17–43,Jan.2012. [Online]. Available: http://dx.doi.org/10.1007/s00454-011-9388-y [30] Congduan Li, John MacLaren Walsh, Steven Weber, “Matroid bounds on the region of entropic vectors,” in 51th Annual Allerton Conference on Communication, Control and Computing,Oct.2013.[Online].Available: http://www.ece.drexel.edu/walsh/Congduan allerton13.pdf 42 [31] H. Q. Nguyen, “Semimodular functions and combinatorial geometries,” Transactions of the American mathematical society,vol.238,pp.355–383,Apr.1978.

[32] Jayant Apte and Congduan Li and John MacLaren Walsh, “Algorithms for Computing Network Coding Rate Regions via Single Element Extensions of Matroids,” submitted to ISIT 2014.

[33] W. Tutte, “Connectivity in matroids,” Canad. J. Math,vol.18,pp.1301–1324,1966.

[34] “On the asymptotic proportion of connected matroids,” European J. Combin.,vol.32.