Calculating rate regions for network coding and distributed storage via polyhedral projection and representable matroid enumeration John MacLaren Walsh Department of Electrical and Computer Engineering Drexel University Philadelphia, PA [email protected] Thanks to NSF CCF-1016588, NSF CCF-1053702, & AFOSR FA9550-12-1-0086. 1 Entropic Vectors and Polyhedral Computation (Development Team) Jayant Apte Congduan Li Daniel Venutolo efficient parallel rate regions & computing non-Shannon infinite precision rate delay tradeoffs inequalities polyhedral computation Prof. Steven Weber Drexel University 2 Outline 1. Bounding Rate Regions in Principle/Theory (a) Distributed Storage/Network Coding Regions & Entropic Vectors (b) Outer Bounds: Shannon/Non-Shannon (c) Inner Bounds: Linear Polymatroids/ Representable Matroids 2. Calculating Polyhedral Rate Regions: Why is na¨ıve? " (a) Di↵erence Between Descriptions (b) Projecting Polyhedra (c) Exploiting Symmetries 3. Linear Polymatroid Enumeration for Rate Regions (a) Review of Non-isomorphic Matroid Enumeration (b) Non-isomorphic Representable Matroid Enumeration (c) Non-isomorphic Extremal Representable Matroid Enumeration (d) Network Constrained Linear Polymatroid Enumeration 4. Related Current & Future Work Outline 1. Bounding Rate Regions in Principle/Theory (a) Distributed Storage/Network Coding Regions & Entropic Vectors (b) Outer Bounds: Shannon/Non-Shannon (c) Inner Bounds: Linear Polymatroids/ Representable Matroids 2. Calculating Polyhedral Rate Regions: Why is na¨ıve? " (a) Di↵erence Between Descriptions (b) Projecting Polyhedra (c) Exploiting Symmetries 3. Linear Polymatroid Enumeration for Rate Regions (a) Review of Non-isomorphic Matroid Enumeration (b) Non-isomorphic Representable Matroid Enumeration (c) Non-isomorphic Extremal Representable Matroid Enumeration (d) Network Constrained Linear Polymatroid Enumeration 4. Related Current & Future Work Networking Coding and Distributed Storage Rate Regions Network Coding Distributed Storage (MDCS DSCSC) . Out(i) . In(i) . Ys i Yj H(Xj) Zl Rl s t Sj El Dm Fm ωs β(t) . e Ue . Ul . Vm . ⊂ S Re A B ⊂ S × E ⊂ E × D D S ¯ T S E¯ (Roughly) intersect ΓN⇤ with (Roughly) intersect ΓN⇤ with hYs !s,s hY = hYs , hYj ,j = hYj ≥ 2S S 2S s j X2S X2S hU Y =0,s hZ (Y ,j U ) =0,l Out(s)| s 2S l| j 2 l 2E hU U =0,i ( ) h(Y ,j F ) (Z ,l V ) =0,m Out(i)| In(i) 2V\ S[T j 2 m | l 2 m 2D hU Re,e hY >H(Xj ),j e 2E j 2S hY U =0,t hZ Rl,l β(t)| In(t) 2T l 2E and project onto !s,Re and project onto H(Xi),Rl { } ¯ Substitute inner/outer bounds for ΓN⇤ to get inner/outer bounds for rate region 5 ¯ Region of Entropic Vectors ΓN⇤ – What is it? 1. X =(X1,...,XN ) N discrete RVs 2. every subset X =(Xi,i ) H(XY ) A 2A A✓ 1,...,N [N] has joint entropy { }⌘ h(X ). A H(Y ) 2N 1 3. h =(h(X ) [N]) R − en- A |A ✓ 2 tropic vector Example: for N =3, h = • H(X) (h1,h2,h3,h12,h13,h23,h123). 2N 1 REV for N =2: 4. a ho R − is entropic if joint PMF 2 9 pX s.t. h(pX )=ho. H(X) H(XY ) 5. Region of entropic vectors =Γ⇤ H(Y ) H(XY ) N 6. Closure Γ¯⇤ is a convex cone [1]. H(XY ) H(X)+H(Y ) N Γ¯⇤ is an unknown non-polyhedral convex cone for N 4. N ≥ determining capacity regions of all networks under network coding complete ¯ , characterization of ΓN⇤ [2] Outline 1. Bounding Rate Regions in Principle/Theory (a) Distributed Storage/Network Coding Regions & Entropic Vectors (b) Outer Bounds: Shannon/Non-Shannon (c) Inner Bounds: Linear Polymatroids/ Representable Matroids 2. Calculating Polyhedral Rate Regions: Why is na¨ıve? " (a) Di↵erence Between Descriptions (b) Projecting Polyhedra (c) Exploiting Symmetries 3. Linear Polymatroid Enumeration for Rate Regions (a) Review of Non-isomorphic Matroid Enumeration (b) Non-isomorphic Representable Matroid Enumeration (c) Non-isomorphic Extremal Representable Matroid Enumeration (d) Network Constrained Linear Polymatroid Enumeration 4. Related Current & Future Work ¯ Bounding the Region of Entropic Vectors ΓN⇤ from the Outside Shannon Outer Bound: Γ . = region of poly- • N matroid rank functions = entropy is submodular: I(X ; X X ) 0 , , A B| C ≥ 8A B C ¯ ¯ Γ2 = Γ2⇤, Γ3 = Γ3⇤. Γ = Γ¯⇤ ,N 4 Γ¯⇤ non-polyhedral convex cone N 6 N ≥ N Non-Shannon Outer Bounds:[3, 4, 5, 6, 7, 8, 9] • Yeung & Zhang, Dougherty & Freiling & Zeger, Matus Start with 4 unconstr. r.v.s add rv. obeying distr. match & Markov. cond. Intersect Γ for N 5 w/ Markov & distr. match N ≥ Project back to orig. 4 unconstr. vars. ΓN Shannon Outer Bound obtain new information inequalities this way! N Non-Shannon Outer Bound Z¯ Region of Entropic Vectors overall: Shannon linear eq./ineq. project ΓN⇤ ! ! H(XY ) Subspace Ranks Bound H(XY ) SN T H(Y ) q GF(q)-Representable Matroid Constraints Projection MN Bound H(Y ) H(Y ) Φ4 binary entropic vectors H(X) conv(Φ4) convex hull H(X) H(X) 8 Outline 1. Bounding Rate Regions in Principle/Theory (a) Distributed Storage/Network Coding Regions & Entropic Vectors (b) Outer Bounds: Shannon/Non-Shannon (c) Inner Bounds: Linear Polymatroids/ Representable Matroids 2. Calculating Polyhedral Rate Regions: Why is na¨ıve? " (a) Di↵erence Between Descriptions (b) Projecting Polyhedra (c) Exploiting Symmetries 3. Linear Polymatroid Enumeration for Rate Regions (a) Review of Non-isomorphic Matroid Enumeration (b) Non-isomorphic Representable Matroid Enumeration (c) Non-isomorphic Extremal Representable Matroid Enumeration (d) Network Constrained Linear Polymatroid Enumeration 4. Related Current & Future Work ¯ Bounding ΓN⇤ from the Inside, 1: Representable Matroids Conceptual Inner Bound Creation Process 2N 1 Matroids: r ΓN Z − ,r( ) all non-isomorphic matroids 2 \ A |A| w/ ground set size N All non-isomorphic matroids for N 9 [10] ’08 • 2N 1 Minor Exclusion: GF (q)-Representable Matroid: r ΓN Z − Purge any w/ U 2 , 4 . M N 2 \ s.t. A GF (q) ⇥ s.t. r( ) = rank(A:, ) 9 2 A A Add isomorphs repr. matroid = scaled EV!: u (GF (q)M ) • ⇠U (permutations) to list by mapping elements in ground X = uA h = r( ) log2 q set to network variables ) A A Key: representability no forbidden minors: Constraint Exclusion: • , Purge vectors not obeying – complete small list known for q 2, 3, 4 2{ } network constraints from list [11, 12, 13, 14, 15] eg.:GF (2) repr. no , Projection: Project rays onto U(2, 4) minor (Tutte 1958) capacity/rate variables, & q remove non-extremal rays Γ¯⇤ bound: Γ conic hull of GF (q)-repr. ma- • N N Result: extreme ray troids. representation of rate region polyhedral inner bound 10 ¯ Bounding ΓN⇤ from the Inside, 2: Inner Bounds from Subspace Ranks 2N 1 Subspace Bounds: r ΓN Z − projec- 2 \ tions of representable matroids, N 0 N,parti- N ≥ tion 1,...,N0 = , = n = k { } n=1 Gn Gn \Gk ; 6 Build inner bound for Γ¯ : S N ( N⇤ r( ) = rank([A:, n n ]) (1) S A G | 2A 1. Obtain Γq using, e.g., N 0 r( ) =scaledEV!=linear polymatroid method from previous slide, · 2. project (remove all but en- Xn = uA:, n h = r( ) log2 q G ) A A tropies where each element in N : conic hull of all subspace ranks S Xn appears together) : Γ Ingleton’s [16, 17, 18] S4 4\ H(XY ) H(XY ) H(Y ) I(X ; X )+I(X ; X X )+I(X ; X X ) I(X ; X ) 0 1 2 3 4| 1 3 4| 2 − 3 4 ≥ Constraints Projection H(Y ) H(Y ) H(X) recently characterized by DFZ + Kinser [19, H(X) H(X) S5 20] unknown for N 6, but can inner bounded sound familiar??? SN ≥ by projecting Γq (see right) Shannon lin. project N ! ! 11 T Outline 1. Bounding Rate Regions in Principle/Theory (a) Distributed Storage/Network Coding Regions & Entropic Vectors (b) Outer Bounds: Shannon/Non-Shannon (c) Inner Bounds: Linear Polymatroids/ Representable Matroids 2. Calculating Polyhedral Rate Regions: Why is na¨ıve? " (a) Di↵erence Between Descriptions (b) Projecting Polyhedra (c) Exploiting Symmetries 3. Linear Polymatroid Enumeration for Rate Regions (a) Review of Non-isomorphic Matroid Enumeration (b) Non-isomorphic Representable Matroid Enumeration (c) Non-isomorphic Extremal Representable Matroid Enumeration (d) Network Constrained Linear Polymatroid Enumeration 4. Related Current & Future Work Why this is na¨ıve : Di↵erence Between Descriptions Inner Bounds Outer Bounds 6 Shannon Outer Bound 5 Binary Inner Bound 10 10 4 5 4 >3.3*10 Inequalities 10 10 Inequalities 4 Extreme rays 3 10 10 Extreme rays 3 2 10 10 2 1 10 10 # in log scale 1 # in log scale 10 0 10 0 3 4 5 10 # of RVs 3 4 5 # of RVs Small # of Extreme Rays Small # of Inequalities HUGE # of Inequalities HUGE # of Extreme Rays 13 Outline 1. Bounding Rate Regions in Principle/Theory (a) Distributed Storage/Network Coding Regions & Entropic Vectors (b) Outer Bounds: Shannon/Non-Shannon (c) Inner Bounds: Linear Polymatroids/ Representable Matroids 2. Calculating Polyhedral Rate Regions: Why is na¨ıve? " (a) Di↵erence Between Descriptions (b) Projecting Polyhedra (c) Exploiting Symmetries 3. Linear Polymatroid Enumeration for Rate Regions (a) Review of Non-isomorphic Matroid Enumeration (b) Non-isomorphic Representable Matroid Enumeration (c) Non-isomorphic Extremal Representable Matroid Enumeration (d) Network Constrained Linear Polymatroid Enumeration 4. Related Current & Future Work Why this is na¨ıve : Projecting Polyhedra – Options[7, 21, 22, 23, 24, 25, 26] Extreme Ray/Point Fourier Motzkin Benson's Outer Convex Hull Method Projection Elimination Approximation Algorithm Successively only deals with Project Extreme Eliminates works directly Pareto frontier in Rays & Points Variables (slow) in projected projected space inequalities space of N dimen. Redundancy polyhedron Removal (convex hull) N-1 dim.