Generalized Sphere-Packing Upper Bounds on the Size of Codes for Combinatorial Channels
Total Page:16
File Type:pdf, Size:1020Kb
Generalized sphere-packing upper bounds on the size of codes for combinatorial channels Daniel Cullina Negar Kiyavash Dep. of Electrical & Computer Eng. Dep. of Industrial and Enterprise Systems Eng. Coordinated Science Laboratory Coordinated Science Laboratory University of Illinois at Urbana-Champaign University of Illinois at Urbana-Champaign Urbana, IL 61801 Urbana, IL 61801 [email protected] [email protected] Abstract—A code for a combinatorial channel is a feasible give a different sphere-packing bound. Cullina and Kiyavash point in an integer linear program derived from that channel. applied this idea to the deletion channel to improve the best Sphere-packing upper bounds are closely related to the fractional known upper bound for a fixed number of errors [5]. We show relaxation of this program. When bounding highly symmetric channels, this formulation can often be avoided, but it is essential that the same phenomenon exists for substitution and erasure in less symmetric cases. We present a few low-complexity upper errors. Channels that perform various mixtures of substitutions bounds on the value of the relaxed linear program. We also and erasures all have the same codes but lead to different discuss a more general bound derived from the codeword bounds: the Hamming bound, Singleton bound, and a family constraint graph for the channel. This bound is not necessarily of intermediate bounds. computationally tractable. When there is a family of channels with the same constraint graph, tractable bounds can be applied In Section I, we discuss the linear programs associated to each channel and the best bound will apply to the whole family. with sphere-packing bounds. In Section II, we discuss various techniques for obtaining nonuniform sphere-packing bounds. Sphere-packing upper bounds on the size of a zero er- In Section III, we discuss families of channels that have the ror code are fundamentally related to linear programming. same codes but give different sphere-packing bounds. However, many classical combinatorial channels are highly I. SPHERE-PACKING BOUNDS AND LINEAR PROGRAMS symmetric. For these, it is often possible to get the best possible sphere-packing bound without directly considering In a combinatorial channel, also known as an adversarial any linear programs. For less symmetric channels, it is still channel, for each channel input there is a set of possible possible to obtain many upper bounds without writing down a outputs. This behavior of the channel can be represented by linear program. Recently a new upper bound, explicitly derived a bipartite graph which we call the channel graph. The left via linear programming, was applied to the deletion channel vertices of this graph are the channel inputs and the right by Kulkarni and Kiyavash [1]. It was subsequently applied vertices are the channel outputs. to grain error channels [2], [3] and multipermutation channels For each output, at most one of the associated inputs can be [4]. We will refer to this as the local degree bound and present in a zero-error code. Consequently, a zero-error code is a pack- a generalized version. ing of the neighborhoods of the inputs into the output space. Sphere-packing and sphere-covering arguments have been Maximum set packing is NP-Hard for general set systems, so applied in an ad hoc fashion throughout the coding theory we seek efficiently computable bounds. Maximum set packing literature. This work aims at presenting a unifying frame- is naturally expressed as an integer linear program. The relaxed work that that permits such arguments in their most general problem, maximum fractional set packing, provides an upper form applicable to both uniform and nonuniform error sphere bound on the original packing problem. sizes. More precisely, we derive a series of bounds resulting A. The linear program from approximiations to packing and covering problems. We Let U be the set of channel inputs and let V be the set characterize each bound as the solution to a linear program of channel outputs. For u 2 U, let N(u) ⊆ V be the and study the relationships between them. These bounds use neighborhood of u in the bipartite channel graph (the set varying levels of information about structure of the error of outputs that can be produced from u). For v 2 V , let model and consequently make tradeoffs between performace N(v) ⊆ U be the neighborhood of v in the bipartite channel and complexity. graph (the set of inputs that can produce v). Alternatively, we can directly consider the constraints on Each neighborhood of an output gives a constraint in the codes and forget the particular channel that produced the primal linear programming problem. constraints. This approach offers a better upper bound at the cost of increased complexity. In some cases, there is a Definition 1. Let A 2 f0; 1gjU|×|V j be the bipartite adjacency family of channels that all have the same codes but each matrix for the channel graph H. ∗ Let 1S be the indicator vector for the set S. Note that If the minimum degree is far from the average degree, pMD T T ∗ A1fvg = 1N(v) and 1fugA = 1N(u). is likely to be a bad approximation of p . A better bound comes from considering all of the input degrees. Definition 2. For a channel graph H, the size of the maximum set packing, is Definition 4. For a channel graph H, define the degree sequence bound p(H) = maxf1T x : x 2 f0; 1gjUj;AT x ≤ 1g ∗ T jUj T T pDS(H) = maxf1 x : x 2 R ; 0 ≤ x ≤ 1; 1 A x ≤ jV jg: and the size of the maximum fractional set packing is p∗ ∗ T jUj T This is the same as the program for MD, except that we p (H) = maxf1 x : x 2 R ; x ≥ 0;A x ≤ 1g: have added the constraint x ≤ 1. Again, it is easy to see that The size of the minimum set covering is this bounds p(H). Each u 2 U is in the neighborhood of at least one v 2 V . Thus in the linear program for p∗(H), u is T jV j κ(H) = minf1 y : y 2 f0; 1g ; Ay ≥ 1g included in at least one constraint and xu ≤ 1. This means ∗ that the feasible set for pDS(H) includes that feasible set for and the size of the minimum fractional set covering is ∗ ∗ ∗ p (H), so pDS(H) ≥ p (H). ∗ T jV j κ (H) = minf1 y : y 2 R ; y ≥ 0; Ay ≥ 1g: The optimum of this program can be found greedily. Sort the points in U by degree from low to high. In each step, By strong linear programming duality, p∗(H) = κ∗(H). take the first point in the list add weight to it until you hit a The channel must output something for each possible input, constraint. If the global constraint is binding, we are done. If so N(u) is nonempty for each u 2 U. This ensures the primal we hit the local constraint xu ≤ 1 first, move on to the next programs are bounded and the dual programs are feasible. point in the list. This algorithm finds a maximum set C ⊆ U The value of the linear program can be computed in poly- such that for all c0 2 U n C, nomial time. However, we are usually interested in a channels X 0 X with exponentially large input and output spaces. To analyze jN(c)j ≤ jV j ≤ jN(c )j + jN(c)j such channels, even simpler bounds are desired. c2C c2C In other words, degree information does not rule out the II. FOUR APPROXIMATIONS TO THE MAXIMUM possibility that C is a code. However, if any other input is FRACTIONAL SET COVER added to C, the resulting set cannot be a code. ∗ ∗ In this section we consider four simple upper bounds on The next bound is intermediate between pMD and pDS. the maximum fractional set packing and minimum fractional Definition 5. For a channel graph H, define the degree set cover number. Each of these bounds is the value of some threshold bound simplified linear program. They are derived either by relaxing the constraints of the primal program or by tightening the ∗ T jUj T pDT (H; d) = maxf1 x : x 2 R ; 0 ≤ x ≤ 1; a x ≤ jV jg constraints of the dual program. where The degree of u in the channel graph is jN(u)j. ( d jN(u)j ≥ d; aT = Definition 3. For a channel graph H, define the minimum u 0 jN(u)j < d: degree bounds ∗ ∗ ∗ T jUj T T Let pDT (H) = mind pDT (H; d). pMD(H) = maxf1 x : x 2 R ; x ≥ 0; 1 A x ≤ jV jg; ∗ This is equivalent to applying the degree sequence bound to κMD(H) = minfjV jz : z 2 R; z ≥ 0;A1z ≥ 1g; a modified degree sequence, one where all degrees less than = minf1T y : y = 1z; z 2 ; z ≥ 0; Ay ≥ 1g: R d have been reduced to 0 and all degrees at least d have been jV j The bounds are equal: reduced to d. The value of the program equals jSj+ d , where S = fu 2 U : jN(u)j < dg, the members of U with small jV j ∗ ∗ degree. If we let d = minu2U jN(u)j, then S is empty and pMD(H) = κMD(H) = : minu2U jN(u)j the bound reduces to the minimum degree bound. The linear program for p∗(H) contains a constraint for each ∗ A. The local degree bound v 2 V : 1N(v)x ≤ 1. In the linear program for p we have P MD Definition 6.