arXiv:1102.3527v2 [cs.IT] 24 May 2011 o ntesbpc pne yteecdn etr already called vectors is encoding It the user. by that by spanned received subspace the in encoding said is not corresponding packet encoded An the users. be all together contains to packet is broadcasted which is encoded and vector header An combination vector a transmitter. linear encoding with the The the encoded by packets. for determined send original coefficients the to the all specifies is ove of combinations idea linear field taking finite by The obtained a [2]. are reliability that packets [1], of terms utilization in channel problem broadcasting wireless the uprom ubro xsigshmsi em faverage field of finite terms binary in schemes with delay. existing scheme worst-case the of and our number up of a speeding show outperforms performance Simulations innova in developed. delay generate is utilized the vectors to encoding be algorithm sparse innovative efficient can and both An is sparsity process. which The decoding vector shown encoding is sparse. it an users, and find of number fie always the finite can to the equal we When or two. than vect is larger encoding size is size innovative field finite an prob the of when the NP-complete existence that the prove determining we of feedback, with broadcasting wireless eoig biul,i l h noe akt generated are for packets transmission packets encoded for enough received transmitter the yet the all by not if have Obviously, who decoding. users all to h vrg ubro rnmsin saaye n[] By autho [5]. some users, in from analyzed information is feedback be transmissions the to exploiting of packet shown number innovative average is an The approach find This to algorithm. tries rate-optimal a probabilistic and consider users a know authors all by transmitter the of the which status [4], in the In feedback, innovativ perfect found. an with users, be system of s always number the the can to if equal that packet is shown field is finite it the of [3], the In packets. obtain innovative generate to minimized. users be can all packets for of set transmissions complete packet of number omte fteHn ogSeilAmnsrtv ein C Region, Administrative Special AoE/E-02/08). Kong No. Hong the of Committee hswr a atal upre yagatfo h Univers the from grant a by supported partially was work This ierntokcdn rvdsa xeln ouinto solution excellent an provides coding network Linear Abstract ouiieterdocanlefiinl,i sipratto important is it efficiently, channel radio the utilize To eateto lcrncEngineering Electronic of Department noaiet user a to innovative iyUiest fHn Kong Hong of University City eeaino noaieadSas Encoding Sparse and Innovative of Generation mi:[email protected] Email: e tr o racs ytm ihFeedback with Systems Broadcast for Vectors I h plcto flna ewr oigto coding network linear of application the —In fteudryn nt edi ufiinl large. sufficiently is field finite underlying the if oYe Kwan Yuet Ho .I I. ftecrepnigecdn etris vector encoding corresponding the if NTRODUCTION innovative innovative h hns nvriyo ogKong Hong of University Chinese The fi sinnovative is it if mi:[email protected] Email: nttt fNtokCoding Network of Institute h total the , ia(Project hina ent .Shum W. Kenneth t Grants ity ris or that that lem tive ize a r nd to ld rs e s ea n oerr h orend ep rc ftestatus the of track keeps node no source has The channel error. feedback no the and that a t delay assume receiving acknowledgement We an node. Upon source sends source. the receiver feedback the a a to successfully, is receiver packet there each that assume from packet We channel received error-free. successfully be unreco to a is assumed while packet discarded, erased and An erable probability. erasure the denotes iuain o ohsaladlrefiiefields. finite large and cofactor small by the schemes both transmission V, for other Section simulation, some with In compared method. is cofactor method the we with called that vector rithm encoding show innovative is we and most IV, field sparse at Section a finite find to In binary always NP-complete. the required can the with be packets and transmission to of file noiseless, shown number the be off the to finish minimizing assumed wher of [11], is in problem channel obtained also broadcast is the vector result encoding related proble innovative A the of that NP-complete. existence prove the we III, determining Section of In exist. not may vector This of [10]. expense retransmissions. [9], the of in number at larger complexity considered computational is what lowers received back approach feeding have without order linear receivers field In random binary the complexity, the devices. over decoding hand-held code and network mobile encoding for reduce finit costly large to with be computation may of The an [8]. field set in utilizing receivers full can analyzed without the the is coding packets feedback of at broadcast reception innovative available Randomized complete are that packets. the packets for so waiting the [7] without once [6], decoded networ in be decodable packets instantly generate coded to algorithms develop rnmte and transmitter oeand node r lhbt ewl locl a a packe call probability transmitted is the also channel receives successfully erasure will receiver Each erasure broadcast We broadcast the alphabet. a to ary as input modeled The is channel. which channel, wireless a node source hntefiiefil iei ml,a noaieencoding innovative an small, is size field finite the When ecnie ieessnl-o ytmcnitn fone of consisting system single-hop wireless a consider We I S II. K YSTEM U i o-eocmoet sn eemnsi algo- deterministic a using components non-zero 1 sthe as S − K at obodatafiet l eevr via receivers all to file a broadcast to wants P M e eevr/sr.W denote We receivers/users. i DLAND ODEL needn fec te,where other, each of independent , t eevr where receiver, -th eateto lcrncEngineering Electronic of Department mi:[email protected] Email: iyUiest fHn Kong Hong of University City P h a Sung Wan Chi ROBLEM q ayapae packet. a alphabet -ary i { ∈ F ORMULATION 1 , S 2 K , . . . , stesource the as } with t The . P k- q v- m is is o y e e e - of each receiver. The transmitted packet is a function of the j-th literal in the i-th clause is ¬xk, then let the k-th and source file and the acknowledgements from the K receivers. the (n + 1)-st component in the j-th row of Bi be 1, and In this paper, we focus on transmission schemes with linear the remaining components be all zero. Let Ci be the matrix network coding. The alphabet size q is a power of prime whose rows form a basis of the orthogonal complement of the and the alphabet set is identified with the finite field GF (q). row space of Bi. We will use the fact that a vector v is in the T The file is packetized into N packets. The transmitted packet row space of Ci if and only if Biv =0. is a linear combination of the N packets, with coefficients Consider an example with n = 4 Boolean variables. From drawn from GF (q). An encoding vector is an N-vector whose the clause ¬x1 ∨ ¬x2 ∨ x3, we get components are the N coefficients used in the generation of a 10001 00010 transmitted packet. Each user returns an acknowledgement to B 01001 C i = , i = 11001 . the source node if a packet is received successfully, until he "00100#   has already received N packets whose encoding vectors are It can be verified that each row in Bi is orthogonal to the rows linearly independent over GF (q). In order to minimize the in Ci, i.e., the row space of Ci is the orthogonal complement delay of each user, it is crucial to generate encoding vectors of the row space of Bi. which are innovative to all users. For the extra user, user m +1, let Bm+1 be the 1 × (n + 1) Our objectives are (i) to determine whether an innovative matrix [0n1], where 0n stands for the 1 × n all-zero vector. encoding vector exists, and (ii) to devise an effective algorithm The problem reduction can be done in polynomial time. for generating innovative and sparse encoding vectors. Let x = [x1 x2 ...xn] be a Boolean row vector and xˆ = x . Obviously, any solution x to a 3-SAT problem would III. NP-COMPLETENESS IN FINDING AN INNOVATIVE [ 1] cause the product B xT a non-zero vector for NCODING ECTOR WHEN j ˆ j =1, 2,...,m E V q =2 T and [0n1]xˆ 6=0. Therefore xˆ is not in the row space of Cj If the field size q is larger than or equal to K, it is for all j. Hence xˆ is also a solution to the derived 2-IEV known that an innovative encoding can always be found [3]. problem. Indeed, the number of non-zero encoding vectors which are Conversely, any solution to the derived 2-IEV problem also ri not innovative to user i is equal to q , where ri is the rank of yields a solution to the original 3-SAT problem as well. Let the subspace spanned by the encoding vectors already received n+1 c = [c1 c2 ...cn cn+1] ∈ GF (2) be a solution to the by user i and ri < N , and hence by the union bound, the derived 2-IEV problem. Note that we must have cn+1 = 1 number of non-zero vectors which are not innovative is at because of Bm+1. Let i be an integer between 1 and m. Since r1 r2 r most (q − 1)+(q − 1) + ... + (q K − 1). When q ≥ K, T c is not in the row space of Ci, the product Bic is a non- the number of non-zero and non-innovative encoding vector zero vector, for otherwise c would belong to the orthogonal is strictly less than qN − 1, and hence we can always find an complement of Bi. Hence, if we assign TRUE to xk if ck =1 innovative packet. and FALSE to xk if ck =0, for k =1, 2,...,n, then the i-th When the underlying finite field is small, an innovative clause will have a TRUE value. Since this is true for all i, the encoding vector may not exist. whole Boolean expression also has a TRUE value. IEV Problem q- : A problem instance consists of K matrices The problem 2-IEV is clearly in NP, since it is efficiently C i over GF (q), i = 1, 2,...,K, and each matrix has N verifiable. Hence it is NP-complete. columns. Determine whether there is an N-dimensional vector ENERATION OF PARSE NCODING ECTORS over GF (q) which is not in the row space of Ci, for all i. IV. G S E V After receiving N packets whose encoding vectors are Theorem 1. 2-IEV is NP-complete. linearly independent over GF (q), a user can recover the source Proof: The idea is to reduce the 3-SAT problem, well- data by solving a system of N linear equations. The standard known to be NP-complete [12], to the 2-IEV problem. Recall Gaussian elimination requires O(N 3) operations over GF (q). that the 3-SAT problem is a Boolean satisfiability problem, One way to reduce the decoding complexity is to choose whose instance is a Boolean expression written in conjunctive encoding vectors which are sparse. A vector is called w-sparse normal form with three variables per clause (3-CNF), and the if there are no more than w non-zero components. question is to decide if there is some assignment of TRUE and Theorem 2. If q ≥ K, we can find an innovative encoding FALSE to the variables such that the given Boolean expression vector which is K-sparse. has a TRUE value. Let E be a given Boolean expression with n variables Proof: Suppose that user k, for k = 1, 2,...,K, has x1,...,xn, and m clauses in 3-CNF. We want to reduce the received rk packets whose encoding vectors are linearly in- 3-SAT problem to the 2-IEV problem with N = n+1 packets dependent. Let Ck be the rk × N matrix obtained by putting and K = m +1 users. together the rk encoding vectors. We want to find a K-sparse To the i-th clause (i = 1, 2,...,m), we first construct a innovative encoding vector x = [x1 x2 ... xN ]. 3 × N matrix Bi. If the j-th literal (j = 1, 2, 3) in the i-th Since Ck is full-rank, we can find rk columns of Ck which clause is xk, then let the k-th component in the j-th row of are linearly independent. Let Ik be a set of indices of rk Bi be 1, and the other elements be all zero. Otherwise, if the linear independent columns in Ck. For each k, we arbitrarily ′ ′ ′ pick a column whose index is not in Ik. We call this the Suppose that I1 = {1, 2}, I2 = I3 = {1, 2, 3}. We have ′ extra column and let Ik be the union of Ik and the index of 1 0 1 1 0 0 this extra column. The cardinality of I′ is r +1. For each ˆ 0 1 ˆ ˆ k k H1 = x x , H2 = 0 1 1 , H3 = 0 2 0 . 1 2 x x x x x x k =1, 2,...,K, we construct an (rk + 1) × (rk + 1) matrix   " 1 2 3# " 1 2 3# Hˆ k, by first appending the vector x = [x1 x2 ... xN ] to the In Hˆ 1 the cofactors of x1 and x2 are −1 and 0 respectively. bottom of matrix Ck, and then deleting all columns of the ′ Hence, i1 = 1. In Hˆ 2, all cofactors of x1, x2 and x3 are resulting matrix except the columns with indices in Ik. non-zero. We thus have i2 = 3. In Hˆ 3, the cofactor of x3 is For each k, we compute the rk +1 cofactors of the entries Hˆ nonzero, and so i3 =3. The index set J is equal to {1, 3}. The in the last row of k. Let xik be the variable with largest two index sets of users are K1 = {1} and K2 = {2, 3}. By the index in the last row of Hˆ k whose cofactor is non-zero. The cofactor method, we first assign 0 to x2. Then we go through column indices i1,...,iK so obtained may not be distinct. the variable indices in J in ascending order. For x1, we can Let J , {j1, j2,...,js} be the set of distinct indices such assign any nonzero value to x1. For example, we pick x1 =1. that J = {i1,i2,...,iK} and j1 < j2 <...determinants of Hˆ 2 t =1, 2,...,s, we let Kt be the set of users such that k ∈ Kt and Hˆ 3, which are −1+ x3 and 2x3 respectively. Finally, we if and only if ik = jt. We remark that J contains at most K want to assign a value to x3 such that −1+ x3 6= 0 and distinct indices, i.e., s ≤ K. 2x3 6= 0. The only choice in this example is x3 = 2. The We obtain a K-sparse innovative encoding vector as follows. resulting encoding vector is [1 0 2]. First, we set all variables x , for i∈ / J , to zero. Then we assign i In the cofactor method, the main complexity is related to the values to x 1 , x 2 ,...,x sequentially, so that the determinant j j js computation of r +1 cofactors in an (r + 1) × (r + 1) matrix. of Hˆ is nonzero for all k. For k ∈ K1, the last row of Hˆ has k k A straightforward calculation of an r × r determinant requires only one variable, namely x 1 , whose value is not yet assigned j O(r3) arithmetic operations. The calculation of all cofactors in (the rest are all set to zero). The cofactor of x 1 in Hˆ is non- j k a matrix would require O(r4) operations per each user in each zero. If we expand the determinant of Hˆ in the last row, we k step. We can use a more efficient algorithm, called the Bareiss see that the determinant can be written as b x , where b is ik ik ik algorithm. The number of arithmetic operations over GF (q) the cofactor of x in Hˆ . We have a non-zero value if x 1 is ik k j required in the computation of cofactors per user can be non-zero, for all . We can assign any non-zero element 3 k ∈ K1 reduced to O(N ). Summing over all K users, the complexity of GF (q) to x 1 , and make Hˆ non-zero for all k ∈ K1. 3 j k for computing all cofactors is O(KN ). The complexity of 2 Inductively, suppose that the values of xj1 ,...xjt−1 have the rest of the cofactor method is of O(K N). The overall 3 2 been assigned. Consider the determinants of Hˆ k for k ∈ Kt. complexity of the cofactor method is O(KN + K N). The only variable in the last row of Hˆ k which has not been If Jaggi-Sanders algorithm in [13] is applied to solve the 2 assigned a value yet is xjt . If we expand the determinant on encoding problem, the complexity is O(KN (K + N)). It the last row, we obtain a linear polynomial in the form of means that the cofactor method is no worse than the algorithm aik + bik xjt , where aik is a constant and bik is the cofactor of in [13] in terms of the encoding complexity. But certainly Hˆ xjt in k. There are at most K such degree-one polynomials, the encoding vector produced by Jaggi-Sanders algorithm is and thus we can assign a value to xjt such that all determinants not sparse. Details on the Bareiss algorithm is given in the of Hˆ k are non-zero. Here we have used the assumption that appendix. q ≥ K. Note that the assignment of xjt does not affect the The cofactor method assumes that q ≥ K. If q

60 1 Worst−case delay

55 0.5 Average total number of additions in decoding

50 0 20 30 40 50 60 70 80 20 30 40 50 60 70 80 Number of users Number of users

Fig. 1. The worst-case delay vs the number of users Fig. 3. The average total no. of additions vs the number of users (q=2)

5 x 10 70 2 SOM Multiplication − RLNC MWVS Addition − RLNC 65 RLNC (q=2) Multiplication − Cofactor Method Cofactor Method (q=2) 1.5 Addition − Cofactor Method RLNC (q=101) Cofactor Method (q=101) 60

1 55 Average delay

0.5 50

45 0 20 30 40 50 60 70 80 Average total number of operations in decoding 20 30 40 50 60 70 80 Number of users Number of users

Fig. 2. The average delay vs the number of users Fig. 4. The average total no. of operations vs the number of users (q=101)

each row of Ck contains exactly one nonzero component. delay performance of the cofactor method with a small finite We initialize Ii to be the set of non-zero columns in Ck. In field size (q =2) is comparable to that of RLNC with a large the second phase, we transmit the packets using the encoding finite field size (q = 101). Next, we consider the average delay vectors generated by the cofactor method. performance. According to information theory, the best we can Each simulation points involved 1000 random realizations do is to have N/(1 − Pe)=45.7 transmissions on average. In and we assume that N = 32 and Pe = 0.3. The worst-case Figure 2, we observe that both the cofactor method and RLNC delay is defined as the average of total number of transmissions with large enough finite field size (q = 101) can achieve the for S to ensure that all users receive an intact file over 1000 limit. From the figure, we also see that although all concerned random realizations. The average delay means the average methods may not be optimal when q =2, the cofactor method number of transmissions for S so that an intact file can be always results in a smaller average delay. received by a user. For the decoding complexity, we count The decoding algorithms in most of the previous work the number of additions and multiplications in decoding. In are basically Gauss-Jordan elimination except the instantly our simulations, an addition operation involving two non-zero decodable schemes in [6], [7]. We implement the Gauss-Jordan operands is counted. A multiplication operation is counted elimination for sparse matrix in our simulation. Note that the when none of the two operands is 1 or 0. K-sparse property of the cofactor method implies an upper Figure 1 shows the worst-case delay performance of our bound on the number of non-zero entries in an encoding system with the cofactor method, the random linear network vector. In practice, the average number of non-zero entries code (RLNC) scheme in which the components are selected is significantly less than both K and N even for K>N. As according to a uniform distribution, the sorted opportunistic a result, significant decoding complexity reduction is expected method (SOM) in [6] and the maximum weight vertex search for a system with the cofactor method. Figures 3 and 4 (MWVS) algorithm in [7] for encoding vector generations, show the average total number of operations for all users where both SOM and MWVS generate instantly decodable in the system when q = 2 and q = 101, respectively. The packets. It is found that, for q =2, the cofactor method always cofactor method indeed yields significant reduction in both the performs better than RLNC, SOM and MWVS in terms of the average total number of addition and multiplication operations worst-case delay. In addition, we also find that the worst-case when compared with RLNC. From Figure 3, we observe that, c c · · · c · · · c with both SOM and MWVS which are instantly decodable, a 11 12 1r 1n ...... receiver enjoys a low decoding complexity at the expense of ...... C =  . larger delay. As a result, the cofactor method which always cr,1 cr,2 · · · cr,r · · · cr,n generates sparse encoding vectors is a promising choice in  ∗ ∗ ··· ∗ ∗ ∗   x x · · · x · · · x  terms of delay performance and decoding complexity.  1 2 r n  The entries marked by “∗” are not known yet and will be VI. CONCLUSIONS revealed later. We can apply the Bareiss algorithm to the submatrix obtained by removing the “∗” entries and the right We devise a cofactor method to generate K-sparse encoding n−r columns. When the value of the (r+1)-st row is known, vector. When q ≥ K, it is guaranteed that the resulting we can run the Bareiss algorithm again on the submatrix encoding vector is innovative, and hence the broadcast system obtained by removing rows r +2 to n − 1 and the n − r − 1 is delay-optimal. The sparsity can be exploited in devising columns on the right. We can see that the r2 entries in the first faster decoding algorithm. Simulation result shows that the r rows and the first r columns are the same as before and we cofactor method outperforms RLNC, SOM and MWVS in do not need to re-calculate them. Only the calculation of the terms of both the worst-case delay and average delay. On 2r+1 new entries are required. For each user, the source node the other hand, when q = 2, the problem of determining the essentially runs the Bareiss algorithm on an N ×N matrix, and existence of an innovative encoding vector is NP-complete. the complexity per user is O(N 3). Summing over all users, APPENDIX the complexity involving the computation of the cofactors is O(KN 3). The Bareiss algorithm is a fraction-free algorithm for com- Acknowledgement: The authors are grateful to Prof. Wai Ho puting determinant [15]. To illustrate the idea, we apply Mow and Dr. Kin-Kwong Leung for fruitful email discussions. Bareiss algorithm to an n × n matrix M whose elements are integers and the last row consists of indeterminates x1, REFERENCES x2,...,xn. At the end of the algorithm, the entry in the lower- [1] S.-Y. R. Li, R. W. Yeung, and N. Cai, “Linear network coding,” IEEE Trans. on Inf. Theory, vol. 49, no. 2, pp. 371–381, Feb. 2003. right corner of M is a linear polynomial in x1, x2,...,xn, [2] R. Koetter and M. M´edard, “An algebraic approach to network coding,” and the coefficient of xi is the corresponding cofactor of xi in IEEE/ACM Trans. Networking, vol. 11, no. 5, pp. 782–795, Oct. 2003. the original M. We remark that this is an in-place algorithm, [3] M. Durvy, C. Fragouli, and P. Thiran, “Towards reliable broadcasting and the complexity is in the order of n3. using ACKs,” in Proc. IEEE Int. Symp. Inform. Theory, Nice, Jun. 2007, pp. 1156–1160. [4] L. Keller, E. Drinea, and C. Fragouli, “Online broadcasting with network Algorithm 1 Bareiss algorithm coding,” in Proc. of the Fouth Workshop on Network Coding, theory, and M applications (NetCod ’08), Hong Kong, Jan. 2008, pp. 68–73. Input: An n × n matrix . Assume that all principle minors [5] X. Xiao, L. Yang, W. Wang, and S. Zhang, “A broadcasting retransmis- of M are nonzero. sion approach based on random linear network coding,” in IEEE Int. Conf. Circuits and Systems for Comm., May 2008, pp. 782–786. Notation: Let mij denote the (i, j)-entry of M, and m00 , 1. [6] P. Sadeghi, D. Traskov, and R. Koetter, “Adaptive network coding for for k =1,...,n − 1 do broadcast channels,” in Fifth Workshop on Network Coding, Theory and mkkmij −mik mkj Compute mij ← , for i, j = k+1,...,n. Applications (NetCod ’09), Lausanne, Jun. 2009, pp. 80–85. mk−1,k−1 [7] S. Sorour and S. Valaee, “On minimizing broadcast completion delay end for for instantly decodable network coding,” in IEEE Int. Conf. Comm. (ICC Output: Return the (n,n)-entry of M. ’10), May 2010, pp. 1–5. [8] A. Eryilmaz, A. Ozdaglar, and M. M´edard, “On delay performance gains from network coding,” in 40th Annual Conf. on Inf. Sci. and Systems, 1 2 3 Princeton, Mar. 2006, pp. 864–870. Example 2. Consider M = 4 5 6 , as an example. [9] M. Ghaderi, D. Towsley, and J. Kurose, “Reliability gain of network "x1 x2 x3# coding in lossy wireless networks,” in IEEE Int. Conf. on Comp. Comm. After the first pass of the for-loop (k = 1), the partial result (INFOCOMM ’08), Phoenix, Apr. 2008, pp. 2171–2179. is [10] J. Heide, M. V. Pedersen, F. H. P. Fitzek, and T. Larsen, “Network 1 2 3 coding for mobile devices – systematic binary random rateless codes,” in IEEE Int. Conf. Comm. Workshops (ICC Workshop 2009), Dresden, M = 4 −3 −6 . x x − x x − x Jun. 2009, pp. 1–6. " 1 2 2 1 3 3 1# [11] S. Y. El Rouayheb, M. A. R. Chaudhry, and A. Sprintson, “On the minimum number of transmissions in single-hop wireless coding After the end of the algorithm, we have networks,” in IEEE Information theory workshop (ITW), Lake Tahoe, Sep. 2007, pp. 120–125. 1 2 3 [12] M. R. Garey and D. S. Johnson, Computers and intractability: A guide M = 4 −3 −6 . to the theory of NP-completeness. W. H. Freeman and Company, 1979. "x1 x2 − 2x1 −3x1 + 6x2 − 3x3# [13] S. Jaggi, P. Sanders, P. Chou, M. Effros, S. Egner, K. Jain, and L. Tolhuizen, “Polynomial time algorithms for multicast network code The coefficients of x1, x2 and x3 of the polynomial in the construction,” IEEE Trans. Inf. Theory, vol. 51, no. 6, pp. 1973–1982, 2 3 1 3 1 2 Jun. 2005. (3, 3)-entry are the cofactors , − , and 5 6 4 6 4 5 [14] C. W. Sung, K. W. Shum, and H. Y. Kwan, “On the sparsity of a linear network code for broadcast systems with feedback,” to appear in Proc. respectively. of Int. Symp. on Network Coding (NetCod ’11), Beijing, Jul. 2011. Furthermore, the algorithm can be run incrementally. Sup- [15] C. K. Yap, Fundamental problems of algorithmic algebra. New York: pose that only the first r rows in a matrix C is available, Oxford University Press, 2000.