Linear Block Codes Vector Spaces Let GF(q) be fixed, normally q=2 • The set Vn consisting of all n-dimensional vectors (a n-1, a n-2,•••a0), ai ∈ GF(q), 0 ≤i ≤n-1 forms an n-dimensional vector space. • In Vn, we can also define the inner product of any two vectors a = (a n-1, a n-2,•••a0) and b = (b n-1, b n-2,•••b0) n−1 i ab= ∑ abi i i=0 • If a•b = 0, then a and b are said to be orthogonal. Linear Block codes • Let C ⊂ Vn be a block code consisting of M codewords. • C is said to be linear if a linear combination of two codewords C1 and C2, a 1C1+a 2C2, is still a codeword, that is, C forms a subspace of Vn. a 1, ∈ a2 GF(q) • The dimension of the linear block code (a subspace of Vn) is denoted as k. M=qk. Dual codes Let C ⊂Vn be a linear code with dimension K. ┴ The dual code C of C consists of all vectors in Vn that are orthogonal to every vector in C. The dual code C┴ is also a linear code; its dimension is equal to n-k. Generator Matrix Assume q=2 Let C be a binary linear code with dimension K. Let {g1,g2,…gk} be a basis for C, where gi = (g i(n-1) , g i(n-2) , … gi0 ), 1 ≤i≤K. Then any codeword Ci in C can be represented uniquely as a linear combination of {g1,g2,…gk}. ⋯ Cxg=1 1 ++ xgxk k, i ∈ GF (2) Generator Matrix (cntd) Construct a matrix G as follows g1 g2 x =(x ,…x ) C=xG =x g +…+x g G = 1 k G 1 1 k k ⋮ gK G is called a generator matrix for G since G can be used to directly encode the K information bits into a codeword C Dual code C┴ is also a linear code. Generator Matrix for Dual Code (Parity Matrix) h1 h H=2 , hhh = (⋯ h ) ⋮ i in(− 1), in ( − 2), i 0 hn− K {h 1,…hk} is a basis for C┴┴┴. Since C and C┴ are orthogonal, it follows that HG’ = 0<---- (n-k) x k matrix with all elements 0 In particular 0 Ci = (C i1 , C i2 , … Cin ) iff HC i’ = The matrix H is called a parity check matrix for C. A linear block code can be described by either its generator matrix or parity check matrix. However, G and H are not unique. Systematic Code ⋮ G= [ IKXK P ] ⋮ H= − P' I (n− K )( X n − K ) P is kx(n-k) matrix Ci = xiG = (xi xip) where xi = (x i1 , x i2 , …xik ) is the k information bits In the codeword Ci, the first k bits are the information bits, and the remaining n-k bits are linear combinations of the k information bits. The remaining n-k bits add correlation into the codeword Ci, and are called parity check bits . Non-Systematic Code • One can always convert a non-systematic code into an equivalent systematic code by performing a proper vector coordinate permutation, for example, using Gaussian elimination and a proper column permutation. • Gaussian elimination don’t alter the codebook. • Column reordering, may change the codebook. However, for AWGN channel and discrete memoryless channel, the resulting linear code has the same performance as the original code. Example 0011101 G = 01 00111 1 00111 0 Express G in system form? Determine the parity check matrix H? Linear Block Code (cntd) ⊂ •A binary linear code C Vn with dimension k is sometimes referred to as an [n,k,d] code, where d is the Hamming distance of C. •Given any vector a=(a n-1, a n-2,•••a0) in Vn, the weight of a is the number of nonzero ai, and is denoted by wt(a). wt(1100110) =4 •The Hamming distance d of a linear code C is equal to the minimum weight of nonzero codewords. d≜ min dab (,)= min wtab ( − ) ab,∈C ab , ∈ C ab≠ ab ≠ = minwt ( a ) a∈C a≠0 Example Hamming code (7,4,3) 1000101 0100111 G = 0010110 0001011 H? Example, encode 1100 Parity check bits? d=3 Linear Block Code (cntd) Theorem 5.1.2: Let C be an [ n, k, d] code with a parity check matrix H. Then every d-1 columns of H are linearly independent and some d columns are linear dependent Theorem 5.1.3( the singleton bound): If C is an [ n, k, d] code, then d≤n-k+1. If d = n – k +1, then the corresponding [n, k, d] code C is called maximum distance separable (MDS). Construction of New codes from Old Adding an Overall Parity Check Let C be an [ n, k, d ] code with a parity check matrix H. suppose that some codewords in C have odd weights. a) We add a 0 bit at the end of every codeword of C with even weight b) We add a 1 bit at the end of every codeword of C with odd weight The resulting code C’ has a parity check matrix 0 H 0 11 ··· 11 Every new codeword in C’ has even weight, C’ is called the extended code of C, and is an [ n, k, d’ ] code d if d is even d ' = d +1 if d is odd Shortening a code by taking a cross-section •Take all codewords in an [n, k, d] code C that begin with 0 and delete the first coordinate, we then get a shortened [n-1,k-1,d’] code C’, where d’≥d. •Repeat this process l times, one may get a shortened [n-l, k-l, d”] code, where d” ≥d. Some Specific Linear Block Codes [2 m-1, 2 m-1-m,3] Hamming codes •The parity check matrix H has the property that the column vectors of H consist of all non-zero m-dimensional vectors. •No two columns of H are linearly dependent. However, for m>1, it is possible to find three columns that add to 0. Extended [2 m, 2 m-1-m,4] Hamming codes 11101000 01110100 H = 11010010 11111111 m=3 Some Specific Linear Block Codes [2 m-1, m, 2 m-1] Simplex codes The dual code of [2 m-1, 2 m-1-m,3] of Hamming code. It is also called a maximum-length shift register code. Every pair of codewords is the same distance apart. The simplex signal set discussed in Chapter 3 is the continuous analogue of the simplex code [2 m, m+1, 2 m-1] First-order Reed-Muller codes The dual code of the extended [2 m, 2 m-1-m,4] Hamming code. The biorthogonal signal set discussed in Chapter 3 is the continuous analogue of this code. Hardmard Codes A Hardmard code consists of all row vectors of a Hardmard matrix. Definition: An nxn binary matrix A=(aij ) (aij is in GF(2)) is said to be a Hardmard matrix if the matrix (− 1)A = ( − 1) aij ( )nxn is orthogonal. Theorem 5.1.4: If A is an nxn Hardmard matrix, then n=2 or 4k for some k. Conjecture: It is strong believed that Hardmard matrices exist for all n’s specified in the above. Construction of Hardmard Codes 0 0 A2 = 0 1 If A n is an nxn Hardmard matrix then An A n A2n = An A n is also a Hardmard matrix. A n is the complement of A n. By using the method repeatedly, we can construct all Hardmard matrices of order n=2 m, m = 1, 2, ···. With this construction, the corresponding Hardmard code is a linear [2 m, m, 2 m-1] code. Cyclic Codes Definition: Fix a finite field GF(q). A linear block code C is said to be cyclic if any cyclic shift of a codeword is also a codeword, i.e., whenever C=(C n-1,C n-2, …C0) is in the codebook, then so is (C n-2,C n-3,…C0, C n-1) . Example : C = {000, 110, 101, 011} Algebraic Description of Cyclic Codes Associate with any vector a = (a n-1, a n-2,•••a0) in Vn a polynomial n-1 n-2 a(x) = a n-1x + a n-2x +••• +a 1x + a 0 n n-1 2 Then xa(x) = a n-1x + a n-2x + ••• + a 1x + a 0x n-1 n = a n-2x + •••+a 0x+a n-1 mod (x -1) is the polynomial associated with the vector (a n-2,•••a0 a,n-1) Under modulo (x n-1) operation, multiplying a(x) by x corresponds to a cyclic shift of a. A vector a with its associated polynomial a(x) a linear block code C with the set of polynomials associated with its codewords. Definition: A linear block code C is cyclic if xC(x) mod (x n-1) єC whenever C(x)єC. Cyclic Codes (cntd) Let g(x)єC be a monic polynomial of minimal degree in C ( A polynomial of degree r is monic if the coefficient before xr is 1) Theorem 5.2.1: a) g(x) is unique b) Any C(x) in C can be written uniquely as C(x)=f(x)g(x), where f(x) is a polynomial having degree <k and n-k=degree of g(x) c) g(x) is a factor of x n-1, i.e., g(x)|x n-1 n-k n-k-1 d) If g(x)=x +g n-k-1x +···g1x+g 0, the C is an [n, k, d] code for some d and has the following generator matrix.