Part 2. Channel Coding p. 1 ELEC 7073 Digital Communications III, Dept. of E.E.E., HKU What’s Channel Coding? ¾ Channel coding: – To add redundancy in the information sequence so that the sequence can be recovered at the receiver even in the presence of noise and interference. – Transmission reliability is improved. –Examples: – Block code Å memoryless – Repetition code, Hamming code, Maximum-length code, BCH code, Reed-Solomon code – Cyclic redundancy check (CRC) code – Convolutional code Å with memory – Turbo code – LDPC code p. 2 ELEC 7073 Digital Communications III, Dept. of E.E.E., HKU Error Control Techniques ¾ Forward error-control (FEC) – Using the redundancy in the transmitted code word for both the detection and correction of errors incurred during the transmission. – Simplex connection ¾ Automatic-repeat request (ARQ) − Using the redundancy merely for error detection. − The receiver sends a feedback to the transmitter, saying that if any error is detected in the received packet or not (Not-Acknowledgement (NACK) and Acknowledgement (ACK), respectively). − The transmitter retransmits the previously sent packet if it receives NACK. − Full-duplex connection ¾ Hybrid ARQ (ARQ+FEC) − Full-duplex connection − Error detection and correction p. 3 ELEC 7073 Digital Communications III, Dept. of E.E.E., HKU FEC Historical Pedigree (1) 1950 1960 1970 Shannon’s Paper 1948 Early practical Hamming Early practical implementations defines basic implementations of RS codes for tape binary codes Gallager’s Thesis and disk drives On LDPCs BCH codes Berlekamp and Massey Proposed Viterbi’s Paper rediscover Euclid’s On Decoding polynomial technique Convolutional Codes and enable practical Reed and Solomon algebraic decoding define ECC Forney suggests Technique concatenated codes p. 4 ELEC 7073 Digital Communications III, Dept. of E.E.E., HKU FEC Historical Pedigree (2) 1980 1990 2000 Ungerboeck’s TCM Paper - 1982 LDPC beats RS codes appear LDPC beats Turbo Codes in CD players Turbo Codes For DVB-S2 Berrou’s Turbo Code First integrated Standard - 2003 First integrated Paper - 1993 Viterbi decoders Renewed interest (late 1980s) Turbo Codes in LDPCs due to TC Adopted into Research Standards TCM Heavily (DVB-RCS, 3GPP, etc.) Adopted into Standards p. 5 ELEC 7073 Digital Communications III, Dept. of E.E.E., HKU Part 2.1 Block codes p. 6 ELEC 7073 Digital Communications III, Dept. of E.E.E., HKU Some Definitions (1) ¾ Field Let F be a set of elements on which two operations addition and multiplication are defined. F is said to be a field if and only if 1. F forms a commutative group under addition operation. ∀ab, ∈⇒+=+∈ F a b b a F 2. F contains an element called zero that satisfies a+0=a 3. F forms a commutative group under multiplication operation. ∀ab, ∈⇒⋅=⋅∈ F a b b a F 4. F contains an element called the identity satisfying a(1)=a 5. The operations “+” and “.” are distributive: abc⋅()()()+=⋅+⋅ ab ac p. 7 ELEC 7073 Digital Communications III, Dept. of E.E.E., HKU Some Definitions (2) ¾Galois field: GF(q) A finite field with q elements Example: when q is a prime, GF(q) has the elements {0, 1, …, q-1}. The addition and multiplication operations are defined modulo q. Binary field: GF(2), {0, 1} Addition Multiplication 0 ⊕ 0 = 0 0⋅0 = 0 0 ⊕1 =1 0⋅1 = 0 1⊕ 0 =1 1⋅0 = 0 1⊕1 = 0 1⋅1 =1 p. 8 ELEC 7073 Digital Communications III, Dept. of E.E.E., HKU Example of Galois Field ¾GF(5) is a set of elements {0, 1, 2, 3, 4} + 01234 i 01234 001234 0 00000 112340 101234 223401 202413 334012 303142 440123 404321 p. 9 ELEC 7073 Digital Communications III, Dept. of E.E.E., HKU Some Definitions (3) ¾ Vector space Let V be a set of vectors and F a field of elements (called scalars). V forms a vector space over F if: 1. Vector addition commutative: ∀u, v∈⇒+=+∈VV u v v u 2. Scalar multiplication: ∀aF∈∀∈⇒⋅=∈, vV a vuV 3. Distributive: (ab+ )⋅=⋅+⋅vvv a b and a ⋅( uvuv + ) =⋅+⋅ a a 4. Associative: ∀ab,,∈∀∈⇒⋅⋅=⋅⋅ Fvvv V () a b a () b 5. ∀vV∈⋅=, 1 vv 6. Zero vector: 0VvV∈ ,∀∈ ⇒ v0v + = 7. ∀vV∈∈+=, there exists an element wV, such that v w0 p. 10 ELEC 7073 Digital Communications III, Dept. of E.E.E., HKU Some Definitions (4) ¾ Vector space (cont’d): Example − The set of binary n-tuples, denoted by Vn V4 = {(0000),(0001),(0010),(0011),(0100),(0101),(0110),(0111), (1000),(1001),(1010),(1011),(1100),(1101),(1110),(1111)} ¾ Vector subspace: − A subset S of the vector space V n is called a subspace if: − The all-zero vector is in S. − The sum of any two vectors in S is also in S. − The scalar multiplication of the vectors in S is also in S. − Example: {(0000),(0101),(1010),(1111)} is a subspace of V4. p. 11 ELEC 7073 Digital Communications III, Dept. of E.E.E., HKU Some Definitions (5) ¾ Spanning set • A collection of vectors G = {vv12,,,…, vn } the linear combinations of which include all vectors in a vector space V, is said to be a spanning set for V or to span V. •Example: {(1000),(0110),(1100),(0011),(1001)} spans V4. ¾ Basis: • A spanning set for V that has minimal cardinality is called a basis for V. • Cardinality of a set is the number of vectors in the set. •Example: {(1000),(0100),(0010),(0001)} is a basis for V4 . V4 = {(0000),(0001),(0010),(0011),(0100),(0101),(0110),(0111), (1000),(1001),(1010),(1011),(1100),(1101),(1110),(1111)} p. 12 ELEC 7073 Digital Communications III, Dept. of E.E.E., HKU Block Codes (1) ¾ (n, k) block code: The information stream is partitioned into blocks of k elements. Each block is mapped to a larger block of n elements, called code word. Æ memoryless The elements of a code/information block word are selected from an alphabet of q symbols. − Binary code : q=2; the alphabet = {0, 1} − Non-binary code: q>2 Channel Data block encoder Code word k elements n elements n-k Redundant elements k R = Code rate c n p. 13 ELEC 7073 Digital Communications III, Dept. of E.E.E., HKU Block Codes (2) (n, k) binary block code n bits : k bits : 2n possible codes k 2 information blocks mapping V V n k C 2k code words Basis of C Example: (n=6,k=3) k code words code rate=3/6, 3 redundant elements (constructing a 3-bit blocks are mapped to 6-bit codewords. generator matrix) (100) Æ (110100); (010) Æ (011010) p. 14 ELEC 7073 Digital Communications III, Dept. of E.E.E., HKU Block Codes (3) ¾ Linear block codes: Suppose ci and cj are two code words in an (n,k) block code and a1 and a2 be any two elements selected from the alphabet. Then the code is linear if and only if a1 ci +a2 cj is also a code word. A linear code must contain the all-zero code word Easy implementation and analysis ¾ Non-linear block codes: Example: (4, 2) code (00) → ( 1000) ()(10→ 0100 ) ()(01→ 0010 ) ()(11→ 0 0 01 ) p. 15 ELEC 7073 Digital Communications III, Dept. of E.E.E., HKU Linear Block Codes (1) ¾ Important parameters (1) Hamming weight of a code word ci , denoted by wi=w(ci), is the number of non-zero elements in ci . Hamming distance between two code words ci and cj, is the number of elements in which they differ. ddij= ()(c,c i j=⊕ w c i c j ) Minimum distance is ddmin ==min (cij ,c ) minw (ci ) i≠ j i≠1 p. 16 ELEC 7073 Digital Communications III, Dept. of E.E.E., HKU Linear Block Codes (2) ¾ Important parameters (2) Error detection capability is given by ed= min −1 Error correcting-capability t of a code, which is defined as the maximum number of guaranteed correctable errors per codeword, is ⎢ ⎥ td=−⎣( min 12) ⎦ Copied from Proakis’s Digital Communications p. 17 ELEC 7073 Digital Communications III, Dept. of E.E.E., HKU Linear Block Codes (3) ¾ Encoding in (n,k) linear block code: The generator matrix cmG= Generally, the rows of G, { g i } , ⎡⎤g1 are linearly independent. ⎢⎥g They form a basis for the (n, (,cc ,…… , c )=⋅ ( mm , , , m )⎢⎥2 12nk 1 2 ⎢⎥ k) codes. ⎢⎥ ⎣⎦gk (,cc12 ,…… , cnkk )= m 1⋅+gg 1 m 2 ⋅++ 2 m ⋅ g p. 18 ELEC 7073 Digital Communications III, Dept. of E.E.E., HKU Example of Linear Block Code (6, 3) Hamming Message vector Codeword weight Generator matrix: 000 000000 0 Basis of the codewords 100 110100 3 ⎡⎤g 1 ⎡ 110100⎤ Gg==⎢⎥⎢ 011010⎥ 010 011010 3 ⎢⎥2 ⎢ ⎥ ⎣⎦⎢⎥g 3 ⎣⎢ 101001⎦⎥ 110 101110 4 001 101001 3 cmG= 101 011101 4 011 110011 4 111 000111 3 dmin = 3 p. 19 ELEC 7073 Digital Communications III, Dept. of E.E.E., HKU Linear Block Codes (4) ¾ Systematic linear block code (n,k) − For a systematic code, the first (or last) k elements in the codeword are information bits. GIP= [ k ] Ik =×kk identity matrix Pk =×knk( − ) matrix c ==(cc , ,..., c ) ( mm , ,..., m , p , p ,..., p ) 12nknk 1 2 1 2 − message bits parity check bits p. 20 ELEC 7073 Digital Communications III, Dept. of E.E.E., HKU Example of Systematic Linear Block Code Message codeword 0000 0000 000 (7,4) systematic code 0001 0001 011 0010 0010 110 0011 0011 101 1000101 ⎡⎤0100 0100 111 ⎢⎥010 0111 G = ⎢⎥0101 0101 100 ⎢⎥0010110 ⎢⎥0110 0110 001 ⎣⎦0001011 0111 0111 010 1000 1000 100 1001 1001 110 1010 1010 010 1011 1011 000 1100 1100 011 1101 1101 001 1110 1110 100 1111 1111 111 p.