CSCI-B609: A Theorist’s Toolkit, Fall 2016 Oct 6 Lecture 14: Hamming and Hadamard Codes
Lecturer: Yuan Zhou Scribe: Kaiyuan Zhu
1 Recap
Recall from the last lecture that error-correcting codes are in fact injective maps from k symbols to n symbols in Σ, Enc: Σk → Σn where k and n are referred to as the message dimension and block length respectively. We also call the image of the encoding function code, which is usually denoted by C, i.e. C = Im(Enc); and an element y ∈ C a codeword. The minimum distance d is defined as the smallest Hamming distance between two distinct codewords, d = min {∆(y1, y2)} = min |{i : y1i 6= y2i}| y16=y2∈C y16=y2∈C We want d to be large so that more errors can be tolerated, but this makes the number of k vertices we can put in Σn smaller. Therefore we have to sacrifice the rate to generate the n same number of codeword. In many ways, coding theory is about exploring a tradeoff.
2 Linear Codes
In coding theory, a linear code is an error-correcting code for which any linear combination of codewords is still a codeword. Linear codes have the following advantages: i. easy to figure out the minimum distance; and ii. simple encoding and decoding algorithms.
Definition 1. (Linear code) Let Σ = Fq be a finite field with q elements, then C is linear n n×k if ∀y1, y2 ∈ C ⊆ Fq , y1 + y2 ∈ C. In other words, let G ∈ Fq be a full rank n × k matrix k n (making the map injective), then Enc: Fq → Fq becomes x 7→ Gx, which defines a linear code with its generator matrix G.
1 0 Example. Let q = 2, n = 3 and k = 2. Then the generator matrix G = 0 1 , so that 1 1
1 Lecture 14: Hamming and Hadamard Codes 2
x1 ( 0 0 1 1 ) x1 G · = x2 . Thus C = Im(G) = 0 , 1 , 0 , 1 . x2 x1 + x2 0 1 1 0
Note that for linear codes, we introduce the following notation [n, k(, d)]q henceforth, where n is the block length, k is the message dimension, and d is the minimum distance if known.
n Definition 2. (Hamming weight) The Hamming weight of x ∈ Fq in a linear code is denoted by wt(x) = ∆(x, 0). Fact 1. In a linear code, the minimum distance d is equal to the minimum Hamming weight of a nonzero codeword.
Proof.
d = min {∆(y1, y2)} = min {∆(y1 − y2, 0)} = min {wt(y)} y16=y2∈C y16=y2∈C y=y1−y26=0∈C
⊥ Definition 3. (Dual code) Given [n, k]q code C, denote the orthogonal space C , {y ∈ n T ⊥ Fq : y x = 0, ∀x ∈ C} as the dual code of C. Note that C has parameters [n, n − k]q. Definition 4. (Parity check matrix) The parity check matrix H of C is defined as an (n − ⊥ ⊥ ⊥ n−k n T k) × n matrix such that C = Im(Enc ), where Enc : Fq → Fq maps w to H w. In other words, HT is the generator matrix of C⊥.
Example. Reconsider the previous example, in which
( 0 0 1 1 ) C = 0 , 1 , 0 , 1 0 1 1 0
( 0 1 ) Therefore C⊥ = 0 , 1 and H = (1, 1, 1). 0 1 Fact 2. y ∈ C ⇔ Hy = 0. (re-express the code as null space of the parity check matrix)
Proof. Notice that HT is the generator matirx of C⊥, i.e. C⊥ is the row span of H. Let T T h1 x = 0 h1 T hT h x = 0 n−k 2 2 X T H = . , then Hx = 0 ⇔ ⇔ ∀a1, a2, ··· , an−k ∈ q, aih x = . . F i . . i=1 T hn−k T hn−kx = 0 0 ⇔ ∀y ∈ C⊥, yT x = 0 ⇔ x ∈ (C⊥)⊥ = C Lecture 14: Hamming and Hadamard Codes 3
Corollary 3. The minimum distance d is the minimum number of columns in H that are linearly dependent.
Proof. d = min {wt(y)} = min{wt(y) | y 6= 0, Hy = 0}. y6=0∈C
Hamming code [1] is defined by the case of linear code that q = 2, which has excellent rate k ≈ 1 but lower distance as we will see later. n Definition 5. (Hamming code) Let r ∈ N+. Define the parity check matrix of a Hamming code as 0 0 0 ··· 1 0 0 0 ··· 1 . . . . . H = . . . . . 0 1 1 ··· 1 1 0 1 ··· 1
r×(2r−1) r i.e. H ∈ F2 , which is spanned by all distinct 2 − 1 nonzero column vectors.
( 0 1 ) 0 1 1 Example. For r = 2, H = , and C = 0 , 1 . 1 0 1 0 1
r r Theorem 4. Hamming code is [2 − 1, 2 − 1 − r, 3]2 code.
Proof. We only need to prove d = 3, which is equivalent to say the minimum number of linearly dependent column is 3. Since 0 is not a column of H, every 2 cloumns are linearly independent. But there exists obviously triple of linearly dependent columns, such 0 0 0 0 . . . . . . . . as, + + = . 0 1 1 0 1 0 1 0
r Remark. Let n = 2 − 1, then Hamming code is [n, n − log2(n + 1), 3]2 code. 3 Since the distance is 3, Hamming code is uniquely decodable for up to = 1 error. In 2 fact, we can correct one error easily. Let y ∈ C be any codeword, and z = y + ei be the received message. Then Hz = H(y + ei) = Hei Lecture 14: Hamming and Hadamard Codes 4
which is just the i the column of H. Otherwise Hz = 0 implies that y is not modified. For 1 1 1 0 1 1 1 example, with y = 1 and z = 1 , Hz = · 1 = . This 1 0 1 1 1 0 0 indicates that index 3 has changed.
Definition 6. (Perfect code) C is a perfect code if Hamming balls centered at codewords of radius t (i.e. max errors) can partition Σn exactly.
Theorem 5. Hamming code is perfect.
n Proof. ∀x ∈ F2 , if Hx = 0, then x ∈ C. Otherwise Hx = hi, where hi is the i-th column of H. Hence H(x + ei) = 0 and therefore x + ei ∈ C.
The Hadamard code is a code with extremely low rate but high distance. It is always used for error detection and correction when transmitting messages over very noisy or unreliable channels.
Definition 7. (Hadamard Code) Let r ∈ N+. The generator matrix of Hadamard code is a r r 2 × r matrix where the rows are all possible binary strings in F2.
0 0 0 1 Example. For r = 2, we have G = , which maps the messages to Gx = 1 0 1 1 0 0 0 0 ( ) 0 1 0 1 , , , . 0 0 1 1 0 1 1 0
r r Fact 6. Hadamard code is a [2 , r, 2 − 1]2 code.
Proof. It suffices to prove the minimum weight of a nonzero codeword is 2r − 1. Let x 6= 0 ∈ Lecture 14: Hamming and Hadamard Codes 5
n F2 , i.e. ∃k s.t. xk = 1. Then
wt(Gx) T = r [g x = 1] 2r Pi∈[2 ] i T = r [y x = 1] Py∈F2 X 0 = P 0 [r]\{k} ykxk + yixi = 1 y ∈F2 ,yk∈F2 i6=k X 0 1 = Ey0∈ [r]\{k} Pyk∈F2 yixi = 1 + yk = F2 2 i:i6=k
T where gi denote the i-th row of G. n Remark. In other words, Hadamard code is [n, log n, ] code with n = 2r. 2 2 2
Reference
[1] Hamming, R. W. (1950). Error detecting and error correcting codes. Bell System tech- nical journal, 29(2), 147-160.
[2] http://www.cs.cmu.edu/~odonnell/toolkit13/lecture10.pdf