Hadamard Matrices
Abstract Since their discovery in 1893, it has been found that Hadamard matrices have many useful applications. In this paper, we consider some examples of Hadamard matrices and how they are constructed. We also explore three applications in depth; namely factor screening, encrypting and error correcting codes, and briefly discuss some further applications.
Max Arnott Jonathan Cound Chlo´eFearn Amiee Rice Julian Villar Emma Wright
Lancaster University Supervised by Jan Grabowski Contents
1 An Introduction to Hadamard Matrices 2 1.1 Properties of Hadamard Matrices ...... 2 1.2 Construction of Hadamard Matrices ...... 3 1.2.1 Sylvester Matrices using the Kronecker Product ...... 3 1.2.2 The Paley Construction ...... 4
2 Applications 7 2.1 Factor Screening ...... 7 2.2 Encryption ...... 8 2.3 Error Correcting Codes ...... 11
1 1 An Introduction to Hadamard Matrices
We begin with the definition of a Hadamard matrix. Definition 1.1 ([1]). A Hadamard matrix, H, is a square matrix whose entries must be ±1 and that satisfies HHT = nI, where n is the order of H and I is the n × n identity matrix. We denote 1 as + and −1 as − as follows: 1 1 1 1 + + + + 1 1 −1 −1 + + − − H = = . 1 −1 1 −1 + − + − 1 −1 −1 1 + − − + Two trivial examples of Hadamard matrices are the single entry matrices where n = 1: [+] , [−] .
1.1 Properties of Hadamard Matrices There are many features and properties that define a Hadamard matrix. We discuss these below.
Proposition√ 1.2 ([2]). The rows and columns of a Hadamard matrix are pairwise orthogonal, with length n. As a direct result of Proposition 1.2, in any Hadamard matrix, if we take two neighbouring rows or columns, half of the adjacent elements have the same sign and half have the opposite sign.
Proposition 1.3. Hadamard matrices are restricted so that n = 1, 2 and 4m, m ∈ N. This proof is taken from notes made available by the University of Colorado [4]. Proof. Suppose that H is a Hadamard matrix of order n > 2. Then we multiply all columns that have the first entry as − by −1. This will result in the top row containing only + values. Rearrange the first three rows so that they look like: + ... + + ... + + ... + + ... + + ... + + ... + − ... − − ... − ...... + ... + − ... − + ... + − ... − x y z w Here, x, y, z and w are the number of columns of each type. Therefore, x + y + z + w = n. When we take the inner products of rows 1 and 2, 1 and 3, and 2 and 3 we get
x + y − z − w = 0 x − y + z − w = 0 x − y − z + w = 0.
2 Solving this system of equations gives
x = y = z = w = n/4.
Thus, the integer n must be divisible by 4.
Remark: Note that this does not prove that a Hadamard matrix exists of order 4m for every positive integer m; it was Jacques Hadamard himself who proposed this conjecture, but it is yet to be proven. Definition 1.4 ([5]). Hadamard matrices are said to be equivalent if they can be obtained from each other by a series of row/column operations which include the negation of row and columns and swapping rows or columns. Hadamard matrices of order 1, 2, 4, 8, and 12 contain a unique Hadamard matrix up to equivalence. There exist 5 non-equivalent Hadamard matrices that have order 16, 3 of order 20 and 60 of order 24.
For a long period of time, the smallest order that could not be constructed was 92, but by using computers, mathematicians at JPL [6] have since found matrices of order 4k up to 4k = 668.
1.2 Construction of Hadamard Matrices Hadamard matrices can be constructed in many ways, some of which are unfortunately beyond the scope of this paper. We explore two of the most elementary methods below.
1.2.1 Sylvester Matrices using the Kronecker Product We begin with Sylvester matrices, which are constructed using the Kronecker product of two ma- trices, which we define below.
Definition 1.5 ([7]). The Kronecker product, A ⊗ B, of an m × n matrix A and a p × q matrix B is a11B ··· a1nB . . A ⊗ B = . . . am1B ··· amnB Lemma 1.6 ([7]). The following two properties hold for all matrices A, B, C and D: • (A ⊗ B)T = AT ⊗ BT ; and • (A ⊗ B)(C ⊗ D) = AC ⊗ BD.
Proofs for the above lemma are omitted, but can be found in notes by Broxson, from the University of North Florida [8]. Theorem 1.7. [[1]] The Kronecker product of Hadamard matrices is a Hadamard matrix.
3 Proof. Let A and B be Hadamard matrices, of size m and n respectively. Then AAT = mI and BBT = nI. Since (A ⊗ B)T = AT ⊗ BT [7], we have, by lemma 1.6 (A ⊗ B)(A ⊗ B)T = (A ⊗ B)(AT ⊗ BT ) = AAT ⊗ BBT = mI ⊗ nI = mnI . So A ⊗ B is a Hadamard matrix of order mn. This gives us the means to construct Hadamard matrices of order st, where matrix A has order s and matrix B has order t.
Example 1.8 (The Sylvester Construction). Let + + A = . + − Then to construct a 4 × 4 Hadamard matrix, we can use the Kronecker product of A with itself:
+ + + + 0 AA + − + − A = A ⊗ A = = . A −A + + − − + − − + We can check that A0 is a Hadamard matrix by computing A0(A0)T and checking that it is equal to 4I4.
Theorem 1.7 gives rise to the following corollary. Corollary 1.9 ([9]). There exist Hadamard matrices of size 2k for all k ∈ N.
1.2.2 The Paley Construction A key property of the Paley construction of Hadamard matrices is that we do not require an inital Hadamard matrix to make a new one as we do when using the Sylvester construction; it is not a recursive process.
We begin with a definition of a Galois field, recalling from MATH225 [10] the definition of a field. Definition 1.10 ([9]). A Galois field, GF(s), is a field with s elements, where s is finite. Next, we define quadratic residues. Definition 1.11 ([9]). Let GF(s) be a Galois field of order s, where s = pn, for a prime number p and a natural number n. Then an element a in GF(s) is a quadratic residue if and only if there exists b in GF(s) such that a = b2. Otherwise, a is a quadratic non-residue. The quadratic character, χ(a), is defined by Wikipedia [11] as, 1 if a is a quadratic residue in GF (s), χ(a) = −1 if a is a quadratic non-residue in GF (s), 0 if a = 0.
The Jacobsthal matrix S for GF (s) is an s × s matrix with entries Sab = χ(a − b) [11]. We have two cases:
4 Case 1: s ≡ 1 mod 4, and Case 2: s ≡ 3 mod 4. Let J be an s-dimensional column vector of 1s. Then the next step for constructing the Hadamard matrix is as follows [11]: Case 1: Replace all zero entries in 0 J T JJS with the matrix + − − − and all ±1 entries with the matrix + + ± . + − Then we have a Hadamard matrix of size 2(s + 1). Case 2: 0 J T I + −JJS is a Hadamard matrix of size s + 1. Below, we use the Paley construction to create a Hadamard matrix of order 12.
Example 1.12 (Paley Construction). Consider the Galois field Z5 = {0, 1, 2, 3, 4}. Then we have 12 ≡ 1, 22 ≡ 4, 32 ≡ 9 ≡ 4, and 42 ≡ 16 ≡ 1.
So 1 and 4 are quadratic residues in Z5, whilst 2 and 3 are quadratic non-residues in Z5. So χ(0) = 0, χ(1) = 1, χ(2) = −1, χ(3) = −1, and χ(4) = 1.
Then by computing χ(a − b) for each entry in the Jacobian matrix S, we obtain
0 + − − + + 0 + − − S = − + 0 + − . − − + 0 + + − − + 0
5 Then, since 5 ≡ 1 mod 4, we can take
0 J T JJS and replace all 0s, 1s, and −1s with
+ − + + − − , , and − − + − − + respectively, which gives
+ − + + + + + + + + + + − − + − + − + − + − + − + + + − + + − − − − + + + − − − + − − + − + + − + + + + + − + + − − − − + − + − − − + − − + − + H = . + + − − + + + − + + − − + − − + + − − − + − − + + + − − − − + + + − + + + − − + − + + − − − + − + + + + − − − − + + + − + − + − − + − + + − − −
We can check that H is Hadamard by multiplying it with its transpose, and we find that
HHT = 12I.
Sylvester first proposed a method of construction that produced Hadamard matrices of order 2n for all positive integers n. The Paley construction produces Hadamard matrices of order either q + 1 or 2(q + 1) where q is congruent to 3 or 1 mod 4 respectively.
6 2 Applications 2.1 Factor Screening Factor screening is an important application of Hadamard matrices. In factor screening, we investi- gate which factors in an experiment are having the largest effect on the response. We begin by using linear regression which investigates the effects of the explanatory variables on the response variable; this is exactly what we investigate in factor screening. The multiple linear regression model is as follows, as stated in part 2.2 of Jin’s work on factor screening [12]:
For i = 1, . . . , n,
Yi = β0 + β1xi,1 + β2xi,2 + ... + βpxi,p + i where the residuals 1, . . . , n, are independent and identically distributed and follow a normal distribution. So for i = 1, . . . , n,
2 i ∼ N(0, σ ) . We can also write this as follows [12]:
Y = Xββ + 0 0 where Y = (Y1,...,Yn) is the response vector , X is the design matrix, β = (β1, . . . , βp) is the 0 vector of coefficients and = (1, . . . , p) is the residual vector.
The design matrix X is an n × p matrix whose columns represent the explanatory variables. The first column in the design matrix is a column of 1s if there is an intercept term in the linear regres- sion model.
As stated in Jin’s thesis [12], if we conduct a 2-level design experiment, where we investigate k factors at 2 levels, then we need 2k runs. The different levels of the factor are recorded as either +1 or −1; we usually use −1 for the first level and +1 for the second level. Because we label the factors with ±1 the matrix created is orthogonal. This means that the resulting matrix is in fact a Hadamard matrix, since it contains only ±1 and has orthogonal columns.
Example 2.1 (Factor Screening). Suppose we wish to do an investigation at 2 levels with 2 factors, A and B, as has been conducted in the ReliaWiki article [13]. Then we have 2k = 22 = 4 runs, 2 main effects A and B and an interaction term AB.
Taken from ReliaWiki [13], the experiment might appear as follows: A B (1) -1 -1 a 1 -1 b -1 1 ab 1 1
7 and this gives us the design matrix
I A B AB + − − + + + − − X = + − + − + + + +
In order to see which factors are the estimated main effects we must calculate 2βˆ where, as stated in Jin’s work [12],
βˆ = (XT X)−1XT YY. For example, if
βˆ = (1, −0.2) , this implies that factor A (left hand side) has a larger impact than factor B (right hand side) since 1 > −0.2. It also tells us that factor A has a large positive effect, whilst factor B has a smaller negative effect. Sometimes we may get three values in the vector βˆ to include the effect of the interaction term AB. We can also use Plackett-Burman designs for screening as explained in the Engineering Statis- tics Handbook [14], since they allow us to evaluate lots of factors quickly in order to see which are the most important ones. The Plackett-Burman designs which represent the experiment are same as the design matrix X previously described. Plackett-Burman designs have runs which are multiples of 4, and a 12 run experiment may contain up to 11 factors to be tested. This means that Plackett-Burman designs can sometimes be more efficient than following the regression based model mentioned above. However, as stated by Steele [15], Plackett-Burman designs don’t provide information about whether the effect of one factor depends on another.
As explained in [17], we create Plackett-Burman designs by setting the first row to be a cyclic arrangement of the N − 1 elements, where N is the number of rows of our arrangement. We then create the second row by shifting all the elements from the first row one place to the right. We repeat this process until N − 1 rows have been created. For the final row, we add a row of −1s. We can then evaluate this matrix in a similar way to as previously mentioned to determine which factors are the main effects in the investigation.
2.2 Encryption In this section we will discuss an encryption scheme that uses Hadamard matrices with a circulant core. This method of encryption is popular as it is sturdy against the brute force, plaintext, and ciphertext varieties of cyber attack. A successful encryption system should aim to satisfy the following objectives:
• only one key can be used to decode
• uses a small sized key
8 • is resistant to cryptographic attacks • is not overly computationally strenuous A brute force method of code breaking is one that involves trying an large number of possible keys. In a plaintext attack, the attacker tries to derive the key using a known encrypted version of an arbitrary message to obtain information about the key. In a ciphertext attack, the attacker uses decrypted versions of known encrypted messages to gather information, so in some sense is the reverse of a plaintext attack [18] [19].
Assume the message we wish to send uses the Latin alphabet and contains n letters. We can represent this message as a vector m with length n, such that each element of m corresponds to a letter of the alphabet, and that the order of the characters in our message is the order of their corresponding numbers in m. For simplicity, let (a, b, c,..., z) correspond to (1, 2, 3,..., 26) with the number 27 corresponding to a space. We could use extra numbers to correspond to punctuation, numbers themselves, or any other characters, but we will not for this example. There is also no need for the elements of m to necessarily be natural numbers. If there is not an n × n Hadamard matrix for the length of your message (suppose your message has an odd number of characters), simply put some spaces at the end of your message before you find your m so that n will be valid.
T We will also make use of a Hadamard matrix A ∈ Mn({1, −1}), with AA = nIn, a chosen constant d ∈ R, and a vector of length n of 1s that we will call en.
We will encrypt the message using the formula:
c = mmA + deen . Then c is our encrypted version of m which we can now transmit securely. The key that we need to decode this message is the pair (A, d). This key will decrypt c uniquely; there is no possibility that using the key correctly on c will output a vector other than m, which can easily be translated back into letters.
To decrypt our message we can simply rearrange the encryption formula as such:
c = mmA + deen
c − deen = mmA T T (c − deen)A = mmAA = mmn −1 T n (c − deen)A = m . Clearly, if we know A, c, and d, we can find m from this formula (n can be figured out from A). The decryption depends on the fact that A is Hadamard because it requires AAT to be a multiple of the identity matrix In.
This method satisfies the four objectives listed earlier for short messages; its key is unique (proof omitted), the key consists of just a matrix and a constant, making it computationally simple to encrypt and decrypt as it only relies on basic matrix multiplication, and it is resistant to crypto- graphic attacks.
9 This method is resistant to brute force attacks because there are infinite choices for d and an extreme amount of Hadamard matrices to choose from when n is large, so for a computer to try to crack our code by guessing many values it is unlikely to succeed. It is resistant to plaintext attacks because the key is in two parts; the vector was matrix multiplied by A and then each element translated by d to encode. They would be reduced to guessing since they have no clue how much they should translate their vector by before they have to multiply by a matrix that they don’t know, so knowing an encrypted and decrypted version of the same message will not help an attacker obtain the key. Finally, it is resistant to a ciphertext attack for the same reason, since the process of encryption and decryption depends on the same key. For this method of encryption, there is no advantage nor disadvantage to using a ciphertext approach over a plaintext approach. Clearly, the larger the size of the Hadamard matrix we choose, the more possible matrices there are and so the more difficult it is to retrieve the key [20]. Example 2.2 (Encoding a Message). Let our message to encrypt be “matrices”. Then our message vector m is (13, 1, 20, 18, 9, 3, 5, 19). We can see that our code will require an 8×8 Hadamard matrix. We will choose d = 3, and
+ + + + + + + + + − + − + − + − + + − − + + − − + − − + + − − + T T A = ,A = A, AA = 8I8 . + + + + − − − − + − + − − + − + + + − − − − + + + − − + − + + −
This gives c = mmA + 3e8 = (91, 9, −33, 33, 19, 25, −9, −7). This is our encrypted message.
Now, to return m from c we simply use the formula from above. Since
T AA = 8I8, x = 8, −1 T m = x (c − 3e8)A
+ + + + + + + + + − + − + − + − + + − − + + − − 1 + − − + + − − + = (88, 6, −36, 30, 16, 22, −12, −10) 8 + + + + − − − − + − + − − + − + + + − − − − + + + − − + − + + − 1 = (104, 8, 160, 144, 72, 24, 40, 152) = (13, 1, 20, 18, 9, 3, 5, 19) 8
10 Since we know that each number i in our decoded vector corresponds to the ith letter of the alphabet, we can convert m into our message: “matrices”.
2.3 Error Correcting Codes Error correcting codes are a way of expressing a sequence of numbers in such a way that any errors which have been introduced can, up to a certain limit, be corrected based on the remaining numbers [21].
To use Hadamard matrices to produce an error correcting code, we take a 4m × 4m symmetric Sylvester-Hadamard matrix. Between the matrices H4m and −H4m there are 8m rows, and pos- sible codewords, in a system using these Hadamard matrices. The Hadamard matrix of 1s and −1s can be converted to 1s and 0s in order that the Hadamard code be used with a system which transmits data in binary form from Malek’s work on Hadamard codes [22].
A Hadamard code can be used by allocating any form of information to a particular codeword and then having a method to decode the received codewords at the end, which we see in a example 2.3, using the H8 matrix produced by the Kronecker product as shown in definition 1.5. Example 2.3. Error Correcting
+ + + + + + + + + − + − + − + − + + − − + + − − + − − + + − − + H8 = H2 ⊗ H2 ⊗ H2 = + + + + − − − − + − + − − + − + + + − − − − + + + − − + − + + −
If a codeword, a, is received, we firstly replace the 0s with −1s, and call this a0 . We then calculate 0 0 b = a H8. If a is a row of H8 then the vector b will have 7 zero elements and one element of 8 0 (or −8 if a is a row of −H8) as stated by van Lint [23], because the inner product of a row of an n × n Hadamard matrix with itself is n and the inner product of two different rows of a Hadamard matrix is 0. The correct codeword is the one corresponding to the position of 8 in b (so if 8 is in the second position in the matrix then the second row of H8 is the correct codeword) [22].
If a = (1, 0, 0, 1, 0, 1, 1, 0), a0 = (1, −1, −1, 1, −1, 1, 1, −1), 0 and so b = a H8 = (0, 0, 0, 0, 0, 0, 0, 8), and so the eighth row of H8 is selected as the correct codeword.
As a Hadamard code with matrices of order 4m can correct m − 1 errors, this code can correct one error in the 8 digits of a codeword. If there is one error in the codeword, then b will contain one
11 element with absolute value at least 6, and no other element greater than 2.
If a = (1, 1, 1, 1, 0, 1, 0, 0), 0 then a = (1, 1, 1, 1, −1, 1, −1, −1) (which is not a row of H8 or −H8.), b = (2, −2, 2, −2, 6, 2, −2, 2); and as the element with the greatest absolute value, 6, is in the fifth position in b, we select the fifth row of H8.
a = (0, 0, 0, 1, 1, 1, 0, 0), so a0 = (−1, −1, −1, 1, 1, 1, −1, −1), and b = (−2, −2, 2, 2, −2, −2, −6, 2).
As the element with the greatest absolute value is −6 in the seventh position in b, we select the seventh row of −H8.
If there are two errors then b contains a number of equally large values not greater than 4
a = (1, 0, 1, 0, 1, 1, 1, 1), and so a0 = (1, −1, 1, −1, 1, 1, 1, 1), then b = (4, 4, 0, 0, −4, 4, 0, 0); and the message cannot be decoded [22]. This method was used in the Mariner 9 spaceprobe to transmit pictures taken by the probe to Earth [22]. The black and white pictures had 64 shades, and each shade could be assigned to one of the possible codes using in the H32 and −H32 matrices. If for example the probability of any particular digit being altered by noise and being incorrectly received on Earth were 5%, this method could reduce the probability of the wrong message being received to just 0.01% - a significant improvement over the error rate of 26% if each digit were simply transmitted once without any form of coding, and also over the 4% probability of errors if an alternative code was used [23], simply repeating each digit 5 times—this method would allow 2 errors to occur without changing the interpretation of the picture, whereas the Hadamard code allowed the correction of up to 7 errors [23]. This Hadamard code could importantly be implemented without requiring transmission of significantly more data than the five repeats code, thus allowing the use of the Hadamard code on a spacecraft with limited capacity to store or transmit data [22].
12 References
[1] P.J Cameron. Hadamard Matrices. Encyclopedia of Design Theory. http://designtheory.org/library/encyc/topics/had.pdf [2] Mathworld article. Hadamard Matrix. http://mathworld.wolfram.com/HadamardMatrix.html [3] University of Illinois at Chicago lecture slides. Hadamard Matrices and Hadamard Codes. http://homepages.math.uic.edu/~leon/mcs425-s08/handouts/Hadamard_codes.pdf [4] University of Colorado lecture slides. Hadamard Matrices and Designs. http://math.ucdenver.edu/~wcherowi/courses/m6406/hadamard.pdf [5] Wikipedia article. Hadamard Matrices. https://en.wikipedia.org/wiki/Hadamard_matrix [6] L. Baumert, S. W. Golomb and M. Hall Jr. Discovery of an Hadamard Matrix of Order 92. Bulletin of the American Mathematical Society, 68 (1962), 237-238. http://www.ams.org/journals/bull/1962-68-03/S0002-9904-1962-10761-7/home.html [7] K. Sch¨acke. On the Kronecker Product. University of Waterloo. 2004. http://www.math.uwaterloo.ca/~hwolkowi/henry/reports/kronthesisschaecke04.pdf [8] B. J. Broxzon. The Kronecker Product. University of North Florida. 2006. http://digitalcommons.unf.edu/cgi/viewcontent.cgi?article=1025&context=etd [9] A. M. Leghwel. On Some Methods of Constructing Hadamard Matrices. Journal of Humanities and Applied Science. http://www.asmarya.edu.ly/journal2/wp-content/uploads/2015/09/Abdurzak-M. -Leghwel3.pdf [10] J. Grabowski. MATH225 lecture notes. Lancaster University. [11] Wikipedia article. Paley Construction. https://en.wikipedia.org/wiki/Paley_construction [12] Y. Jin. Factor Screening in a 12 Run Plackett-Burman Design Assuming Four Active Factors. https://brage.bibsys.no/xmlui/bitstream/handle/11250/2433757/15731_FULLTEXT. pdf?sequence=1 [13] ReliaWiki article. Two Level Factorial Experiments. http://reliawiki.org/index.php/Two_Level_Factorial_Experiments [14] Engineering Statistics Handbook. Plackett-Burman Designs. http://www.itl.nist.gov/div898/handbook/pri/section3/pri335.htm [15] C. Steele. What the Heck is a Plackett-Burman Design, and Why Would I Want It? The Minitab Blog. http://blog.minitab.com/blog/statistics-and-quality-improvement/ what-the-heck-is-a-plackett-burman-design-and-why-would-i-want-it
13 [16] Wikipedia article. Hadamard’s Inequalities. https://en.wikipedia.org/wiki/Hadamard\%27s_inequality [17] H. Evangelaras, C. Koukouvinos, and J. Seberry. Applications of Hadamard Matrices. Univer- sity of Wollongong.
[18] Wikipedia article. Chosen Plaintext. https://en.wikipedia.org/wiki/Chosen-plaintext_attack [19] Wikipedia article. Chosen Ciphertext Attack. https://en.wikipedia.org/wiki/Chosen-ciphertext_attack\#Introduction [20] Scienpress article. http://www.scienpress.com/Upload/JAMB/Vol\%203_1_2.pdf [21] Mathworld article. http://mathworld.wolfram.com/Error-CorrectingCode.html [22] M. Malek. Hadamard Codes. California State University, East Bay. http://www.mcs.csueastbay.edu/~malek/TeX/Hadamard.pdf [23] J. H. van Lint. Coding, Decoding, and Combinatorics. In R. Wilson (Ed.), Applications of Combinatorics (pp. 67-74). (Shiva Mathematics Series; Vol. 6). Nantwick: Shiva. 1982.
14