Hadamard Matrices and Orthonormal Bases
Calvin Hotchkiss
CBMS Boot Camp Iowa State University
June 2, 2018
Hadamard Matrices and ONB Iowa State Let v T be the transpose of a vector (or matrix) v, v its (pointwise) conjugate, and v ∗ its conjugate transpose.
Definition 1 (Inner product) T T Let x = (x1, x2,..., xn) and y = (y1, y2,..., yn) be column n ∗ vectors in C . Then hx, yi = y x.
Two vectors x and y are called orthogonal if hx, yi = 0.
The norm of a vector x, ||x|| = phx, xi.
Preliminaries
n T We start out today in C := {(x1, x2,... xn) : xj ∈ C}. This is a complete vector space with an inner product, what we call a Hilbert Space.
Hadamard Matrices and ONB Iowa State Definition 1 (Inner product) T T Let x = (x1, x2,..., xn) and y = (y1, y2,..., yn) be column n ∗ vectors in C . Then hx, yi = y x.
Two vectors x and y are called orthogonal if hx, yi = 0.
The norm of a vector x, ||x|| = phx, xi.
Preliminaries
n T We start out today in C := {(x1, x2,... xn) : xj ∈ C}. This is a complete vector space with an inner product, what we call a Hilbert Space.
Let v T be the transpose of a vector (or matrix) v, v its (pointwise) conjugate, and v ∗ its conjugate transpose.
Hadamard Matrices and ONB Iowa State Two vectors x and y are called orthogonal if hx, yi = 0.
The norm of a vector x, ||x|| = phx, xi.
Preliminaries
n T We start out today in C := {(x1, x2,... xn) : xj ∈ C}. This is a complete vector space with an inner product, what we call a Hilbert Space.
Let v T be the transpose of a vector (or matrix) v, v its (pointwise) conjugate, and v ∗ its conjugate transpose.
Definition 1 (Inner product) T T Let x = (x1, x2,..., xn) and y = (y1, y2,..., yn) be column n ∗ vectors in C . Then hx, yi = y x.
Hadamard Matrices and ONB Iowa State The norm of a vector x, ||x|| = phx, xi.
Preliminaries
n T We start out today in C := {(x1, x2,... xn) : xj ∈ C}. This is a complete vector space with an inner product, what we call a Hilbert Space.
Let v T be the transpose of a vector (or matrix) v, v its (pointwise) conjugate, and v ∗ its conjugate transpose.
Definition 1 (Inner product) T T Let x = (x1, x2,..., xn) and y = (y1, y2,..., yn) be column n ∗ vectors in C . Then hx, yi = y x.
Two vectors x and y are called orthogonal if hx, yi = 0.
Hadamard Matrices and ONB Iowa State Preliminaries
n T We start out today in C := {(x1, x2,... xn) : xj ∈ C}. This is a complete vector space with an inner product, what we call a Hilbert Space.
Let v T be the transpose of a vector (or matrix) v, v its (pointwise) conjugate, and v ∗ its conjugate transpose.
Definition 1 (Inner product) T T Let x = (x1, x2,..., xn) and y = (y1, y2,..., yn) be column n ∗ vectors in C . Then hx, yi = y x.
Two vectors x and y are called orthogonal if hx, yi = 0.
The norm of a vector x, ||x|| = phx, xi.
Hadamard Matrices and ONB Iowa State n Any basis B = {b1, b2,..., bn} for C has the property that, for n any v ∈ C , there exist unique a1, a2,..., an in C such that:
v = a1b1 + a2b2 + ··· + anbn.
An orthonormal basis is particularly convenient, since it has the property that aj = hv, bj i for all j, so that
n v = hv, bj i bj . j=1 X
Orthonormal Bases
A set of mutually orthogonal vectors is called orthonormal if they n all have norm 1. If they are additionally a basis for C , they are called an orthonormal basis.
Hadamard Matrices and ONB Iowa State An orthonormal basis is particularly convenient, since it has the property that aj = hv, bj i for all j, so that
n v = hv, bj i bj . j=1 X
Orthonormal Bases
A set of mutually orthogonal vectors is called orthonormal if they n all have norm 1. If they are additionally a basis for C , they are called an orthonormal basis.
n Any basis B = {b1, b2,..., bn} for C has the property that, for n any v ∈ C , there exist unique a1, a2,..., an in C such that:
v = a1b1 + a2b2 + ··· + anbn.
Hadamard Matrices and ONB Iowa State Orthonormal Bases
A set of mutually orthogonal vectors is called orthonormal if they n all have norm 1. If they are additionally a basis for C , they are called an orthonormal basis.
n Any basis B = {b1, b2,..., bn} for C has the property that, for n any v ∈ C , there exist unique a1, a2,..., an in C such that:
v = a1b1 + a2b2 + ··· + anbn.
An orthonormal basis is particularly convenient, since it has the property that aj = hv, bj i for all j, so that
n v = hv, bj i bj . j=1 X
Hadamard Matrices and ONB Iowa State u1 Let u1 = v1, then e1 = . ||u1||
u2 Let u2 = v2 − hv2, e1i e1. Then e2 = . ||u2||
u3 Let u3 = v3 − hv3, e1i e1 − hv3, e2i e2. Then e3 = . Continue ||u3|| until: uk uk = vk − hvk , e1i e1 − ··· − hvk , ek−1i ek−1. Then ek = . ||uk ||
The Gram–Schmidt Process
Given a set of linearly independent vectors S = {v1,..., vk }, the Gram–Schmidt process generates an orthonormal set 0 S = {e1,..., ek } that spans the same subspace as S. In the process it will generate a set {u1,..., uk } that is orthogonal but not necessarily orthonormal.
Hadamard Matrices and ONB Iowa State u2 Let u2 = v2 − hv2, e1i e1. Then e2 = . ||u2||
u3 Let u3 = v3 − hv3, e1i e1 − hv3, e2i e2. Then e3 = . Continue ||u3|| until: uk uk = vk − hvk , e1i e1 − ··· − hvk , ek−1i ek−1. Then ek = . ||uk ||
The Gram–Schmidt Process
Given a set of linearly independent vectors S = {v1,..., vk }, the Gram–Schmidt process generates an orthonormal set 0 S = {e1,..., ek } that spans the same subspace as S. In the process it will generate a set {u1,..., uk } that is orthogonal but not necessarily orthonormal.
u1 Let u1 = v1, then e1 = . ||u1||
Hadamard Matrices and ONB Iowa State u3 Let u3 = v3 − hv3, e1i e1 − hv3, e2i e2. Then e3 = . Continue ||u3|| until: uk uk = vk − hvk , e1i e1 − ··· − hvk , ek−1i ek−1. Then ek = . ||uk ||
The Gram–Schmidt Process
Given a set of linearly independent vectors S = {v1,..., vk }, the Gram–Schmidt process generates an orthonormal set 0 S = {e1,..., ek } that spans the same subspace as S. In the process it will generate a set {u1,..., uk } that is orthogonal but not necessarily orthonormal.
u1 Let u1 = v1, then e1 = . ||u1||
u2 Let u2 = v2 − hv2, e1i e1. Then e2 = . ||u2||
Hadamard Matrices and ONB Iowa State The Gram–Schmidt Process
Given a set of linearly independent vectors S = {v1,..., vk }, the Gram–Schmidt process generates an orthonormal set 0 S = {e1,..., ek } that spans the same subspace as S. In the process it will generate a set {u1,..., uk } that is orthogonal but not necessarily orthonormal.
u1 Let u1 = v1, then e1 = . ||u1||
u2 Let u2 = v2 − hv2, e1i e1. Then e2 = . ||u2||
u3 Let u3 = v3 − hv3, e1i e1 − hv3, e2i e2. Then e3 = . Continue ||u3|| until: uk uk = vk − hvk , e1i e1 − ··· − hvk , ek−1i ek−1. Then ek = . ||uk ||
Hadamard Matrices and ONB Iowa State The columns (and also the rows) of U form an orthonormal basis n for C .
n If B = {b1, b2,..., bn} is an orthonormal basis for C , then UB = {Ub1, Ub2,..., Ubn} is also an orthornomal basis for B.
n If x is a column vector in C , written in terms of the standard basis, then Ux is the same vector written in terms of the ONB formed by the columns of U.
Unitary matrices
∗ ∗ ∗ An n × n matrix U is called unitary if U U = UU = In, where U represents the conjugate transpose of the matrix U.
Hadamard Matrices and ONB Iowa State n If B = {b1, b2,..., bn} is an orthonormal basis for C , then UB = {Ub1, Ub2,..., Ubn} is also an orthornomal basis for B.
n If x is a column vector in C , written in terms of the standard basis, then Ux is the same vector written in terms of the ONB formed by the columns of U.
Unitary matrices
∗ ∗ ∗ An n × n matrix U is called unitary if U U = UU = In, where U represents the conjugate transpose of the matrix U.
The columns (and also the rows) of U form an orthonormal basis n for C .
Hadamard Matrices and ONB Iowa State Unitary matrices
∗ ∗ ∗ An n × n matrix U is called unitary if U U = UU = In, where U represents the conjugate transpose of the matrix U.
The columns (and also the rows) of U form an orthonormal basis n for C .
n If B = {b1, b2,..., bn} is an orthonormal basis for C , then UB = {Ub1, Ub2,..., Ubn} is also an orthornomal basis for B.
n If x is a column vector in C , written in terms of the standard basis, then Ux is the same vector written in terms of the ONB formed by the columns of U.
Hadamard Matrices and ONB Iowa State This means that the columns (and also the rows) of H form an n √ orthogonal basis for C , with each vector having norm n.
Sometimes the term Hadamard matrix refers to the scaled √1 version, n H, which is also a unitary matrix.
A major class of Hadamard matrices are the discrete Fourier transform matrices, which exist for all dimensions n > 1.
Hadamard matrices
Definition 2 An n × n matrix H is called a (complex) Hadamard matrix if 1. all of its entries have norm 1, ∗ 2. H H = nIn.
Hadamard Matrices and ONB Iowa State Sometimes the term Hadamard matrix refers to the scaled √1 version, n H, which is also a unitary matrix.
A major class of Hadamard matrices are the discrete Fourier transform matrices, which exist for all dimensions n > 1.
Hadamard matrices
Definition 2 An n × n matrix H is called a (complex) Hadamard matrix if 1. all of its entries have norm 1, ∗ 2. H H = nIn.
This means that the columns (and also the rows) of H form an n √ orthogonal basis for C , with each vector having norm n.
Hadamard Matrices and ONB Iowa State A major class of Hadamard matrices are the discrete Fourier transform matrices, which exist for all dimensions n > 1.
Hadamard matrices
Definition 2 An n × n matrix H is called a (complex) Hadamard matrix if 1. all of its entries have norm 1, ∗ 2. H H = nIn.
This means that the columns (and also the rows) of H form an n √ orthogonal basis for C , with each vector having norm n.
Sometimes the term Hadamard matrix refers to the scaled √1 version, n H, which is also a unitary matrix.
Hadamard Matrices and ONB Iowa State Hadamard matrices
Definition 2 An n × n matrix H is called a (complex) Hadamard matrix if 1. all of its entries have norm 1, ∗ 2. H H = nIn.
This means that the columns (and also the rows) of H form an n √ orthogonal basis for C , with each vector having norm n.
Sometimes the term Hadamard matrix refers to the scaled √1 version, n H, which is also a unitary matrix.
A major class of Hadamard matrices are the discrete Fourier transform matrices, which exist for all dimensions n > 1.
Hadamard Matrices and ONB Iowa State Discrete Fourier Transform Matrices
Let ω = e−2πi/n, a primitive nth root of 1. A discrete Fourier transform (DFT) matrix has the form
jk Fn = ω j,k=0,...,n−1 or: 1 1 1 1 ··· 1 2 3 n−1 1 ω ω ω ··· ω 1 ω2 ω4 ω6 ··· ω2(n−1) Fn = 1 ω3 ω6 ω9 ··· ω3(n−1) . ...... ...... 1 ωn−1 ω2(n−1) ω3(n−1) ··· ω(n−1)(n−1)
Hadamard Matrices and ONB Iowa State Some small DFT matrices: 1 1 1 1 1 F = (1) F = F = 1 e−2πi/3 e−4πi/3 1 2 1 −1 3 1 e−4πi/3 e−2πi/3
Discrete Fourier Transform
T Multiplying a column vector x = (x0, x1,..., xn−1) by DFT matrix applies the discrete Fourier transform to it:
n−1 − 2πi kj (Fnx)k = xj · e n j=0 X which is the definition of the discrete Fourier Transform.
Hadamard Matrices and ONB Iowa State Discrete Fourier Transform
T Multiplying a column vector x = (x0, x1,..., xn−1) by DFT matrix applies the discrete Fourier transform to it:
n−1 − 2πi kj (Fnx)k = xj · e n j=0 X which is the definition of the discrete Fourier Transform.
Some small DFT matrices: 1 1 1 1 1 F = (1) F = F = 1 e−2πi/3 e−4πi/3 1 2 1 −1 3 1 e−4πi/3 e−2πi/3
Hadamard Matrices and ONB Iowa State For n = 4, all Hadamard matrices are equivalent to a matrix of the form: 1 1 1 1 1 ieia −1 −ieia 1 −1 1 −1 1 −ieia −1 ieia for some a ∈ [0, π).
Small Hadamard Matrices
For n = 1, 2, 3 and 5, all Hadamard matrices are equivalent to the DFT matrix of the respective size.
Hadamard Matrices and ONB Iowa State Small Hadamard Matrices
For n = 1, 2, 3 and 5, all Hadamard matrices are equivalent to the DFT matrix of the respective size.
For n = 4, all Hadamard matrices are equivalent to a matrix of the form: 1 1 1 1 1 ieia −1 −ieia 1 −1 1 −1 1 −ieia −1 ieia for some a ∈ [0, π).
Hadamard Matrices and ONB Iowa State Construction of Hadamard Matrices
T ∗ I If H is a Hadamard matrix, so are H , H, and H . These may or may not be equivalent to H or to each other.
I If A and B are n × n Hadamard matrices, so is AB H = A −B
and for any diagonal unitary matrix E,
A EB H = A −EB
I If A and B are Hadamarad matrices, than their tensor product A ⊗ B is also a Hadamard matrix.
Hadamard Matrices and ONB Iowa State Dit¸ˇa’sconstruction
If A is a K × K Hadamard matrix, B is an M × M Hadamard matrix, and E1,..., EK−1 are M × M unitary diagonal matrices, then the KM × KM block matrix H defined by: a00B a01E1B ... a0(K−1)EK−1B a10B a11E1B ... a EK−1B 1(K−1) . . .. . . . . . a(K−1)0B a(K−1)1E1B ... a(K−1)(K−1)EK−1B
is a Hadamard matrix. Exercise 3 Use Dit¸ˇa’sconstruction to construct some 6 × 6 and 8 × 8 Hadamard matrices.
Hadamard Matrices and ONB Iowa State It is an open problem whether a real Hadamard matrix exists for all multiples of 4.
One family of real Hadamard matrices are the Walsh matrices of dimensions 2k : 1 1 1 1 1 1 1 2 1 −1 1 −1 H(2 ) = , H(2 ) = , 1 −1 1 1 −1 −1 1 −1 −1 1
and in general H(2k ) = H(2) ⊗ H(2k−1).
Real Hadamard Matrices
Real Hadamard matrices must only have entries of 1 and −1. They only exist for n = 1, 2 and multiples of 4.
Hadamard Matrices and ONB Iowa State One family of real Hadamard matrices are the Walsh matrices of dimensions 2k : 1 1 1 1 1 1 1 2 1 −1 1 −1 H(2 ) = , H(2 ) = , 1 −1 1 1 −1 −1 1 −1 −1 1
and in general H(2k ) = H(2) ⊗ H(2k−1).
Real Hadamard Matrices
Real Hadamard matrices must only have entries of 1 and −1. They only exist for n = 1, 2 and multiples of 4.
It is an open problem whether a real Hadamard matrix exists for all multiples of 4.
Hadamard Matrices and ONB Iowa State Real Hadamard Matrices
Real Hadamard matrices must only have entries of 1 and −1. They only exist for n = 1, 2 and multiples of 4.
It is an open problem whether a real Hadamard matrix exists for all multiples of 4.
One family of real Hadamard matrices are the Walsh matrices of dimensions 2k : 1 1 1 1 1 1 1 2 1 −1 1 −1 H(2 ) = , H(2 ) = , 1 −1 1 1 −1 −1 1 −1 −1 1
and in general H(2k ) = H(2) ⊗ H(2k−1).
Hadamard Matrices and ONB Iowa State Exercise
Show why a real Hadamard matrix can only have size n = 1, 2 and multiples of 4.
Hadamard Matrices and ONB Iowa State Hadamard Pairs
Definition 4 d A set of vectors B = {b0,..., bn−1} ⊂ R and a set of vectors d L = {l0,..., ln−1} ⊂ R are called a Hadamard pair if the matrix:
H := e2πibj ·lk (1) 06j,k6n−1 is Hadamard. B is called the Hadamard dual of L and vice versa. 2 For example, in R , B = {(0, 0), (0, 2/3), (2/3, 0) and L = {(0, 0), (2, 1), (4, 2)} are a Hadamard pair, with
1 1 1 H = 1 e4πi/3 e2πi/3 , 1 e2πi/3 e4πi/3
which is equal to the DFT matrix F3.
Hadamard Matrices and ONB Iowa State Construct some Hadamard pairs for n = 4.
Exercise 5 Recall that for n = 4, all Hadamard matrices are equivalent to a matrix of the form: 1 1 1 1 1 ieia −1 −ieia 1 −1 1 −1 1 −ieia −1 ieia
for some a ∈ [0, π).
Hadamard Matrices and ONB Iowa State Exercise 5 Recall that for n = 4, all Hadamard matrices are equivalent to a matrix of the form: 1 1 1 1 1 ieia −1 −ieia 1 −1 1 −1 1 −ieia −1 ieia
for some a ∈ [0, π).
Construct some Hadamard pairs for n = 4.
Hadamard Matrices and ONB Iowa State Exercise List
I Use Dit¸ˇa’sconstruction to construct some 6 × 6 and 8 × 8 Hadamard matrices.
I Show why a real Hadamard matrix can only have size n = 1, 2 and multiples of 4.
I Construct some Hadamard pairs for n = 4.
Hadamard Matrices and ONB Iowa State Example 6
Fe4 below has dephased form F4.
i −1 −i 1 1 1 1 1 −1 1 −1 1 1 i −1 −i Fe4 = F4 = −i −1 i 1 1 −1 1 −1 1 1 1 1 1 −i −1 i
Dephased Forms
A complex Hadamard matrix is called dephased when the entries of its first row and column are all equal to 1. For any complex Hadamard matrix H, there exist unique diagonal unitary matrices, Dr and Dc , with (Dc )1,1 = 1 for which
Dr · H · Dc
is dephased.
Hadamard Matrices and ONB Iowa State Dephased Forms
A complex Hadamard matrix is called dephased when the entries of its first row and column are all equal to 1. For any complex Hadamard matrix H, there exist unique diagonal unitary matrices, Dr and Dc , with (Dc )1,1 = 1 for which
Dr · H · Dc
is dephased. Example 6
Fe4 below has dephased form F4.
i −1 −i 1 1 1 1 1 −1 1 −1 1 1 i −1 −i Fe4 = F4 = −i −1 i 1 1 −1 1 −1 1 1 1 1 1 −i −1 i
Hadamard Matrices and ONB Iowa State Two Hadamard matrices H1 and H2 are called equivalent and written H1 ' H2 if there exist diagonal unitary matrices D1 and D2 and permutation matrices P1 and P2 such that
H1 = D1P1H2P2D2.
Two Hadamard matrices with the same dephased form are equivalent. However, the converse is not true; it can be difficult to determine whether two Hadamard matrices are equivalent, particularly for large n.
Equivalence of Hadamard Matrices
The Hadamard matrix properties are preserved under multiplication by a diagonal unitary matrix or a permutation matrix.
Hadamard Matrices and ONB Iowa State Two Hadamard matrices with the same dephased form are equivalent. However, the converse is not true; it can be difficult to determine whether two Hadamard matrices are equivalent, particularly for large n.
Equivalence of Hadamard Matrices
The Hadamard matrix properties are preserved under multiplication by a diagonal unitary matrix or a permutation matrix.
Two Hadamard matrices H1 and H2 are called equivalent and written H1 ' H2 if there exist diagonal unitary matrices D1 and D2 and permutation matrices P1 and P2 such that
H1 = D1P1H2P2D2.
Hadamard Matrices and ONB Iowa State Equivalence of Hadamard Matrices
The Hadamard matrix properties are preserved under multiplication by a diagonal unitary matrix or a permutation matrix.
Two Hadamard matrices H1 and H2 are called equivalent and written H1 ' H2 if there exist diagonal unitary matrices D1 and D2 and permutation matrices P1 and P2 such that
H1 = D1P1H2P2D2.
Two Hadamard matrices with the same dephased form are equivalent. However, the converse is not true; it can be difficult to determine whether two Hadamard matrices are equivalent, particularly for large n.
Hadamard Matrices and ONB Iowa State