MATH 355 Supplemental Notes Complex Inner Product Spaces Complex Inner Product Spaces The Cn spaces The prototypical (and most important) real vector spaces are the Euclidean spaces Rn. Any study of complex vector spaces will similar begin with Cn. As a set, Cn contains vectors of length n whose entries are complex numbers. Thus, 2 i ` 3 5i C3, » ´ fi P i — ffi – fl 5, 1 is an element found both in R2 and C2 (and, indeed, all of Rn is found in Cn), and 0, 0, 0, 0 p ´ q p q serves as the zero element in C4. Addition and scalar multiplication in Cn is done in the analogous way to how they are performed in Rn, except that now the scalars are allowed to be nonreal numbers. Thus, to rescale the vector 3 i, 2 3i by 1 3i, we have p ` ´ ´ q ´ 3 i 1 3i 3 i 6 8i 1 3i ` p ´ qp ` q ´ . p ´ q « 2 3iff “ « 1 3i 2 3i ff “ « 11 3iff ´ ´ p ´ qp´ ´ q ´ ` Given the notation 3 2i for the complex conjugate 3 2i of 3 2i, we adopt a similar notation ` ´ ` when we want to take the complex conjugate simultaneously of all entries in a vector. Thus, 3 4i 3 4i ´ ` » 2i fi » 2i fi if z , then z ´ . “ “ — 2 5iffi — 2 5iffi —´ ` ffi —´ ´ ffi — 1 ffi — 1 ffi — ´ ffi — ´ ffi – fl – fl Both z and z are vectors in C4. In general, if the entries of z are all real numbers, then z z. “ The inner product in Cn In Rn, the length of a vector x ?x x is a real, nonnegative number. The modulus, or length, } }“ ¨ of a complex number z a ib is real and nonnegative as well: “ ` z zz a ib a ib a2 b2, or z 2 zz. | |“ “ p ` qp ´ q“ ` | | “ a b a A natural idea, therefore, is to define an inner product between vectors z z ,...,z , w “p1 nq “ w ,...,w Cn in this manner: p 1 nqP n z, w z w w z w z wHz zTw. (1) x y “ j j “ 1 1 `¨¨¨` n n “ “ j 1 ÿ“ 2 MATH 355 Supplemental Notes Complex Inner Product Spaces Here, wH stands for wT, the conjugate transpose of w. For instance, 3 4i ´ » 2i fi if z then zH zT 3 4i 2i 2 5i 1 . “ 2 5 “ “ ` ´ ´ ´ ´ 1 4 — iffi ˆ —´ ` ffi ” ı — 1 ffi — ´ ffi4 1 – fl ˆ Remarks On p. 439 (Section 9.2) of Strang’s text, he defines the inner product of complex vectors u, • v to be the conjugate transpose of the first vector multiplied by the second—i.e., uHv.A few paragraphs down, he acknowledges that some authors do it as we have, the conjugate transpose of the second vector times the first one. The two definitions do not yield the same result. For example, if u 2 i, 1 3i, 8 and v i, 3 2i, 1 i , then my definition of “p ` ´ q “p´ ` ´ q inner product between u and v gives 2 i ` u, v vHu i 3 2i 1 i 1 3i 4 i, x y “ “ ´ ` » ´ fi “ ´ ” ı 8 — ffi – fl but Strang’s definition, which would coincide with my v, u , yields x y i ´ uHv 2 i 1 3i 8 3 2i 4 i. “ ´ ` » ` fi “ ` ” ı 1 i — ´ ffi – fl That is, one is always the complex conjugate of the other. Though my sample is limited, the linear algebra books with which I am familiar define the inner product in Cn as I have above, not the other (Strang’s) way. He is correct that it is a matter of preference which definition you use. The inner product of vectors in Cn no longer exclusively produces real numbers, as seen in • the example above. However, when taking an inner product of z z , z ,...,z with itself, “p 1 2 nq the result n n z, z zHz z z z 2, x y “ “ j j “ | j| j 1 j 1 ÿ“ ÿ“ is the sum of the moduli of the components of z, guaranteed to be nonnegative. Thus, we define length for vectors z in Cn to be z z, z , } }“ x y b and note that the only instance in which z 0 is when z is, itself, the zero vector. } }“ When the entries of z, w are all real numbers (that is, z, w Rn), our definition for inner • P product exactly matches the dot product—that is, z, w z w. x y “ ¨ 3 MATH 355 Supplemental Notes Complex Inner Product Spaces The essential list of properties that the inner product in Cn has, for all vectors u, v, w Cn and all P scalars c, is (i) v, v 0. x y • (ii) v, v 0 implies v 0. x y “ “ (iii) u w, v u, v w, v . x ` y “ x y ` x y (iv) cu, v c u, v . x y “ x y (v) u, v v, u . x y “ x y Note that (iii) and (v) together imply that u, v w u, v u, w , x ` y “ x y ` x y while (iv) and (v) together give that u, cv c u, v . x y “ x y Conjugate transpose of a matrix Suppose, now, that A is an m-by-n matrix whose entries are complex numbers. The idea of a conjugate transpose AH makes sense, as it did for vectors in Cn. In fact, it is precisely what is computed when the prime symbol is invoked in Octave. octave:127> A = randi(7,3,2) 4 ones(3,2) + (randi (5,3,2) 4 ones(3,2)) i ⌥ ´ ⇤ ´ ⇤ ⇤ ⌅ A = 2 3i 2 + 1i ´ ´ 0 1i 1 1i ´ ´ ´ 2 2i 1 + 1i ´ octave:128> A’ ans = 2 + 3i 0 + 1i 2 + 2i ´ 2 1i 1 + 1i 1 1i ´ ´ ´ ⌃ ⇧ Of course, taking the conjugate transpose of a matrix twice returns one to the original: AH H A. p q “ For complex matrices A, B of appropriate size, one can take the conjugate transpose of the product AB. If we denote the matrix full of conjugates of entries found in A by A, then we have T T AB H AB T A B T B A BHAH. p q “p q “p q “ “ 4 MATH 355 Supplemental Notes Symmetric Matrices In particular, the conjugate transpose of a matrix-vector product Av is Av H vHAH, p q “ and if we need to take an inner product between Au and v, we have the convenient formula Au, v vHAu AHv Hu u, AHv . (2) x y “ “p q “ @ D Formula (2) holds whether or not A is square. In the context of matrices, AH is sometimes called the adjoint matrix (as opposed to the conjugate transpose). If the entries in A are real numbers only, then AH AT. Any square matrix which “ satisfies AH A is said to be self-adjoint, or Hermitian. “ Symmetric Matrices We use the term symmetric to describe a matrix A whose entries are real numbers, and which satisfies AT A. Note that this implies a symmetric matrix is self-adjoint. Symmetric matrices are “ very important in applications. They have some very favorable properties. The first is that their eigenvalues are real. Theorem 1: Eigenvalues of a symmetric matrix are real. Proof: To prove this, we note first that any complex number z can be expressed in the form z a ib; here a, b are real numbers, called the real and imaginary parts of “ ` z, respectively. The number z is, in fact, real precisely when its imaginary part b 0. “ Furthermore, the di↵erence of z and its conjugate is z z a ib a ib i 2b , ´ “p` q´p ´ q“ p q which is zero if and only if z R. P Now, suppose λ, v is an eigenpair (with v , 0) of a symmetric matrix A. Consider p q the quantity λ v 2, which may alternatively be expressed as } } λ v, v λv, v Av, v v, AHv v, Av v,λv λ v, v . x y “ x y “ x y “ “ x y “ x y “ x y @ D 5 MATH 355 Supplemental Notes Symmetric Matrices Subtracting the expression at one end from that on the other gives 0 λ v, v λ v, v λ λ v, v λ λ v 2. “ x y ´ x y “p ´ q x y “p ´ q} } Since v , 0, it follows that λ λ 0, which implies λ R. ⇤ } } ´ “ P As well, symmetric matrices generate eigenvectors which are naturally orthogonal. Theorem 2: Eigenvectors corresponding to distinct eigenvalues of a symmetric matrix are orthogonal. Proof: Suppose µ, u , λ, v are both eigenpairs of a symmetric matrix A with µ , λ. p q p q By the previous theorem, µ and λ are real numbers, so λ λ. We have “ µ λ u, v µ u, v λ u, v µu, v u,λv Au, v u, Av p ´ q x y “ x y ´ x y “ x y ´ x y “ x y ´ x y u, AHv u, Av u, Av u, Av 0. “ ´ x y “ x y ´ x y “ @ D Since µ λ u, v 0 and µ λ , 0, it follows that u, v 0. ⇤ p ´ q x y “ ´ x y “ Now, if A is symmetric and has n distinct eigenvalues (all real, of course) λ1,...,λn, then the corresponding eigenspaces Null A λ I are all 1-dimensional (since GM = AM = 1 for • p ´ j q each eigenvalue) with basis vector vj, eigenvectors are the vectors v ,...,v are linearly independent (proved earlier), and form a basis of Rn, and • t 1 nu the basis v ,...,v is orthogonal, not requiring a Gram-Schmidt process to make them so.