Operators on Inner-Product Spaces Self-Adjoint: T ∗ = T . Normal: TT ∗ = T ∗T . Positive: Self-adjoint and hT v, vi ≥ 0 for all v. Isometry: ||T v|| = ||v|| for all v. Let V be a finite-dimensional inner-product space. We wish to character- ize those linear operators T : V → V that have an orthonormal basis of eigenvectors. Complex : When V is a complex vector space, V has an orthonormal basis of eigenvectors with respect to a linear operator T if and only if T is normal. Proof: If V has an orthonormal basis of eigenvectors with respect to a linear operator T then T has a diagonal matrix representation A, which implies m(T ∗) = A∗ with respect to the same basis. Since the matrix representations of T and T ∗ are diagonal, they commute, hence T commutes with T ∗. Conversely, suppose TT ∗ = T ∗T . Since V is a complex vector space, T has an upper-triangular matrix representation A with respect to an orthonormal basis (method: apply Gram-Schmidt to an upper-triangular basis). It will suffice to show that this matrix is diagonal. It will further suffice to show 2 2 that ||Ri|| = ||Ci|| for each i. Let {u1, . . . , un} be the orthonormal basis. ∗ 2 2 We must show ||T ei|| = ||T ei|| for all i. However, for any v ∈ V , ||T ∗v||2 = hT ∗v, T ∗vi = hTT ∗v, vi = hT ∗T v, vi =

= hT v, T vi = ||T v||2.

Real Spectral Theorem: When V is a real vector space, V has an or- thonormal basis of eigenvectors with respect to a linear operator T if and only if T is self-adjoint. Proof: If V has an orthonormal basis of eigenvectors with respect to a real operator T then T has a diagonal matrix representation A which satisfies AT = A. This implies that T is self-adjoint. Conversely, suppose that a real operator T : V → V is self-adjoint. We will define a complex normal operator S : W → W , use the Complex Spectral

1 Theorem to find an orthonormal basis of eigenvectors for W , and use this to find an orthonormal basis of eigenvectors for V . Let W = V × V with scalar multiplication defined by

(a + bi)(v, w) = (av − bw, aw + bv) and inner product defined by

h(v1, w1), (v2, w2)i = (hv1, v2i + hw1, w2i) + i(−hv1, w2i + hw1, v2i). Let S : W → W be defined by

S(v, w) = (T v, T w).

S is self-adjoint, hence normal, hence V × V has an orthonormal basis of eigenvectors {(v1, w1),..., (vn, wn)}. The equation S(v, w) = (a + bi)(v, w) implies T (v) = av − bw and T (w) = aw + bv. Making these substitutions into hT v, wi = hv, T wi and rearranging yields b(hv, vi + hw, wi) = 0. This forces b = 0. Hence for each (vi, wi) there is a ai such that T vi = aivi and T wi = aiwi. Since {(v1, w1),..., (vn, wn)} is a basis for V ×V over C, every (x, 0) is in the span of {(v1, w1),..., (vn, wn)} over C, which implies that each x ∈ V is in the span of {v1, w1, . . . , vn, wn} over R. Hence V has a basis which is a subset of {v1, w1, . . . , vn, wn}, hence V has a basis of eigenvectors of T .

Now let Ba be the set of those basis eigenvectors corresponding to eigen- value a. Applying Gram-Schmidt to these produces an orthonormal set of eigenvectors Oa corresponding to eigenvalue a. If u and v are eigenvectors corresponding to distinct eigenvalues of T then they are orthogonal to each other: suppose T u = au and T v = bv where a 6= b. Then

ahu, vi = hau, vi = hT u, vi =

hu, T vi = hu, bvi = bhu, vi.

This forces hu, vi = 0. The union of the Oa bases forms an orthonormal eigenvector basis of V . Remark: Taken together, the two spectral theorems yield an algorithm for finding an orthonormal basis of eigenvectors for an V given a linear operator T : V → V which is normal (V complex) or self- adjoint (V real): First, find all the eigenvalues of T . Second, compute each

2 eigenspace. Since V has in principle an orthonormal basis of eigenvectors, V is the sum of its eigenspaces. Since eigenvectors corresponding to distinct eigenvalues are linearly independent, the sum is direct. Hence the union of the eigenspace bases forms a basis of eigenvectors for V . When V is complex and T is normal we know that applying Gram-Schmidt to a basis of eigenvectors produces an orthonormal basis of eigenvectors since the original basis produces an upper-triangular matrix representation. When V is real and T is self-adjoint we know that eigenvectors corresponding to distinct eigenvalues are orthogonal, so applying Gram-Schmidt to a eigenvector basis produces an orthonormal eigenvector basis. To summarize, in either case we find a basis for each eigenspace, then apply Gram-Schmidt to the union of these bases. Characterization of Real Normal Linear Operators: Let T : V → V be a normal operator on a real inner product space V . We will show that T has a block-diagonal matrix representation with respect to an orthonormal basis, where each of the blocks has dimension ≤ 2 and each of the blocks a −b of dimension 2 has the form . We can use an induction argument: b a for dimension 1 this is trivial. More generally, find an invariant subspace U of dimension ≤ 2, find an orthonormal basis for it, and expand to an orthonormal basis for V . The matrix representation of T restricted to U has the desired form, using the algebra steps which appear in the proof of the Complex Spectral Theorem. The same algebra steps can be used to show A 0  that the matrix representation of T with respect to this basis is 1 . 0 B So we have the foundation for an induction argument: we have decomposed V into U L U ⊥ where T maps U into U and U ⊥ into U ⊥ and the matrix representation of T restricted to these invariant subspaces are A1 and B and U has dimension ≤ 2. The induction hypothesis enables us to find an orthonormal basis for U ⊥ of the desired form, and adding these vectors to the orthonormal basis for U produces the desired orthonormal basis for V . Conversely, if T has such a matrix representation with respect to an orthonormal basis then it is normal. Note also that T is normal but not self- a −b adjoint if and only if T has at least one 2 × 2 block of the form with b a b 6= 0. By replacing one basis vector with its additive inverse if necessary we can assume b > 0.

3   a11 a12 (Note: if T is represented by with respect to the basis {u1, u2} a21 a22  a (β/α)a  then it is represented by the matrix 11 12 with respect to (α/β)a21 a22 the basis {αu1, βu2}.) Characterization of Positive Operators: An operator T : V → V on a complex or real finite-dimensional inner product space is said to be positive if it is self-adjoint and satisfies hT v, vi ≥ 0 for each v ∈ V . We know that the self-adjoint operators are precisely those that have a diagonal matrix representation with respect to some orthonormal basis of eigenvectors, where the diagonal entries rii are real numbers. The equation

X X X 2 hT ( αiei), αieii = rii|αi| ≥ 0 i i i holds for all choices of (α1, . . . , αn), which happens if and only if the diag- onal entries are non-negative real numbers. So the positive operators are precisely those that have a diagonal matrix representation with respect to some orthonormal basis of eigenvectors, in which the diagonal entries (i.e. the eigenvalues) are all non-negative. Theorem 7.27 is a list of equivalent conditions for a positive operator T . Given this characterization, each positive operator has a positive square root: just use the same orthonormal basis of eigenvectors, but let the matrix rep- resentation be that in which the the diagonal entries are replaced by their non-negative square roots. This square root is unique: let T be a positive 2 operator and let S be a positive operator√ that satisfies S = T . Computing S on the eigenvector√ basis yields Sv = λv where T v = λv. This determines S. Notation: S = T . For any linear operator T : V → V on an inner product space V , the operator ∗ ∗ 2 T T is always positive:√ hT T v, vi =√hT v, T vi = ||T v|| ≥ 0. Hence one can always construct T ∗T . The map T ∗T has some interesting properties: √ √ 1. For all u, v ∈ V , h T ∗T u, T ∗T vi = hT u, T vi. Proof: √ √ h T ∗T u, T ∗T vi = hT ∗T u, vi = hT u, T vi.

√ 2. null( T ∗T ) = null(T ). Proof: First note that null(T ) ⊆ null(T ∗T ). On the other hand, if u ∈ null(T ∗T ) then T ∗T u = 0, hence hT ∗T u, ui = 0,

4 therefore hT u, T ui = 0, therefore ||T u||2 = 0, therefore T u = 0, there- ∗ fore u ∈ null(T ). Hence null(T ) = null(T T ). Using the diagonal√ matrix ∗ ∗ ∗ representation√ of T T we can show that null(T T ) = null( T T ). Hence null(T ) = null( T ∗T ). √ 3. rank( T ∗T ) = rank(T ). Proof: This follows from #2 above and the Rank-Nullity Theorem. Isometries: An operator S : V → V on a complex or real finite-dimensional inner product space is said to be an isometry if satisfies ||Sv|| = ||v|| for each v ∈ V . Theorem: The following are equivalent: (a) S : V → V is an isometry. (b) For all u, v ∈ V , hSu, Svi = hu, vi. (c) S∗S = I. (d) S maps an orthonormal basis to an orthonormal basis. Proof: (a) implies (b): Assume that S is an isometry. Expanding the equation ||S(λu + v)||2 = ||λu + v||2 yields 2Re(λhSu, Svi) = 2Re(λhu, vi). If V is real then (b) holds. If V is complex then setting λ = 1, then λ = i, we see that (b) holds. (b) implies (c): Assume that S satisfies hSu, Svi = hu, vi for all u, v ∈ V . Then hS∗Su, vi = hu, vi, therefore h(S∗S − I)u, vi = 0. Setting v = (S∗S − I)u we obtain

||(S∗S − I)u|| = 0 for all u ∈ V , hence S∗S = I.

5 ∗ (c) implies (d): Assume that S S = I. Let {e1, . . . , en} be an orthormal basis for V . Then

∗ hSei, Seji = hS Sei, eji = hei, eji for all i and j, hence {Se1, . . . , Sen} is orthonormal.

(d) implies (a): Assume that S maps the orthonormal basis {e1, . . . , en} into the orthonormal basis {f1, . . . , fn}. Then hSei, Seji = hei, eji for all i and i, which implies hSu, Svi = hu, vi for all u, v ∈ V . This implies ||Sv|| = ||v|| for all v. Note that S∗S = I implies that isometries are normal. We have already char- acterized normal operators above: complex normal operators have a diagonal matrix representation with respect to an orthonormal basis of eigenvectors, and real normal operators have a block-diagonal matrix representation in a −b which the 2 × 2 blocks are of the form . The equation SS∗ = I b a implies that the 1-dimensional blocks have 1 and the 2-dimensional cos θ − sin θ blocks can be assumed to be in the form for some θ ∈ (0, π) sin θ cos θ (where some of the othornormal basis vectors may need to be replaced by their additive inverses). Conversely, if a linear operator as a matrix repre- sentation of this form with respect to an orthonormal basis then it satisfes S∗S = I hence it is an isometry. and Singular Decomposition Theorem 7.41 (Polar Decomposition of a Linear Operator): Let T be a linear operator on a finite-dimensional inner√ product space V . Then there is an isometry S ∈ L(V ) such that T = S T ∗T . √ √ ∗ ∗ Proof: The√ operator T T maps V onto range( T T ). The mapping ∗ S0 : range( T T ) → range(T ) defined by √ ∗ S0( T T v) = T v is well-defined, because if √ √ ∗ ∗ T T v1 = T T v2 then √ ∗ T T (v1 − v2) = 0,

6 therefore √ √ ∗ ∗ h T T (v1 − v2), T T (v1 − v2)i = 0, therefore hT (v1 − v2),T (v1 − v2) = 0, therefore T v1 = T v2. Moreover, √ √ √ √ ∗ ∗ ∗ ∗ hS0( T T u),S( T T v)i = hT u, T vi = h T T u, T T vi. √ ∗ Hence S0 is an√ inner-product preserving . Since rank( T T ) = ∗ ⊥ ⊥ rank(T ), range( T T ) and range(T√) have the same dimension. Let {e1, . . . , ek} ∗ ⊥ be an orthonormal basis for range( T T ) and let {f1, . . . , fk} be an or- ⊥ thonormal basis for range(T ) . We will extend S0 to a linear map S : V → V by declaring that Sei = fi for each i. By construction, hSu, Svi =√hu, vi for each u, v ∈ V , therefore S is an isometry. By construction, T = S T ∗T . √ ∗ Note that if we choose an orthonormal basis√ of eigenvectors for T T of the ∗ form {e1, . . . , en} then we have T (ei) = S T T ei = Ssiei = siSei. Setting fi = Sei for each i, the matrix representation of T with respect to the pair of orthonormal bases {e1, . . . , en} and {f1, . . . , fn} is diag(s1, . . . , sn). This choice of bases is called the Singular Value Decomposition of T . Miscellaneous Results:

Proposition 7.1: Every eigenvalue of a self-adjoint operator is real. Proof: Let T : V → V be self-adjoint and V a complex vector space. Then T is normal and has a diagonal matrix representation A. Since A = A∗, its diagonal entries are real, hence the eigenvalues of T are real. Proposition 7.3: Let T : V → V be a linear operator on a complex inner- product space. Then T is self-adjoint if and only if hT v, vi ∈ R for all v ∈ V .

Proof: Assume T is self-adjoint. Then it is normal, hence has a diago- nal matrix representation with real diagonal entries rii with respect to an orthonormal basis of eigenvectors. This implies

X X X 2 hT ( αiei), αieii = rii|αi| ∈ R. i i i

7 Conversely, if hT v, vi ∈ R for all v ∈ V then hT v, vi = hv, T vi = hv, T vi, hence T is self-adjoint. Proposition 7.4: If T is a self-adjoint operator on V such that hT v, vi = 0 for all v ∈ V then T = 0. Proof: T has a diagonal matrix representation with respect to an orthonor- mal basis of eigenvectors. Then

X X X 2 hT ( αiei), αieii = rii|αi| = 0 i i i holds for all choices of (α1, . . . , αn). This implies that all the eigenvalues rii are 0, which implies that T is represented by the 0 matrix, which says T = 0. Proposition 7.6: A linear operator T : V → V on an inner-product space is normal if and only if ||T v|| = ||T ∗v|| for all v. Proof: T is normal iff T ∗T − TT ∗ = 0 iff h(T ∗T − TT ∗)v, vi = 0 iff hT ∗T v, vi = hTT ∗v, vi iff hT v, T vi = hT ∗v, T ∗vi iff ||T v|| = ||T ∗v|| for all v. Corollary 7.7: Let T : V → V be normal. Then v is an eigenvector of T with respect to eigenvalue λ if and only if v is an eigenvector of T ∗ with respect to eigenvalue λ. Proof: Let v be an eigenvector of T . Then ||(T − λI)v|| = 0. Since T is normal, T − λI is normal with (T − λI)∗ = T ∗ − λI. Hence ||(T ∗ − λ)v|| = 0. This makes v an eigenvector of T ∗ with eigenvalue λ. The converse holds as well. Proposition 7.8: Let T : V → V be normal. Eigenvectors corresponding to distinct eigenvalues of T are orthogonal. Proof: Suppose T u = αu and T v = βv where α 6= β. Then

αhu, vi = hT u, vi = hu, T ∗vi

= hu, βvi = βhu, vi. Hence hu, vi = 0.

8