Linear Algebra: Graduate Level Problems and Solutions

Home , Dual basis, Inner product space, Orthogonal complement, Product (mathematics)

Igor Yanovsky

1 Linear Algebra Igor Yanovsky, 2005 2

Disclaimer: This handbook is intended to assist graduate students with qualifying examination preparation. Please be aware, however, that the handbook might contain, and almost certainly contains, typos as well as incorrect or inaccurate solutions. I can not be made responsible for any inaccuracies contained in this handbook. Linear Algebra Igor Yanovsky, 2005 3

Contents

1 Basic Theory 4 1.1 Linear Maps ...... 4 1.2 Linear Maps as Matrices ...... 4 1.3 Dimension and Isomorphism ...... 4 1.4 Matrix Representations Redux ...... 6 1.5 Subspaces ...... 6 1.6 Linear Maps and Subspaces ...... 7 1.7 Dimension Formula ...... 7 1.8 Matrix Calculations ...... 7 1.9 Diagonalizability ...... 8

2 Inner Product Spaces 8 2.1 Inner Products ...... 8 2.2 Orthonormal Bases ...... 8 2.2.1 Gram-Schmidt procedure ...... 9 2.2.2 QR Factorization ...... 9 2.3 Orthogonal Complements and Projections ...... 9

3 Linear Maps on Inner Product Spaces 11 3.1 Adjoint Maps ...... 11 3.2 Self-Adjoint Maps ...... 13 3.3 Polarization and Isometries ...... 13 3.4 Unitary and Orthogonal Operators ...... 14 3.5 Spectral Theorem ...... 15 3.6 Normal Operators ...... 15 3.7 Unitary Equivalence ...... 16 3.8 Triangulability ...... 16

4 Determinants 17 4.1 Characteristic Polynomial ...... 17

5 Linear Operators 18 5.1 Dual Spaces ...... 18 5.2 Dual Maps ...... 22

6 Problems 23 Linear Algebra Igor Yanovsky, 2005 4

1 Basic Theory

1.1 Linear Maps

Lemma. If A ∈ Matmxn(F) and B ∈ Matnxm(F), then

tr(AB) = tr(BA).

Pn Proof. Note that the (i, i) entry in AB is j=1 αijβji, while (j, j) entry in BA is Pm i=1 βjiαij. Xm Xn Thus tr(AB) = αijβji, i=1 j=1 Xn Xm tr(BA) = βjiαij. j=1 i=1

1.2 Linear Maps as Matrices n Example. Let Pn = {α0 + α1t + ··· + αnt : α0, α1, . . . , αn ∈ F} be the space of polynomials of degree ≤ n and D : V → V the diﬀerential map

n n−1 D(α0 + α1t + ··· + αnt ) = α1 + ··· + nαnt .

If we use the basis 1, t, . . . , tn for V then we see that D(tk) = ktk−1 and thus the (n + 1)x(n + 1) matrix representation is computed via   0 1 0 ··· 0    0 0 2 ··· 0    2 n n−1 2 n  ..  [D(1) D(t) D(t ) ··· D(t )] = [0 1 2t ··· nt ] = [1 t t ··· t ]  0 0 0 . 0   . . . .   ...... n  0 0 0 ··· 0

1.3 Dimension and Isomorphism A linear map L : V → W is isomorphism if we can ﬁnd K : W → V such that LK = IW and KL = IV .

V −−−−→L W x x   IV  IW

V ←−−−−K W Linear Algebra Igor Yanovsky, 2005 5

Theorem. V and W are isomorphic ⇔ there is a bijective linear map L : V → W .

Proof. ⇒ If V and W are isomorphic we can ﬁnd linear maps L : V → W and K : W → V so that LK = IW and KL = IV . Then for any y = IW (y) = L(K(y)) so we can let x = K(y), which means L is onto. If L(x1) = L(x2) then x1 = IV (x1) = KL(x1) = KL(x2) = IV (x2) = x2, which means L is 1 − 1. ⇐ Assume L : V → W is linear and a bijection. Then we have an inverse map L−1 −1 −1 which satisﬁes L ◦ L = IW and L ◦ L = IV . In order for this inverse map to be allowable as K we need to check that it is linear. Select α1, α2 ∈ F and y1, y2 ∈ W . −1 Let xi = L (yi) so that L(xi) = yi. Then we have −1 −1 −1 L (α1y1 + α2y2) = L (α1L(x1) + α2L(x2)) = L (L(α1x1 + α2x2)) −1 −1 = IV (α1x1 + α2x2) = α1x1 + α2x2 = α1L (y1) + α2L (y2).

Theorem. If Fm and Fn are isomorphic over F, then n = m.

m n n m Proof. Suppose we have L : F → F and K : F → F such that LK = IFn and KL = IFm . L ∈ Matnxm(F) and K ∈ Matmxn(F). Thus

n = tr(IFn ) = tr(LK) = tr(KL) = tr(IFm ) = m.

Deﬁne the dimension of a vector space V over F as dimF V = n if V is isomorphic to Fn.

Remark. dimC C = 1, dimR C = 2, dimQ R = ∞. The set of all linear maps {L : V → W } over F is homomorphism, and is denoted by homF(V,W ).

Corollary. If V and W are ﬁnite dimensional vector spaces over F, then homF(V,W ) is also ﬁnite dimensional and

dimF homF(V,W ) = (dimF W ) · (dimF V )

Proof. By choosing bases for V and W there is a natural mapping

(dimF W )·(dimF V ) homF(V,W ) → Mat(dimF W )×(dimF V )(F) ' F

This map is both 1-1 and onto as the matrix represetation uniquely determines the linear map and every matrix yields a linear map. Linear Algebra Igor Yanovsky, 2005 6

1.4 Matrix Representations Redux

L : V → W , bases x1, . . . , xm for V and y1, . . . , yn for W . The matrix for L interpreted as a linear map is [L]: Fm → Fn. The basis isomorphisms deﬁned by the choices of basis for V and W : m 1 [x1 ··· xm]: F → V , n [y1 ··· yn ]: F → W .

V −−−−→L W x x   [x1···xm] [y1···yn]

[L] Fm −−−−→ Fn

L ◦ [x1 ··· xm] = [y1 ··· yn][L]

1.5 Subspaces A nonempty subset M ⊂ V is a subspace if α, β ∈ F and x, y ∈ M, then αx+βy ∈ M. Also, 0 ∈ M. If M,N ⊂ V are subspaces, then we can form two new subspaces, the sum and the intersection: \ M + N = {x + y : x ∈ M, y ∈ N},M N = {x : x ∈ M, x ∈ N}. T M and N have trivial intersection if M N = {0}. M and N are transversal if M + N = V . Two spaces are complementary if theyT are transversal and have trivial intersection.L M,N form a direct sum of V if M N = {0} and M + N = V . Write V = M N.

Example. V = R2. M = {(x, 0) : x ∈ R}, x-axis, and N = {(0, y): y ∈ R}, y-axis.

Example. V = R2. M = {(x, 0) : x ∈ R}, x-axis, and N = {(y, y): y ∈ R}, a diagonal. L Note (x, y) = (x − y, 0) + (y, y), which gives V = M N. L If we have a direct sum decomposition V = M N, then we can construct the projection of V onto M along N. The map E : V → V is deﬁned using that each z = x + y, x ∈ M, y ∈ N and mapping z to x. E(z) = E(x + y) = E(x) + E(y) = E(x) = x. Thus im(E) = M and ker(E) = N.

Deﬁnition. If V is a vector space, a projection of V is a linear operator E on V such that E2 = E.

1 m [x1 ··· xm]: F → V means   α1  .  [x1 ··· xm]  .  = α1x1 + ··· + αmxm αm Linear Algebra Igor Yanovsky, 2005 7

1.6 Linear Maps and Subspaces L : V → W is a linear map over F. The kernel or nullspace of L is ker(L) = N(L) = {x ∈ V : L(x) = 0} The image or range of L is im(L) = R(L) = L(V ) = {L(x) ∈ W : x ∈ V } Lemma. ker(L) is a subspace of V and im (L) is a subspace of W .

Proof. Assume that α1, α2 ∈ F and that x1, x2 ∈ ker(L), then L(α1x1 + α2x2) = α1L(x1) + α2L(x2) = 0 ⇒ α1x1 + α2x2 ∈ ker(L). Assume α1, α2 ∈ F and x1, x2 ∈ V , then α1L(x1) + α2L(x2) = L(α1x1 + α2x2) ∈ im (L). Lemma. L is 1-1 ⇔ ker(L) = {0}. Proof. ⇒ We know that L(0·0) = 0·L(0) = 0, so if L is 1−1 we have L(x) = 0 = L(0) implies that x = 0. Hence ker(L) = {0}. ⇐ Assume that ker(L) = {0}. If L(x1) = L(x2), then linearity of L tells that L(x1 − x2) = 0. Then ker(L) = {0} implies x1 − x2 = 0, which shows that x1 = x2 as desired. Lemma. L : V → W , and dim V = dim W . L is 1-1 ⇔ L is onto ⇔ dim im (L) = dim V . Proof. From the dimension formula, we have dim V = dim ker(L) + dim im(L). L is 1-1 ⇔ ker(L) = {0} ⇔ dim ker(L) = 0 ⇔ dim im (L) = dim V ⇔ dim im (L) = dim W ⇔ im (L) = W , that is, L is onto.

1.7 Dimension Formula Theorem. Let V be ﬁnite dimensional and L : V → W a linear map, all over F, then im(L) is ﬁnite dimensional and

dimF V = dimF ker(L) + dimF im(L)

Proof. We know that dim ker(L) ≤Tdim V and that it has a complement M of dimension k = dim V − dim ker(L). Since M ker(L) = {0} the linear map L must be 1-1 when restricted to M. Thus L|M : M → im(L) is an isomorphism, i.e. dim im(L) = dim M = k.

1.8 Matrix Calculations 2 Change of Basis Matrix. Given the two basis of R , β1 = {x1 = (1, 1), x2 = (1, 0)} and β2 = {y1 = (4, 3), y2 = (3, 2)}, we ﬁnd the change-of-basis matrix P from β1 to β2. Write y1 as a linear combination of x1 and x2, y1 = ax1 + bx2. (4, 3) = a(1, 1) + b(1, 0) ⇒ a = 3, b = 1 ⇒ y1 = 3x1 + x2. Write y2 as a linear combination of x1 and x2, y2 = ax1 + bx2. (3, 2) = a(1, 1) + b(1, 0) ⇒ a = 2, b = 1 ⇒ y2 = 2x1 + x2. · ¸ 3 2 Write the coordinates of y and y as columns of P . P = . 1 2 1 1 Linear Algebra Igor Yanovsky, 2005 8

1.9 Diagonalizability Deﬁnition. Let T be a linear operator on the ﬁnite-dimensional space V . T is diagonalizable if there is a basis for V consisting of eigenvectors of T .

Theorem. Let v1, . . . , vn be nonzero eigenvectors of distinct eigenvalues λ1, . . . , λn. Then {v1, . . . , vn} is linearly independent.

Alternative Statement. If L has n distinct eigenvalues λ1, . . . , λn, then L is diagonalizable. (Proof is in the exercises).

Definition. Let L be a linear operator on a finite-dimensional vector space V , and let λ be an eigenvalue of L. Define Eλ = {x ∈ V : L(x) = λx} = ker(L − λIV ). The set Eλ is called the eigenspace of L corresponding to the eigenvalue λ. The algebraic multiplicity is defined to be the multiplicity of λ as a root of the characteristic polynomial of L, while the geometric multiplicity of λ is defined to be the dimension of its eigenspace, dim Eλ = dim(ker(L − λIV )). Also,

dim(ker(L − λIV )) ≤ m.

Eigenspaces. A vector v with (A − λI)v = 0 is an eigenvector for λ.

Generalized Eigenspaces. Let λ be an eigenvalue of A with algebraic multiplicity m. A vector v with (A − λI)mv = 0 is a generalised eigenvector for λ.

2 Inner Product Spaces

2.1 Inner Products The three important properties for complex inner products are: 1) (x|x) = ||x||2 > 0 unless x = 0. 2) (x|y) = (y|x). 3) For each y ∈ V the map x → (x|y) is linear. The inner product on Cn is deﬁned by

(x|y) = xty¯

Consequences:(α1x1 + α2x2|y) = α1(x1|y) + α2(x2|y), (x|β1y1 + β2y2) = β¯1(x|y1) + β¯2(x|y2), (αx|αx) = αα¯(x|x) = |α|2(x|x).

2.2 Orthonormal Bases

Lemma. Let e1, . . . , en be orthonormal. Then e1, . . . , en are linearly independent and any element x ∈ span{e1, . . . , en} has the expansion

x = (x|e1)e1 + ··· + (x|en)en.

Proof. Note that if x = α1e1 + ··· + αnen, then (x|ei) = (α1e1+···+αnen|ei) = α1(e1|ei)+···+αn(en|ei) = α1δ1i+···+αnδni = αi. Linear Algebra Igor Yanovsky, 2005 9

2.2.1 Gram-Schmidt procedure

Given a linearly independent set x1, . . . , xm in an inner product space V it is possi- ble to construct an orthonormal collection e1, . . . , em such that span{x1, . . . , xm} = span{e1, . . . , em}.

x1 e1 = . ||x1|| z2 z2 = x2 − projx1 (x2) = x2 − proje1 (x2) = x2 − (x2|e1)e1, e2 = . ||z2|| zk+1 zk+1 = xk+1 − (xk+1|e1)e1 − · · · − (xk+1|ek)ek, ek+1 = . ||zk+1||

2.2.2 QR Factorization

  (x1|e1)(x2|e1) ··· (xm|e1)   £ ¤ £ ¤  0 (x2|e2) ··· (xm|e2)  A = x1 ··· xm = e1 ··· em  . . . .  = QR  . . .. .  0 0 ··· (xm|em)

3 Example. Consider the vectors x1 = (1, 1, 0), x2 = (1, 0, 1), x3 = (0, 1, 1) in R . Perform Gram-Schmidt:¡ ¢ x1 (1,1,0) 1 1 e1 = = √ = √ , √ , 0 . ||x1|| 2 2 2 ¡ ¢ ¡ ¢ ( 1 ,− 1 ,1) ¡ ¢ √1 √1 √1 1 1 z2 2√ 2 √1 √1 √2 z2 = (1, 0, 1)− , , 0 = 2 , − 2 , 1 , e2 = ||z || = = , − , . 2 2 2 ¡ 2 ¢ 3/¡2 6 ¢6 ¡6 z = x − (x |e )e − (x |e )e = (0, 1, 1) − √1 √1 , √1 , 0 − √1 √1 , − √1 , √2 = − 3 3 ¢ 3 1 1 3 2 2 ¡2 2 2 ¢ 6 6 6 6 1 1 1 z3 1 1 1 √ , √ , √ , e3 = = − √ , √ , √ . 3 3 3 ||z3|| 3 3 3

2.3 Orthogonal Complements and Projections The orthogonal projection of a vector x onto a nonzero vector y is deﬁned by ³ ¯ ´ ¯ y y (x|y) ¡ ||(x|y)||¢ proj (x) = x¯ = y, The length of this projection is |proj (x)| = . y ||y|| ||y|| (y|y) y ||y||

The deﬁnition of projy(x) immediately implies that it is linear from the linearity of the inner product.

The map x → projy(x) is a projection.

Proof. Need to show projy(projy(x)) = projy(x). µ ¶ (x|y) (x|y) (x|y) (y|y) (x|y) proj (proj (x)) = proj y = proj (y) = y = y = proj (x). y y y (y|y) (y|y) y (y|y) (y|y) (y|y) y Linear Algebra Igor Yanovsky, 2005 10

Cauchy-Schwarz Inequality. V complex inner product space.

|(x|y)| ≤ ||x||||y||, x, y ∈ V.

Proof. First show projy(x) ⊥ x − projy(x): ³ ¯ ´ ³ ¯ ´ ³ ¯ ´ (x|y) ¯ (x|y) (x|y) ¯ (x|y) ¯(x|y) (proj (x)|x − proj (x)) = y¯x − y = y¯x − y¯ y y y ||y||2 ||y||2 ||y||2 ||y||2 ||y||2 (x|y) (x|y) (x|y) (x|y) (x|y) = (y|x) − (y|y) = (y|x) − (x|y) = 0. ||y||2 ||y||2 ||y||2 ||y||2 ||y||2 ¯¯ ¯¯ ¯ ¯ ¯¯(x|y) ¯¯ ¯(x|y)¯ |(x|y)| ||x|| ≥ ||proj (x)|| = ¯¯ y¯¯ = ¯ ¯||y|| = . y (y|y) (y|y) ||y||

Triangle Inequality. V complex inner product space.

||x + y|| ≤ ||x|| + ||y||.

Proof. ||x + y||2 = (x + y|x + y) = ||x||2 + 2Re(x|y) + ||y||2 ≤ ||x||2 + 2|(x|y)| + ||y||2 ≤ ||x||2 + 2||x||||y|| + ||y||2 = (||x|| + ||y||)2.

Let M ⊂ V be a ﬁnite dimensional subspace of an innter product space, and e1, . . . , em an orthonormal basis for M. Using that basis, deﬁne E : V → V by

E(x) = (x|e1)e1 + ··· + (x|em)em

Note that E(x) ∈ M and that if x ∈ M, then E(x) = x. Thus E2(x) = E(x) implying that E is a projection whose image is M. If x ∈ ker(E), then

0 = E(x) = (x|e1)e1 + ··· + (x|em)em ⇒ (x|e1) = ··· = (x|em) = 0.

This is equivalent to the condition (x|z) = 0 for all z ∈ M. The set of all such vectors is the orthogonal complement to M in V is denoted

M ⊥ = {x ∈ V :(x|z) = 0 for all z ∈ M}

L ⊥ Theorem. Let V be an inner product space. Assume V = M M , then im(projM ) = ⊥ L ⊥ M and ker(projM ) = M . If M ⊂ V is ﬁnite dimensional then V = M M and

projM (x) = (x|e1)e1 + ··· + (x|em)em for any orthonormal basis e1, . . . , em for M. Proof. For E deﬁned as above, ker(E) = M ⊥. x = E(x) + (I − E)(x) and (I − E)(x) ∈ ker(E) = M ⊥. Choose z ∈ M.

2 2 2 2 ||x − projM (x)|| ≤ ||x − projM (x)|| + ||projM (x) − z|| = ||x − z|| ,

2 where equality holds when ||projM (x)−z|| = 0, i.e., projM (x) is the only closest point to x among the points in M. Linear Algebra Igor Yanovsky, 2005 11

Theorem. LLet E : V → V be a projection on to M ⊂ V with the property that V = ker(E) ker(E)⊥. Then the following conditions are equivalent. 1) E = projM . 2) im(E)⊥ = ker(E). 3) ||E(x)|| ≤ ||x|| for all x ∈ V .

Proof. We have already seen that (1) ⇔ (2). Also (1),(2) ⇒ (3) as x = E(x)+(I−E)(x) is an orthogonal decomposition. So ||x||2 = ||E(x)||2 + ||(I − E)(x)||2 ≥ ||E(x)||2. Thus, we only need to show that (3) implies that E is orthogonal. Choose x ∈ ker(E)⊥ and observe that E(x) = x − (1 − E)(x) is an orthogonal decomposition. Thus ||x||2 ≥ ||E(x)||2 = ||x − (1 − E)(x)||2 = ||x||2 + ||(1 − E)(x)||2 ≥ ||x||2 ⊥ This means that (1 − E)(x) = 0 and hence x = E(x) ∈ im(E) ⇒ ker(LE) ⊂ im(E). Conversely, if z ∈ im(E) = M, then we can write z = x + y ∈ ker(E) ker(E)⊥. This implies that z = E(z) = E(y) = y, where the last equality follows from ker(E)⊥ ⊂ im(E). This means that x = 0 and hence z = y ∈ ker(E)⊥.

3 Linear Maps on Inner Product Spaces

3.1 Adjoint Maps ∗ The adjoint of A is the matrix A such that aij =a ¯ji. A : Fm → Fn, A∗ : Fn → Fm. (Ax|y) = (Ax)ty¯ = xtAty¯ = xt(A¯ty) = (x|A∗y).

dim(M) + dim(M ⊥) = dim(V ) = dim(V ∗)

Theorem. Suppose S = {v1, v2, . . . , vk} is an orthonormal set in an n-dimensional inner product space V . Then a) S can be extended to an orthonormal basis {v1, v2, . . . , vk, vk+1, . . . , vn} for V . ⊥ b) If M = span(S), then S1 = {vk+1, . . . , vn} is an orthonormal basis for M . c) If M is any subspace of V , then dim(V ) = dim(M) + dim(M ⊥).

0 Proof. a) Extend S to a basis S = {v1, v2, . . . , vk, wk+1, . . . , wn} for V . Apply Gram- Schmidt process to S0. The ﬁrst k vectors resulting from this process are the vectors in S. S0 spans V . Normalizing the last n − k vectors of this set produces an orthonormal set that spans V . b) Because S1 is a subset of a basis, it is linearly independent. Since S1 is clearly a subset of M ⊥, we need only show that it spans M ⊥. For any x ∈ V , we have

Xn x = (x|vi)vi. i=1

⊥ If x ∈ M , then (x|vi) = 0 for 1 ≤ i ≤ k. Therefore, Xn x = (x|vi)vi ∈ span(S1). i=k+1 c) Let M be a subspace of V . It is ﬁnite-dimensional inner product space because V is, and so it has an orthonormal basis {v1, v2, . . . , vk}. By (a) and (b), we have dim(V ) = n = k + (n − k) = dim(M) + dim(M ⊥). Linear Algebra Igor Yanovsky, 2005 12

L Theorem. Let M be a subspace of V . Then V = M M ⊥.

Proof. By Gram-Schmidt process, we can obtain an orthonormal basis {v1, v2, . . . , vk} of M, and by theorem above, we can extend it to an orthonormal basis {v1, v2, . . . , vn} ⊥ of V . Hence vk+1, . . . , vn ∈ M . If x ∈ V , then ⊥ x = a1v1+···+anvn, where a1v1+···+akvk ∈ M and ak+1vk+1+···+anvn ∈ M . ⊥ 0 T ⊥ 0 0 Accordingly, V = M + M .T On the other hand, if x ∈ M M , then (x |x ) = 0. This yields x0 = 0. Hence M M ⊥ = {0}. Theorem. a) M ⊆ M ⊥⊥. b) If M is a subset of a finite-dimensional space V , then M = M ⊥⊥. ⊥ ⊥⊥ Proof. a) LetL x ∈ M. Then (x|z) = 0,L∀z ∈ M ; hence x ∈ M . b) V = M M ⊥ and, also V = M ⊥ M ⊥⊥. Hence dim M = dim V − dim M ⊥ and dim M ⊥⊥ = dim V − dim M ⊥. This yields dim M = dim M ⊥⊥. Since M ⊆ M ⊥⊥ by (a), we have M = M ⊥⊥. Fredholm Alternative. L : V → W be a linear map between finite dimensional inner product spaces. Then ker(L) = im(L∗)⊥, ker(L∗) = im(L)⊥, ker(L)⊥ = im(L∗), ker(L∗)⊥ = im(L). Proof. Using L∗∗ = L and M ⊥⊥ = M, the four statements are equivalent. ker L = {x ∈ V : Lx = 0}.V →L W ∗ im(L) = {Lx : x ∈ V },W →L V im(L∗) = {L∗y : y ∈ W }, im(L∗)⊥ = {x ∈ V :(x|L∗y) = 0 for all y ∈ W } = {x ∈ V :(Lx|y) = 0 for all y ∈ W }. If x ∈ ker L ⇒ x ∈ im(L∗)⊥. Conversely, if (Lx|y) = 0 for all y ∈ W ⇒ Lx = 0 ⇒ x ∈ ker L. Rank Theorem. L : V → W be a linear map between finite dimensional inner product spaces. Then rank(L) = rank(L∗). Proof. dim V = dim(ker(L)) + dim(im(L)) = dim(im(L∗))⊥ + dim(im(L)) = dim V − dim(im(L∗)) + dim(im(L)). Corollary. For an n × m matrix A, the column rank equals the row rank. Proof. Conjugation does not change the rank. rank(A) is the column rank. rank(A∗) is the row rank of the conjugate of A. Corollary. Let L : V → V be a linear operator on a finite dimensional inner product space. Then λ is an eigenvalue for L ⇔ λ¯ is an eigenvalue for L∗. Moreover, these eigenvalue pairs have the same geometric multiplicity: ∗ dim(ker(L − λIV )) = dim(ker(L − λI¯ V )). ∗ ∗ Proof. Note that (L − λIV ) = L − λI¯ V . Thus we only need to show dim(ker(L)) = dim(ker(L∗)). dim(ker(L)) = dim V − dim(im(L)) = dim V − dim(im(L∗)) = dim(ker(L∗)). Linear Algebra Igor Yanovsky, 2005 13

3.2 Self-Adjoint Maps A linear operator L : V → V is self-adjoint (Hermitian) if L∗ = L, and skew-adjoint if L∗ = −L. Theorem. If L is self-adjoint operator on a ﬁnite-dimensional inner product space V , then every eigenvalue of L is real. Proof. Method I: Suppose L is a self-adjoint operator in V . Let λ be an eigenvalue of L, and let x be a nonzero vector in V such that Lx = λx. Then

λ(x|x) = (λx|x) = (Lx|x) = (x|L∗x) = (x|Lx) = λ¯(x|x).

Thus λ = λ¯, which means that λ is real.

Proof. Method II: Suppose that L(x) = λx for x 6= 0. Because a self-adjoint operator is normal, then if x is an eigenvector of L then x is also an eigenvector of L∗. Thus, λx = L(x) = L∗(x) = λx¯ .

Proposition. If L is self- or skew-adjoint, then for each invariant subspace M ⊂ V the orthogonal complement is also invariant, i.e., if L(M) ⊂ M, then also L(M ⊥) ⊂ M ⊥. Proof. Assume that L(M) ⊂ M. If x ∈ M and z ∈ M ⊥, then since L(x) ∈ M we have

0 = (z|L(x)) = (L∗(z)|x) = ±(L(z)|x).

Since this holds ∀x ∈ M, it follows that L(z) ∈ M ⊥.

3.3 Polarization and Isometries Real inner product on V : (x + y|x + y) = (x|x) + 2(x|y) + (y|y) 1 1 ⇒ (x|y) = ((x + y|x + y) − (x|x) − (y|y)) = (||x + y||2 − ||x||2 − ||y||2). 2 2 Complex inner products (are only conjugate symmetric) on V : (x + y|x + y) = (x|x) + 2Re(x|y) + (y|y) 1 ⇒ Re(x|y) = (||x + y||2 − ||x||2 − ||y||2). 2 Re(x|iy) = Re(−i(x|y)) = Im(x|y). In particular, we have 1 Im(x|y) = (||x + iy||2 − ||x||2 − ||iy||2). 2 We can use these ideas to check when linear operators L : V → V are 0. First note that L = 0 ⇔ (L(x)|y) = 0, ∀x, y ∈ V . To check the ⇐ part, let y = L(x) to see that ||L(x)||2 = 0, ∀x ∈ V . Theorem. Let L : V → V be self-adjoint. L = 0 ⇔ (L(x)|x) = 0, ∀x ∈ V . Proof. ⇒ If L = 0 ⇒ (L(x)|x) = 0, ∀x ∈ V . ⇐ Assume that (L(x)|x) = 0, ∀x ∈ V . 0 = (L(x + y)|x + y) = (L(x)|x) + (L(x)|y) + (L(y)|x) + (L(y)|y) = (L(x)|y) + (y|L∗(x)) = (L(x)|y) + (y|(L(x))) = 2Re(L(x)|y). Now insert y = L(x) to see that 0 = Re(L(x)|L(x)) = ||L(x)||2. Linear Algebra Igor Yanovsky, 2005 14

Theorem. Let L : V → V be a linear map on a complex inner-product space. Then L = 0 ⇔ (L(x)|x) = 0, ∀x ∈ V .

3.4 Unitary and Orthogonal Operators A linear transformation A is orthogonal is AAT = I, and unitary if AA∗ = I, i.e. A∗ = A−1.

Theorem. L : V → W is a linear map between inner product spaces. TFAE: ∗ 1) L L = IV , (L is unitary) 2) (L(x)|L(y)) = (x|y) ∀x, y ∈ V , (L preserves inner products) 3) ||L(x)|| = ||x|| ∀x ∈ V . (L preserves lengths)

∗ ∗ Proof. (1) ⇒ (2). L L = IV ⇒ (L(x)|L(y)) = (x|L L(y)) = (x|Iy) = (x|y), ∀x ∈ V . Also note: L takes orthonormal sets of vectors to orthonormalp sets of vectors.p (2) ⇒ (3). (L(x)|L(y)) = (x|y), ∀x, y ∈ V ⇒ ||L(x)|| = (L(x),L(x)) = (x, x) = ||x||. (3) ⇒ (1). ||L(x)|| = ||x||, ∀x ∈ V ⇒ (L∗L(x)|x) = (L(x)|L(x)) = (x|x) = (Ix|x) ⇒ ((L∗L − I)(x)|x) = 0, ∀x ∈ V . Since L∗L − I is self-adjoint (check), L∗L = I.

Two inner product spaces V and W over F are isometric, if we can ﬁnd an isometry L : V → W , i.e. an isomorphism such that (L(x)|L(y)) = (x|y).

Theorem. Supposet L is unitary, then L is an isometry on V .

Proof. An isometry on V is a mapping which preserves distances. Since L is unitary, ||L(x) − L(y)|| = ||L(x − y)|| = ||x − y||. Thus L is an isometry. Linear Algebra Igor Yanovsky, 2005 15

3.5 Spectral Theorem Theorem. Let L : V → V be a self-adjoint operator on a ﬁnite dimensional inner product space. Then we can ﬁnd a real eigenvalue λ for L.

Spectral Theorem. Let L : V → V be a self-adjoint operator on a ﬁnite dimensional inner product space. Then there exists an orthonormal basis e1, . . . , en of eigenvectors, i.e. L(e1) = λ1e1,...,L(en) = λnen. Moreover, all eigenvalues λ1, . . . , λn are real. Proof. Prove this by induction on dim V . Since L = L∗, we can ﬁnd v ∈ V , λ ∈ R such that L(v) = λv (Lagrange multipliers). Let v⊥ = {x ∈ V :(x|v) = 0}, an orthogonal complement to v. dim v⊥ = dim V − 1. We show L leaves v⊥ invariant, i.e. L(v⊥) ⊂ v⊥. Let x ∈ v⊥, then

∗ (L(x)|v) = (x|L (v)) = (x|L(v)) = (x|λv)|{z} = λ(x|v) = 0. λ∈R

⊥ ⊥ Thus, L|v⊥ : v → v is again self-adjoint, because (L(x)|y) = (x|L(y)), ∀x, y ∈ V ⇒ ⊥ v ⊥ x, y ∈ v . Let e1 = ||v|| ; e2, . . . , en - orthonormal basis for v so that L(ei) = λiei, i = ⊥ ⊥ 2, . . . , n. Check: (e1|ei) = 0, ∀i ≥ 2 since ei ∈ v = e1 , i = 2, . . . , n. Corollary. Let L : V → V be a self-adjoint operator on a ﬁnite dimensional inner product space. Then there exists an orthonormal basis e1, . . . , en of eigenvectors and a real n × n diagonal matrix D such that   λ1 ··· 0 ∗  . . .  ∗ L = [e1 ··· en] D [e1 ··· en] = [e1 ··· en]  . .. .  [e1 ··· en] . 0 ··· λn

3.6 Normal Operators An operator L : V → V on an inner product space is normal if LL∗ = L∗L. Self-adjoint, skew-adjoint and isometric operators are normal. These are precisely the operators that admit the orthonormal basis that diagonalizes them.

Proposition. LL∗ = L∗L ⇔ ||L(x)|| = ||L∗(x)|| for all x ∈ V .

Proof. ||L(x)|| = ||L∗(x)|| ⇔ ||L(x)||2 = ||L∗(x)||2 ⇔ (L(x)|L(x)) = (L∗(x)|L∗(x)) ⇔ (x|L∗L(x)) = (x|LL∗(x)) ⇔ (x|(L∗L − LL∗)(x)) = 0 ⇔ L∗L − LL∗ = 0, since L∗L − LL∗ is self-adjoint.

Theorem. If V is a complex inner product space and L : V → V is normal, then ∗ ker(L − λIV ) = ker(L − λI¯ V ) for all λ ∈ C.

Proof. Observe L − λIV is normal and use previous proposition to conclude that ∗ ||(L − λIV )(x)|| = ||(L − λI¯ V )(x)||. Linear Algebra Igor Yanovsky, 2005 16

Spectral Theorem for Normal Operators. Let L : V → V be a normal operator on a complex inner product space. Then there exists an orthonormal basis e1, . . . , en such that L(e1) = λ1e1,...,L(en) = λnen. Proof. Prove this by induction on dim V . Since L is complex linear, we can use the Fundamental Theorem of Algebra to ﬁnd λ ∈ C ∗ ∗ and x ∈ V \{0}, so that L(x) = λx ⇒ L (x) = λx¯ . ker(L−λIV ) = ker(L −λI¯ V ). Let x⊥ = {z ∈ V :(z|x) = 0}, an orthogonal complement to x. To get induction, we need to show that x⊥ is invariant under L, i.e. L(x⊥) ⊂ x⊥. Let z ∈ x⊥ and show Lz ∈ x⊥. (L(z)|x) = (z|L∗(x)) = (z|λx¯ ) = λ(z|x) = 0.

Check that L|x⊥ is normal. Similarly, x⊥ is invariant under L∗, i.e. L∗ : x⊥ → x⊥, since (L∗(z)|x) = (z|L(x)) = (z|λx) = λ¯(z|x) = 0. ∗ ∗ ∗ ⊥ ⇒ L |x⊥ = (L|x⊥ ) since (L(z)|y) = (z|L y), z, y ∈ x .

3.7 Unitary Equivalence Two nxn matrices A and B are unitary equivalent if A = UBU ∗, where U is an nxn ∗ ∗ matrix such that U U = UU = IFn . Corollary. (nxn matrices) 1. A normal matrix is unitary equivalent to a diagonal matrix. 2. A self-adjoint matrix is unitary or orthogonally equivalent to a real diagonal. 3. A skew-adjoint matrix is unitary equivalent to a purely imaginary diagonal. 4. A unitary matrix is unitary equivalent to a diagonal matrix whose diagonal elements are unit scalars.

3.8 Triangulability Schur’s Theorem. Let L : V → V be a linear operator on a ﬁnite dimensional complex inner product space. Then we can ﬁnd an orthonormal basis e1, . . . , en such that the matrix representation [L] is upper triangular in this basis, i.e.,   α11 α12 ··· α1n  0 α ··· α  ∗  22 2n  ∗ L = [e1 ··· en][L][e1 ··· en] = [e1 ··· en]  . . . .  [e1 ··· en] .  . . .. .  0 0 ··· αnn Linear Algebra Igor Yanovsky, 2005 17

Generalized Schur’s Theorem. Let L : V → V be a linear operator on an n dimensional vector space over F. Assume that χL(t) = (t − λ1) ··· (t − λn) for λ1, . . . , λn ∈ F, then V admits a basis x1, . . . , xn such that the matrix representation with respect to x1, . . . , xn is upper triangular. Proof. The proof is by induction on the dimension n of V . The result is immediate if n = 1. So suppose that the result is true for linear operators on (n − 1)-dimensional inner product spaces whose characteristic polynomials split. We can assume that L∗ has a unit eigenvector z. Suppose that L∗(z) = λz and that W = span({z}). We show that W ⊥ is L-invariant. If y ∈ W ⊥ and x = cz ∈ W , then (L(y)|x) = (L(y)|cz) = (y|L∗(cz)) = (y|cL∗(z)) = (y|cλz) = cλ(y|z) = cλ(0) = 0. ⊥ So L(y) ∈ W . It is easy to show that the characteristic polynomial of LW ⊥ divides the characteristic polynomial of L and hence splits. Thus, dim(W ⊥) = n − 1, so we ⊥ may apply the induction hypothesis to LW ⊥ and obtainS an orthonormal basis γ of W such that [LW ⊥ ]γ is upper triangular. Clearly, β = γ {z} is an orthonormal basis for V such that [L]β is upper triangular.

4 Determinants

4.1 Characteristic Polynomial n n−1 The characteristic polynomial of A is deﬁned as χA(t) = t +αn−1t +. . . α1t+α0. The characteristic polynomial of L : V → V can be deﬁned by

χL(t) = det(L − tIV ).

−1 1 T Facts: det L = det L . det A = det A . If A is orthogonal, det A = ±1, since det(I) = det(AAT ) = det(A) det(AT ) = (det A)2. If U is unitary, | det A| = 1, and all |λi| = 1. Linear Algebra Igor Yanovsky, 2005 18

5 Linear Operators

5.1 Dual Spaces For a vector space V over F, we deﬁne the dual space V 0 = hom(V, F) as the set of linear functions of V , i.e. V 0 = {f : V → F | f is linear }. Let x1, . . . , xn be a basis for V . For each i, there is a unique linear functional fi on V s.t.

fi(xj) = δij.

In this way we obtain from x1, . . . , xn a set of n distinct linear functionals f1, . . . , fn on V . These functionals are also linearly independent. For, suppose Xn f = cifi. i=1 Xn Xn Then f(xj) = cifi(xj) = ciδij = cj. i=1 i=1

In particular, if f is the zero functional, f(xj) = 0 for each j and hence the scalars cj are all 0. Now f1, . . . , fn are n linearly independent functionals, and since we know 0 0 that V has dimension n, it must be that f1, . . . , fn is a basis for V . This basis is called the dual basis. 0 We have shown that ∃! dual basis {f1,..., fn} for V . If f is a linear functional on V then f is some linear combination of the fi, and the scalars cj must be given by cj = f(xj).

Xn f = f(xi)fi i=1

Similarly, if Xn x = αixi is a vector in V , then i=1 Xn Xn fj(x) = αifj(xi) = αiδij = αj i=1 i=1 so that the unique expression for x as a linear combination of the xi is

Xn x = fi(x)xi i=1

th fi(x) = αi = i coordinate for x. Let M ⊂ V be a subspace and deﬁne the annihilator2 to M in V as

0 0 0 0 M = {f ∈ V : f(x) = 0 for all x ∈ M} = {f ∈ V : f(M) = {0}} = {f ∈ V : f|M = 0}

2annihilaror is the counterpart of orthogonal complement Linear Algebra Igor Yanovsky, 2005 19

2 Example. Let β = {x1, x2} = {(2, 1), (3, 1)} be a basis for R . (xi(ξ1, ξ2)). We ﬁnd 0 the dual basis of β given by β = {f1, f2}. To determine formulas for f1 and f2, we seek functionals f1(ξ1, ξ2) = a1ξ1 + a2ξ2 and f2(ξ1, ξ2) = b1ξ1 + b2ξ2 such that f (x ) = 1, f (x ) = 0, f (x ) = 0, f (x ) = 1. Thus ( 1 1 1 2 2 1 ( 2 2 1 = f1(x1) = f(2, 1) = 2a1 + a2 0 = f2(x1) = f(2, 1) = 2b1 + b2

0 = f1(x2) = f(3, 1) = 3a1 + a2 1 = f2(x2) = f(3, 1) = 3b1 + b2

The solutions yield a1 = −1, a2 = 3 and b1 = 1, b2 = −2. Hence f1(ξ1, ξ2) = −ξ1 + 3ξ2 and f2(ξ1, ξ2) = ξ1 − 2ξ2, or f1 = (−1, 3), f2 = (1, −2), form the dual basis.

3 Example. Let β = {x1, x2, x3} = {(1, 0, 1), (0, 2, 0), (−1, 0, 2)} be a basis for R . 0 (xi(ξ1, ξ2, ξ3)). We ﬁnd the dual basis of β given by β = {f1, f2, f3}. To determine formulas for f1,f2,f3 we seek functionals f1(ξ1, ξ2, ξ3) = a1ξ1 + a2ξ2 + a3ξ3, f2(ξ1, ξ2, ξ3) = b1ξ1 + b2ξ2 + b3ξ3, and f3(ξ1, ξ2, ξ3) = c1ξ1 + c2ξ2 + c3ξ3 such that fi(xj) = δij.   1 = f1(x1) = f(1, 0, 1) = a1 + a3 0 = f2(x1) = f(1, 0, 1) = b1 + b3 0 = f1(x2) = f(0, 2, 0) = 2a2 1 = f2(x2) = f(0, 2, 0) = 2b2   0 = f1(x3) = f(−1, 0, 2) = −a1 + 2a3 0 = f2(x3) = f(−1, 0, 2) = −b1 + 2b3  2 1 0 = f3(x1) = f(1, 0, 1) = c1 + c3 Thus a1 = 3 , a2 = 0, a3 = 3 , 0 = f (x ) = f(0, 2, 0) = 2c b = 0, b = 1 , b = 0,  3 2 2 1 2 2 3  1 1 1 = f3(x3) = f(−1, 0, 2) = −c1 + 2c3 c1 = − 3 , c2 = 0, c3 = 3 .

2 1 1 1 1 Hence f1(ξ1, ξ2, ξ3) = 3 ξ1 + 3 ξ3, f2(ξ1, ξ2, ξ3) = 2 ξ2, f3(ξ1, ξ2, ξ3) = − 3 ξ1 + 3 ξ3, 2 1 1 1 1 or f1 = ( 3 , 0, 3 ), f2 = (0, 2 , 0), f3 = (− 3 , 0, 3 ), form the dual basis. 4 Example. Let W is the subspace of R spanned by x1 = (1, 2, −3, 4) and x2 = (0, 1, 4, −1). We find the basis for W◦, the annihilator of W . If suffices to find a basis of the set of linear functionals f(ξ1, ξ2, ξ3, ξ4) = a1ξ1 + a2ξ2 + a3ξ3 + a4ξ4 for which f(x1) = 0 and f(x2) = 0 : f(1, 2, −3, 4) = a1 + 2a2 − 3a3 + 4a4 = 0 f(0, 1, 4, −1) = a2 + 4a3 − a4 = 0 The system of equations in a1, a2, a3, a4 is in echelon form with free variables a3 and a4. Set a3 = 1, a4 = 0 to obtain a1 = 11, a2 = −4, a3 = 1, a4 = 0 ⇒ f1(ξ1, ξ2, ξ3, ξ4) = 11ξ1 − 4ξ2 + ξ3. Set a3 = 0, a4 = −1 to obtain a1 = 6, a2 = −1, a3 = 0, a4 = −1 ⇒ f2(ξ1, ξ2, ξ3, ξ4) = 6ξ1 − ξ2 − ξ4. ◦ The set of linear functionals {f1, f2} is a basis of W . Linear Algebra Igor Yanovsky, 2005 20

Example. Given the annihilator described by the three linear functionals in R4:

f1(ξ1, ξ2, ξ3, ξ4) = ξ1 + 2ξ2 + 2ξ3 + ξ4

f2(ξ1, ξ2, ξ3, ξ4) = 2ξ1 + ξ4

f3(ξ1, ξ2, ξ3, ξ4) = −2ξ1 − 4ξ3 + 3ξ4 we ﬁnd the subspace it annihilates. After the row reduction, we ﬁnd that the functionals below ahhihilate the same subspace

g1(ξ1, ξ2, ξ3, ξ4) = ξ1 + 2ξ3

g2(ξ1, ξ2, ξ3, ξ4) = ξ2

g3(ξ1, ξ2, ξ3, ξ4) = ξ4

The subspace annihilated consists of the vectors with ξ1 = −2ξ3, ξ2 = ξ4 = 0. Thus the subspace that is annihilated is given by span{(−2, 0, 1, 0)}.

Proposition. If M ⊂ V is a subspace of a ﬁnite dimensional space and x1, . . . , xn is ◦ a basis for V such that M = span{x1, . . . , xm}, then M = span{fm+1, . . . , fn}, where f1, . . . , fn is the dual basis. In particular we have

dim(M) + dim(M ◦) = dim(V ) = dim(V 0).

Proof. Let x1, . . . , xm be a basis for M; M = span{x1, . . . , xm}. Extend to {x1, . . . , xn}, 0 a basis for V . Construct a dual basis f1, . . . , fn for V , fi(xj) = δij. ◦ ◦ We show that fm+1, . . . , fn is a basis for M . First, show M = span{fm+1, . . . , fn}. ◦ 0 Pn Pn Pm Pn Let f ∈ M ⊂ V . f = i=1 cifi = i=1 f(xi)fi = i=1 f(xi)fi + i=m+1 f(xi)fi = Pn i=m+1 f(xi)fi ∈ span{fm+1, . . . , fn}. Second, {fm+1, . . . , fn} are linearly independent, since {fm+1, . . . , fn} is a subset of basis for V 0. Thus, dim(M ◦) = n − m = dim(V ) − dim(M).

Theorem. W1 and W2 are subspaces of a ﬁnite-dimensional vector space. Then ◦ ◦ W1 = W2 ⇔ W1 = W2 . ◦ ◦ Proof. ⇒ If W1 = W2, then of course W1 = W2 . ⇐ If W1 6= W2, then one of the two subspaces contains a vector which is not in the other. Suppose there is a vector x ∈ W2 but x∈ / W1. There is a linear functional f ◦ ◦ such that f(z) = 0 for all z ∈ W1, but f(x) 6= 0. Then f ∈ W1 but f∈ / W2 and ◦ ◦ W1 6= W2 . Theorem. Let W be a subspace of a ﬁnite-dimensional vector space V . Then W = W ◦◦.

Proof. dim W + dim W ◦ = dim V , dim W ◦ + dim W ◦◦ = dim V 0, and since dim V = dim V 0 we have dim W = dim W ◦◦. Since W ⊂ W ◦◦, we see that W = W ◦◦. Linear Algebra Igor Yanovsky, 2005 21

L 0 Proposition.L Assume that the ﬁnite dimensional space V = M N, then also V = M ◦ N ◦ and the restriction maps V 0 → M 0 and V 0 → N 0 give isomorphisms

M ◦ ≈ N 0, N ◦ ≈ M 0.

Proof. Select a basis x1, . . . , xn for V such that M = span{x1, . . . , xm} and N = span{xm+1, . . . , xn}. Then let f1, . . . , fn be the dual basis and simply observe that ◦ ◦ 0 ◦ L ◦ M = span{fm+1, . . . , fn} and N = span{f1, . . . , fm}. This proves that V = M N . Next we note that

dim(M ◦) = dim(V ) − dim(M) = dim(N) = dim(N 0).

◦ 0 So at least M and N have the same dimension. Also if we restrict fm+1, . . . , fn to N, then we still have that fi(xj) = δij for j = m + 1, . . . , n. As N = span{xm+1, . . . , xn}, 0 ◦ 0 this means that fm+1|N , . . . , fn|N form a basis for N . The proof that N ≈ M is similar. Linear Algebra Igor Yanovsky, 2005 22

5.2 Dual Maps The dual space construction leads to dual map L0 : W 0 → V 0 for a linear map L : V → W . This dual map is a substitute for the adjoint to L and is related to the transpose of the matrix representation of L. The deﬁnition is L0(g) = g ◦ L. Thus if g ∈ W 0 we get a linear function g ◦ L : V → F since L : V → W . The dual to L is often denoted L0 = Lt. The dual map satisﬁes (L(x)|g) = (x|L0(g)) for all x ∈ V and g ∈ W 0.

Generalized Fredholm Alternative. L : V → W be a linear map between ﬁnite dimensional inner product spaces. Then

ker(L) = im(L0)◦, ker(L0) = im(L)◦, ker(L)◦ = im(L0), ker(L0)◦ = im(L).

Proof. Using L00 = L and M ◦◦ = M, the four statements are equivalent. ker L = {x ∈ V : Lx = 0}. im(L) = {Lx : x ∈ V }, im(L0) = {L0(g): g ∈ W }, im(L0)◦ = {x ∈ V :(x|L0(g)) = 0 for all g ∈ W } = {x ∈ V : g(L(x)) = 0 for all g ∈ W }. If x ∈ ker L ⇒ x ∈ im(L0)◦. Conversely, if g(L(x)) = 0 for all g ∈ W ⇒ Lx = 0 ⇒ x ∈ ker L.

Rank Theorem. L : V → W be a linear map between ﬁnite dimensional inner product spaces. Then rank(L) = rank(L0).

Proof. dim V = dim(ker(L)) + dim(im(L)) = dim(im(L0)◦) + dim(im(L)) = dim V − dim(im(L0)) + dim(im(L)). Linear Algebra Igor Yanovsky, 2005 23

6 Problems

Cross Product: a = (a1, a2, a3), b = (b1, b2, b3): ¯ ¯ ¯ ¯ ¯ i j k ¯ ¯ ¯ a × b = ¯ a1 a2 a3 ¯ = (a2b3 − a3b2)i + (a3b1 − a1b3)j + (a1b2 − a2b1)k ¯ ¯ b1 b2 b3 µ ¯ ¯ ¯ ¯ ¯ ¯ ¶ ¯ a a ¯ ¯ a a ¯ ¯ a a ¯ = ¯ 2 3 ¯ , ¯ 3 1 ¯ , ¯ 1 2 ¯ . ¯ b2 b3 ¯ ¯ b3 b1 ¯ ¯ b1 b2 ¯ Problem (F’03, #9). Consider a 3x3 real symmetric matrix with determinant 6. Assume (1, 2, 3) and (0, 3, −2) are eigenvectors with eigenvalues 1 and 2. a) Give an eigenvector of the form (1, x, y) for some real x, y which is linearly independent of the two vectors above. b) What is the eigenvalue of this eigenvector.

Proof. a) Since A is real and symmetric, A is self-adjoint. By the spectral theorem, its eigenvectors are orthonormal. v1 · v2 = (1, 2, 3) · (0, 3, −2) = 0, so the two vectors are orthogonal. We¯ cross the v1¯ and v2 to obtain a linearly indepentent vector v3. ¯ i j k ¯ µ ¯ ¯ ¯ ¯ ¯ ¯ ¶ ¯ ¯ ¯ 2 3 ¯ ¯ 3 1 ¯ ¯ 1 2 ¯ v = v × v = ¯ 1 2 3 ¯ = ¯ ¯ , ¯ ¯ , ¯ ¯ = (−13, 2, 3). 3 1 2 ¯ ¯ ¯ 3 −2 ¯ ¯ −2 0 ¯ ¯ 0 3 ¯ ¯ 0 3 −2 ¯ v3 2 3 Thus, the required vector is e3 = −13 = (1, − 13 , − 13 ). b) Since A is self-adjoint, by the spectral theorem, there exists an orthonormal basis of eigenvectors and a real diagonal matrix D such that ¯ ¯ ¯ ¯ ¯ λ1 0 0 ¯ ∗ ¯ ¯ ∗ A = ODO = [e1 e2 e3] ¯ 0 λ2 0 ¯ [e1 e2 e3] . ¯ ¯ 0 0 λ3

Since O is orthotonal, OO∗ = I, i.e. O∗ = O−1, and A = ODO−1. Note det A = −1 −1 1 det(ODO ) = det(O) det(D) det(O ) = det(O) det(D) det(O) = det(D) = λ1λ2λ3. Thus 6 = det A = det D = λ1λ2λ3, and λ1 = 1, λ2 = 2, then λ3 = 3. Linear Algebra Igor Yanovsky, 2005 24

Problem (S’02, #9). Find the matrix representation in the standard basis for either rotation by an angle θ in the plane perpendicular to the subspace spanned by vectors (1, 1, 1, 1) and (1, 1, 1, 0) in R4.

Proof. x1 = (1, 1, 1, 1), x2 = (1, 1, 1, 0). span{(1, 1, 1, 1), (1, 1, 1, 0)} = span{e1, e2}, orthogonal complement → span{e3, e4}, where rotation happens.   1 0 0 0 £ ¤  0 1 0 0  £ ¤∗ T = e e e e   e e e e 1 2 3 4  0 0 cos(θ) ∓ sin(θ)  1 2 3 4 0 0 ± sin(θ) cos(θ)

            1 1 Ã 1 1 ! 1 1 1  1   1   1  1  1  1  1  1  1  e =   , z = x − (x |e )e =   −   ·     =   , 1 2  1  2 2 2 1 1  1   1  2  1  2  1  4  1  1 0 0 1 1 −3   1   z2 1  1  ⇒ e2 = = √   ||z2|| 2 3 1 −3         1 1 1 1  −1   1  1  −1  1  1  orthogonal complement   ,   basis for ⊥; ⇒ e3 = √   , e4 = √    0   −2  2  0  6  −2  0 0 0 0 Linear Algebra Igor Yanovsky, 2005 25

Problem (F’01, #8). T : R3 → R3 rotation by 60◦ counterclockwise about the plane perpendicular to (1, 1, 1). S : R3 → R3 reﬂection about the plane perpendicular to (1, 0, 1). Determine the matrix representation of S◦T in the standard basis {(1, 0, 0), (0, 1, 0), (0, 0, 1)}. Proof. Rotation:   1 0 0 £ ¤ £ ¤∗ T = e1 e2 e3  0 cos(θ) − sin(θ)  e1 e2 e3 0 sin(θ) cos(θ)   1 0 0 √ £ ¤   £ ¤∗ T = e e e 0 1 − 3 e e e 1 2 3  √2 2  1 2 3 3 1 0 2 2 e1, e2, e3 orthonormal basis; e1 = e2 × e3, e2 = e3 × e1, e3 = e1 × e2.       1 1 1 1 1 1 e1 = √  1  , e2 = √  −1  , e3 = ±√  1  . 3 1 2 0 6 −2 ¯ ¯ ¯ i j k ¯ µ ¯ ¯ ¶ ¯ ¯ ¯ 1 1 ¯ Check: e = e ×e = √1 √1 ¯ 1 1 1 ¯ = √1 √1 ¯ ¯ , ··· , ··· = ( √1 , ··· , ··· ), 3 1 2 3 2 ¯ ¯ 3 2 ¯ −1 0 ¯ 6 ¯ 1 −1 0 ¯   1 1 ⇒ e3 = +√  1  . 6 −2       √1 √1 √1 1 0 0 √1 √1 √1  3 2 6   √   3 3 3  T = √1 − √1 √1 0 1 − 3 √1 − √1 0 = PT P −1.  3 2 6   √2 2   2 2  θ √1 0 − √2 3 1 √1 √1 − √2 3 6 0 2 2 6 6 6 Reﬂection:   −1 0 0 £ ¤ £ ¤∗ S = f1 f2 f3  0 1 0  f1 f2 f3 0 0 1       1 0 1 1 1 f1 = √  0  , f2 =  1  , f3 = ±√  0  . 2 1 0 2 −1 ¯ ¯ ¯ i j k ¯ µ ¯ ¯ ¶ ¯ ¯ ¯ 0 1 ¯ Check: f = f × f = √1 ¯ 1 0 1 ¯ = √1 ¯ ¯ , ··· , ··· = (− √1 , ··· , ··· ), 3 1 2 2 ¯ ¯ 2 ¯ 1 0 ¯ 2 ¯ 0 1 0 ¯   1 1 ⇒ f3 = −√  0  . 2 −1       √1 0 − √1 √1 0 √1 2 2 −1 0 0 2 2     −1 S =  0 1 0   0 1 0   0 1 0  = OSxO . √1 0 √1 0 0 1 − √1 0 √1 2 2 2 2 −1 −1 S ◦ T = OSxO PTθP . Linear Algebra Igor Yanovsky, 2005 26

Problem (F’02, #8). Let T be the rotation of an angle 60◦ counterclockwise about the origin in the plane perpendicular to (1, 1, 2) in R3. i) Find the matrix representation of T in the standard basis. Find all eigenvalues and eigenspaces of T . ii) What are the eigenvalues and eigenspaces of T if R3 is replaced by C3.

Proof. i)   1 0 0 √ £ ¤   £ ¤∗ T = e e e 0 1 − 3 e e e 1 2 3  √2 2  1 2 3 3 1 0 2 2 e1, e2, e3 orthonormal basis; e1 = e2 × e3, e2 = e3 × e1, e3 = e1 × e2.       1 1 1 1 1 1 e1 = √  1  , e2 = √  −1  , e3 = ±√  1  . 6 2 2 0 3 −1 ¯ ¯ ¯ i j k ¯ µ ¯ ¯ ¶ ¯ ¯ ¯ 1 2 ¯ Check: e = e ×e = √1 √1 ¯ 1 1 2 ¯ = √1 √1 ¯ ¯ , ··· , ··· = ( √1 , ··· , ··· ), 3 1 2 6 2 ¯ ¯ 6 2 ¯ −1 0 ¯ 3 ¯ 1 −1 0 ¯   1 1 ⇒ e3 = +√  1  . 3 −1 3 Know T (e1) = e1, so λ = 1 is an eigenvalue. Eigenspace = span{e1}. If z ∈ R , z = αe + w, w ∈ span{e , e }. T (z) = αe +T (w) . So if T (z) = λz, must have 1 2 3 1 | {z } rot. by 60◦ λ(αe + w) = αe +T (w) . ⇒ λαe = αe and T (w) = λw ← impossible unless 1 1 | {z } 1 1 ∈ span{e2,e3} w = 0. No more eigenvalues or eigenvectors.

Any 3-D rotation has 1 for an eigenvalue: any vector lying along the axis of the rotation is unchanged by the rotation, and is therefore an eigenvector corresponding to eigenvalue 1. The line formed by these eigenvectors is the axis of rotation. For the spe- cial case of the null rotation, every vector in 3-D space is an eigenvector corresponding to eigenvalue 1. Any 3-D reflection has two eigenvalues: -1 and 1. Any vector orthogonal to the plane of the mirror is reversed in direction by the reflection, without its size being changed; that is, the reflected vector is -1 times the original, and so the vector is an eigenvector corresponding to eigenvalue -1. The set formed by all these eigenvectors is the line orthogonal to the plane of the mirror. On the other hand, any vector in the plane of the mirror is unchanged by the reflection: it is an eigenvector corresponding to eigenvalue 1. The set formed by all these eigenvectors is the plane of the mirror. Any vector that is neither in the plane of the mirror nor orthogonal to it is not an eigenvector of the reflection.  1 0 0  √  ii) T : C3 → C3; 0 1 − 3 , χ = (t − 1)(t2 − t + 1) ⇒ roots are t =  √2 2  3 1 0 2 2 i π −i π 1, e 3 , e 3 . These are then the three distinct eigenvalues, each with one complex Linear Algebra Igor Yanovsky, 2005 27

" √ # 1 3 · ¸ ±i π − α dimensional eigenspace. Find e 3 eigenspaces for √2 2 , and then , 3 1 β · ¸ 2 2 γ 2 3 3 i π ∈ C eigenspaces for T : C → C , span{e } ↔ λ = 1, span{αe + βe } ↔ e 3 , δ 1 2 3 −i π span{γe2 + δe3} ↔ e 3 . Problem (S’03, #8; W’02, #9; F’01, #10). Let V be n-dimensional complex vector space and T : V → V a linear operator. χT has n distinct roots. Show T is diagonalizable. Let V be n-dimensional complex vector space and T : V → V a linear operator. Let v1, . . . , vn be non-zero vectors of distinct eigenvalues in V . Prove that {v1, . . . , vn} is linearly independent.

Proof. Since F = C, any root of χT is also an eigenvalue, so we have λ1, . . . , λn distinct eigenvalues. Induction on n = dim V . n = 1 ⇒ trivially linearly independent. n > 1, v1, . . . , vn are non-zero vectors in V with λ1, . . . , λn distinct eigenvalues. If

α1v1 + ··· + αnvn = 0, (6.1) want to show αi’s = 0.

T (α1v1 + ··· + αnvn) = T (0) = 0,

α1T v1 + ··· + αnT vn = 0,

α1λ1v1 + ··· + αnλnvn = 0. (6.2)

Multiplying (6.1) by λn and subtracting oﬀ (6.2), we get

α1(λn − λ1)v1 + ··· + αn−1(λn − λn−1)vn−1 = 0.

Since {v1, . . . , vn−1} are linearly independent, and λi 6= λj, i 6= j, ⇒ α1 = ··· = αn−1 = 0. Then by (6.1), αnvn = 0 ⇒ αn = 0, since vn is non-zero. Thus, α1 = ··· = αn = 0, and {v1, . . . , vn} are linearly independent. Having shown {v1, . . . , vn} are linearly independent, they generate an n-dimensional subspace which is then all of V . Hence {v1, . . . , vn} gives a basis. Linear Algebra Igor Yanovsky, 2005 28

Problem (F’01, #9). Let A be a real symmetric matrix. Prove that there exists an invertible matrix P such that P −1AP is diagonal.

Proof. Let V = Rn with the standard inner product. Let A be real symmetric matrix ⇒ At = A ⇒ A is self-adjoint. Let T be the linear operator on V which is represented by A in the standard order basis.

[A]S VS −−−−→ VS x x   P  P

[A]B VB −−−−→ VB

Since T is self-adjoint on V , there exists an orthonormal basis β = {v1, . . . , vn} of eigenvectors of T . Then T vi = λvi, i = 1, . . . , n, where λi’s are eigenvalues of T . Let D = [A]B. Let P be the matrix with v1, . . . , vn as column vectors. Then

−1 −1 −1 −1 [A]B = P [A]SP = P [A]S[v1 ··· vn] = P [Av1 ··· Avn] = P [λ1v1 ··· λnvn]

Since with choice, P is orthonormal with real entries, detP = 1 (P invertible) ⇒ P −1 = P t.     v1 λ1 ··· 0 t  .   . . .  [A]B = P [λ1v1 ··· λnvn] =  .  [λ1v1 ··· λnvn] =  . .. .  vn 0 ··· λn

t −1 since vi’s are orthonormal. Then D = P [A]SP , D = P AP . Problem (S’03, #9). t t Let A ∈ M3(R) satisfy det(A) = 1 and A A = AA = IR3 . Prove that the characteristic polynomial of A has 1 as a root (i.e. 1 is an eigenvalue of A).

3 Proof. χA(t) = t + · · · − 1 = (t − λ1)(t − λ2)(t − λ3), λ1, λ2, λ3 ∈ C, using the fundamental theorem of algebra. A real ⇒ 1 root of odd degree. λ1 ∈ R. Case 1: λ2, λ3 ∈ R; Case 2: λ2 = λ, λ3 = λ¯. det(A) = 1 = λ1λ2λ3, since determinant is a product of eigenvalues. t t ∗ ¯T T A A = AA = IR3 ⇒ A orthogonal ⇒ A ∈ M3(C) is unitary since A = A = A . λ1, λ2, λ3 eigenvalues for A as a unitary transformation, so if Axi = λixi, then

∗ 2 (xi|xi) = (U Uxi|xi) =(U is unitary)= (Uxi|Uxi) = (λixi|λixi) = |λi| (xi|xi)

2 ⇒ |λi| = 1. Case 1: λ2, λ3 ∈ R ⇒ λ2, λ3 = ±1 and λ2λ3 = 1, so one or three eigenvalues = +1. 2 Case 2: λ1λ2λ3 = λ1λλ¯ = λ1|λ| = λ1 = 1. Linear Algebra Igor Yanovsky, 2005 29

Problem (S’03, #10). Let T : Rn → Rn be symmetric3, tr(T 2) = 0. Show that T = 0.

Proof. By spectral theorem, T = ODO∗, O is orthogonal and D is diagonal with real entries.   λ1 ··· 0 2 ∗ ∗ 2 ∗  . . .  T = ODO ODO = OD O , where D =  . .. .  . 0 ··· λn

2 2 ∗ ∗ 2 2 2 2 0 = tr(T ) = tr(OD O ) = tr(O OD ) = tr(D ) = λ1 + ··· + λn. 2 ⇒ λi = 0 since λi ≥ 0. Problem (W’02, #10). Let V be a ﬁnite dimensional complex inner product space and f : V → C a linear functional. Show f(x) = (x|y) for some y.

Proof. Select e1, . . . , en orthonormal basis, and let y = f(e1)e1 + ··· + f(en)en.

(x|y) = (x|f(e1)e1 + ··· + f(en)en) = f(e1)(x|e1) + ··· + f(en)(x|en)

= f((x|e1)e1 + ··· + (x|en)en) = f(x), since f is linear. We can also show that y is unique. Suppose y0 is another vector in V for which f(x) = (x|y0) for every x ∈ V . Then (x|y) = (x|y0) for all x, so (y − y0|y − y0) = 0, and y = y0.

Problem (S’02, #11). Let V be a ﬁnite dimensional real inner product space and T,S : V → V two commuting (i.e. ST = TS) self-adjoint linear operators. Show that there exists an orthonormal basis that simultaneously diagonalizes S and T .

Proof. Since T,S are self-adjoint, ∃ an ordered orthonormal basis {v1, . . . , vn} of eigenvectors corresponding to eigenvalues λ1, . . . , λn for T . vi ∈ Eλi (T ).

V = Eλ1 (T ) ⊕ · · · ⊕ Eλn (T ). vi ∈ Eλi (T ) ⇒ T vi = λivi.

T Svi = ST vi = Sλivi = λiSvi ⇒ Svi ∈ Eλi (T ).

Thus Eλi (T ) is invariant under S, i.e. S : Eλi (T ) → Eλi (T ).

Since S|Eλ (T ) is self-adjoint, ∃ an ordered orthonormal basis βi of eigenvectors of S for i Sn Eλi (T ). β = i=1 βi.

3symmetric in R ⇒ self-adjoint ≡ hermitian. Linear Algebra Igor Yanovsky, 2005 30

Problem (S’02, #10). Let V be a complex inner product space and W a ﬁnite dimensional subspace. Let v ∈ V . Prove that there exists a unique vector vW ∈ W such that

||v − vW || ≤ ||v − w||, ∀w ∈ W.

Deduce that equality holds if and only if w = vW .

Proof. vW is supposed to be the orthogonal projection of v onto W . Choose an orthonormal basis e1, . . . , en for W . Then deﬁne

projW (x) = (x|e1)e1 + ··· + (x|en)en.

Claim: xW = projW (x). Show: x − projW (x) ⊥ projW (x). ¡ ¯ ¢ x − (x|e )e − · · · − (x|e )e ¯ (x|e )e + ··· + (x|e )e ¡ ¯ 1 1 n ¢n ¡ 1 1 n n ¯ ¢ = x¯ (x|e1)e1 + ··· + (x|en)en − (x|e1)e1 + ··· + (x|en)en ¯ (x|e1)e1 + ··· + (x|en)en ¡ Xn Xn ¢ = (x|e1)(x|e1) + ··· + (x|en)(x|en) − (x|ei)(x|ej)(ei|ej) i=1 j=1 ¡ Xn ¢ = (x|e1)(x|e1) + ··· + (x|en)(x|en) − (x|ei)(x|ei) = 0 since (ei|ej) = δij. i=1

In fact, projW (x) ⊥ x − projW (x) and W 3 w ⊥ x − projW (x). 2 2 ||x − projW (x) + projW (x) − w|| = ||x − w|| ⇒ ||x − projW (x)|| ≤ ||x − w||, with equality when ||projW (x) − w|| = 0 ⇔ projW (x) = w. Show: x − projW (x) ⊥ w ∈ W . ¡ ¯ ¢ x − (x|e )e − · · · − (x|e )e ¯ (w|e )e + ··· + (w|e )e ¡ ¯ 1 1 n n¢ ¡ 1 1 n n ¯ ¢ = x¯ (w|e1)e1 + ··· + (w|en)en − (x|e1)e1 + ··· + (x|en)en ¯ (w|e1)e1 + ··· + (w|en)en ¡ Xn Xn ¢ = (w|e1)(x|e1) + ··· + (w|en)(x|en) − (x|ei)(w|ej)(ei|ej) i=1 j=1 ¡ Xn ¢ = (w|e1)(x|e1) + ··· + (w|en)(x|en) − (x|ei)(w|ei) = 0 since (ei|ej) = δij. i=1

Problem (F’03, #10). a) Let t ∈ R such that t is not an integer multiple of π. For the matrix · ¸ cos(t) sin(t) A = − sin(t) cos(t)

−1 prove there does not exist a real valued· matrix¸ B such that BAB is a diagonal matrix. 1 λ b) Do the same for the matrix A = , where λ ∈ R \{0}. 0 1 · ¸ cos(t) − λ sin(t) Proof. a) det(A − λI) = = λ2 − 2λ cos t + 1. − sin(t) cos(t) − λ √ 2 ⇒ λ1,2 = cos t± cos t − 1. ⇒ λ1,2 = a±ib, b 6= 0, i.e. λ1,2 ∈/ R. Hence, eigenvectors Linear Algebra Igor Yanovsky, 2005 31

−1 are not real, and @B ∈ M(R), such· that BAB¸ · is¸ diagonal.· ¸ 0 λ w 0 b) λ = 1. We ﬁnd eigenvectors, 1 = . Thus, both eigenvectors 1,2 0 0 w 0 · ¸ 2 1 are v = , i.e. linearly dependent ⇒ there does not exist a basis for R2 1,2 0 consisting of eigenvectors of A. Therefore, @B ∈ M(R), such that BAB−1 is diagonal. Linear Algebra Igor Yanovsky, 2005 32

Problem (F’02, #10) (Spectral Theorem for Normal Operators). ∗ ∗ Let A ∈ Matn×n(C) satisfying A A = AA , i.e. A is normal. Show that there is an orthonormal basis of eigenvectors of A. Rephrase: For L : V → V , V complex ﬁnite dimensional inner product space.

Proof. Prove this by induction on dim V . Since L is complex linear, we can use the Fundamental Theorem of Algebra to ﬁnd λ ∈ C ∗ ∗ and x ∈ V \{0}, so that L(x) = λx ⇒ L (x) = λx¯ . ker(L−λIV ) = ker(L −λI¯ V ). Let x⊥ = {z ∈ V :(z|x) = 0}, an orthogonal complement to x. To get induction, we need to show that x⊥ is invariant under L, i.e. L(x⊥) ⊂ x⊥. Let z ∈ x⊥ and show Lz ∈ x⊥. (L(z)|x) = (z|L∗(x)) = (z|λx¯ ) = λ(z|x) = 0.

Problem (W’02, #11). Let V be a ﬁnite dimensional complex inner product space and L : V → V a linear transformation. Show that we can ﬁnd an orthonormal basis so [L] is upper triangular.

Proof. Assume [L] is upper triangular with respect to e1, . . . , en   α11 α12 ··· α1n    0 α22 ··· α2n  ⇒ [L(e1) ··· L(en)] = [e1 ··· en]  . . . .  .  ......  0 ··· 0 αnn

L(e1) = α11e1 ⇒ e1 is an eigenvector with eigenvalue α11. dim V = 2: L : V → V complex ⇒ can pick e1 ∈ V so that Le1 = α11e1; pick e2 ⊥ e1. · ¸ α11 α12 ⇒ [L(e1) L(e2)] = [e1 e2] ← upper triangular. 0 α22

L(e1) = α11e1, L(e2) = α12e1 + α22e2. Observe: L(ek) ∈ span{e1, . . . , ek} ⇒ L(e1),...,L(ek) ∈ span{e1, . . . , ek}. So we have {0} = M0 ⊂ M1 ⊂ · · · ⊂ Mk−1 ⊂ Mk = V with the property dim Mk = k, L(Mk) ⊂ Mk = span{e1, . . . , ek}. Enough to show that any linear transformation on an n-dimensional space has an (n − 1)-dim subspace M ⊂ V . Keep using this result then we can generate such an increasing sequence {0} = M0 ⊂ M1 ⊂ · · · ⊂ Mk−1 ⊂ M = V that are all invariant under L. M1 = span{e1}, |e1| = 1. T ⊥ e2 = M2 M1 , e2 is an orthogonal complement of e1. ··· Pick e1, . . . , ek orthonormal basis such that span{e1, . . . , ek} = Mk. L(e1) ∈ M1, L(e1) = α11e1, L(e2) ∈ M2, L(e2) = α12e1 + α22e2, · · · · · · · · · ← Get an upper triangular form for [L]. L(ek) ∈ Mk, L(ek) = α1ke1 + α2ke2 + ··· + αkkek. To construct M ⊂ V we select x ∈ V \{0} such that L∗(x) = λx.(L∗ : V → V is complex and linear, so by the Fundamental Theorem of Algebra can ﬁnd x, λ, so that L∗(x) = λx). Then M = x⊥. dim M = k − 1 ← have to show M is invariant under L. Take z ⊥ x: (L(z)|x) = (z|L∗(x)) = (z|λx) = λ¯(z|x) = 0. ⇒ L(z) ∈ x⊥. So L(M) ⊂ M and M has dimension k − 1. Linear Algebra Igor Yanovsky, 2005 34

Problem (F’02, #7; F’01, #7). Let T : V → W be a linear transformation of ﬁnite dimensional real vector spaces. Deﬁne the transpose of T and then prove both of the following: 1) im(T )0 = ker(T t). 2) rank(T ) = rank(T t) (dim im(T ) = dim im(T t)).

Proof. Transpose = dual. Let T : V → W be linear. Let T t = T 0 : W 0 → V 0, where 0 t 0 0 X = homR(X, R). T : W → V is linear.

t T 0(g) = g ◦ TV →T W, V 0 ←T W 0

1) This is a proof of the Generalized Fredholm Alternative. ker T 0 = {g ∈ W 0 : T 0(g) = g ◦ T = 0} im(T ) = {T (x): x ∈ V } im(T )◦ = {g ∈ W 0 : g(T (x)) = 0 for all x ∈ V } g(T (x)) = 0 for all x ∈ V ⇔ g ◦ T = 0 ⇔ g ∈ ker T 0. 2) This is a proof of the Generalized Rank Theorem. rank(T ) = dim(im(T )), rank(T t) = dim(im(T t)). T t : W 0 → V 0. Dimension formula: dim W 0 = dim(ker(T t)) + dim(im(T t)) = dim(im(T ))◦ + dim(im(T t)) = dim W 0 − dim(im(T )) + dim(im(T t)).

Problem (W’02, #8). Let T : V → W and S : W → X be linear transformations of ﬁnite dimensional real vector spaces. Prove that

rank(T ) + rank(S) − dim(W ) ≤ rank(S ◦ T ) ≤ min{rank(T ), rank(S)}. T Proof. Note: V →T W →S X. rank(S ◦ T ) = rank(T ) − dim(im T ker S). rank(T ) + rank(S) − dim(W ) = rank(T ) + rank(S) − dim(ker S) − rank(S) \ = rank(T ) − dim(ker S) = rank(S ◦ T ) + dim(im T ker S) − dim(ker S) ≤ rank(S ◦ T ) | {z } ≤0 Note: M ⊂ V subspace, dim(L(M)) ≤ dim M, a consequence of dimension formula. rank(S ◦ T ) = dim((S ◦ T )(V )) = dim(S(T (V ))) ≤ dim(T (V )) = rank(T ) ≤ dim(S(W )) = rank(S). Alternatively, to prove rank(S ◦ T ) ≤ rank(S) , note that since T (V ) ⊂ W , we also have S(T (V )) ⊂ S(W ) and so dim S(T (V )) ≤ dim S(W ) . Then rank(S ◦ T ) = dim((S ◦ T )(V )) = dim(S(T (V ))) ≤ dim S(W ) = rank(S). Linear Algebra Igor Yanovsky, 2005 35

Problem (S’03, #7). Let V be a ﬁnite dimensional real vector space. Let W ⊂ V be a subspace and 0 W = {f : V → F linear | f = 0 on W }. Let W1,W2 ⊂ V be subspaces. Prove that \ ◦ ◦ ◦ W1 W2 = (W1 + W2) .

0 0 Proof. W = {f ∈ V | f|W = 0}. ◦ ◦ ◦ ◦ T ◦ Write similar deﬁnitions for W1 ,W2 , (W1+W2) , and W1 W2 and make observations. \ ◦ ◦ ◦ ◦ ◦ ◦ 1) (W1 + W2) ⊂ W1 ,W2 ⇒ (W1 + W2) ⊂ W1 W2 . \ ◦ ◦ ◦ 2) Suppose f ∈ W1 W2 ⇒ f|W1 = 0, f|W2 = 0 ⇒ f|W1+W2 = 0 ⇒ f ∈ (W1 + W2) . ◦ T ◦ ◦ Thus, W1 W2 ⊂ (W1 + W2) . Problem (S’02, #8). Let V be a ﬁnite dimensional real vector space. Let M ⊂ V be a subspace and M 0 = {f : V → F linear | f = 0 on M}. Prove that

dim(V ) = dim(M) + dim(M ◦).

Problem (F’03, #8). Prove the following three statements. You may choose an order of these statements and then use the earlier statements to prove the later statements. a) If L : V → W is a linear transformation between two ﬁnite dimensional real vector spaces V,W , then

dim im L = dim V − dim ker(L). b) If L : V → V is a linear transformation on a ﬁnite dimensional real inner product space and L∗ is its adjoint, then im(L∗) is the orthogonal complement of ker(L) in V . c) Let Anxn be a real matrix, then the maximal number of linearly independent rows (row rank) equals the maximal number of linearly independent columns (column rank).

Proof. We prove (a) and (b) separately. (a),(b) ⇒ (c). a) We know that dim ker(L) ≤ dimTV and that it has a complement M of dimension k = dim V − dim ker(L). Since M ker(L) = {0} the linear map L must be 1-1 when restricted to M. Thus L|M : M → im(L) is an isomorphism, i.e. dim im(L) = dim M = k. b) We want to show ker(L)⊥ = im(L∗). Since M ⊥⊥ = M, we can prove ker(L) = im(L∗)⊥. ker L = {x ∈ V : Lx = 0}.V →L W ∗ im(L) = {Lx : x ∈ V },W →L V im(L∗) = {L∗y : y ∈ W }, im(L∗)⊥ = {x ∈ V :(x|L∗y) = 0 for all y ∈ W } = {x ∈ V :(Lx|y) = 0 for all y ∈ W }. If x ∈ ker L ⇒ x ∈ im(L∗)⊥. Conversely, if (Lx|y) = 0, ∀y ∈ W ⇒ Lx = 0 ⇒ x ∈ ker L. c) Using Dimension formula (a) and Fredholm Alternative (b), we have the Rank theorem: dim V = dim(ker(A)) + dim(im(A)) = dim(im(A∗))⊥ + dim(im(A)) = dim V − dim(im(A∗)) + dim(im(A)). Thus, rank(A) = rank(A∗). Conjugation does not change the rank, so rank(A) = rank(AT ). rank(A) is the column rank. rank(AT ) is the row rank of A. Thus, row rank (A) = column rank (A). We have not proved that the conjugation does not change the rank. To establishL this result easier, use (b) where we showed im(L∗) = ker(L)⊥. Since V = ker L imL, ker(L)⊥ = im(L), which establishes im(L∗) = im(L).