MTH 102A - Part 1 Linear Algebra 2019-20-II Semester

Arbind Kumar Lal1

February 3, 2020

1Indian Institute of Technology Kanpur   1 2 • Matrix A =   behaves like the scalar 3 when multiplied with 2 1         1 1 2 1 1 x =  . That is,     = 3  . 1 2 1 1 1 • Physically: The Linear function f (x) = Ax magnifies the nonzero   1 2 vector   ∈ C three (3) times. 1     1 1 • Similarly, A   = −1  . So, behaves by changing the −1 −1   1 direction of the vector   −1   1 2 2 • Take A =  . Do I have a nonzero x ∈ C which gets 1 3 magnified by A? • So I am looking for x 6= 0 and α s.t. Ax = αx. Using x 6= 0, we have Ax = αx if and only if [αI − A]x = 0 if and only if det[αI − A] = 0.   √ α − 1 −2 2 • det[αI − A] = det   = α − 4α + 1. So α = 2 ± 3. −1 α − 3  √  √ 1 + 3 −2 • Take α = 2 + 3. To find x, solve  √  x = 0: −1 3 − 1 √  3 − 1 using GJE, for instance. We get x =  . Moreover 1   √  √  √  1 2 3 − 1 3 + 1 √ 3 − 1 Ax =     =  √  = (2 + 3)   . 1 3 1 2 + 3 1 n • We call λ ∈ C an eigenvalue of An×n if there exists x ∈ C , x 6= 0 s.t. Ax = λx. We call x an eigenvector of A for the eigenvalue λ. We call (λ, x) an eigenpair. • If (λ, x) is an eigenpair of A, then so is (λ, cx), for each c 6= 0, c ∈ C. • Th The number λ is an eigenvalue of An×n if and only if det(λI − A) = 0. Proof: Follows from: there exists a nontrivial solution of [λI −A]x = 0 if and only if det[λI −A] = 0.   1 3 • Find eigenpairs of A =  . −2 5 √ • To find the eigenvalues, solve det(λI − A) = 0. So λ = 3 ± 2i.   x √ • To find an eigenvector   for the eigenvalue 3 + 2i: y  √  2  √  E21( √ ) 2 + 2i −3 0 2+ 2i 2 + 2i −3 0 •  √  ⇒  . So 2 2i − 2 0 0 0 0     x 3√   =  2+ 2i . y 1    √ 3√ 3 + 2i,  2+ 2i . 1

• Let D be a diagonal matrix. Then eigenpairs of D are (dii , ei ).

Ques: Should An×n have n eigenvalues (including multiplicities)?. Yes. • While discussing eigenvalues etc, we consider square matrices only. • Ques: What are the eigenvalues of an upper triangular matrix? Diagonal entries. • The multiset of eigenvalues of A is called the spectrum of A. Notation: σ(A). Ques: Let λ ∈ σ(A). Must we have at least one eigenvector for λ? Yes, by GJE. • The multiplicity of λ in σ(A) is called the algebraic multiplicity of A. Ques: Suppose that algebraic multiplicity of λ in σ(A) is 2. Must we have 2 lin.ind eigenvectors for λ? No. For example, take   0 1 A= . Then σ(A)={0, 0} and e1 is an eigenvector. That is, 0 0 Ae1 = 0. If x is another eigenvector, then Ax = 0. That is,         0 1 x1 x2 0     =   =  . So x2 = 0 and x = x1e1. 0 0 x2 0 0 • [If (λ1, x1),..., (λk , xk ) are eigenpairs, then x1,..., xk may not

be linearly independent unless λi ’s are distinct!!] • If λ ∈ σ(A), then the maximum number of linearly independent eigenvectors for λ is the geometric multiplicity of λ. [Geometry: maximum k such that the LT magnifies a

k-dimensional subspace by λ. Take LT: e1 → e1, e2 → e2,

e3 → e1 + e3. • Thm: Let (λi , vi ) be some eigenpairs of An×n, λi ’s are distinct.

Then v1,..., vk are linearly independent. [Contrast it with the earlier comment.]

Proof: Suppose that v1,..., vk are lin.dep. Then there exists `

smallest, and β s.t. v`+1 = β1v1 + ··· + β`v`. Note that β 6= 0. So

Av`+1 = A[β1v1 +···+ β`v`] = λ1β1v1 + ··· + λ`β`v`

λ`+1v`+1 = λ`+1β1v1 + ··· + λ`+1β`v`

0 = [λ`+1 − λ1]β1v1 + ··· + [λ`+1 − λl ]β`v`.

So v` ∈ LS(v1,..., v`−1), a contradiction. • The converse of the above is not true. Think of I. • 0 ∈ σ(A) if and only if Null SpA 6= {0} if and only if A is singular. • Let A be invertible. Then λ ∈ σ(A) if and only if λ−1 ∈ σ(A−1). • pA(x) = det(xI − A) is called the characteristic polynomial of A.

The equation pA(x) = 0 is called the characteristic equation of A. So eigenvalues are roots of the characteristic equation. • Let λ ∈ σ(A). The eigenspace for λ = {x : Ax = λx}. dim ({x : Ax = λx}) = the geometric multiplicity of λ. • Ques: Is σ(A) = σ(AT )? Yes, same characteristic polynomial:

pA(x) = pAT (x).

• Let A∈Mn(R). Then pA(λ) = 0 if and only if pA(λ) = 0. So λ∈σ(A) if and only if λ∈σ(A). [This is not true for complex matrices.] • Similar: A and B are said to be similar matrices if there exists an invertible matrix S s.t. A = SBS−1. Thm: Similar matrices have the same characteristic polynomial. Proof: Let A be similar to B. So there exists S s.t B = S−1AS. ⇒ det(xI − B) = det(xI − S−1AS) = det(S−1(xI − A)S) = det(xI − A). • Let B be invertible. Then AB is similar to BA, as −1 BA = B(AB)B . So pAB(x) = pBA(x) and σ(AB) = σ(BA). −1 • Let B =S AS, λ∈σ(A) and v1,..., vk be lin.ind eigenvectors of A for λ. −1 −1 −1 Are S v1,..., S vk lin.ind? Yes, as S is invertible. Is S vi an eigenvector of B? −1 −1 −1 −1 −1 Yes, B(S vi ) = S ASS vi = S Avi = λ(S vi ). So the geometric multiplicity of λ in A and B are the same. [Geometry: if A has eigenvector v and T x = Ax, then T is said to have v as its eigenvector.] • Thm: Let λ ∈ σ(A). Then geo. multip of λ ≤ alg multip of λ.

Proof: Let geometric multiplicity of λ be k and v1,..., vk be

lin.ind eigenvectors for λ. Apply GS: let w1,..., wk be the orthonormal eigenvectors for λ. Extend it to an orthonormal basis:

{w1,..., wk , wk+1,..., wn}. Put P[:, i] = wi . ∗ ∗ h i • Then P AP = P Aw1 ··· Awk Awk+1 ··· Awn =  ∗  w1    .   .     ∗   wk  h i   λw ··· λw ∗ · · · ∗  ∗  1 k wk+1    .   .    ∗ wn   λ ··· 0 ∗ · · · ∗    ..  0 . 0 ∗ · · · ∗     0 ··· λ ∗ · · · ∗ =  .   0 ··· 0 ∗ · · · ∗    .   .    0 ··· 0 ∗ · · · ∗ • As P∗ = P−1 (P is unitary), ∗ k pA(x) = pP∗AP (x) = det(xI − P AP) = (x − λ) det(xI − D). So alg. multip of λ in A = alg. multip of λ in P∗AP ≥ k. ∗ • Th Let A ∈ Mn(C). Then there exists U unitary, s.t. T = U AU is upper triangular. Further, if A and σ(A) are real, then U can be real orthogonal. Proof: Use induction on n. Case n = 1 is trivial. Let n > 1. Let

(λ1, w1) be an eigenpair of A, kw1k = 1. Take an orthonormal h i basis {w1,..., wn} and form W = w1 ··· wn . Then,   ∗ λ1 ∗ 0 W AW =  , A ∈ Mn−1. 0 A0   ∗ λ1 ∗ 0 W AW =  , A ∈ Mn−1. 0 A0 • By induction hypothesis, there exists U0 unitary, s.t. T 0 = U0∗A0U0 is upper triangular.   1 0 • Put U = W  . Note: U is unitary. Now 0 U0     ∗ 1 0 ∗ 1 0 U AU =   W AW   0 U0∗ 0 U0           1 0 λ1 ∗ 1 0 λ1 ∗ λ1 ∗ =       =   =  . 0 U0∗ 0 A0 0 U0 0 U0∗A0U0 0 T 0 Further: ‘if A, λ are real then there exists x 6= 0 real, s.t. Ax = λx’. • Remark In SUT, as U∗AU = T , we have

{λ1, . . . , λn} = σ(A) = σ(T ) = {t11,..., tnn}.

Further, we can get the λi ’s in the diagonal of T in any prescribed order. Q • Cor Let σ(A) = {λ1, . . . , λn}. Then det A = λi and P TraceA = λi . Proof: By SUT, there exists U unitary, s.t. U∗AU = T is upper P P triangular. So σ(T ) = σ(A). So λi = tii = Trace(T ) = Trace(U∗AU) = Trace(AUU∗) = Trace(A). Q Q ∗ And λi = tii = det T = det U AU = det A. • By SUT, U∗AU = T and

{λ1, . . . , λn} = σ(A) = σ(T ) = {t11,..., tnn}.

• By diagA we denote the diagonal {a11,..., ann} of A. By diagv

we denote the diagonal matrix A with a11 = v1,..., ann = vn. • Th[spectral Theorem] Let A be normal (AA∗ = A∗A). Then there exists U unitary, s.t. U∗AU = D is diagonal. Further, diagD = σ(A). Proof: By SUT, there exists U unitary, s.t U∗AU = T is upper triangular. As A∗A = AA∗, we get T ∗T = TT ∗. Note: P 2 ∗ ∗ 2 |t1i | = (TT )11 = (T T )11 = |t11| . So t12 = ··· = t1n = 0. Repeating the process, T is diagonal. • Cor Let A be Hermitian. Then there exists U unitary, s.t. U∗AU = D is real diagonal. Further, if A is real, then U can be chosen as real orthogonal. Proof: By SUT, we can find U unitary, s.t. U∗AU = T is upper triangular. As A∗ = A, we get T ∗ = T . So T is real diagonal. • Multiply 0 ∗ ∗ ∗ ∗ ∗ ∗ ∗ 0 0 ∗ ∗       0 ∗ ∗ ∗ 0 0 ∗ ∗ 0 0 ∗ ∗           =   0 0 ∗ ∗ 0 0 ∗ ∗ 0 0 ∗ ∗       0 0 0 ∗ 0 0 0 ∗ 0 0 0 ∗

0 0 ∗ ∗ ∗ ∗ ∗ ∗ 0 0 0 ∗       0 0 ∗ ∗ 0 ∗ ∗ ∗ 0 0 0 ∗           =   0 0 ∗ ∗ 0 0 0 ∗ 0 0 0 ∗       0 0 0 ∗ 0 0 0 ∗ 0 0 0 ∗ 0 0 0 ∗ ∗ ∗ ∗ ∗ 0 0 0 0       0 0 0 ∗ 0 ∗ ∗ ∗ 0 0 0 0 •           =   0 0 0 ∗ 0 0 ∗ ∗ 0 0 0 0       0 0 0 ∗ 0 0 0 0 0 0 0 0

0 ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗         0 ∗ ∗ ∗ 0 0 ∗ ∗ 0 ∗ ∗ ∗ 0 ∗ ∗ ∗         So         = 0. 0 0 ∗ ∗ 0 0 ∗ ∗ 0 0 0 ∗ 0 0 ∗ ∗         0 0 0 ∗ 0 0 0 ∗ 0 0 0 ∗ 0 0 0 0 • Cor[Cayley-Hamilton]: Every matrix satisfies its characteristic equation.

Proof: Let σ(A) = {λ1, . . . , λn}. By SUT, there exists U unitary, ∗ s.t. U AU = T is upper triangular; tii = λi for each i. Let p(x) be Q the characteristic polynomial of A. So p(x) = (x − λi ). Q Q ∗ ∗ ⇒ p(A) = (A − λi I) = (UTU − λi UIU ) h i ∗ ∗ = U (T − λ1I)(T − λ2I) ··· (T − λnI) U = U0U = 0. • We say ‘A is diagonalizable’ if there exists S invertible, s.t. −1 S AS = D is diagonal with λi = dii as the eigenvalues of A.

Thm: An×n is diagonalizable if and only if it has n linearly independent eigenvectors. Proof: ⇒: Let A be diagonalizable. So there exists S invertible, −1 s.t. S AS = D is diagonal. Put si = S[:, i] and λi = dii . Then h i h i As1 ··· Asn = A s1 ··· sn = AS = SD h i h i = s1 ··· sn D = λ1s1 ··· λnsn .

That is, Asi = λi si . So s1,..., sn are n linearly independent eigenvectors. • ⇐: let s1,..., sn be n linearly independent eigenvectors, say

Asi = λi si . h i Put S = s1 ··· sn and D = diag(λ1, . . . , λn). Then h i h i AS = As1 ··· Asn = λ1s1 ··· λnsn = SD. So S−1AS = D.

• Cor Let An×n have n distinct eigenvalues. Then A is diagonalizable. Proof: Follows as the corresponding eigenvectors are linearly independent.   0 1 • Ques: Is A =   diagonalizable? No. Assume it is. Then 0 0 there exists S invertible, s.t. S−1AS = D is diagonal. As   −1 0 0 σ(A) = σ(S AS) = σ(D), we must have D =  . But then 0 0   −1 0 0 A = SDS =  , a contradiction. 0 0