Linear Algebra Lecture 14 - Quadratic Forms

Oskar Kędzierski

27 January 2020 Quadratic Form

Deﬁnition A function Q : Rn R is called a quadratic form if ÝÑ 2 2 Q x1,..., xn a11x1 . . . annxn aij xi xj , pp qq “ ` ` ` 1ďiăjďn ÿ that is, it is a function given by a homogeneous polynomial of degree 2 in variables x1,..., xn. Quadratic Form

2 2 Q x1, x2 x1 x2 pp qq “ ´ Quadratic Form

2 2 Q x1, x2 x1 x2 pp qq “ ´

2 2 2 Q x1, x2, x3 x1 2x2 5x3 2x1x2 2x1x3 2x2x3 pp qq “ ` ` ` ´ ` Symmetric Matrix

Recall Deﬁnition ⊺ Matrix A aij M n n; R is called symmetric if A A, i.e. “ r s P p ˆ q “ aij aji for i, j 1,..., n. “ “ Symmetric Matrix

Recall Deﬁnition ⊺ Matrix A aij M n n; R is called symmetric if A A, i.e. “ r s P p ˆ q “ aij aji for i, j 1,..., n. “ “ Example

0 2 5 matrix 2 4 3 is symmetric » ´ ﬁ 5 3 1 ´ – ﬂ Symmetric Matrix

Recall Deﬁnition ⊺ Matrix A aij M n n; R is called symmetric if A A, i.e. “ r s P p ˆ q “ aij aji for i, j 1,..., n. “ “ Example

0 2 5 matrix 2 4 3 is symmetric » ´ fi 5 3 1 ´ – fl 0 2 6 matrix 2 4 3 is not symmetric » ´ fi 5 3 1 ´ – fl Matrix of a Quadratic Form

Deﬁnition n 2 Let Q x1,..., xn 1 aii x 1 aij xi xj be a quadratic pp qq “ i“ i ` ďiăjďn form. The matrix of the quadratic form Q is a symmetric matrix ř ř 1 M bij M n n; R such that bii aii and bij bji aij for “ r s P p ˆ q “ “ “ 2 1 i j n. ď ă ď Example 2 2 1 0 The matrix of the form Q x1, x2 x1 x2 is M . pp qq “ ´ “ 0 1 „ ´  Matrix of a Quadratic Form

Proposition Let M be a matrix of the quadratic form Q : Rn R. Then ÝÑ ⊺ Q x1,..., xn x Mx, pp qq “ x1 where x . . “ » . fi x — n ffi – fl Matrix of a Quadratic Form (continued)

Proposition Let M be a matrix of the quadratic form Q : Rn R. Then ÝÑ ⊺ Q x1,..., xn x Mx, pp qq “ x1 where x . . “ » . fi x — n ffi – fl Proof. Entries of matrix M in the i-th row get multiplied by xi and elements in the j-th column get multiplied by xj . For every i j ‰ the monomial xi xj comes from the entry in the i-th row, j-th column and from the entry in the j-th row, i-th column. Positive/Negative Definite Quadratic Form

Definition Quadratic form Q : Rn R (resp. symmetric matrix ÝÑ M M n n; R ) is positive definite if Q x 0 (resp. P p ˆ q p qą x ⊺Mx 0) for any x Rn, x 0. ą P ‰ Positive/Negative Definite Quadratic Form

Definition Quadratic form Q : Rn R (resp. symmetric matrix ÝÑ M M n n; R ) is positive definite if Q x 0 (resp. P p ˆ q p qą x ⊺Mx 0) for any x Rn, x 0. ą P ‰ Quadratic form Q (resp. symmetric matrix M) is negative definite if Q x 0 (resp. x ⊺Mx 0) for any x Rn, x 0. p qă ă P ‰ Example The quadratic form 2 : Rn R is positive definite since 2 2 2}¨} ÝÑ x x1 . . . xn 0 for x 0. } } “ ` ` ą ‰ 2 2 The quadratic form Q x1, x2 x1 x2 is not positive definite pp qq “ ´ since Q 0, 1 1 0. It is not negative definite since pp qq “´ ă Q 1, 0 1 0. pp qq “ ą 2 2 2 The quadratic form Q x1, x2, x3 x1 2x2 5x3 2x1x2 pp qq2 “ ` `2 ` 2x1x3 2x2x3 x1 x2 x3 x2 2x3 is not positive ´ ` “ p ` ´ q ` p ` q definite since Q 3, 2, 1 0. It is not negative definite. pp ´ qq “ Recall 2 2 2 2 a1 a2 . . . an a1 a2 . . . a 2a1a2 2a1a3 . . . 2a1an p ` ` ` q “ ` ` ` n ` ` ` ` ` 2a2a3 2a2a4 . . . 2a2an 2a3a4 . . . 2a3an ...... 2an 1an ` ` ` ` ` ` ` ` ` ´ Recall 2 2 2 2 a1 a2 . . . an a1 a2 . . . a 2a1a2 2a1a3 . . . 2a1an p ` ` ` q “ ` ` ` n ` ` ` ` ` 2a2a3 2a2a4 . . . 2a2an 2a3a4 . . . 2a3an ...... 2an 1an ` ` ` ` ` ` ` ` ` ´ For example 2 x1 3x2 2x3 p ´ ` q “ Recall 2 2 2 2 a1 a2 . . . an a1 a2 . . . a 2a1a2 2a1a3 . . . 2a1an p ` ` ` q “ ` ` ` n ` ` ` ` ` 2a2a3 2a2a4 . . . 2a2an 2a3a4 . . . 2a3an ...... 2an 1an ` ` ` ` ` ` ` ` ` ´ For example 2 x1 3x2 2x3 p ´ ` q “

2 2 2 2 2 x1 3 x2 2 x3 2 3 x1x2 2 2x1x3 2 3 2x2x3 “ ` p´ q 2` 2 ` ¨ p2´ q ` ¨ ` ¨ p´ q ¨ “ x1 9x2 4x3 6x1x2 4x1x3 12x2x3 “ ` ` ´ ` ´ Recall 2 2 2 2 a1 a2 . . . an a1 a2 . . . a 2a1a2 2a1a3 . . . 2a1an p ` ` ` q “ ` ` ` n ` ` ` ` ` 2a2a3 2a2a4 . . . 2a2an 2a3a4 . . . 2a3an ...... 2an 1an ` ` ` ` ` ` ` ` ` ´ For example 2 x1 3x2 2x3 p ´ ` q “

2 2 2 2 2 x1 3 x2 2 x3 2 3 x1x2 2 2x1x3 2 3 2x2x3 “ ` p´ q 2` 2 ` ¨ p2´ q ` ¨ ` ¨ p´ q ¨ “ x1 9x2 4x3 6x1x2 4x1x3 12x2x3 “ ` ` ´ ` ´

Proposition A quadratic form Q : Rn R can expressed (possibly after a ÝÑ 2 2 2 change of coordinates) as Q x1,..., xn l1 l2 . . . l pp qq“˘ ˘ ˘ ˘ n where l1,..., ln are linear functions such that li ,..., ln do not depend on the variables x1,..., xi 1 for i 2,..., n. ´ “ Recall 2 2 2 2 a1 a2 . . . an a1 a2 . . . a 2a1a2 2a1a3 . . . 2a1an p ` ` ` q “ ` ` ` n ` ` ` ` ` 2a2a3 2a2a4 . . . 2a2an 2a3a4 . . . 2a3an ...... 2an 1an ` ` ` ` ` ` ` ` ` ´ For example 2 x1 3x2 2x3 p ´ ` q “

2 2 2 2 2 x1 3 x2 2 x3 2 3 x1x2 2 2x1x3 2 3 2x2x3 “ ` p´ q 2` 2 ` ¨ p2´ q ` ¨ ` ¨ p´ q ¨ “ x1 9x2 4x3 6x1x2 4x1x3 12x2x3 “ ` ` ´ ` ´

2 2 2 Q x1, x2, x3 x1 2x2 5x3 2x1x2 2x1x3 2x2x3 pp qq2 “ 2` ` `2 ´ ` 2 “ 2 x1 x2 x3 x2 4x2x3 4x3 x1 x2 x3 x2 2x3 p ` ´ q ` ` ` “ p ` ´ q ` p ` q Example

2 2 2 Q x1, x2, x3 x1 2x2 5x3 2x1x2 2x1x3 2x2x3 pp qq2 “ 2` ` `2 ´ ` 2 “ 2 x1 x2 x3 x2 4x2x3 4x3 x1 x2 x3 x2 2x3 p ` ´ q ` ` ` “ p ` ´ q ` p ` q 2 2 2 Q x1, x2, x3 x1 x2 x3 2x1x2 4x1x3 pp qq2 “ 2 ´ ` `2 ´ “2 2 2 x1 x2 2x3 2x2 4x2x3 3x3 x1 x2 2x3 2 x2 x3 x3 p ` ´ q ´ ` ´ “ p ` ´ q ´ p ´ q ´ What to do if there is no square? Do the following substitution: x1“y1´y2 Q x1, x2, x3 x1x2 2x1x3 x2“y1`y2 x3“y3 pp qq “ ` “ 2 2 “ y1 y2 y1 y2 2 y1 y2 y3! y1 y)2 2y1y3 2y2y3 p ´ qp2 `2 q`2 p ´ q “ ´2 ` ´2 “ y1 y3 y2 y3 2y2y3 y1 y3 y2 y3 p ` q ´ ´ ´ “ p ` q ´ p ` q Sylvester’s Criterion

Proposition Let M M n n; R be a symmetric matrix. Let Wi denote the P p ˆ q determinant of the upper-left i-by-i submatrix of M. Matrix M is positive deﬁnite if and only if Wi 0 for i 1,..., n. ą “ Sylvester’s Criterion

Consider the symmetric matrix

1 1 1 ´ M 1 2 1 “ » ﬁ 1 1 6 ´ – ﬂ Example

Consider the symmetric matrix

1 1 1 ´ M 1 2 1 “ » ﬁ 1 1 6 ´ – ﬂ and compute its leading principal minors Example

Consider the symmetric matrix

1 1 1 ´ M 1 2 1 “ » ﬁ 1 1 6 ´ – ﬂ and compute its leading principal minors W1 det 1 1 0, “ “ ą “ ‰ Example

Consider the symmetric matrix

1 1 1 ´ M 1 2 1 “ » ﬁ 1 1 6 ´ – ﬂ and compute its leading principal minors W1 det 1 1 0, “ “ ą 1 1 W2 det “ ‰ 1 0, “ 1 2 “ ą „  Example

Consider the symmetric matrix

1 1 1 ´ M 1 2 1 “ » fi 1 1 6 ´ – fl and compute its leading principal minors W1 det 1 1 0, “ “ ą 1 1 W2 det “ ‰ 1 0, “ 1 2 “ ą „  1 1 1 c1`c3 0 0 1 ´ c2`c3 ´ W3 det 1 2 1 det 2 3 1 1 0. “ » fi “ » fi “ ą 1 1 6 5 7 6 ´ – fl – fl Example

Consider the symmetric matrix

1 1 1 ´ M 1 2 1 “ » fi 1 1 6 ´ – fl and compute its leading principal minors W1 det 1 1 0, “ “ ą 1 1 W2 det “ ‰ 1 0, “ 1 2 “ ą „  1 1 1 c1`c3 0 0 1 ´ c2`c3 ´ W3 det 1 2 1 det 2 3 1 1 0. “ » fi “ » fi “ ą 1 1 6 5 7 6 ´ 2 2 2 By Sylverster’s– criterion thefl quadratic– form x1 2xfl2 6x3 2x1x2 ` ` ` 2x1x3 2x2x3 is positive definite. ´ ` Another Example

Consider the symmetric matrix

1 1 1 ´ M 1 2 1 “ » ﬁ 1 1 5 ´ – ﬂ Another Example

Consider the symmetric matrix

1 1 1 ´ M 1 2 1 “ » ﬁ 1 1 5 ´ – ﬂ and compute its leading principal minors Another Example

Consider the symmetric matrix

1 1 1 ´ M 1 2 1 “ » ﬁ 1 1 5 ´ – ﬂ and compute its leading principal minors W1 det 1 1 0, “ “ ą “ ‰ Another Example

Consider the symmetric matrix

1 1 1 ´ M 1 2 1 “ » ﬁ 1 1 5 ´ – ﬂ and compute its leading principal minors W1 det 1 1 0, “ “ ą 1 1 W2 det “ ‰ 1 0, “ 1 2 “ ą „  Another Example

Consider the symmetric matrix

1 1 1 ´ M 1 2 1 “ » fi 1 1 5 ´ – fl and compute its leading principal minors W1 det 1 1 0, “ “ ą 1 1 W2 det “ ‰ 1 0, “ 1 2 “ ą „  1 1 1 c1`c3 0 0 1 ´ c2`c3 ´ W3 det 1 2 1 det 2 3 1 0 0. “ » fi “ » fi “ č 1 1 5 4 6 5 ´ – fl – fl Another Example

Consider the symmetric matrix

1 1 1 ´ M 1 2 1 “ » fi 1 1 5 ´ – fl and compute its leading principal minors W1 det 1 1 0, “ “ ą 1 1 W2 det “ ‰ 1 0, “ 1 2 “ ą „  1 1 1 c1`c3 0 0 1 ´ c2`c3 ´ W3 det 1 2 1 det 2 3 1 0 0. “ » fi “ » fi “ č 1 1 5 4 6 5 ´ By Sylverster’s– criterion thefl quadratic– form fl 2 2 2 Q x1, x2, x3 x1 2x2 5x3 2x1x2 2x1x3 2x2x3 is not pp qq “ ` ` ` ´ ` positive definite. Another Example

Consider the symmetric matrix

A quadratic form Q is positive deﬁnite if and only if Q is ´ negative deﬁnite. Sylvester’s Criterion (continued)

A quadratic form Q is positive deﬁnite if and only if Q is ´ negative deﬁnite. Proposition

Let M M n n; R be a symmetric matrix. Let Wi denote the P p ˆ q determinant of the upper-left i-by-i submatrix of M. Matrix M is negative deﬁnite if and only if Sylvester’s Criterion (continued)

A quadratic form Q is positive deﬁnite if and only if Q is ´ negative deﬁnite. Proposition

Let M M n n; R be a symmetric matrix. Let Wi denote the P p ˆ q determinant of the upper-left i-by-i submatrix of M. Matrix M is negative deﬁnite if and only if

Wi 0 for odd i, ă

Wi 0 for even i, ą for i 1,..., n. “ Example

Consider the symmetric matrix

1 1 M ´ ´ “ 1 2 „ ´ ´  Example

Consider the symmetric matrix

1 1 M ´ ´ “ 1 2 „ ´ ´  and compute its leading principal minors Example

Consider the symmetric matrix

1 1 M ´ ´ “ 1 2 „ ´ ´  and compute its leading principal minors W1 det 1 1 0, “ ´ “´ ă “ ‰ Example

Consider the symmetric matrix

1 1 M ´ ´ “ 1 2 „ ´ ´  and compute its leading principal minors W1 det 1 1 0, “ ´ “´ ă 1 1 W2 det “ ´ ‰ ´ 1 0, “ 1 2 “ ą „ ´ ´  Example

Consider the symmetric matrix

1 1 M ´ ´ “ 1 2 „ ´ ´  and compute its leading principal minors W1 det 1 1 0, “ ´ “´ ă 1 1 W2 det “ ´ ‰ ´ 1 0, “ 1 2 “ ą „ ´ ´  2 2 2 2 The quadratic form x1 2x1x2 2x2 x1 x2 x2 is ´ ´ ´ “ ´p ` q ´ negative deﬁnite. Postive/Negative Semideﬁnite Form

Definition Quadratic form Q : Rn R (resp. symmetric matrix ÝÑ M M n n; R ) is positive semidefinite if Q x 0 (resp. P p ˆ q p qě x ⊺Mx 0) for any x Rn. ě P Postive/Negative Semidefinite Form

Definition Quadratic form Q : Rn R (resp. symmetric matrix ÝÑ M M n n; R ) is positive semidefinite if Q x 0 (resp. P p ˆ q p qě x ⊺Mx 0) for any x Rn. ě P Quadratic form Q (resp. symmetric matrix M) is negative semidefinite if Q x 0 (resp. x ⊺Mx 0) for any x Rn. p qď ď P Quadratic form Q (resp. symmetric matrix M) is indefinite if there exist x, y Rn such that Q x 0, Q y 0 (resp. P p qą p qă x ⊺Mx 0, y ⊺My 0). ą ă Postive/Negative Semidefinite Form

2 2 The quadratic form Q x1, x2 x1 x2 is indeﬁnite since pp qq “ ´ Q 1, 0 0 and Q 0, 1 0. pp qq ą pp qq ă Examples

2 2 The quadratic form Q x1, x2 x1 x2 is indefinite since pp qq “ ´ Q 1, 0 0 and Q 0, 1 0. pp qq ą pp qq ă 2 2 2 The quadratic form Q x1, x2, x3 x1 2x2 5x3 2x1x2 pp qq2 “ ` `2 ` 2x1x3 2x2x3 x1 x2 x3 x2 2x3 is positive ´ ` “ p ` ´ q ` p ` q semidefinite. It is not positive definite. Examples

The quadratic form 2 2 2 2 Q x1, x2 x1 2x1x2 2x2 x1 x2 x2 is negative pp qq“´ ´ ´ “ ´p ` q ´ deﬁnite. Examples

The quadratic form 2 2 2 2 Q x1, x2 x1 2x1x2 2x2 x1 x2 x2 is negative pp qq“´ ´ ´ “ ´p ` q ´ deﬁnite.

The quadratic form 2 2 2 2 Q x1, x2, x3 x1 2x1x2 2x2 x1 x2 x2 is not pp qq“´ ´ ´ “ ´p ` q ´ negative deﬁnite since Q 0, 0, 1 0. It is negative semideﬁnite. pp qq “ Warning

2 Consider the quadratic form Q x1, x2 x2 . pp qq“´ Warning

2 Consider the quadratic form Q x1, x2 x2 . It is negative pp qq“´ semideﬁnite. Warning

2 Consider the quadratic form Q x1, x2 x2 . It is negative pp qq“´ semideﬁnite. Its matrix is 0 0 M “ 0 1 „ ´  Warning

2 Consider the quadratic form Q x1, x2 x2 . It is negative pp qq“´ semideﬁnite. Its matrix is 0 0 M “ 0 1 „ ´  Compute the leading principal minors Warning

2 Consider the quadratic form Q x1, x2 x2 . It is negative pp qq“´ semideﬁnite. Its matrix is 0 0 M “ 0 1 „ ´  Compute the leading principal minors W1 det 0 0 0, “ “ ě “ ‰ Warning

2 Consider the quadratic form Q x1, x2 x2 . It is negative pp qq“´ semideﬁnite. Its matrix is 0 0 M “ 0 1 „ ´  Compute the leading principal minors W1 det 0 0 0, “ “ ě 0 0 W2 det “ ‰ 0 0, “ 0 1 “ ě „ ´  Warning

2 Consider the quadratic form Q x1, x2 x2 . It is negative pp qq“´ semidefinite. Its matrix is 0 0 M “ 0 1 „ ´  Compute the leading principal minors W1 det 0 0 0, “ “ ě 0 0 W2 det “ ‰ 0 0, “ 0 1 “ ě This shows„ there´ is no direct analogue of Sylvester’s criterion for positive/negative semidefinite matrices. Positive Definite Quadratic Form Proposition Let A M m n; R be a matrix. Then matrix A⊺A is positive P p ˆ q semidefinite. Moreover, the matrix A⊺A is positive definite if and only if r A n (i.e. columns of matrix A are linearly p q“ independent). Positive Definite Quadratic Form Proposition Let A M m n; R be a matrix. Then matrix A⊺A is positive P p ˆ q semidefinite. Moreover, the matrix A⊺A is positive definite if and only if r A n (i.e. columns of matrix A are linearly p q“ independent). Proof. For any x Rn P 2 x ⊺ A⊺A x Ax ⊺ Ax kAxk 0. p q “ p q p q“ ě If r A n (i.e. the linear transformation ϕ: Rn Rm given pðq p q“ Ñ by A M ϕ st is injective by the rank–nullity theorem) then “ p qst kAxk 0 Ax 0 x 0. “ ðñ “ ðñ “ Positive Definite Quadratic Form Proposition Let A M m n; R be a matrix. Then matrix A⊺A is positive P p ˆ q semidefinite. Moreover, the matrix A⊺A is positive definite if and only if r A n (i.e. columns of matrix A are linearly p q“ independent). Proof. For any x Rn P 2 x ⊺ A⊺A x Ax ⊺ Ax kAxk 0. p q “ p q p q“ ě If r A n (i.e. the linear transformation ϕ: Rn Rm given pðq p q“ Ñ by A M ϕ st is injective by the rank–nullity theorem) then “ p qst kAxk 0 Ax 0 x 0. “ ðñ “ ðñ “ if Ax 0 x 0 then ker ϕ 0 which, by the pñq “ ñ “ “ t u rank–nullity theorem, gives r A n. p q“ Example

1 2 Let A 2 1 M 3 2; R where r A 2. The matrix “ » fi P p ˆ q p q“ 1 3 – fl 1 2 1 2 1 6 7 A⊺A 2 1 , “ 2 1 3 » fi “ 7 14 „  1 3 „  – fl is positive definite and the matrix

1 2 5 4 7 1 2 1 A⊺ ⊺ A⊺ AA⊺ 2 1 4 5 5 , p q “ “ » fi 2 1 3 “ » fi 1 3 „  7 5 10 – fl – fl is positive semidefinite and it is not positive definite (this will be justified later). Eigenvalues and Positivity

Theorem (Spectral Theorem) Symmetric matrix M M n n; R is diagonalizable. P p ˆ q Eigenvalues and Positivity

Theorem (Spectral Theorem) Symmetric matrix M M n n; R is diagonalizable. P p ˆ q In particular, the characteristic polynomial wM λ det M λI p q“ p ´ q has n real roots (=eigenvalues) counted with multiplicities . Theorem Let Q : Rn R be a quadratic form and let M be its matrix. Let ÝÑ λ1,...,λn R be the roots of wM λ . Then P p q i) form Q is positive definite λ1,...,λn 0, ðñ ą ii) form Q is positive semidefinite λ1,...,λn 0, ðñ ě iii) form Q is negative definite λ1,...,λn 0, ðñ ă iv) form Q is negative semidefinite λ1,...,λn 0, ðñ ď v) form Q is indefinite λi 0, λj 0 for some ðñ ă ą 1 i, j n. ď ď Eigenvalues and Positivity (continued) Proof. n n Let v1,..., vn R be a basis of R consisting of eigenvectors of P M, that is

Mvi λi vi for i 1,..., n, “ “ ˚ R . R where λi is an eigenvalue of M and vi . M n 1; is P “ ˚ P p ˆ q taken to be a n-by-1 matrix. „  Eigenvalues and Positivity (continued) Proof. n n Let v1,..., vn R be a basis of R consisting of eigenvectors of P M, that is

Mvi λi vi for i 1,..., n, “ “ ˚ . where λi R is an eigenvalue of M and vi . M n 1; R is P “ . P p ˆ q „ ˚  taken to be a n-by-1 matrix. Let vi , vj be vectors such that λi λj . Then ‰ ⊺ ⊺ ⊺ v Mvj v Mvj v λj vj λj vi vj , i “ i p q“ i p q“ p ¨ q Eigenvalues and Positivity (continued) Proof. n n Let v1,..., vn R be a basis of R consisting of eigenvectors of P M, that is

Mvi λi vi for i 1,..., n, “ “ ˚ . where λi R is an eigenvalue of M and vi . M n 1; R is P “ . P p ˆ q „ ˚  taken to be a n-by-1 matrix. Let vi , vj be vectors such that λi λj . Then ‰ ⊺ ⊺ ⊺ vi Mvj vi Mvj vi λj vj λj vi vj , ⊺ ⊺ “⊺ p q“ ⊺ p q“ ⊺ p ¨ q v Mvj v M vj Mvi vj λi vi vj λi vi vj . i “ p i q “ p q “ p q “ p ¨ q

This is possible only if vi vj 0, i.e. vectors vi , vj are ¨ “ perpendicular. Eigenvalues and Positivity (continued) Proof. n n Let v1,..., vn R be a basis of R consisting of eigenvectors of P M, that is

This is possible only if vi vj 0, i.e. vectors vi , vj are ¨ “ perpendicular. Using Gram-Schmidt process for eigenspaces Vpλi q one can assume the basis v1,..., vn is orthonormal. Eigenvalues and Positivity (continued)

Proof. That is ⊺ 0 for i j vi vj v vj ‰ . ¨ “ i “ 1 for i j " “ n For any v R there exist unique α1,...,αn R such that P P

v α1v1 . . . αnvn. “ ` ` Eigenvalues and Positivity (continued)

Proof. That is ⊺ 0 for i j vi vj v vj ‰ . ¨ “ i “ 1 for i j " “ n For any v R there exist unique α1,...,αn R such that P P

v α1v1 . . . αnvn. “ ` ` Now ⊺ ⊺ Q v v Mv v M α1v1 . . . αnvn p q“ “ p ` ` q“ ⊺ 2 2 α1v1 . . . αnvn λ1α1v1 . . . λnαnvn λ1α1 . . . λn α . “ p ` ` q p ` ` q“ ` ` n Eigenvalues and Positivity (continued)

Proof. That is ⊺ 0 for i j vi vj v vj ‰ . ¨ “ i “ 1 for i j " “ n For any v R there exist unique α1,...,αn R such that P P

v α1v1 . . . αnvn. “ ` ` Now ⊺ ⊺ Q v v Mv v M α1v1 . . . αnvn p q“ “ p ` ` q“ ⊺ 2 2 α1v1 . . . αnvn λ1α1v1 . . . λnαnvn λ1α1 . . . λn α . “ p ` ` q p ` ` q“ ` ` n In particular ⊺ Q vi v Mvi λi . p q“ i “ Example

Let 1 0 M . “ 0 1 „ ´  The eigenvalues are λ1 1 0, λ2 1 0 therefore the “ ą 2 “´2 ă quadratic form Q x1, x2 x1 x2 is indeﬁnite. pp qq “ ´ Example

Let 1 0 0 M 0 2 2 . “ » ﬁ 0 2 2 – ﬂ Example

Let 1 0 0 M 0 2 2 . “ » fi 0 2 2 The characteristic polynomial– fl 2 wM λ 1 λ 2 λ 4 λ 1 λ λ 4 has p q “ p ´ qpp ´ q ´ q“ p ´ qp ´ q non-negative roots λ1 0, λ2 1, λ3 4, λ1, λ2, λ3 0. “ “ “ ě Therefore the quadratic form 2 2 2 2 2 Q x1, x2, x3 x1 2x2 2x3 4x2x3 x1 2 x2 x3 is pp qq “ ` ` ` “ ` p ` q positive semidefinite. Example (continued) 1 2 Let A 2 1 M 3 2; R where r A 2. The matrix “ »1 3fi P p ˆ q p q“ – fl 1 2 54 7 1 2 1 A⊺A 2 1 45 5 , “ » fi 2 1 3 “ » fi 1 3 „  7 5 10 – fl – fl is positive semidefinite and it is not positive definite.

5 λ 4 7 5 λ 4 1 c1`p5´λqc`3 c3 2c2 c2 4c3 det ´4 5 λ 5 ´ det ´4 5 λ 2λ´ 5 ` » 7´ 5 10 λfi “ » 7´ 5 ´λ fi “ ´ ´ – fl – fl 0 0 1 det 2λ2 15λ 21 7λ 15 2λ´ 5 “ »´ λ2 `5λ ´7 5 ´4λ ´λ fi “ ´ ` ´ ´ – 2 fl 2λ 15λ 21 7λ 15 det ´ 2 ` ´ ´ “´ λ 5λ 7 5 4λ “ „ ´ ` ´  2 λ λ 20λ 35 . “´ p ´ ` q Example (continued)

Therefore one eigenvalue of A⊺A is equal to 0, and, by the Vieta’s formulas, λ1 λ2 20 0, ` “ ą λ1λ2 35 0, “ ą the other two eigenvalues are non–negative. In fact

54 7 5 5 1 3 45 5 1 0. ´ ´ »7 5 10fi »´3fi “ “ ‰ ´ – fl – fl Spectral Theorem

Proposition Let ϕ: Rn Rn be an endomorphism. If there exists an Ñ n A orthonormal basis A of R such that M ϕ A is symmetric then for n p q B any orthonormal basis B of R matrix M ϕ B is symmetric. p q Proof. A B Let M M ϕ A, N M ϕ B. Matrix “ p q “ p q A A st Q M id B M id M id B , “ p q “ p qst p q is orthonormal, i.e. Q´1 Q⊺. Because “ N Q⊺MQ, “ we have N⊺ Q⊺M⊺ Q⊺ ⊺ N. “ p q “ Spectral Theorem (continued) Proposition Let ϕ: Rn Rn be an endomorphism such that matrix M ϕ st is Ñ p qst symmetric. Then there exists µ R and v Rn, v 0 such that P P ‰ ϕ v µv, i.e µ is an eigenvalue of ϕ and v is an eigenvector for p q“ µ. Spectral Theorem (continued) Proposition Let ϕ: Rn Rn be an endomorphism such that matrix M ϕ st is Ñ p qst symmetric. Then there exists µ R and v Rn, v 0 such that P P ‰ ϕ v µv, i.e µ is an eigenvalue of ϕ and v is an eigenvector for p q“ µ. Proof. st Let M M ϕ and let wM λ det M λIn be the “ p qst p q“ p ´ q characteristic polynomial of ϕ. By the fundamental theorem of algebra there exists a complex root µ C of wM and a complex n P eigenvector v C , i.e. wM µ 0 and Mv µv. Matrix M is P p q“ “ real therefore Mv µv. Moreover “ v ⊺Mv Mv ⊺ v µv ⊺v µkvk, “ p q “ “ v ⊺Mv v ⊺ Mv v ⊺ µv µkvk. “ p q“ p q“ This implies µ R and since V is given a system of linear P pµq equations with real coeﬃcients one can choose v Rn. P Spectral Theorem (continued)

Proposition Let ϕ: Rn Rn be an endomorphism such that matrix M ϕ st is Ñ p qst symmetric. Then for any subspace W Rn such that ϕ W W , Ă p qĂ ϕ W K W K. p qĂ Spectral Theorem (continued)

Proposition Let ϕ: Rn Rn be an endomorphism such that matrix M ϕ st is Ñ p qst symmetric. Then for any subspace W Rn such that ϕ W W , Ă p qĂ ϕ W K W K. p qĂ

Let M M ϕ st and let w W K be any vector. We need to “ p qst P check that v ⊺ Mw 0 for a any v W . p q“ P Spectral Theorem (continued)

Proposition Let ϕ: Rn Rn be an endomorphism such that matrix M ϕ st is Ñ p qst symmetric. Then for any subspace W Rn such that ϕ W W , Ă p qĂ ϕ W K W K. p qĂ

Let M M ϕ st and let w W K be any vector. We need to “ p qst P check that v ⊺ Mw 0 for a any v W . In fact p q“ P v ⊺Mw Mv ⊺w 0. “ p q “ Spectral Theorem

Theorem Symmetric matrix M M n n; R is diagonalizable. P p ˆ q Spectral Theorem

Theorem Symmetric matrix M M n n; R is diagonalizable. P p ˆ q Proof. Let ϕ be an endomorphism given by M M ϕ st . Assume “ p qst W Rn is a subspace spanned by pairwise perpendicular Ă eigenvectors of ϕ. Let V W K. Matrix of ϕ relative to some “ orthonormal basis of V is symmetric and ϕ V V . Therefore p qĂ there exists µ R and a non–zero vector v V such that P P ϕ v µv. p q“ Spectral Theorem

Corollary For any symmetric matrix M M n n; R there exists matrix P 1 p ˆ q Q M n n; R such that Q´ Q⊺ and the matrix P p ˆ q “ D Q⊺MQ, “ is diagonal and D M n n; R . P p ˆ q Spectral Theorem (continued)

Corollary For any symmetric matrix M M n n; R there exists matrix P 1 p ˆ q Q M n n; R such that Q´ Q⊺ and the matrix P p ˆ q “ D Q⊺MQ, “ is diagonal and D M n n; R . P p ˆ q Proof. n Let A v1,..., vn be an orthonormal basis of R consisting of “ p q st ´1 eigenvectors of M. If Q M id A then the matrix Q MQ is ⊺ “ p ´q1 ⊺ diagonal and Q Q In, i.e. Q Q . “ “ Bilinear Form