Norms of Vectors and Matrices and Eigenvalues and Eigenvectors - (7.1)(7.2) Vector norms and matrix norms are used to measure the difference between two vectors or two matrices, respectively, as the absolute value function is used to measure the distance between two scalars. 1. Vector Norms: a. Definition: A vector norm on Rn is a real-valued function, ||!||, with the following properties: Let x and y be vectors and ! be real number. i. ||x|| ! 0, ||x|| ! 0, if and only if x is a zero vector; ii. ||!x|| ! |!|||x||; iii. ||x " y|| " ||x|| " ||y||. T b. Commonly used the vector norms: let x ! x1 " xn 2 2 i. 2-norm (or Euclidean norm): #x#2 ! x1 "..."xn ii. 1-norm: #x#1 ! |x1 | "..."|xn | iii. infinity-norm: #x## ! max1"i"n |xi | T c. Cauchy-Schwarz Inequality: x y " ||x||2#y#2. $1"cos#"$"1 T x y ! #x#2||y||2 cos#"$ " #x#2||y||2 T Example Let x ! 1 $23$4 . Compute #x#2, #x#1 and #x##. 2 2 2 2 #x#2 ! 1 " 2 " 3 " 4 ! 30 , ||x||1 ! 1 " 2 " 3 " 4 ! 10 , ||x||# ! 4 T T Example Let D1 ! x ! x1 x2 | #x#1 " 1 , D2 ! x ! x1 x2 | #x#2 " 1 and T D# ! x ! x1 x2 | #x## " 1 . Show that D1 % D2 % D# We want to show that #x## " #x#2 " #x#1. 1 Vectors in the green diamond satisfy the inequality 0.8 #x#1 " 1. 0.6 Vectors in the red circle satisfy the inequality 0.4 0.2 #x#2 " 1. Vectors in the blue square satisfy the inequality -1 -0.8 -0.6 -0.4 -0.20 0.2 0.4 0.6 0.8 1 -0.2 #x## " 1. -0.4 Hence, #x# " #x# " #x# . # 2 1 -0.6 -0.8 -1 blue - #!##, red - #!#2, green - #!#1 Example Show that (i) #x## " #x#2 " n #x##; (ii) #x## " #x#1 " n#x##; and (iii) #x#2 " #x#1 " n #x#2. T Let x ! x1 " xn (i) Let |xk | ! max1"i"n|xi |. Then #x## ! |xk | and 1 2 2 2 2 2 |xk | ! xk " #x#2 ! x1 " " " xn " xk " " " xk ! nxk ! n |xk |. (iii) Since x 2 x x 2 x 2 x 2 2 x x x 2 x 2 x 2, x x . # #1 ! #| 1 | " " " | n |$ ! | 1 | " " " | n | " &i$j | i || j | ! | 1 | " " " | n | ! || ||2 # #2 " # #1 T T #k$ #k$ #k$ #k$ Example Let x ! x1 " xn and x ! x1 " xn . The sequence %x & converges to x if #k$ #k$ and only if the sequence %#x $ x#& converges to 0 (limk%##x $ x# ! 0) for any vector norm. 2. Matrix Norms: a. Definition: A matrix norm on the set of all n & n matrices is a real-valued function, ||!||, with the following properties: Let A and B be matrices and ! be real number. i. ||A|| ! 0, ||A|| ! 0, if and only if A is a zeros matrix; ii. ||!A|| ! |!|||A||; iii. ||A " B|| " ||A|| " ||B||; iv. ||AB|| " ||A|| ||B||. n b. Commonly used the matrix norms: Let A ! 'aij (i,j!1. i. 1-norm: A max n a (maximum column-sum) # #1 ! 1"j"n &i!1| ij | ii. infinity-norm: A max n a (maximum row-sum) # ## ! 1"i"n &j!1| ij | iii. Frobenius-norm: A n n a2 # #F ! &i!1 &j!1 ij 1 $23 Example Let A ! $45$6 . Compute #A#1, #A## and #A#F. 7 $89 row sums: 6, 15, 24, column sums: 12, 15, 18 9#10$#19$ #A# ! 18, #A# ! 24, #A# ! 1 " 22 " "92 ! ! 285 . 1 # F 6 c. Distance between two n & n matrices A and B with respect to a matrix norm is: ||A $ B||. d. Natural or induced matrix norm: Theorem 7.9 Let ||!|| be a vector norm. Then ||A|| ! max ||Ax|| ||x||!1 is a matrix norm and is called a natural or induced matrix norm. Then we have #A## ! max #Ax##; #A#2 ! max #Ax#2 #x##!1 ||x||2!1 Corollary 7.10: For any z ' 0, ||Az|| " ||A|| ||z|| for any natural norm. e. The following inequalities hold for A: (i) #A#2 " #A#F " n #A#2 (ii) #A#2 " #A#1#A## (iii) 1 #A# " #A# " n #A# n # 2 # (iv) 1 #A# " #A# " n #A# n 1 2 1 3. Eigenvalues and Eigenvectors: 2 a. Definition: Let Abe an n & n real matrix. A nonzero vector x in Rn is said to be an eigenvector corresponding to the eigenvalue # if (*) Ax ! #x. The pair ##,x$ is called an eigenpair of A. Note that if ##,x$ is an eigenpair of A then #!#,x$ is again an eigenpair of A. b. Characteristic polynomial of A : Note also that the equation #*$ is equivalent to that the equation: Ax $ #Inx ! #A $ #In $x has infinitely many solutions or det#A $ #In $ ! 0. Let P##$ ! det#A $ #In $. P##$ is called characteristic polynomial of A. Note that P##$ is an nth degree polynomial in # and zeros of P##$ are eigenvalues of A. c. Spectral radius of A : Let #i be an eigenvalue of A. The spectral radius of A,$#A$ ! max1"i"n|#i |. 4 $10 0 $14$10 Example Let A ! . Compute $#A$. 0 $14$1 00$14 9 1 1 9 7 1 1 7 Find all eigenvalues: 2 $ 2 5 , 2 5 " 2 , 2 $ 2 5 , 2 5 " 2 . 9 1 1 9 7 1 1 7 1 9 Then $#A$ ! max 2 $ 2 5 , 2 5 " 2 , 2 $ 2 5, 2 5 " 2 ! 2 5 " 2 ! 5. 618. 102 Example Let A ! 01$1 . Compute $#A$. $11 1 Find all eigenvalues: 1,1 $ i 3 ,i 3 " 1 $#A$ ! max 1, i 3 " 1 , 1 $ i 3 ! max 1, 2, 2 ! 2 d. Properties: T i. #A#2 ! $#A A$ (When A is symmetric, #A#2 ! $#A$. ii. $#A$ " #A# for any natural norm #!#. 102 Example Let A ! 01$1 . Compute #A#2. $11 1 T 102 102 2 $11 ATA ! 01$1 01$1 ! $120 $11 1 $11 1 106 T Eigenvalues of A A: 0.8972, 2.8536, 6.2491 , #A#2 ! 6.2491 ! 2. 4998 e. Convergent matrices: 3 k A matrix A is convergent if limk%# A ! 0n&n . The following statements are equivalent. i. A is a convergent matrix. k ii. limk%##A # ! 0 form all natural norm. iii. $#A$ ' 1. k iv. limk%# A x ! 0 for every x. 1 1 1 1 $ 4 0 2 1 $ 4 Example Let A1 ! , A2 ! , eigenvalues: and A3 ! . Find $ Ai 1 1 1 1 # $ 4 2 2 0 4 1 and determine if Ai are convergent. 3 3 1 1 1 Eigenvalues of A1 are 4 , 4 , eigenvalues of A2 are 2 ,$ 2 and eigenvalues of A3 are 1 ( 4 i. Hence, 3 1 1 1 2 $ Ai , $ A2 ! , $ A3 ! 1 " i ! 1 " ! 1. 030776407, # $ 4 # $ 2 # $ 4 4 both A1 and A2 are convergent and A3 divergent. Among A1 and A2, A2 converges faster than A1 does. 4.