CS 515: Homework 1 Yingwei Wang ∗ Department of Mathematics, Purdue University, West Lafayette, IN, USA 1 Matrix product 1.1 (a) 1 3 2 1 2 ∗ 3, 2, 1 = 6 4 2 . 3 9 4 3 1.2 (b) 1 3, 2, 1 ∗ 2 = 10. 3 1.3 (c) 0 0 1 1 2 3 7 8 9 0 1 0 ∗ 4 5 6 = 4 5 6 . 1 0 0 7 8 9 1 2 3 2 Symmetric matrices Give an example to show that the product of two symmetric matrices need not to be symmetric. ∗E-mail address: [email protected]; Tel: 765 237 7149 1 Yingwei Wang Numerical Linear Algebra 1 0 0 1 0 1 1 0 1 0 2 0 ∗ 0 1 0 = 0 2 0 . 0 0 3 1 0 1 3 0 3 3 Elementary reflector Let H = I − 2xxT , where xT x = 1. 3.1 H is symmetric HT = IT − 2(xxT )T = I − 2xxT = H. So H is symmetric. 3.2 H is involuntary H2 =(I − 2xxT )(I − 2xxT ) = I2 − 4xxT +4xxT xxT = I − 4xxT +4x(xT x)xT = I − 4xxT +4xxT = I. So H is involuntary. 3.3 H is orthogonal Since H is symmetric and involuntary, we have HT H = H2 = I, HHT = H2 = I. So H is orthogonal. Remark 3.1. H is often called “elementary reflector ”or a “Householder transformation”. 2 Yingwei Wang Numerical Linear Algebra 4 Property about zero vectors Proposition 4.1. Let A ∈ Rm×n. Show that xT AT Ax =0 if and only if Ax =0. Proof. If Ax = 0, then it is obvious that xT AT Ax = 0. T Conversely, let y = Ax =(y1, ··· ,yj, ··· ,ym) be a column vector, then xT AT Ax =0, ⇒ yT y =0, m 2 ⇒ yj =0, j=1 X ⇒ yj =0, ∀j, ⇒ y =0, ⇒ Ax =0. 5 Block matrices Assuming the partition conform, compute 5.1 (a) (X,Y )T ∗ A ∗ (X,Y ) XA = ∗ (X,Y ) Y A XAX XAY = . Y AX Y AY 5.2 (b) β bT α (α, xT ) ∗ ∗ b B x α = (αβ + xT b, αbT + xT B) ∗ x = α2β + αxT b + αbT x + xT Bx. 3 Yingwei Wang Numerical Linear Algebra 5.3 (c) α aT β bT ∗ a A b B αβ + aT b αbT + aT B = . βa + Ab abT + AB 6 Matrix equation n T T T Proposition 6.1. Let x ∈ R be partitioned in the form x = (ξ1,y ) where y = n−1 (ξ2,ξ3, ··· ,ξn). Show that if ξi =06 , there is a unique vector b ∈ R such that 1 0 ξ ξ ∗ 1 = 1 (6.1) b I − y 0 n 1 Proof. Choose b = −y/ξi, then 1 0 ξ ∗ 1 b In− y 1 ξ = 1 ξb + y ξ = 1 0 If there are two vectors b1, b2 satisfying (6.1), then ξ1b1 + y =0, ξ1b2 + y =0, ⇒ ξ1(b1 − b2)=0, ⇒ b1 = b2. 7 Nonsingular matrix Proposition 7.1. Let A = diag (λ1,λ2, ··· ,λn). Show that A is nonsingular if and only if λi =06 for 1 ≤ i ≤ n. 4 Yingwei Wang Numerical Linear Algebra n Proof. Since det(A) = Πi=1λi, we know that if A is nonsingular then det(A) =6 0, which implies that λi =06 for 1 ≤ i ≤ n. Conversely, if λi =06 for 1 ≤ i ≤ n, then det(A) =6 0, which means A is nonsingular. −1 −1 −1 −1 Besides, then inverse of A is A = diag (λ1 ,λ2 , ··· ,λn ). 8 Inverse Proposition 8.1. Let u, v ∈ Rn and σ =6 0. Suppose that vT u − σ−1 =6 0. Show that (I − σuvT ) is nonsingular and has an inverse given by (I − τuvT ) where σ−1 + τ −1 = vT u. T T Proof. Let An = In − σuv and Dn = det(An). Let u = (u1,u2, ··· ,un) and v = T (v1, v2, ··· , vn) . Then by the knowledge of linear algebra, it is easy to know that 1 − σu1v1 −σu1v2 ··· −σu1vn −σu v 1 − σu v ··· −σu v det(A ) = det 2 1 2 2 2 n (8.1) n ··· −σunv −σunv ··· 1 − σunvn 1 2 1 − σu1v1 −σu1v2 ··· 0 1 − σu1v1 −σu1v2 ··· σu1vn −σu v 1 − σu v ··· 0 −σu v 1 − σu v ··· σu v = det 2 1 2 2 − det 2 1 2 2 2 n(8.2) ··· ··· −σunv −σunv ··· 1 −σunv −σunv ··· σunvn 1 2 1 2 1 0 ··· 0 0 1 ··· 0 = D − − det (8.3) n 1 ··· −σunv −σunv ··· σunvn 1 2 = Dn−1 − σunvn. which means Dn = Dn−1 − σvnun. (8.4) Besides, we know that D1 =1 − σu1v1. Now we can get n T Dn =1 − σ ujvj =1 − σv u. j=1 X 5 Yingwei Wang Numerical Linear Algebra Since vT u − σ−1 =6 0, we know that det(I − σuvT ) =6 0, which implies that (I − σuvT ) is nonsingular. Now we want to find the inverse of (I − σuvT ). Choose τ such that 1 1 + = vT u, σ τ σ + τ ⇒ = vT u, στ ⇒ σ + τ = στvT u. Consider the product (I − τuvT )(I − σuvT ): (I − τuvT )(I − σuvT ) = I − (σ + τ)uvT + στ(uvT )2, = I − στvT uuvT + στ(uvT )2. T T T 2 Let X = v uuv =(xpq)n×n, Y =(uv ) =(ypq)n×n, then n xpq = ukvk upvq, k=1 ! nX ypq = (upvk)(ukvq) k=1 Xn = ukvk upvq. k ! X=1 It implies that X = Y and further (I − τuvT )(I − σuvT )= I. Similarly, we can prove that (I − σuvT )(I − τuvT )= I. Then we are done. Remark 8.1. From (8.2) to (8.3), we assume that un and vn are not 0. But if un = 0 or v0 =0, then (8.4) is still true. So, anyway, (8.4) is true. Besides, from (8.2) to (8.3), if vj =0, then it is fine; if vj =06 , then we can use the last T row to make jth row become ej = (0, ··· , 0, 1, 0, ··· , 0) where 1 is in the jth place. Finally, from (8.1) to (8.2), it is just a common trick for computing the det(An). You can find that in any textbook on linear algebra. 9 Matrices Proposition 9.1. Let B ∈ Rn×n,S ∈ Rk×k and U, V ∈ Rn×k. Assuming that B,S and [V T B−1U − S−1] are nonsingular, show that (B − USV T )−1 = B−1 − B−1UTV T B−1, 6 Yingwei Wang Numerical Linear Algebra where S−1 + T −1 = V T B−1U. Proof. On one hand, (B − USV T )(B−1 − B−1UTV T B−1), = I − UTV T B−1 − USV T B−1 + USV T B−1UTV T B−1, = I − U(T + S)V T B−1 + US(S−1 + T −1)TV T B−1, = I. On the other hand, (B−1 − B−1UTV T B−1)(B − USV T ), = I − B−1USV T − B−1UTV T + B−1UTV T B−1USV T , − − − − = I − B 1U(S + T )V T + B 1UT (S 1 + T 1)SV T , = I. Then we are done. 10 Hermitian matrix Question: A ∈ Rn×n is hermitian if AH = A. If A = B + iC, then it is easy to show that BT = B and CT = −C. Suppose that we represent A in an array A.herm with the property that A.herm(i, j) houses bij if i ≥ j and cij if j > i. Using this data structure write a matrix-vector multiply function that computes ℜ(z) and ℑ(z) from ℜ(x) and ℑ(x) so that z = Ax. Solution: Let x =(xj) be a column vector, and each element xj = pj + iqj, where pj = ℜ(xj) and qj = ℑ(xj). Let A = B + iC be hermitian and A.herm =(ajk), B =(bjk), C =(cjk). We know that ajk = bjk if j ≥ k, & ajk = cjk if j ≤ k. Besides, bjk = bkj, & cjk = −ckj. 7 Yingwei Wang Numerical Linear Algebra Let z =(zj) be a column vector. Then from z = Ax, we know that n zj = (bjk + icjk)(pk + iqk) k=1 Xn = (bjkpk − cjkqk)+ i(cjkpk + bjkqk). k=1 X j n j n ⇒ ℜ(zj)= ajkpk − akjpk − akjqk + ajkqk, (10.1) k k j k k j X=1 X= +1 X=1 X= +1 j n j n ℑ(zj)= − akjpk + ajkpk + ajkqk − akjqk. (10.2) k k j k k j X=1 X= +1 X=1 X= +1 From the formula (10.1) and (10.2), we can write some code using any language. 11 The algebra of triangular matrices Proposition 11.1. Here are some properties about the products and inverses of triangular and unit triangle matrices. 1. The inverse of an upper (lower) triangular matrix is upper (lower) triangular. 2. The product of two upper (lower) triangular matrices is upper (lower) triangular. 3. The inverse of a unit upper (lower) triangular matrix is upper (lower) triangular. 4. The product of two unit upper (lower) triangular matrices is upper (lower) triangular. Proof. (1). Let Tn×n be the upper triangular matrix, then T (i, j) = 0 if i > j. Induction by n. If n = 1, then it is obviously true. −1 Suppose T(n−1)×(n−1) is an upper triangular matrix, then solving the equation TS = I, we can get S(n, n)=1/T (n, n), S(n, 1 : n − 1)=0, S(1 : n − 1, n)= −T (1 : n − 1, 1 : n − 1)−1T (1 : n − 1, n)S(n, n), S(1 : n − 1, 1 : n − 1) = T (1 : n − 1, 1 : n − 1)−1, which is triangular by assumption.