Appendix A Facts from Linear Algebra Abstract We introduce the notation of vector and matrices (cf. Section A.1), and recall the solvability of linear systems (cf. Section A.2). Section A.3 introduces the spectrum σ(A), matrix polynomials P (A) and their spectra, the spectral radius ρ(A), and its properties. Block structures are introduced in Section A.4. Subjects of Section A.5 are orthogonal and orthonormal vectors, orthogonalisation, the QR method, and orthogonal projections. Section A.6 is devoted to the Schur normal form (§A.6.1) and the Jordan normal form (§A.6.2). Diagonalisability is discussed in §A.6.3. Finally, in §A.6.4, the singular value decomposition is explained. A.1 Notation for Vectors and Matrices We recall that the field K denotes either R or C. Given a finite index set I, the linear I space of all vectors x =(xi)i∈I with xi ∈ K is denoted by K . The corresponding square matrices form the space KI×I . KI×J with another index set J describes rectangular matrices mapping KJ into KI . The linear subspace of a vector space V spanned by the vectors {xα ∈V : α ∈ I} is denoted and defined by α α span{x : α ∈ I} := aαx : aα ∈ K . α∈I I×I T Let A =(aαβ)α,β∈I ∈ K . Then A =(aβα)α,β∈I denotes the transposed H matrix, while A =(aβα)α,β∈I is the adjoint (or Hermitian transposed) matrix. T H Note that A = A holds if K = R . Since (x1,x2,...) indicates a row vector, T (x1,x2,...) is used for a column vector. Exercise A.1. Prove the following rules for T and H (where λ ∈ K): (A + B)T = AT + BT, (AB)T = BTAT, (λA)T = λAT, (A + B)H = AH + BH, (AB)H = BHAH, (λA)H = λA¯ H, (A−1)T =(AT)−1, (A−1)H =(AH)−1 = A−H. © Springer International Publishing Switzerland 2016 401 W. Hackbusch, Iterative Solution of Large Sparse Systems of Equations, Applied Mathematical Sciences 95, DOI 10.1007/978-3-319-28483-5 402 Appendix A The inverse of a transposed or adjoint matrix is shortly denoted by A−T := (AT)−1,A−H := (AH)−1. Definition A.2. A matrix A ∈ KI×I is called symmetric if A = AT, Hermitian if A = AH, regular if A−1 exists, unitary if AHA=I (i.e., A regular and A−1 = AH), normal if AAH = AHA . Remark A.3. (a) Hermitian or unitary matrices are also normal. (b) All matrix properties of Definition A.2 carry over from A to the adjoint AH. (c) Products of regular (unitary) matrices are again regular (unitary). A diagonal matrix D is completely described by its diagonal entries. We write ! dα for α = β, D =diag{dα : a ∈ I} for D with Dαβ = (A.1) 0 for α = β. If I is ordered, we may also write D =diag{d1,d2,...,dn}. For an arbitrary matrix A ∈ KI×I , D =diag{A} denotes the diagonal part diag{aαα : α ∈ I} of A. In the case of an ordered index set, a matrix T is called tridiagonal if Tij =0 for all |i − j| > 1; i.e., if T has the band width 1 (cf. Definition 1.6). The entries αi = Ti,i−1 define the lower side diagonal, βi = Tii the (main) diagonal, and γi = Ti,i+1 the upper side diagonal, while all other entries of T vanish. Such a matrix is abbreviated as T =tridiag{(αi,βi,γi): i ∈ I} (A.2) (here the values α1 and γ#I are meaningless). By tridiag{A} we denote the tridiagonal part of an arbitrary matrix A. Assuming again an ordered index set, a matrix T is called a lower triangular matrix if Tij =0for all i<j. Similarly, T is called upper triangular if Tij =0 for all i>j. T is a strictly lower or upper triangular matrix if, in addition, Tii =0 for all i ∈ I. A.2 Systems of Linear Equations Let A ∈ KI×I and b ∈ KI . The system of equations to be solved is Ax = b, i.e., aαβ xβ = bα for all α ∈ I. β∈I A.2 Systems of Linear Equations 403 Since the right-hand side b may be perturbed (by rounding errors, etc.), the relevant question is: when is Ax = b solvable for all b ∈ KI ? The following theorem recalls that this property is equivalent to the regularity of A. Theorem A.4. For A ∈ KI×I , the following properties are equivalent: (a) A is regular, (b) rank(A)=#I, (c) det(A)=0, (d) Ax =0 has only the trivial solution x =0, (e) Ax = b is solvable for all b ∈ KI , (f) Ax = b has at most one solution, (g) Ax = b is uniquely solvable for all b ∈ KI . A.3 Eigenvalues and Eigenvectors The spectrum of a matrix A ∈ KI×I is defined by σ(A):={λ ∈ C :det(A − λI)=0}. Each λ ∈ σ(A) is called an eigenvalue of A. An eigenvalue has the algebraic multiplicity k if it is a k-fold root of the characteristic polynomial det(A − λI). Since det(A − λI) is a polynomial in λ of degree n =#I, there exist exactly n eigenvalues when they are counted according to their algebraic multiplicity. The geometric multiplicity of λ is the dimension of ker(A − λI). The properties of the determinant prove the next properties. Remark A.5. σ(AT)=σ(A) and σ(AH)=σ(A¯)=σ(A):={λ¯ : λ ∈ σ(A)}. A vector e ∈ CI is called an eigenvector of the matrix A,ife =0 and Ae = λe. (A.3) By Theorem A.4c,d, we conclude from (A.3) that λ must be an eigenvalue. Vice versa, the same theorem proves the following lemma. Lemma A.6. For each λ ∈ σ(A), there exists an eigenvector e satisfying the eigen- value problem (A.3). Hence the geometric multiplicity is at least one. Exercise A.7. Let A =(aij)i,j∈I be an upper or lower triangular matrix or a diagonal matrix. Prove that σ(A)={aii : i ∈ I}. Definition A.8. Two matrices A, B ∈ KI×I are called similar if there is a regular matrix T such that A = T −1BT. (A.4) If T is unitary, the matrices A and B are called unitarily similar. 404 Appendix A Theorem A.9. (a) The eigenvalues of similar matrices A and B coincide: σ(A)=σ(B). The algebraic multiplicities of the eigenvalues are also equal as well as the geometric multiplicities. (b) If T is the similarity transformation in (A.4) and e is an eigenvector of A , then Te is an eigenvector of B. Proof. The algebraic multiplicities are equal since det(A − λI)=det(T −1(B − λI)T )=det(T −1)det(B − λI)det(T ) 1 = det(B − λI)det(T )=det(B − λI). det(T ) ker(A − λI)=ker(T −1(B − λI)T )=ker(B − λI)T proves identical dimensions of ker(A − λI) and ker(B − λI) and therefore of the geometric multiplicities. Part (b) uses B(Te)=TT−1BTe = TAe = T (λe)=λ (Te). Theorem A.10. The products AB and BA have the same spectra with a possible exception of a zero eigenvalue: σ(AB)\{0} = σ(BA)\{0}. This statement is also true for rectangular matrices A ∈ KI×J and B ∈ KJ×I . Proof. Let the eigenvector e =0 belong to the eigenvalue 0 = λ ∈ σ(AB): ABe = λe. Since λe =0 , the vector v := Be does not vanish. Multiplying by B yields BABe = λBe, i.e., BAv = λv with v =0 . λ ∈ σ(BA)\{0} proves σ(AB)\{0}⊂σ(BA)\{0}. The reverse inclusion is analogous. ν Given a polynomial P (ξ)= ν aν ξ in ξ ∈ C, we can extend the domain of definition of P by ν I×I P (A):= aν A for arbitrary A ∈ K ν to the set of square matrices. Here, A0 is defined as the identity I. The proof of the following lemma is postponed to the end of §A.6.1. Lemma A.11. (a) The spectra of A and P (A) satisfy σ(P (A)) = P (σ(A)) := {P (λ):λ ∈ σ(A)}. (b) The algebraic multiplicity of the eigenvalues P (λ) of P (A) is the sum of the multiplicities of all eigenvalues λ1,λ2,...,λk of A with P (λj)=P (λ) (1≤j ≤k). (c) Each eigenvector of A associated with the eigenvalue λ is also an eigenvector of P (A) with the eigenvalue P (λ). A.3 Eigenvalues and Eigenvectors 405 Exercise A.12. Prove the following: (a) If σ(A) contains no zeros of the polynomial P (ξ), then the matrix P (A) is regular. (b) The properties ‘diagonal’, ‘upper triangular matrix’, ‘lower triangular ma- trix’ carry over from A to P (A). This statement is also true for the properties ‘symmetric’ and ‘Hermitian’, provided that P has real coefficients. (c) Let A be regular. All properties mentioned in (b) carry over from A to A−1 . Lemma A.13. Let A ∈ KI×I be a strictly (upper or lower) triangular matrix. Then Am =0 holds for all m>#I. Proof. One proves by induction that Am (m ∈ N) has a vanishing main diagonal m and m − 1 vanishing side diagonals: (A )ij =0for |i − j| <m.Form>#I, the inequality |i − j| <mholds for all indices; hence Am =0. Two matrices A and B are called commutative (or ‘A and B commute’) if AB = BA.