A Appendix: Linear Algebra and Functional Analysis
A Appendix: Linear Algebra and Functional Analysis In this appendix, we have collected some results of functional analysis and matrix algebra. Since the computational aspects are in the foreground in this book, we give separate proofs for the linear algebraic and functional analyses, although finite-dimensional spaces are special cases of Hilbert spaces. A.1 Linear Algebra A general reference concerning the results in this section is [47]. The following theorem gives the singular value decomposition of an arbitrary real matrix. Theorem A.1. Every matrix A ∈ Rm×n allows a decomposition A = UΛV T, where U ∈ Rm×m and V ∈ Rn×n are orthogonal matrices and Λ ∈ Rm×n is diagonal with nonnegative diagonal elements λj such that λ1 ≥ λ2 ≥ ··· ≥ λmin(m,n) ≥ 0. Proof: Before going to the proof, we recall that a diagonal matrix is of the form ⎡ ⎤ λ1 0 ... 00... 0 ⎢ ⎥ ⎢ . ⎥ ⎢ 0 λ2 . ⎥ Λ = = [diag(λ ,...,λm), 0] , ⎢ . ⎥ 1 ⎣ . .. ⎦ 0 ... λm 0 ... 0 if m ≤ n, and 0 denotes a zero matrix of size m×(n−m). Similarly, if m>n, Λ is of the form 312 A Appendix: Linear Algebra and Functional Analysis ⎡ ⎤ λ1 0 ... 0 ⎢ ⎥ ⎢ . ⎥ ⎢ 0 λ2 . ⎥ ⎢ ⎥ ⎢ . .. ⎥ ⎢ . ⎥ diag(λ ,...,λn) Λ = ⎢ ⎥ = 1 , ⎢ λn ⎥ ⎢ ⎥ 0 ⎢ 0 ... 0 ⎥ ⎢ . ⎥ ⎣ . ⎦ 0 ... 0 where 0 is a zero matrix of the size (m − n) × n. Briefly, we write Λ = diag(λ1,...,λmin(m,n)). n Let A = λ1, and we assume that λ1 =0.Let x ∈ R be a unit vector m with Ax = A ,andy =(1/λ1)Ax ∈ R , i.e., y is also a unit vector. We n m pick vectors v2,...,vn ∈ R and u2,...,um ∈ R such that {x, v2,...,vn} n is an orthonormal basis in R and {y,u2,...,um} is an orthonormal basis in Rm, respectively.
[Show full text]