Elements of Linear Algebra, Topology, and Calculus
Total Page:16
File Type:pdf, Size:1020Kb
00˙AMS September 23, 2007 © Copyright, Princeton University Press. No part of this book may be distributed, posted, or reproduced in any form by digital or mechanical means without prior written permission of the publisher. Appendix A Elements of Linear Algebra, Topology, and Calculus A.1 LINEAR ALGEBRA We follow the usual conventions of matrix computations. Rn×p is the set of all n p real matrices (m rows and p columns). Rn is the set Rn×1 of column vectors× with n real entries. A(i, j) denotes the i, j entry (ith row, jth column) of the matrix A. Given A Rm×n and B Rn×p, the matrix product Rm×p ∈ n ∈ AB is defined by (AB)(i, j) = k=1 A(i, k)B(k, j), i = 1, . , m, j = 1∈, . , p. AT is the transpose of the matrix A:(AT )(i, j) = A(j, i). The entries A(i, i) form the diagonal of A. A Pmatrix is square if is has the same number of rows and columns. When A and B are square matrices of the same dimension, [A, B] = AB BA is termed the commutator of A and B. A matrix A is symmetric if −AT = A and skew-symmetric if AT = A. The commutator of two symmetric matrices or two skew-symmetric matrices− is symmetric, and the commutator of a symmetric and a skew-symmetric matrix is skew-symmetric. The trace of A is the sum of the diagonal elements of A, min(n,p ) tr(A) = A(i, i). i=1 X We have the following properties (assuming that A and B have adequate dimensions) tr(A) = tr(AT ), (A.1a) tr(AB) = tr(BA), (A.1b) tr([A, B]) = 0, (A.1c) tr(B) = 0 if BT = B, (A.1d) − tr(AB) = 0 if AT = A and BT = B. (A.1e) − An n n matrix A is invertible (or nonsingular) if there exists an n n matrix × × B such that AB = BA = In, where In denotes the n n identity matrix with ones on the diagonal and zeros everywhere else. If this× is the case, then B is uniquely determined by A and is called the inverse of A, denoted by A−1.A matrix that is not invertible is called singular. A matrix Q is orthonormal if QT Q = I. A square orthonormal matrix is termed orthogonal and satisfies Q−1 = QT . The notion of an n-dimensional vector space over R is an abstraction of Rn endowed with its operations of addition and multiplication by a scalar. Any For general queries, contact [email protected] 00˙AMS September 23, 2007 © Copyright, Princeton University Press. No part of this book may be distributed, posted, or reproduced in any form by digital or mechanical means without prior written permission of the publisher. 190 APPENDIX A n-dimensional real vector space is isomorphic to Rn. However, producing a diffeomorphism involves generatingE a basis of , which may be computa tionally intractable; this is why most of the followingE material is presented on abstract vector spaces. We consider only finite-dimensional vector spaces over R. A normed vector space is a vector space endowed with a norm, i.e., a mapping x x RE with the following properties. For all a R and all x, y ,∈ E 7→ k k ∈ ∈ ∈ E 1. x 0, and x = 0 if and only if x is the zero vector; 2. kaxk ≥ = a x k ; k 3. kx +k y | | kxk + y . k k ≤ k k k k Given two normed vector spaces and , a mapping A : is a (linear) operator if A[αx + βy] =E αA[x]F + βA[y] for all x, yE 7→ Fand all α, β R. The set L( ; ) of all operators from to is ∈a Evector space. An∈ operator A L(ERnF; Rm) can be represented byE an Fm n matrix (also denoted by A) such∈ that A[x] = Ax for all x Rn . This× represen tation is an isomorphism that matches the composition∈ of operators with the multiplication of matrices. Let , , and be normed vector spaces, E F G let L be a norm on L( ; ), L be a norm on L( ; ), and k · k (E;F) E F k · k (F;G) F G L be a norm on L( ; ). These norms are called mutually consistent if k·k (E;G) E G B A L(E;G) A L(E;F) B L(F;G) for all A L( ; ) and all B L( ; ). Ak consistent◦ k or≤ submultiplicative k k k k norm is a norm∈ thatE F is mutually ∈consistentF G with itself. The operator norm or induced norm of A L( ; ) is ∈ E F A[x] A := max k k. k k x∈E,x6=0 x k k Operator norms are mutually consistent. Given normed vector spaces , , and , a mapping A from E1 E2 F E1 × E2 to is called a bilinear operator if for any x2 2 the linear mapping x F A[x , x ] is a linear operator from to ,∈ and E for any x the 1 7→ 1 2 E1 F 1 ∈ E1 linear mapping x2 A[x1, x2] is a linear operator from 2 to . The set of bilinear operators from7→ to is denoted by L( ,E ; F), and we use E1 × E2 F E1 E2 F the notation L2( ; ) for L( , ; ). These definitions are readily extended to multilinear operators.E F A bilinearE E F operator A L ( ; ) is symmetric if ∈ 2 E F A[x, y] = A[y, x] for all x, y . A symmetric bilinear operator A L2( , R) is positive-definite if A[x, x]∈ > E 0 for all x , x = 0. ∈ E By Euclidean space we mean a finite-dimensional∈ E 6 vector space endowed with an inner product, i.e., a bilinear, symmetric positive-definite form , . The canonical example is Rn, endowed with the inner product h· ·i x, y := x T y. h i An orthonormal basis of an n-dimensional Euclidean space is a sequence (e , . , e ) of elements of such that E 1 n E 1 if i = j, ei, ej = h i 0 if i = j. ( 6 For general queries, contact [email protected] 00˙AMS September 23, 2007 © Copyright, Princeton University Press. No part of this book may be distributed, posted, or reproduced in any form by digital or mechanical means without prior written permission of the publisher. ELEMENTS OF LINEAR ALGEBRA, TOPOLOGY, AND CALCULUS 191 Given an orthonormal basis of , the mapping that sends the elements of to their vectors of coordinates in ERd is an isomorphism (an invertible mappingE that preserves the vector space structure and the inner product). An operator T : is termed symmetric if T [x], y = x, T [y] for all x, y . GivenE an → operator E T : betweenh two Euclideani h spacesi and ∈, the E adjoint of T is the operatorE → F T ∗ : satisfying T [x], y E = x,F T ∗[y] for all x and all y . TheF kernel→ E of the operatorh Ti is theh lineari subspace ker(∈ ET ) = x ∈ : T F [x] = 0 . The range (or image) of T is the set range(T ) = T [x]: {x ∈ E . Given a }linear subspace of , the orthogonal complement {of is ∈⊥ E}= x : x, y = 0 for allS y E . Given x , there is a uniqueS decompositionS { ∈ Ex =h x i+ x with x ∈ S} and x ∈⊥ E; 1 2 1 ∈ S 2 ∈ S x1 is the orthogonal projection of x onto and is denoted by ΠS (x). The Moore-Penrose inverse or pseudo-inverse ofS an operator T is the operator † −1 T : : y (T ⊥ ) [Π y], F → E 7→ |(ker(T )) range(T ) ⊥ where the restriction T ⊥ : (ker(T )) range(T ) is invertible by |(ker(T )) → construction and Πrange(T ) is the orthogonal projector in onto range(T ). The Euclidean norm on a Euclidean space is F E x := x, x . k k h i n The Euclidean norm on R is p x := √xT x. k k The Euclidean norm on Rn×p endowed with the inner product X, Y = tr(XT Y ) is the Frobenius norm given by A = ( (A(i, j))2h)1/2 . Thei k kF i,j operator norm on Rn×n L(Rn; Rn), where Rn is endowed with its Eu P clidean norm, is the spectral≃ norm given by A = λ (AT A), k k2 max T q where λmax(A A) is the largest eigenvalue of the positive-semidefinite matrix AT A. An operator T in L( ; ) is invertible if for all y there exists x such that y = T [x]. AnE operatorE that is not invertible∈ E is termed singular∈ E. Let idE denote the identity operator on : idE [x] = x for all x . Let T L( ; ) be a symmetric operator. A realE number λ is an eigenvalue∈ E of ∈ E E T if the operator T λ id is singular; any vector x = 0 in the kernel of − E 6 T λ idE is an eigenvector of T corresponding to the eigenvalue λ. −References: [GVL96], [Hal74], [Die69]. A.2 TOPOLOGY A topology on a set X is an abstraction of the notion of open sets in Rn . Defining a topology on X amounts to saying which subsets of X are open while retaining certain properties satisfied by open sets in Rn. Specifically, a topology on a set X is a collection of subsets of X, called open sets, such that T For general queries, contact [email protected] 00˙AMS September 23, 2007 © Copyright, Princeton University Press.