Linear Algebra Michael Taylor Math. Dept., UNC E-mail address: [email protected] 2010 Mathematics Subject Classification. 15-00, 15-01 Key words and phrases. vector spaces, linear transformations, matrices, determinants, eigenvectors, eigenvalues, generalized eigenvectors, inner products, norms, trace, adjoint, unitary, orthogonal transformations, Jordan canonical form, polar decomposition, singular value decomposition, tensor products, exterior algebra, fields, rings, modules, principal ideal domains, groups, quaternions, Clifford algebra, octonions, Noetherian rings Contents Preface xi Some basic notation xvii Chapter 1. Vector spaces, linear transformations, and matrices 1 x1.1. Vector spaces 3 Exercises 7 x1.2. Linear transformations and matrices 9 Exercises 14 x1.3. Basis and dimension 17 Exercises 21 x1.4. Matrix representation of a linear transformation 24 Exercises 26 x1.5. Determinants and invertibility 27 Exercises 35 The Vandermonde determinant 38 x1.6. Applications of row reduction and column reduction 40 Exercises 54 Chapter 2. Eigenvalues, eigenvectors, and generalized eigenvectors 55 x2.1. Eigenvalues and eigenvectors 58 Exercises 61 x2.2. Generalized eigenvectors and the minimal polynomial 65 Exercises 70 vii viii Contents x2.3. Triangular matrices and upper triangularization 72 Exercises 76 x2.4. The Jordan canonical form 78 Exercises 82 Chapter 3. Linear algebra on inner product spaces 83 x3.1. Inner products and norms 86 Exercises 90 x3.2. Norm, trace, and adjoint of a linear transformation 96 Exercises 98 x3.3. Self-adjoint and skew-adjoint transformations 101 Exercises 108 x3.4. Unitary and orthogonal transformations 112 Exercises 117 x3.5. Schur's upper triangular representation 123 Exercises 127 x3.6. Polar decomposition and singular value decomposition 128 Exercises 136 x3.7. The matrix exponential 137 Exercises 147 x3.8. The discrete Fourier transform 152 Exercises 164 Chapter 4. Further basic concepts: duality, convexity, positivity 167 x4.1. Dual spaces 170 Exercises 171 x4.2. Convex sets 173 Exercises 177 x4.3. Quotient spaces 178 Exercises 180 x4.4. Positive matrices and stochastic matrices 181 Exercises 187 Chapter 5. Multilinear algebra 189 x5.1. Multilinear mappings 191 Exercises 192 x5.2. Tensor products 194 Contents ix Exercises 196 x5.3. Exterior algebra 197 Exercises 205 x5.4. Isomorphism Skew(V ) ≈ Λ2V and the Pfaffian 209 Chapter 6. Linear algebra over more general fields 211 x6.1. Vector spaces over more general fields 213 Exercises 222 x6.2. Rational matrices and algebraic numbers 225 Chapter 7. Rings and modules 227 x7.1. Rings and modules 231 Exercises 243 x7.2. Modules over principal ideal domains 245 Exercises 254 x7.3. The Jordan canonical form revisited 258 Exercises 260 x7.4. Integer matrices and algebraic integers 261 Exercises 265 x7.5. Noetherian rings and Noetherian modules 266 x7.6. Polynomial rings over UFDs 273 Chapter 8. Special structures in linear algebra 277 x8.1. Quaternions and matrices of quaternions 280 Exercises 287 x8.2. Algebras 290 x8.3. Clifford algebras 299 Exercises 309 x8.4. Octonions 310 Appendix A. Complementary results 333 xA.1. The fundamental theorem of algebra 335 xA.2. Averaging rotations 337 xA.3. Groups 343 xA.4. Finite fields and other algebraic field extensions 351 Bibliography 363 Index 365 Preface Linear algebra is an important gateway connecting elementary mathematics to more advanced subjects, such as multivariable calculus, systems of dif- ferential equations, differential geometry, and group representations. The purpose of this work is to provide a treatment of this subject in sufficient depth to prepare the reader to tackle such further material. In Chapter 1 we define the class of vector spaces (real or complex) and discuss some basic examples, including Rn and Cn, or, as we denote them, Fn, with F = R or C. We then consider linear transformations between such spaces. In particular, we look at an m × n matrix A as defining a linear transformation A : Fn ! Fm. We define the range R(T ) and null space N (T ) of a linear transformation T : V ! W . In x1.3 we define the notion of basis of a vector space. Vector spaces with finite bases are called finite dimensional. We establish the crucial property that any two bases of such a vector space V have the same number of elements (denoted dim V ). We apply this to other results on bases of vector spaces, culminating in the \fundamental theorem of linear algebra," that if T : V ! W is linear and V is finite dimensional, then dim N (T ) + dim R(T ) = dim V , and discuss some of its important consequences. A linear transformation T : V ! V is said to be invertible provided it is one-to-one and onto, i.e., provided N (T ) = 0 and R(T ) = V . In x1.5 we define the determinant of such T , det T (when V is finite dimensional), and show that T is invertible if and only if det T =6 0. One useful tool in the study of determinants consists of row operations and column operations. In x1.6 we pursue these operations further, and show how applying row reduction to an m × n matrix A works to display a basis of its null space, while applying column reduction to A works to display a basis of its range. xi xii Preface In Chapter 2 we study eigenvalues λj and eigenvectors vj of a linear transformation T : V ! V , satisfying T vj = λjvj. Results of x1.5 imply that λj is a root of the \characteristic polynomial" det(λI − T ). Section 2.2 extends the scope of this study to a treatment of generalized eigen- vectors of T , which are shown to always form a basis of V , when V is a finite-dimensional complex vector space. This ties in with a treatment of properties of nilpotent matrices and triangular matrices, in x2.3. Combining the results on generalized eigenvectors with a closer look at the structure of nilpotent matrices leads to the presentation of the Jordan canonical form for an n × n complex matrix, in x2.4. In Chapter 3 we introduce inner products on vector spaces and endow them with a Euclidean geometry, in particular with a distance and a norm. In x3.2 we discuss two types of norms on linear transformations, the \opera- tor norm" and the \Hilbert-Schmidt norm." Then, in xx3.3{3.4, we discuss some special classes on linear transformations on inner product spaces: self- adjoint, skew-adjoint, unitary, and orthogonal transformations. In x3.5 we establish a theorem of Schur that for each n × n matrix A, there is an or- thonormal basis of Cn with respect to which A takes an upper triangular form. Section 3.6 establishes a polar decomposition result, that each n × n complex matrix can be written as KP , with K unitary and P positive semi- definite, and a related result known as the singular value decomposition of a complex matrix (square or rectangular). In x3.7 we define the matrix exponential etA, for A 2 M(n; C), so that x(t) = etAv solves the differential equation dx=dt = Ax; x(0) = v. We produce a power series for etA and establish some basic properties. The matrix exponential is fundamental to applications of linear algebra to ODE. Here, we use this connection to produce another proof that if A is an n × n complex matrix, then Cn has a basis consisting of generalized eigenvectors of A. The proof given here is completely different from that of x2.2. Section 3.8 takes up the discrete Fourier transform (DFT), acting on functions f : Z ! C that are periodic, of period n. This transform diag- onalizes an important class of operators known as convolution operators. This section also treats a fast implementation of the DFT, known as the Fast Fourier Transform (FFT). Chapter 4 introduces some further basic concepts in the study of linear algebra on real and complex vector spaces. In x4.1 we define the dual space V 0 to a vector space. We associate to a linear map A : V ! W its transpose At : W 0 ! V 0 and establish a natural isomorphism V ≈ (V 0)0 when dim V < 1. Section 4.2 looks at convex subsets of a finite-dimensional vector space. Section 4.3 deals with quotient spaces V=W when W is a linear subspace of V . Preface xiii In x4.4 we study positive matrices, including the important class of sto- chastic matrices. We establish the Perron-Frobenius theorem, which states that, under a further hypothesis called irreducibility, a positive matrix has a positive eigenvector, unique up to scalar multiple, and draw useful corollaries for the behavior of irreducible stochastic matrices. Chapter 5 deals with multilinear maps and related constructions, includ- ing tensor products in x5.2 and exterior algebra in x5.3, which we approach as a further development of the theory of determinants, initiated in x1.5. Results of this chapter are particularly useful in the development of differ- ential geometry and manifold theory, involving studies of tensor fields and differential forms. In Chapter 6 we extend the scope of our study of vector spaces, adding to R and C more general fields F. We define the notion of a field, give a number of additional examples, and describe how results of Chapters 1, 2, 4, and 5 extend to vector spaces over a general field F. Specific fields considered include both finite fields, such as Z=(p), and fields of algebraic numbers. In x6.2 we show that the set A of algebraic numbers, which are roots of polynomials with rational coefficients, are precisely the eigenvalues of square matrices with rational entries. We use this, together with some results of x5.2, to obtain a proof that A is a field, different from that given in x6.1.