Linear Algebra with Applications John T. Scheick Duke University Boston, Massachusetts Burr Ridge, Illinois Dubuque, Iowa Madison, Wisconsin New York, New York San Francisco, California St. Louis, Missouri TABLE OF CONTENTS AND OUTLINE Preface xiii 0 Systems of Equations and Matrices 1 0.1 Systems of Linear Equations and Matrices 1 0.2 Solution of Homogeneous Systems 4 Gauss elimination / Row-reduced echelon form / Existence of nontrivial solutions and description of the solution of Ax = 0 0.3 Solution of Inhomogeneous Systems 10 Gauss elimination revisited / Existence and uniqueness of solutions 0.4 Matrix Algebra 16 Matrix multiplication, transposition, and conjugation / Identities and rules 0.5 Systems of Equations and Matrix Inverses 23 Conditions for the existence of an inverse / Computation of the inverse / Elementary matrices 0.6 Fields 29 1 Vector Spaces 32 1.1 Vector Spaces 32 Axioms / Principal examples 1.2 Subspaces 37 Subspaces / Linear combinations and span / Connection to matrices 1.3 Linear Independence 43 Examples / Connection to matrices 1.4 Basis and Dimension 53 Basic theory / Examples / Applications of the theory to some fundamental vector spaces, matrices, and systems of equations 1.5 Coordinates with Respect to a Basis 62 Change of basis and change of coordinates 2 Linear Operators 73 2.1 Preliminary Topics 73 Functions /Linear operators /Null space and range / Examples IX x Table of Contents and Outline 2.2 The Rank and Nullity Theorem 80 Rank and nullity theorem / Operator inverses /Application to matrix theory / Computation of the range and null space of a matrix 2.3 Linear Operators and Matrices 91 Matrix of an operator / Operator algebra / Change of basis and similar matrices /Applications Inner Product Spaces 103 3.1 Preliminaries 103 Definitions and examples /Norms; angle between vectors 3.2 Orthogonal Sets 108 Computational advantages of orthogonal sets / Fourier coefficients and Parseval 's identity / Gram-Schmidt process / QR factorization 3.3 Approximation and Orthogonal Projection 120 Equivalence of the problems / Computations using orthogonal and nonorthogonal sets / Normal equations / Projection operator / M1 / Examples in many settings 3.4 Applications of Projection Theory 133 Projections in Fm / Weighted linear regression / Data fitting with polynomials, weighted and unweighted 3.5 Orthogonal Complements 151 Decomposition of the vector space / Applications to approximation and matrix theory / Fredholm alternative 3.6 The Gram Matrix and Orthogonal Change of Basis 158 Matrix representation of inner products / Orthogonal change of basis / Rank of a Gram matrix Diagonalizable Linear Operators 164 4.1 Eigenvalues and Eigenvectors 164 Definitions / Spectrum and eigenspaces of an operator / Theoretical computations using determinants / Properties of the characteristic polynomial / Geometric and algebraic multiplicities / Remarks on numerical calculations 4.2 Linear Operators with an Eigenbasis 173 Diagonalizable operators and their computational advantages / Similarity to a diagonal matrix 4.3 Functions of Diagonalizable Operators 186 Two competing definitions / Functions of matrices / General properties of functions of diagonalizable operators /Minimal polynomial Table of Contents and Outline xi 4.4 First-Order Matrix Differential Equations 196 Decoupling the differential equations: Two viewpoints for diagonalizable matrices / eAl 4.5 Estimates of Eigenvalues: Gershgorin's Theorems 202 4.6 Application to Finite Difference Equations 207 Biological models, Markov chain examples, and finite difference equations Appendix: Review of determinants 216 5 The Structure of Normal Operators 219 5.1 Adjoints and Classification of Operators 219 Definitions / Normal, Hermitian, and unitary operators / Matrix characterization 5.2 The Spectral Theorem 228 Spectral theorem and resolution / Functions of normal operators / Simultaneous diagonalization of normal operators 5.3 Applications to Matrix Theory 237 Functions of normal matrices / Generalized eigenvalue problem 5.4 Extremum Principles for Hermitian Operators 250 The Rayleigh quotient and its extremal properties / Courant- Fischer theorem / Interlacing theorem for bordered matrices 5.5 The Power Method 257 Estimating the dominant eigenvalue and eigenvector / Sharpening estimates / Approximation of secondary eigenvalues and eigenvectors / Inverse power method / Subspace methods 5.6 The Rayleigh-Ritz Method 269 Approximation of a finite number of eigenvalues and eigenvectors of a Hermitian operator defined on an infinite dimensional space 6 Bilinear and Quadratic Forms 282 6.1 Preliminaries 282 Definitions and examples / Elementary properties / Polar identities / The matrix of a bilinear form and change of basis; congruent matrices 6.2 Classification of Hermitian Quadratic Forms 291 Classification of quadratic forms, operators, and matrices / Diagonalization and the law of inertia /Minimization problems xii Table of Contents and Outline 6.3 Orthogonal Diagonalization 301 Orthogonal diagonalization and the Principal Axis theorem / Applications: level sets and the strain ellipsoid 6.4 Other Methods of Diagonalization 307 Completion of squares, simultaneous row and column operations, a method of Jacobi 6.5 Simultaneous Diagonalization of Quadratic Forms 317 Simultaneous diagonalization of two and three quadratic forms and the connection to the generalized eigenvalue problem Small Oscillations 324 7.1 Differential Equations of Small Oscillations 324 Derivation of the differential equations by Newton's laws, Kirchoff's laws, Lagrange's equations of motion, and the Ritz method for continuous systems 7.2 Undamped Small Oscillations 331 Solution of the differential equations using two viewpoints / The language of small oscillations / Application of the power method / Examples, including the Ritz method / Response to harmonic excitation 7.3 Damped Small Oscillations 354 Conditions in order that the differential equations can be decoupled / Solution of the differential equations / Response to harmonic excitation 7.4 Galerkin's Method for Partial Differential Equations 363 Illustration of the method for the wave equation Factorizations and Canonical Forms 373 8.1 The Singular Value and Polar Decompositions 373 The SVD and its interpretations / Polar decomposition 8.2 Applications of the SVD 380 Solutions of Ax = y and the pseudoinverse / Applications in numerical analysis / Applications in pattern recognition 8.3 Schur's Theorem 389 Schur's theorem / Matrix norms and the sequence A" / Iterative solution of equations 8.4 Jordan Canonical Form 395 Applications to differential equations / eM Answers to Selected Problems 405 Bibliography 423 Index 425.