Cambridge University Press 978-1-107-10381-8 — a Second Course in Linear Algebra Stephan Ramon Garcia , Roger A

Cambridge University Press 978-1-107-10381-8 — A Second Course in Linear Algebra Stephan Ramon Garcia , Roger A. Horn Index More Information

Index

A0 = I,4 arithmetic–geometric mean inequality, 99 AB vs. BA associative law for complex multiplication, 400, 409 criterion for similarity, 268 augmented matrix, see matrix, augmented eigenvalues, 214 Autonne’s theorem, 363 Hermitian, 301 Jordan forms, 267 backward substitution, 167, 315 one factor is invertible, 218 Balmer series, 200 singular values, 343 base case, see mathematical induction, base case trace, see trace, tr AB = tr BA basis absolute row sum and invertible matrices, 34 deleted, 188 change of and similarity, 44 largest, 188 coordinate vector, 40 additive law for complex conjugation, 402 definition, 33 additivity axiom for inner product, 87 is a finite list, 33 adjoint of eigenvectors, 184 definition, 5, 111 orthonormal, 104, 108 in a finite-dimensional space, 113 standard, 35 is idempotent, 5 basis representation of a block matrix, 66 function, 40 of an isometry, 126 of a linear transformation, 43 of inverse, 5 of a vector, 40 reverses products, 5 with respect to an orthonormal basis, 105, 110 similar to, 270 Bauer–Fike theorem, 362, 394 singular value decomposition, 341 Bessel’s inequality, 114, 121 unique, 111 best approximation adjugate linear model, 165 and inverse, 9 solution of normal equations, 161 Cauchy expansion, 71 theorem, 159 definition, 9 bidiagonal matrix, 4 rank of, 147 bilinear, 88 algebraic multiplicity Binet’s formula, see Fibonacci numbers, Binet’s ≥ geometric multiplicity, 206 formula definition, 203 biorthogonality, see principle of biorthogonality all-ones matrix, 198 block Gaussian elimination, 70, 76 all-ones vector, 26 block matrix annihilating polynomial 2 × 2,63 2 × 2 matrix, 181 addition, 63 and eigenvalues, 181 adjoint, 66 characteristic polynomial, see Cayley–Hamilton centrosymmetric, 77, 218 theorem complex type, 77 definition, 180 conjugate transpose, 66 existence, 180 definition, 63 roots include all eigenvalues, 183 multiplication, 63 area and determinant, 10 transpose, 66 argument of a complex number block upper triangular matrix definition, 401 blocks with distinct eigenvalues, 232 multiplicative law, 406 eigenvalues, 205 arithmetic mean, 99 invariant subspace, 169

© in this web service Cambridge University Press www.cambridge.org Cambridge University Press 978-1-107-10381-8 — A Second Course in Linear Algebra Stephan Ramon Garcia , Roger A. Horn Index More Information

412 Index

bordered matrix subspace, 288 Cauchy expansion of determinant, 70, 77 sums and products, 288 definition, 66 circular shift matrix, 287 determinant, 70, 77 codomain, 1 eigenvalue interlacing, 378 coefficient matrix, see matrix, coefficient Jordan block, 246 coefficients, 6 reduction formula for determinant, 70 cofactor, see determinant, cofactor bordering, see bordered matrix column inclusion, 307 Born, Max, 200 column of a matrix, 2 Brauer’s theorem, 237 column partition, see partitioned matrix column rank, see rank, column cancellation theorem for direct sums, 275 column space Cartesian decomposition and span, 24 and spectral norm, 355 definition, 23 definition, 290 inclusion, 80 normal matrix, 290 same dimension as row space, 59 Cauchy expansion of determinant, 70, 77 two direct sums have same, 80 Cauchy matrix two matrices have same, 80 definition, 325 column vector is a Gram matrix, 325 definition, 7 positive semidefinite, 325 zero, 7 Cauchy–Schwarz inequality commutant alternative proof, 98, 121 definition, 213 case of equality, 93 dimension, 213, 300 in Fn, 323 is a subspace, 213 in the plane, 85 commutative law for complex multiplication, 399 inner product space, 93 commutator Cayley transform characterization, see Shoda’s theorem definition, 300 definition, 71 of a skew-Hermitian matrix, 300 nonzero scalar matrix is not a, 72 of a unitary matrix, 300 zero trace, 72 Cayley–Hamilton theorem commuting matrices false proof, 239 block diagonal form, 235 proof, 225 circulant matrices, 288 Cecil Sagehen, 47, 273 common eigenvector, 194, 195 change of basis matrix definition, 3 definition, 45 diagonalizable, 209 inverse, 46 eigenvalues of sum and product, 236 is invertible, 46 normal matrices, 284, 285 characteristic polynomial one has distinct eigenvalues, 219 companion matrix, 230 real symmetric matrices, 284 definition, 202 simultaneous upper triangularization, 235 of inverse, 239 simultaneously diagonalizable matrices, 209 roots are eigenvalues, 202 sum and product of spectra, 239 similarity invariant, 205 compact singular value decomposition upper triangular matrix, 203 column vector, 334 Cholesky factorization definition, 333 Hadamard inequality, 324 orthonormal basis of column space, 334 min matrix, 316, 323 orthonormal basis of row space, 334 theorem, 316 row vector, 334 circulant matrix theorem, 333 circular shift matrix, 287 uniqueness, 335 cyclic permutation matrix, 287 companion matrix definition, 287 characteristic polynomial, 220 diagonalized by Fourier matrix, 288 definition, 230 eigenvalues, 288 eigenvectors, 241 eigenvectors, 288 geometric multiplicity of eigenvalues, 241 polynomial in cyclic permutation matrix, 288 Jordan canonical form, 274

Index 413

minimal polynomial = characteristic polynomial, 230 permutation similarity, 384 complete mathematical induction, see mathematical real orthogonal similarity, 384 induction, complete two unitary matrices, 389 complex conjugation unitary similarity, 384 additive law, 402 conic section definition, 401 ellipse, 317 multiplicative law, 402 hyperbola, 317 of a matrix, see conjugate of a matrix conjugate linearity of inner product, 88 complex exponential function conjugate of a complex number, see complex definition, 407 conjugation power series, 409 conjugate of a matrix complex numbers definition, 5 absolute value, 401 does not reverse products, 5 and 2 × 2 real matrices, 399 is idempotent, 5 and Cartesian coordinates, 398 similar to, 269 argument, 401 conjugate symmetry axiom for inner product, 87 conjugate, 401 conjugate transpose, see adjoint de Moivre’s formula, 406 conjugation Euler’s formula, 407 complex, see complex conjugation imaginary part, 398 consistent linear system, see linear system, consistent imaginary unit, 398 constant terms, 6 inverse, 400 contraction modulus, 401 convex combination of unitaries, 369 notation, 1, 398 definition, 369 polar form, 406 mean of two unitaries, 369 product, 398 principal submatrix of a unitary matrix, 369 purely imaginary, 398 convergent matrix real part, 398 definition, 259 sum, 398 similarity, 259 triangle inequality, 404 spectral criterion, 260 complex plane, 398 convex set, 101 complex symmetric matrix coordinates of a vector, 40 Autonne’s theorem, 363 Cramer’s rule, 55, 121 definition, 5, 363 CS decomposition, 144, 147 every Jordan block similar to a, 272 cyclic permutation matrix every matrix similar to a, 272 definition, 287 full-rank factorization, 367 diagonalized by Fourier matrix, 287 need not be diagonalizable, 365 spectrum, 287 polar decomposition, 367 pseudoinverse is symmetric, 368 de Moivre’s formula, 406 simultaneous diagonalization with a positive definite defect from normality, 285 matrix, 367 degree, see polynomial, degree singular value decomposition, 363 dependent, see linear dependence conditioning derived norm, see norm, derived eigenvalues, 362, 388 descending sequence of subspaces, 39 linear systems, 361 determinant perfect, 361 2 × 2 block matrix, 77 conformal partitions, see partitioned matrix 2 × 2 matrix, 9 congruence 3 × 3 matrix, 9 definition, 396 and permutations, 10 equivalence relation, 396 area in R2,10 ∗congruence bordered matrix, 70 definition, 384 Cauchy expansion, 70 equivalence relation, 384 cofactor, 9 of two Hermitian matrices, 385 compute via minors, 9 of two inertia matrices, 390 compute via row reduction, 10 of two normal matrices, 390 criterion for positive definiteness, 382 of two real inertia matrices, 384 invertibility criterion, 9

414 Index

determinant (cont.) implies diagonalizable, 207 Laplace expansion, 9 linearly independent eigenvectors, 184 of a triangular matrix, 10 distinct scalars, 67 of the inverse, 10 distributive law for complex multiplication, preserved by similarity, 49 400 product of eigenvalues, 204, 224 dividend, see division algorithm, dividend product rule, see product rule for determinants division algorithm Schur complement, 70 and Cayley–Hamilton theorem, 227 Sylvester identity, 79 definition, 13 Vandermonde matrix, see Vandermonde matrix, dividend, 13 determinant divisor, 13 volume in R3,10 quotient, 13 diagonal dominance remainder, 13 definition, 191 divisor, see division algorithm, divisor sufficient condition for positive semidefinite, 310 domain, 1 diagonal entries dot product definition, 4 and Euclidean length, 85 equal, 73 definition, 85 majorization and eigenvalues, 383 properties, 85 special structures, 81 diagonal matrix eA, see functional calculus, matrix exponential commute with a, 67 eigenpair definition, 4 definition, 175 left multiplication by a, 67 eigenspace right multiplication by a, 67 definition, 186 similarity of two, 292 dimension, see geometric multiplicity diagonalizable matrix is a subspace, 186 T criterion via minimal polynomial, 233 of A and A , 187 criterion via unitary triangularization, 233 eigenvalue interlacing definition, 207 bordered matrix, 378 direct sum of diagonalizable matrices, 208, 241 principal submatrix, 381 equal algebraic and geometric multiplicities, rank-1 additive perturbation, 374 206, 208 rank-2 additive perturbation, 376 exponential of a, 272 tridiagonal matrix, 395 rank, 209, 216 eigenvalues rank of powers, 240 0 ∈/ spec A ⇐⇒ A is invertible, 180 simultaneously diagonalizable, see simultaneously 2 × 2 matrix, 182 diagonalizable matrices AB vs. BA, 214 unispectral, 233 algebraic multiplicity, see algebraic multiplicity unitarily, 282 and determinant, 204 differential equations and main diagonal entries, 203 diagonalizable matrix, 256 and trace, 204 uncoupled systems, 257 block upper triangular matrix, 205 dimension conditioning issues, 362, 388 definition, 35 definition, 175 finite, 35 distinct, see distinct eigenvalues infinite, 35 existence, 182 of a subspace, 37 finitely many, 183 of a sum, 37 geometric multiplicity, see geometric multiplicity of an intersection, 37 idempotent matrix, 198 theorem for linear transformations, 50 inverse problems, 396 theorem for matrices, 51 majorization and diagonal entries, 383 zero, 35 monotonicity theorem, 394 direct sum of matrices, 66 multiplicity, see multiplicity, eigenvalues direct sum of subspaces, see subspace, direct sum nilpotent matrix, 216 discriminant, 216 of a low-rank matrix, 216 distinct, 1 of a real matrix, 204 distinct eigenvalues of adjoint, 204 definition, 184, 203 of conjugate, 204

Index 415

of inverse, 197 forward substitution, 315 of transpose, 204 Fourier roots of characteristic equation, 202 Jean Baptiste Joseph, 124 sequential integers matrix, 219 series, 118 shift by a rank-1 adjustment, 237, 374 Fourier matrix similarity invariant, 205 2 × 2, 130 simple, 203 4 × 4, 130 two diagonalizable matrices with the same, 292 diagonalizes circulant matrices, 287 two normal matrices with the same, 292 inverse = conjugate, 130 two real normal matrices with the same, 292 is symmetric, 130 Weyl inequalities, 386 is unitary, 130 eigenvectors spectrum, 199 basis of, 184, 206 Fredholm Alternative, 174 definition, 175 Frobenius linearly independent, 184 condition number, 370 must be nonzero, 175 inner product, 88 of commuting matrices, 194 norm, see Frobenius norm elementary matrices rank inequality, 79 definition, 8 Frobenius norm invertible, 8 and singular values, 329 represent row operations, 8 definition, 91 elementary row operations, 7 of a unitary matrix, 144 entrywise product, see Hadamard product self-adjoint, 367 EP matrix submultiplicative, 349 definition, 301 submultiplicative for Hadamard product, 367 rank AB = rank BA, 302 Fuglede–Putnam theorem, 286, 299 equivalence relation full column rank, see rank, full column ∗congruence, 384 full rank, see rank, full congruence, 396 full row rank, see rank, full row definition, 49 full-rank factorization real orthogonal similarity, 145 and eigenvalues, 216 unitary equivalence, 367 and the pseudoinverse, 368 unitary similarity, 145 block matrices, 78 error bounds in a linear system, 360 complex symmetric matrix, 367 Euclidean length definition, 62 and dot product, 85 function in the plane, 84 definition, 1 properties, 86 jump discontinuity, 118 Euclidean norm periodic, 116 derived from standard inner product, 91 piecewise continuous, 118 Euler functional calculus formula, 407 2 × 2 matrices, 221, 272, 322 sum of reciprocal squares, 123 general functions, 212 exp A, see functional calculus, matrix exponential matrix exponential, 212 exponential function, see complex exponential function polynomials, 210 square root, 220 fast Fourier transform, 147 Fundamental Theorem of Algebra, 13 FFT, see fast Fourier transform fundamental theorem of calculus, 112, 200, 300 Fibonacci numbers approximation, 216 Gaussian elimination Binet’s formula, 16, 211, 216 and inertia of a Hermitian matrix, 386, 393 definition, 211 example, 76 field of scalars geometric mean, 99 complex, 19 geometric multiplicity general, 32 ≤ algebraic multiplicity, 206 real, 19 and rank, 186 field of values, see numerical range Conjugate eigenvalues of a real matrix, finite dimensional, see vector space, finite-dimensional 197 Flanders, Harley, 277 definition, 186

416 Index

Gersgorinˇ homogeneous linear system circle, 188 definition, 6 disk, 188 more unknowns than equations, 7 region, 188 nontrivial solution, 6 Semyon Aranovich, 200 trivial solution, 6 sufficient condition for positive semidefinite, 310 two possibilities for solution, 7 theorem, 188 Householder matrix theorem for geometric multiplicity > 1, 199 definition, 134 Goldbach, C., 17 is unitary, Hermitian, involutive, 134 golden ratio, 211 Householder transformation Google matrix, 238, 239 definition, 134 Gram determinant, 164 mapping one given vector to another, 136, 137 Gram matrix hydrogen spectral lines, 200 Cauchy matrix is a, 325 definition, 164 idempotent matrix determinant, 164 definition, 6 in an inner product space, 324 eigenvalues, 198 Gram–Schmidt process invertible, 18 definition, 106 singular values, 365 determine minimal polynomial, 240 trace, 18, 221 linearly dependent list, 120 unitarily similar to a block diagonal matrix, modified, 123 365 Grammian, see Gram matrix unitary similarity of two, 368 unitary similarity to a block upper triangular matrix, Hadamard 234 gate, 127 identity linear transformation, see linear transformation, inequality, 145, 324 identity Hadamard product identity matrix, see matrix, identity definition, 319 ill conditioned, 361 principal submatrix of Kronecker product, 326 inconsistent linear system, see linear system, properties, 319 inconsistent Schur product theorem, 320 independent, see linear independence submultiplicative for Frobenius norm, 367 index of a matrix, 63 submultiplicative for rank, 326 induction hypothesis, see mathematical induction, submultiplicative for spectral norm, 395 induction hypothesis Hadamard, Jacques, 326 inductive step, see mathematical induction, inductive Haynsworth inertia theorem, 393 step Heisenberg uncertainty principle, 71 inertia matrix Heisenberg, Werner, 200 ∗congruence of two, 390 Hermitian axiom, see conjugate symmetry axiom for ∗congruence to a normal matrix, 388 inner product ∗congruent to a Hermitian matrix, 384 Hermitian matrix, see also self-adjoint operator definition, 388 ∗congruence of two, 385, 391 non-normal ∗congruent to an, 395 definition, 5, 112 real, 384 diagonal entries, 16 Sylvester’s theorem, 385 diagonal entries and eigenvalues, 326, 383 inertia theorem product of two, 270 Haynsworth, 393 rank(AB)k vs. rank(BA)k, 301 normal matrix, 390 simultaneous diagonalization by ∗congruence, 324 Sylvester, 385 spectrum, 289 infinite dimensional, see vector space, infinite- unitarily similar to a real tridiagonal matrix, 145 dimensional Hermitian part of a matrix, 290, 326 inner product Hilbert, David, 200 definition, 87 Hilbert–Schmidt norm, 91 derived norm, see norm, derived homogeneity axiom determined by derived norm, 95 derived norm, 92 Frobenius, 88 inner product, 87 in matrix multiplication, 57 norm, 96 L2,88

Index 417

properties, 87 Jordan block standard, 88 bordered matrix in two ways, 246 inner product space, 88 definition, 244 interlacing, see eigenvalue interlacing multiplicity, 244 intersection of subspaces, see subspace, intersection nilpotent, see nilpotent Jordan block intertwining ranks of powers, 246 and commuting, 15 second difference formula for multiplicity, 271 and polynomials, 15 with eigenvalue λ, 244 and similarity, 15 Jordan canonical form definition, 15 algorithm to determine, 255 Fuglede–Putnam theorem, 286 conventional presentation of the, 255 invariant subspace definition, 255 and block upper triangular matrix, 169 Markov matrix, 263 definition, 169 similar to complex conjugate, 269 inverse eigenvalue problems, 396 Jordan form inverse of a matrix definition, 250 2 × 2,4 every square matrix has a, 250 2 × 2 block matrix, 79 Jordan matrix adjugate form, 9 definition, 244 at most one, 4 nilpotent, see nilpotent Jordan matrix block 2 × 2,65 Jordan, Camille, 277 characteristic polynomial, 239 Jordan, Pascual, 200 complex vs. real, 52 jump discontinuity, see function, jump discontinuity definition, 4 is a polynomial in the matrix, 227 kernel Kronecker product, 75 definition, 42 left, 39 linear transformation, 42 left = right, 39, 55 Kronecker delta, 3 product of elementary matrices, 8 Kronecker product reverses products, 5 and vec operator, 75 right, 39 definition, 74 side-by-side method, 59 determinant, 241 upper triangular, 65 eigenvalues, 241 invertible matrix Hadamard product is a principal submatrix, 326 and strict diagonal dominance, 191 identities, 75 columns are a basis, 34 inverse, 75 consistent linear systems, 55 mixed product property, 75, 80 definition, 4 of normal matrices, 302 determinant nonzero, 9 of unitary matrices, 147 is a change of basis matrix, 46 singular value decomposition, 344 square root, 274 singular values, 344 zero is not an eigenvalue, 180 upper triangular matrices, 81 involution Kronecker sum definition, 6 definition, 241 reversal matrix is a symmetric, 128 eigenvalues, 241 singular values, 369 unitary similarity of two, 369 Lagrange interpolation unitary similarity to a block diagonal matrix, 369 real data, 14 unitary similarity to a block upper triangular matrix, theorem, 14 240 unique, 14 involutory matrix, see involution Laplace expansion by minors, see determinant, isometry Laplace expansion and adjoint, 126 law of cosines, 82 and inner product, 126 leading principal minor, 382 definition, 125 leading principal submatrix, see submatrix, leading invertible in finite-dimensional case, 125 principal properties, 125 least squares isomorphism of vector spaces, 41 linear model, 165

418 Index

least squares (cont.) linear transformation minimization, 164 basis representation function, 41 normal equations, 164 commutator, 71 quadratic model, 173 definition, 41 solution to an inconsistent linear system, 164, differentiation, 42 358 identity, 44 left shift induced by a matrix, 41 definition, 42 integration, 42 identities, 245 matrix representation, 43 onto but not one to one, 52 one to one, 42 right inverse=left inverse, 52 one to one if and only if onto, 51 linear combination lists definition, 24 linearly dependent, 28 nontrivial, 24 linearly independent, 29 trivial, 24 maximal linearly independent, 36 linear dependence minimal spanning, 36 and the zero vector, 28 of vectors, 24 basic facts, 28 span of, 24 definition, 28 vs. sets, 24 for sets, 29 lower triangular matrix, see matrix, lower triangular one vector, 28 Lyapunov equation three or more vectors, 28 definition, 321 two vectors, 28 solution, 322 linear functional annihilates commutators, 122 Mn = span{unitaries}, 325 definition, 108 main diagonal, 4 dimension of kernel, 52 majorization Riesz vector, 109 bottom-up, 383, 391 linear independence definition, 383 and extension to a basis, 68 top-down inequalities, 394 and null spaces, 29 Markov matrix and span, 31 definition, 262 and unique representations, 30 eigenvalues with modulus 1, 262 basic facts, 29 Hadamard product of a unitary matrix and its columns extend to an invertible matrix, 68 conjugate, 325 columns of a square matrix, 29 Jordan canonical form, 263 complex exponentials, 29 limit theorem, 263 T definition, 29 limit theorem if A and A are both Markov, 273 for sets, 29 normal, 301 monomials, 29 positive entries, 263 one vector, 29 power bounded, 262 orthonormal system, 104 spectral radius, 262 preserved by invertible matrix, 61 stationary distribution, 264 three or more vectors, 29 mathematical induction two vectors, 29 base case, 11 linear operator, 41 complete, 12 linear regression, see least squares, linear model induction hypothesis, 11, 12 linear system inductive step, 11 coefficients of, 6 principle of, 11 consistent, 6 matrix constant terms, 6 addition, 2 definition, 6 augmented, 7 homogeneous, see homogeneous linear system bidiagonal, 4 inconsistent, 6 block, see block matrix matrix representation, 7 block centrosymmetric, 77 minimum norm solution , see minimum norm block diagonal, 66, 170 solution block upper triangular, 169 solution, 6 bordered, see bordered matrix three possibilities, 6 change of basis, see change of basis matrix

Index 419

circulant, see circulant matrix matrix exponential, see functional calculus, matrix coefficient, 7 exponential commutator, 71 matrix representation of a linear transformation, see complex symmetric, see complex symmetric linear transformation, matrix representation matrix migration patterns, 264, 273 complex type, 77, 217 min matrix convergent, see convergent matrix and the Volterra operator, 343 cyclic permutation, 287 Cholesky factorization, 316, 323 definition, 1 inverse, 316, 323 diagonal, see diagonal matrix positive definite, 316 diagonalizable, see diagonalizable matrix minimal polynomial distinct eigenvalues, 184 basic properties, 228 elementary, see elementary matrices companion matrix, 230 equal, 2 criterion for diagonalizability, 233, 240 Fourier, 129 definition, 229 Google, 238 divides characteristic polynomial, 228 Hermitian, see Hermitian matrix necessary condition for diagonalizability, 231 Hessenberg, see upper Hessenberg matrix of a diagonalizable matrix, 231 idempotent, see idempotent matrix of a direct sum, 241 identity, 3 similarity invariant, 229 inertia, 388 via Gram–Schmidt, 240 invertible, see invertible matrix minimum norm solution involution, see involution and pseudoinverse, 358 lower triangular, 4 consistent linear system, 152 Markov, see Markov matrix definition, 152 min matrix, 316 in column space of adjoint, 152 multiplication, 3, 54, 57, 58 via QR factorization, 171 multiply by a scalar, 2 minor nilpotent, see nilpotent matrix compute determinant via, 9 noninvertible, 4 definition, 9 normal, see normal matrix leading principal, 382 partitioned, see partitioned matrix principal, 382 permutation, see permutation matrix modified Gram–Schmidt process, see Gram–Schmidt positive definite, see positive definite matrix process, modified positive semidefinite, see positive semidefinite matrix modulus of a complex number power bounded, see power bounded matrix definition, 401 powers, 4 multiplicative law, 404, 406 product, see matrix, multiplication modulus of a matrix product of two symmetrics, 265 2 × 2 matrix, 342 real, see real matrix and polar decomposition, 337 real inertia, 384 definition, 336 real orthogonal, see real orthogonal matrix via singular value decomposition, 337 reversal, see reversal matrix momentum operator, see operator, momentum row reduce, 7 Mona Lisa, 347 scalar, 4 monic polynomial, see polynomial, monic skew Hermitian, see skew-Hermitian matrix monotonicity theorem, 394 skew symmetric, see skew-symmetric matrix Moore–Penrose pseudoinverse, see pseudoinverse square, 2 multiplicative law for complex conjugation, 402 strictly lower triangular, 4 multiplicity strictly upper triangular, 4 eigenvalues, 203, 216, 224 symmetric, see symmetric matrix Jordan block, 244 triangular, 4 zeros of a polynomial, 13 tridiagonal, see tridiagonal matrix unispectral, see unispectral matrix negative definite, 304 unitary, see unitary matrix negative semidefinite, 304 upper Hessenberg, see upper Hessenberg matrix nilpotent Jordan block upper triangular, see upper triangular matrix definition, 245 Vandermonde, see Vandermonde matrix index, 246 zero, 2 perturbation of a, 255, 272

420 Index

nilpotent Jordan matrix Hermitian, 289 algebraic multiplicity, 246 idempotent, 299 definition, 245 invariant subspaces same as for adjoint, 298 geometric multiplicity, 246 Markov matrix, 301 index, 246 nilpotent, 299 nilpotent matrix orthogonal eigenvectors, 299 characteristic polynomial, 216, 243 orthogonal projection, 289 definition, 6 polar decomposition, 341 direct sum of, 271 pseudoinverse, 368 eigenvalues, 216, 243 rank AB = rank BA, 301, 325 index-two unitary similarity, 368 real symmetric, see real symmetric matrix minimal polynomial, 243 similarity of two, 292 similar to a nilpotent Jordan matrix, 248 skew Hermitian, 289 square root, 275 spectral resolution, 296 sum or product of, 271 spectral theorem, 282 nonderogatory matrix symmetric, see symmetric normal matrix definition, 274 unitary, 289 similarity and characteristic polynomial, 274 normalization, 92, 98 noninvertible matrix, see matrix, noninvertible normed vector space, 97 nonnegativity axiom null space derived norm, 92 definition, 23 inner product, 87 two matrices have same, 174 norm, 96 nullity, 51, 78 nontrivial solution, see homogeneous linear system, numerical range, 391 nontrivial solution norm off-diagonal entries, 4 definition, 96 one to one derived, 91, 92 definition, 1 Euclidean, 91 linear transformation, 42 Frobenius, 91 onto, 1 Hilbert–Schmidt, 91 operator, see also linear operator L2,91 definition, 41 ℓ1 (absolute sum), 97 momentum, 71 ℓ2,91 position, 71 ℓ∞ (max), 97 orthogonal complement Schur, 91 definition, 149 normal equations of col A∗, 152 conditioning issues, 361 of kernel, 152 definition, 161 of null A, 152 solution via QR factorization, 167 of orthogonal complement, 151 normal matrix of range, 152 2 × 2, 279, 300 properties, 150 2 × 2 block form, 281 orthogonal projection ∗congruence of two, 390 idempotent, 157 adjoint is a polynomial in the matrix, 293 matrix, 167 adjoint is normal, 280 mutually orthogonal, 294 and unitary similarity, 281 of a vector onto a subspace, 155 arbitrary principal submatrix, 299 operator, 156 Cartesian decomposition, 290 properties, 157 characterizations, 293 representation with respect to an orthonormal basis, collinear spectrum, 301 158 commuting family, 284 resolution of the identity, 294 definition, 6, 279 self-adjoint, 157 diagonal entries and eigenvalues, 301 spectrum, 289 direct sum of normal matrices, 281, 302 orthogonality distinct singular values, 341 complex exponentials, 121 eigenvalues and diagonal entries, 325 properties, 90 examples, 279 sets, 90

Index 421

sines and cosines, 91, 117 column vector, 340 vectors, 90 complex symmetric matrix, 367 orthonormal factors commute, 341 basis, 104 factors in a, 337 columns extend to a unitary matrix, 131 left, 338 definition, 102 normal matrix, 341 list extends to an orthonormal basis, 108, 131 right, 338 system, 102 theorem, 337 orthonormalization uniqueness of factors, 338 via Gram–Schmidt, 145 polarization identity, 95, 126 via QR factorization, 142 polynomial outer product and intertwining, 15 definition, 58 and similarity, 16 in matrix multiplication, 58 annihilates a matrix, see annihilating polynomial characteristic, see characteristic polynomial PageRank, 238 coefficient, 12 parallelogram identity complex, 12 characterization of derived norm, 101 constant, 12 derived norm, 92 definition, 12 Euclidean length, 86 degree, 12 parity of a permutation, see permutation, parity division, 13 Parseval’s identity, 114 even, 23 partition function, 271 factor, 13 partitioned matrix functional calculus, see polynomial functional according to its columns, 2, 54 calculus according to its rows, 57 least common multiple, 241 column blocks, 58 location of zeros, 220 conformal partitions, 63 monic, 12 Pauli nonconstant, 12 equation, 127 odd, 23 gates, 127 of a block triangular matrix, 77 spin matrices, 127, 144 of a matrix, 14 Wolfgang, 200 orthogonal, 108 Penrose identities, 367 prime-generating, 17 perfectly conditioned, 361 real period, 116 definition, 12 periodic function, see function, periodic non-real zeros in conjugate pairs, 17 permutation odd degree has a real zero, 17 composition of transpositions, 10 real factors, 17 even, 10 zero, 12 odd, 10 zero of a, 13 of a list, 10 polynomial functional calculus parity, 10 for a normal matrix, 297 sign, 10 for diagonalizable matrices, 210 transposition, 10 position operator, see operator, position permutation matrix positive definite matrix definition, 131 Cholesky factorization is unique, 316 is real orthogonal, 132 column inclusion, 307 permutation similarity definition, 304 definition, 132 determinant positive, 306 of 2 × 2 block diagonal matrices, 133 determinant test, 382 rearranges diagonal entries, 132 geometrical interpretation, 306 rearranges rows and columns, 132 Gram matrix with linearly independent columns, 306 plane rotation matrix inverse of square root, 323 and addition formulae for cosine and sine, 144 leading principal submatrix, 324 definition, 127 monotonicity theorem, 394 polar decomposition must be Hermitian, 304 2 × 2 matrix, 344 positive eigenvalues, 306

422 Index

positive definite matrix (cont.) Pythagorean theorem product of two, 324 classical, 82 simultaneous diagonalization with a Hermitian derived norm, 92 matrix, 318 inner product space, 103 Sylvester criterion, 382 trace positive, 306 QR algorithm, 145 positive semidefinite matrix QR factorization Cholesky factorization, 316 in orthonormalization, 142 column inclusion, 307 narrow, 140 ∗congruence, 309 solution of normal equations, 167 definition, 304 theorem, 139 determinant is monotone, 315 used to compute minimum norm solution, determinant nonnegative, 306 171 direct sums, 309 via Householder transformations, 139 Gram matrix, 304 wide, 140 monotonicity theorem, 394 QS decomposition of a unitary matrix, 291 must be Hermitian, 304 quadratic form nonnegative eigenvalues, 304 change of variables and congruence, 317 polynomial square root, 312 diagonalization, 317 product of two, 314 simultaneous diagonalization, 318 product with a Hermitian, 323 quantum mechanics, 19, 71, 89, 127 square root, 312 quotient, see division algorithm, quotient square root of 2 × 2, 313 trace nonnegative, 306 range unique square root, 312 definition, 1 x∗Ax = 0 ⇒ Ax = 0, 308 linear transformation, 42 positivity axiom rank derived norm, 92 and invertible submatrix, 147 inner product, 87 and nonzero eigenvalues, 209, 238 norm, 96 column, 60 power-bounded matrix definition, 36 definition, 261 dimension of column space, 36 Markov matrix, 262 dimension of row space, 60 primary matrix function, 277 Frobenius inequality, 79 principal minor, 382 full, 60 principal submatrix, see submatrix, principal full column, 60 principle of biorthogonality, 219 full row, 60 principle of mathematical induction, see mathematical Hadamard product, 326 induction, principle of of a product, 61, 64, 79 probability vector, 261 of a sum, 64 product rule for determinants, 10 of powers, 62 projection of one vector onto another of transpose, 8 in an inner product space, 93 perturb one entry, 78 in the plane, 86 rank A = rank AA∗ = rank A∗A, 307 pseudoinverse revealed by reduced row echelon form, 8 and a full-rank factorization, 368 row, 60 and least squares, 358 second differences determine Jordan form, 271 and minimum norm solution, 358 rank-1 projection matrix definition, 356 and Householder matrix, 134 diagonal matrix, 368 definition, 133 normal matrix, 368 Rayleigh Penrose identities, 367 Baron, see Strutt, John William properties, 357, 367 quotient theorem, 372 rank-1 matrix, 368 real matrix solution to a consistent linear system, 359 conjugate eigenvalues, 197, 204 solution to an inconsistent linear system, conjugate eigenvectors, 197 359 odd size, 220 well defined, 356 real eigenvalues have real eigenvectors, 197 purely imaginary number, 398 similarity of two, 196

Index 423

unitary similarity of two, 292 in a vector space, 19 real orthogonal matrix Schur 2 × 2, 144 complement, 70, 76 definition, 6, 127 determinant formula, 70 diagonal, 144 inequality, 285, 351 plane rotation matrices, 127 Issai, 327 real orthogonal similarity majorization of diagonal entries and eigenvalues, 383 definition, 131 norm, 91 equivalence relation, 145 product, see Hadamard product real matrices, 292 product theorem, 320 real symmetric matrix triangularization theorem, 223 orthogonal real eigenvectors, 282 self-adjoint operator, see also Hermitian matrix real orthogonally diagonalizable, 282 definition, 112 reduced row echelon form sesquilinear, 88 definition, 8 sets vs. lists, 24 reveals rank, 8 shift property of similarity, 16, 49 unique, 8 Shoda’s theorem, 73 reduction formula, 70 sign of a permutation, see permutation, sign reflexive relation, 49 similarity remainder, see division algorithm, remainder and change of basis, 44, 48 replacement lemma, 34 and characteristic polynomial, 205 resolution of the identity and eigenvalues, 205 construct from a unitary matrix, and intertwining, see intertwining, and similarity 295, 300 and minimal polynomial, 229 definition, 294 convergent matrices, 259 reversal matrix definition, 48 definition, 128 matrix similar to its transpose, 265 symmetric unitary involution, 128 of two diagonal matrices, 292 Riemann–Lebesgue lemma, 123 of two diagonalizable matrices, 292 Riesz of two normal matrices, 292 representation theorem, 109 of two real matrices, 196 vector of two real normal matrices, 292 definition, 109 permutation, see permutation similarity of a linear functional, 109 preserves determinant, 49 right shift preserves trace, 49 adjoint, 302 real orthogonal, see real orthogonal similarity definition, 42 shift property, 16, 49 invariant subspace, 302 to a block diagonal matrix, 232 is an isometry, 125 to a diagonal matrix, 207 left inverse=right inverse, 52 to a matrix with equal diagonal entries, 73 one to one but not onto, 52 to adjoint, 270 root of a polynomial equation, 13 to conjugate, 269 roots of unity, 406 unitary, see unitary similarity roots vs. zeros, 13 via S or S−1,48 row of a matrix, 2 simultaneous real orthogonal diagonalization, 284 row partition, see partitioned matrix simultaneous unitary diagonalization, 284 row rank, see rank, row simultaneously diagonalizable matrices row reduce, 8 commuting matrices, 209 row space definition, 209 column space of transpose, 59 simultaneously upper triangularizable matrices, 235 definition, 59 singular value decomposition same dimension as column space, 59 column vector, 331 compact, see compact singular value decomposition scalar, 1 complex symmetric matrix, 363 scalar matrix, see matrix, scalar definition, 330 scalar multiplication outer product form, 345 compatible with field multiplication, 20 theorem, 329 distributes over vector addition, 20 truncated, 347 identity element, 20 uniqueness, 342

424 Index

singular values of a vector, 349 AB vs. BA, 343 of inverse, 350 and Frobenius norm, 329 properties, 348 and similarity, 352 self-adjoint, 367 and trace, 353 submultiplicative, 349 and unitary matrices, 352 submultiplicative for Hadamard product, 395 definition, 328 upper bound, 355 distinct, 332 spectral projection, 296 idempotent matrix, 365 spectral radius involution, 369 and spectral norm, 350 multiplicity, 331 definition, 260 of a normal matrix, 351 Markov matrix, 262 of a positive semidefinite matrix, 351 not a norm, 260 of a unitary matrix, 352 spectral resolution, 296 properties, 329 spectral theorem simple, 332 normal matrix, 282, 295 spectral condition number, 359 symmetric normal matrix, 290 spectral norm, 349 symmetric unitary matrix, 290 unitarily invariant, 352 spectrum singular vectors definition, 183 definition, 331 historical remarks, 200 left, 331 nonempty, 181 right, 331 square matrix, see matrix, square skew-Hermitian matrix square root definition, 5 2 × 2 positive semidefinite matrix, 313, 322 diagonal entries, 16 invertible matrix, 274 spectrum, 289 Jordan block, 274 skew-Hermitian part of a matrix, 290 nilpotent matrix, 275 skew-symmetric matrix positive semidefinite matrix, 313 definition, 5 symmetric unitary matrix, 291 diagonal entries, 16 standard basis, see basis, standard solution of a linear system, see linear system, solution standard inner product, see inner product, standard span (noun) stationary distribution, see Markov matrix, stationary and linear independence, 31 distribution definition, 24 strict diagonal dominance is a subspace, 25 definition, 191 is an intersection of subspaces, 31 implies invertibility, 191 is idempotent, 25 sufficient condition for positive definite, 310 of the columns of a matrix, 24 strictly lower triangular matrix, see matrix, strictly lower the whole space, 25 triangular span (verb), 25 strictly upper triangular matrix, see matrix, strictly spanning upper triangular list, 25 Strutt, John William, 372 set, 25 subdiagonal, 4 spectral condition number submatrix definition, 359 definition, 4 properties, 359 eigenvalue interlacing, 381 relative error in solution of a linear system, 360 invertible and rank, 147 unitarily invariant, 360 leading principal, 4 spectral decomposition of a normal matrix principal, 4 definition, 284 trailing principal, 4 recipe, 284 subspace spectral lines of hydrogen, 200 always nonempty, 22 spectral norm criterion, 22 2 × 2 matrix, 366 definition, 22 and Cartesian decomposition, 355 direct sum, 27 block matrices, 370 direct sum of orthogonal complements, 151 definition, 349 intersection, 26, 39

Index 425

invariant, see invariant subspace derived norm, 94 span of union, 26 Euclidean length, 86 subset need not be a, 23 in the plane, 83 sum, 26 norm, 96 union, 26 triangular matrix, 4 zero, 22 tridiagonal matrix sum of subspaces, see subspace, sum definition, 4 superdiagonal, 4 distinct eigenvalues, 395 Sylvester eigenvalue interlacing, 395 criterion for positive definiteness, 382 normal, 301 criterion for positive semidefiniteness, 396 real eigenvalues, 395 determinant identity, 79 trivial solution, see homogeneous linear system, trivial inertia theorem, 385, 391 solution law of nullity, 78 theorem on linear matrix equations, 231 uncertainty principle, see Heisenberg uncertainty Sylvester, James Joseph, 396 principle symmetric matrix union of subspaces, see subspace, union complex, see complex symmetric matrix unispectral matrix definition, 5 definition, 232 product of two, 265 diagonalizable, 233 real, see real symmetric matrix unit ball real vs. complex, 301 convex set, 101 symmetric normal matrix definition, 97 real and imaginary parts, 301 unit vector, 92, 98 spectral theorem, 290 unitarily diagonalizable, 282 symmetric relation, 49 unitary annihilation of lower entries, 138 symmetric unitary matrix unitary equivalence spectral theorem, 291 definition, 367 square root, 291 equivalence relation, 367 system of linear equations, see linear system unitary matrix 1 × 1, 127 tensor product, see Kronecker product ∗congruence of two, 389 Toeplitz matrix 2 × 2, 144 commutes with Jordan block, 276 adjoint is inverse, 127 definition, 276 characterizations, 129 trace CS decomposition, 147 tr AB = tr BA, 6, 80, 120 definition, 6, 127 and singular values, 353 Frobenius norm, 144 definition, 6 orthonormal columns, 129 idempotent matrix, 18, 221 orthonormal rows, 129 is a linear transformation, 52 properties, 128 preserved by similarity, 49 QS decomposition, 291 sum of eigenvalues, 204, 224 real, see real orthogonal matrix zero, see Shoda’s theorem singular values, 352 trailing principal submatrix, see submatrix, trailing spectrum, 289 principal symmetric, see symmetric unitary matrix transitive relation, 49 takes a given vector into a multiple of e1, 138 transpose with given first column, 138 definition, 5 unitary similarity is idempotent, 5 2 × 2 matrices, 273 of a block matrix, 66 definition, 131 of inverses, 5 equivalence relation, 145 reverses products, 5 index-two matrices, 368 similarity of a matrix and its, 265 matrix and its transpose, 274 transposition, see permutation, transposition of two idempotent matrices, 368 triangle inequality of two involutions, 369 case of equality, 405 of two real matrices, 292 complex numbers, 404 to a block upper triangular matrix, 232

426 Index

unitary similarity (cont.) real-valued functions, 22 to a diagonal matrix, 282 rectangular matrices, 21 to an upper Hessenberg matrix, 143 zero, 21 to an upper triangular matrix, 223 zero dimensional, 35 upper Hessenberg matrix Volterra operator definition, 142 adjoint, 121 unitary similarity to a, 143 Cartesian decomposition, 300 upper triangular matrix definition, 121, 199 definition, 4 eigenpairs of, 199 determinant, 203 volume and determinant, 10 eigenvalues, 203 Schur’s theorem, 223 Weierstrass approximation theorem, 174 well conditioned, 361 Vandermonde determinant, see Vandermonde matrix, Weyl inequalities determinant for eigenvalues, 386 Vandermonde matrix imply additive interlacing, 388 determinant, 16 Weyr characteristic invertible, 16, 17 and algebraic multiplicity, 272 vec operator and the Jordan canonical form, 255 definition, 75 associated with an eigenvalue, 253 of a triple product, 75 definition, 253 vector addition monotone decreasing, 253 additive inverse, 20 Weyr matrix associative, 20 definition, 277 commutative, 20 matrix commutes with a, 276 distributes over scalar multiplication, 20 Weyr, Eduard, 277 identity element, 20 in a vector space, 19 zero dimensional, see vector space, zero-dimensional vector space zero matrix, see matrix, zero column vectors, 21 zero of a polynomial, see polynomial, zero of a complex, 20 zero polynomial, see polynomial, zero complex-valued functions, 22 zero vector definition, 19 axiom, 20 dimension, see dimension column vectors, 21 finite-dimensional, 35 complex-valued functions, 22 finitely nonzero sequences, 22 finitely nonzero sequences, 22 four things in harmony, 19 polynomials, 21 infinite dimensional, 35 real-valued functions, 22 nonzero, 21 rectangular matrices, 21 polynomials, 21 zeros vs. roots, 13 real, 20