Index

Bauer–Fike bound, 528 A beads on a string, 559 Bellman, Richard, xii absolute uncertainty or error, 414 Beltrami, Eugenio, 411 absorbing Markov chains, 700 Benzi, Michele, xii absorbing states, 700 Bernoulli, Daniel, 299 addition, properties of, 82 Bessel, Friedrich W., 305 additive identity, 82 Bessel’s inequality, 305 additive inverse, 82 best approximate inverse, 428 adjacency matrix, 100 biased estimator, 446 adjoint, 84, 479 binary representation, 372 adjugate, 479 Birkhoff, Garrett, 625 affine functions, 89 Birkhoff’s theorem, 702 affine space, 436 bit reversal, 372 algebraic group, 402 bit reversing permutation matrix, 381 algebraic multiplicity, 496, 510 block diagonal, 261–263 amplitude, 362 rank of, 137 Anderson-Duffin formula, 441 using real arithmetic, 524 Anderson, Jean, xii block matrices, 111 Anderson, W. N., Jr., 441 determinant of, 475 angle, 295 linear operators, 392 canonical, 455 block matrixmultiplication, 111 between complementary spaces, 389, 450 block triangular, 112, 261–263 maximal, 455 determinants, 467 principal, 456 eigenvalues of, 501 between subspaces, 450 Bolzano–Weierstrass theorem, 670 annihilating polynomials, 642 Boolean matrix, 679 aperiodic Markov chain, 694 bordered matrix, 485 Arnoldi’s algorithm, 653 eigenvalues of, 552 Arnoldi, Walter Edwin, 653 branch, 73, 204 associative property Brauer, Alfred, 497 of addition, 82 Bunyakovskii, Victor, 271 of matrixmultiplication, 105 of scalar multiplication, 83 C asymptotic rate of convergence, 621 augmented matrix, 7 cancellation law, 97 Autonne, L., 411 canonical angles, 455 canonical form, reducible matrix, 695 B Cantor, Georg, 597 Cauchy, Augustin-Louis, 271 back substitution, 6, 9 determinant formula, 484 backward error analysis, 26 integral formula, 611 backward triangle inequality, 273 Cauchy–Bunyakovskii–Schwarz inequality, 272 band matrix, 157 Cauchy–Goursat theorem, 615 Bartlett, M. S., 124 Cauchy–Schwarz inequality, 287 base-b, 375 Cayley, Arthur, 80, 93, 158, 460 basic columns, 45, 61, 178, 218 Cayley–Hamilton theorem, 509, 532, 597 combinations of, 54 to determine f(A) , 614 basic variables, 58, 61 Cayley transformation, 336, 556 in nonhomogeneous systems, 70 CBS inequality, 272, 277, 473 basis, 194, 196 general form, 287 change of, 253 centered difference approximations, 19 characterizations, 195 Ces`aro, Ernesto, 630 orthonormal, 355 Ces`aro sequence, 630 basis for Ces`aro summability, 630, 633, 677 direct sum, 383 for stochastic matrix, 697 intersection of spaces, 211 chain, Jordan, 575 space of linear transformations, 241 change of basis, 251, 253, 258 706 Index change of coordinates, 252 complexnumbers, the set of, 81 characteristic equation, 491, 492 component matrices, 604 coefficients, 494, 504 component vectors, 384 characteristic polynomial, 491, 492 composition of a product, 503 of linear functions, 93 characteristic values and vectors, 490 of linear transformations, 245, 246 Chebyshev, Pafnuty Lvovich, 40, 687 of matrixfunctions, 608, 615 checking an answer, 35, 416 computer graphics, 328, 330 Cholesky, Andre-Louis, 154 condition number Cholesky factorization, 154, 314, 558, 559 for eigenvalues, 528 Cimmino, Gianfranco, 445 generalized, 426 Cimmino’s reflection method, 445 for matrices, 127, 128, 414, 415 circuit, 204 condition of circulant matrix, 379 eigenvalues, hermitian matrices, 552 with convolution, 380 linear system, 128 eigenvalues, eigenvectors, 523 conditioning and pivots, 426 classical Gram–Schmidt algorithm, 309 conformable, 96 classical least squares, 226 conformably partitioned, 111 clock cycles, 539, 694 congruence transformation, 568 closest point conjugate, complex, 83 to an affine space, 436 conjugate gradient algorithm, 657 with Fourier expansion, 440 conjugate matrix, 84 theorem, 435 conjugate transpose, 84 closure property reverse order law, 109 of addition, 82, 160 conjugate vectors, 657 of multiplication, 83, 160 connected graph, 202 coefficient matrix, 7, 42 connectivity and linear dependence, 208 coefficient of linear correlation, 297 connectivity matrix, 100 cofactor, 477, 487 consistent system, 53, 54 expansion, 478, 481 constituent matrices, 604 Collatz, Lothar, 666 continuity Collatz–Wielandt formula, 666, 673, 686 of eigenvalues, 497 column, 7 of inversion, 480 equivalence, 134 of norms, 277 and nullspace, 177 continuous Fourier transform, 357 operations, 14, 134 continuous functions, maxand min, 276 rank, 198 convergence, 276, 277 relationships, 50, 136 convergent matrix, 631 scaling, 27 converse of a statement, 54 space, 170, 171, 178 convolution spanning set for, 172 with circulants, 380 vector, 8, 81 definition, 366 Comdico, David, xii operation count, 377 commutative law, 97 theorem, 367, 368, 377 commutative property of addition, 82 Cooley, J. W., 368, 375, 651 commuting matrices, eigenvectors, 503, 522 cooperating species model, 546 companion matrix, 648 coordinate matrix, 242 compatibility of norms, 285 coordinates, 207, 240, 299 compatible norms, 279, 280 change of, 252 competing species model, 546 of a vector, 240 complementary projector, 386 coordinate spaces, 161 complementary subspaces, 383, 403 core-nilpotent decomposition, 397 angle between, 389, 450 correlation, 296 complete pivoting, 28 correlation coefficient, 297 numerical stability, 349 cosine complete set of eigenvectors, 507 of angle, 295 complexconjugate, 83 minimal angle, 450 complexexponential, 362, 544 discrete, 361 Index 707

Courant–Fischer theorem, 550 diffusion equation, 563 alternate, 557 diffusion model, 542 for singular values, 555 dimension, 196 Courant, Richard, 550 of direct sum, 383 covariance, 447 of fundamental subspaces, 199 Cramer, Gabriel, 476 of left-hand nullspace, 218 Cramer’s rule, 459, 476 critical point, 570 of nullspace, 218 cross product, 332, 339 of orthogonal complement, 339 Cuomo, Kelly, xii of range, 218 curve fitting, 186, 229 of row space, 218 of space of linear transformations, 241 D of subspace, 198 of sum, 205 defective, 507 direct product, 380, 597 deficient, 496, 507 direct sum, 383 definite matrices, 559 of linear operators, 399 deflation, eigenvalue problems, 516 of several subspaces, 392 dense matrix, 350 of symmetric and skew-symmetric matrices, 391 dependent set, 181 directed distance between subspaces, 453 derivative of a determinant, 471, 474, 486 directed graph, 202 of a linear system, 130 Dirichlet, Johann P. G. L, 563, 597 of a matrix, 103, 226 Dirichlet problem, 563 operator, 245 discrete Fourier transform, 356, 358 determinant, 461 discrete Laplacian, 563 computing, 470 eigenvalues of, 598 of a product, 467 discrete sine, cosine, and exponential, 361 as product of eigenvalues, 494 disjoint subspaces, 383 of a sum, 485 distance, 271 and volume, 468 to lower-rank matrices, 417 deviation from symmetry, 436 between subspaces, 450 diagonal dominance, 639 to symmetric matrices, 436 diagonal matrix, 85 between a vector and a subspace, 435 eigenvalues of, 501 inverse of, 122 distinct eigenvalues, 514 diagonalizability, 507 distributions, 532 being arbitrarily close to, 533 distributive property in terms of minimum polynomial, 645 of matrixmultiplication, 105 in terms of multiplicities, 512 of scalar multiplication, 83 summary, 520 domain, 89 diagonalization doubly stochastic, 702 of circulants, 379 Drazin generalized inverse, 399, 401, 422, 640 Jacobi’s method, 353 Cauchy formula, 615 of normal matrices, 547 integral representation, 441 simultaneous, 522 Drazin, M. P., 399 diagonally dominant, 184, 499, 622, 623, 639 Duffin, R. J., 441 systems, 193 Duncan, W. J., 124 difference equations, 515, 616 difference of matrices, 82 E difference of projectors, 393 differential equations, 489, 541, 542 independent solutions, 481 Eckart, C., 411 nonhomogeneous, 609 economic input–output model, 681 solution of, 546 edge matrix, 331 stability, 544, 609 edges, 202 systems, 608 eigenpair, 490 uncoupling, 559 eigenspace, 490 708 Index eigenvalues, 266, 410, 490 extending to a basis, 201 bounds for, 498 extending to an orthonormal basis, 325, 335, 338, 404 continuity of, 497 extension set, 188 determinant and trace, 494 distinct, 514 F generalized, 571 indexof, 401, 587, 596 Faddeev and Sominskii, 504 perturbations and condition of, 528, 551 fail-safe system, 701 semisimple, 596 fast Fourier transform (FFT), 368 sensitivity, hermitian matrices, 552 FFT algorithm, 368, 370, 373, 381, 651 unit, 696 FFT operation count, 377 eigenvalues of fast integer multiplication, 375 bordered matrices, 552 filtering random noise, 418 discrete Laplacian, 566, 598 finite difference matrix, 522, 639 triangular and diagonal matrices, 501 finite-dimensional spaces, 195 tridiagonal Toeplitz matrices, 514 finite group, 676 eigenvectors, 266, 490 Fischer, Ernst, 550 of commuting matrices, 503 five-point difference equations, 564 generalized, 593, 594 fixed points, 386, 391 independent, 511 of a reflector, 338 of tridiagonal Toeplitz matrices, 514 flatness, 164 electrical circuits, 73, 204 floating-point number, 21 elementary matrices, 131–133 forward substitution, 145 interchange matrices, 135, 140 four fundamental subspaces, 169 elementary orthogonal projector, 322, 431 summary, 178 rank of, 337 Fourier coefficients, 299 elementary reflector, 324, 444 Fourier expansion, 299 determinant of, 485 and projection, 440 elementary row and column operations, 4, 8 Fourier, Jean Baptiste Joseph, 299 and determinants, 463 Fourier matrix, 357 elementary triangular matrix, 142 Fourier series, 299, 300 ellipsoid, 414 Fourier transform continuous, 357 degenerate, 425 discrete, 356, 358 elliptical inner product, 286 Frame, J. S., 504 elliptical norm, 288 Francis, J. F. G., 535 EP matrices, 408 Fr´echet, Maurice, R., 289 equal matrices, 81 free variables, 58, 61 equivalence, row and column, 134 in nonhomogeneous systems, 70 testing for, 137 frequency, 362 equivalent norms frequency domain, 363 matrices, 425 Frobenius, Ferdinand Georg, 44, 123, 215, 662 vectors, 276 Frobenius form, 680 equivalent statements, 54 Frobenius inequality, 221 equivalent systems, 3 Frobenius matrixnorm, 279, 425, 428 ergodic class, 695 and inner product, 288 error, absolute and relative, 414 of rank-one matrices, 391 essentially positive matrix, 686 unitary invariance of, 337 estimators, 446 Frobenius test for primitivity, 678 euclidean norm, 270 full-rank factorization, 140, 221, 633 unitary invariance of, 321 for determining index, 640 evolutionary processes, 616 of a projector, 393 exponential function complex, 544 affine, 89 discrete, 361 composition of, 93, 615, 608 matrix, 441, 525 domain of, 89 inverse of, 614 linear, 89, 238 products of, 539 norm of, 288 sums of, 614 range of, 89 Index 709 functional matrixidentities, 608 GMRES, 655 functions of Golub, Gene H., xii diagonalizable matrices, 526 gradient, 570 of Jordan blocks, 600 Gram, Jorgen P., 307 matrices, 601 Gram matrix, 307 using Cauchy integral formula, 611 Gram–Schmidt algorithm using Cayley–Hamilton theorem, 614 classical version, 309 nondiagonalizable matrices, 603 implementations of, 319 fundamental mode of vibration, 562 and minimum polynomial, 643 fundamental problem of matrixtheory, 506 modified version, 316 fundamental subspaces, 169 numerical stability of, 349 dimension of, 199 and volume, 431 orthonormal bases for, 407 Gram–Schmidt process, 345 projector onto, 434 Gram–Schmidt sequence, 308, 309 fundamental theorem of algebra, 185, 492 graph, 202 fundamental theorem of linear algebra, 405 of a matrix, 209, 671 graphics, 3-D rotations, 328, 330 G Grassmann, Hermann G., 160 Graybill, Franklin A., xii gap, 453, 454 grid norm, 274 Gauss, Carl F., ix, 2, 93, 234, 488 grid points, 18 as a teacher, 353 group, finite, 676 Gaussian elimination, 2, 3 group inverse, 402, 640, 641 and LU factorization, 141 growth in Gaussian elimination, 26 effects of roundoff, 129 Guttman, L., 124 modified, 43 numerical stability, 348 H operation counts, 10 Gaussian transformation, 341 Hadamard, Jacques, 469, 497 Gauss–Jordan method, 15, 47, 48 Hadamard’s inequality, 469 for computing a matrixinverse, 118 Halmos, Paul, xii, 268 operation counts, 16 Hamilton, William R., 509 Gauss–Markov theorem, 229, 448 harmonic functions, 563 Gauss–Seidel method, 622 Haynsworth, Emilie V., 123 general solution heat equation, 563 algebraic equations Helfrich, Laura, xii homogeneous systems, 59, 61, Hermite, Charles, 48 nonhomogeneous systems, 64, 66, 70, 180, 221 Hermite interpolation polynomial, 607 difference equations, 616 Hermite normal form, 48 differential equations, 541, 609 Hermite polynomial, 231 generalized condition number, 426 hermitian matrix, 85, 409, 410 generalized eigenvalue problem, 571 condition of eigenvalues, 552 generalized eigenvectors, 593, 594 eigen components of, 549 generalized inverse, 221, 393, 422, 615 Hessenberg matrices 350 Drazin, 399 QR factorization of, 352 group, 402 Hessian matrix, 570 and orthogonal projectors, 434 Hestenes, Magnus R., 656 generalized minimal residual (GMRES), 655 hidden surfaces, 332, 339 genes and chromosomes, 543 Hilbert, David, 307 geometric multiplicity, 510 Hilbert matrix, 14, 31, 39 geometric series, 126, 527, 618 Hilbert–Schmidt norm, 279 Gerschgorin circles, 498 Hohn, Franz, xii Gerschgorin, S. A., 497 H¨older, Ludwig O., 278 Givens reduction, 344 H¨older’s inequality, 274, 277, 278 and determinants, 485 homogeneous systems, 57, 61 numerical stability, 349 Hooke, Robert, 86 Givens rotations, 333 Hooke’s law, 86 Givens, Wallace, 333 Horn, Roger, xii 710 Index

Horst, Paul, 504 integer multiplication, 375 Householder, Alston S., 324 integral formula Householder reduction, 341, 342 for generalized inverses, 441 and determinants, 485 for matrixfunctions, 611 and fundamental subspaces, 407 intercept model, 447 numerical stability, 349 interchange matrices, 135, 140 Householder transformations, 324 interlacing of eigenvalues, 552 hyperplane, 442 interpolation formula for f(A), 529 I Hermite polynomial, 607 Lagrange polynomial, 186 idempotent, 113, 258, 339, 386 intersection of subspaces and projectors, 387 basis for, 211 identity matrix, 106 projection onto, 441 identity operator, 238 invariant subspace, 259, 262, 263 ill-conditioned matrix, 127, 128, 415 inverse Fourier transform, 358 ill-conditioned system, 33, 535 inverse iteration, 534 normal equations, 214 inverse matrix, 115 image and image space, 168, 170 best approximation to, 428 dimension of, 208 Cauchy formula for, 615 image of unit sphere, 417 computation of, 118 imaginary, pure, 556 operation counts, 119 imprimitive matrices, 674 continuity of, 480 maximal root of, 676 determinants, 479 spectrum of, 677 eigenvalues of, 501 test for, 678 existence of, 116 imprimitivity, indexof, 679, 680 generalized, 615 incidence matrix, 202 integral representation of, 441 inconsistent system, 53 norm of, 285 independent columns, 218 properties of, 120 independent eigenvectors, 511 of a sum, 220 independent rows, 218 invertible operators, 246, 250 independent set, 181 invertible part of an operator, 399 basic facts, 188 involutory, 113, 325, 339, 485 maximal, 186 irreducible Markov chain, limits, 693 independent solutions irreducible matrix, 209, 671 for algebraic equations, 209 isometry, 321 for differential equations, 481 iteration matrix, 620 index iterative methods, 620 of an eigenvalue, 401, 587, 596 of imprimitivity, 674, 679, 680 J of nilpotency, 396 of a square matrix, 394, 395 Jacobi’s diagonalization method, 353 by full-rank factorization, 640 Jacobi’s iterative method, 622, 626 induced matrixnorm, 280, 389 Jacobi, Karl G. J., 353 − of A 1, 285 Johnson, Charlie, xii elementary properties, 285 Jordan blocks, 588, 590 of rank-one matrices, 391 functions of, 600 unitary invariance of, 337 nilpotent, 579 inertia, 568 Jordan chains, 210, 401, 575, 576, 593 infinite-dimensional spaces, 195 construction of, 594 infinite series and matrixfunctions, 527 Jordan form, 397, 408, 589, 590 infinite series of matrices, 605 for nilpotent matrices, 579 information retrieval, 419 preliminary version, 397 inner product, 286 Jordan, Marie Ennemond Camille, 15, 411, 589 geometric interpretation, 431 Jordan segment, 588, 590 input–output economic model, 681 Jordan structure of matrices, 580, 581, 586, 589 integer matrices, 156, 473, 485 uniqueness of, 580 Index 711

Jordan, Wilhelm, 15 Legendre’s differential equation, 319 Leibniz, Gottfried W., 459 K length of a projection, 323 Leontief’s input–output model, 681 Kaczmarz’s projection method, 442, 443 Leontief, Wassily, 681 Kaczmarz, Stefan, 442 Leslie, P. H., 684 Kaplansky, Irving, 268 Leslie population model, 683 Kearn, Vickie, xi, 12 Leverrier–Souriau–Frame Algorithm, 504 kernel, 173 Kirchhoff’s rules, 73 Leverrier, U. J. J., 504 loop rule, 204 L´evy, L., 497 Kline, Morris, 80 limiting distribution, 531, 636 Kowa, Seki, 459 limits Kronecker, Leopold, 597 and group inversion, 640 Kronecker product, 380, 597 in Markov chains and the Laplacian, 573 irreducible Markov chains, 693 Krylov, Aleksei Nikolaevich, 645 reducible Markov chains, 698 Krylov of powers of matrices, 630 method, 649 and spectral radius, 617 sequence, 401 of vector sequences, 639 subspaces, sequences, matrices, 646 in vector spaces, 276, 277 Kummer, Ernst Eduard, 597 Lindemann, Carl Louis Ferdinand von, 662 linear L algebra, 238 combination, 91 Lagrange interpolating polynomial, 186, 230, 233, 529 correlation, 296, 306 Lagrange, Joseph-Louis, 186, 572 dependence and connectivity, 208 Lagrange multipliers, 282 estimation, 446 Lancaster, Peter, xii functions, 89, 238 Lanczos algorithm, 651 defined by matrixmultiplication, 106 Lanczos, Cornelius, 651 Laplace’s determinant expansion, 487 defined by systems of equations, 99 Laplace’s equation, 624 models, 448 Laplace, Pierre-Simon, 81, 307, 487, 572 operators, 238 Laplacian, 563 and block matrices, 392 latent semantic indexing, 419 regression, 227, 446 latent values and vectors, 490 spaces, 169 law of cosines, 295 stationary iterations, 620 LDU factorization, 154 transformation, 238 leading principal minor, 558 linearly independent and dependent sets, 181 leading principal submatrices, 148, 156 basic facts, 188 least common multiple, 647 maximal, 186 least squares, 226, 439 and rank, 183 and Gram–Schmidt, 313 linearly independent eigenvectors, 511 and orthogonal projection, 437 lines in n not through the origin, 440 and polynomial fitting, 230 lines, projection onto, 440 and pseudoinverse, 438 long-run distribution, 531 and QR factorization, 346 loop, 73 total least squares, 223 equations, 204 why least squares?, 446 LeBlanc, Kathleen, xii rule, 74 left-hand eigenvectors, 490, 503, 516, 523, 524 simple, 75 in inverses, 521 lower triangular, 103 and projectors, 518 LU factorization, 141, 144 left-hand nullspace, 174, 178, 199 existence of, 149 spanning set for, 176 with interchanges, 148 Legendre, Adrien–Marie, 319, 572 operation counts, 146 Legendre polynomials, 319 summary, 153 712 Index

minimum polynomial, 642 M determination of, 643 of a vector, 646 main diagonal, 41, 85 minimum variance estimator, 446 Markov, Andrei Andreyevich, 687 Minkowski, Hermann, 184, 278, 497, 626 Markov chains, 532, 638, 687 Minkowski inequality, 278 absorbing, 700 minor determinant, principal, 559, 466 periodic, 694 MINRES algorithm, 656 mass-stiffness equation, 571 Mirsky, Leonid, xii matrices, the set of, 81 M-matrix, 626, 639, 682, 703 matrix, 7 modern least squares, 437 diagonal 85 modified gaussian elimination, 43 exponential, 441, 525, 529 modified Gram–Schmidt algorithm, 316 and differential equations, 541, 546, 608 monic polynomial, 642 inverse of, 614 Montgomery, Michelle, xii products, 539 Moore, E. H., 221 sums, 614 Moore–Penrose generalized inverse, 221, 422, 400 functions, 526, 601 best approximate inverse, 428 as infinite series, 527 integral representation, 441 as polynomials, 606 and orthogonal projectors, 434 group, 402 Morrison, W. J., 124 multiplication, 96 multiplication by blocks, 111 of integers, 375 as a linear function, 106 of matrices, 96 properties of, 105 of polynomials, 367 relation to linear transformations, 244 multiplicities, 510 norms, 280 and diagonalizability, 512 1-norm, 283 multiplier, 22, 25 2-norm, 281, 425 in partial pivoting, 26 ∞-norm, 127, 283 Frobenius norm, 425 N induced norm, 285 polynomials, 501 negative definite, 570 product, 96 Neumann series, 126, 527, 618 representation of a projector, 387 Newton, 86 representations, 262 Newton’s identities, 504 triangular 41 Newton’s second law, 560 maximal angle, 455 nilpotent, 258, 396, 502, 510 maximal independent set, 218 Jordan blocks, 579 maximal linearly independent subset, 186, 196 part of an operator, 399 maximum and minimum of continuous functions, 276 Noble, Ben, xii McCarthy, Joseph R., 651 node, 18, 73, 202, 204 mean, 296, 447 rule, 74, 204 Meyer no-intercept model, 447 Bethany B., xii noise removal with SVD, 418 nonbasic columns, 50, 61 Carl, Sr., xii nonderogatory matrices, 644, 648 Holly F., xii nondiagonalizable, spectral resolution, 603 Louise, xii nonhomogeneous differential equations, 609 Margaret E., xii nonhomogeneous systems, 57, 64 Martin D., xii general solution, 64, 66, 70 min-maxtheorem, 550 summary, 70 alternate formulation, 557 nonnegative matrices, 661, 670 for singular values, 555 nonsingular matrices, 115 minimal angle, 450 and determinants, 465 minimal spanning set, 196, 197 and elementary matrices, 133 minimum norm least squares solution, 438 products of, 121 minimum norm solution, 426 sequences of, 220 Index 713 norm, 269 orthogonal projection, 239, 243, 248, 299, 305, 385, 429 compatibility, 279, 280, 285 and 3-D graphics, 330 elliptical, 288 onto an affine space, 436 equivalent, 276, 425 and least squares, 437 of a function, 288 orthogonal projectors, 322, 410, 427, 429 on a grid, 274 elementary, 431 of an inverse, 285 formulas for, 430 for matrices, 280 onto an intersection, 441 1-, 2-, and ∞-norms, 281, 283 and pseudoinverses, 434 Frobenius, 279, 337 sums of, 441 induced, 280, 285, 337 orthogonal reduction, 341 of a projection, 323 to determine full-rank factorization, 633 for vectors, 275 to determine fundamental subspaces, 407 1-, 2-, and ∞-norms, 274 orthogonal triangularization, 342 p-norms, 274 orthogonal vectors, 294 of a waveform, 382 orthonormal basis, 298 normal equations, 213, 214, 221, 226, 313, 437 extending to, 325, 335, 38 normalized vector, 270 for fundamental subspaces, 407 normal matrix, 304, 400, 409, 547 by means of orthogonal reduction, 355 nullity, 200, 220 orthonormal set, 298 nullspace, 173, 174, 178, 199 Ostrowski, Alexander, 626 equality, 177 outer product, 103 of an orthogonal projector, 434 overrelaxation, 624 of a partitioned matrix, 208 P of a product, 180, 220 spanning set for, 175 Painter, Richard J., xii and transpose, 177 parallelepiped, 431, 468 number of pivots, 218 parallelogram identity, 290, 291 parallelogram law, 162 numerical stability, 347 parallel sum, 441 O parity of a permutation, 460 Parseval des Chˆenes, M., 305 Parseval’s identity, 305 oblique projection, 385 partial pivoting, 24 method for linear systems, 443 and diagonal dominance, 193 oblique projectors from SVD, 634 and LU factorization, 148 Ohm’s law, 73 and numerical stability, 349 p Oh notation O(h ), 18 particular solution, 58, 65–68, 70, 180, 213 one-to-one mapping, 250 partitioned matrix, 111 onto mapping, 250 and linear operators, 392 operation counts rank and nullity of, 208 for convolution, 377 Peano, Giuseppe, 160 for Gaussian elimination, 10 Penrose equations, 422 for Gauss–Jordan method, 16 Penrose, Roger, 221 for LU factorization, 146 perfect shuffle, 372, 381 for matrixinversion, 119 period of trigonometric functions, 362 operator, linear, 238 periodic extension, 302 operator norm, 280 periodic function, 301 periodic Markov chain, 694 Ortega, James, xii permutation, 460 orthogonal complement, 322, 403 symmetric, 671 dimension of, 339 permutation counter, 151 involving range and nullspace, 405 permutation matrix, 135, 140, 151 orthogonal decomposition theorem, 405, 407 perpendicular, 294 orthogonal diagonalization, 549 perp, properties of, 404, 409 orthogonal distance, 435 Perron–Frobenius theory, 661, 673 orthogonal matrix, 320 Perron, Oskar, 661 determinant of, 473 Perron root, 666, 668 714 Index

Perron vector, 665, 668, 673 principal submatrix, 494, 558 perturbations and interlaced eigenvalues, 553 affecting rank, 216 of an M-matrix, 626 eigenvalues, 528 of a stochastic, 703 hermitian eigenvalues, 551 products in inverses, 128 of matrices, 96 in linear systems, 33, 128, 217 of nonsingular matrices, 121 rank-one update, 208 of orthogonal projectors, 441 singular values, 421 of projectors, 393 Piazzi, Giuseppe, 233 product rule for determinants, 467 pivot projection, 92, 94, 322, 385, 429 conditioning, 426 and Fourier expansion, 440 determinant formula for, 474, 558 method for solving linear systems, 442, 443 elements and equations, 5 onto positions, 5, 58, 61 affine spaces, 436 in partial pivoting, 24 fundamental subspaces, 434 uniqueness, 44 hyperplanes, 442 pivoting lines, 440, 431 complete, 28 oblique subspaces, 385 partial, 24 orthogonal subspaces, 429 plane rotation, 333 symmetric matrices, 436 determinant of, 485 projectors, 239, 243, 339, 385, 386 p-norm, 274 complementary, 386 Poisson’s equation, 563, 572 from core-nilpotent decomposition, 398 Poisson, Sim´eon D., 78, 572 difference of, 393 polar factorization, 572 from full-rank factorization, 633, 634 polarization identity, 293 as idempotents, 387 polynomial induced norm of, 389 equations, 493 matrixrepresentation of, 387 in a matrix, 501 oblique, 386 and matrixfunctions, 606 orthogonal, 429 minimum, 642 product of, 393 multiplication and convolution, 367 spectral, 517, 603 polytope, 330, 339 sum of, 393 ponderal index, 236 proper values and vectors, 490 poor man’s root finder, 649 pseudoinverse, 221, 422, 615 population distribution, 532 as best approximate inverse, 428 population migration, 531 Drazin, 399 population model, Leslie, 683 group, 402 positive definite form, 567 inner, outer, reflexive, 393 positive definite matrix, 154, 474, 558, 559 integral representation of, 441, 615 positive matrix, 661, 663 and least squares, 438 positive semidefinite matrix, 558, 566 Moore–Penrose, 422 Poulson, Deborah , xii and orthogonal projectors, 434 power method, 532, 533 pure imaginary, 556 powers of a matrix, 107 Pythagorean theorem, 294, 305, 423 limiting values, 530 and closest point theorem, 435 powers of linear transformations, 248 for matrices with Frobenius norm, 428 precision, 21 preconditioned system, 658 Q predator–prey model, 544 primitive matrices, 674 QR factorization, 345, 535 test for, 678 and Hessenberg matrices, 352 principal angles, 456 and least squares, 346 principal minors, 494, 558 and minimum polynomial, 643 in an M-matrix, 626, 639 rectangular version of, 311 nonnegative, 566 and volume, 431 positive, 559 quadratic form, 567 Index 715

reducible matrices, 209, 671 quaternions, 509 canonical form for, 695 R in linear systems, 112 reflection, 92, 94 random integer matrices, 156 about a hyperplane, 445 random walk, 638 method for solving linear systems, 445 range reflector, 239, 324, 444 of a function, 89, 169 determinant of, 485 of a matrix, 170, 171, 178, 199 reflexive pseudoinverse, 393 of an operator, 250 regression, 227, 446 of an orthogonal projector, 434 relative uncertainty or error, 414 of a partitioned matrix, 179 relaxation parameter, 445, 624 of a product, 180, 220 residual, 36, 416 of a projector, 391 resolvent, 285, 611 of a sum, 206 restricted operators, 259, 393, 399 range-nullspace decomposition, 394, 407 restricted transformations, 424 range-symmetric matrices, 408 reversal matrix, 596 rank, 45, 139 reverse order law of a block diagonal matrix, 137 for inversion, 120, 121 and consistency, 54 for transpose and conjugate transpose, 109 and determinants, 466 reversing binary bits, 372 of a difference, 208 Richardson iterative method, 622 of an elementary projector, 337 right angle, 294 and free variables, 61 right-hand rule, 340 of an incidence matrix, 203 right-hand side, 3 and independent sets, 183 Ritz values, 651 and matrixinverses, 116 roots of unity, 356 and nonhomogeneous systems, 70 and imprimitive matrices, 676 and nonsingular submatrices, 218 Rose, Nick, xii numerical determination, 421 rotation, 92, 94 of a partitioned matrix, 208 determinant of, 485 of a perturbed matrix, 216 plane (Givens rotations), 333 of a product, 210, 211, 219 in 2, 326 of a projector, 392 in 3, 328 and submatrices, 215 in n, 334 of a sum, 206, 221 rotator, 239, 326 summary, 218 rounding convention, 21 and trivial nullspaces, 175 roundoff error, 21, 129, 347 rank normal form, 136 row, 7 rank-one matrices echelon form, 44 characterization of, 140 reduced, 48 diagonalizability of, 522 equivalence, 134, 218 perturbations of, 208 and nullspace, 177 rank-one updates operations, 134 determinants of, 475 rank, 198 eigenvalues of, 503 relationships, 136 rank plus nullity theorem, 199, 410 scaling, 27 Rayleigh, Lord, 550 space, 170, 171, 178, 199 Rayleigh quotient, 550 spanning set for, 172 iteration, 535 vector, 8, 81 real numbers, the set of, 81 RPN matrices, 408 real Schur form, 524 Rutishauser, Heinz, 535 real-symmetric matrix, 409, 410 rectangular matrix, 8 S rectangular QR factorization, 311 rectangular systems, 41 Saad, Yousef, 655 reduced row echelon form, 48 saw-toothed function, 306 reducible Markov chain, 698 scalar, 7, 81 716 Index

singular values, 553 scalar multiplication, 82, 83 Courant–Fischer theorem, 555 scale, 27 and determinants, 473 scaling a linear system, 27, 28 as eigenvalues, 555 scaling in 3-D graphics, 332 and the SVD, 412 Schmidt, Erhard, 307 size, 8 skew-hermitian matrices, 85, 88 Schr¨odinger, Erwin, 651 skew-symmetric matrices, 85, 88, 391, 473 Schultz, Martin H., 655 eigenvalues of, 549, 556 Schur complements, 123, 475 as exponentials, 539 Schur form for real matrices, 524 vector space of, 436 Schur, Issai, 123, 508, 662 SOR method, 624 Schur norm, 279 Souriau, J. M., 504 Schur triangularization theorem, 508 spanning sets, 165 Schwarz, Hermann A., 271, 307 for column space, 172 search engine, 418, 419 for four fundamental subspaces, 178 sectionally continuous, 301 for left-hand nullspace, 176 secular equation, 503 minimal, 197 Seidel, Ludwig, 622 for nullspace, 175 Sellers, Lois, xii for row space, 172 semiaxes, 414 test for, 172 semidefinite, 566 sparse least squares, 237 semisimple eigenvalue, 510, 591, 593, 596 sparse matrix, 350 semistable, 544 spectral circle, imprimitive matrices, 676 spectral mapping property, 539, 613 sensitivity, 128 spectral projectors, 517, 602, 603 minimum norm solution, 426 sequence commuting property, 522 limit of, 639 interpolation formula for, 529 of matrices, 220 positivity of, 677 series for f(A) , 605 in terms of eigenvectors, 518 shape, 8 spectral radius, 497, 521, 540 Collatz–Wielandt formula, 666, 673, 686 shell game, 635 as a limit, 619 Sherman, J., 124 and limits, 617 Sherman–Morrison formula, 124, 130 spectral representation of matrixfunctions, 526 SIAM, 324, 333 spectral resolution of f(A) , 603 signal processing, 359 spectral theorem for diagonalizable matrices, 517 signal-to-noise ratio, 418 spectrum, 490 sign of a permutation, 461 of imprimitive matrix, 677 similar matrices, 255, 473, 506 spheres, 275 similarity, 505 splitting, 620 and block-diagonal matrices, 263 spring-mass vibrations, 570 and block-triangular matrices, 263 springs, 86 and eigenvalues, 508 square invariant, 256 matrix, 8 and orthogonal matrices, 549 system, 5 transformation, 255, 408, 506 wave function, 301 and transpose, 596 stable, 544 algorithm, 217, 317, 347, 422 unitary, 547 matrix, 544 simple eigenvalue, 510 system, 544, 609 simple loops, 75 standard simultaneous diagonalization, triangularization, 522 basis, 194, 240, 299 simultaneous displacements, 622 coordinates, 240 sine, discrete, 361 deviation, 296 singular matrix, 115 inner product, 95, 271 eigenvalues of, 501 scores, 296 sequences of, 220 standardization of data, 296 singular systems, practical solution of, 217 stationary distribution, 531, 693 Index 717

T steady-state distribution, 531, 636 steepest descent, 657 Taussky-Todd, Olga, 497 step size, 19 Taylor series, 18, 570, 600 Stewart, G. W., xii t-digit arithmetic, 21 Stiefel, Eduard, 656 tensor product, 380, 597 stiffness and the Laplacian, 573 constant, 86 term-by-document matrix, 419 matrix, 87 text mining, 419 stochastic matrix, 685, 687 three-dimensional rotations, 328, 330 doubly, 702 time domain, 363 summability of, 697 Todd, John, 497 unit eigenvalues of, 696 Toeplitz matrices, 514 Strang, Gilbert, xii Toeplitz, Otto, 514 strongly connected graph, 209, 671 total least squares, 223 Strutt, John W., 550 trace, 90 stuff in a vector space, 197, 200 and characteristic equation, 504 subgroup, 402 of imprimitive matrices, 678 submatrix, 7 inequalities, 293 as a block matrix, 111 of a linear operator, 256 and rank, 215 of a product, 110, 114 subscripts, 7 of a projector, 392 subset, 162 as sum of eigenvalues, 494 subspace, 162 transformation, linear, 238 angles or gaps between, 450 transient behavior, 532 dimension of, 198 transient class, 695 directed distance between, 453 transition diagram, 108, 531 four fundamental, 169 transition matrix, 108, 531, 688 invariant, 259, 262, 263 transitive operations, 257 maximal angle between, 455 translation, in 3-D graphics, 332 sum of, 205 transpose, 83 substochastic matrix, 685 and determinants, 463 nullspace, 177 successive displacements, 623 properties of, 84 successive overrelaxation method, 624 reverse order law for, 109 sum and similarity, 596 of matrices, 81 trapezoidal form, 342 of orthogonal projectors, 441 trend of observations, 231 of projectors, 393 triangle inequality, 220, 273, 277 of vector spaces, 166, 383 backward version, 273 dimension of, 205 triangular matrices, 41, 103 summable matrixand summability, 631, 633, 677 block versions, 112 stochastic matrices, 697 determinant of, 462 superdiagonal, 575 eigenvalues of, 501 SVD, 412 elementary, 142 and full-rank factorization, 634 inverses of, 122 and oblique projectors, 634 triangularization, simultaneous, 522 switching circuits, 539 triangularization using elementary reflectors, 342 Sylvester, James J., 44, 80, 411 triangular system, 6 Sylvester’s law of inertia, 568 tridiagonal matrix, 20, 156, 352 Sylvester’s law of nullity, 220 Toeplitz matrices, 514 symmetric trivial functions, 494 nullspaces, 175 matrices, 85 solution, 57, 60, 69 diagonalization and eigen components of, 549 and nonhomogeneous systems, 70 reduction to tridiagonal form, 352 and nonsingular matrices, 116 space of, 436 subspace, 162, 197 permutation, 671 Tukey, J. W., 368, 375, 651 718 Index two-point boundary value problem, 18 Wronski, Jozef M., 189 U Wronski matrix, 189, 190 unbiased estimator for variance, 449, 446 X, Y, Z uncertainties in linear systems, 414 underrelaxation, 624 Young, David M., 625 Young, G., 411 unique solution Zeeman, E. Christopher, 704 for differential equations, 541 zero nullspace, 175 and free variables, 61 zero transformation, 238 for homogeneous systems, 61 Z-matrix, 628, 639, 296 for nonhomogeneous systems, 70 z-scores, 296 unitarily invariant norm, 425, 337 unitary diagonalization, 547 unitary matrices, 304, 320 determinant of, 473 unit columns, 102, 107 unit eigenvalues of stochastic matrices, 696 units, 27 unit sphere, 275 image of, 414, 425 unstable, 544 upper-trapezoidal form, 342, 344 upper triangular, 103 URV factorization, 406, 407 and full-rank factorization, 634

V

Vandermonde, Alexandre-Theophile, 185 Vandermonde determinant, 486 Vandermonde matrices, 185, 230, 357 Van Loan, Charlie, xii variance, 447 vector, 159 norms, 274 spaces, 160 vertexmatrix, 330 vibrations, small, 559 volume by determinants, 468 by Gram–Schmidt, and QR, 431 von Mises, R., 533 von Neumann, John, 289

W

Weierstrass, Karl Theodor Wilhelm, 589, 662 well conditioned, 33, 127, 415 Weyl, Hermann, 160 why least squares?, 446 Wielandt, Helmut, 534, 666, 675, 679 Wielandt’s matrix, 685 Wielandt’s theorem, 675 Will, Marianne, xii wire frame figure, 330 Woodbury, M., 124 Wronskian, 474, 481, 486