Addition Matrix, 16, 20 Vector, 2, 5 Adjoint, 75, 95, 96 Aitken's Theorem

Home , Commutation matrix, Hessian matrix

Cambridge University Press 0521822890 - Matrix Algebra Karim M. Abadir and Jan R. Magnus Index More information

Index

Addition equation, 155, 159, 174, 190 matrix, 16, 20 polynomial, 155, 156, 160, 161, 163, 170, vector, 2, 5 181 Adjoint, 75, 95, 96 root, see Eigenvalue Aitken’s theorem, 384 Cofactor, 75, 90, 91, 370 Almost sure, 50 Column Analytic continuation, 270 block, 97 Angle, 3, 10, 23 rank, 74, 77, 80 Asymptotic equivalence, 407 space, 73, 75–77 Commutation matrix, 300–307 Basis, 45, 56, 57 as derivative of X , 363 dimension, 59, 60 commutation property, 301, 302 existence, 58 deﬁnition, 299 extension to, 58 determinant, 305 reduction to, 58 eigenvalues, 305 Bayesian sensitivity, 377 explicit expression, 302, 303 Bilinear form, 356, 357 is orthogonal, 300 Binomial coefﬁcient, 405 trace, 304, 305 Block Completeness, 46, 47, 67, 68, 403 block-diagonal, 97 Completion of square, 216 column, 97 Complex numbers, 4, 11–13, 18–19, 39–42, 155, row, 97 398–400 Bolzano-Weierstrass theorem, 229, 402 argument, 398 complex conjugate, 4, 11, 399 Cauchy, 13, 96 modulus, 4, 12, 398 Cauchy-Schwarz, see Inequality, Cauchy-Schwarz polar form, 270, 398 criterion, 403 Concavity inequality, 324 of λn, 344 rule of invariance, 353 of log |A|, 334, 391 not valid for second differential, 353 of |A|1/n, 391 sequence, 47, 67, 402 Conformable, 17 Cayley’s transform, 264 Continuity argument, 96, 116, 165, 223, 229, 322, Cayley-Hamilton theorem, 190, 201, 271 333 Characteristic Contraction, 229, 242 426

Index 427

Convexity, 410 Direct sum, 70, 97 of λ1, 344 Distance, 46, 64 of Lagrangian, 354, 413 Duplication matrix, 311–317 sufficient condition, 354 and commutation matrix, 313 Craig-Sakamoto lemma, 181, 208 and Kronecker product, 315 Cramer’s rule, 146, 147, 154 definition, 299 properties of Dn(A ⊗ A)Dn, 317 Derivative, 351 properties of DnDn, 314 + identification, 352 properties of Dn (A ⊗ A)Dn, 315–317 notation, 351 with respect to symmetric matrix, 366, 367, Eigenvalue 373 algebraic multiplicity, 163 Determinant, 74, 87–96, 173 complex, 160 axiomatic definition, 92 concavity of λn, 344 definition, 74 convexity of λ1, 344 differential, 369–373 definition, 155 elementary operation, 89 distinct, 157, 170, 171, 175 equality, 116, 117, 167 geometric multiplicity, 163 expansion by row (column), 90 inequality, 343–350 Hessian matrix, 380, 381 monotonicity, 346 inequality, 225, 325, 326, 333, 339, 349 multiple, 155, 178 of 2 × 2 matrix, 75 multiplicity, 155 of 3 × 3 matrix, 87 of 2 × 2 matrix, 158 of commutation matrix, 305 of 3 × 3 matrix, 159 of conjugate transpose, 88 of AB and BA, 167 of diagonal matrix, 90 of commutation matrix, 305 of elementary matrix, 136 of diagonal matrix, 164 of inverse, 95 of Hermitian matrix, 175 of orthogonal matrix, 95 of idempotent matrix, 232 of partitioned matrix, 109–118 of inverse, 163 of positive (semi)definite matrix, 215, 216 of orthogonal matrix, 165, 175 of product, 94, 112 of positive (semi)definite matrix, 215 of skew-Hermitian matrix, 255 of power, 163 of transpose, 88 of rank-one matrix, 172 of triangular matrix, 92 of skew-Hermitian matrix, 255 of tridiagonal matrix, 92 of skew-symmetric matrix, 164, 255 product of eigenvalues, 167, 189 of symmetric matrix, 175, 181 Vandermonde, 93, 148 of transpose, 163 zero, 89, 94 of triangular matrix, 164 Deviations from the mean, 239, 242 of unitary matrix, 165 dg-function, 17 ordering, 322 diag-function, 17 product of, 167, 189 Differential, 351–395 quasilinear representation, 343, 346 first, 352, 355–373 real, 175, 225 second, 353 simple, 155, 174, 190 with respect to symmetric matrix, 366, 367, 373 sum of, 168, 189 Dimension, 45 variational description, 345 finite, 45, 55, 56, 59, 60 zero, 164, 190 infinite, 45, 56, 147 n Eigenvector of C ,60 definition, 157 of column space, 76, 77 example, 162 of kernel, 73, 82, 149 existence, 161 of orthogonal complement, 70, 73, 76, 77

428 Index

Eigenvector (Cont.) diagonalization generalized, 157, 201–208 conditions, 171, 187, 200 degree, 157 of idempotent matrix, 234, 236 left, 173 of normal matrix, 158, 191 linear combination, 161 of positive (semi)definite matrix, 210, 215, linearly independent, 170, 171, 176 219 normalized, 157 of symmetric matrix, 158, 177, 189 of symmetric matrix, 175, 176, 178, of triangular matrix, 158, 186 179 simultaneous, 158, 174, 180, 211, 225 orthogonal, 170, 171, 173, 175, 176 with distinct eigenvalues, 158, 171 orthonormal, 187, 192 Jordan’s theorem, 157, 158, 199, 245 right, 157 polar decomposition, 211, 226, 398 uniqueness, 157, 161, 176 QR, 158, 172 Elementary rank, 80, 158 matrix, 132–137 Schur’s triangularization theorem, 157, 158, determinant, 136 187 does not commute, 136 singular-value decomposition, 211, 225, 226 explicit expression, 133 using echelon matrices, 140, 158 inverse, 134 Fibonacci sequence, 35, 42 product, 135, 140 Fischer transpose, 134 inequality, 228, 341 operation, 89, 100, 101, 131, 133, 136 min-max theorem, 346 symmetric function, 156, 169 Frobenius’s inequality, 122, 129 Elimination Fundamental theorem of algebra, 155, 401 backward, 132, 145 forward, 132, 145 Gauss, 154 Gaussian, 132, 143–148 Gauss-Markov theorem, 384 Equation, linear, 131–153 Generalized inverse, 295 characterization, 152 and the solution of linear equations, 296 consistency, 151, 152, 293, 294, 296 definition, 274 general solution, 294, 296 existence, 295 homogeneous, 79, 132, 148–150, 292 explicit expression, 295 matrix, 294–295 rank, 296 nonhomogeneous, 132, 151–153 Gram-Schmidt orthogonalization, 67, 98, 172 nontrivial solution, 148, 149 number of solutions, 148, 151 Hadamard product, 321, 340 trivial solution, 148, 149 definition, 340 unique solution, 294, 295 positive definite, 340 vector, 79, 292–294 Heine-Borel theorem, 402 Equivalence, 141, 157 Hermitian form, 19, 39, 210 class, 50 Euclid, 328 Idempotent matrix, 18, 231–242 Euclidean space, 1 checks, 210 m-dimensional, 1 definition, 18, 210 Euler, 143, 144, 154, 398 differential, 365 Exponential of a matrix eigenvalues, 232, 233 differential, 368 idempotent operation, 37, 210 series expansion, 244, 249, 252, 256, 257, 260, in econometrics, 238 262, 265–269 necessary and sufficient condition, 235 nonsymmetric, 37, 210, 232 Factorization of order two, 37 as product of elementary matrices, 140 rank equals trace, 235 Cholesky, 210, 220, 242 sum of, 236, 240–242

Index 429

Inequality Kato’s lemma, 218, 331 arithmetic-geometric mean, 328, 392, 393 Kernel, 73, 149 Bergstrom, 323 Kronecker delta, 66 Bunyakovskii, 13 Kronecker product, 274–281 Cauchy, 324 and vec-operator, 282 Cauchy-Schwarz, 7, 8, 13, 62, 322, 325, 330 deﬁnition, 273 concerning eigenvalues, 343–350 determinant, 279 derived from Schur complement, 341–343 differential, 355 determinantal, 225, 325, 326, 333, 339, 349 eigenvalues, 278 Fischer, 228, 341 eigenvectors, 279 Frobenius, 122, 129 inverse, 278 Hadamard, 337, 349 multiplication rules, 275–277 Kantorovich, 331 noncommutativity, 275 Minkowski, 329 rank, 279 Olkin, 340 trace, 277 rank, 78, 81, 120–122, 124, 179, 324 tranpose, 277 Schur, 305, 325, 331 Sylvester, 122, 129, 179, 180 Lagrange trace, 329, 338, 348 function, 411 triangle, 7, 12, 13, 38, 63 multiplier method, 411–413 geometric interpretation, 8 multipliers, 411 Inner product matrix of, 354, 413 continuity, 65 symmetric matrix of, 354 in vector space, 45 Laplace expansion, 74 induced by norm, 63, 64 Leading element, 132, 138 of two complex vectors, 4, 12 Least squares of two real matrices, 18, 38 and best linear unbiased estimation, 382–387 oftworealvectors,2,6,20 constrained (CLS), 383 2 Inverse, 74, 83–87, 95 estimation of σ , 386 and rank, 83 generalized (GLS), 339, 342, 383 differential, 364–367 multicollinearity, 385 existence, 83 ordinary (OLS), 339, 342, 382 Jacobian of transformation, 373, 374 residuals, 376 of A + ab, 87, 173, 248 sensitivity analysis, 375–377 of A − BD−1C, 107 Leibniz, 154 of partitioned matrix, 103–109 Length, 46, 62, 212 of product, 84 l’Hopital’sˆ rule, 408 of triangular matrix, 186 Line, 6 series expansion, 249 in the plane, 327 uniqueness, 83 line segment, 6 Linear Jacobian combination, 44, 52 matrix, 351 dependence, 44, 53, 54 of transformation, 354, 373–375 conditions for, 53 Jordan, 192–200 in triangular matrix, 53 block, 192–194 difference equation, 36, 409–410 power, 258 equation, see Equation, linear symmetrized, 194 form, 18, 19 chain, 157, 201–208 independence, 44 lemma, 195, 196 and span, 59 matrix, 157, 158, 197, 198, 200 of powers, 147 number of blocks, 200 space, see Vector space representation, 260 structure, 300, 318–320, 374 theorem, 199

430 Index

Logarithm of a matrix negative (semi)definite, 209 differential, 368–369 see also Positive (semi)definite matrix series expansion, 244, 250, 253 nilpotent, 23, 183, 192 index, 23 Matrix nonsingular, 74, 83, 140 augmented, 145, 151 normal, 18, 19, 33, 41, 171, 182, 191, 192 block-diagonal, 97 notation, 15 cofactor, 75, 91, 187 null, 16, 20 commutation, see Commutation matrix order, 15, 17, 19 commuting, 17, 23, 24, 29, 34, 101, 174 orthogonal companion, 173 2 × 2 example, 31, 95, 254 complex, 16, 18–19, 39–42 definition, 18 conformable, 17 determinant, 95 conjugate transpose, 18, 39 eigenvalues, 175 definition, 15 preserves length, 212 diagonal, 17, 28, 29, 337 properties, 86 duplication, see Duplication matrix real versus complex, 19, 85, 166 echelon, 132, 137–143 representation, 31, 254, 263 definition, 132 partitioned, see Partitioned matrix factorization, 140 permutation, 18, 32 finding inverse, 142 is orthogonal, 33 rank, 132, 137, 141 positive, 243 reduction of, 139 positive (semi)definite, reduction to, 137 see Positive (semi)definite matrix element, 15 power, 18, 34, 35, 217 elementary, see Elementary, matrix and difference equations, 36 equality, 16, 19 Fibonacci sequence, 35 equicorrelation, 241 noninteger, 261 function, 243–271 product, 16, 21, 22, 25 determinant, 262 different from scalar multiplication, 22 of diagonal matrix, 246, 247 real, 16 of idempotent matrix, 248 scalar, 17 of nilpotent matrix, 247 scalar multiplication, 16 of symmetric matrix, 255 semi-orthogonal, 84 trace, 262 singular, 74, 149, 227 generalized inverse, see Generalized inverse skew-Hermitian, 18, 40 Gramian, bordered, 230–231, 242 determinant, 255 Hermitian, 18, 39, 175 diagonal elements, 40 diagonal elements, 39 eigenvalues, 255 Hessian, 378–382 skew-symmetric, 30 identification, 353, 378, 379 definition, 18 in maximum likelihood estimation, 390 diagonal elements, 30 of composite function, 353 eigenvalues, 164, 255 symmetry, 353 representation, 255 idempotent, see Idempotent matrix skew-symmetrizer, see Skew-symmetrizer ma- identity, 17, 21, 28 trix indefinite, 209, 219 square, 17 inverse, see Inverse square root, see Positive (semi)definite matrix, invertible, see Matrix, nonsingular square root Jacobian, 351 submatrix see also Derivative definition, 15 Jordan, 157, 158, 197, 198, 200 in partitioned matrix, 26 Moore-Penrose inverse, see Moore-Penrose in- leading principal, 156, 168, 337 verse

Index 431

Matrix (Cont.) Norm principal, 156, 222 general definition, 63 rank, 82 in vector space, 46, 62 sum of matrices, 16, 20 induced by inner product, 63 symmetric, 17, 29, 51, 157, 171, 175–182 of complex vector, 4, 12 definition, 17 of real matrix, 18, 38 power, 35 of real vector, 2, 7, 8 real eigenvalues, 175 Normal distribution, 377, 386–391 real eigenvectors, 175 and symmetrizer matrix, 309–311 real versus complex, 19, 175, 191 Notation, 415–421 special properties, 157 Null space, see Kernel symmetrizer, see Symmetrizer matrix transpose, 15, 20, 26 O, o, oh notation, 407, 419 triangular, 17, 51, 157, 182–187 Orthogonal inverse, 186 complement, 46, 66, 70, 73, 76, 237 linearly independent columns, 53 matrix, see Matrix, orthogonal lower, 17 projection, 68 power, 35 set, 46 product, 29, 184 subspace, 46, 66, 73 strictly, 17, 183, 195 vectors, 3, 8–10, 46 unit, 17, 186 Orthonormal upper, 17 basis, 67 tridiagonal, 92 set, 46, 66 unipotent, 101 vectors, 3, 66 unitary, 18, 40 representation, 264 Vandermonde, 93, 96, 147, 148 Parallelogram equality, 62 Maximum likelihood estimation, 387–391 Partitioned matrix, 26, 97–129 Hessian matrix, 390 3-by-3 block matrix, 107, 118, 125 information matrix, 390 bordered, 108, 118, 125 log-likelihood, 387 commuting, 101 treatment of positive definiteness, 389 determinant, 109–118 treatment of symmetry, 388, 389 diagonal, 100 Minimum inverse, 103–109 global, 354 positive (semi)definite, 228–231 local, 354 determinant, 114, 335–336 under constraints, 354, 410–413 inverse, 107 Minor, 370 necessary and sufficient conditions, 228, 229 leading principal, 156, 223 power, 108 principal, 156, 223 product, 98 sum of, 156 rank, 119–125 Modulus, see Complex numbers, modulus sum, 98 Moore-Penrose inverse, 284–292 symmetric, 100 and least squares, 384 trace, 100 and the solution of linear equations, 292–295 transpose, 99 definition, 274 triangular, 100 existence, 284 Permutation of positive semidefinite matrix, 290 matrix, see Matrix, permutation of symmetric matrix, 289 of integers, 74, 89 rank, 286 Pivot, 98, 132, 145 uniqueness, 285 Poincare’s´ separation theorem, 347–348 Multiplication Polynomial, 56, 93, 96, 147, 168, 169, 400–401 scalar, 2, 5, 16 characteristic, 155, 156, 160, 161, 163, 170, in vector space, 43 181

432 Index

Polynomial (Cont.) full row rank, 74 matrix, 243–271 inequality, 78, 81, 120–122, 124, 179, 324 order of, 243 matrix of rank one, 80, 172 representation, 265–270 matrix of rank two, 218 monic, 401 number of nonzero eigenvalues, 165, 179, 190 Positive (semi)definite matrix, 211–231 of diagonal matrix, 79 checks, 210 of partitioned matrix, 119–125 definite versus semidefinite, 210 of product, 82 definition, 209 of submatrix, 82 determinant, 215, 216 of triangular matrix, 79 diagonal elements, 213 theorem, 77 eigenvalues, 215, 222 Rayleigh quotient, 181, 343 inequality, 325–340 Reflection, 32, 95 inverse, 216 Rotation, 23, 32, 95, 254 matrix quadratic form, 221–222 Row of rank two, 218 block, 97 partitioned, 228–231 rank, 74, 77, 80 power, 217, 332 principal minors criterion, 223, 224 Scalar, 1 principal submatrices, 222 Scalar product, see Inner product quadratic form, 209, 211 Scaling, 24 square root, 220, 221, 227 Schur uniqueness, 220 complement, 102, 106, 228, 322 trace, 215, 216 inequality derived from, 341–343 transforming symmetric matrix into, 218 notation for, 102 upper bound for elements, 213, 323 decomposition theorem, 157, 158, 187 variance matrix, 56 inequality, 305, 325, 331 versus negative (semi)definite, 209, 213 Schwarz, 13 Positivity, treatment of (in optimization problems), inequality, see Inequality, Cauchy-Schwarz 386, 389 Sensitivity analysis, 375–378 Postmultiplication, 17 Bayesian, 377 Power sum, 156, 169 least squares, 375–377 Premultiplication, 17 Series expansion, 401–409 Projection, 239 absolutely convergent, 243, 260, 403 oblique, 210 binomial orthogonal, 68, 210 alternative expansion, 246 theorem, 68, 69 definition, 244, 405 Proof radius of convergence, 244, 260, 405 by contradiction, 398 with two matrices, 251 by contrapositive, 397 conditionally convergent, 245, 403 by deduction, 398 definition, 243 by induction, 398 exponential function Pythagoras, 9, 65, 69, 70 as limit of binomial, 249 definition, 244, 398 Quadratic form, 18, 211, 212 Jordan representation, 257 differential, 356, 357, 359, 363 multiplicative property, 252, 256 Hessian matrix, 378, 381 nonsingularity, 262 Quasilinearization, 322, 324, 328, 329, 343, 344 polynomial representation, 265–269 radius of convergence, 244, 260, 404 Rank, 74, 75 inverse, 249 and zero determinant, 94 Jordan representation, 255–264 equality, 81, 82, 85, 123, 124 logarithmic function full column rank, 74 additive property, 253

Index 433

Series expansion (Cont.) idempotent, 307 explicit expression, 250 orthogonal to skew-symmetrizer, 307 explicit representation, 244, 403, 405 rank, 307 implicit definition, 244, 405 radius of convergence, 244, 403 Trace, 18, 30 matrix-polynomial representation, 265–270 and vec-operator, 283 nonconvergent, 403 differential, 357, 358 not unique, 243, 246, 261 Hessian matrix, 380 power, 261 inequality, 329, 338, 348 radius of convergence, 244, 403 linear operator, 18, 30 summable, 248, 403–405, 414 of A A, 31, 38, 214, 322 Series representation, see Series expansion of commutation matrix, 304, 305 Set of conjugate transpose, 39 compact, 402 of matrix product, 18, 31 of real numbers, 1 of positive (semi)definite matrix, 215, 216 Shift of power, 168, 169, 189 backward, 193 of transpose, 18, 30 forward, 193 sum of eigenvalues, 168, 189 Similarity, 157, 185 Transposition, 74, 89, 96 and eigenvalues, 166 Singular value, 226 Unit circle, 244, 400 Skew-symmetrizer matrix, 305 definition, 305 Vec-operator, 281–284 idempotent, 305 and Kronecker product, 282 orthogonal to symmetrizer, 307 and trace, 283 Space definition, 273 complex space, 4 differential, 355 Euclidean space, 1 linearity, 281 vector space, see Vector space notation, 273 Span, 44, 54, 55, 81 Vech-operator, 311, 312 and linear independence, 59 definition, 299 Spectral radius, 243, 369 Vector, 1–13 Rm Spectral theorem, see Factorization, diagonalization as a point in ,1 Stirling’s series, 407 as an arrow, 1 Subspace, 44, 50, 51 collinear, 2, 7, 13, 20 closed, 68 norm of, 8 dimension, 60, 61 column vector, 1 intersection, 44, 52 component of, 1 of R2,50 elementary, see Vector, unit of R3,51 equality, 1, 4 sum, 44, 52, 61 inequality, 1 union, 44, 52 length, 3 Sweep operator, 98, 126–129 normal to plane, 65 calculates inverse, 127 normalized, 3, 8, 32 pivot, 98 notation, 1 solves linear equation, 128 null vector, 1, 5, 47 Sylvester, 327, 350 uniqueness, 5, 47 law of nullity, 122, 129, 179, 180 order, 1, 4 Symmetrizer matrix, 307–311 orthogonal, 3, 8–10, 46, 65 and Kronecker product, 307 orthonormal, 3, 9, 32 and normal distribution, 309–311 row vector, 16 and Wishart distribution, 310 scalar multiplication, 2, 5 definition, 299 sum of vectors, 2, 5

434 Index

† Vector (Cont.) L2-space, 49, 56, 68, 71 sum vector, 1, 10, 31 real, 43, 48 unit, 3 representation in terms of basis, 58 Vector analysis scalar multiplication, 43, 47 algebraic, 1 vector in, 43 geometric, 1, 2 Vector space, 43–71 addition, 43, 47 Weierstrass, 92, 96 axioms, 43–44 Wishart distribution complex, 43 central, 310 Hilbert space, 44, 46, 47, 67–71 deﬁnition, 310 completeness, 47 noncentrality matrix, 310 inner-product space, 44, 45, 61–67 variance, 310, 317 l2-space, 49, 56, 61, 71 L2-space, 50, 56 Zorn’s lemma, 71