APPENDIX A Additional Details and Fortification for Chapter 1 A.1 Matrix Classes and Special Matrices The matrices can be grouped into several classes based on their operational prop- erties. A short list of various classes of matrices is given in Tables A.1 and A.2. Some of these have already been described earlier, for example, elementary, sym- metric, hermitian, othogonal, unitary, positive definite/semidefinite, negative defi- nite/semidefinite, real, imaginary, and reducible/irreducible. Some of the matrix classes are defined based on the existence of associated matri- ces. For instance, A is a diagonalizable matrix if there exists nonsingular matrices T such that TAT −1 = D results in a diagonal matrix D. Connected with diagonalizable matrices are normal matrices. A matrix B is a normal matrix if BB∗ = B∗B.Normal matrices are guaranteed to be diagonalizable matrices. However, defective matri- ces are not diagonalizable. Once a matrix has been identified to be diagonalizable, then the following fact can be used for easier computation of integral powers of the matrix: A = T −1DT → Ak = T −1DT T −1DT ··· T −1DT = T −1DkT and then take advantage of the fact that ⎛ ⎞ dk 0 ⎜ 1 ⎟ K . D = ⎝ .. ⎠ K 0 dN Another set of related classes of matrices are the idempotent, projection, invo- lutory, nilpotent, and convergent matrices. These classes are based on the results of integral powers. Matrix A is idempotent if A2 = A, and if, in addition, A is hermitian, then A is known as a projection matrix. Projection matrices are used to partition an N-dimensional space into two subspaces that are orthogonal to each other. A matrix B is involutory if it is its own inverse, that is, if B2 = I. For example, a reflection matrix such as the Householder matrix is given by 2 H = I − vv∗ v∗v where v is a nonzero vector, and then H = H−1. A convergent matrix (also known as k stable matrix) C is a matrix for which limk→∞ C = 0. These matrices are important 561 562 Appendix A: Additional Details and Fortification for Chapter 1 Table A.1. Matrix classes (based on operational properties) Class Definition Remarks Convergent (Stable) lim Ak = 0 k→∞ r α Ak = 0 Defective (Deficient) k k=0 αr = 0; r < N T −1AT is diagonal Diagonalizable for some nonsingular T Any matrix that scales, interchanges, or adds Elementary multiples of rows or • Used in Gaussian elimination columns of another matrix B Gram A = B∗B for some B • Are Hermitian • (B + B∗) /2 is the hermitian Hermitian A∗ = A part of B. • B∗B and BB∗ are hermitian Idempotent A2 = A • det(A) = 1ordet(A) = 0 • Examples: identity matrix Involutory A2 = I,i.e.A = A−1 reverse unit matrices, symmetric orthogonal matrices Negative definite x∗Ax < 0 x = 0 Negative semidefinite x∗Ax ≤ 0 x = 0 k Nilpotent (of degree k) A = 0; k > 0 • det(A) = 0 Normal AA∗ = A∗A • Are diagonalizable Nonsingular (Invertible) |A| = 0 for procedures that implement iterative computations. If, in addition, k < ∞ for Ck = 0, then the stable matrix will belong to the subclass of nilpotent matrices. Aside from the classifications given in Tables A.1 and A.2, we also list some spe- cial matrices based on the structure and composition of the matrices. These are given in Table A.3. Some of the items in this table serve as a glossary of terms for the special matrices already described in this chapter. Some of the matrices refer to matrix struc- tures based on the positions of zero and nonzero elements such as banded, sparse, triangular, tridiagonal, diagonal, bidiagonal, anti-diagonal, and Hessenberg. Some involve additional specifications on the elements themselves. These include iden- tity, reverse identity, shift, real, complex, polynomial, rational, positive/negative, or nonpositive/nonnegative matrices. For instance, positive (or nonnegative) matrices Appendix A: Additional Details and Fortification for Chapter 1 563 Table A.2. Matrix classes (based on operations) Class Definition Remarks Orthogonal AT = A−1 Positive definite x∗Ax > 0;x = 0 Positive semidefinite x∗Ax ≥ 0;x = 0 Projection Idempotent and Hermitian There exists permutation P Reducible such that A = PAPT is block triangular • det(A) = 0ifN is odd • a = 0, thus trace(A) = 0 Skew-symmetric AT =−A ii • (B − BT )/2isthe skew-symmetric part of B • aii = 0 or pure imaginary Skew-hermitian A∗ =−A • (B − B∗)/2isthe skew-hermitian part of B • BT B and BBT are both symmetric but generally not equal Symmetric A = AT • (B + BT )/2isthe symmetric part of B Unitary A∗ = A−1 are matrices having only positive (or nonnegative) elements.1 Some special matrices depend on specifications on the pattern of the nonzero elements. For instance, we have Jordan, Toeplitz, Shift, Hankel, and circulant matrices, as well as their block matrix versions, that is, block-Jordan, block-Toeplitz, and so forth. There are also special matrices that depend on collective properties of the rows or columns. For instance, stochastic matrices are positive matrices in which the sum of the elements within each row should sum up to unity. Another example are diagonally dominant matrices, where for the elements of any fixed row, the sum of the magnitudes of off-diagonal elements should be less than the magnitude of the diagonal element in that row. Finally, there are matrices whose entries depend on their row and column indices, such as Fourier, Haddamard, Hilbert, and Cauchy matrices. Fourier and Haddamard matrices are used in signal-processing applications. As can be expected, these tables are not exhaustive. Instead, the collection shows that there are several classes and special matrices found in the literature. They often contain interesting patterns and properties such as analytical formulas for determinants, trace, inverses, and so forth, that could be taken advantage of during analysis and computations. 1 Note that positive matrices are not the same as positive definite matrices. For instance, with 15 1 −2 A = B = 51 02 A is positive but not positive definite, whereas B is positive definite but not positive. 564 Appendix A: Additional Details and Fortification for Chapter 1 Table A.3. Matrices classes (based on structure and composition) Name Definition Remarks • AB (or BA) will reverse ⎛ ⎞ sequence of rows (columns) α α 0 1 of B,scaledby i ⎝ ⎠ N Antidiagonal A = ... • det(A) = (−1) αi αN 0 • MATLAB: A=flipud(diag(alpha)) where alpha=(α1,...,αN) ⎧ ⎨ i > j + p • p is the right-bandwidth Band (or banded) a = 0if or ij ⎩ • q is the left-bandwidth j > i + q • = N α det(A) i=1 i • Let B = A−1 then if j > i, bij = 0 ⎛ ⎞ α 1 1 0 if j = i, b = ⎜ β α ⎟ ii α ⎜ 1 2 ⎟ i Bidiagonal = ⎜ ⎟ i−1 A ⎜ . ⎟ 1 βk (Stieltjes) ⎝ .. .. ⎠ if i > j, bij = − αi αk k=j 0 βN−1 αN • MATLAB: A=diag(v)+diag(w,-1) where v= (α1,...,αN) w= (β1,...,βN−1) • Often used to indicate Binary aij = 0or1 incidence relationship between i and j • Are nonsingular (but often ill-conditioned for large N) For given x and y • = det(A) 1 N i−1 a = ; x + y = 0 = = fij Cauchy ij x + y i j i2 j 1 i j N N + and elements of x and y i=1 j=1(xi yj ) = − − are distinct where fij (xi xj )(yi yj ) • MATLAB: A=gallery(‘cauchy’,x,y) ⎛ ⎞ • α α ··· α Are normal matrices 1 2 N • ⎜ α α ··· α ⎟ Are special case of Toeplitz = ⎜ N 1 N−1 ⎟ • Circulant A ⎝ ··· ⎠ MATLAB: A=gallery(‘circul’,alpha) α2 α3 ··· α1 where alpha= (α1, ···,αN) ⎛ ⎞ − ··· − − pn−1 p1 p0 • ⎜ ⎟ pk are coefficients of a ⎜ ⎟ ⎜ ⎟ polynomial: ⎜ 100 ⎟ N n−1 Companion A = ⎜ ⎟ s + pN−1s + p1s + p0 ⎜ ⎟ . • MATLAB: A=compan(p) ⎝ .. ⎠ where p= (1, p − ,...,p , p ) 010 n 1 1 0 Complex aij are complex-valued Appendix A: Additional Details and Fortification for Chapter 1 565 Name Definition Remarks ⎛ ⎞ α 1 0 • = α ⎜ ⎟ det(A) i i = ⎜ . ⎟ • Diagonal A ⎝ .. ⎠ MATLAB: A=diag(alpha) where alpha= (α1,...,αN) 0 αN |a | > a • Nonsingular (based on Diagonally dominant ii ij i=j Gersgorin’s theorem) i = 1, 2,...,N • Are orthogonal √ (i−1)(j−1) • Used in Fourier transforms aij = (1/ N)W • MATLAB: Fourier h=ones(N,1)*[0:N-1]; √ 2π W = exp − −1 W=exp(-2*pi/N*1i); N A=W.ˆ(h.*h’)/sqrt(N) Identity matrix with 4 • Used to rotate points elements replaced based on in hyperplane Givens (Rotation) given p and q: • Useful in matrix reduction app = aqq = cos(θ) to Hessenberg form apq =−aqp = sin(θ) • Are orthogonal H [=]2k × 2k k • Elements either 1 or −1 11 Hadamard H = ⊗ H − • Are orthogonal k 1 −1 k 1 • MATLAB: A=hadamard(2ˆk) H0 = 1 ⎛ ⎞ • ··· βα Each anti-diagonal has the ⎜ ··· ··· γ ⎟ same value = ⎜ ⎟ • Hankel A ⎝ β ··· ··· ⎠ MATLAB: A=hankel([v,w]) = ...,β,α αγ ··· where v ( ) w= (α,γ,...) • Useful in finding eigenvalues • For square B, there is unitary Q such that A = Q∗BQ is a + , = 0 Hessenberg j k j upper hessenberg 2 ≤ k ≤ (N − j) • MATLAB: [Q,A]=hess(B); where A=(Q’)(B)(Q) • Symmetric and positive definite 1 • MATLAB: Hilbert a = ij i + j − 1 h=[1:N]; A=gallery(‘cauchy’,h,h-1) ⎛ ⎞ • 10 Often denoted by IN ⎜ ⎟ • det(A) = 1 Identity A = ⎜ .. ⎟ ⎝ . ⎠ • AB = BA = B 01 • MATLAB: A=eye(N) (continued) 566 Appendix A: Additional Details and Fortification for Chapter 1 Table A.3 (continued) Name Definition Remarks A = iB Imaginary where B√is real and i = −1 ⎛ ⎞ s 10 ⎜ ⎟ • Are bidiagonal ⎜ . ⎟ ⎜ .. .. ⎟ • det(A) = sN Jordan block A = ⎜ ⎟ ⎜ .