sign in sign up
<<
Home , Design matrix

APPENDIX Some results

The main matrix results required for this book are given in this Appendix. Most of them are stated without proof as they are relatively straightforward and can be found in most books on matrix algebra; see for instance Searle (1966), Basilevsky (1983) and Graybill (1983). The properties of circulant matrices are considered in more detail as they have particular relevance in the book and are perhaps not widely known. A detailed account of circulant matrices can be found in Davis (1979).

A.1 Notation

An n x m matrix A = ((aij)) has n rows and m columns with aij being the element in the ith row andjth column. A' will denote the transpose of A. A is symmetrie if A = A'. If m = 1 then the matrix will be called a eolumn veetor and will usually be denoted by a lower ca se letter, a say. The transpose of a will be a row veetor a'. A whose diagonal elements are those ofthe vector a, and whose ofT-diagonal elements are zero, will be denoted by a6• The inverse of a6 will be denoted by a- 6• More generally, if each element in a is raised to power p then the corresponding diagonal matrix will be denoted by a P6• Some special matrices and vectors are:

In' the n x 1 vector with every element unity In = 1~, the n x n Jnxm = I nl;", the n x m matrix with every element unity

J n = J nxn

Kn = (l/n)Jn the n x n matrix with every element n -1. Suffices can be omitted if, in doing so, no ambiguity results. 206 SOME MATRIX RESULTS A.2 Trace and rank

If A = ((au)) is a square matrix of order n, i.e. an n x n matrix, then

n trace(A) = L ajj ;= 1 The number of linearly independent rows or columns of the n x n matrix A is given by r = rank(A). If r = n then A is non-singular, if r < n A is singular. Provided the matrices are conformable, and not necessarily square, then

trace(AB) = trace(BA) (A.l)

rank (AB) :s.;; min[rank(A), rank(B)] (A.2)

rank(AA') = rank(A' A) = rank(A) = rank(A') (A.3) It follows from (A.l) that the trace of the product of matrices is invariant under any cyclic permutation ofthe matrices. For example, trace(ABC) = trace(BCA) = trace(CAB)

A.3 Eigenvalues and eigenvectors Let A be a square symmetrie matrix of order n. An eigenvalue of A is a seal ar A such that Ax = AX for some vector x # o. The vector x is called an eigenvector of A. If Al' A2' ... , An are the eigen val ues of Athen

n trace(A) = L A; (A.4) ;= 1 rank(A) = number of non-zero eigenvalues (A.5)

n lAI = n A; (A.6) ;= 1 where lAI is the determinant of A. If Al ,A2 , ••• ,Am are the distinct eigenvalues of A (m:s.;; n) then the determinant can be written as

lAI = A~I Ai' ... A~'" where n; is the multiplicity of Ai> and where Ln; = n. Repeated application ofAx = AX shows that, for some positive integer h, x is also an eigenvector of Ah with corresponding eigtmvalue IDEMPOTENT MATRICES 207 A\ i.e. (h = 1,2, ... ) (A.7) The eigenvalues of a symmetrie matrix are real. For eaeh eigenvalue of a symmetrie matrix there exists areal eigenveetor. If B is a non-square matrix then B'B and BB' have the same non-zero eigenvalues. For every symmetrie matrix A there exists a non-singular matrix X sueh that

(A.8) where the elements of Aare the eigenvalues of A. For the symmetrie matrix A there exists a set of orthogonal and normalized eigenveetors XI' X2, •.. , Xn satisfying , {1, i=j xix j = 0, i#j The canonical or spectral deeomposition of A is then given by

n A = LAiXiX; (A.9) i= I where Ai is the eigenvalue eorresponding to Xi' If A is non-singular then

n -1 "1-1 , A = L... Ai XiXi (A.1O) i= I

It also follows from (A.9) that the eigenveetors Xi (i = 1,2, ... , n) satisfy

n L xix;=I (A.ll) i= I

A.4 Idempotent matrices

A square matrix A is idempotent if A 2 = A. If A is idempotent then so is I - A. land Kare examples of idempotent matriees. The eigenvalues of an A of order n are 1 with multiplieity r, and 0 with multiplieity n - r, where r = rank(A). It then follows from (A.4) and (A.5) that for an idempotent matrix

rank(A) = traee(A) (A.12) 208 SOME MATRIX RESULTS If A and Bare idempotent then AB is idempotent if and only if AB=BA.

A.5 Generalized inverses Let A be a square matrix of order n with rank(A) = r. If r = n then A has an unique inverse A - 1 which satisfies (A.13) Further, an unique solution to the equations Ax = y is then given by x=A-ly (A.14)

If r < n, A - 1 does not exist. There will, however, be an infinite number of solutions of the consistent equations Ax = y. As an example, consider the equations

The second equation is twice the first so that rank(A) = 1. Replacing 6 by 5, say, in the vector y would lead to an inconsistent set of equations for which no solution will exist. One solution to the above equations is given by Xl = 3, X2 = 0 and can be written as x = By where B=(~ ~}

Although A - 1 does not ex ist, the matrix B plays the role of an inverse in that its use leads to a solution to the equations. Bis called a generalized inverse or g-inverse of A, and is denoted by A -; it is sometimes called a pseudo-inverse or conditional inverse of A. It is not unique since

A-=G _~) also leads to the solution Xl = 3, X2 = 0, whereas

A-= C~3 1~6) leads to a different solution, namely Xl = x2 = 1. GENERALIZED INVERSES 209 A - is ag-inverse of A if and only if it satisfies (A.15) A solution to the eonsistent equations Ax = y is then given by (AI6) It ean be verified that the three g-inverses given in the above example satisfy (A.15). The matrix A + whieh satisfies the following four eonditions: AA+A=A A+AA+ =A+ (A.l7) (AA +l' = AA + (A + Al' = A + A is ealled the Moore-Penrose g-inverse of A; it ean be shown to be an unique matrix. If A is given in eanonieal form as n A= L AjXjX; j= I where Al, A , .•• , A, (r< n) are the non-zero eigenvalues, then: 2 , + "1-1 I (a) A = L Aj XjXj (AI8) j= 1 is the Moore-Penrose g-inverse of A;

, n (b) A - = L Aj- 1 XjX; + L (XjXjX; (A.19) i=l i=r+l is ag-inverse of A for a given set of eonstants C(, + l' C(, + 2, ... , C(n. If these eonstants are zero then A - '7 A + .

If A is asymmetrie idempotent matrix then two useful g-inverses of Aare the Moore-Penrose inverse A + = A and the inverse A - = I. The following two results are frequently used in the book: (a) AA - is an idempotent matrix with rank(AA -) = traee(AA -) = rank(A) (A.20) (b) If (A/A)- is ag-inverse of A/A then A(A/A)- A' A = A (A21) 210 SOME MATRIX RESULTS A.6 Kronecker products

Let A = ((aij)) be a p x q matrix and B be an r x s matrix. The Kronecker product of A and B, denoted by A ® B, is the pr x qs matrix defined as

allB al2B ... alqB) A®B= ( f21 B f22 B i·· f2qB aplB ap2B ... apqB

Two special results are that In ® Im = Inm and Kn ® Km = Knm . If a is a scalar and matrices are conformable, then

(aA)®B = A®(aB) = a(A®B) (A®B)®C = A®(B®C) (A®B)' = A'®B' (A.22) (A®B)(C®D) = AC®BD (A + B)®C = A®C + B®C Let A and B be square matrices of order n and m respectively; then (A®B)- =A-®B­ (A.23) trace(A ® B) = trace(A)·trace(B) (A.24)

IfxI isan eigenvector of A with eigenvalue Al' and x2 is an eigenvector of B with eigenvalue A2' then

(A ® B)(x I ® X2) = Al A2(X I ® X2) (A.25) i.e. Xl ®X2 is an eigenvector of A®B with eigenvalue AI A2.

A.7 Circulant matrices If A = ((ai)) is an n x n matrix with aij = alm where j - i + 1, r;~ i { m= n-(j-i+ 1), j