The Concept of a Generalized Inverse for Matrices Was Introduced by Moore(1920)

J. Japanese Soc. Comp. Statist. 2(1989), 1-7

THE MOORE-PENROSE INVERSE MATRIX POR THE BALANCED ANOVA MODELS

Byung Chun Kim* and Ha Sik Sunwoo*

ABSTRACT

Since the design matrix of the balanced linear model with no interactions has special form, the general solution of the normal equations can be easily found. From the relationships between the minimum norm least squares solution and the Moore-Penrose inverse we can obtain the explicit form of the Moore-Penrose inverse X+ of the design matrix of the model y = XƒÀ + ƒÃ for the balanced model with no interaction.

1. Introduction The concept of a generalized inverse for matrices was introduced by Moore(1920). He developed it in the context of linear transformations from n-dimensional to m-dimensional vector space over a complex field with usual Euclidean norm. Unaware of Moore's work, Penrose(1955) established the existence and the uniqueness of the generalized inverse that satisfies the Moore's condition under L2 norm. It is commonly known as the Moore-Penrose inverse. In the early 1970's many statisticians, Albert (1972), Ben-Israel and Greville(1974), Graybill(1976),Lawson and Hanson(1974), Pringle and Raynor(1971), Rao and Mitra(1971), Rao(1975), Schmidt(1976), and Searle(1971, 1982), worked on this subject and yielded sev- eral texts that treat the mathematics and its application to statistics in some depth in detail. Especially, Shinnozaki et. al.(1972a, 1972b) have surveyed the general methods to find the Moore-Penrose inverse in two subject: direct methods and iterative methods. Also they tested their accuracy for some cases. The Moore-Penrose inverse is a theoretical tool in statistical application where singular matrices occur. For instance, the design matrix in the general linear model is not of full rank. Kim and Lee(1986) found the Moore-Penrose inverse of design matrix using the Greville's method and from this the minimum norm least squares solution of the normal equations is estimated for one- and two-way factorial designs. In this paper a simple derivation of the Moore-Penrose inverse matrices of matrices with special form which arise in statistics, especially called the design matrices of linear models, is described using the relationship between the Moore-Penrose inverse and the minimum norm least squares solution. In section 2 we introduce the Moore-Penrose inverse of a matrix and describe the relationship between the Moore-Penrose inverse and the minimum norm least squares solution. This relationship is very useful to find the explicit form of ,the Moore- Penrose inverse of matrix that has special form. In particular the design matrices of the

* Korea Advanced Institute of Science and Technology , Seoul, Korea. Key words : design matrix; linear model; M-P inverse; M-P solution.

1 BYUNG CHUN KIM and HA SIK SUNWOO linear models can be represented in some regular arrangements of 0's and 1's. Then we can find the general form of the solutions of the normal equations. From this result we can find the explicit form of the Moore-Penrose inverse of the design matrix. In section 3 this procedure will be described in detail. Also some examples will be given.

2. Preliminaries Given any matrix X, there is a unique matrix G such that

(i) XGX = X, (ii) GXG = G, (iii) (XC)' = XG, (iv) (CX)' = GX which is called the Moore-Penrose inverse of X and is denoted by Xt Any matrix satisfies (i) is a generalized inverse of X and denoted by X-. Hereafter M-P inverse means the Moore-Penrose inverse. The M-P inverse plays an important role in statistics. The minimum least squares solution to the inconsistent linear equations Xb = y is defined to be the solution that has the minimum norm in the class of the least squares solution. Then the following two propositions tell us the close relationship between the M-P inverse and the minimum norm least squares solution of the equation Xb = y. Proposition 2.1. If Xb = y is consistent, then b = X+y is the minimum norm least squares solution. The proof of proposition 2.1 can be easily verified. Proposition 2.2. Suppose that b = Wy is a solution of the equation X'Xb = X'y where W is independent of y. Then b = W y is the minimum norm least squares solution if and only if W = X+. The proof of proposition 2.2 can be found in Rao and Mitra(1971). Because of this reason the minimum norm least squares solution is often called the M-P solution. In most cases the direct derivation of the M-P inverse is very difficult, but if we use the proposition 2.2 to obtain the explicit form of the M-P solution, the M-P inverse is simply the premultiplied matrix of the solution. Hereafter we shall use the following notations. 1n is the n-dimensional column vector of unities. In is the identity matrix of order n. Jn1Xnz = 1n11n2.

3. The M-P inverse In this section we will find the M-P inverse of a matrix satisfying some conditions. Theorem 3.1. If, for N x mj matrices {X11i= 0, 1, ... , k} providedmo =1, it hold that

and

2 The Moore-Penrose inverse matrix for the balanced ANOVA models

then the M -P inverse X+ of the partitioned matrix X = [1N X1 X2 Xk] is given by

(3.1)

where (3.2)

Proof. For simplicit we write

Js, = Jm~ xm,, Ji . = Jm1XN.

The conditions given in the theorem yields

Consider the consistent equations X'Xb = X'y for some non-zero vector y. Then we can easily show that the general form of the solution is given by

wherex1i x2, ... , xk arearbitrary numbers and Y =1Ny. Notethat therank of thecoehi- cientmatrix X'X is 1+ ~k 1(m2-1) whereasthe system consists of 1 + 1m~ equations with1 + ~k 1m1 unknowns. From the proposition 2.2, the M-P solution is given by minimizing the norm b'b which is a quadratic form of x1i x2, ... , xk. Differentiating b'b with respect to x1i x2,.. . , xk and setting the result to zero yield

j = 1,2,...,k.

Hence the norm b'b attains its minimum for

3. BYUNG CHUN KIM and HA SIK SUN WOO

(3.3)

where (3.4)

r Then by proposition 2.2 the matrix in (3.3) is the M-P inverse of X.

As an application consider the balanced multi-way factorial design with no interactions and n observations per cell. Then the model equation can be expressed as the form

Y= 1NIL+X1/91+...+XkQk+E where an m= x 1 vector of parameters of i-th factor; X1: an N x mi incidence matrix of i-th factor; E : an N x 1 vector of random errors; y : an N x 1 vector of observations; N : the total number of observations (= n f ms); n : the number of replications (> 1).

The incidence matrices can be expressed using Kronecker products of identity matrix and of matrices with all elements equal to 1. The Kronecker or direct product of the m x n matrix A and the p x q matrix B is the mp x nq matrix with mn submatrices a13 B; it is symbolized by A ® B = (a1,B). This product is associative but not commutative. If we write the design matrix X as

J(. (jN A'1 Jf2 ... XkJ

Then Xi=1m1®...®Im;®...®lmk®1n for i=1,2,...,k.

It is easy to see that these matrices satisfy the conditions of theorem 3.1, therefore the M-P inverse of X is

where Ms is defined by the same as in (3.4). This method can be applied to every balanced linear models with no interactions. But for the one way model the similar procedure can be applied whether the model is balanced or not. These are summarized as examples.

4 The Moore-Penrose inverse matrix for the balanced ANOVA models

Example 1. One way classification model Consider the one way classification model with a levels

y=1xµ+X,O+E where this model may not be balanced, say, nt (> 1) replications for each level i. If we let

d' = (n1 , n2, ... , na) D = diag(n1, n2~ •, na)

Then the normal equations become

Therefore the general solution of the normal equations is represented as the form

for arbitray number x. Some elementary calculations give us the value of x that minimizes the norm b'b as follows:

Hence the M-P inverse of [1 X1 is

(3.5) by proposition 2.2. If the model is balanced, that is, if D = nI, then W1 is

(3.6)

Example 2. The additive Latin square arrangements The general formulation of the model for the additive Latin square arrangements is

y= lµ+Xia+X218+X3'y+E where a' = (ai, ..., at), N' = (/31,. . . , fit), f = (')'1,... , 7t~

Vo = It and Vi = Q;(It); a cyclic change of the rows of It.

5. BYUNG CHUN KIM and HA SIK SUNW00

It is easy to see that the design matrix satisfies the conditions of theorem 3.1 with Mo = 3/t and Mi = 2/t for i =1, 2, 3. Therefore the M-P inverse of the matrix [1 Xl X2 X3] is given by

(s.7)

4. Conclusions The M-P solution has many advantages in many areas, especially in statistics as men- tioned earlier. For a given linear system if the coefficient matrix has certain regular ar- rangement, its M-P inverse and the solution have very close relationship. For a balanced linear model with no interactions the general solution of the normal equations can be found with some auxiliary parameters. Under the relationships between the M-P inverse and the solution we have given the explicit form of the M-P inverse of the design matrix.

Acknowledgements The authors wish to thank the referees and the associate editor for their helpful sug- gestions.

REFERENCES

Albert, A.(1972). Regression and the Moore-Penrose Inverse, New York: Academic Press. Ben-Israel, A. and Greville, T. N. E.(1974). GeneralizedInverses: Theory and Applications, New York: John Wiley. Graybill, G.(1976). Theory and Applications of the Linear Model, North Scituate, Mass.: Duxbury Press. Kim, B. C. and Lee, J. T.(1986). The Moore-Penrose Inverse for the Classification Models, J. of the Korean Statist., 15, 46-61. Lawson, C. and Hanson, R.(1974). Solving Least Squares Problems, Englewood Cliffs: N. J.: Prentice-Hall. Moore, E.(1920). Abstract, Bull. Amer. Math. Soc., 26, 394-395. Penrose, R. (1955). A Generalized Inverse for Matrices, Proc. Cambridge Philos. Soc., 51, 406-413. Pringle, R. and Raynor, A.(1971). Generalized Inverse for Matrices with Applications in Statistics, New York: Hafner. Rao, C. R.(1975). Linear Statistical Inference and Its Applications, New York: John Wiley,. Rao, C. R. and Mitra, S.(1971). GeneralizedInverse of Matrices with Applications in Statis- tics, New York: John Wiley. Schmidt, F.(1976). Econometrics, New York: Marcel Dekker. Shinozaki, N., Sibuya, M., and Tanabe, K.(1972a), Numerical algorithms for the Moore- Penrose inverse of a matrix: direct methods, Ann. Inst. Statist. Math., 24, 193-203.

6 The Moore-Penrose inverse matrix for the balanced ANOVA models

Shinozaki, N., Sibuya, M., and Tanabe, K.(1972b), Numerical algorithms for the Moore- Penrose inverse of a matrix: iterative methods, Ann. Inst. Statist. Math., 24, 621-629. Searle, S. R.(1971). Linear Models, New York: John Wiley and Sons. Searle, S. R.(1982). Matrix Algebra Usefulfor Statistics, New York: John Wiley and Sons.

(Received August 1989;Revised January 1990.)