Tutorial in Quantitative Methods for Psychology 2008, Vol. 4(2), p. 65‐78. General Linear Models: An Integrated Approach to Statistics Sylvain Chartier Andrew Faulkner University of Ottawa Generally, in psychology, the various statistical analyses are taught independently from each other. As a consequence, students struggle to learn new statistical analyses, in contexts that differ from their textbooks. This paper gives a short introduction to the general linear model (GLM), in which it is showed that ANOVA (one‐way, factorial, repeated measure and analysis of covariance) is simply a multiple correlation/regression analysis (MCRA). Generalizations to other cases, such as multivariate and nonlinear analysis, are also discussed. It can easily be shown that every popular linear analysis can be derived from understanding MCRA. The most commonly used statistical analyses are concepts – which is absolutely necessary to do any work in ANOVA, t‐tests, and regressions (Cousineau, 2005). These research. According to Tatsuoka (1988): are taught as independent modules during the training of “… much is to be gained by the student’s going psychology students, which sadly fragments their through the calculations by hand … Students who knowledge into pieces that seem different and disconnected have undergone this sort of learning experience will from one another. Multivariate statistic books are no be more likely to develop a thorough understanding stranger to this practice (e.g. Howell, 2002; Shavelson, 1996; of the major steps involved in a sequence of Stevens, 1992; Tabachnick & Fidell, 2001) – the vast majority computation than will those who, from the outset, give only a very brief introduction to the subject. This is leave all the “busy work” to the computer.” even worse in univariate books, in which even such cursory Consequently, after their mandatory courses, students still treatment is lacking. With this is combined with the fact that usually have difficulties choosing the correct statistical test mathematical reasoning is not a priority in most psychology for their data. A student who masters software does not curricula (Giguère, Hélie, & Cousineau, 2004), the net result indicate his comprehension – all he/she shows is a bit of is a general misunderstanding of statistics among many technical competence. students of social science. Although most psychology teachers know that ANOVA In addition, more time is devoted to using how‐to‐do‐it and regression are linked through the general linear model computer tools (due to their accessibility and user‐friendly (GLM), few actually teach it in their courses. GLM offers a interfaces), and less time is spent on understanding the unique pedagogical perspective to provide such a unified view of statistical testing. To provide a more detailed explanation, it will be shown that ANOVA ‐ as well as the t‐ Address correspondence to Sylvain Chartier, University of test ‐ are simply special cases of multiple Ottawa, School of Psychology, 125 University, Ottawa, correlation/regression analysis (MCRA). Table 1 shows the Ontario, K1N 6N5, E‐mail: [email protected]. The different analyses that can be derived from MCRA given the authors are grateful to an anonymous reviewer and Jean‐ number of independent variables (IV), and the number of François Ferry for their help in reviewing the manuscript dependant variables (DV). Therefore, if the IV is nominal and Jean‐François Allaire and Isabelle Smith for their and there is only one continuous DV, the ANOVA is a comments in reviewing a previous version of the special case of MCRA. Thus, a general framework can be manuscript. 65 66 Table 1. Univariate and multivariate representations of the GLM. DV Form IV Form Type of analysis 1 nominal 1 nominal Phi coefficient / Chi‐square 1 continuous 1 nominal t‐test 1 nominal ≥1 continuous and/or Logistic regression / GLM nominal Discriminant function 1 continuous 1 continuous Simple correlation / regression 1 continuous ≥2 nominal ANOVA Univariate (one‐wa y, factorial, repeated measure) 1 continuous ≥2 continuous and ANCOVA nominal 1 continuous ≥2 continuous Multiple correlation / regression ≥ 2 nominal ≥2 nominal Correspondence ≥ 2 nominal ≥2 continuous and/or Multivariate logistic regression / Discriminant nominal functions ≥ 2 continuous ≥2 nominal MANOVA GLM ≥ 2 continuous ≥1 latent Principal component / Factor ≥ 2 continuous ≥1 latent Multidimensional scaling Multivariate and/or nominal ≥ 2 continuous ≥1 continuous and/or Structural equation modeling and/or latent latent ≥ 2 continuous ≥2 continuous Canonical correlation Note: DV, dependent va riable, IV, independent variable taught in which statistical methods are viewed as a whole, ANOVA (repeated, one‐way, factorial, covariance), the same which would facilitate their comprehension and application. three steps are always involved: compute the appropriate However, to really understand statistical analysis, linear coding matrix, create the SSCP matrix, and calculate the R‐ algebra must be used, which, paradoxically, is not part of squared. More precisely, a description and review of the mandatory psychology curriculum in many universities. multiple correlation analysis (MCRA) and simple This once again reinforces the disadvantage that psychology correlation/regression analysis (SCRA) is provided. students (unwittingly) endure in their training, compared to Subsequently, ANOVA, factorial ANOVA, and repeated‐ other sciences (Giguère, Hélie, & Cousineau, 2004). While is measures are presented as special cases of MCRA, in that should be noted that linear algebra is briefly introduced in order. The second part of the paper asserts the various links some multivariate statistics books (e.g. Tabachnick & Fidell, between MCRA and ANOVA with some numerical 2001; Tatsuoka, 1988), far stronger material is found in books examples using one‐way ANOVA and repeated‐measures about linear algebra itself (e.g. Lipschutz & Lipson, 2001; ANOVA. The final section discusses the multivariate case Strang, 1988). and nonlinear analysis (the generalized linear model). The information in this paper is primarily a synthesis of knowledge first presented in Cohen & Cohen (1983), Multiple Correlation/Regression Analysis Tatsuoka (1988), Kutner, Nachtsheim, Neter, & Li (2005), The purpose of MCRA is to determine the strength of and Morrison (1976), and it is mainly divided into three correlation between a criterion (the dependent variable) and parts. The first shows that when using MCRA or any kind of multiple predictors (the independent variables). If a 67 functional relationship is desired, then multiple regression junction of the second row and second column will be used analysis can be performed. to estimate the criterion variance: ⎡ nn⎤ ()x −−xxxy2 ()()−y Sum of square and cross product (SSCP) matrix and the R‐ ⎢ ∑∑iii⎥ ⎢ ii==11⎥ squared SSCPSCRA = nn (5) ⎢ 2 ⎥ ⎢∑∑()()yyxxii−− () yyi − ⎥ Different variables can be expressed, using standard ⎣⎢ ii==11⎦⎥ matrix notations, as follows. The predictor variables are where x and y represent the mean of the predictors and of defined as: the criterion, respectively. ⎡⎤x11xx 12 L 1p ⎢⎥ By partitioning the SSCP matrix correctly, the coefficient x21xx 22 L 2 p 2 X = ⎢⎥ (1) of determination ‐ R‐squared (R ) ‐ can be obtained. This is ⎢⎥MMOM ⎢⎥ done by dividing the SSCP into four sectors, which we x xx ⎣⎦⎢⎥nn12L np name Spp, Spc, Scp, Scc. These are (in order), the sum of squares and the criterion variable is defined as: of the predictors alone, the sum of cross‐products between ⎡⎤y1 the predictors and the criterion, the sum of cross‐products ⎢⎥ y between the criterion and the predictors (note that Scp = SpcT), y = ⎢⎥2 (2) ⎢⎥M and finally the sum of squares of the criterion alone. ⎢⎥ ⎣⎦yn ⎡sspp pc ⎤ SSCP = ⎢ ⎥ (6) where X is matrix of dimension n×p, y a matrix (vector) of ⎢ss⎥ ⎣ cp cc ⎦ 1×n and, n and p are the number of participants and Once this is drawn up, the coefficient of determination predictors, respectively. These two matrices can be put into (0≤R2≤1) can be obtained with the following matrix a single matrix : multiplication: ⎡⎤x11xx 12 L 1p y 1 2T1−−1 R = SSSS (7) ⎢⎥x xxy pcpppccc ⎢⎥21 22 L 2p 2 M = (3) In the particular case of SCRA, each element of the SSCP ⎢⎥MMOMM ⎢⎥matrix is a scalar, and thus the R2 will also be a scalar. It is ⎣⎦⎢⎥xnn12xxL npny given by the following: where xij represents the jth predictor of the ith participant, yi nn−1 th 22⎛⎞ the criterion of the i participant. From the M matrix, the RyyxxxxSCRA =−−∑∑()()()i i ⎜⎟i − ii==11⎝⎠ sum of squares and cross product (SSCP) matrix can be nn−1 computed. Useful information such as the variance, ⎛2 ⎞ ×−−∑∑()()()xxyyii⎜ yy i −⎟ (8) covariance, and R‐squared can also be extracted from the ii==11⎝⎠ 2 SSCP matrix, which is obtained by: ⎛⎞n ()()xxyy−− SSCP=−()() M MT M − M ⎜⎟∑ ii = ⎝⎠i=1 TT nn =−MM MM (4) 22 ∑∑()()xxii−− yy =−MMTTTT()()/ 1M 1M n ii==11 where 1 is defined as a vector (of dimension n) in which all Equation 8 is the standard way to obtain the coefficient of elements are equal to 1, M is a means‐score matrix (of determination. If the standard bivariate correlation (RSCRA) is 2 dimension n × p+1) where the mean of each column is desired, then one must find the square root of R SCRA, as repeated over n lines, and T denotes the matrix transpose found by equation 8, which must then be multiplied by the operation. If the SSCP is divided by the corresponding sign (+ or ‐) of Spc (direction of the covariance). 2 R2 degrees of freedom (n‐1), then the variance/covariance For an unbiased estimate of R ( % ), the following matrix is obtained. Thus, the SSCP is a convenient way to correction must be applied: (1−−Rn2 )( 1) represent a lot of information about variability in a single R% 2 =−1 (9) (1)np−− matrix. Naturally, the same can be found for SCRA using Equation 4. In that case, the matrix will be reduced to a 2 by This is called the shrunken R‐squared, or the adjusted R‐ 2 format.