Introduction the General “General Linear Model” Linear Model (GLM) • “Model” means that we are usually interested in predicting or “modeling” the values of one variable (criteria) from the values of one or more others (the predictors)
z what “model,” “linear” & “general” mean • “Linear” means that the variables will be “linearly transformed (* & /) and “linearly combined” (+ & -) to produce the z bivariate, univariate & multivariate GLModels model’s estimates z kinds of variables
z some common models • “General” means that the model intends to provide a way to model & test RHs: about any combination of criterion and predictor variables (i.e., any model), and to test RHs: about comparisons among models
Regression “vs.” GLM
The “constant” is often represented differently in GLM than in multiple regression …
Single predictor models
Æ single predictor regression y’ = bx + a
Æ single predictor GLM y’ = b0 + b1x1 Multiple predictor models
Æmultiple predictor regression y’ = b1x1 b2x2+ a
Æ multiple predictor GLM y’ = b0 + b1x1 + b2x2 Common kinds of GLModels Common kinds of variables •BivariateÆ one criterion & one predictor Quantitative variables Æ simple regression •Raw variable y’ = b0 + b1x • Centered variables X – mean •Univariate Æ one criterion & multiple predictors Æ multiple regression in all its forms •Mean Æ 0 simplifies math of more complicated models y’ = b + b x + b x + b x 0 1 1 2 2 3 3 • Re-centered variables X Æ a more meaningful value •Multivariate Æ multiple criterion & multiple predictors • Change “start” or “stop” values Æ canonical regression in all its forms • E.g., “aging & intellectual decline” • Mathematical “trick” to get the desired model/weights b0 + b1y1 + b2y2 =b0 + b1x1 + b2x2 + b3x3 • selecting which group or value will be represented in model’s bs
Common kinds of variables
Quadratic quantitative variables
•X2 – actually represents combination of linear + quadratic
2 •Xcen – represents the “pure” quadratic term
2 2 2 • Model with X will have ≈ R as model with Xcen + Xcen
• A model with a quadratic term should always include the linear term for that variable Common kinds of variables Common kinds of variables 2-group variables k-group variables
• Raw coding (usually 1-2-3, etc.) • Unit coding (usually 1-2) • Dummy Coding • Dummy Coding • “control” or “comparison” group coded 0 • “treatment” or “target” groups coded 1 on one variable & 0 • “control” or “comparison” group coded 0 on all others • “treatment” or “target” group coded 1 • the full set of codes must be included in the model
•Effect Coding •Effect Coding • “control” or “comparison” group coded -1 • “control” or “comparison” group coded -1 • “treatment” or “target” group coded 1 on one variable & 0 on all others • “treatment” or “target” group coded 1 • the full set of codes must be included in the model
Common kinds of variables K-groups variables, cont.
• Comparison coding • Combining simple and complex analytical comparison codes to represent specific, hypothesis driven, group comparisons • E.g., Say you have 4 groups and RH: that… • Group 1 has higher scores that the average scores of groups 2-4 the codes would be gp1 = 3 gp2 = -1 gp3 = -1 gp4 = -1 • Groups 2 & 3 have higher average scores than do 1 & 4 the codes would be gp1 = -1 gp2 = 1 gp3 = 1 gp4 = -1 • Group 2 has higher scores than the average scores of groups 3-4 the codes would be gp1 = 0 gp2 = 2 gp3 = -1 gp4 = -1 • Usually havea set of k-1 codes Common kinds of variables Common kinds of variables K-groups variables, cont. Ordered-category variables Sometimes you have a quantitative variable that you want to Polynomial coding change into a set of ordered categories If the groups represent a quantitative continuum, you use e.g. Æ % grade into “A” “B” “C” “D” “F” codes to represent different polynomial functions (linear, e.g. Æ % grade into “Pass” “Fail” quadratic, cubic, etc.) to explore the shape of the e.g. Æ aptitude test scores into “remedial” “normal” “gifted” relationship between that variable and the criterion Sometimes this is done to help with “ill-behaved distributions” E.g., for a 5-group variable, the polynomial codes are … e.g. Æ frequency variable with mean=1.1, std=8.4, sk=4.2 • Linear -2 -1 0 1 2 e.g. Æ frequency variable with 60% “0” 38% “1” max = 118 • Quadratic 2 -1 -2 -1 2 • Cubic -1 2 0 -2 1 Important because Æ skewed univariate distributions can “create” • Quartic 1 -4 6 -4 1 apparently nonlinear bivariate relationships • the full set of codes must be included in the model
Common kinds of variables Ordered-category variables, cont. Once you form the ordered categories (using “IF,” “RECODE” or other transformations), you can enter those variables into the GLM in different ways • Using the category values (e.g., 1, 2, 3, etc) *** • Centering or re-centering the category values *** • Dummy codes of the category values • Effect codes of the category values • Polynomial codes of the category values***
*** indicates approaches that make assumptions about the interval nature of the variable and/or its normal distribution, with which not everyone agrees! Common kinds of variables Common kinds of variables Interactions Interaction Æ “Guidelines” • Interactions represent the “joint effect” or “non-additive • When including a 2-way interaction, both related main effects combination” of 2 or more predictors as they relate to a must be included criterion (or set of criteria in the multivariate case). • When including a 3-way interaction, all 3 main effects and all 3 • They are the “moderation,” “it depends,” “sometimes,” or 2-way interactions must be included “maybe” that makes our science and statistical analyses so • When including a non-linear interaction term, the related linear interesting. and nonlinear main effects, and linear interaction terms must • Interactions can be formed from the combination of any 2 or be included more variables of the types just discussed. • The associated terms can not exceed the df of the variables • There are some “guidelines” about forming, including and involved (except for quantitative variables) interpreting interaction terms.
Common kinds of GLModels
“Linear” Multiple regression models
y’ = b0 +b1x1 +b2x2 +b3x3
Can include any of the variable types: • Quantitative (raw, centered or re-centered) • 2- or k-group (with dummy, effect, or comparison coding) • Ordered category (coded) Common kinds of GLModels Common kinds of GLModels
“Non-Linear” Multiple regression models with quant variables “2-way Interaction” Multiple regression models
2 2 y’ = b0 +b1x1 +b2x1 +b3x2 + + b4x2 y’ = b0 +b1x+b2z+b3xz
Can include any of the variable types: Can include any of the variable types: • Linear terms should be centered • Quantitative variables should be centered • Non-linear terms should be centered then powered • 2- or k-group variables should be coded • Non-linear terms above quadratic should be based on theory • Interaction terms formed as product of main effect terms • Include linear term for all non-linear terms, at least at first • Must included main effects terms for any interaction variable
Common kinds of GLModels
“3-way Interaction” Multiple regression models
y’ = b0 +b1x+b2z+b3v+b4xz + b5xv + b6zv + b7xzv
Can include any of the variable types: • Quantitative variables should be centered • 2- or k-group variables should be coded • Interaction terms formed as product of main effect terms • Must included main effects terms for any interaction variable Common kinds of GLModels Common kinds of GLModels
2-group ANOVA models
3-group ANOVA models y’ = b0 +b1x1 y’ = b0 +b1x1 +b2x2 “X” is a dummy or effect coded 2-group variable
“X1”& “X2” are a dummy or effect codes for a 3-group variable
Common kinds of GLModels
4-group ANOVA models
y’ = b0 +b1x1 +b2x2 +b3x3
“X1,” “X2”& “X3” are a dummy or effect codes for a 4-group variable Common kinds of GLModels Common kinds of GLModels
2x2 Factorial ANOVA model 2x3 Factorial ANOVA model
y’ = b0 +b1x+b2z+b3xz y’ = b0 + b1x1 +b2z1 +b3z2 +b4xz1 +b5xz2
st “X” is a dummy or effect code of 1 2-group variable “X ” is a dummy or effect code of 1st 2-group variable nd 1 “Z” is a dummy or effect code of 2 2-group variable “Z ”& “Z” are dummy or effect codes of 2nd k-group variable “XZ” represents the interaction of “X” and “Z” 1 2 “XZ1”& “ZX2” represent the interaction of “X” and “Z”
Common kinds of GLModels
2-group ANCOVA models
y’ = b0 +b1x+b2z
“X” is a dummy or effect coded 2-group variable “Z” is the covariate (dummy coded or quantitative) Common kinds of GLModels Common kinds of GLModels
2-group ANCOVA models with covariate interaction 3-group ANCOVA models
y’ = b0 +b1x1 +b2x2 +b3z y’ = b0 +b1x+b2z+b3xz
“X1”& “X2” are a dummy or effect codes for a 3-group variable “X” is a dummy or effect coded 2-group variable “Z” is the covariate (dummy coded or quantitative) “Z” is the covariate (dummy coded or quantitative) “XZ” represents the interaction of “X” and “Z”
Common kinds of GLModels
3-group ANCOVA model with covariate interaction
y’ = b0 +b1x1 +b2x2 +b3z+b4xz1 +b5xz2
“X1”& “X2” are a dummy or effect codes for a 3-group variable “Z” is the covariate (dummy coded or quantitative)
“XZ1”& “ZX2” represent the interaction of “X” and “Z” Common kinds of GLModels Common kinds of GLModels
2x2 Factorial ANCOVA model 2x2 Factorial ANCOVA model with covariate interactions
y’ = b +bx+bz+bxz + b v+bxv + b zv + b xzv y’ = b0 +b1x+b2z+b3xz + b4v 0 1 2 3 4 5 6 7 “X” is a dummy or effect code of 1st 2-group variable “Z” is a dummy or effect code of 2nd 2-group variable “X” is a dummy or effect code of 1st 2-group variable “XZ” represents the interaction of “X” and “Z” “Z” is a dummy or effect code of 2nd 2-group variable “V” represents the covariate “XZ” represents the interaction of “X” and “Z” “V” represents the covariate “XV” represents the interaction of “X” and “V” “ZV” represents the interaction of “Z” and “V” “XZV” represents the interaction of “X,” “Z” and “V”