Generalized Linear Models The general linear model approach used so far: • consists of specifications for the mean and covariance struc- ture; • mean response is a linear function of parameters and covari- ates; • has strong theoretical support when the responses are mul- tivariate normal; • works well when the response is continuous. 1 Generalized Linear Models When the response is discrete, or the mean response is naturally modeled as nonlinear in the parameters, we need an alternative. Other members of the exponential family of distributions provide various alternatives. Each distribution has a natural way to relate the mean response to the parameters and covariates, and implies a relationship be- tween the mean response and the variance. Begin with independent responses; longitudinal data handled by allowing for dependence or with random effects. 2 Distributions The most important distributions are: • normal • Bernoulli • binomial • Poisson where the last three are used for modeling count data. 3 Variance Functions For each of these, the variance is in the form φv(µ) for some dispersion parameter φ and variance function v(µ): • normal: Var(Y ) = σ2, so v(µ) = 1, φ = σ2; • Bernoulli: E(Y ) = p, Var(Y ) = p(1 − p), so v(µ) = µ(1 − µ), φ = 1; • binomial: E(Y ) = np, Var(Y ) = np(1−p), so v(µ) = µ(n − µ)/n, φ = 1; • Poisson: E(Y ) = Var(Y ) = µ, so v(µ) = µ, φ = 1. 4 Link Functions The link function links the mean response to the parameters and covariates: 0 g(µ) = η = β1X1 + β2X2 + ··· + βpXp = X β. Each distribution has a canonical link function, derived from its natural parametrization in the exponential family. Other link functions can also be used, when the subject matter suggests one. Distribution Variance Function, v(µ) Canonical Link Normal v(µ) = 1 Identity: µ = η µ Bernoulli v(µ) = µ(1 − µ) Logit: log 1−µ = η Poission v(µ) = µ Log: log(µ) = η 5 Logistic Regression • Response Y is Bernoulli: P(Y = 1) = 1 − P(Y = 0) = µ. • Variance function: Var(Y ) = µ(1 − µ) (whence φ = 1). • Covariates: µ = µ(X, β). • Require 0 ≤ µ(X, β) ≤ 1: rules out linear model µ = X0β. 6 Logistic Link Function The log odds ratio µ ! log = logit(µ) = η 1 − µ can be solved for µ: exp(η) µ = . 1 + exp(η) Any value for the log odds gives µ satisfying 0 < µ < 1. So a linear model for logit(µ) η = X0β always gives valid values of µ. 7 Interpretation of Parameters If one covariate changes by +1 and the others are not changed, η changes by the corresponding β coefficient. If µ ≈ 0, logit(µ) ≈ log(µ), so log(µ) also changes by approxi- mately β. If β is also small, a change of β in log(µ) is approximately a fractional change of β in µ. For example, if β = 0.1, a change of +1 in the covariate implies a 10% increase in µ. At the other extreme, if µ ≈ 1, logit(µ) ≈ − log(1 − µ). So now, if β is still 0.1, a change of +1 in the covariate implies a 10% decrease in 1 − µ. 8 Binomial Case If Y1,Y2,...,Ym are independent Bernoulli with the same covari- ates (or at least, the same µ), then Y = Y1 + Y2 + ··· + Ym is Bin(m, µ). • The logistic link function is still reasonable. • The binomial variance function is Var(Y ) = mµ(1 − µ). 9 • In practice, Y is often over-dispersed: Var(Y ) > mµ(1 − µ). • The variance function therefore often includes the dispersion parameter: v(µ) = φmµ(1 − µ). • Over-dispersion can be caused by: dependence: conditionally on the covariates, the individual Yjs are not independent (e.g., common cause of failure); heterogeneity: the individual Yjs do not have exactly the same covariates, or covariates are measured with error. 10.