Lecture 58 Generalized Linear Modeling Non-Normal Distributions

11/2/2016

CHE384, From Data to Decisions: Measurement, Generalized Linear Model (GLM) Uncertainty, Analysis, and Modeling

• Let yi have any probability distribution so long as it Lecture 58 is from the “exponential” family – e.g., normal, log-normal, exponential, gamma, Generalized Linear Modeling chi-squared, beta, Bernoulli, Poisson, etc. – Not included are Student’s t, mixed distributions Chris A. Mack • Allow for any transformation of y (the link function) Adjunct Associate Professor – Must be monotonic and differentiable • Linear in the model parameters Link function http://www.lithoguru.com/scientist/statistics/ ⋯ © Chris Mack, 2016Data to Decisions 1 © Chris Mack, 2016Data to Decisions 2

Non-Normal Distributions Typical Distribution/Link Pairings

• For a non-normal distribution, Least Distribution Typical uses Link Name Link function Squares ≠ MLE Normal Linear-response data Identity • Combine any link function with any Exponential Exponential-response data, Inverse 1/ (exponential family) probability distribution or Gamma scale parameters for y, then find the maximum likelihood count of occurrences in Poisson Log ln estimates for the parameters fixed amount of time/space – Solve with iteratively reweighted least squares Bernoulli or outcome of single yes/no Logit ln – Many software packages can do this Binomial occurrence (logistic model) regression

What if our Response is Binary? Predicting Proportions • Sometimes the response is binary • We want to predict a proportion (or – The patient lives or dies probability), p, for a categorical variable – The part passes or fails – Fraction of people that die of a heart attack – The customer buys or doesn’t buy – Fraction of molecules that decompose • The response y will follow a Bernoulli • Consider the following linear model distribution Suppose ⋯ – , the probability of “success” (y = 1) – 1 (a function of the mean) • Problem: p is constrained to between 0 and 1 • We want to model the probability of success – This model does not force a constraint on – – Additionally, the variance is not constant

1 11/2/2016

Probit Regression Logistic Distribution

/ 1 • Instead of a linear model, assume p is 1 / 1 / sigmoidally shaped 1.2 1.0 – Note that virtually every cdf 0.8 PDF CDF (cumulative distribution function) 0.6 0.4

has a sigmoidal shape 0.2

0.0 • Example: Probit model -4 -2 0 2 4 – Assume p is Bernoulli distributed, then 3 ⋯ Excess kurtosis = 1.2 Link function

Logistic Distribution Logistic Regression

• The logistic cdf can be analytically inverted • Instead of the probit model’s inverse normal function, use the inverse logistic cdf (and 1 / ln assume p is Bernoulli distributed) 1 1 ln ⋯ • We’ll model p, our output fraction of a binary 1 variable, with this cdf • Note: • This is called logistic regression

Uses of Logistic Regression Lecture 58: What have we learned?

• Predict binary outcome events • What are the three requirements of a – Patient mortality after surgery (as a function of generalized linear model? age, prior health indicators, sex) • What are some common distribution/link – Probability of contracting a certain disease (as a function of age, ethnicity, fitness, sex) function pairs? – Probability customer will make a purchase (as • What is the distribution/link function pair a function of income, age, sex, neighborhood) for logistic regression? – Probability of defaulting on a mortgage (as a function of price, income, interest rate, • Name three examples of where you might mortgage type) want to use a logistic regression