Regression Analysis Chih-Chiang Yang, Ph.D. [email protected] 27016855x2151

Correlation Coefficient

 Pearson’s Product-moment Correlation Coefficient

 Spearman’s Correlation Coefficient

 Point-biserial Correlation Coefficient

 Biserial Correlation Coefficient

 Phi Coefficient

Testing Hypothesis for ρ

Terms for Regression Analysis

 Regression Analysis

 Dependent Variable/Explained Variable/Regressand/y

 Independent Variable/Explanatory Variable/Predictor/xi

 Simple Regression

 Multiple Regression

 Linear Regression

 Non-linear Regression

 Simple Linear Regression

 Multiple Linear Regression

Simple Linear Regression

2  y0   1 x   ；~NID (0,  ) ˆ ˆ  Estimations of Parameters（OLS）： yˆ   0  1 x1

1 ANOVA for Simple Linear Regression

 ANOVA Table Source SS df MS F P Regression SSR 1 MSR MSR P[F>F*] F *  MSE Residual SSE n-2 MSE Total SST n-1

 Testing Hypothesis for i s

 Testing Hypothesis for 1

 Testing Hypothesis for 0

2 SSR  Coefficient of Determination： R  SST

Model Adequacy Checking

 Residual Analysis  Normal Plot

 Plot of Residuals Against yˆi

 Plot of Residuals Against xi  Plot of Residuals Against Times

 Detection of Outliers

 Lack of Fit  Transformation to a Straight Line

Multiple Linear Regression

2  y0   1 x 1   2 x 2 ...  k x k   ；~NID (0,  ) ˆ ˆ ˆ ˆ  Estimations of Parameters（OLS）： yˆ   0  1 x1   2 x2  ...   k xk

ANOVA for Multiple Linear Regression

 ANOVA Table Source SS df MS F P

2 Regression SSR k MSR MSR P[F>F*] F *  MSE Residual SSE n-k-1 MSE Total SST n-1

 Testing Hypothesis for i s

 Testing Hypothesis for i

2 SSR  Coefficient of Determination： R  SST

2n 1 2  Adjusted R1  (1  R ) n p

Stepwise Regression Selection Procedures

 Forward Selection

 Backward Selection

 Stepwise Selection

Multicollinearity Diagnostics

 Examination of the Correlation Matrix

 Variance of Inflation Factors (VIF)

 Eigenvalues

Dealing with Multicollinearity

 Collecting Additional Data

 Model Respecification/Variable Elimination

 Ridge Regression

 Generalized Ridge Regression

 Principal Component Regression

 Latent Root Regression

3 Other Topics

 Autocorrelation

 Robust Regression

 Nonlinear Regression

 Logistic Regression

4