Announcements Announcements Unit 6: Simple Linear Regression Lecture 2: Outliers and inference Statistics 101 Project 2... Mine C¸etinkaya-Rundel April 4, 2013 Statistics 101 (Mine C¸etinkaya-Rundel) U6 - L2: Outliers and inference April 4, 2013 2 / 27 Types of outliers in linear regression Types of outliers in linear regression Types of outliers Types of outliers ●● ● 0 ● ● ● ● ● 10 ● ● ●● ● ●●●●● ● ● How do(es) the outlier(s) in- ● ● ● ● ● ● ● fluence the least squares ● ● ● ● ● line? −10 5 ● ●● ● ● How do(es) the outlier(s) in- ● ●● ● ● ● ●●●●● ● ● ● ●● ● ● ● ● ●● ● fluence the least squares ●● ●● ● ● ● ● ● ●● ●●● To answer this question ● ● ● ●● ● ●● ● ●● ●●● ● ●● ● 0 ● ●●● ● ● ● ● ●● ● ●●●● ● ● line? ● ● think of where the ● ● ● ● ● −20 ● ●● ● ● regression line would be with and without the ● ● ● ● ● ● ● 2 ● ● outlier(s). 5 ● ●● ● ● ● ● ●●●● ●● ● ● ●● ● ●● ●● ● ●● ●● ● ● ● ●●●● ● ● ●●● ●● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● 0 ● ● ● ● 0 ● ● ●●● ●● ● ● ● ● ● ●●● ●●● ●●●● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● −5 −2 ●● ●● Statistics 101 (Mine C¸etinkaya-Rundel) U6 - L2: Outliers and inference April 4, 2013 3 / 27 Statistics 101 (Mine C¸etinkaya-Rundel) U6 - L2: Outliers and inference April 4, 2013 4 / 27 Types of outliers in linear regression Types of outliers in linear regression Some terminology Influential points Data are available on the log of the surface temperature and the log of the light intensity of 47 stars in the star cluster CYG OB1. Outliers are points that fall away from the cloud of points. Outliers that fall horizontally away from the center of the cloud ● ● w/ outliers are called leverage points. 6.0 ● w/o outliers ● ● High leverage points that actually influence the slope of the ● ● ● ● 5.5 ● ●● regression line are called influential points. ● ● ● ● ● ● ● ● ● ● ● In order to determine if a point is influential, visualize the ● ● 5.0 ● ● regression line with and without the point. Does the slope of the ● ● ● ● ● ● ● line change considerably? If so, then the point is influential. If log(light intensity) 4.5 ● ● ● ● ● ● not, then it’s not. ● ● ● ● 4.0 ● 3.6 3.8 4.0 4.2 4.4 4.6 log(temp) Statistics 101 (Mine C¸etinkaya-Rundel) U6 - L2: Outliers and inference April 4, 2013 5 / 27 Statistics 101 (Mine C¸etinkaya-Rundel) U6 - L2: Outliers and inference April 4, 2013 6 / 27 Types of outliers in linear regression Types of outliers in linear regression Types of outliers Types of outliers 15 ●●●● ● 40 ●●● ● ● ● ● ● ● ● ●●●● ● ●●●●●● ● ● ● ● ●●● ● ● ● Clicker question ●●● ● ● ● ●●●● ● ● ●● ●●●● 10 ● ● ●● ● ● ● ● ●●●●● ● ●●●●●● ● ●●● ● ●●●● ● ●●●● 20 ●●●●●●● Which of the below best de- ● ●● ●● ●● ● ● ● ● ● ● ● ● ●●●● ● ● ●●●●● ● ● scribes the outlier? ●●●●● 5 ● ● ●● ●● ● ●● ● ● ● ● ●●●● ● ● ●● Does this outlier influence ● ●●●●●● ● ● 0 ● ●● ● ● the slope of the regression ● ● 0 ● (a) influential line? (b) leverage −20 ● ● (c) leverage −5 ● (d) none of the above ● ● ● ● 2 ● ● ● ●● ● ● 5 ● ●●● ● ●● ●● ● ● ● ● ● ● ●● ● ● ●● ● ● ●● ● ● ● ●● ● ● ● ● ● ●● ●● ● ● ● ● ● (e) there are no outliers ●● ●● ● ●●●● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0 ● ● ● ●●● ●● ● ● ●● 0 ● ● ● ● ● ● ●●●● ●● ● ● ● ●● ● ● ●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ●●● ● ● ● ● −2 ● ● ● −5 ● ● Statistics 101 (Mine C¸etinkaya-Rundel) U6 - L2: Outliers and inference April 4, 2013 7 / 27 Statistics 101 (Mine C¸etinkaya-Rundel) U6 - L2: Outliers and inference April 4, 2013 8 / 27 Types of outliers in linear regression Types of outliers in linear regression Recap Recap (cont.) Clicker question R = 0:08; R2 = 0:0064 R = 0:79; R2 = 0:6241 Which of following is true? 10 (a) Influential points always change the intercept of the regression 8 line. 6 2 (b) Influential points always reduce R . 4 ● (c) It is much more likely for a low leverage point to be influential, 2 ● ● ●● ● ● ●● ●●● ● ●● ● ●●●●● ●●● ●●● ● ● than a high leverage point. ●● ● ● ● ● ● 0 ● ● ●● ●● ●●● ●●●●● ● ● ●● ● ● ● ● ●● ● (d) When the data set includes an influential point, the relationship −2 ● between the explanatory variable and the response variable is ●● ● 2 ● ● ● ● ● ● ●● ● ● ● always nonlinear. ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ● ● ● ● ● ● 0 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● (e) None of the above. ● ● ● ● ● ● ● ● ● ● ● −2 ● Statistics 101 (Mine C¸etinkaya-Rundel) U6 - L2: Outliers and inference April 4, 2013 9 / 27 Statistics 101 (Mine C¸etinkaya-Rundel) U6 - L2: Outliers and inference April 4, 2013 10 / 27 Inference for linear regression Understanding regression output from software Inference for linear regression Understanding regression output from software Nature or nurture? Clicker question Which of the following is false? In 1966 Cyril Burt published a paper called “The genetic determination of differences in intelligence: A study of monozygotic twins reared apart?” The Coefficients: ● data consist of IQ scores for [an assumed random sample of] 27 identical Estimate Std. Error t value Pr(>|t|) 10 twins, one raised by foster parents, the other by the biological parents. (Intercept) 9.20760 9.29990 0.990 0.332 bioIQ 0.90144 0.09633 9.358 1.2e-09 130 ● R = 0.882 Residual standard error: 7.729 on5 25 degrees of freedom 120 ● ● Multiple R-squared: 0.7779, Adjusted R-squared: 0.769 ● ● ● ● ● ● 110 ● ● F-statistic: 87.56 on 1 and 25 DF, p-value:●●●●● 1.204e-09 ● ● ●● ● ● ● ●● ● ●● ●● ● ● ● ●● ●●● ● ●● ● ● ● ●● ●●● ● 100 ● 0 ● ●●● ● ● ● ● ●● ● ● ●●●● ● ● ● ● ● ● ● ● (a) For each 10 point increase in the biological● ●● twin’s IQ, we would ● 90 ● ● ● foster IQ foster ● ● ● expect the foster twin’s IQ to increase on average by 9 points. ● 80 ● ● (b) Roughly 78% of the foster twins’ IQs can be accurately predicted ● ● 70 ● ● ● ● by the model. 2 ● ● ● ●●●● ● ● ●● ● ●● ●● ●● ● ● [ ● ●● ●● (c) The linear model is fosterIQ = 9:2●● + 0:9 × bioIQ. 0 ● ● ●●●● ●● ● ●●● ●●●●● 70 80 90 100 110 120 130 ● ●● ● ●● ● ● ● (d) Foster twins with IQs higher than● average● ● IQs have biological −2 ●● biological IQ ● twins with higher than average IQs as● well. Statistics 101 (Mine C¸etinkaya-Rundel) U6 - L2: Outliers and inference April 4, 2013 11 / 27 Statistics 101 (Mine C¸etinkaya-Rundel) U6 - L2: Outliers and inference April 4, 2013 12 / 27 Inference for linear regression HT for the slope Inference for linear regression HT for the slope Testing for the slope Testing for the slope (cont.) Clicker question Estimate Std. Error t value Pr(>jtj) (Intercept) 9.2076 9.2999 0.99 0.3316 Assuming that these 27 twins comprise a representative sample of all bioIQ 0.9014 0.0963 9.36 0.0000 twins separated at birth, we would like to test if these data provide convincing evidence that the IQ of the biological twin is a significant We always use a t-test in inference for regression. predictor of IQ of the foster twin. What are the appropriate hypothe- − Remember: Test statistic, T = point estimate null value ses? SE Point estimate = b1 is the observed slope. (a) H : b = 0; H : b 0 0 0 A 0 , SEb1 is the standard error associated with the slope. (b) H0 : β1 = 0; HA : β1 , 0 Degrees of freedom associated with the slope is df = n − 2, (c) H0 : b1 = 0; HA : b1 , 0 where n is the sample size. Remember: We lose 1 degree of freedom for each parameter we estimate, and in simple (d) H0 : β0 = 0; HA : β0 , 0 linear regression we estimate 2 parameters, β0 and β1. Statistics 101 (Mine C¸etinkaya-Rundel) U6 - L2: Outliers and inference April 4, 2013 13 / 27 Statistics 101 (Mine C¸etinkaya-Rundel) U6 - L2: Outliers and inference April 4, 2013 14 / 27 Inference for linear regression HT for the slope Inference for linear regression HT for the slope Testing for the slope (cont.) % College graduate vs. % Hispanic in LA What can you say about the relationship between of % college gradu- ate and % Hispanic in a sample of 100 zip code areas in LA? Estimate Std. Error t value Pr(>jtj) (Intercept) 9.2076 9.2999 0.99 0.3316 Education: College graduate Race/Ethnicity: Hispanic 1.0 1.0 bioIQ 0.9014 0.0963 9.36 0.0000 0.8 0.8 0.9014 − 0 T = = 9.36 0.0963 0.6 0.6 df = 27 − 2 = 25 p − value = P(jTj > 9:36) < 0:01 0.4 0.4 0.2 0.2 Freeways Freeways No data No data 0.0 0.0 Statistics 101 (Mine C¸etinkaya-Rundel) U6 - L2: Outliers and inference April 4, 2013 15 / 27 Statistics 101 (Mine C¸etinkaya-Rundel) U6 - L2: Outliers and inference April 4, 2013 16 / 27 Inference for linear regression HT for the slope Inference for linear regression HT for the slope % College educated vs. % Hispanic in LA - another look % College educated vs. % Hispanic in LA - linear model What can you say about the relationship between of % college gradu- Clicker question ate and % Hispanic in a sample of 100 zip code areas in LA? Which of the below is the best interpretation of the slope? Estimate Std. Error t value Pr(>jtj) 100% (Intercept) 0.7290 0.0308 23.68 0.0000 %Hispanic -0.7527 0.0501 -15.01 0.0000 75% (a) A 1% increase in Hispanic residents in a zip code area in LA is 50% associated with a 75% decrease in % of college grads. (b) A 1% increase in Hispanic residents in a zip code area in LA is 25% associated with a 0.75% decrease in % of college grads. % College graduate (c) An additional 1% of Hispanic residents decreases the % of 0% college graduates in a zip code area in LA by 0.75%. 0% 25% 50% 75% 100% (d) In zip code areas with no Hispanic residents, % of college % Hispanic graduates is expected to be 75%. Statistics 101 (Mine C¸etinkaya-Rundel) U6 - L2: Outliers and inference April 4, 2013 17 / 27 Statistics 101 (Mine C¸etinkaya-Rundel) U6 - L2: Outliers and inference April 4, 2013 18 / 27 Inference for linear