<p>Simple Linear Regression</p><p>Regression Analysis – a statistical technique that attempts to model, that is specify, the relationship between a dependent variable and one or more variables.</p><p>Linear Regression – </p><p>Ordinary Least Squares (OLS) Regression</p><p>Slope (Regression Coefficient): cov(xy) sd(y) cov(xy) b   r(xy)  1 var(x) sd(x) sd(x)sd(x)</p><p> cov(xy) r  sd(x)sd(y)</p><p>The least squares slope is equal to the Cross Product of X and Y divided by the Total Sum of Squared Deviations of X from its Mean.</p><p> cov(xy)  (X  X )(Y Y ) / n 1  (X  X )(Y Y ) CP(xy) b     1 var(x)  (X  X ) 2 / n 1  (X  X ) 2 TSS(x)</p><p>Standardized slope coefficient in bivariate regression. sd(x) b  r(xy)  b* 1 sd(y) 1</p><p>The unstandardized coefficient of X represent the unit change in Y, resulting from a 1 unit change in X. The standardized coefficient of X represent the standard deviation change in Y, resulting from a 1 standard deviation change in X.</p><p>The correlation coefficient represent the bivariate relationship between X and Y. In simple regression, the standardized regression coefficient represent the bivariate relationship between X and Y in standard deviation units. In bivariate regression, the standardized regression coefficient is equal to the correlation coefficient.</p><p>Y-Intercept b0  Y  b1 X</p><p>Regression Equation:</p><p>Population:</p><p>Y    X Y    X   </p><p>Sample: ˆ ˆ Y  a  bX  e Y  a  bX Y  b0  b1 X</p><p>Y-hat</p><p>The estimated regression line passes through the means.</p><p>If there is no association between X and Y, that is the correlation coefficient is equal to zero, then the regression slope is zero (0), the intercept is the mean of Y and the best predictor of Y remains the mean of Y.</p><p> sd(y)  sd(y)  b1  r(xy)  0   0 sd(x)  sd(x) </p><p> b0  Y  b1 X  Y  0(X )  Y  0  Y ˆ Y  b0  b1 X  Y  0(X )  Y  0  Y</p><p>Estimated Error – a) the difference between the observed value of Y and the predicted value of Y from the regression equation. The sum of the squared errors is the Residual Sum of Squares. The difference between the Total Sum of Squares and the Residual Sum of Squares is the Explained Sum of Squares. The sum of the residuals is zero.</p><p>Another way to calculate the Coefficient of Determination (See above) is ESS divided by TSS, or the proportion of the variation in the observed values of Y that is explained by the regression.</p><p>2 TSS  (Yi Y ) e  (Y Yˆ)</p><p>OLS minimizes RSS. OLS selects b0 and b1 such that the RSS is as small as possible  e 2   (Y Yˆ) 2  RSS TSS  RSS  ESS</p><p>ESS /TSS  R 2 ˆ 2 ESS  TSS  RSS  (Yi Yi ) RSS s  e n  k</p><p> s s  e bx</p><p> s x b   t  Null</p><p> sb</p><p>H1:  1  0 b t </p><p> sb</p><p>1 X 2 sb  se  0 n 1 2  (X i  X )</p><p>Multiple Regression – a) the specification of a linear equation that links multiple independent variables to a dependent variable and includes the Y-Intercept, the slopes, and (sometimes) error.</p><p>ˆ Y  b0  b1 X 1  b2 X 2 ...bk 1 X k 1  e</p><p>Adjusted Rsquared</p><p>ˆ 2 (Yi Yi ) /(N  K 1) R 2  2  (Yi Yi ) /(N 1) </p>