DISPLAYING THE RELATIONSHIP
1 DEFINITIONS:
Studies are often conducted to attempt to show that some explanatory variable “causes” the values of some response variable to occur.
The response or dependent variable is the response of interest, the variable we want to predict, and is usually denoted by y.
The explanatory or independent variable attempts to explain the response and is usually denoted by x.
A scatterplot shows the relationship between two quantitative variables x and y. The values of the x variable are marked on the horizontal axis, and the values of the y variable are marked in the vertical axis. Each pair of observations (xi, yi), is represented as a point in the plot.
Two variables are said to be positively associated if, as x increases, the values of y tends to increase. Two variables are said to be negatively associated if, as x increases, the values of y tends to decrease.
When a scatterplot does not show a particular direction, neither positive, nor negative, we say that there is no linear association.
Scatterplot of Final vs Midterm Scores 100 90 80 Final 70 60 50 The 10th student 40 (21 , 38) 30 20 10 0 0 10 20 30 40 50 Midterm 2 X Y Student Midterm Final Number Score Score 1 39 62 2 44 69 3 32 68 4 40 86 5 45 88.5 6 46 88.5 7 33 76 8 39 66.5 9 32.5 75 10 21 38 11 30 71 12 39 88 13 44 96.5 14 28.5 71.5 15 38 96 16 43 82.5 17 42 85 18 25.5 28 19 47 95 20 36 39 21 31.5 58 22 32 49 23 42 62 24 21 59 25 41 90
Let's Do It! 1
The data below was obtained in a study of age and systolic blood pressure of six randomly selected subjects. Make a scatter plot to examine the relationship between (x) = age and (y) = pressure. Comment on the relationship with respect to form, direction, strength, and any departures or usual values.
Subject Age x Pressure y A 43 128 B 48 120 C 56 135
3 D 61 143 E 67 141 F 70 152
Notes of Caution 1. An observed relationship between two variables does not imply that there is some causal link between the two variables. For example, consider the following scatter-plot of IQ score versus shoe size:
IQ
4 Shoe Size
As a person ages their shoe size increases as well as their IQ. Although there is a positive association, there is no causal link between the two variables shoes size and IQ.
Most studies attempt to show that some explanatory variable "causes" the values of the response to occur. While we can never positively determine whether or not there is a distinct cause-and-effect relationship, we can assess if there appears to be such relationship.
2. A relationship between two variables can be influenced by confounding variables. Consider the following scatter-plot of the number of sport magazines read in a month versus the height of the person:
Number of magazines read
: women : men
Height
Overall there appears to be a positive association between height and number of magazines. However, if for each gender, there does not appear to be an association. Gender is a confounding variable and aggregating the datap. 811 across gender can result in misleading conclusions. Any study, especially an observational study, has the potential to be wrongly interpreted because of confounding variables.
3. Unusual data points (outliers) can mislead the association, especially if the data set is small. Consider the following scatter-plot of the percentage of people who speak English versus population size.
Percent who speak English Outlier
5 Population Size
The eight points in the scatter-plot represents eight countries from Central and South America selected at random. The outlier is Mexico City.
4. Sometimes a scatter plot, such as the one in Figure below, shows a curvilinear relationship between the data. In this situation, Methods for curvilinear relationships are beyond the scope of this course.
Simple Linear Regression
6 Scatterplot of Final vs Midterm Scores 100 90 Final 80 70 60 50 40 L i n e # 1 30 20 10 L i n e # 2 0
0 10 20 Midterm 30 40 50
So the question remains as to how to find a “best-fitting” line?
Equation of a Line y = a + b x where b = slope - the amount y changes when x is increased by 1 unit. a = y-intercept - the value of y when x is set equal to zero.
DEFINITION::
7 The least squares regression line, given by y a bx, is the line that makes the sum of the squared vertical deviations of the data points from the line as small as possible. Performing the regression is often stated as regress y on x.
Least squares regression line for regressing final exam scores, y, on midterm exam scores, x, is given by y 7.5 1.75x.
Estimated slope of b=1.75 tells us that for a 1-point increase on the midterm we would expect, on average, an increase of 1.75 points on the final exam.
Estimated y-intercept of a=7.5 tells us that if someone were to score 0 points on the midterm, we would predict they would get 7.5 points on the final exam.
Suppose a new student scores 40 points on the midterm. Based on our model, what would be their predicted final exam score? Plug the value of x=40 into our estimated equation. The predicted final Exam score is y 7.5 1.75(40) 77.5 points.
Let's Do It! 2 13.2 Childhood Growth
8 The growth of children from early childhood through adolescence generally follows a linear pattern. Data on the heights of female Americans during childhood, from four to nine years old, were compiled and the least squares regression line was obtained as y 80 6x, where y is height in centimeters and x is age in years. Note that 1 inch is equal to 2.54 centimeters.
(a) Interpret the value of the estimated slope b= 6.
(b) Would interpretation of the value of the estimated y- intercept, a= 80, make sense here? If yes, interpret it. If no, explain why not.
(c) What would you predict the height to be for a female American at 8 years old? Give your answer first in centimeters then in inches.
(d) What would you predict the height to be for a female American at 25 years old? Give your answer first in centimeters then in inches.
(e) Why do you think your answer to part (d) was so inaccurate?
Calculating the Least Squares Regression Line 9
The Least Squares Regression Line
The least squares regression line is given by y a bx where
xi x yi y n xi yi xi yi slope = b 2 2 2 xi x n xi xi
y – intercept = a y bx
Example Test 1 versus Test 2—Obtaining the Regression Line “By Hand”
(a) Look at the relationship graphically with a scatter-plot to confirm initially that a linear model seems appropriate.
(b) Calculate the estimated regression line by completing the calculation table shown below.
10 n xi yi xi yi 5884 60 70 220 b 1.1. 2 2 5 760 60 2 200 n xi xi 70 60 a y bx 1.1 0.8. 5 5
Least squares equation: y 0.8 1.1x.
Slope of the line is b= 1.1. This means that Test 2 scores are expected to go up by 1.1 points on average for each additional point scored on test 1.
A student who scored 15 points on Test 1 is predicted to score y 0.8 1.1(15) 17.3 points on Test 2.
Test 1 versus Test 2—Obtaining the Regression Line Using the TI Calculator
11 To obtain the least squares regression line using the TI graphing calculator we would first need to enter the data.
L1 L2 Enter the values of the quantitative 8 9 variable x = Test 1 into L1 and enter to 10 13 corresponding value of the quantitative 12 14 variable y=Test 2 into L2. To get the 14 15 least squared regression equation we 16 19 use the following sequence of buttons
Your output screen should provide the least squares regression equation as y = a + b x with the y-intercept of a=0.8 and the slope of b=1.1.
Caution: There are two linear regression options-namely LinReg(ax+b) and LinReg(a+bx). We request the latter option, which uses b to represent the slope.
12 Let's Do It! 13.3 Oil-Change Data
The table below presents data on x = the number of oil changes per year and y = the cost of repairs for a random sample of 10 cars of a certain make and model, from a given region.
(a) Make a scatter-plot of the points as a check for linearity and outliers. Comment on your plot.
(b) Find the least squares regression line for regressing cost on number of oil changes. Describe what the estimated y- intercept and estimated slope represent.
(c) Use your least squares regression line to predict the cost of car repairs for a car that had four oil changes.
Homework page11.3 546: 1, 13, 21, 23, 30, 32, 38, 60, 62, 65, 93, 97
13 p. 836
11.4 STATISTICALLY SIGNIFICANT RELATIONSHIP?
Researchers must rely on data from only a sample in order to assess if a relationship exists between two variables.
Even though a relationship may be apparent in the sample, it is possible that it will not extend to the population.
Researchers use statistics to assess the significance of an observed relationship by measuring the chance that a relationship as strong or stronger would be observed, assuming there really is no relationship in the population.
Think About It
Slope of Zero Consider the equation of a line for relating two variables: yˆ a bx . Suppose the y -intercept, a, is equal to 10 and slope, b, is equal to 0.
What would be the value of the response, yˆ , if x were equal to 2?
What would be the value of the response, yˆ , if x were equal to 12?
What would be the value of the response, yˆ , for any value of x?
What would it mean if the slope for the regression line for a population were equal to 0?
14 The hypothesis of interest in linear regression is: Main Hypothesis: The slope of the linear regression line using all of the population values is equal to 0 (i.e. the linear relationship is insignificant).
Alternative Hypothesis: The slope of the linear regression line using all of the population values is NOT 0 (i.e. the linear relationship is significant).
After obtaining the regression line, one should test the significance of the components of the line (mainly, the slope b .)We should remember that we are testing for a linear relationship between x and y . It is possible that x may determine y nonlinearly.
The test of the main hypothesis above is called the Slope F-Test of Significance. Microsoft EXCEL output: Regression of final exam score, y, on midterm exam score, x.
Lines 12 through 14 are used to assess the significance of the respective coefficients. Line 14 presents information about the slope. The p-value in Line 14 of 0.0001 is used to assess whether the estimated slope is statistically significantly different from zero. Line 13 presents information about the y-intercept. The p-value in line 13 is used to test whether the y -intercept for the population linear regression is equal to 0.
15 Example Service Time A computer-repair technician recorded data on the number of computers serviced and the amount of time to complete the service for 11 randomly selected service visits.
200 Service Time 180
160
140
120
100
80
60
40 20 0 1 2 3 4 5 6 7 8 Number of Computers Serviced x = number of computers serviced and y = time to complete the service
(a) What type of association, if any, does the scatter plot show?
Output from SPSS:
(b) Obtain the estimated linear regression equation: yˆ a bx
Note: The column with the heading contains the values for b and a
SPSS reports the estimated slope first, in the *row, and the y- intercept information in the **row. yˆ 10.19 24.83x
16 (c) Is there evidence of a significant (non-zero) linear relationship between the number of computers serviced and the service time? Explain.
Note: p-value is nearly 0 (p-value < 0.00005) => The number of computers serviced appears to be a significant linear predictor of service time.
(d) Predict the number of minutes required for service when it is reported that 5 pc's are down. y 10.19 24.83(5) 134.3 minutes
Let's Do It! Size of Homes and Selling Price
There are many factors that affect the selling price of a home. The total dwelling size and the assessed value are just two factors. Data were gathered on homes in a Milwaukee, Wisconsin, neighborhood. Scatter-plots revealed a linear relationship between the total dwelling size of a home in 100 square feet and its selling price in dollars. Below is the regression output for the least squares regression of selling price on total dwelling size.
Dep var: PRICE N: 20 Multiple R: .913 Squared multiple R: .834 Adjusted squared multiple R: .825 Standard error of estimate: 3377.192
Variable Coefficient Std error Std coef Tolerance T P(2 tail) CONSTANT 11947.010 4748.133 0.000 . 2.516 0.022 SIZE 2749.622 288.980 0.913 .100E+01 9.515 0.000
Analysis of Variance Source Sum-of-squares DF Mean-square F-ratio P Regression .103257E+10 1 .103257E+10 90.533 0.000 Residual .205298E+09 18 .114054E+08 (a) How many homes were included in this study?
(b) Obtain the least squares regression line for predicting selling price from the size of the home.
(c) Is there a significantly (nonzero) linear relationship between price and size? Explain.
17 (d) The total dwelling size for another home in this neighborhood is 1620 square feet. Use the least squares regression equation to estimate the selling price of this home.
11.7 CORRELATION: HOW STRONG IS THE LINEAR RELATIONSHIP? 18 DEFINITION:
The sample correlation coefficient r measures the strength of the linear relationship between two quantitative variables. It describes the direction of the linear association and indicates how closely the points in a scatter-plot are to the least squares regression line.
Features of the correlation coefficient. 1. Range 1 r 1
2. Sign The sign of the correlation coefficient indicates direction of association — negative [- 1 , 0) or positive (0 , +1].
3. Magnitude The magnitude of the correlation coefficient indicates the strength of the linear association. If the data follow a straight line r 1 (if the slope is positive) or r 1 (if the slope is negative), indicating a perfect linear association. If r 0 then there is no linear association.
4. Measures Strength The correlation only measures the strength of the linear association.
5. Unit-less The correlation is computed using standard scores of the two variables. It has no unit of measure and the absolute value of r will not change if the units of measurement for x or y are changed. The correlation between x and y is the same as the correlation between y and x.
19 Some Pictures....
y x x x x x x x x x x x
x x x x Positive, moderate to strong linear association, r 0.8 .
y
x x x x x x x x x x x x x r 0.2 x
x Negative, weak linear association,
y
x x x x x x
x x x A strong association, just not a linear one, r 0.
Let's Do It! 113.8Matching Graphs The scatter-plot #1 to the right yields a regression line of y = -2.6 + 1.1x and a correlation of r = 0.84.
Using this information as a base, match each of the four scatter- plots below to the correct description of its regression line and
20 correlation coefficient. The scales on the axes of the scatter-plots are the same.
How to Calculate the Correlation Coefficient r
Dep var: PERCENT N:18 Multiple R: .840 Squared multiple R: .706 Adjusted squared multiple R: .688 Standard error of estimate: 10.547
Variable Coefficient Std error Std coef Tolerance T P(2 tail) CONSTANT 96.681 6.289 0.000 . 15.373 0.000 LENGTH -5.970 0.963 -0.840 .100E+01 -6.201 0.000
Analysis of Variance Source Sum-of-squares DF Mean-square F-ratio P Regression 4276.908 1 4276.908 38.448 0.00 Residual 1779.815 16 111.238
“Multiple R: 0.840” = absolute value of the correlation coefficient r.
21 The sign of r can be determined by looking at the sign of the slope, which here is -5.970.
Correlation between length of putt and percentage of putts made is r 0.84.
The formula:
n x y x y r i i i i 2 2 2 2 n xi xi n yi yi
Example Test 1 v e r s us Test 2 Obtaining t he Correlation Coefficient “By Hand”
We already have computed the summation quantities needed for finding r, shown in the calculation table.
Completed Calculation Table 2 2 x i y i x i x i y i y i
8 9 64 72 81 10 13 100 130 169 12 14 144 168 196 14 15 196 210 225 16 19 256 304 361
2 2 Total: x i 6 0 y i 7 0 x i 7 6 0 x i y i 8 8 4 y i 1 0 3 2
The large positive
22 correlation coefficient and the scatter-plot indicate a strong, positive, linear association between Test 1 and Test 2 scores.
Let’s Do It! 2 Birth Rates We gathered data from 1970 for twelve nations on the percentage of women aged 14 or older who were economically active and the crude birth rate. (We define the crude birth rate as the number of births in a year per 1000 population size) We are interested in the relationship of the crude birth rate (y) on the percentage of women who were economically active (x) Nation x y Algeria 2 48 Argentina 19 21 Denmark 34 14 a. Create the scatter-plot. E. Germany 40 11 Determine if there is a Guatemala 8 41 positive, negative, or India 12 37 association between x and y. Ireland 20 22 Jamaica 20 31 Japan 37 19 Philippines 19 42 USA 30 15 Soviet Union 46 18
b. Find the equation of the regression line. Interpret the slope.
c. Find the correlation coefficient r.
Obtaining the Correlation Using the TI
23 To get the regression line and the correlation coefficient using the TI we first need to turn on the diagnostic option. If the x data is in
L1 and the y data is in L2, then the steps are as follows:
Let’s Do It! 3 Birth Rates
Check the value of r you obtained in activity2 above using TI- calculator.
Let’s Do It! 4 Data on Milk Production 24 Milk samples were obtained from 14 Holstein-Friesian cows, and each was analyzed to determine uric acid concentration (Y), measured in mol/L. In addition to acid concentration, the total mild production (X), measured in kg/day, was recorded for each cow. The data was entered into a computer and the following regression output was obtained.
(a) What is the equation of the least squared regression line?
(b) What is the correlation between x and y? r = ______.
(c) We are interested if the linear relationship is significant by testing the following hypothesis. Main Hypothesis: The slope of the regression line equals 0.
Circle the p-value in the output that is used to test this hypothesis. At the level of significance of 0.05, we would (circle one):
Accept H0 Reject H0 Can’t Tell
THE SQUARED CORRELATION r 2 — WHAT DOES IT TELL US? 25 r = correlation coefficient, gives the strength and the direction of the linear relationship between two quantitative variables x and y; –1 r 1. Note that when we square r we get => 0 r2 1. The value of r2 Is the percentage of variation of dependent variable that are explained the independent variable x.
The quantity r2 is generally denoted in computer output as R2, and is often reported as a percent. r2 = 0.75 => about 75% of the variation in the response variable y can be explained by the linear relationship between x and y.
Homework page 504 : 1-4all ,13-15all,18,20, 24,25,26,28.
26