Statistical Significance for 2 X 2 Tables

CHAPTER 13

STATISTICAL SIGNIFICANCE FOR 2 x 2 TABLES

SECTION 13.1

MEASURING THE STRENGTH OF THE RELATIONSHIP

FREE RESPONSE QUESTIONS

1. Whether or not we can rule out chance as an explanation for the relationship observed in a sample depends on what two things? ANSWER: 1) THE STRENGTH OF THE RELATIONSHIP OBSERVED IN THE SAMPLE; AND 2) HOW MANY PEOPLE WERE INVOLVED IN THE STUDY.

2. Suppose you observe a relationship between two categorical variables in a sample based on 22,000 people. If this observed relationship were based on a sample of 600 people rather than 22,000, how would this affect the believability of the results (if at all)? Explain your answer. ANSWER: AN OBSERVED RELATIONSHIP IS MUCH MORE BELIEVABLE IF IT IS BASED ON A MUCH LARGER SAMPLE SIZE. SAMPLE RESULTS FROM LARGER SAMPLES ARE LESS LIKELY TO VARY FROM SAMPLE TO SAMPLE AND ARE MORE LIKELY TO REPRESENT THE TRUTH ABOUT THE POPULATION.

MULTIPLE CHOICE QUESTIONS

3. Which of these does not apply to a ‘statistically significant’ relationship between two categorical variables? a. The relationship between the two variables is very important from a practical standpoint. b. The relationship observed in the sample was unlikely to have occurred unless there really is a relationship in the population. c. The notion that this relationship could have happened by chance is deemed to be implausible. d. All of the above apply to a ‘statistically significant’ relationship. ANSWER: A

4. Which of the following differences in percentages for the two categories of an explanatory variable would be considered to be statistically significant? a. 50% - 49 % = 1% b. 50% - 45% = 5% c. 50% - 40% = 10% d. Not enough information to tell. ANSWER: D

FILL-IN-THE-BLANK QUESTIONS

5. When researchers look for a relationship between two categorical variables for individuals in the ______, they measure those categorical variables on individuals in the ______. ANSWERS (RESPECTIVELY): POPULATION; SAMPLE

6. The question researchers are really asking when they are looking for a relationship between two variables is whether or not that relationship is present in the ______. ANSWER: POPULATION

SECTION 13.2

STEPS FOR ASSESSING STATISTICAL SIGNIFICANCE

7. The first two basic steps for conducting a hypothesis test are to: 1) determine the null and alternative hypotheses; and 2) collect the data and summarize it with a ‘test statistic.’ What is the third step in the process that allows you to then make a decision? ANSWER: DETERMINE HOW UNLIKELY THE TEST STATISTIC WOULD BE IF THE NULL HYPOTHESIS WERE TRUE.

8. Suppose you want to investigate whether there is a relationship between the gender of college students and whether or not they wear hats in school. What would be your null hypothesis and your alternative hypothesis (in words)? Be sure to label clearly which hypothesis is which. ANSWER: NULL HYPOTHESIS: NO RELATIONSHIP BETWEEN GENDER AND HAT WEARING IN COLLEGE STUDENTS; ALTERNATIVE HYPOTHESIS: THERE IS A RELATIONSHIP.

MULTIPLE CHOICE QUESTIONS . 9. Suppose you find a statistically significant relationship between two categorical variables (with no other supporting evidence available). When can such results correctly lead you conclude a cause and effect relationship? a. Never. b. Only when the data were from a randomized experiment. c. Only when the data were from a random sample. d. Always; a statistically significant relationship wouldn’t be significant unless there is a cause and effect relationship. ANSWER: B

10. Your mother-in-law says that rubbing a carpet stain with toothpaste before applying stain remover helps take the stain out. You are skeptical. You asked an impartial neighbor to set up a randomized experiment to find out whether there is a relationship between the stain removing method and the outcome. She went home to collect and analyze the data, then called you and said “In a chi-square test, I would choose the alternative hypothesis,” and then hung up. What do you conclude? a. The toothpaste didn’t make a difference. b. The toothpaste did help after all. c. The toothpaste had a significant effect but you don’t know in which way yet. d. The toothpaste made things worse. ANSWER: C FILL-IN-THE-BLANK QUESTIONS

11. Another name for the alternative hypothesis is the ______hypothesis. ANSWER: RESEARCH 12. The ______hypothesis is usually written to express the fact that ‘nothing is happening.’ ANSWER: NULL

SECTION 13.3

THE CHI-SQUARE TEST

For Questions 13-15, use the following narrative Narrative: Aspirin and polyps A well-designed experiment by the Ohio State University Medical Center studied whether or not taking an aspirin a day would affect colon cancer patients’ chances of getting subsequent colon polyps. In the study, 635 patients with colon cancer participated; 317 of them were randomly assigned to the aspirin group, and the other 318 patients were assigned to a placebo (non-aspirin) group. The results showed that 17% of the patients in the aspirin group developed subsequent polyps, compared to 27% of the patients in the non-aspirin group. The results are summarized in the table below.

POLYPS NO POLYPS TOTAL ASPIRIN 54 263 317 PLACEBO 86 232 318 TOTAL 140 495 635

13. {Aspiring and polyps narrative} Write the expected counts in each of the four cells of the table in parentheses next to each of the observed counts. ANSWER: SEE ANSWERS IN PARENTHESES IN THE TABLE BELOW.

POLYPS NO POLYPS TOTAL ASPIRIN 54 (69.89) 263 (247.11) 317 PLACEBO 86 (70.11) 232 (247.89) 318 TOTAL 140 495 635

14. {Aspirin and polyps narrative} Compute the test statistic for the data. ANSWER: 9.25

15. {Aspirin and polyps narrative} Is there a statistically significant relationship between taking aspirin and developing subsequent polyps in colon cancer patients? If so, what is the relationship? Justify your answer. ANSWER: THE CHI-SQUARE STATISTIC IS 9.25 > 3.84 (P-VALUE < .05), THE RELATIONSHIP IS STATISTICALLY SIGNIFICANT. SINCE THE ASPIRIN GROUP EXPERIENCED FEWER POLYPS (17% COMPARED TO 27%), IT APPEARS THAT ASPIRIN REDUCES THE CHANCES OF DEVELOPING SUBSEQUENT POLYPS IN COLON CANCER PATIENTS.

16. Suppose a researcher wanted to find out which of two chemicals works best to kill poison ivy. He conducted a controlled experiment involving 200 poison ivy plants, randomly assigned half of them to each treatment (chemical #1 or chemical #2). Five days after applying the chemicals, he recorded the data below. Is there a statistically significant relationship between type of chemical used and the results when applied to poison ivy? Conduct a hypothesis test to find out.

DIED LIVED TOTAL CHEMICAL #1 78 22 100 CHEMICAL #2 89 11 100 TOTAL 167 33 200

ANSWER: YES; THE EXPECTED CELL COUNTS ARE IN PARENTHESES. THE CHI- SQUARE TEST STATISTIC IS 4.39 > 3.84, SO THE RELATIONSHIP IS STATISTICALLY SIGNIFICANT. NOTE THAT CHEMICAL #2 SEEMS TO DO BETTER SINCE IT KILLED A HIGHER PERCENTAGE OF PLANTS IN THE SAMPLE (89% COMPARED TO 78%).

DIED LIVED TOTAL CHEMICAL #1 78 (83.5) 22 (16.5) 100 CHEMICAL #2 89 (83.5) 11 (16.5) 100 TOTAL 167 33 200

17. Using the criterion of .05, which of the following results allows a researcher to conclude that a relationship between two categorical variables is statistically significant? a. p-value = .04 b. p-value = .50 c. p-value = .95 d. None of the above. ANSWER: A

18. Which of the following statements is true about chi-square tests? a. A large chi-square test statistic results in a large p-value. b. A large p-value means that there is a good chance that the relationship is statistically significant. c. If the two variables are not related in the population, then less than 5% of the samples you could ever take would give you a test statistic of 3.84 or larger. d. All of the above. ANSWER: C

19. Which of the following statements is not true about the chi-square statistic for a 2 x 2 contingency table? a. If it is greater than 3.84, reject the null hypothesis and accept the alternative hypothesis. b. If it is greater than 3.84, the relationship in the table is considered to be statistically significant. c. 95% of the tables for sample data from populations in which there is no relationship will have a chi-square statistic of 3.84 or greater. d. All of the above are true. ANSWER: C

20. Suppose you wanted to test to see if there is a relationship between gender and driving an SUV. You take a random sample of 200 vehicles and their owners and use Minitab to analyze your data. Your computer output is below, but unfortunately the last line of the output (which would have contained the p-value) didn’t print. What can you conclude? (Assume the study was conducted properly otherwise.)

Expected counts are printed below observed counts SUV Non-SUV TOTAL Male 45 67 112 49.84 62.16 Female 44 44 88 39.16 48.84

Total 89 111 200

Chi-Sq = .47 + .38 + .60 + .48 = 1.92

a. The relationship between gender and driving an SUV is not statistically significant. b. There is a statistically significant relationship. Women are more likely to drive SUVs than men. c. There is a statistically significant relationship. More men are likely to drive SUVs than women. d. Not enough information to tell. ANSWER: A

21. A measure of how unlikely the test statistic would be if the null hypothesis were true is the ______. ANSWER: P-VALUE

22. A ______test is a statistical procedure that is used to determine whether or not there is a relationship between two categorical variables. ANSWER: CHI-SQUARE

SECTION 13.4

PRACTICAL VERSUS STATISTICAL SIGNIFICANCE

23. If two variables have a statistically significant relationship that does not necessarily mean the two variables have a relationship of practical importance. Describe how this could happen. ANSWER: IT DEPENDS ON THE SAMPLE SIZE; A MINOR RELATIONSHIP OBSERVED IN A SAMPLE BASED ON A LARGE ENOUGH NUMBER OF OBSERVATIONS CAN EASILY ACHIEVE STATISTICAL SIGNIFICANCE.

24. Is it possible for an important relationship in a population to fail to achieve statistical significance in the sample and therefore go undetected, even if the sample is selected properly and the data are collected correctly? Explain your answer. ANSWER: YES; WHY: 1) THERE MAY HAVE ONLY BEEN A FEW OBSERVATIONS; OR 2) THE SAMPLE IS NOT REPRESENTATIVE OF THE POPULATION, JUST BY CHANCE.

For Questions 25-26, use the following narrative Narrative: Teenage voting Suppose you take a random sample of teenagers aged 18-19, and you find that 53% of the males in your sample are Democrats, and 54% of the females in your sample are Democrats, respectively. 25. {Teenage voting narrative} Should these results be considered significant from a practical standpoint? Why or why not? ANSWER: NO; THEY ARE NOT SIGNIFICANTLY DIFFERENT FROM A PRACTICAL STANDPOINT. THE DIFFERENCE IS TOO SMALL.

26. {Teenage voting narrative} Could these results ever be statistically significant? Why or why not? ANSWER: YES, IF THE SAMPLE SIZE IS LARGE ENOUGH, ANY NON-ZERO DIFFERENCE CAN BECOME STATISTICALLY SIGNIFICANT.

27. Suppose a chi-square test statistic from a sample of size 1,000 is 38.2 with a p-value of .0001, so the relationship is statistically significant. If the sample size for this study had only been based on a sample size of 100 (but the percentages remained the same), what would the chi-square test statistic have been and what conclusion would have been drawn? a. Chi-square = 38.2; p-value = .0001; the relationship is statistically significant. b. Chi-square = 3.82; p-value = .001; the relationship is statistically significant. c. Chi-square = 3.82; p-value = .052; the relationship is not statistically significant. d. None of the above. ANSWER: C

28. Suppose researchers say they ‘failed to find a relationship’ between two variables that they thought might have been related. What does this say to you, as an educated consumer of statistical information? a. You should check to make sure the study was not based on a small number of individuals. b. The researchers must not have observed any relationship in their sample. c. There must be no relationship between these two variables in the population. d. All of the above. ANSWER: A

29. Suppose there is not enough evidence to conclude that the relationship in the population is real. Which of the following is not an equivalent way of saying this? a. The relationship is not statistically significant. b. We cannot reject the null hypothesis. c. We accept the null hypothesis. d. All of the above. ANSWER: C

30. Suppose that a medical study found a statistically significant relationship between wearing gold jewelry and developing skin cancer. Suppose the study was based on a sample of 100,000 people and had a p-value of .049. How would you react to these results? a. You feel these results are very convincing since the sample size was so large. b. You are impressed by the sample size but the p-value is too close to .05 to feel very convinced; the result is marginal. c. You are skeptical because the high sample size would lead to a small p-value even with a miniscule relationship, and thus gives misleading results. d. None of the above ANSWER: C

31. A minor relationship that is found to be ______significant is not necessarily ______significant. ANSWER: STATISTICALLY; PRACTICALLY 32. An interesting relationship in the population may fail to achieve ______significance if there are too _____ observations. ANSWER: STATISTICAL; FEW

33. A minor relationship in the population may achieve ______significance but fail to achieve ______significance if there are an extremely large number of observations. ANSWERS (RESPECTIVELY): STATISTICAL; PRACTICAL