Topic 3: Characteristics of Good Sample Surveys and Comparative Studies

‘Poll Monday’, ‘Thinking Tuesday’ and ‘Results Wednesday’

OVERVIEW

The poll this week focuses on reading research studies. Students are asked on Monday to find a news story about a comparative research study. On Tuesday, the instructor selects two examples from those that were submitted and asks follow up questions. On Wednesday the instructor provides commentary regarding the two questions that were asked on Tuesday.

‘POLL MONDAY’ QUESTIONS

Poll Question - Submit the url or website address of a news story about a comparative research study. (Be sure this is recent - the research studies should be 2014 or later!)

‘THINKING TUESDAY’ QUESTIONS

Here are the urls from two research studies submitted by fellow students:

1) E-cigarettes: Teens who vape are more likely to smoke later, study says: http://www.latimes.com/science/sciencenow/la-sci-sn-e-cigarettes-vape-teens- smoke-tobacco-20150908-story.html

2) New study says caffeine can help strengthen memory function http://www.washingtonpost.com/national/health-science/new-study-says- caffeine-can-help-strengthen-memory-function/2014/01/13/bff02b5a-7c7c-11e3- 93c1-0e888170b723_story.html

Poll Question 1 - Which of the above was an experiment, and which was an observational study? A. The caffeine study was an experiment and the e-cigarette study was an observational study. B. The e-cigarette study was an experiment and the caffeine study was an observational study

Poll Question 2 - The news report for one study claims that caffeine can help strengthen memory, while the report for the other study claims that smoking e-cigarettes can lead non-smoking teens to take up smoking. Which study provides better evidence for the reported claim? Explain briefly

‘RESULTS WEDNESDAY’ COMMENTARY

This week we examined two news reports suggested by class members that describe comparative studies:

1) A story in the LA Times about teens use of e-Cigarettes and their later smoking habits at http://www.latimes.com/science/sciencenow/la-sci-sn-e-cigarettes-vape- teens-smoke-tobacco-20150908-story.html and

2) A study in the Washington Post about caffeine and it’s possible effect on memory at https://www.washingtonpost.com/national/health-science/new-study-says- caffeine-can-help-strengthen-memory-function/2014/01/13/bff02b5a-7c7c-11e3- 93c1-0e888170b723_story.html

The caffeine study was a randomized experiment where the subjects were randomized to either take a 200-milligram caffeine tablet or a placebo tablet while the e-Cigarette study was a prospective observational study since the researchers just followed the teens over time and did not assign them to use or not use the e- Cigarettes. (81% of the class got that right)

As a randomized experiment, the caffeine study provided better evidence for its causal claim. The e-Cigarette study provided very poor evidence of causality. Since the e- Cigarette study was just observational there might have been many differences between the group that tried e-Cigarettes and those that did not – and any of those differences could provide an alternate explanation of the data. For example, a key predictor of whether a teen will start smoking is whether members of his/her family and social circle are smokers. If there was an imbalance in that characteristic between the groups being compared, that could easily explain the association seen.

Also the number of teens that tried e-Cigarettes in this study was very small (only 16) and thus the data about smoking rates for that group is subject to a great deal of variability.

The news report in the LA Times did a better job of reporting the e-Cigarettes study by including important details and caveats (like the variability issue) while the Washington Post story failed to mention important details like the sample size and researcher blinding – but did give a link to the actual scientific paper so the reader can get the details if interested.