ASSOCIATION FOR CONSUMER RESEARCH Labovitz School of Business & Economics, University of Minnesota Duluth, 11 E. Superior Street, Suite 210, Duluth, MN 55802 Choosing One At a Time? Simultaneously Presented Options Lead to Normatively Better Choices Than Sequentially Presented Options Krishna Savani, National University of Singapore, Singapore Shankha Basu, Nanyang Technological University, Singapore Four experiments investigate the effect of choosing among simultaneously (versus sequentially) presented options. Findings suggest that people are more likely to choose the normatively best option when they view the options simultaneously. Mediation analysis reveals that greater deliberation, when considering options simultaneously, may be a possible mechanism for the phenomenon. [to cite]: Krishna Savani and Shankha Basu (2015) ,"Choosing One At a Time? Simultaneously Presented Options Lead to Normatively Better Choices Than Sequentially Presented Options", in AP - Asia-Pacific Advances in Consumer Research Volume 11, eds. Echo Wen Wan and Meng Zhang, Duluth, MN : Association for Consumer Research, Pages: 201-103. [url]: http://www.acrwebsite.org/volumes/1018740/volumes/ap11/AP-11 [copyright notice]: This work is copyrighted by The Association for Consumer Research. For permission to copy or use this work in whole or in part, please contact the Copyright Clearance Center at http://www.copyright.com/. Asia-Pacific Advances in Consumer Research (Volume 11) / 201 Pieters, Rik (2013), “Bidirectional Dynamics of Materialism and Loneliness: Not Just a Vicious Cycle,” Journal of Consumer Research, 40 (4), 615-31. Rosenbloom, Stephanie and Michael Barbaro (2009), “Green-light Specials, Now at Wal-Mart,” New York Times, January 25. Trudel, Remi, and June Cotte (2009), “Does It Pay To Be Good,” Sloan Management Review, 50 (2), 61-8. Vouloumanos, Athena, Kristine H. Onishi, and Amanda Pogue (2012), “Twelve-Month-Old Infants Recognize That Speech Can Communicate Unobservable Intentions,” Proceedings of the National Academy Of Sciences, 109, 12933-37. Weiner, Bernard (1980), Human Motivation, NY: Holt, Rinehart & Winston. Weiner, Bernard (2000), “Attributional Thoughts about Consumer Behavior,” Journal of Consumer Research, 27 (3), 382-87. White, Katherine and Bonnie Simpson (2013), “When Do (and Don’t) Normative Appeals Influence Sustainable Consumer Behaviors?” Journal of Marketing, 77 (2), 78-95. Yuan, Hong, Uday Rajan, and Aradhna Krishna (2012), “Why Are Green Items More Prevalent at Higher Priced Stores?” working paper, University of Michigan. Blinding Us to the Obvious? The Effect of Statistical Training on the Evaluation of Evidence Blakeley B. McShane, Kellogg School of Management, Northwestern University, USA David Gal, College of Business Administration, University of Illinois at Chicago, USA EXTENDED ABSTRACT In this paper, we investigate one way in which the NHST para- Null hypothesis significance testing (NHST) is the dominant digm may lead researchers to misinterpret evidence. In particular, paradigm in academic training and reporting in the biomedical and given the focus on NHST and the concomitant dichotomization of social sciences [Morrison and Henkel, 1970, Gigerenzer, 1987, Saw- results into statistically significant and not statistically significant in yer and Peter, 1983, McCloskey and Ziliak, 1996, Gill, 1999, An- academic training and reporting, we hypothesized that researchers– derson et al., 2000, Gigerenzer, 2004, Hubbard, 2004]. A prominent despite general knowledge that the conventional 5% level of statisti- feature of the NHST paradigm is the enshrinement of the eponymous cal significance is arbitrary–tend to think of evidence in dichotomous null hypothesis, which typically posits that there is no difference be- terms: evidence that reaches the conventionally defined threshold of tween two or more groups with respect to some underlying popula- statistical significance (i.e., p < 0.05) is interpreted as a demonstra- tion parameter of interest (e.g., a mean or proportion). Pitted against tion of a difference whereas evidence that fails to reach this threshold the null hypothesis is the alternative hypothesis, which, in typical ap- is interpreted as a demonstration of no difference. In fact, evidence is plications, posits that there is a difference between the groups. Stan- more accurately viewed “as a fairly continuous function of the mag- dard practice involves collecting data, computing a p-value which nitude of p” [Rosnow and Rosenthal,1989] and the conventional 5% is a function of the data and the null hypothesis, and then retaining level of statistical significance (and the concomitant dichotomization or rejecting the null hypothesis depending on whether the p-value is of results into “significant” and “not significant”) is arbitrary. respectively above or below the size α of the hypothesis test where α To systematically examine whether researchers might be led by is conventionally set to 0.05. the notion of statistical significance to misconstrue evidence, we sur- Despite the overwhelming dominance of the NHST paradigm in veyed researchers across a wide variety of fields (including medicine, practice, it has received no small degree of criticism over the decades. cognitive science, psychology, consumer behavior and quantitative Consider, for instance, the following passage from Gill [1999]: marketing researchers, and economics) regarding their interpretation of data. Researchers were presented with a scenario like the below: It [NHST] has been described as a “strangle-hold” [Rozen- boom, 1960], “deeply flawed or else ill-used by research- • A study aimed to test how different interventions might ers” [Serlin and Lapsley, 1993], “a terrible mistake, ba- affect terminal cancer patients’ survival. Participants sically unsound, poor scientific strategy, and one of the were randomly assigned to one of two groups. Group worst things that ever happened in the history of psychol- A was instructed to write daily about positive things ogy” [Meehl, 1978], “an instance of the kind of essential they were blessed with while Group B was instructed mindlessness in the conduct of research” [Bakan, 1966], to write daily about misfortunes that others had to en- “badly misused for a long time” [Cohen, 1994], and that dure. Participants were then tracked until all had died. it has “systematically retarded the growth of cumulative Participants in Group A lived, on average, 8.2 months knowledge” [Schmidt, 1996]. Or even more bluntly: “The post-diagnosis whereas participants in Group B lived, significance test as it is currently used in the social sciences on average, 7.5 months post-diagnosis (p = XXX). just does not work.” [Hunter, 1997] We then asked researchers questions pertaining to: Clearly NHST is not without its critics. • Descriptive statements: Speaking only of the subjects who Despite this widespread criticism, relatively little attention has took part in this particular study, the average number of been devoted to whether researchers are in fact misled by the NHST post-diagnosis months lived by the participants who were in paradigm in their evaluation of evidence. However, exceptions exist. Group A was greater / less / no different than that lived by the For instance, it is well-known that statistical significance and practi- participants who were in Group B. We also included an op- cal importance are often confused; indeed, this confusion is so ram- tion indicating it could not be determined based on the data. pant that, to preempt it, introductory statistics textbooks repeatedly affirm, with a frequency rivaled only by declarations that correlation • Likelihood judgments (for this question, the outcome was does not imply causation, that statistical significance is distinct from presented in terms of a recovery probability rather than practical importance [Freedman et al., 2007]. Another ill effect of the months lived post-diagnosis): A person drawn randomly dichotomization of results into statistically significant and not statis- from the same patient population as the patients in the study tically significant is that researchers treat results that attain statistical is more / less / equally likely to recover from the disease if significance as evidence for an effect while they treat results that given Drug A than if given Drug B. We also included an op- fail to attain statistical significance as evidence of the absence of an tion indicating it could not be determined based on the data. effect. Gelman and Stern [2006] have discussed one important impli- cation of this practice, namely that researchers commonly infer that • Treament choice (for this question, the outcome was two treatments are significantly different when one treatment attains presented in terms of a recovery probability rather statistical significance while the other fails to do so. In reality, the than months lived post-diagnosis): I prefer Drug A / two treatments may have a statistically similar effect, or as Gelman B. We also included an option indicating indifference. and Stern [2006] conclude, “the difference between ‘significant’ and ‘not significant’ is not itself statistically significant.” Asia-Pacific Advances in Consumer Research 202 Volume 11, © 2015 Asia-Pacific Advances in Consumer Research (Volume 11) / 203 In presenting these results, we varied whether the p-value pre- sented was below or above 0.05. We also presented some of our re- spondents with a posterior probability based on a Bayesian calcula- tion. A substantial majority