University of Pennsylvania ScholarlyCommons Finance Papers Wharton Faculty Research 3-2015 The Psychology of Intelligence Analysis: Drivers of Prediction Accuracy in World Politics Barbara Mellers University of Pennsylvania Eric Stone Pavel Atanasov Nick Rohrbaugh S. Emlen Metz See next page for additional authors Follow this and additional works at: https://repository.upenn.edu/fnce_papers Part of the Finance and Financial Management Commons, and the Social and Behavioral Sciences Commons Recommended Citation Mellers, B., Stone, E., Atanasov, P., Rohrbaugh, N., Metz, S. E., Ungar, L., Bishop, M. M., Horowitz, M. C., Merkle, E., & Tetlock, P. E. (2015). The Psychology of Intelligence Analysis: Drivers of Prediction Accuracy in World Politics. Journal of Experimental Psychology: Applied, 21 (1), 1-14. http://dx.doi.org/ 10.1037%2Fxap0000040 This paper is posted at ScholarlyCommons. https://repository.upenn.edu/fnce_papers/60 For more information, please contact [email protected]. The Psychology of Intelligence Analysis: Drivers of Prediction Accuracy in World Politics Abstract This article extends psychological methods and concepts into a domain that is as profoundly consequential as it is poorly understood: intelligence analysis. We report findings from a geopolitical forecasting tournament that assessed the accuracy of more than 150,000 forecasts of 743 participants on 199 events occurring over 2 years. Participants were above average in intelligence and political knowledge relative to the general population. Individual differences in performance emerged, and forecasting skills were surprisingly consistent over time. Key predictors were (a) dispositional variables of cognitive ability, political knowledge, and open-mindedness; (b) situational variables of training in probabilistic reasoning and participation in collaborative teams that shared information and discussed rationales (Mellers, Ungar, et al., 2014); and (c) behavioral variables of deliberation time and frequency of belief updating. We developed a profile of the best forecasters; they were better at inductive reasoning, pattern detection, cognitive flexibility, and open-mindedness. They had greater understanding of geopolitics, training in probabilistic reasoning, and opportunities to succeed in cognitively enriched team environments. Last but not least, they viewed forecasting as a skill that required deliberate practice, sustained effort, and constant monitoring of current affairs. Disciplines Finance and Financial Management | Social and Behavioral Sciences Author(s) Barbara Mellers, Eric Stone, Pavel Atanasov, Nick Rohrbaugh, S. Emlen Metz, Lyle Ungar, Michael M. Bishop, Michael C. Horowitz, Ed Merkle, and Philip E. Tetlock This journal article is available at ScholarlyCommons: https://repository.upenn.edu/fnce_papers/60 A Collection of Neuroscience & Cognition Articles AMERICAN PSYCHOLOGICAL ASSOCIATION A Collection of Neuroscience & Cognition Articles from APA Journals DEAR MEMBER, t APA, we understand how challenging it can be for you to stay abreast of the latest A and best academic research on psychology. To help you save time, APA is compiling noteworthy articles from a variety of research disciplines. This booklet features some of the most influential scholars and cutting-edge scientific researchers on topics that range from increasing cognitive reserves to an analysis of learning and recall. When curating this selection of articles for you, APA’s scientific staff drew from half a dozen scholarly APA journals and publications that focus on the latest neuroscience and cognition research. APA’s Journals Program houses hundreds of academic papers in this field, and you can access them by visiting the journals area on our website at www.apa.org/pubs/journals and browsing by subject: Neuroscience & Cognition. Moving forward, we will continue to bring you even greater access to cutting-edge research. And we’d like your assistance in this effort: Please share your feedback about this booklet at [email protected]. We look forward to hearing from you as we work on your behalf to make your APA membership more valuable and enjoyable. Sincerely, Ian King, MBA Executive Director, Membership American Psychological Association A Collection of Neuroscience & Cognition Articles from APA Journals CONTENTS 1 MODELING THE INTERPLAY BETWEEN AFFECT AND DELIBERATION Decision April 2015 by George Loewenstein, Ted O'Donoghue, and Sudeep Bhatia 28 RECOGNITION WITHOUT AWARENESS: ENCODING AND RETRIEVAL FACTORS Journal of Experimental Psychology: Learning, Memory, and Cognition September 2015 by Fergus I. M. Craik, Nathan S. Rose, and Nigel Gopie 39 THE TIP-OF-THE-TONGUE HEURISTIC: HOW TIP-OF-THE-TONGUE STATES CONFER PERCEPTIBILITY ON INACCESSIBLE WORDS Journal of Experimental Psychology: Learning, Memory, and Cognition September 2015 by Anne M. Cleary and Alexander B. Claxton 46 SEARCHING FOR EXPLANATIONS: HOW THE INTERNET INFLATES ESTIMATES OF INTERNAL KNOWLEDGE Journal of Experimental Psychology: General June 2015 by Matthew Fisher, Mariel K. Goddu, and Frank C. Keil 60 STRESS INCREASES CUE-TRIGGERED "WANTING" FOR SWEET REWARD IN HUMANS Journal of Experimental Psychology: Animal Learning and Cognition April 2015 by Eva Pool, Tobias Brosch, Sylvain Delplanque, and David Sander 69 MEMORY AS A HOLOGRAM: AN ANALYSIS OF LEARNING AND RECALL Canadian Journal of Experimental Psychology / Revue canadienne de psychologie expérimentale March 2015 by Donald R. J. Franklin and D. J. K. Mewhort A Collection of Neuroscience & Cognition Articles from APA Journals 90 THE PSYCHOLOGY OF INTELLIGENCE ANALYSIS: DRIVERS OF PREDICTION ACCURACY IN WORLD POLITICS Journal of Experimental Psychology: Applied March 2015 by Barbara Mellers, Eric Stone, Pavel Atanasov, Nick Rohrbaugh, S. Emlen Metz, Lyle Ungar, Michael M. Bishop, Michael Horowitz, Ed Merkle, and Philip Tetlock 104 WHAT CAN 1 BILLION TRIALS TELL US ABOUT VISUAL SEARCH? Journal of Experimental Psychology: Human Perception and Performance February 2015 by Stephen R. Mitroff, Adam T. Biggs, Stephen H. Adamo, Emma Wu Dowd, Jonathan Winkle, and Kait Clark 109 SENDING YOUR GRANDPARENTS TO UNIVERSITY INCREASES COGNITIVE RESERVE: THE TASMANIAN HEALTHY BRAIN PROJECT Neuropsychology July 2016 by Megan E. Lenehan, Mathew J. Summers, Nichole L. Saunders, Jeffery J. Summers, David D. Ward, Karen Ritchie, and James C. Bickering 116 A COMPLEMENTARY PROCESSES ACCOUNT OF THE DEVELOPMENT OF CHILDHOOD AMNESIA AND A PERSONAL PAST Psychological Review April 2015 by Patricia J. Bauer Decision © 2015 American Psychological Association 2015, Vol. 2, No. 2, 55–81 2325-9965/15/$12.00 http://dx.doi.org/10.1037/dec0000029 Modeling the Interplay Between Affect and Deliberation George Loewenstein Ted O’Donoghue Carnegie Mellon University Cornell University Sudeep Bhatia University of Warwick Drawing on diverse lines of research in psychology, economics, and neuroscience, we develop a model in which a person’s behavior is determined by an interaction between deliberative processes that assess options with a broad, goal-based perspective, and affective processes that encompass emotions and other motivational states. Our model provides a framework for understanding many departures from rationality discussed in the literature and captures the familiar feeling of being “of 2 minds.” Most important, by focusing on factors that moderate the relative influence of the 2 processes, our model generates a variety of novel testable predictions. We apply our model to intertemporal choice, risky decisions, and social preferences. Keywords: decision making, dual process, dual system, willpower, intertemporal choice, risk, social preferences From the writings of the earliest philosophers example, controlled versus automatic processes to the present, there has been an almost unbro- (Shiffrin & Schneider, 1977), symbolic and as- ken belief that human behavior is best under- sociative processes (Sloman, 1996; Smith & stood as the product of two interacting and often DeCoster, 2000), impulsive and reflective pro- competing processes. Many recent dual process cesses (Lieberman, 2003; Strack & Deutsch, perspectives have focused on the differences 2004), and System I and II (Kahneman & Fred- between two different modes of thinking—for erick, 2002). In this article, we also propose a dual-process framework; however, our focus is on choice behavior rather than judgment. Fol- lowing a long tradition of perspectives drawing George Loewenstein, Department of Social and Decision a distinction between, for example, “passion Sciences, Carnegie Mellon University; Ted O’Donoghue, Department of Economics, Cornell University; Sudeep Bha- versus reason,” “the id and the ego,” and more tia, Department of Psychology and Warwick Business recently, “emotion and cognition,” we argue School, University of Warwick. that choice behavior can be seen as the product This work was supported by Integrated Study of the of two motivational processes, one more delib- Human Dimensions of Global Change at Carnegie Mellon University (NSF Grant SBR-9521914 to George Loewen- erative and focused on broader goals and the stein), the National Science Foundation (Grant SES- other more reflexive and driven by emotions 0214043 to Ted O’Donoghue), and the Economic and So- and other motivational states. This document is copyrighted by the American Psychological Association or one of its allied publishers. cial Research Council (Grant ES/K002201/1 to Sudeep Although both affect and deliberation have Bhatia). For useful comments,