BEHAVIORAL AND BRAIN SCIENCES (2016), Page 1 of 77 doi:10.1017/S0140525X15000965, e229 Cognition does not affect perception: Evaluating the evidence for “top-down” effects Chaz Firestone Department of Psychology, Yale University, New Haven, CT 06520-8205 chaz.fi[email protected] Brian J. Scholl Department of Psychology, Yale University, New Haven, CT 06520-8205 [email protected] Abstract: What determines what we see? In contrast to the traditional “modular” understanding of perception, according to which visual processing is encapsulated from higher-level cognition, a tidal wave of recent research alleges that states such as beliefs, desires, emotions, motivations, intentions, and linguistic representations exert direct, top-down influences on what we see. There is a growing consensus that such effects are ubiquitous, and that the distinction between perception and cognition may itself be unsustainable. We argue otherwise: None of these hundreds of studies – either individually or collectively – provides compelling evidence for true top-down effects on perception, or “cognitive penetrability.” In particular, and despite their variety, we suggest that these studies all fall prey to only a handful of pitfalls. And whereas abstract theoretical challenges have failed to resolve this debate in the past, our presentation of these pitfalls is empirically anchored: In each case, we show not only how certain studies could be susceptible to the pitfall (in principle), but also how several alleged top-down effects actually are explained by the pitfall (in practice). Moreover, these pitfalls are perfectly general, with each applying to dozens of other top-down effects. We conclude by extracting the lessons provided by these pitfalls into a checklist that future work could use to convincingly demonstrate top-down effects on visual perception. The discovery of substantive top-down effects of cognition on perception would revolutionize our understanding of how the mind is organized; but without addressing these pitfalls, no such empirical report will license such exciting conclusions. 1. Introduction and cognition deliver conflicting evidence about the world – as in most visual illusions. Indeed, there may be How does the mind work? Though this is, of course, the no better way to truly feel the distinction between percep- central question posed by cognitive science, one of the tion and cognition for yourself than to visually experience deepest insights of the last half-century is that the question the world in a way you know it not to be. does not have a single answer: There is no one way the There is a deep sense in which we all know what per- mind works, because the mind is not one thing. Instead, ception is because of our direct phenomenological the mind has parts, and the different parts of the mind acquaintance with percepts – the colors, shapes, and operate in different ways: Seeing a color works differently sizes (etc.) of the objects and surfaces that populate than planning a vacation, which works differently than our visual experiences. Just imagine looking at an understanding a sentence, moving a limb, remembering a apple in a supermarket and appreciating its redness (as fact, or feeling an emotion. opposed, say, to its price). That is perception. Or look The challenge of understanding the natural world is to at Figure 1A and notice the difference in lightness capture generalizations – to “carve nature at its joints.” between the two gray rectangles. That is perception. Where are the joints of the mind? Easily, the most Throughout this paper, we refer to visual processing natural and robust distinction between types of mental simply as the mental activity that creates such sensations; processes is that between perception and cognition. This we refer to percepts as the experiences themselves, and distinction is woven so deeply into cognitive science as we use perception (and, less formally, seeing)toencom- to structure introductory courses and textbooks, differen- pass both (typically unconscious) visual processing and tiate scholarly journals, and organize academic depart- the (conscious) percepts that result. ments. It is also a distinction respected by common sense: Anyone can appreciate the difference between, 1.1. The new top-down challenge on the one hand, seeing a red apple and, on the other hand, thinking about, remembering, or desiring ared Despite the explanatorily powerful and deeply intuitive apple. This difference is especially clear when perception nature of the distinction between seeing and thinking, a © Cambridge University Press 2016 0140-525X/16 Downloaded from https:/www.cambridge.org/core. NYU Medical Center: Ehrman Medical Library, on 15 May 2017 at 21:10:16, subject to the Cambridge Core terms of use, available at1 https:/www.cambridge.org/core/terms. https://doi.org/10.1017/S0140525X15002800 Firestone and Scholl: Cognition does not affect perception ACB DE Figure 1. Examples of lightness illusions can be subjectively appreciated as “demonstrations” (for references and explanations, see Adelson 2000). (A) The two columns of gray rectangles have the same luminance, but the left one looks lighter. (B) The rectangles are uniformly gray, but they appear to lighten and darken along their edges. (C) Uniformly colored squares of increasing luminance produce an illusory light “X” shape at their corners. (D) The two central squares have the same objective luminance, but the left one looks lighter. (E) The two rectangles are identical segments of the same gradient, but the right one looks lighter. Similar demonstrations abound, for nearly every visual feature. vocal chorus has recently and vigorously challenged the nitive states routinely “penetrate” perception, such that extent of this division, calling for a generous blurring of what we see is an alloy both of bottom-up factors and of the lines between visual perception and cognition (for beliefs, desires, motivations, linguistic representations, recent reviews, see Balcetis 2016; Collins & Olson 2014; and other such states. In other words, these views hold Dunning & Balcetis 2013; Goldstone et al. 2015; Lupyan that the mental processes responsible for building percepts 2012; Proffitt & Linkenauger 2013; Riccio et al. 2013; Ste- can and do access radically more information elsewhere in fanucci et al. 2011; Vetter & Newen 2014; Zadra & Clore the mind than has traditionally been imagined. 2011). On this increasingly popular view, higher-level cog- At the center of this dispute over the nature of visual per- ception and its relation to other processes in the mind has been the recent and vigorous proliferation of so-called top- down effects on perception. In such cases, some extraper- ceptual state is said to literally and directly alter what we CHAZ FIRESTONE is a graduate student in the Depart- see. (As of this writing, we count more than 175 papers ment of Psychology at Yale University. As of July published since 1995 reporting such effects; for a list, 2017, he will be an Assistant Professor in the Depart- ment of Psychological and Brain Sciences at Johns see http://perception.yale.edu/TopDownPapers.) For example, it has been reported that desiring an object makes it look Hopkins University. He holds an Sc.B. in cognitive neu- fl roscience and an A.M. in philosophy, both from Brown closer (Balcetis & Dunning 2010), that re ecting on uneth- University. His research explores the border between ical actions makes the world look darker (Banerjee et al. perception and cognition, and he was recognized for 2012), that wearing a heavy backpack makes hills look this work with the 2013 William James Prize from the steeper (Bhalla & Proffitt 1999), that words having to do Society for Philosophy and Psychology. He hasn’t pub- with morality are easier to see (Gantman & Van Bavel lished very many papers, but this one is his favorite. 2014), and that racial categorization alters the perceived lightness of faces (Levin & Banaji 2006). BRIAN SCHOLL is Professor of Psychology and Chair of If what we think, desire, or intend (etc.) can affect the Cognitive Science program at Yale University, where he also directs the Perception & Cognition what we see in these ways, then a genuine revolution in Laboratory. He and his research group have published our understanding of perception is in order. Notice, for more than 100 papers on various topics in cognitive example, that the vast majority of models in vision science, with a special focus on how visual perception science do not consider such factors; yet, apparently, interacts with the rest of the mind. He is a recipient such models have been successful! For example, today’s of the Distinguished Scientific Award for Early Career vision science has essentially worked out how low-level Contribution to Psychology and the Robert L. Fantz complex motion is perceived and processed by the Memorial Award, both from the American Psychologi- brain, with elegant models of such processes accounting cal Association, and is a past President of the Society for extraordinary proportions of variance in motion pro- for Philosophy and Psychology. At Yale he is a recipient cessing (e.g., Rust et al. 2006) – and this success has of both the Graduate Mentor Award and the Lex Hixon Prize for Teaching Excellence in the Social Sciences, come without factoring in morality, hunger, or language and he has great fun teaching the Introduction to (etc.). Similarly, such factors are entirely missing from Cognitive Science course. contemporary vision science textbooks (e.g., Blake & Sekuler 2005; Howard & Rogers 2002; Yantis 2013). If Downloaded from https:/www.cambridge.org/core2 BEHAVIORAL AND. NYU BRAIN Medical SCIENCES, Center: Ehrman 39 (2016) Medical Library, on 15 May 2017 at 21:10:16, subject to the Cambridge Core terms of use, available at https:/www.cambridge.org/core/terms. https://doi.org/10.1017/S0140525X15002800 Firestone and Scholl: Cognition does not affect perception such factors do influence how we see, then such models in neighboring disciplines such as philosophy of mind and textbooks are scandalously incomplete.