The Impending Demise of the Item in Visual Search
Total Page:16
File Type:pdf, Size:1020Kb
BEHAVIORAL AND BRAIN SCIENCES (2017), Page 1 of 69 doi:10.1017/S0140525X15002794, e132 The impending demise of the item in visual search Johan Hulleman Division of Neuroscience and Experimental Psychology, School of Biological Sciences, The University of Manchester, Manchester M13 9PL, United Kingdom. [email protected] https://www.research.manchester.ac.uk/portal/johan.hulleman.html Christian N. L. Olivers Department of Experimental and Applied Psychology, Institute for Brain & Behaviour Amsterdam, VU University, 1081 BT Amsterdam, The Netherlands. [email protected] http://www.vupsy.nl/staff-members/christian-olivers/ Abstract: The way the cognitive system scans the visual environment for relevant information – visual search in short – has been a long-standing central topic in vision science. From its inception as a research topic, and despite a number of promising alternative perspectives, the study of visual search has been governed by the assumption that a search proceeds on the basis of individual items (whether processed in parallel or not). This has led to the additional assumptions that shallow search slopes (at most a few tens of milliseconds per item for target-present trials) are most informative about the underlying process, and that eye movements are an epiphenomenon that can be safely ignored. We argue that the evidence now overwhelmingly favours an approach that takes fixations, not individual items, as its central unit. Within fixations, items are processed in parallel, and the functional field of view determines how many fixations are needed. In this type of theoretical framework, there is a direct connection between target discrimination difficulty, fixations, and reaction time (RT) measures. It therefore promises a more fundamental understanding of visual search by offering a unified account of both eye movement and manual response behaviour across the entire range of observed search efficiency, and provides new directions for research. A high-level conceptual simulation with just one free and four fixed parameters shows the viability of this approach. Keywords: attention; eye movements; features; fixations; functional field of view; oculomotor control; visual search; visual selection 1. Introduction GS, and AET as the leading theories (Chan & Hayward Whether we are trying to find a friend amongst disembark- 2013). However, although these dominant theoretical ing passengers or looking for a street name to establish our frameworks have inspired great advances in the study of whereabouts, searching for targets is a ubiquitous part of visual attention, in our opinion, further progress is hindered our lives, and it involves fundamental cognitive mecha- by what appears to be an implicit yet central assumption, nisms of perception, attention, and memory. Therefore, namely that the primary unit of selection in visual search determining how we scan the visual environment for rele- is the individual item. vant information is a fundamental goal of vision science. In the lab, the typical search experiment involves a single Visual search behaviour has been studied for a long time. known target, which can range from a simple geometrical For example, the very first issue of the Quarterly Journal of shape to a more complex alphanumeric character or an Experimental Psychology contained a paper on it (Mack- everyday object. Participants are usually instructed to worth 1948), and Neisser wrote about finding a face in determine its presence amongst a varying number of dis- the crowd for Scientific American back in 1964. But the tractor items, although there are variants of the task in two most seminal years in the field of visual search probably which the target is always present and observers make a lie in the 1980s. At the start of that decade, Treisman and decision on some orthogonally varied property (e.g., the Gelade (1980) published their classic Feature Integration identity of a letter inside of a target that is defined by Theory (FIT). At the end, Wolfe et al. (1989), as well as colour). The effect of the number of distractor items on Duncan and Humphreys (1989) proposed their very influ- RT – the slope of the search function – is an important ential alternatives, Guided Search (GS) and Attentional measure, because it indicates how efficiently observers Engagement Theory (AET). These contributions made detect the target. Although theories of visual search visual search a burgeoning research area. In fact, they broadly recognize that there is a large amount of parallel have been so successful that a recent review of visual processing within the visual field, this has had surprisingly search, published almost 25 years later, still listed FIT, little impact on what has been assumed to be the core © Cambridge University Press 2017 0140-525X/17 Downloaded from https://www.cambridge.org/core. Universiteitsbibliotheek Utrecht, on 05 Dec 2017 at 14:38:33, subject to the Cambridge Core terms of use, available at 1 https://www.cambridge.org/core/terms. https://doi.org/10.1017/S0140525X16000285 Hulleman & Olivers: The impending demise of the item in visual search process, namely the selection of individual items that are search. As we will see later, this qualitative distinction either rejected as distractors or recognized as a target. prompted an enduring empirical focus on the shallower There are a number of promising alternative perspectives end of the search slope spectrum as the most informative that ground the search process in eye fixations rather about the fundamental mechanisms of visual search. than covert selections of individual items. We argue that After all, somewhere between 0 ms/item and around 25 these approaches, when unified, provide a more compre- ms/item (for target-present trials) the transition to item hensive framework for explaining the oculomotor and search occurs. Consequently, search beyond this range manual response dimensions of visual search behaviour. has been considered to have little additional theoretical The goal of this paper is to provide this unification. value. FIT opened up an abundance of research questions. It predicted binding errors, where features are combined 2. Setting the stage: Feature Integration Theory incorrectly (e.g., Treisman & Schmidt 1982). It also and its assumptions inspired a taxonomy of basic features, by providing the diagnostic of flat search slopes (see Wolfe & Horowitz FIT was never intended as a theory of visual search pro- 2004, for an overview). And importantly, because of its fun- per, but rather used the visual search paradigm to test damental distinction between parallel feature search and its predictions about the way early sensory processing serial conjunction search, FIT encouraged other research- fi produces object representations. Nevertheless, it is dif - ers to challenge the core of the theory by finding conjunc- fl cult to overestimate its in uence on the formulation of tions of features that nevertheless yielded flat search slopes. the visual search problem. The fundamental distinction Success in this endeavour (e.g., Nakayama & Silverman fl between search with at slopes, where the time taken to 1986; McLeod et al. 1988; Wolfe et al. 1989) gave rise to fi nd the target is independent of the number of distractors new models (Duncan & Humphreys 1989; Wolfe et al. (e.g., / amongst |), and search with steeper slopes, where 1989) and to adaptations of FIT (Treisman & Sato 1990; search time increases with set size (e.g., red / amongst Treisman 1991). green / and red |), had been made before (e.g., Jonides & Gleitman 1972). But Treisman and Gelade’s(1980) FIT provided an attractive explanation. In its original version, 3. Popular alternative theories: Guided Search, visual features (e.g., colour, orientation, motion) are pre- Attentional Engagement Theory, and Signal attentively registered in parallel in separate feature maps. Detection approaches So, whenever the target differs from the distractors by a single feature (e.g., red amongst green), search time is 3.1. Guided Search independent of set size. Target presence is simply estab- Guided Search, the hitherto most successful model, was lished by inspecting activity in the relevant feature map. conceived to challenge FIT’s fundamental distinction Identifying a target that is a conjunction of features (e.g., between parallel feature and serial conjunction search. red | amongst green | and red /), however, requires serially Wolfe et al. (1989) adapted FIT such that information applied attention to bind the features together, using a from the feature maps guides attention towards conjunc- map that contains the item locations. Consequently, when- tions as well. Across several updates (Wolfe 1994; 2007; fi ever the target is de ned by a combination of features, Wolfe & Gancarz 1996) the basic principle has remained RTs increase with set size. Thus, FIT explained the quan- unchanged: Guided Search combines signals from different titative difference between single feature and conjunc- feature maps into a single activation map via broadly tion search slopes as a qualitative difference between tuned (“categorical”) channels (e.g., “red,”“green,”“verti- “ ” “ ” parallel, map -based search and serial, item -based cal,”“horizontal”). The activation map holds the locations of the individual items, and attention is guided towards the location with the highest activation. If it contains the target, a target-present response follows. However, because of inherent noise, it may contain a distractor. In JOHAN HULLEMAN is a lecturer in the School of Biolog- that case, attention is guided to the location with the ical Sciences at The University of Manchester. He next-highest activation. This continues until the target is received his Ph.D. from the University of Nijmegen. found or search is terminated with a target-absent His main interest is in visual perception, visual attention, response. and visual search. He has (co-)authored over 30 publica- fi tions on these topics. Top-down weighting or ltering of the channels improves search efficiency. For example, for a green-hori- CHRIS OLIVERS received his Ph.D. in 2001 from the zontal target and distractors that are red-horizontal and University of Birmingham, United Kingdom.