Visual Memory Capacity in Transsaccadic Integration
Total Page:16
File Type:pdf, Size:1020Kb
Exp Brain Res (2007) 180:609–628 DOI 10.1007/s00221-007-0885-4 RESEARCH ARTICLE Visual memory capacity in transsaccadic integration Steven L. Prime Æ Lia Tsotsos Æ Gerald P. Keith Æ J. Douglas Crawford Received: 14 June 2006 / Accepted: 9 January 2007 / Published online: 16 February 2007 Ó Springer-Verlag 2007 Abstract How we perceive the visual world as stable by inputting a noisy extra-retinal signal into an eye- and unified suggests the existence of transsaccadic centered feature map. Our results suggest that trans- integration that retains and integrates visual informa- saccadic memory has a similar capacity for storing tion from one eye fixation to another eye fixation simple visual features as basic visual memory, but this across saccadic eye movements. However, the capacity capacity is dependent both on the metrics of the sac- of transsaccadic integration is still a subject of con- cade and allocation of attention. troversy. We tested our subjects’ memory capacity of two basic visual features, i.e. luminance (Experiment 1) Keywords Visual perception Á Saccades Á Visual and orientation (Experiment 2), both within a single working memory fixation (i.e. visual working memory) and between separate fixations (i.e. transsaccadic memory). Exper- iment 2 was repeated, but attention allocation was Introduction manipulated using attentional cues at either the target or distracter (Experiment 3). Subjects were able to A central question in cognitive neuroscience is how we retain 3–4 objects in transsaccadic memory for lumi- perceive a unified and continuous visual world despite nance and orientation; errors generally increased as viewing it in a disjointed and discontinuous manner. saccade size increased; and, subjects were more accu- The typical observer makes 2–5 eye movements, called rate when attention was allocated to the same location saccades, per second (Rayner 1978, 1998). This means as the impending target. These results were modelled that a visual scene is processed in what is often per- ceived as discrete ‘snapshots’ during eye fixations of approximately 300 ms between saccades (Henderson S. L. Prime Á G. P. Keith Á J. D. Crawford (&) Centre for Vision Research, York University, and Hollingworth 1998). In some way, the brain retains 4700 Keele Street, Toronto, ON, Canada M3J 1P3 and integrates visual input from each eye fixation to e-mail: [email protected] form a unified mental percept of the visual sce- ne—often this leaves observers with the impression S. L. Prime Á G. P. Keith Á J. D. Crawford CIHR Group for Action and Perception, that their perception of the visual world is much like York University, 4700 Keele Street, putting together a jigsaw puzzle. This process is called Toronto, ON, Canada M3J 1P3 transsaccadic integration (e.g. Irwin 1991; Prime et al. 2006). S. L. Prime Á G. P. Keith Á J. D. Crawford Department of Psychology, York University, A related, but separate, process that is necessary for 4700 Keele Street, Toronto, ON, Canada M3J 1P3 transsaccadic integration is transsaccadic memory: the ability to retain information from previous fixations. L. Tsotsos Á J. D. Crawford However, the capacity of such memory is a much- Department of Biology and Kinesiology and Health Sciences, York University, debated topic. Some early studies assumed that de- 4700 Keele Street, Toronto, ON, Canada M3J 1P34 tailed visual information is retained across saccades 123 610 Exp Brain Res (2007) 180:609–628 in an internal ‘spatial buffer’ (Jonides et al. 1982; when reporting on the object closest to the next to-be- McConkie and Rayner 1976). Evidence of the exis- fixated target (i.e. saccade-target). This finding has tence of such a highly detailed spatial buffer has been since been replicated and expanded in several other questioned by several studies (Bridgeman and Mayer studies (Currie et al. 2000; Henderson and Holling- 1983; Irwin et al. 1983; McConkie and Zola 1979; worth 1999; McConkie and Currie 1996). This has led O’Regan and Levy-Schoen 1983; Rayner and Pollatsek to the suggestion that the allocation of attention to 1983). In particular, it has been noted that observers intended saccade-targets plays a crucial role in trans- are largely insensitive to unexpected changes in the saccadic integration—the ‘saccade-target’ theory (see visual scene that occur during saccades and other visual Deubel and Schneider 1996; Kowler et al. 1995). disruptions (Grimes 1996; Simons 1996; Simons and Several important questions remain. First, in most Levin 1997; O’Regan et al. 2000; Rensink et al. 1997). previous studies of transsaccadic memory, the These ‘change blindness’ experiments have questioned remembered objects, such as letters and toys, had the very existence of transsaccadic memory and semantic meaning for human subjects (e.g. Irwin and prompted some researchers to propose that visual Zelinsky 2002; cf. Carlson et al. 2001; Deubel et al. memory is essentially wiped clean with the acquisition 2002). It is possible that these stimuli engaged memory of a new fixation after an eye movement (Bridgeman systems or top–down processes that are not available to et al. 1994; O’Regan 1992; Tatler 2001). early vision. Moreover, while this may show that However, there are other explanations for trans- meaningful objects can be retained across saccades saccadic change blindness that do not rely on the (e.g., for the purpose of complex scene analysis), little assumption that memory is wiped clean (Niemeier is still known how many pre-semantic visual features et al. 2003). Moreover, a ‘‘no memory’’ theory of visual can be stored, for example to recognize larger objects search is inconsistent with evidence that scene per- that are not visible in a single fixation. Second, a few ception is more accurate when observers are free to studies have shown evidence of transsaccadic memory make eye movements (Schilingensiepen et al. 1986) in the absence of allocentric cues (De Graef et al. 2001; requiring an accumulation of visual information across Hayhoe et al. 1991; Verfaillie 1997). However, in these saccades during visual search (McCarley et al. 2003; studies allocentric spatial cues were not entirely elim- Peterson et al. 2001) even for more complex natural inated and remained visible across saccades (e.g. the scenes (Melcher 2001; Tatler et al. 2003, 2005). edges of the computer monitor), which likely aided in Most recent studies propose an intermediate view of solving the problem of spatial correspondence between transsaccadic memory, where some, but not all infor- pre- and post-saccadic stimuli. Moreover, accuracy in mation is retained across saccades to influence pro- the transsaccadic integration study by Hayhoe et al. cessing of visual information in subsequent fixations (1991) still depended on a reference point remaining (e.g. Pollatsek et al. 1984; Rayner and Pollatsek 1983; present during the saccade trials for performance to be Rayner et al. 1980). It is well known that visual on par with a similar fixation condition. Thus, it is still working memory can maintain about four objects each unknown if transsaccadic memory has the same as units of integrated feature conjunctions (Luck and capacity when forced to rely on the egocentric sense of Vogel 1997; Vogel et al. 2001), depending on the eye movement. It is also not known how saccade complexity of the object’s features (Alvarez and Cav- metrics might influence this process. If signals from the anagh 2004). These studies, however, did not take into oculomotor system are used to ‘update’ visual memory, account eye movements during the memory delay then one would expect performance to degrade with interval. If transsaccadic memory does not exist as saccade size, because larger saccades are underrepre- suggested by some of the studies mentioned earlier sented compared to smaller saccades in oculomotor (e.g. O’Regan 1992), subjects in the study by Luck and topography (Gandhi and Keller 1999; Robinson 1972; Vogel (1997), for example, should have difficulty in Sparks and Hartwich-Young 1989). maintaining objects in memory when a saccade is ad- Finally, in previous studies, the attended location ded to the task. This does not seem to be the case. usually coincided with the target of the next saccade; Irwin and colleagues have found that subjects can re- thus, little is still known how the dissociation of tain up to 3–4 objects when cued to remember several attention from the saccade-target modulates trans- visual stimuli across saccades (Irwin 1992; Irwin and saccadic memory. One exception is a study by Irwin Andrews 1996; Irwin and Gordon 1998). and Gordon (1998). They manipulated their subjects’ Moreover, these studies by Irwin and colleagues attention by instructing subjects to pay attention to a (Irwin 1992; Irwin and Andrews 1996; Irwin and Gor- particular location of a stimulus array about to be don 1998) have found that subjects were most accurate presented. In the present study, however, we used a 123 Exp Brain Res (2007) 180:609–628 611 more rigorous method of manipulating attention by not the case. Another possibility may be that the two using exogenous cues intended to draw the subjects’ memory systems are identical except transsaccadic attention to a specific stimulus—either the critical to- memory preferentially retains items related to the be-remembered target or a distracter—rather than a saccade due to attention (e.g. Kowler et al. 1995). If general location as in Irwin and Gordon (1998). this were the case we would expect judgement accuracy In a recent paper we showed that humans can spa- of unattended items (e.g. unrelated to the saccade) to tially retain and integrate a single pre-saccadic feature be at chance. However, this did not occur in our study; with a single post-saccadic feature in the absence of non-saccade targets were remembered above chance any allocentric visual cues, although small errors oc- for at least 3–4 objects.