I0035-8711-380-1-06978.Pdf —
Total Page:16
File Type:pdf, Size:1020Kb
Journal of Vision (2020) 20(4):17, 1–24 1 Intra-saccadic motion streaks as cues to linking object locations across saccades Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany Bernstein Center for Computational Neuroscience Berlin, Berlin, Germany Berlin School of Mind and Brain, Humboldt-Universität Richard Schweitzer zu Berlin, Berlin, Germany Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany Bernstein Center for Computational Neuroscience Berlin, Berlin, Germany Berlin School of Mind and Brain, Humboldt-Universität Martin Rolfs zu Berlin, Berlin, Germany When visual objects shift rapidly across the retina, they produce motion blur. Intra-saccadic visual signals, Introduction caused incessantly by our own saccades, are thought to be eliminated at early stages of visual processing. Here The dystopian science-fiction television series “Black we investigate whether they are still available to the Mirror” featured an episode in which people could visual system and could—in principle—be used as cues record memories through their pupils (Armstrong & for localizing objects as they change locations on the Welsh, 2011). Replays of these memories revealed a retina. Using a high-speed projection system, we common but wrong intuition of how our eyes capture developed a trans-saccadic identification task in which the world around us. They showed skillfully crafted brief but continuous intra-saccadic object motion was videos, with smooth camera movements from one key to successful performance. Observers made a location to the next. In reality, we make several saccadic saccade to a target stimulus that moved rapidly either up eye movements every second that rapidly shift the or down, strictly during the eye movement. Just as the entire image of the visual scene across the retina. Upon target reached its final position, an identical distractor each new fixation, each object in the scene is projected stimulus appeared on the opposite side, resulting in a onto a new part of the retina and processed by new display of two identical stimuli upon saccade landing. populations of neurons throughout retinotopic visual Observers had to identify the original target using the only available clue: the target’s intra-saccadic cortex. Yet, these jerky displacements are not part of movement. In an additional replay condition, we our perceptual experience—the visual world is stable. presented the observers’ own intra-saccadic retinal Whereas this phenomenon has received attention for stimulus trajectories during fixation. Compared to the centuries and has inspired research and theory to this replay condition, task performance was impaired during date (Binda & Morrone, 2018; Burr & Morrone, 2011; saccades but recovered fully when a post-saccadic blank Cavanagh, Hunt, Afraz, & Rolfs, 2010; Hall & Colby, was introduced. Reverse regression analyses and 2011; Higgins & Rayner, 2015; Marino & Mazer, 2016; confirmatory experiments showed that performance Wurtz, 2018; Ziesche & Hamker, 2014), a fundamental increased markedly when targets had long movement question remains unanswered: How does the visual durations, low spatial frequencies, and orientations system keep track of an object that is changing locations parallel to their retinal trajectory—features that on the retina as the eyes move (Rolfs, 2015; Wurtz, promote intra-saccadic motion streaks. Although the 2008)? That is, how do we determine a correspondence potential functional role of intra-saccadic visual signals between two successive views of an object across a is still unclear, our results suggest that they could saccade? provide cues to tracking objects that rapidly change Here we hypothesized that intra-saccadic information locations across saccades. could contribute to object identification across saccades. Citation: Schweitzer, R. & Rolfs, M. (2020). Intra-saccadic motion streaks as cues to linking object locations across saccades. Journal of Vision, 20(4):17, 1–24, https://doi.org/10.1167/jov.20.4.17. https://doi.org/10.1167/jov.20.4.17 Received July 13, 2019; published April 25, 2020 ISSN 1534-7362 Copyright 2020 The Authors Downloaded from jov.arvojournals.orgThis work ison licensed 06/10/2020 under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. Journal of Vision (2020) 20(4):17, 1–24 Schweitzer & Rolfs 2 For example, motion streaks, which occur due to the prolonged post-movement durations of the light source slow integration of visual signals at early stages of being on, perceived streaks were still significantly visual processing (Geisler, 1999), are generated each longer during fixation than during saccades, suggesting time eye movements cause object movement across an additional attenuation of smear around saccades. the retina (Bedell & Yang, 2001; Brooks, Impelman, More recently, Duyck et al. (2016) used an objective & Lum, 1981; Duyck, Collins, & Wexler, 2016; technique to quantify streak efficiency by tasking their Matin, Clymer, & Matin, 1972). Motion streaks—and participants with localizing a gap (realized by briefly intra-saccadic smear in general—imposed by our dimming a light-emitting diode during the saccade) in own eye movements have been widely considered a an intra-saccadic motion streak. hindrance to perceptual stability that are counteracted All of the studies showed that the perceived streak by specialized mechanisms and thus eliminated from was directly related to presentation duration, revealed perception (for a collection of examples, see Castet, the location of the inducing light source, and could 2010). These mechanisms range from passive accounts, be attenuated by presenting prolonged post-movement such as shearing forces in the retina that reduce visual endpoints. Although it is again impossible to conclude sensitivity during saccades (Richards, 1969) and pre- from these results whether or not intra-saccadic motion and post-saccadic masking (Castet, 2010; Matin et al., streaks are relevant to the visual system, the hypothesis 1972), to active suppression of information in the that they could be is informed by three insights. First, magnocellular pathway (Burr, Morrone, & Ross, 1994; motion and contrast detection during saccades have Ross, Morrone, Goldberg, & Burr, 2001), as well as been shown to be possible, provided that the presented combinations of these (Volkmann, Riggs, White, & stimulus has been optimized for the direction and Moore, 1978; Wurtz, 2018). We know, however, that velocity of the saccade (Castet & Masson, 2000; intra-saccadic motion perception is possible if the Castet et al., 2002; García-Pérez & Peli, 2011; Mathôt, stimulus has a velocity similar to that of the saccade Melmi, & Castet, 2015; Schweitzer & Rolfs, 2019). (Castet & Masson, 2000; Castet, Jeanjean, & Masson, Second, information undergoing perceptual omission 2002). Similarly, if the visual scene is briefly illuminated is not eliminated from visual processing (Watson & during a saccade, observers have a clear impression Krekelberg, 2009). Third, in a static visual environment, of a smeared and blurry visual scene (Campbell self-induced retinal input is related to any ongoing eye & Wurtz, 1978). In addition, stimuli undergoing movement and could thus provide information about saccadic suppression can still influence post-saccadic the direction, amplitude, and velocity of eye movements judgments, even if the observer is unaware of them (Matin et al., 1972). (Watson & Krekelberg, 2009). Recently, the hypothesis To test the hypothesis whether it is, in principle, has been proposed that effectively modulating the possible to use strictly intra-saccadic continuous object spatiotemporal power distribution in the retinal image motion to link the identities of objects across saccades saccades enhances low spatial frequencies (SFs) and as they change locations on the retina, we developed a thus facilitates a coarse-to-fine strategy of post-saccadic trans-saccadic identification paradigm. Observers (N visual processing (Boi, Poletti, Victor, & Rucci, 2017; = 15) made a saccade toward a target stimulus in the Rucci, Ahissar, & Burr, 2018). Indeed, Boi et al. (2017) visual periphery. Upon saccade landing, the display showed that contrast sensitivity to post-saccadic low contained two stimuli—the target stimulus and an SF (but not high SF) information is greater if the identical distractor—one above and one below the stimulus has its onset during the saccade than if the target’s previous location. Observers had to identify same stimulus is presented with a contrast ramp during the original target stimulus, which had moved rapidly fixation. A functional role of visual processing during but continuously to its new location as the eyes were in the saccade might thus be facilitating the processing of flight (see Methods section). The distractor stimulus, coarse information during early fixation. In particular, in turn, merely appeared on the opposite side as soon a potential function role of intra-saccadic visual as the target stimulus had reached its final location. information, such as motion streaks, remains elusive. To identify the target stimulus in a two-alternative Perception of intra-saccadic motion streaks induced forced choice task (Figure 1a), therefore, observers by saccades across a static stimulus (often presented in a could not simply rely on detecting a displacement dimly lit, uniform background or in complete darkness) (Brooks, Yates, & Coleman, 1980; Wexler & Collins, has been investigated in past studies.