Journal of Vision (2003) 3, 761-770 http://journalofvision.org/3/11/10/ 761

Smooth anticipatory eye movements alter the memorized position of flashed targets

Center for Systems Engineering and Applied Mechanics and Laboratory of Neurophysiology, Université catholique Gunnar Blohm de Louvain, Louvain-la-Neuve, Belgium Laboratory of Neurophysiology, Université catholique de Marcus Missal Louvain, Brussels, Belgium Center for Systems Engineering and Applied Mechanics and Laboratory of Neurophysiology, Université catholique Philippe Lefèvre de Louvain, Louvain-la-Neuve, Belgium

Briefly flashed visual stimuli presented during smooth object- or self-motion are systematically mislocalized. This phenomenon is called the “flash-lag effect” (Nijhawan, 1994). All previous studies had one common characteristic, the subject’s sense of motion. Here we asked whether motion perception is a necessary condition for the flash-lag effect to occur. In our first experiment, we briefly flashed a target during smooth anticipatory eye movements in darkness and subjects had to orient their toward the perceived flash position. Subjects reported to have no sense of eye motion during anticipatory movements. In our second experiment, subjects had to adjust a cursor on the perceived position of the flash. As a result, we show that gaze orientation reflects the actual perceived flash position. Furthermore, a flash-lag effect is present despite the absence of motion perception. Moreover, the time course of gaze orientation shows that the flash- lag effect appeared immediately after the egocentric to allocentric reference frame transformation.

Keywords: anticipation, , , localization, flash perception, flash-lag effect

One condition in which localization errors happen is Introduction during smooth object- or self-motion. An impressive It seems natural that in our everyday life, we demonstration of such a perceptual mislocalization has experience the visual environment to be spatially stable. first been conducted by MacKay (1958) and has later been However, when we orient gaze or move through the visual rediscovered by Nijhawan (1994). In their experiment, scene, self-motion induces an optic flow. Thus, for a two strobed segments that were flashed in alignment with stable space perception during self-motion, the central a continuously lit rotating line lagged the moving object, nervous system (CNS) has to use extraretinal signals to and the size of the lag increased with angular velocity. compensate for retinal motion. In this condition, the This phenomenon is called the “flash-lag effect” and has question arises, “How does the brain combine visual been extensively studied using the original, rotational signals from the environment with internal signals related configuration (Brenner & Smeets, 2000; Eagleman & to self-motion?” Sejnowski, 2000; Krekelberg & Lappe, 1999; Lappe & An interesting way to address the issue of a stable Krekelberg, 1998; Purushothaman, Patel, Bedell, & percept of the environment is to perform a localization Ogmen, 1998), as well as using constant linear motion task. To localize a flashed object, the brain has to (Brenner, Smeets, & van den Berg, 2001; Whitney, integrate the object’s retinal location with an extra-retinal Cavanagh, & Murakami, 2000; Whitney & Murakami, signal about the direction of gaze (Bridgeman, 1995; 1998). However, the flash-lag effect is not restricted to Festinger & Canon, 1965; Mergner, Nasios, Maurer, & moving objects but also appears during smooth pursuit Becker, 2001; Mon-Williams & Tresilian, 1998; van eye movements (Kerzel, 2000; Nijhawan, 2001; van Beers Beers, Wolpert, & Haggard, 2001). It is particularly et al., 2001). This is also the case for head or whole body difficult for the CNS to match moving and flashed movements (Schlag, Cai, Dorfman, Mohempour, & stimuli because continuously changing variables (i.e., Schlag-Rey, 2000). In both conditions, retinal signals position and velocity signals) have to be matched at key about motion are absent. Even more spectacular is the events, although they are processed with different delays. finding that in certain conditions a flash-lag appears A systematic bias of the perceived position shows the without any motion at all. Illusory motion perception can limits of this process (for a review, see Schlag & Schlag- induce a perceptual bias of a flashed target (Cai, Rey, 2002). doi:10.1167/3.11.10 Received March 21, 2003; published December 4, 2003 ISSN 1534-7362 © 2003 ARVO Blohm, Missal, & Lefèvre 762

Jacobson, Baloh, Schlag-Rey, & Schlag, 2000; Nishida & moving 0.2° red laser target was back-projected onto the Johnston, 1999; Watanabe, Nijhawan, & Shimojo, 2002). screen. The target was controlled via an M3-Series mirror The flash-lag effect shows two characteristic features. galvanometer (GSI Lumonics) and by using a dedicated First, there is a difference in flash localization error computer running LabViewRT (National Instruments, depending on the retinal location of the flash with respect Austin, TX, USA) software. Movements of one eye were to the motion direction (i.e., a flash presented ahead or recorded with the scleral coil technique, Skalar Medical behind the gaze direction is mislocalized differently) BV (Collewijn, van der Mark, & Jansen, 1975; Robinson, (Kerzel, 2000; Nijhawan, 2001; van Beers et al., 2001; 1963). Whitney et al., 2000; Whitney & Murakami, 1998). Paradigm Second, the final gaze orientation error depends linearly on the eye and/or target velocity at the moment of the Recording sessions were composed of a series of flash (Brenner et al., 2001; Nijhawan, 2001; van Beers et blocks, each containing 40 trials. During the first block of al., 2001). trials, the moving target was always present. These trials In all previous studies of the flash-lag effect, it could were used to build up an anticipatory response and will be hypothesized that it was exclusively due to the be referred to as build-up trials. After one block of build- perception of motion. In these experiments, a perception up trials, several blocks of test trials randomly mixed with of motion was induced either by target motion, self- build-up trials were presented. motion, or illusory motion. Is motion perception Build-up trials (Figure 1A) started with an 800-ms necessary to evoke a flash-lag? Here we tested whether a fixation period at the center of the screen. After a 300-ms flash-lag could be induced by self-motion but in the target extinction period (gap), the target moved from the absence of motion perception. This was done by testing center of the screen for 800-ms at 40°/s, always in the subjects in a situation where there is no perception of same direction. The trial ended with another 500-ms motion despite smooth anticipatory eye movements. In fixation period. The gap duration was chosen to provide our first experiment, we designed a paradigm inducing an optimal smooth anticipatory (Morrow smooth anticipatory eye movements that were not & Lamb, 1996). perceived by subjects (Kowler & Steinman, 1979). During these smooth eye movements, we presented briefly a flashed target and asked subjects to orient gaze toward the A remembered target position. In our second experiment, 800ms, 0¼ we validated our gaze orientation approach by a perceptual localization task. As a result, we rule out the hypothesis that the bias in spatial perception is due to 300ms, gap motion perception. Indeed, we show that spatial perception of human subjects can be altered by self- 800ms, 40¼/s motion in the absence of the sense of motion. 500ms, 32¼ Methods B

Experiment 1 800ms, 0¼ 100-500ms, gap Experimental set-up Healthy human subjects without any known oculomotor abnormalities participated in the experiment 10ms, [-15¼..15¼] after informed consent. Among the seven subjects, three were completely naïve of oculomotor experiments. Mean 1.100-1.500ms age was 29 years, ranging from 22 to 36 years. All procedures were conducted with approval of the Université catholique de Louvain Ethics Committee, in compliance with the Helsinki declaration (1996). Experiments were conducted in a completely dark Figure 1. Experimental paradigm. A. Build-up trials. After an room. Subjects sat in front of a 1-m distant tangent 800-ms fixation and a 300 ms gap period, the target moved screen, which spanned about ±45° of their visual field. for 800 ms at 40°/s, always in the same direction. B. Test Their heads were restrained by a chin-rest. A horizontally trials. Test trials started like build-up trials with an 800-ms fixation. After a variable gap of 100-500 ms, a 10-ms flash appeared at a random position between -15° and 15°.

Blohm, Missal, & Lefèvre 763

In the second part of the recording session, build-up trials were randomly interleaved with 30% of test trials 20 flash Eye (Figure 1B). Test trials started in the same way as build-up 15 trials with an 800-ms fixation period in the center of the 10 PE(t)

screen. Afterwards, instead of the fixed gap followed by a flash 5 ramp target motion, the target disappeared for a random PE EV duration lasting between 100 and 500-ms. This variable Position (deg) flash 0 gap was followed by a 10-ms flash presented at a random position ±15° around the expected eye position (= target 0 500 500 position of build-up trials). All trials lasted for 2400-ms. Subjects were instructed to follow the target as accurately Time (ms) as possible during build-up trials, and to orient gaze to the Figure 2. Data analysis. After the occurrence of the first memorized target (flash) position during test trials. orientation , eye position error (PE) was sampled Data acquisition and analysis every 50 ms. Dotted black lines represent the initial fixation point and the flash position. The star stands for the flash. The Eye and target position were sampled at 500-Hz and red line represents eye position (saccades in bold). At the stored on the hard disk of a PC for offline analysis with moment of the flash, position error (PEflash) and eye velocity Matlab (Mathworks) scripts. Position signals were low-pass (EV ) were also measured. filtered using a zero-phase digital filter (autoregressive flash forward-backward filter, cutoff frequency: 50-Hz). Velocity rate CMOS sensors (Clarke, Ditterich, Druen, Schonfeld, and acceleration were derived from position signals using & Steineke, 2002). a central difference algorithm. Paradigm In our analysis, only test trials were considered. We were interested in the gaze orientation mechanism toward We used a paradigm similar to the one used in the flashed target. Position error (PE) and eye velocity Experiment 1, except that we had to reduce the range of (EV) signals were measured at the moment of the flash. the flash position to –10°…10° because of the limited The position error is the difference between target (T) screen size. Recording sessions were composed of a series and eye (E) position at a given moment in time: PE=T–E. of blocks containing 40 trials each. Each block started All trials were aligned on the flash onset. In order to with three build-up trials (Figure 1A). Afterwards, build- describe the flash localization process, PE was measured up and test trials (Figure 1B) were mixed with 50% every 50-ms starting at the end of the first saccade until probability. However, in contrast with Experiment 1, after 1000-ms after the flash (see Figure 2). each test trial there was a perceptual localization task (Figure 3) and the next trial was always a build-up trial. Afterwards, there was again a 50% probability for either a Experiment 2 build-up or a test trial to appear. Experimental set-up Three out of the seven subjects of Experiment 1 800ms, 0¼ participated in this experiment after informed consent. All procedures were conducted with approval of the 100-500ms, gap Université catholique de Louvain Ethics Committee, in compliance with the Helsinki declaration (1996). 10ms, [-10¼..10¼] Experiments were conducted in a completely dark room. Subjects sat in front of a 0.4-m distant, 21 in. Sony 1.100-1.500ms GDM-F520 computer screen on which we presented either a 0.5° red circular target or a white vertical cursor. The screen refresh rate and resolution were 100-Hz and cursor alignment 640 x 480 pixels, respectively. Subjects were asked to use a computer mouse to move the cursor. The cursor was mouse click at perceived composed of two vertical 0.1° large and 1.0° high bars flash location that were aligned horizontally and separated vertically by 0.65°. Target and cursor presentation were controlled in Figure 3. Perceptual localization task. The trial started with an real time by a VSG2/5 Visual Stimulus Generator (32MB 800-ms fixation period, a 100-500-ms gap and a 10-ms flash at VRAM) (Cambridge Research Systems Ltd). a random position –10°…10°. After each test trial, subjects Subjects’ heads were restrained by a chin-rest. Their localized the memorized perceived position of the flash by eye movements were recorded using a Chronos eye means of a cursor. The cursor position could be adjusted by a tracker, Skalar Medical BV, which is based on high-frame computer mouse, and subjects had to press the mouse button to validate their choice of the perceived flash position.

Blohm, Missal, & Lefèvre 764

The perceptual localization task consisted of the alignment of the cursor with the memorized perceived Results position of the flash. Subjects had to press the mouse button to validate their choice of the position of the Experiment 1 cursor. Data acquisition and analysis General observations The target position was sampled at 100-Hz and stored At the end of each trial, the eyes pointed at the on the hard disk of a PC. Images of the eyes were sampled perceived location of the flashed target. Figure 4 shows independently of the target at 100-Hz and stored on the four typical trials that illustrate the behavior and the gaze hard disk of a second PC. The eye position was extracted orientation performance. If the eyes move toward the offline from the eye images using the polar correlation flash, the condition is called foveopetal (FP); otherwise, algorithm for an ellipse approximation of the iris (Clarke we will refer to it as foveofugal (FF). The smooth eye et al., 2002), as implemented in the Iris software (Skalar displacement during the latency period of the first Medical BV). The cursor position of the perceptual orientation saccade resulted in an overshoot of the first localization task was also recorded. Synchronization saccade in the FP condition, whereas first saccades in FF between eye and target signals was performed by means of trials undershot. Although in Figure 4B and 4D the flash a TTL signal. localization is rather precise (FF condition), in Figure 4A In this experiment, four parameters of interest were and 4C, we observe a remaining error on the final eye extracted from the recording files (i.e., position error position (FP condition). In fact, in the FP condition, subjects generally localized the flash ahead of its actual PEflash and eye velocity EVflash at the moment of the flash as well as the actual and perceived [= cursor] position of position. When asked after the experiment, subjects the flash). reported that they had no sense of performing smooth anticipatory eye movements during the gap. This is in accordance with previous findings (Kowler & Steinman, 1979).

15 A 15 B 10 10 5 5 0 0

-5 -5

Position (deg) Position Position (deg) Position -10 FP -10 FF EV = 9.5¼/s; PE = 9.9¼ EV = 8.8¼/s; PE = -6.9¼ -15 flash flash -15 flash flash -500 0 500 1000 1500 -500 0 500 1000 1500 Time (ms) Time (ms)

15 15 C D 10 10 5 5 0 0

-5 -5 Position (deg) Position

Position (deg) Position -10 FP -10 FF EV ==19.2¼/s; PE = 9.1¼ EV -15 flash flash -15 flash 18.9¼/s; PE flash = -10.4¼

-500 0 500 1000 1500 -500 0 500 1000 1500 Time (ms) Time (ms)

Figure 4. Typical trials. Black solid and dotted lines represent the fixation target and the flash position, respectively. The star stands for the flash. Eye position (red lines) and saccades (bold red lines) are shown for four different conditions. A and C. Flash presented at a foveopetal (FP) position during medium (A) and high (C) eye velocity. B and D. Flash presented at a foveofugal (FF) position during medium (B) and high (D) eye velocity.

Blohm, Missal, & Lefèvre 765

On average, subjects needed 2–3 saccades to orient perceptual offset. If –PEend > 0 (and thus PEend < 0 ), the their eyes to the memorized position of the flash. The flash is perceived ahead of its actual position in the general properties and dynamics of this orientation direction of the smooth eye movement. process have been previously described in detail (Blohm, Table 1. Mean Values and Ranges of Parameters That Missal, & Lefèvre, 2003). Here we will concentrate on the Characterize the Visual Localization Data Set in Experiment 1. directional bias in space perception, which results from the fact that the eyes were moving at the moment of the Variable Case Values Range N flash. (mean ± SD) [25...75]% To quantify smooth eye velocity after the flash, we |PEflash| FF 6.643 ± 4.100 [3.061...9.578] 765 provide in Figure 5A the mean behavior and raw data. FP 6.530 ± 3.775 [3.454…9.515] 762 Panel B shows the smooth eye displacement, which is the EVflash FF 10.115 ± 8.595 [4.535...14.246] 765 FP [4.653...13.994] 762 eye displacement after removing saccades. The smooth 9.916 ±8.398 -PE FF [0.129...1.070] 765 eye displacement is thus the integration of the smooth eye end 0.587 ± 1.068 FP 1.592 ± 2.212 [0.559...2.692] 762 velocity. The total smooth eye displacement depended on the eye velocity at the moment of the flash: the higher the FP indicates foveopetal; FF, foveofugal. eye velocity at the moment of the flash, the larger the total smooth eye displacement. The two signatures of the flash-lag phenomenon are analyzed in Figure 6. Figure 6A represents the perceptual offset of the flash for the different flash locations on the 40 A

30

g) 3 20 A FF FP

2.5 locity (deg/s) locity

10 fset (de 2 e

V 1.5 0 1 0.5 -10 0

Perceptual of 10 B -15 -10 -5 0 5 10 15

Pos. (deg) Pos. 0 Flash position relative to fovea (deg)

0 200 400 600 800 1000 3 B Time (ms) 2.5 FP fset (deg) 2 Figure 5. Smooth eye movement. A. Mean and SD of smooth 1.5 eye velocity (solid and dotted black lines) aligned on the flash 1 onset. All foveofugal (FF) and foveopetal (FP) conditions are 0.5 pooled. B. Mean and SD of the smooth eye displacement (solid and dotted black lines) accumulated since the flash onset. 0 FF

Perceptual of Gray lines show individual trials. Trials were aligned on flash 0 5 10 15 20 25 30 onset (time zero). Eye Velocity EV (deg/s) flash In our analysis, we will consider only the eye position PEflash and velocity EVflash at the moment of the flash. Figure 6. Final gaze orientation toward flashed targets during Indeed, PEflash is the only retinal signal that is available to smooth anticipation. A. Dependency of the perceptual offset the system to localize the flash (see “Discussion”). (-PEend) on the retinal target position relative to the fovea. B. Localization error The perceptual offset depends strongly on the eye velocity at the moment of the flash EV in foveopetal (FP) condition. In We examined in this section the two characteristic flash foveofugal (FF) condition, only a weak influence of the smooth features of a putative flash-lag effect. Table 1 summarizes eye velocity is observed. Whiskers indicate the SEM (see text the different parameters that were analyzed. Note that for details). Bins of 2.5° (panel A) and 5°/s (panel B) are –PE (PE = PE(1000-ms)) will be the measure of the end end centered on the binning interval.

Blohm, Missal, & Lefèvre 766 in the gaze orientation experiment. Figure 6A et al. (2003) showed that the first orientation saccade did clearly shows that there is an asymmetric perceptual bias not take into account the smooth eye displacement. in the flash localization. Thus, a flash presented during Therefore, at this time, the gain element β(t) is the same smooth anticipatory eye movements is mislocalized in the in the FP and FF condition (t test, p >.05). However, same way as a flash presented during smooth pursuit. afterwards, the p level that quantifies the difference of the A closer look at this asymmetrical perceptual effect is regression parameter β( t) between FP and FF conditions provided in Figure 6B. Here we separated the data into decreased. After 450-ms, the regression parameters β(t) two populations (i.e., foveopetal [FP] and foveofugal [FF] became significantly different (p <.05) and even highly flash presentations). We observed a clear effect of the significantly different after 650-ms (p<.001). Furthermore, smooth eye velocity at the moment of the flash EVflash on in Figure 7, the 95% confidence intervals of the the perceptual offset for the FP condition (see regressions for FP and FF conditions separately also Equation 2), whereas the effect was much more decrease, which indicates that individual regressions attenuated for the FF condition (see Equation 1). improve over time. Hence, although Equation 3 is not a signature of the flash-lag effect, the resulting separation of FF: -PEend = 0.163 + 0.029 ⋅ EVflash (N = 765, R both FP and FF populations in Figure 7B shows the = 0.1393, p = .023) (1) relevance of this analysis in characterizing the temporal evolution of the flash-lag effect. FP: -PEend = -0.073 + 0.129 ⋅ EVflash (N = 762, R = 0.3781, p < .001) (2) Figure 6B shows separately the dependence of the 2

(deg) A final gaze orientation error on the smooth eye velocity α α 0 EVflash at the moment of the flash for both FF and FP conditions. Individual analyses of the data for all subjects fset fset -2 resulted in regression coefficients that varied between Of FP –0.021…0.064 (N=78…190, p =.001…0.914) for the FF FF condition in Equation 1 and between 0.070…0.234 0.3 (N=83…194, p<.001…0.017) in the FP condition in B Equation 2.

Taken all together, we showed that targets flashed (s) 0.2 during smooth anticipatory eye movements are affected β by a flash-lag illusion. This was the case although subjects

had no perception of any eye movements when the flash 0.1 Gain Gain occurred. Temporal evolution of the error 0 After having shown in the previous section that an anticipatory smooth eye movement distorts the perceived 0 200 400 600 800 1000 space when tested with a short flash, we wondered Time (ms) whether we could reveal the temporal evolution of the flash-lag effect. Therefore, we performed the following regression analysis independently for both FF and FP Figure 7. Temporal evolution of the gaze orientation error. conditions and for each time step during gaze orientation: Regression variables of Equation 3 are represented for both foveofugal (FF) and foveopetal (FP) conditions and for each time PE(t) = α(t) - β(t) ⋅ EVflash (3) step. The gray zone shows the transition between the early and The results of the regression in Equation 3 are late orientation (see text for details) when the difference between represented in Figure 7. Individual regression coefficients FP and FF conditions became significant. Whiskers indicate the for each time step and FF or FP condition ranged from R 95% confidence intervals of the means. A. Offset α(t). B. Gain = 0.1393…0.7597 (p < .001…0.023). The nonzero offset α β(t). in the early orientation (Figure 7A) was essentially due to the saccadic undershoot strategy (Gellman & Fletcher, 1992) and the system compensated for this error later on Experiment 2 in the orientation process. To demonstrate that the final gaze orientation reflects Figure 7B shows the evolution of the error the perceived position of the flash, we performed a dependence on EVflash over time. Interestingly, in the perceptual localization experiment. As in Experiment 1, earlier orientation process, there was no difference subjects reported to have no sense of performing any between FP and FF relationships (p >.05). Indeed, Blohm smooth eye movements during the gap period.

Blohm, Missal, & Lefèvre 767

Table 2 summarizes the results of this experiment. Figure 8 illustrates the results of the perceptual Here the cursor localization error ERRloc replaces the eye localization experiment. Panel A shows the dependence position error PEend of Experiment 1. We verified that the of the localization error on the retinal location of the overall eye velocity at the moment of the cursor flash. Compared to the gaze orientation experiment in appearance was small (EV =–0.043 ± 0.928°/s, mean Figure 6, the flash-lag is qualitatively the same, although ±SD). the effect is more selective concerning the eye position error at the moment of the flash (see “Discussion”). Table 2. Mean Values and Ranges of Parameters That Figure 8B recapitulates the effect of the eye velocity Characterize the Perceptual Localization Data Set in (EV ) at the moment of the flash on the perceptual Experiment 2. flash localization of the target. Qualitatively, we obtained the Variable Case Values Range N same results as for Experiment 1: FP flashes are (mean ± SD) [25...75]% mislocalized in the direction of the eye movement |PEflash| FF 5.902 ± 3.568 [2.655...8.301] 312 whereas this behavior is much reduced for FF flashes. FP 4.299 ± 3.251 [1.924...6.875] 284 This is expressed in the following regression equations: EVflash FF 10.160 ± 9.401 [3.846...16.034] 312 FP 9.520 ± 9.498 [3.665...16.237] 284 FF: -ERRloc = 0.053 + 0.023 ⋅ EVflash (N = 312, R = -ERRloc FF 0.602 ± 1.310 [-0.325...1.619] 312 0.0831, p = .094) (4) FP 1.957 ± 2.021 [0.403...3.934] 284

FP indicates foveopetal; FF, foveofugal. FP: -ERRloc = 0.451 + 0.116 ⋅ EVflash (N = 284, R = 0.1967, p = .002) (5) Individual values of the regression coefficients for all 3 A FF FP subjects ranged from 0.017…0.029 (N = 84…131, p = 2.5 .143…0.697) for the FF condition in Equation 4 and

fset (deg) 2 0.083…0.154 (N = 81…128, p=.015…0.072) for the FP 1.5 condition in Equation 5. However, we would like to 1 point out that this perceptual localization task was 0.5 conducted to confirm that the final gaze orientation really 0 represents the perceived position of the flash. We claim

Perceptual of that our results show that both localization procedures -15 -10 -5 0 5 10 15 reveal the same phenomenon. Flash position relative to fovea (deg) Discussion A flash presented during perceived movement is 3 B perceptually mislocalized (Schlag & Schlag-Rey, 2002). 2.5 FP Such a bias has previously been observed for targets fset (deg) 2 presented briefly during smooth self-, object-, or illusory- 1.5 FF motion and is called “flash-lag.” Here we asked whether 1 the observed asymmetrical perceptual offset might be due 0.5 to motion perception. Therefore, a briefly flashed target 0 was presented during unperceived smooth anticipatory

Perceptual of eye movements in darkness, and subjects had to localize 0 5 10 15 20 25 30 the flash. This revealed the presence of a self-movement Eye Velocity EV (deg/s) induced flash-lag illusion despite the absence of the sense flash of self-movement. Thus, space perception is decoupled from movement perception, although a perceived Figure 8. Perceptual localization of flashed targets during movement might influence the perceived space (Cai et al., smooth anticipatory eye movements. A. The perceptual 2000; Nishida & Johnston, 1999; Watanabe et al., 2002). localization error (-ERRloc) was strongly modulated by the Gaze orientation and perceptual localization retinal location of the flash. B. Asymmetrical influence of the eye velocity (EVflash) at the moment of the flash on space Our gaze orientation and perceptual localization perception. Whiskers indicate the SEM. Gaze orientation results show the same trend (i.e., almost the same results (gray lines) are transposed from Figure 6 for dependency of the perceptual offset on the eye velocity at comparison. the moment of the flash and a very similar behavior for the influence of retinal flash position). However, we observed slightly different shapes and regression

Blohm, Missal, & Lefèvre 768 parameters in the gaze orientation task and the perceptual Next we discuss the egocentric to allocentric reference localization experiment. There might be a difference frame transformation and propose candidates for the between gaze orientation and manual localization motor possible underlying neural structures of the flash-lag strategies. Indeed, it has been shown that visually guided effect. manual pointing to remembered targets leads to an Origin of the perceptual bias overshoot for small target eccentricities, whereas larger distances are more likely to be undershot (Berkinblit, The asymmetry between FP and FF conditions Fookson, Smetanin, Adamovich, & Poizner, 1995; suggests a hemifield effect, whereas the time course of the Medendorp, Van Asselt, & Gielen, 1999). Furthermore, a flash-lag effect could indicate the involvement of two comparison of gaze orientation and perceptual different pathways. It has been suggested (Blohm et al., localization of briefly presented targets reveals systematic 2003) that the early orientation involves a direct pathway undershoots for gaze orientation compared to an from the primary visual areas to the saccade generator (for overshoot in perceptual localization (Eggert, Ditterich, & a review, see Krauzlis & Stone, 1999). Therefore, first Straube, 2001; Eggert, Sailer, Ditterich, & Straube, 2002). orientation saccades are coded in an egocentric reference However, despite these possible differences, we observe frame. A second pathway involving middle temporal (MT) very similar regression coefficients in Equations 1 and 2 and medial superior temporal (MST) areas and the compared to Equations 4 and 5. posterior parietal cortex (PPC) has been proposed to The perceptual localization task allowed us to validate compensate for the smooth eye displacements (Blohm et our approach and to confirm that the final gaze direction al., 2003). PPC is also known to code the egocentric to reflects the perceived position of the flashed target. In allocentric reference frame transformation (Andersen, addition, the position of the cursor when the mouse Essick, & Siegel, 1985; Pack, Grossberg, & Mingolla button was pressed gave us direct information on the 2001; Heide, Blankenburg, Zimmermann, & Kompf, perceived flash position. 1995) and areas MT/MST contain neurons that are direction and speed selective (Born & Tootell, 1992; Time course of the flash-lag illusion Mikami, Newsome, & Wurtz, 1986; Newsome, Wurtz, & Our analysis in Figure 7 answered the question about Komatsu, 1988; van Wezel & Britten, 2002) and that the timing of the flash-lag effect in more details. Indeed, carry information about eye movements (Bradley, we showed when the visual illusion begins affecting the Maxwell, Andersen, Banks, & Shenoy, 1996; Eifuku & eye movements. This is in accordance with previous Wurtz, 1998; Komatsu & Wurtz, 1988a, 1988b; results showing that the first orientation saccade does not Newsome et al., 1988; Squatrito & Maioli, 1997). take into account any movement-related information but Interestingly, Krekelberg, Kubischik, Hoffmann, and is only based on the position error PEflash at the moment Bremmer (2003) recently showed that areas MT/MST are of the flash (Blohm et al., 2003). Furthermore, Blohm et strongly involved in the spatial localization of a flashed al. (2003) revealed a 400-ms delay to account for the target, and the neural activity in these areas codes the smooth eye displacement. If the eye moved smoothly in perisaccadic mislocalization of flashes. Therefore, we darkness, the system corrected for this smooth eye propose that the same brain regions might be responsible displacement around 400-ms later by means of a for the flash-lag effect. corrective saccade. Thus, the 400-ms delay represents the Our manual localization data strongly support this time needed for the manifestation of the egocentric to hypothesis, whereas the gaze orientation results might allocentric reference frame transformation. This finding reflect other, more complex mechanisms (see “Gaze can be compared to the transition period for the visual Orientation and Perceptual Localization”). It has been illusion to influence the ocular orientation process we suggested that other visual illusions, such as the Filehne found here. Indeed, we observe a separation of FF and FP illusion, might also be mediated by area MST (Erickson data immediately after the 400-ms delay (Figure 7B). We & Thier, 1991). It would be interesting to test the conclude that the early localization of the flash is not hypothesis that MT/MST neurons might code the flash- affected by the visual illusion, but once the flash position lag effect as it has been shown for perisaccadic flashes has been transformed in allocentric coordinates, eye (Krekelberg et al., 2003). movements reflect the actual perceived flash location. Thus, the bias in space perception might be due to an Conclusions erroneous reference frame transformation. The relatively long delay for egocentric to allocentric The sense of motion was a common factor in all reference frame transformation may also account partly previous experiments on motion-related visual illusions. for the observation that target motion can influence the Here we showed that motion perception is not a flash-lag effect even some time after the flash occurred. necessary condition for such a bias in space perception. Durations between 60-ms and 600-ms have been Indeed, a flash-lag illusion was observed during previously proposed for this process (Eagleman & unperceived smooth anticipatory eye movements. Sejnowski, 2000; Krekelberg & Lappe, 2000). Furthermore, we showed that gaze orientation to briefly

Blohm, Missal, & Lefèvre 769 presented targets follows the perceptual localization of the Clarke, A. H., Ditterich, J., Druen, K., Schonfeld, U., & flash. In addition, the gaze orientation analysis reveals the Steineke, C. (2002). Using high frame rate CMOS time course of the flash-lag effect. We suggest that this sensors for three-dimensional eye tracking. Behavior reflects the time needed by the CNS to perform the Research Methods, Instruments & Computers, 34, 549- egocentric to allocentric reference frame transformation. 560. [PubMed] Collewijn, H., van der Mark, F., & Jansen, T. C. (1975). Acknowledgments Precise recording of human eye movements. Vision Research, 15, 447-450. [PubMed] This work was supported by the Fonds National de la Recherche Scientifique; the Fondation pour la Recherche Eagleman, D. M., & Sejnowski, T. J. (2000). Motion Scientifique Médicale; the Belgian program on inter- integration and postdiction in visual awareness. university poles of attraction initiated by the Belgian state, Science, 287, 2036-2038. [PubMed] Prime Minister’s office for Science, Technology and Eggert, T., Ditterich, J., & Straube, A. (2001). Culture (SSTC); and internal research grant (Fonds Mislocalization of peripheral targets during fixation. Spéciaux de Recherche) of the Université catholique de Vision Research, 41, 343-352. [PubMed] Louvain. Commercial relationships: none. Eggert, T., Sailer, U., Ditterich, J., & Straube, A. (2002). Differential effect of a distractor on primary saccades References and perceptual localization. Vision Research, 42(28), 2969-2984. [PubMed] Andersen, R. A., Essick, G. K., & Siegel, R. M. (1985). Encoding of spatial location by posterior parietal Eifuku, S., & Wurtz, R. H. (1998). Response to motion neurons. Science, 230, 456-458. [PubMed] in extrastriate area MSTl: center-surround interactions. Journal of Neurophysiology, 80, 282-296. Berkinblit, M. B., Fookson, O. I., Smetanin, B., [PubMed] Adamovich, S. V., & Poizner, H. (1995). The interaction of visual and proprioceptive inputs in Erickson, R. G., & Thier, P. (1991). A neuronal correlate pointing to actual and remembered targets. of spatial stability during periods of self- induced Experimental Brain Research, 107(2), 326-330. visual motion. Experimental Brain Research, 86, 608- [PubMed] 616. [PubMed] Blohm, G., Missal, M., & Lefèvre, P. (2003). Interaction Festinger, L., & Canon, L. K. (1965). Information about between smooth anticipation and saccades during spatial location based on knowledge about efference. ocular orientation in darkness. Journal of Psychological Review, 72, 373-384. [PubMed] Neurophysiology, 89, 1423-1433. [PubMed] Gellman, R. S., & Fletcher, W. A. (1992). Eye position Born, R. T., & Tootell, R. B. (1992). Segregation of signals in human saccadic processing. Experimental global and local motion processing in primate Brain Research, 89(2), 425-434. [PubMed] middle temporal visual area. Nature, 357, 497-499. Heide, W., Blankenburg, M., Zimmermann, E., & [PubMed] Kompf, D. (1995). Cortical control of double-step Bradley, D. C., Maxwell, M., Andersen, R. A., Banks, M. saccades: Implications for spatial orientation. Annals S., & Shenoy, K. V. (1996). Mechanisms of heading of Neurology, 38, 739-748. [PubMed] perception in primate . Science, 273, Kerzel, D. (2000). Eye movements and visible persistence 1544-1547. [PubMed] explain the mislocalization of the final position of a Brenner, E., & Smeets, J. B. (2000). Motion extrapolation moving target. Vision Research, 40, 3703-3715. is not responsible for the flash-lag effect. Vision [PubMed] Research, 40, 1645-1648. [PubMed] Komatsu, H., & Wurtz, R. H. (1988a). Relation of Brenner, E., Smeets, J. B., & van den Berg, A. V. (2001). cortical areas MT and MST to pursuit eye Smooth eye movements and spatial localisation. movements. I. Localization and visual properties of Vision Research, 41, 2253-2259. [PubMed] neurons. Journal of Neurophysiology, 60, 580-603. [PubMed] Bridgeman, B. (1995). A review of the role of efference copy in sensory and oculomotor control systems. Komatsu, H., & Wurtz, R. H. (1988b). Relation of Annals of Biomedical Engineering, 23, 409-422. cortical areas MT and MST to pursuit eye [PubMed] movements. III. Interaction with full-field visual stimulation. Journal of Neurophysiology, 60, 621-644. Cai, R. H., Jacobson, K., Baloh, R., Schlag-Rey, M., & [PubMed] Schlag, J. (2000). Vestibular signals can distort the perceived spatial relationship of retinal stimuli. Experimental Brain Research, 135, 275-278. [PubMed]

Blohm, Missal, & Lefèvre 770

Kowler, E., & Steinman, R. M. (1979). The effect of Nijhawan, R. (2001). The flash-lag phenomenon: object expectations on slow oculomotor control. I. Periodic motion and eye movements. Perception, 30, 263-282. target steps. Vision Research, 19, 619-32. [PubMed] [PubMed] Krauzlis, R. J., & Stone, L. S. (1999). Tracking with the Nishida, S., & Johnston, A. (1999). Influence of motion mind's eye. Trends in Neurosciences, 22, 544-550. signals on the perceived position of spatial pattern. [PubMed] Nature, 397, 610-612. [PubMed] Krekelberg, B., Kubischik, M., Hoffmann, K. P., & World Medical Organization (1996). Declaration of Bremmer, F. (2003). Neural correlates of visual Helsinki. British Medical Journal, 313, 1448-1449. localization and perisaccadic mislocalization. Neuron, [Article] 37, 537-545. [PubMed] Pack, C., Grossberg, S., & Mingolla, E. (2001). A neural Krekelberg, B., & Lappe, M. (1999). Temporal model of smooth pursuit control and motion recruitment along the trajectory of moving objects perception by cortical area MST. Journal of Cognitive and the perception of position. Vision Research, 39, Neuroscience, 13, 102-120. [PubMed] 2669-2679. [PubMed] Purushothaman, G., Patel, S. S., Bedell, H. E., & Ogmen, Krekelberg, B., & Lappe, M. (2000). A model of the H. (1998). Moving ahead through differential visual perceived relative positions of moving objects based latency. Nature, 396, 424. [PubMed] upon a slow averaging process. Vision Research, 40, Robinson, D. A. (1963). A method of measuring eye 201-215. [PubMed] movement using a scleral search coil in a magnetic Lappe, M., & Krekelberg, B. (1998). The position of field. IEEE, Transactions on Biomedical Engineering, moving objects. Perception, 27, 1437-1449. [PubMed] BME-10, 137-145. [PubMed] MacKay, D. M. (1958). Perceptual stability of a Schlag, J., Cai, R. H., Dorfman, A., Mohempour, A., & stroboscopically lit visual field containing self- Schlag-Rey, M. (2000). Extrapolating movement luminous objects. Nature, 181, 507-508. [PubMed] without retinal motion. Nature, 403, 38-39.[PubMed] Medendorp, W. P., Van Asselt, S., & Gielen, C. C. Schlag, J., & Schlag-Rey, M. (2002). Through the eye, (1999). Pointing to remembered visual targets after slowly: Delays and localization errors in the visual active one-step self- displacements within reaching system. Nature Reviews Neuroscience, 3, 191-200. space. Experimental Brain Research, 125(1), 50-60. [PubMed] [PubMed] Squatrito, S., & Maioli, M. G. (1997). Encoding of Mergner, T., Nasios, G., Maurer, C., & Becker, W. smooth pursuit direction and eye position by (2001). Visual object localisation in space: neurons of area MSTd of macaque monkey. Journal Interaction of retinal, eye position, vestibular and of Neuroscience, 17, 3847-3860. [PubMed] neck proprioceptive information. Experimental Brain van Beers, R. J., Wolpert, D. M., & Haggard, P. (2001). Research, 141, 33-51. [PubMed] Sensorimotor integration compensates for visual Mikami, A., Newsome, W. T., & Wurtz, R. H. (1986). localization errors during smooth pursuit eye Motion selectivity in macaque visual cortex. I. movements. Journal of Neurophysiology, 85, 1914-1922. Mechanisms of direction and speed selectivity in [PubMed] extrastriate area MT. Journal of Neurophysiology, 55, van Wezel, R. J., & Britten, K. H. (2002). Multiple uses of 1308-1327. [PubMed] visual motion: The case for stability in sensory Mon-Williams, M., & Tresilian, J. R. (1998). A cortex. Neuroscience, 111, 739-759. [PubMed] framework for considering the role of afference and Watanabe, K., Nijhawan, R., & Shimojo, S. (2002). Shifts efference in the control and perception of ocular in perceived position of flashed stimuli by illusory position. Biological Cybernetics, 79, 175-189.[PubMed] object motion. Vision Research, 42, 2645-2650. Morrow, M. J., & Lamb, N. L. (1996). Effects of fixation [PubMed] target timing on smooth-pursuit initiation. Whitney, D., Cavanagh, P., & Murakami, I. (2000). Experimental Brain Research, 111, 262-270. [PubMed] Temporal facilitation for moving stimuli is Newsome, W. T., Wurtz, R. H., & Komatsu, H. (1988). independent of changes in direction. Vision Research, Relation of cortical areas MT and MST to pursuit 40, 3829-3839. [PubMed] eye movements. II. Differentiation of retinal from Whitney, D., & Murakami, I. (1998). Latency difference, extraretinal inputs. Journal of Neurophysiology, 60, 604- not spatial extrapolation. Nature Neuroscience, 1, 656- 620. [PubMed] 657. [PubMed] Nijhawan, R. (1994). Motion extrapolation in catching. Nature, 370, 256-257. [PubMed]