Microsaccade Characterization Using the Continuous Wavelet Transform and Principal Component Analysis

View metadata, citation and similar papers at core.ac.uk brought to you by CORE

provided by Journal of Eye Movement Research

Journal of Eye Movement Research 3(5):1, 1-14

Microsaccade characterization using the continuous wavelet transform and principal component analysis

Mario Bettenbuhl¨ Claudia Paladini Department of Mathematics Department of Mathematics and Department of Psychology University of Potsdam University of Potsdam

Konstantin Mergenthaler Reinhold Kliegl Department of Psychology Department of Psychology University of Potsdam University of Potsdam

Ralf Engbert Matthias Holschneider Department of Psychology Department of Mathematics University of Potsdam University of Potsdam

During visual fixation on a target, humans perform miniature (or fixational) eye movements consisting of three components, i.e., tremor, drift, and microsaccades. Microsaccades are high velocity components with small amplitudes within fixational eye movements. However, microsaccade shapes and statistical properties vary between individual observers. Here we show that microsaccades can be formally represented with two significant shapes which we identfied using the mathematical definition of singularities for the detection of the former in real data with the continuous wavelet transform. For characterization and model selection, we carried out a principal component analysis, which identified a step shape with an overshoot as first and a bump which regulates the overshoot as second component. We conclude that microsaccades are singular events with an overshoot component which can be detected by the continuous wavelet transform.

Keywords: Fixational eye movement, Microsaccade characterization, Microsaccade detection, Continuous wavelet transform, Principal component analysis

Introduction perceptual fading is caused by the adaptation of retinal receptor systems to constant input and can occur The act of visual fixation is the basis for perception rapidly (Coppola & Purves, 1996). Thus, while our eyes of a stationary target objects. In the beginning eye- fixate a stimulus for the visual analysis of fine details, movement research, miniature eye movements have ironically, miniature eye movements must be produced been discovered during fixation of the eyes. Further- to counteract perceptual fading. Consequently, the more, under suppressed eye movements, perception is term fixational eye movements (FEM) was introduced to disturbed. In the 1950s it was demonstrated in a labo- capture this seemingly paradoxical behavior. Percep- ratory experiment that stationary objects rapidly fade tual performance as a function of self-generated noise from perception when our eyes are artificially stabi- is an unimodal function which lends support to un- lized Ditchburn and Ginsborg (1952); Pritchard (1961); derlying nonlinear mechanisms (Starzynski & Engbert, Riggs, Ratliff, Cornsweet, and Cornsweet (1953)). This 2009). Fixational eye movement is classified as tremor, drift and microsaccades (e.g., Ciuffreda & Tannen, 1995). We thank Marco Rusconi for valuable discussions, reading The largest component of fixational eye movements the manuscript and comments. Additionally, we also thank is produced by microsaccades which are high-velocity Hannes Matuschek for the supply of the source codes for movements with small amplitudes. Recent find- TM the wavelet transformation in python . This research was ings demonstrated various neural, perceptual and be- supported by Deutsche Forschungsgemeinschaft (Research havioral functions of microsaccades (Martinez-Conde, Group 868 “Computational Modeling of Behavioral, Macknik, & Hubel, 2004; Martinez-Conde, Macknik, Cognitive, and Neural Dynamics“, Grant No. EN 471/3-1). Troncoso, & Hubel, 2009; Rolfs, 2009). The relevance of microsaccades to diverse neural and cognitive sys-

DOI 10.16910/jemr.3.5.1 This article is licensed under a ISSN 1995-8692 Creative Commons Attribution 4.0 International license. Journal of Eye Movement Research Bettenbuhl,¨ M., Paladini, C., et al. (2010) 3(5):1, 1-14 Microsaccade characterization using the continuous wavelet transform and PCA tems offers a possible explanation for the difficulties (Gowen, Abadi, Poliakoff, Hansen, & Miall, 2007). in identifying a specific function for microsaccades The list of results demonstrates that microsaccades (for recent overviews see Martinez-Conde et al., 2004). are associated with a wide range of research areas in Here, the following section gives a very simple and behavior, cognition and neural functioning. For the brief overview of recent findings about functions of mi- detection of microsaccades in trajectories of fixational crosaccades: eye movements, methods were developed by Boyce Neural activity. Microsaccades are correlated with (1967), Martinez-Conde et al. (2000) and Engbert and bursts of spikes across the visual pathway (Martinez- Kliegl (2004) that used the microsaccade property of Conde, Macknik, & Hubel, 2000; Martinez-Conde et al., their high velocity, i.e. they used their amplitude. The 2004; Martinez-Conde et al., 2009). Theoretical analyses idea underlying their methods is to describe microsac- suggest that they help to decorrelate neural responses cades as high velocity events of certain length. It was in natural viewing (Rucci & Casile, 2004). Mergenthaler and Engbert (2007) who reported that fix- Oculomotor control of fixation. Microsaccades enhance ational eye movements can be modelled as fractional retinal image slip (to counteract retinal fatigue) on a Brownian motion with persistent and anti-persistant short time scale and control fixational errors on a long behavior on a short and long time scale, respectively. time scale (Engbert & Kliegl, 2004). Moreover, recent They also investigated the influence of microsaccades evidence suggests that microsaccades are triggered on on the scaling exponent which determines the charac- perceptual demand based on estimation of retinal im- teristics of the underlying structure. Our working hy- age slip (Engbert & Mergenthaler, 2006). pothesis is the definition of microsaccades as events Perception. Microsaccades are important for periph- that compared to the general structure in fixational eye eral and parafoveal vision. During the perception movement trajectories have low regularities. We pro- of bistable visual scenes, microsaccades induce tran- pose a scale-free detection method for microsaccades sitions to visibility and counteract transitions to per- using the continuous wavelet transform (Holschneider, ceptual fading (Engbert, 2006; Martinez-Conde, Mack- 1995; Mallat, 1998). It uses structural properties of the nik, Troncoso, & Dyar, 2006; Rucci, Iovin, Poletti, & trajectory and the unlikeliness of microsaccadic events Santini, 2007). Moreover, fixational eye movements in the underlying drift movements, to detect microsac- and microsaccades represent noise sources that en- cades. hance perception (Starzynski & Engbert, 2009). Fur- Using the results obtained by a detection method thermore during fixation, microsaccades play a part in that uses structure properties of fixational eye move- supporting second-order visibility (Troncoso, Macknik, ments, we continue with an analysis of the shapes ob- & Martinez-Conde, 2008). tained. We show, how to arrive at a data-driven mi- Attention. Microsaccades can be suppressed volun- crosaccade shape characterization by the use of prin- tarily with focused attention (Bridgeman & Palca, 1980; cipal component analysis (Jolliffe, 2002). We will see Gowen, Abadi, & Poliakoff, 2005). They are also mod- that already two components are enough to describe ulated by crossmodal attention with a pronounced sig- the microsaccade shape within an appropriate range. nature in both rate and orientation (e.g., Engbert & Kliegl, 2003; Galfano, Betta, & Turatto, 2004; Hafed Methods & Clark, 2002; R. Laubrock, Engbert, & Kliegl, 2005; Rolfs, Engbert, & Kliegl, 2005). The hypothesis that mi- Microsaccades are rapid small-amplitude events with typical durations between 6 to 30 ms and ampli- crosaccades represent an index of covert attention has ◦ been criticized by Horowitz, Fine, Fencsik, Yurgenson, tudes below 1 (for an overview see (Engbert, 2006)). In and Wolfe (2007) (but see Horowitz, Fencsik, Fine, Yur- this paper however we propose to not use their higher genson, & Wolfe, 2007; J. Laubrock, Engbert, Rolfs, & velocity as in (Engbert & Kliegl, 2003) or (Engbert & Kliegl, 2007), however new work by J. Laubrock, Kliegl, Mergenthaler, 2006), but rather use their local singu- Rolfs, and Engbert (2010) lends support to the coupling larity structure. Wavelet analysis is a well suited toll between attention and microsaccades. to detect and characterize singularities within a more Saccadic latency. Microsaccades interact with upcom- regular background. ing saccadic responses which can result in prolonged as Singularities and local regularity well as shortened latencies for saccadic reactions (Rolfs, Laubrock, & Kliegl, 2006). Recently, Sinn and Engbert One first might ask how the mathematically mo- (2009) demonstrated that this effect contributes to the tivated term singularity applies to the description of saccadic facilitation effect in nature background. a microsaccade. Following Zuber, Stark, and Cook Individual differences. The pattern of successive mi- (1965), we sketch a prototypical microsaccade shape in crosaccades (called saccadic intrusion in this study) has Figure1. The y-axis is the horizontal eye movement been proposed as a stable characteristic between per- plotted over time (x-axis). Thus, in this example, after sons (Abadi & Gowen, 2004). In general, there is much an initial period of rest, the eye moves quickly towards overlap but also a few differences tied to the distinc- the right and returns partially towards the left before tion between microsaccades and saccadic intrusions arriving at the new horizontal position. The illustration

Figure 1. Microsaccades can mathematically be deﬁned as superposition of two functions with singular points. (a) We 0.5 have labeled several attributes which allow a further description of the eye’s displacement. The eye’s trajectory is drawn

1.0

on the Y-axis against time on the X-axis. Smoothing the (a) 4 3 2 1 0 1 2 3 4 Heavyside and (b) Dirac delta function and superpose them, t [a.u.] will return a microsaccade shape.

This schematic representation of a microsaccade Figure 2. The Morlet wavelet at its smallest internal fre- shape can mathematically be described by a superposi- quency ω0 = 5.336. The oscillations o the real R (t) and imag- tion of smoothed versions of the singularities depict in inary part I (t) of the complex-valued wavelet are enclosed in Ψ(t) Figure1b and1c. These two schematically show shapes the positive and negative modulus of . of functions that have scale invariant singularities: the Heavyside and Dirac delta function. Both functions are the one-dimensional signal is transformed into the two- singular at zero. In fixational eye movement trajecto- dimensional time-frequency plane that tells us when ries, the drift and tremor movement describe the base- (parameter b) which frequency (parameter 1/a) occurs. line of position-time displacements. Microsaccades do Other time-frequency transformations exist as for in- not share the same property of self-similarity at this stance the windowd Fourier transform or the Gabor baseline (compare Mergenthaler & Engbert, 2007), in- transform (Feichtinger & Strohmer, 1998). However fluence the latter and appear as singular events in the only wavelet analysis is capable of characterizing local more regular drift movement. singularity because it has no a priori limit to its time- resolution. For our analysis, we used the progressive Morlet The continuous wavelet transform wavelet, defined by A powerful tool to analyze local regularity and iω t −t2/2 to detect local singularites is the continuous wavelet Ψ(t) = e 0 e (2) transform (Arneodo, Grasseau, & Holschneider, 1988; ω0 Holschneider & Tchamitchian, 1991; Mallat & Hwang, with a = ω and ω0 as internal frequency. It is an 1992). This time-frequency analysis tool has been ap- oscillating wavelet, i.e., the parameter a changes the plied for various reasons in signal processing and in average frequency of the scaled wavelet. We see the 2 general data analysis (see e.g., Daubechies I., 1992; Gaussian envelope given by e−t /2 illustrated in Fig- Daubechies & Teschke, 2005; Diallo, Holschneider, ure2 with the oscillating part of the wavelet. We Kulesh, Scherbaum, & Adler, 2006; Holschneider, 1995; checked that our findings do not depend on this partic- Quiroga, Nadasdy, & Ben-Shaul, 2004). Therefore, we ular choice of wavelet by using the Cauchy wavelet in will apply the wavelet method for the detection of sin- comparison. The results are not sensitive to the choice gularities in fixational eye movement data. of wavelet as long as they respect generic properties The continuous wavelet transform of a real-valued such as localization in time and frequency.

Singularity detection for horizontal orientation (Engbert & Kliegl, 2003; Eng- bert & Kliegl, 2003). For the analysis of microsaccades, we exploit the idea that microsaccades are more irregular than some Microsaccade characterization background movement. In real data however perfect singularities cannot be observed. Perfect singularities In the previous section we have shown how the anal- are physiologically impossible because they would re- ysis of the continuous wavelet transform helps us to quire infinite accelerations. We rather expect to have detect singularities in fixational eye movement trajecto- smoothed singularity. This is a shape which at large ries. We use this information to extract an area around scale looks singular, whereas at small scale the physi- these singularities to investigate the characteristics of ologal limitations enforce a more regular behavior. the eye’s position as e.g., the shape of a microsaccade. A pure singularity will give rise to a conelike struc- For each of the binocular singularities detected with ture of strong wavelet coefficients with the top at the the method described in the previous section, we ex- time-point, where the singularity occures. A smoothed tract K data samples corresponding to an epoch of the singularity will behave the same at all scales, which are time series around the location of a singularity. We call large compared to the smoothing scale. In Figure3, this segment a signal snippet. To investigate if typical such small cones that cross the whole frequency range shapes for microsaccades exist in fixational eye move- can be identified by eye. ment data, we made use of the principal component anal- Numerically singularity detection with wavelets is ysis (PCA, see Jolliffe, 2002; Smith, 2002, for a short usually done using the so called method of maximum tutorial) to systematically describe the large variabil- modulus lines (see Marr & Hildreth, 1980; Witkin, 1983). ity of all possible microsaccadic shapes. The principal A maximum modulus line is a line in the time-scale component analysis is a way to represent a given data plane on which the modulus of the wavelet transform set in a reference frame whose dimensions, the princi- has a local maximum with respect to small variation in pal components, are such that: the first accounts for as b0 such that much of the variance in the data as possible, the second for as much as possible of the remaining variance and |W s(a,b0)| > |W s(a,b0 ± ε)| (3) so on. All pci together represent an empirical orthonormal system. In addition to the principal components Connecting such points will give the maximum modu- pci, we obtain a measure of the importance of each di- lus lines. It can be shown that if a signal has a singu- mension in relation to the others, given by the singular larity at point then there is a maximum modulus line values s. Now, one rewrites the shape of a microsac- which at small scale converges towards the location cade (ms) with a linear combination of these principal of the singularity (e.g., (Mallat & Hwang, 1992)). In shapes: smoothed singularities the maximumum modulus line may end at a scale which is about the smoothing scale ms = c0 pc0 + c1 pc1 + c2 pc2 + ... + cK−1 pcK−1 (4) of the singularity. For this reason we consider those maximum modulus lines, which go from a fixed high- where pci is the ith principal component, ci is the coef- est frequency / smallest scale to a smallest frequency ficient that explains the contribution of this ith shape, / largest scale. The estimated position of the singular- and K is the length of a signal snippet. Due to the in- ity is simply the small-scale end of the corresponding terpretation of the obtained principal components for maximum modulus line. fixational eye movements, we will use principal shapes Figure3b shows a typical example of a wavelet and principal components as synonyms. transform for the horizontal component of fixational Before the analysis of our data for their pci, we need eye movements. The maximum modulus in the (a,b)- to preprocess the input data. First, we remove the con- plane, here highlighted in red. stant signal offset by subtracting the mean value from Taking previous works into consideration (Ditch- each signal snippet, i.e. burn & Ginsborg, 1952; Krauskopf, Cornsweet, & Riggs, 1960), microsaccades are generally binocular, K−1 1 (l) conjugated eye movements and we consider only ms(l) = ms(l) − ms (5) K ∑ i binocular singularities. This means, positions of a sin- i=0 gularity in one eye are not allowed to differ more than τ from the positions of a singularity in the other eye, with l indicating one individual signal snippet. Then i.e., microsaccades in one eye have their simultaneous we compose a matrix M of dimension NxK with N appearing microsaccades in the other eye in a time win- given by the total number of detected singularities and K the length of each snippet. We subtract the ensemble dow (t0 −τ,t0 +τ). We will refer to this criterion as binocularity criterion. mean from M, i.e., In this study, we restrict the analysis to the horizon- K−1 N−1 tal component of fixational eye movements. Previous ˜ 1 M = M − ∑ ∑ mi j (6) work suggested that microsaccades show a preference K · N i=0 j=0

]

0.4 0.0 -0.4 Position [ Position 50.0

40.0

30.0

Frequency [Hz] 25.0

20.0 4 8 12 16 20 Time [s]

Figure 3. One horizontal fixational eye movement trajectory in position-time and time-frequency representation. The marker indicate the positions of singularities (red dots and arrows) and binocular (green dots and arrows) singularities. The modulus of the wavelet transform allows the identification of maximum modulus lines. The time point at which a singularity occurs is taken at the highest frequency. The positions of maximum modulus lines match candidates in the time-position trajectory which e.g., by visual inspection would be identified as microsaccades.

with mi j being the elements of the matrix M. Using sin- nents. We will see how these representations vary be- gular value decomposition (SVD, see Venegas, 2001) we tween subjects. write M˜ as a product of three matrices In summary, we identify binocular singularities that give us candidates for binocular microsaccades in our M˜ = USV T (7) fixational eye movemement study. The representation of a signal snippet with principal components gives where U is an orthogonal KxK matrix which contains possibilities for further discussion about the impor- as columns the orthonormal vectors that represent the tance of each component contributing to the variability orthonormal system for all row vectors of M˜ . The ma- of microsaccadic eye movement. This will be discussed trix V is an orthonormal NxN matrix whose columns in the result section below. represent the orthonormal system for all column vectors of M˜ and S is a rectangular, diagonal KxN matrix Experiment containing the K singular values in its diagonal. After SVD, the columns of U contain the principal compo- Experimental data presented here was published in nents pci, i = 0,...,K −1 which best describe the collec- earlier work (Mergenthaler & Engbert, 2007; Engbert & tion of the singularities along the K dimensions. The Mergenthaler, 2006). diagonal entries of S, namely s0,0,...,sk−1,k−1 give us a measurement for the importance of each pci. Therefore Participants we evaluate −1 K−1 ! Twenty-four participants with an average age of 22 2 2 2 (range: 19 to 51 years) participated in the experiment. s˜hh = shh · ∑ sll (8) l=0 All participants had normal or corrected to normal vision. The experiment was performed in accordance with hh = (0,0),...,(k − 1,k − 1). Having this measure with the declaration of Helsinki. we are able to reduce the dimensions of the obtained orthonormal system to a lower complexity but still con- Task taining sufficient information to describe the variability of inlying functions to a high level. Participants had to fixate a black square on a white In the process of this work we reconstruct the shape background (3-by-3 pixels on a computer display of binocular singularities with the principal compo- (Iiyama, Vision Master Pro 514, 40 by 30 cm, 100Hz,

1024 by 768 pixels) which corresponds to a spatial ex- is allowed to differ of most by 30 ms. In Figure3a we tent of 7.2 arc min). They were required to perform 30 have marked the positions of singularities detected in trials of 20 s each. To reduce loss of data, the partici- the wavelet plane. In Table1 we summarize the num- pants were asked to avoid blinking during each trial. ber of detected monocular singularities as well as the In addition, we remonitored for eye blinks and trials binocular singularities per participant in left and right with occuring blinks were repeated. Every ﬁxation trial eye, respectively. is followed by a presentation of a photograph for 10 s, In the total number of 682 trials, we detected 35531 allowing participants to relax and perform inspection and 35066 singularities in left and right eyes, respec- saccades or blinks. A typical trajectory from a trial is tively. The difference in the number of detections be- shown in Figure4. tween eyes is lower than 1.3%, yielding good agreement for the microsaccadic processes in both eyes. Af- ter application of the binocularity criterion as described (a) (b) ]

¡0.6 in section , we retain a total number of 16947 binocu- [ lar singularities. The mean rate of binocular singulari- 0.2 ties is 1.2 per second with a standard deviation of 0.5. In this study, seven participants contributed less than -0.2 one binocular singularity per second in their ﬁxational eye movement. As the number of detected singulari-

Vertical position position Vertical -0.6 ties in their eye movement trials is in agreement with

-0.6 -0.2 0.2 0.6 -0.6 -0.2 0.2 0.6

Horizontal position [ ] Horizontal position [ ] those of other participants, their results are suggestive of monocular events.

Figure 4. Representation of the trajectories in a fixational eye = c0 · + c1 · + c2 · + c3 · +... movement trial for both eyes. Recorded with EyeLink II system we obtained the horizontal (x-axis) and vertical (y-axis) position of (a) left and (b) right eye. The trajectories are corrected to center. Figure 5. Representation of a time series snippet around Eye movement recording the location of a singularity with its principal shapes pro- cessed by the principal component analysis. The left side of Eye movements of our participants were tracked this equationlike representation is the horizontal position in with a head mounted eye tracker (EyeLink II, SR a 60 ms time window around an identified singularity. Here, Research, Osgoode, Ontario, Canada). They were it is defined by the linear combination of principal shapes. recorded binocularly with a sampling rate of 500 Hz We identified that already two shapes are sufficient to represent roughly 95% of the variances contained in each individ- and a spatial resolution for a dark pupil in root mean ◦ ual shape. The coefficients c0,c1,... are individual for each square of higher than 0.01 visual angle. Participants binocular singularity and explain the contribution of the cor- were seated on a chair with their head placed on a chin responding shape to the individual shape. rest. The viewing distance to the computer screen was 50 cm. Next, we investigate the signal snippets around the Results detected binocular singularities for common features. Equation (4) enables us to rewrite each shape as a lin- The method of singularity detection identified seg- ear combination of reliably measured basic shapes. The ments of fixational eye movements that represent can- variance contribution of each is measured in the indi- didates for microsaccades. Figure3 displays the vidual coefficients for the shape. We have taken snip- wavelet transform of a time series of one single trial pets of the fixational eye movement trajectory with the of fixational eye movement. All local maximum mod- length K = 31 around a binocular singularity position. ulus lines which passed without interruption between We are interested in the 30 ms before and after the time 20 Hz to 50 Hz and met the condition in Equation (3) point we detected. We preprocess the data as described with at maximum ε = 5 ms at 20 Hz are included in the in section Microsaccade characterization and obtain a rep- analyses. We have chosen this frequency range because resentation as shown in Figure5 which is a visual rep- the left and right eye are well correlated over this range. resentation of the components in Equation (4). Additionally we want to work above a threshold of 20 The PCA of the detected snippets reveals that the Hz as the wavelet transform maximum modulus lines first two principal components pc0 and pc1 account for will get influenced by modulus maxima of the other roughly 95% of the variability of microsaccadic shapes structures present in our data at lower frequency. A (compare Equation (8) which measures the importance binocular singularity is defined by the time point a sin- of each single pci or combination of the same). At gularity is detected in left and right eye. The time point this, each component separately accounts (in average

Detected Binocular Participant Number singularities singularities 1 1 of trials rate of [ s ] rate of [ s ] left right 1 30 2.6 2.7 1.5 2 29 3.3 3.3 2.4 3 30 2.7 2.8 1.4 4 30 2.2 2.1 0.3 5 22 2.3 2.3 0.6 6 30 2.8 2.9 1.8 7 30 2.8 2.8 1.7 8 30 2.9 2.8 1.7 9 30 2.1 2.0 0.5 10 17 2.6 2.5 1.0 11 28 2.5 2.4 1.0 12 30 2.4 2.4 0.8 13 29 2.3 2.3 0.6 14 30 2.6 2.5 1.2 15 29 3.1 2.9 1.8 16 30 2.7 2.5 1.2 17 29 2.4 2.4 0.8 18 23 2.6 2.5 1.4 20 29 2.8 2.7 1.7 21 29 2.9 2.9 1.9 22 30 2.1 2.1 0.3 23 29 2.6 2.5 1.2 24 30 2.8 2.7 1.7 25 29 2.5 2.5 1.2 Total 682 2.6±0.3 2.6±0.3 1.2±0.5

Table 1 Rates of detected singularities in the horizonal eye movement in our ﬁxation task experiment. The number of binocular singularities is a subset of all detected. The total rates are given as mean ± standard deviation.

0.4 of both eyes) for 82.7 and 81.2% as well as 12.2 and (a) (b) 13.4% for pc0 and pc1 in left and right eye, respectively. ] 0.2 We restrict therefore our analysis to these first two prin- [ cipal shapes. We decompose each single microsaccade 0.0 (l) ms into the following five terms: Left Left Position Position • a linear combination of pc0 and pc1 -0.2 Right Right • a vector ρ, representing the mean of the whole en- 0.0 0.02 0.04 0.06 0.0 0.02 0.04 0.06 semble for each eye (see Equation (6)) Time [s] Time [s] • a scalar κ(l), representing the mean of each snippet (see Equation (5)) • a small residual vector r capturing numerical er- Figure 6. Graph of the shapes of the first two principal com- rors ponents which go into our model for a microsaccade. (a) We Except for the residual vector, all these components are have for pc0 a steplike shape which has the tendency to re- directly computed from the data. Now, we write our turn after it reached the maximum amplitude. This overshoot, typical for microsaccades, dominates together with the model for a typical microsaccade by the linear combi- almost linear increasing shoot part this shape. (b) The shape nation of the second component pc1 is bumplike. It identifies how (l) (l) (l) (l) much overshoot each microsaccade has. The left (blue solid ms = c0 pc0 + c1 pc1 + κ + ρeye + r (9) line) and right (red dashed line) eye agree in the shape of the first two principal components. while l denotes the index of each individual microsaccade. The shapes of pc0 and pc1 are shown in Figure6a and6b, respectively. The first principal shape pc0 represents a movement

With respect to these coefficient pairs we define mi- Singularity detection and crosaccades as those binocular shapes whose variabil- characterization in ity is described by more than 80% of the first two prin- amplitude-adjusted surrogate cipal components. Expressed in an equation we write data 30 (l) 2 (l) 2 (l) 2 In this section, we validate the applicability of our (c0 ) + (c1 ) ≥ 0.8 ∑hms , pc ji (11) j=0 detection and characterization methods. First, we

0.6

0.5

0.4

0.3

0.2 probability density

0.1

0.0 ¡

¡ ¡ ¡ /2 0 /2 3/2

-pc1 pc0 pc1 -pc0 -pc1

position [ ¢]

Figure 8.(Upper graph) Gaussian kernel density estimation for binocular microsaccades which fit to our model and (lower graph) representation of the most probable shape combination of pc0 and pc1 for each individual participant. Each microsaccade coef- (l) ficient pair (c0,c1) is transformed to the one dimensional α with l being the index of a microsaccade. The density estimation reveals the dominance of the first principal shape pc0 (localization around 0 and π) above pc1. The lower graph shows the distribution of the most probable shapes in left and right direction and supports the former statement. This result holds for all participants. The contribution of the second component pc1 to the microsaccade shape differs between them which we see in the different widths of the peaks. check the reliability of singularities detected with the via continuous wavelet transform by comparison of orig- s(t + 2∆t) + s(t + ∆t) − (s(t − ∆t) + s(t − 2∆t)) inal data and time series which mimic properties of v(t) = the original fixational eye movement data. Second we 6∆t want to support that the identified shapes of the first (13) principal components are typical for microsaccadic eye where s(t) is the signal at position t and ∆t = 0.002s. The movements. generation of amplitude-adjusted surrogates is split into the following steps: 1. Sort v and obtain a rank series r of v

2. Generate a series g of Gaussian distributed We generate a time series which mimic properties of random numbers with the same length as v, sort it fixational eye movements while destroying microsac- and rearrange it according to the rank series r. In this cades, by applying an appropriate surrogate data gen- way, we generate a time series h that is a rescaled time eration method. In fixational eye movements studies series of v with the property that the amplitudes of the we observe a persistent behavior on a short time scale samples belong to a normal distribution (Engbert & Kliegl, 2004; Engbert & Mergenthaler, 2006) which is reflected in a positive autocorrelation func- 3. Transform h to the Fourier space and obtain h˜ tion of the velocities for small lags. We need to reject ˆ i the null hypothesis that positively autocorrelated sam- 4. Randomize the phase: h˜ = he˜ ϕ. ϕ is a series ples in the drift are the reason for the observation of containing equally distributed random numbers high-velocity epochs in fixational eye movements de- between −π and π with identical values for positive tected as singularities. A surrogate data type allow- and negative frequency ing to test the null hypothesis is amplitude-adjusted phase-randomized surrogate data (Theiler, Galdrikian, 5. Calculate the inverse of the Fourier transform: Longtin, Eubank, & Farmer, 1992) which maintains (F−1h˜ˆ)(ω) = h˜(t) the velocity distribution and approximate the auto- ˜ correlation function. The velocity of our fixational eye 6. Obtain a rank series of hˆ and rearrange v in movements is obtained as in Engbert and Kliegl (2003) accordance with the new rank series

0.4 (a) left original To return to a position-time series one sums the ve- right original locities divided by the sampling frequency cumula- left surrogate

] 0.2 tively. In the following we refer to the latter as the sur- right surrogate rogates or surrogate data of ﬁxational eye movement [ time series. We take all 682 trials of surrogates and perform the same analysis as performed for the orig- 0.0 inal data. The results for the detected singularities are position shown in Table2. -0.2 We explain the higher number of singularities detected in the surrogate data by the conservation of each original velocity sample during surrogate data gener- 0.4 (b) left original ation. Nevertheless, epochs of high velocities in ﬁxa- right original tional eye movement data, corresponding to microsac- left surrogate

] 0.2 cades, can be split into two or more singularities in right surrogate the surrogates. But this is only true for monocular [ singularities. However, we also observe a high number of binocular singularities, on average 10 per trial. 0.0

The high number of binocular events results from the position probability of randomly co-occuring extended events of length 2τ. Co-occuring means: A singularity in one -0.2 eye happens within the window of τ milliseconds before or after a singularity in the other eye. Estimating 0.0 0.02 0.04 0.06 the probability of this co-occurrence, we take a monoc- time [s] ular time series of length T and a number of monocular singularities N. As each singularity is in the center of a 2τ window and Figure 9. Representation of the first principal components’ at least on sample ∆t apart, a significant number of data shapes for the original data and surrogate data. Compare (a) pc pc samples belong to those which are possible candidates 0 and 0s, the typical overshoot behavior in the smooth step is not present. Left (blue solid line) and right (red dashed for a co-occurrence. Thus, the probability to observe a line) eye again show approximately the same principal com- binocular event E by chance is given by ponent in original as well as in surrogate data (green dotted and magenta dash-dotted line). (b) The second component (2τ + ∆t) · N for the original pc is a bump whether pc is a smooth peak p(E) = (14) 1 1s T shape. which is the ratio of time, belonging to points of possible co-occurrence candidates in the full trajectory. For case, the first two components explain even more than the presented data the values are: τ = 30 ms, ∆t = 2 ms, 95% of the variance in our data and we choose pc T = 20000 ms, and N = 57, which is the average num- 0s and pc to reduce our 31 dimensional system of shapes ber of detected monocular singularities in our surro- 1s to this two dimensions. Both components look simi- gate data. Inserting these values one gets p(E) · N ≈ 10 lar but differ strongly in their shapes, compared to the binocular events simply by chance in our surrogate principal components obtained for the original data. data. This result is in very good agreement with the This becomes obvious by consideration of combina- numbers of observed binocular events in the surrogate tions of the two components. For surrogates the prin- data: On average 10 binocular events. Now, we argue cipal component pc is a smoothed step and does not that the obtained number of binocular events in surro- 0s show any overshoot. The second component pc is a gates mainly depends on the probabilitiy p(E) of ran- 1s very smooth peak-like bump. Both pc and pc are dom co-occurence, which is set by our time frame to 0s 1s smoothed version of singularities, i.e. a step and a sin- define an event as binocular. gle peak as illustrated in Figure1. These randomly occurring binocular singularities, which also can happen in the original data, cannot be Importantly, the pc0 and pc1 of the original data explained by the first principal components pc0 and pc1 show a directionality in time: They cannot be reversed. by more than 80%. Thus, applying Equation (11) one The principal components pc0s and pc1s for the surro- filters out random binocular singularities in the anal- gate data can be reversed. Both fulfill the condition of ysis of original data. Further processing of the binoc- symmetry, i.e. f (−t) = − f (t) and do not show any di- ular singularities with the principal component analy- rectionality. On the basis of the described differences sis (see section Microsaccade characterization) results in in pc0 and pc1, we reject the described null hypoth- the principal components shown in Figure9. In this esis and conclude that the observed binocular singu-

Detected Binocular Participant Number singularities singularities 1 1 of trials rate of [ s ] rate of [ s ] LR 1 30 2.5 2.6 0.4 2 29 3.1 3.1 0.6 3 30 2.8 2.7 0.4 4 30 2.4 2.2 0.4 5 22 2.7 2.5 0.5 6 30 2.9 2.9 0.5 7 30 3.1 3.0 0.6 8 30 3.2 3.2 0.7 9 30 2.5 2.5 0.4 10 17 2.7 2.6 0.4 11 28 3.1 3.0 0.6 12 30 3.0 3.2 0.6 13 29 2.3 2.3 0.3 14 30 2.8 2.4 0.5 15 29 3.1 3.0 0.6 16 30 2.7 2.5 0.4 17 29 2.4 2.3 0.4 18 23 3.3 3.3 0.7 20 29 3.2 3.2 0.7 21 29 3.4 3.3 0.7 22 30 2.5 2.4 0.4 23 29 3.3 3.3 0.7 24 30 2.8 2.7 0.4 25 29 3.4 3.3 0.7 Total 682 2.9±0.3 2.8±0.4 0.5±0.1

Table 2 Rates for the detected monocular and binocular singularities in the surrogates of horizontal fixational eye movement data sets. The total rates are given as mean ± standard deviation. larities in fixational eye movement time series result cades, our alternative approach can identify large scale from high-velocity epochs in fixations with a distinct saccades and microsaccades within one detection pro- shape given by a linear combination as explained in cedure, e.g., in eye movements recorded during scene our model equation (9) with the components pc0 and viewing or reading. Methods that use velocities require pc1 of the original data, shown in Figure6. predefined thresholds such that an analysis either uses two detection runs to first separate saccades and then Discussion microsaccades (i.e., one uses a threshold related to the variance in the trajectory for saccade detection and sub- We investigated the hypothesis that microsaccades sequently another threshold related to the remaining can be modelled as events of lower regularity and see variance), or instead uses a threshold defined by the that the continuous wavelet transform successfully dis- variance obtained in co-recorded fixation-task experi- tinguishes microsaccades of fixational eye movement ments. Using the shared property of lower regularity from background activity (i.e., drift). Our methods (between microsaccades and saccades) allows the iden- are based on a pre-defined minimal set of parame- tification of both within the same detection process. ters (related to maximum modulus lines, binocular- A very brief analysis between performances of the ity, and the fit to the model). A validation that uses velocity threshold and continuous wavelet transform amplitude-adjusted surrogate data verifies that almost method on eye movement trajectories in a fixation task simultaneously appearing structures of low regularity can be found in the Appendix. Seemingly, the de- in fixational eye-movement trajectories cannot be ex- tected microsaccade positions in fixational eye move- plained by randomly co-occuring autocorrelated sam- ments look quite similar. ples in both eyes’ drift movements. We continued our analysis with a comparison of In comparison to current methods which use ampli- detected microsaccades between participants and per- tudes (or its derivatives) for the detection of microsac- formed a principal components analysis, yielding two

] velocity method tics of saccadic intrusions. Two microsaccadic shapes [ 0.6 wavelet method with a Single Saccadic Pulse (SSP), and Double Saccadic 0.3 Pulse (DSP) were introduced by the authors. Addi- tionally, two sequences of microsaccades were investi- 0.0 position position gated which took two and three subsequent microsac- -0.3 cadic movements into account, the Monophasic Square 0 1 2 3 4 5 Wave Intrusion (MSWI) and Biphasic Square Wave Intru- time [s] sion (BSWI). Our microsaccade model is able to separate SSP and DSP simply by investigation of the more dominant principal shape, i.e., if the first prinicipal Figure A1. Detections of microsaccades in an example hor- shape pc0 is dominant, we count a SSP whereas if the izontal fixational eye-movement trajectory. The two meth- second principal shape pc1 is leading, we see a DSP. ods perform quite similar. The wavelet detection method also In perspective, an analysis of sequences of microsac- identifies small scale events, compare e.g., at around 1.3s. cadic shapes allows studies on short- and long-term de- pendencies as well as investigations with co-registered The detected rates for binocular events with the con- EEG data which profits from a better understanding tinuous wavelet transform and the velocity threshold of microsaccades and the induced potentials (Dimigen, algorithm are significantly correlated (r = 0.96; P < Sommer, Hohlfeld, Jacobs, & Kliegl, under revision). 0.0001). A comparison of the detected time points for Future work will consider a stronger comparison binocular events shows an overlap of 74% (number of of the two methods and the possibility to extend the binocular microsaccades detected with both methods detection method to horizontal and vertical (2D) eye divided with the number of detections by the contin- movements and analyze their interplay in the context uous wavelet transform method). We called an event of microsaccadic movements. A subsequent classifica- occuring at the same time point in both methods if it tion of microsaccade shapes seems promising to inves- did not differ more than τ = 16 ms. tigate the spatiotemporal dynamics of microsaccades In Figure A1 we show five seconds of one fixational and may lead to a new model for the dynamics of fixa- eye movement trial with marked time points which tional eye movements. An understanding of the latter were detected either by the continuous wavelet trans- under the well-defined conditions in fixation-task ex- form or velocity threshold method. We see that the de- periments allows the modeling of eye movements at its tected time points are in nice agreement. baseline. Establishing a model at this point will pro- A detailed investigation of distinct properties describ- vide a foundation to study trajectories and reactions to ing microsaccades detected in one but not the other attractions of the eye in more complex experimental se- method is in preparation. tups.

Participant Number Binocular events 1 of trials rate of [ s ] wavelet velocity both methods 1 30 1.5 1.1 0.9 2 29 2.4 2.4 2.0 3 30 1.4 1.4 1.1 4 30 0.3 0.1 0.1 5 22 0.6 0.5 0.3 6 30 1.8 1.9 1.5 7 30 1.7 2.0 1.4 8 30 1.7 2.0 1.4 9 30 0.5 0.2 0.2 10 17 1.0 0.7 0.4 11 28 1.0 0.9 0.8 12 30 0.8 0.9 0.5 13 29 0.6 0.2 0.1 14 30 1.2 0.8 0.6 15 29 1.8 1.9 1.4 16 30 1.2 1.4 0.9 17 29 0.8 0.3 0.2 18 23 1.4 1.5 1.1 20 29 1.7 1.9 1.5 21 29 1.9 1.9 1.7 22 30 0.3 0.2 0.2 23 29 1.2 1.2 0.9 24 30 1.7 1.7 1.4 25 29 1.2 1.1 0.9 Total 682 1.2±0.5 1.2±0.7 0.9±0.6

Table A1 Detection of microsaccadic events in a ﬁxational eye movement experiment with the continuous wavelet transform (WT) and velocity threshold (VT) method. Results are compared after application of the individual settings for each method. The VT algorithm detects just 5% less binocular events. The total rates are given as mean ± standard deviation.

References Adler, F. (2006). Characterization of polarization attributes of seisimic waves using continuous wavelet transforms. Abadi, R., Gowen, E. (2004). Characteristics of saccadic in- Geophysics, 71, 67–77. trusions. Vision Research, 44, 2675-2690. Dimigen, O., Sommer, W., Hohlfeld, A., Jacobs, A., Kliegl, R. Arneodo, A., Grasseau, G., Holschneider, M. (1988). On the (under revision). Co-registration of eye movements and wavelet transform of multifractals. Physical Review Letters, EEG in natural reading: Analyses and review. 61, 2281-2284. Ditchburn, R., Ginsborg, B. (1952). Vision with a stabilized Boyce, P. (1967). Monocular fixation in human eye move- retinal image. Nature, 170, 36–37. ment. Proceedings of the Royal Society of London. Series B, Engbert, R. (2006a). Flick-induced flips in perception. Neuron, Biological Sciences, 167(1008), 293–315. 49, 168–170. Bridgeman, B., Palca, J. (1980). The role of microsaccades in Engbert, R. (2006b). Microsaccades: A microcosm for re- high acuity observational tasks. Vision Research, 20, 813– search on oculomotor control, attention, and visual per- 817. ception. Progress in Brain Research, 154, 177–192. Ciuffreda, K., Tannen, B. (1995). Eye movement basics for the Engbert, R., Kliegl, R. (2003a). Binocular coordination in clinician. St. Louis: Mosby. microsaccades. In: J. Hyn, R. Radach & H. Deubel (eds.) Coppola, D., Purves, D. (1996). The extraordinarily rapid The Mind’s Eyes: Cognitive and Applied Aspects of Eye Move- disappearance of entopic images. Proceedings of the Na- ments., 103–117. tional Academy of Sciences of the United States of America, 93, Engbert, R., Kliegl, R. (2003b). Microsaccades uncover the 8001–8004. orientation of covert attention. Vision Research, 43(9), 1035- Daubechies, I., Teschke, G. (2005). Variational image restora- 1045. tion by means of wavelets: simultaneous decomposition, Engbert, R., Kliegl, R. (2004). Microsaccades keep the eyes’ deblurring and denoising. Applied and Computational Har- balance during fixation. Psychological Science, 15, 431–436. monic Analysis, 19, 1–16. Engbert, R., Mergenthaler, K. (2006). Microsaccades are Daubechies I. (1992). Ten lectures on wavelets. SIAM. triggered by low retinal image slip. Proceedings of the Na- Diallo, M. S., Holschneider, M., Kulesh, M., Scherbaum, F., tional Academy of Sciences of the United States of America, 103,

7192–7197. D. H. (2009). Microsaccades: a neurophysiological analy- Feichtinger, H., Strohmer, T. (1998). Gabor analysis and algo- sis. Trends Neurosci., 32, 463–475. rithms: Theory and applications. Birkhauser. Mergenthaler, K., Engbert, R. (2007). Modeling the control of Galfano, G., Betta, E., Turatto, M. (2004). Inhibition of return fixational eye movements with neurophysiological delays. in microsaccades. Experimental Brain Research, 159, 400– Physical Review Letters, 98(13), 138104. 404. Pritchard, R. M. (1961). Stabilized images on the retina. Sci- Gowen, E., Abadi, R., Poliakoff, E. (2005). Paying attention to entific American, 204, 72–78. saccadic intrusions. Cognitive Brain Research, 25, 810-825. Quiroga, R. Q., Nadasdy, Z., Ben-Shaul, Y. (2004). Unsuper- Gowen, E., Abadi, R. V., Poliakoff, E., Hansen, P. C., Miall, vised spike detection and sorting with wavelets and su- R. C. (2007). Modulation of saccadic intrusions by exoge- perparamagnetic clustering. Neural Computation, 16, 1661– nous and endogenous attention. Brain Res., 1141, 154–167. 1687. Hafed, Z., Clark, J. (2002). Microsaccades as an overt mea- Riggs, L., Ratliff, F., Cornsweet, J., Cornsweet, T. (1953). The sure of covert attention shifts. Vision Research, 42, 2533- disappearance of steadily fixated visual test objects. Jour- 2545. nal of the Optical Society of America, 43, 495–501. Hafed, Z., Goffart, L., Krauzlis, R. (2009). A Neural Mecha- Rolfs, M. (2009). Microsaccades: small steps on a long way. nism for Microsaccade Generation in the Primate Superior Vision Research, 49, 2415–2441. Colliculus. Science, 323(5916), 940-943. Rolfs, M., Engbert, R., Kliegl, R. (2005). Crossmodal coupling Holschneider, M. (1995). Wavelets: An analysis tool. Oxford of oculomotor control and spatial attention in vision and University Press. audition. Experimental Brain Research, 166, 427–439. Holschneider, M., Tchamitchian, P. (1991). Pointwise anal- Rolfs, M., Laubrock, J., Kliegl, R. (2006). Shortening and pro- ysis of Riemann’s nondifferentiable function. Inventiones longation of saccade latencies following microsaccades. Mathematicae, 105, 157–175. Experimental Brain Research, 169, 369–376. Horowitz, T. S., Fencsik, D. E., Fine, E. M., Yurgenson, S., Rucci, M., Casile, A. (2004). Decorrelation of neural activity Wolfe, J. M. (2007). Microsaccades and attention: Does during fixational instability: Possible implications for the a weak correlation make an index? Reply to Laubrock, refinement of v1 receptive fields. Visual Neuroscience, 21, Engbert, Rolfs, and Kliegl (2007). Psychological Science, 18, 725–738. 367–368. Rucci, M., Iovin, R., Poletti, M., Santini, F. (2007). Miniature Horowitz, T. S., Fine, E. M., Fencsik, D. E., Yurgenson, S., eye movements enhance fine spatial detail. Nature, 447, Wolfe, J. M. (2007). Fixational eye movements are not an 852–855. index of covert attention. Psychological Science, 18, 356–363. Sinn, P., Engbert, R. (2009). Saccadic facilitation by modula- Jolliffe, I. (2002). Principal component analysis. Springer Verlag. tion of microsaccades in natural backgrounds. Manuscript Krauskopf, J., Cornsweet, T., Riggs, L. (1960). Analysis of submitted. eye movements during monocular and binocular fixation. Smith, L. (2002). A tutorial on principal components analy- Journal of the Optical Society of America, 50, 572-578. sis. Laubrock, J., Engbert, R., Rolfs, M., Kliegl, R. (2007). Mi- Starzynski, C., Engbert, R. (2009). Noise-enhanced target crosaccades are an index of covert attention: commentary discrimination under the influence of fixational eye move- on horowitz, fine, fencsik, yurgenson, and wolfe (2007). ments and external noise. Chaos, 19, 015112: 1–7. Psychological Science, 18, 364–366. Theiler, J., Galdrikian, B., Longtin, A., Eubank, S., Farmer, J. Laubrock, J., Kliegl, R., Rolfs, M., Engbert, R. (2010). When (1992). Testing for nonlinearity in time series: the method do microsaccades follow attention? Attention, Perception, of surrogate data. Physica D: Nonlinear Phenomena, 58, 77– & Psychophysics, 72, 683–694. 94. Laubrock, R., Engbert, R., Kliegl, R. (2005). Microsaccade Troncoso, X. G., Macknik, S., Martinez-Conde, S. (2008, 11). dynamics during covert attention. Vision Research, 45, 721– Microsaccades counteract perceptual filling-in. Journal of 730. Vision, 8(14), 1-9. Mallat, S. (1998). A wavelet tour of signal processing. Aca- Venegas, S. (2001). Statistical methods for signal detection in demic Press. climate. Danish Center for Earth System Science Report, 2, 96. Mallat, S., Hwang, W. (1992). Singularity detection and Witkin, A. P. (1983). Scale-space filtering. In Ijcai’83: Pro- processing with wavelets. IEEE Transactions on Information ceedings of the eighth international joint conference on artificial Theory, 38, 617–643. intelligence (pp. 1019–1022). Marr, D., Hildreth, E. (1980). Theory of Edge Detection. Zuber, B. L., Stark, L., Cook, G. (1965). Microsaccades and the Proceedings of the Royal Society of London. Series B. Biological Velocity-Amplitude Relationship for Saccadic Eye Move- Sciences, 207, 187-217. ments. Science, 150, 1459-1460. Martinez-Conde, S., Macknik, S., Hubel, D. (2000). Microsac- cadic eye movements and firing of single cells in the striate cortex of macaque monkeys. Nature Neuroscience, 3, 251– 258. Martinez-Conde, S., Macknik, S., Hubel, D. (2004). The role of fixational eye movements in visual perception. Nature Reviews Neuroscience, 5, 229–240. Martinez-Conde, S., Macknik, S., Troncoso, X., Dyar, T. (2006). Microsaccades counteract visual fading during fixation. Neuron, 49, 297-305. Martinez-Conde, S., Macknik, S. L., Troncoso, X. G., Hubel,

DOI 10.16910/jemr.3.5.1 This article is licensed under a ISSN 1995-8692 Creative Commons Attribution 4.0 International license.