V R V 3 1 ( 0 E ScienceLtd.All rightsr S ““ P G B 0 $ + 0

Stability of Binocular Depth Perception with Moving Head and Eyes RAYMOND van EE,*~ CASPER J. ERKELENS* R F 1 r f D 1 infinalform M 1

s p o

r h h a r p d m p m a e @

Binocular disparity Binocular vision Depth perception Stereopsis Robot vision

I movement, the disparity field before and after the The separationof the human eyes causeseach eye to see a movements will also change. However, this time the disparityof the object and environmentchange together. disparate image of the outside world. Generally, it has In short, during eye and head movements,the images of been accepted that positional disparitiesare sufficientto the entirevisualworld are continuouslychangingon both generate a three-dimensional(3D) percept (e.g. Whets- retinae, which means that there are also continuously tone, 1838; Ogle, 1950; Julesz, 1971). Wheatstone’s changing disparities. One would expect these changing developmentof the stereoscopein 1838was based on this disparitiesto reduce the stability of stereopsis. idea. Recently, this knowledgehas been used in the field In principle, the visual system can utilize the signals of binocular robots. However, many phenomenarelating that control the eye and neck muscles (efference copies) to disparity and perception of depth are still not under- in order to correct stereopsis for disparities induced by stood, including the fact that binocular vision is largely controlled eye and head movements. However, dispa- unaffected by eye and head movements (Westheimer & rities are not only due to controlled eye and head McKee, 1978 concerning lateral eye movements; Stein- movements,they can also be due to uncontrolledeye and man et al., 1985 and Patterson & Fox, 1984 concerning head movements. These uncontrolled movements are head movement). caused by noise in the motor system. Experiments have In binocular robots the quality of 3D analysis is demonstrated large discrepancies between the level of severely reduced by the instability of the cameras (the stereoacuity and the relative sloppiness of oculomotor disparity acquisition system; Eklundh, 1993). By ana- control. Optimal stereoacuity thresholds in the fovea logy, one would expect the stability of human binocular typically attain mean standard deviationsof the order of vision to be reduced by eye and head movements.In the 5 sec of arc (Berry, 1948;Westheimer & McKee, 1978; case of a simpleobjectlike a chessboardit is immediately McKee, 1983),which is about one-sixth of the diameter clear that the imagesof the chessboardon our two retinae of the smallestfoveal cones (Westheimer, 1979a).These differ according to whether the chessboard is positioned thresholds for stereoacuity can be obtained even for a in front of us or eccentrically.Since the disparityfield is 200-msec exposure (Westheimer & McKee, 1978) and composed of the positional differences between the are similar in magnitude to the best monocular hyper- retinal images, the disparity field will depend on the acuities for motion displacement, vernier tasks and position of the object. On the other hand, if the object is relative width (Westheimer & McKee, 1979; McKee et static but the binocular observer makes an eye or head al., 1990b). Given these values, binocular vision can be very sensitive. It can be regarded as a hyperacuity mechanism.On the other hand, duringnaturalbehaviour, *Utrecht University, Faculty of Physics and Astronomy, Helmholtz Instituut,Princetonplein5, NL-3584CC,Utrecht,The Netherlands. vergence position errors of up to 1–2 deg (Collewijn & +Towhom all correspondenceshouldbe addressed. Erkelens, 1990),vergence velocity errors of up to 1 deg/ 3827 3828 R. van EE and C. J. ERKELENS sec (Steinman & Collewijn, 1980) and errors in importantly,Mayhew& Longuet-Higgins(1982)showed cyclovergenceof 10 min arc (Enright, 1990;van Rijn that information about gaze parameters in principle can a 1994) are easily generated and introduce disparities be calculatedfrom the horizontaland vertical disparities. that are similar in size to the errors. The measured This gaze parameter information could then be used to sloppinessof oculomotorcontrolis not due to artefactsin interpret disparities. In addition, G&ding et al. (1995) experimental methods (Ferman a 1987). Besides proposed a decompositionof the disparity interpretation oculomotor system instability,there is another factor of process into disparity correction, which is used to uncertaintywhich affects the interpretationof disparities, compute three-dimensional structure up to a relief namely the exact orientation of the head relative to the transformation, and disparity normalization, which is body.Head stabilityis no better than oculomotorstability used to resolve the relief ambiguity to obtain metric (Schor et al., 1988). structure.Discussingthe existingliteraturebased on this Although oculomotorand head control is sloppy, it is decomposition into disparity correction and disparity neverthelesspossiblethat a feedback systemis at work in normalization, they showed that in relief tasks depth binoculardepth perception.* The noisein the oculomotor perceptionexhibitsa large and stable dependenceon the system, though producing (cyclo)vergenceerrors, could structure of the vertical disparity field, whereas metric be known to the visual system (for instanceby means of tasks are hardly affected. Glirding et al. (1995) also muscle sensors) and utilized in order to interpret reported on the fact that visual tasks that actually require disparities. There is evidence, however, that there is no a full metric reconstruction of the three-dimensional such feedback system in binocular depth perception. visualworld are fairly uncommon.The relief transforma- Firstly, fast side-to-siderotationsof the head, or pressing tion preservesmany importantpropertiesof visual shape, against the eyeball, do not influence depth perception notablythe depthorder as well as all projectiveproperties (Steinmanet al., 1985).Secondly,the resultspresentedin such as coplanarity and collinearity. Therefore, a several reports (Foley, 1980; Erkelens & Collewijn, disparity processing system that computes a reconstruc- 1985a, b; Regan a 1986; Collett et al., 1991; tion of the three-dimensional visual world relying on Cumming et al., 1991; Logvinenko & Belopolskii retinaldisparitiesaloneis very attractiveeven if it doesso (1994); Rogers & Bradshaw, 1995 and Backus et al., up to a relief transformation. 1996) show that in situations where (conflicting) eye Important exceptions to the idea that the metric tasks muscle information is available changing eye posture are hardly affected by vertical disparitiesare the studies does not lead to changingperceptionof depth in the case of Rogers& Bradshaw(1993, 1995).Rogers& Bradshaw of large field stimuli (large displays)or they lead to only (1993)showedthat subjectscan use vertical disparitiesin weak perception of depth in the case of small field order to estimate the perceived peak-to-troughdepth of stimuli. corrugationsfor large-fieldstimuli.However,the amount The discrepancies between the sensitivity of stereo- of perceiveddepthin the full-disparity-cueconditionwas scopic vision and the sloppiness of oculomotor control very much less than would be required for complete mean that oculomotor stability is at least one order of depth constancy. In the Appendix of their 1995 paper, magnitude less precise than measured stereoacuity (also Rogersand Bradshawshowedthat absolutedistancefrom reported by Nelson, 1977 and Collewijn et al., 1991). the observer is altered by modifying vertical disparities. Even if we assume that subjects can obtain very good [See also the paper by Friedman et al. (1978) who also stereoacuity by using relative depth differences (which found that vertical disparity influencesmetrical percep- are unaffected by noisy eye and head movements; tualtasks.]Yet, thesestudieshave notbeen conductedfor Westheimer, 1979b; Erkelens & Collewijn, 1985a, b) limited observationperiods. we still do not know why the stereoacuity stimulus as a Despite these findings about a disparity processing whole does not tremble in depth as a result of the system that computes a metrical reconstruction there is trembling eye and head movements. no evidence yet that such a system is effective in human A possible way for the visual system to deal with the vision on a shorttime-scale.In the next sectionwe report effects of sloppy motor control is to utilize all available on perceptual studies using simple stereograms which retinal information.Frisby, Mayhew & co-workers have show that several classes of basic stimuli which mimic proposedthat gaze (eye posture)parameterstheoretically real world stimuli (containing both realistic horizontal can be calibrated by “shape-from-texture”. However, and vertical disparities)do not elicit reliable perception recently they showed that this hypothesis was not of metric aspectsof depthfor limitedobservationperiods confirmed experimentally (Frisby et al., 1995). More (up to the order of seconds).On the otherhand it hasbeen reported that relief tasks in stereopsis can be effective even to the order of milliseconds(e.g. Kumar & Glaser, *Binocular3D vision involvesperceptionof directionsand perception 1993;Uttal et al., 1994). of depth. Regarding binocular perception of directions there is evidence that vestibular and proprioceptiveinformationis used to maintain stability (Howard, 1982; Carpenter, 1988).For instance, Stereogramsand depthperception fast side-to-side rotations of the head (Steinman a 1985), or Our knowledge about binocular depth perception is pressing against the eyeball, impair the correct couplingbetween extra-retinal signals and perceived directions and result in obtained to a large extent from experimentswith stereo- impairmentof the stability of the visual world in lateral directions. grams. In such experimentsthe subjectviews (with static

. . STABILITYOF BINOCULARDEPTH PERCEPTION 3829

Another perceptual study (Howard & Zacher, 1991) showedthat differentialrotationof the entire half-images of a stereogram induces cyclovergence with a gain unequal to one (cyclodisparity)but elicits poor percep- tion of depth. Again, two different cyclodisparities, simultaneouslypresentin the visualfield,are requiredfor reliabledepthperception.Collewijnet al. (1991)reported FIGURE 1. An example of a horizontal scale and a horizontal shear that thresholdsfor perception of depth caused by cyclo- transformation between the observed half-images. Horizontal scale between the half-images of the stereogram leads to perceived slant disparity increase by a factor of 7 when the visual about the vertical axis. Horizontalshear leads to perceived slant about reference is removed. Cyclodisparities can have con- the horizontal axis. M is the magnitude of the horizontal scale siderable magnitudes and can occur frequently during transformation expressed as a fraction, /l is the magnitude of the natural behaviour. Howard (1993) suggested that horizontal shear transformation,expressed as an angle. whole-fieldcyclodisparitiescould indicate that the eyes are misalignedand that thereforethe perceptualsystemis inclined to ignore these cyclodisparities when judging slant. head) two half-imagesof a stereogram, one transformed Finally, slant from horizontal scale and horizontal relative to the other, projected on a screen. In accordance shear between the entire half-images of a stereogram is with geometricalrules,local horizontallateral shiftof the relatively poorly perceived (Shipley & Hyson, 1972; half-images relative to each other alters the perceived Mitchison & Westheimer, 1984, 1990; Stevens & distance. Horizontal scale between parts of the half- Brookes, 1988; Gillam et al., 1988). Recently, van Ee images of a stereogramleads to perceived slant aboutthe & Erkelens(1996a)investigatedtemporalaspectsof slant vertical axis. Local horizontal shear, on the other hand, perception induced by whole-field horizontal scale and leads to perceived slant about the horizontalaxis. Figure horizontal shear. They quantitatively corroborated the 1 shows an example of the horizontal scale and shear earlier findingthat when observationperiods last up to a transformation. few seconds, perception of slant caused by whole-field Lateral movements of the entire half-images of a horizontal scale and shear is relatively poor. Using stereogramrelative to each other lead to vergence of the argumentssimilarto those presented in the previoustwo eyes (with a gain unequal to one), but are not interpreted paragraphs we now attempt to relate this experimental as changes in distance (Erkelens & Collewijn, 1985a,b; result to the orientation of the head. Head rotation in a Regan et al., 1986). In contrast, differential movements stationaryvisualworld shouldcause similardisparitiesas of parts of the half-imagesgive rise to vivid perceptionof rotationof the entire visual world about the centre of the motion in depth. In addition,Regan et al. (1986) showed headwhen the head is stationary.This idea is explainedin that under stabilizedretinal conditionsabrupt changesin Fig. 2 where two similar drawings are depicted. The the image vergence angle produced no impression of a drawingin Fig. 2(a) representsthe geometryof viewing a step change in depth. These authors suggested that the horizontallyscaled stereogram.The drawing in Fig. 2(b) explanation for their results may be that the brain inter- represents the geometry of viewing an (initially) frontal prets lateral shifts between the entire parts of the stereo- plane with rotated head (initially, fixation was at grams as movementsof the eyes and thereforethese shifts eccentricityu). In the horizontalplane the retinal images are best ignored as signals for depth. (As we will see of the two situations are the same. Analogously, one below this explanationis not entirely correct.) expectsdisparitiescausedby forwardrotationof the head

a) Slant ,. b) h / i , , / LE half- ‘ i ~ d

?!Head i rotation FIGURE2. (a) Showsthe geometryof a horizontallyscaled stereogramwith unrotatedhead.(b) Showsthe geometryof an (initially)frontafplane with rotated head. The retinal images in the horizontalplane are identical in both situations. LE and RE denote left and right eye, respectively. Note that the geometryof a frontal plane at a distance ofzoviewed after a head rotationover a deg is similar to a slantedplane over u degbut at a distance of z~cos 3830 R. van EE and C. J. ERKELENS to correspondto disparitiescaused by horizontalshear of poor depth perception. So far, this knowledge has not the half-images of a stereogram. The argumentswe use been suppliedby the literature. are similar to those used by Erkelens & Collewijn (1985a) and Howard et al. (1993). We suggest that the reasonwhy depth perceptionof one linear transformation within the stereogramis poor and depthperceptionof two Headcentric coordinatesand head movement different linear transformations is vivid is that the In order to identify a test point P in three-dimensional disparity field caused by only one linear transformation space relative to the head we define a right-handed is ambiguous.In otherwords, head rotationscould induce orthogonalcoordinate system with the origin above the the same disparity fields as the scaled and sheared vertebralcolumnand at the same level as the eyes.The x- stereograms.We argue that the disparityfieldscaused by axis points from right to left parallel to the interocular horizontalscale and shear are thereforeprimarilyignored axis, the y-axis points vertically upwards, and the z-axis as signals for perception of slant. We also argue that the points in the primary direction (straight ahead). After a disparity field caused by two different, simultaneously head rotation or translation the headcentric coordinates present,linear transformationscannotbe similarto a field (#,~,#) of test point P are (x~,y~, #). Head causedby ego-movementand, therefore,such a fieldis an translation correspondsto a trivial coordinate modifica- effective stimulus for the slant perception of one plane tion.For example,a head translationalongthey-axis over relative to the other. an arbitrary distance (ad) modifiesy$ coordinates into # = ~ – ad. Head rotation is not trivial. The ccror&- Hypothesis nates before and after a head rotation are related to each Thus, during (noisy) eye and head movements the other by an Euler rotation matrix:

fX51 /E)

cos~cos~ + sin@’sin@sinq!+’ cos@sin@ –sinq#cos@ + cos@sinVsinq!P’ RH = –cos@sin@ + sin@sin@cos@+’ cos~cosfl sin@F’sin~ + cos@sin@cos@ , (1) ( sin@cos# –sin@ cos@cosOH ) disparity field changes continuously. Why do we not where ~H, ~ and ~H denote angles of head rotation perceive a visual world trembling in depth as a result of about the vertical, horizontal and primary direction, our trembling disparity acquisition system? One could respectively. The signs of the angles are again defined think of two opposite hypotheses. Either the visual accordingto a right-handedcoordinatesystem.The order system compensates completely for the disparities of rotations is described in a Fick manner, which means induced by these (noisy) eye and head movements or that the head is first rotated about the vertical axis, then the visual system is blind for these disparities. The findings about (1) using the signals that control the eye and head muscles(efferencecopies),(2) using a feedback loop based on muscle sensors and (3) using all the available (horizontal and vertical) disparities, suggest that the compensation hypothesis does not provide a sufficientanswer to our question,at least not for limited (realistic) observationperiods. Taken together, the above-mentioned suggested ex- planationsfor the poor sensitivityof depth perception to several transformationsbetween half-imagesof a stereo- gram lead to a generalized hypothesis.We hypothesize x that a possibleway for the visual system to deal with the effects of sloppy eye and head movementsis to use only FIGURE3. In Fick’s coordinatesystem a target is uniquelyidentified that part of disparityinformationwhich is invariantunder relative to the left eye by its longitude O: and its latitude ~~. In this eye and head movements. Investigations about the figurethe eye points to a fixationpoint which is at infinity.The origin validity of this hypothesis require precise knowledge of the oculocentric coordinate system is located at the centre of the eyeball. The x-axis of the coordinate system points from right to left, about what sort of disparity is induced, on the one hand the y-axis points vertically upwards, and the z-axis points in the by eye and head movements and on the other hand by primarydirection. In the directionof the arrow the sign of the angle is transformed stereogramswhich are known to elicit only definedas positive.

.— STABILITYOF BINOCULARDEPTHPERCEPTION 3831 about the horizontal axis and lastly about the primary infinity,which means that the visual axis coincideswith direction.* the primary direction.As shown in Fig. 3 the coordinates Headcentric coordinatesand stereograms (~, fi,z$) of a test point P relative to the left eye are If stereogramsare involved,then a separatecalculation parametrizedin a Fick mannerby its longitude~P and its has to be performed to find the headcentric coordinates latitude ~: for each of the two transformed half-images:

# — COS(0,54Y)+ horscale –sin(O.5t/P)+ horshear y? 1.J3eyeimuge- sin(O.5w) c ) -

– cos(-O.5479 –sin(–O.5~) # o.5shift # (2) – sin(–O.5@) cos(–O.5#P) (y~ r i ( )(M )righteyeirnage+ [) 0 ‘ where @, shift, horscale, horshear denote the rotation, lateral shift, horizontal scale and horizontal shear between the entire two half-images of the stereogram, respectively.For simplicitywe assume that there is only one transformation at a time between the parts of the where ,Z1is the distance between the centre of the stereogram relative to each other. oculocentric coordinate system and the headcentric coordinate system along the z-axis. 1 denotes the interoculardistance.Similarnotationis used for the right Oculocentriccoordinates eye. In addition to defininga coordinate system relative to The directionof a new fixationpoint relative to the left the head, we also have to define retinal coordinate eye is denoted by longitude #F and latitude @F.The systems.As before, thex-axispointsfrom rightto left, the coordinatesof a point before and after an eye rotation to y-axis points vertically upwards, and the z-axis points in the new fixation point are related by an Euler matrix the primary direction. The centre of the oculocentric similar to the one given above. After an eye rotation to coordinate system is positioned in the centre of the eye. the fixationpointthe coordinatesof pointP relativeto the Initially, we assume that the eye fixates a target at left eye are (xf,yf, Y~)

cose~cos~~ + sin@~sin@Fsin!/$ cos+$sin~~ ‘sin~~cos+’$ + cos@~sint@intiF RL = –cos@Fsin@F+ sin$$fsin~~cos~~ cos@Fcos@F sin@Fsin@~+ cos$$Fsin@Fcos$$~. (4) ( sin@Fcos#F –sin@F cos@Fcos@F )

*Fick’s coordinate system (Fick, 1854) and Helmholtz’s coordinate From these three equationsthe longitudeOf and latitude system (von Helmholtz, 1911) originate from eye movement studies. Rotations do not commute under summation. Decisions ~~ in the rotated eye coordinatesystemcan be calculated should be made about the order in which rotations should be for arbitrary test points and fixation points. An identical performed.In Fick’s system, the vertical axis of the eye ball is assumed to be fixed to the skull and the horizontal axis of the eye procedurehas to be performedfor the righteye in orderto ballisassumedtorotategimbal-fashionaboutthe vertical axis. In find 4$ and r9$. Disparity is computed from the Helmholtz’ssystem it is the horizontalaxis which is assumedto be differencesbetween retinal coordinatesin the two eyes. fixed to the skull (Howard, 1982; p. 181). Which system is preferablewilldependonthesituation.TheadvantageofFick’s Horizontal (in fact longitudinal)disparity is defined by systemis thatiso~ergencesurfacesareequivalenttoisodisparitysubtracting #p from q$~.Vertical(latitudinal)disparityis surfaces.TheadvantageofHelmholtz’ssystemisthatit is based on epi-polar geometry. obtained by subtracting8$ from 19~. 3832 R. van EE and C. J. ERKELENS

a) b) EI[deg] 40

dl Verf. 1 Her. L d2 disp. f 1 disp. [arcm!; 2 [arcmin]1

FIGURE4. Horizontal(a) and vertical disparity(b) of a frontal plane at a distance of 250cm (all, white patch) and 50 cm (d2, grey patch) in front of the eyes. ~ denotes longitude,19denotes latitude. Both angles are taken relative to the head.

N C sely, for stimuli further away than the horopter, The above-mentioneddefinitionsare implementedin a horizontal disparity is negative. Since the horizontal computer program in which we compute disparityfields disparity does not depend on latitude (in Fick’s descrip- generated by planar surfaces. The disparity fields are tion),the horizontalcomponentof the disparityfield [Fig. computed for a range of eye, head and object positions, 4(a)] doesnot dependon 9 either.Disparityis zero for the on the one hand, and for severaltransformationsbetween fixation point. When fixation is on the plane in the half-imagesof a stereogram,on the other hand. Through- primary direction, points of the frontal plane have out the text and figures, disparity is calculated in negative horizontal disparity because all points are located further away than the horopter. oculocentriccoordinatesfor a field of 80 x 80 deg which is centered around the fixation point. This field is The vertical disparity fields of the frontal planes at provided with a (virtual) lattice of 12x 12 evenly 250 cm (all) and 50 cm (d2) in front of the eyes are shown in Fig. 4(b). These fields depend both on ~ and 0. Each distributeddirections (the angle between adjacent direc- point located outside the plane of fixation and nearer to tions is 80/11= 7.3 deg). Disparity is calculated for each one eye than to the other eye has vertical disparity.Since of the 144 directions.Results are plotted as a function of fixationis chosen to be in the primary direction,vertical longitude(~) and latitude (0) which are taken relative to disparity is zero along the directions ~ = Oor 6 = Oand the head. In our calculationswe use planar surfaces,since anti-symmetricalwith respect to these axes. planar surfaces have simple computationalproperties.In addition, the disparity fields of these surfaces have Eye and head movements vs stereograms several symmetricalproperties,as is shown in the figures It will be demonstrated that disparity fields that are throughout this paper, which makes them easier to brought about by eye and head movements can be interpret.However, in principleit is not relevantwhat the adequately simulated by stereograms. In the rest of the source of the disparity field is. We are not primarily paper we compare the disparity field caused by a interested in the disparity fieldper se. We are interested particular eye or head movementwith the disparity field in how a disparityfield transformsas a result of eye and caused by the stereogram that correspondstheoretically head movements.In the calculationswe take the centreof to the eye or head movement.The basic stimulus(before head rotation to be 10 cm behind the eyes and the the eye or head movement)is always a frontal plane at a interoculardistanceto be 6.5 cm. Again, the exact values distance of 100cm in front of the eyes. of these quantitiesare not relevant for the purposeof our study. We make the assumptionthat the nodal point and Cyclovergencevs differentialrotation within the stereo- the centre of eye rotation coincide. Cormack & Fox gram (1985) found almost no effect of nodal point motion for According to Donders’ law (Donders, 1876) and different fixations, except under the most extreme Listing’slaw (Listing, 1854)the eyes are slightlyrotated conditions. relativeto each other aboutthe line of sightwhile fixating a tertiary position (for a review see Alpern, 1962). This Disparity and the distance of the object implies that after a change of fixation cyclodisparity is Figure 4 shows how the disparity field depends on introduced.The disparityfieldof the frontalplane caused viewing distance and direction. In this figure the by pure cyclovergence is depicted in Fig. 5. The horizontal and vertical disparity fields are shown for magnitude of cyclovergence is chosen to be 1.26 deg two fronto-parallel(frontal)planes at distancesof 250 cm (each eye 0.63 deg). Figure 5(a and b) shows the (the white patch, dl, in Fig. 4) and 50 cm (grey patch, horizontal disparity and vertical disparity, respectively, d2). The figure shows that disparity fields of frontal of the plane after such cyclovergence. planes are curved when viewed at a finite distance. Figure 5(c and d) shows the horizontal and vertical Objects that are curved along the horopter have zero disparityfield of a stereogramwith differentiallyrotated disparity. For stimuli nearer than the horopter the half-images.Each part of the stereogram is rotated over horizontal disparity is, by definition, positive. Conver- 0.63 deg in oppositedirections.The disparity field (both STABILITYOF BINOCULARDEPTHPERCEPTION 3

Cyclovergence a) b)

-20 -20 -40 Ven. 100 disp. [arcmin]1-1$ -40 -20

@[deg]

Differential rotation of stereogram c) El[deg]40 d) 13[deg]40

-20/-7 I

Her. 1 disp. [arcmin]1-1 ::MJ2”W -40 -20 < 20 - o [deg] 40

Difference between cyclovergence and differential rotation e) f)

-20 -40 -40 20 20 Her. Vett. disp. disp. [arcmin]t -z: [arcmin]t -2: -40 -40 -20 < CD[deg] 40

FIGURE5. The top two panels show horizontal(a) and vertical disparity(b) of a frontal plane at a distance of 100cm viewed with cyclovergenceof 1.26deg. The middlepanels showthe horizontal(c) and vertical disparity(d) after a differentialrotation of 1.26deg within the correspondingstereogram.The bottom two panels show the difference in horizontal(e) and vertical (~ disparitybetween cyclovergenceof the eyes and differentialrotationwithin the stereogram.Note that the dimensionsalongthe disparity axes of the bottompanels are different from the other figures.

horizontal and vertical) is more curved for negative O entireparts of a stereogramare approximatelyequivalent because the points of intersection of the light rays that [Fig. 5(e, f)]. come from correspondingpoints of both half-images of Head translationin the primary direction vs horizontal the stereogramare nearer for negative Othan for positive shift within the stereogram $. In the case of cyclovergencethe axis of rotation is the Generally,the disparitiescaused by a head translation towardsa frontalplane can be simulatedby the following visual axis. In the case of differential rotation within a lateral shift between the two half-images of the stereo- stereogram the axis of rotation of either half-image is gram: perpendicularto the projectionscreen.Therefore,it is not TZ.1 trivial that the disparity fields induced by cyclovergence shift = ———— (5) Z. —Tz‘ and differentialrotationare equal.The similaritybetween the numerical results of Fig. 5(a, c) and the numerical where T=denotes the translationof the head towards the results of Fig. 5(b, d) implies that the disparity fields plane, Z the interocular distance and Z. the distance caused by cyclovergenceand differential rotation of the between the stimulus and the eyes. We can derive this 3834 R. van EE and C. J. ERKELENS

Left eye relationshipin a straightforwardmanner as is illustrated is[ hift half-image in Fig. 6. From this figure it immediatelyfollows that: Frontal screen * t ,..’ ,., ~shift ~1 :6[ ‘; tan :6 = — — (6) ‘. .’““”TZ () T,= Z. —TZ“ ,.,, .’ .. ‘, ...” c; ‘...... Frontal plane’’~,, - ,... Figure 7(a, b) shows the horizontaland vertical disparity .’ ., ,..” ; ,’ ... ,., fields of the plane (which was initially positioned at ‘ .28 ,.:: 100cm) after a head translation of 25 cm towards the .. -. ,., plane. Effectively these fields are similar to the fields ,., ,...,, -- ZO-TZ .,” caused by a plane at 75 cm, which means that both the ,’ ., ... .. ; ..” ,.,, -. horizontaland vertical disparityfields are more strongly ,’ ,..’ ... : ,’ ,.. ,.,,’.’,, curved than those of a plane at 100cm. According to the .,,., .,., 112 ..4 2 a v given relationshipbetween lateral shift and head transla- Left eye Right eye tion the disparitiescaused by a head translationof 25 cm towards the frontal plane (at 100cm in front of the eyes) FIGURE6. A lateral shiftbetweenthe twohalf-imagesof a stereogram can be simulatedby a lateral shift of 2.2 cm between-the leads (within the horizontal plane) to similar disparities as a head translation in the primary direction. T, denotes the translation of the half-images of a stereogram (at 100cm in front of the head towards the plane, Zthe interocrdardistance and Z. the distance eyes). Figure7(c),showsthe horizontaldisparityfieldand between the stimulus and the eyes. Fig. 7(d) the vertical disparityfield of such a stereogram.

Head translation in primary direction a)

-20 -20 -40 -40 Her. 100 Veri. 100 disp. disp. [arcmin]1-lo: [arcmin]I.lJ -40 -40 -20 -20 0 1. 20 ‘cD [deg] 40

Offset of stereogram c) d) -

-20 -40 Her. 100 diap. [arcmin]1-lJ -40 .- -20 -20

Difference between head translation and uffset e) f)

-20 -20 -40 -40 20 20 Hor Veri dlsp o dlsp [arcmm]1-20 [arcmm]t -2; -40 -40 -20 -20 <

FIGURE7. The top twopanels showhorizontal(a) andvertical disparity(b) of the frontalplane (initiallyat 100cm) after a head translationof 25 cm in the primary direction.The middlepanels showthe horizontal(c) and vertical disparity(d) after a lateral shift of 2.2cm within the theoretically correspondingstereogram. The bottom two panels show the difference in disparity between head translation and a stereogram-inducedlateral shift. STABILITYOF BINOCULARDEPTHPERCEPTION 3835 Head rotation about the vertical axis a) b)

-20 -40 -40 Her. 100 Velt. 100 disp. disp. [arcmin]t-lo: [arcmin]1-lo: -40 -40 -20 -20

Horizontal scale of stereoaram c)

-20 -20 -40 -40 Her. ‘M Vert. 100 disp. disp. [arcmin]1-lo: [arcmin]I.l& -40 -40 -20 -20

Difference between head rotation about vertical axis and horizontal scale e) f)

-20 -20 -40 -40 20 20 Her. Vert. disp. disp. [arcmin]t -,: [arcmin]t -~ -40 -40 -20 -20 <

FIGURE8. The top two panels show horizontal(a) and vertical disparity (b) of the initially frontal plane after a head rotation over 20 deg aboutthe vertical axis. The middlepanels showthe horizontal(c) and vertical disparity(d) after horizontalscaling of 2.2%within the theoreticallycorrespondingstereogram.The bottomtwo panels showthe difference in disparity inducedby head rotation about the vertical axis and the stereogram-inducedhorizontal scale.

The similarity between Fig. 7(a) and (c) as well as Rotation of the head about the verticalaxis vs horizontal between Fig. 7(b) and (d) implies that in the disparity scale within the stereogram domain head translation in the primary direction and Considera frontalplane at 100cm which is fixatedat a lateral shift of the two entire parts of a stereogram are longitudeof 20 deg. Next, consider a clockwise rotation almost equivalent. Figure 7(e,f) show the difference in of the head about the vertical axis over an angle of 20 disparity between the Fig. 7(a,b) and (c,d). deg. Figure 8(a) shows the horizontal disparity of the The results reported so far can be related to the results plane after the rotation. The distance between the plane of Erkelens & Collewijn (1985a,b). They recognizedthat and the head is shorter for negative than for positive ~. As a result the disparityfield is more curved for negative a change of fixation causes a translation of the retinal ~, which is in agreement with the results shown in Fig. images. They also suggested that an offset in the 4(a). Figure 8(b) showsthe vertical disparityof the plane disparity domain corresponds to a lateral shift between underthe sameviewingconditions.The vertical disparity the two parts of a stereogram. Figure 7 shows that the is anti-symmetricalwith respect to the line # =20 and latter suggestion is not entirely correct. A lateral shift with respectto the line 0 = O.The fact that there is a larger within a stereogram corresponds to a head translation difference in distance between the plane and either eye towards the stimulus,but not to a vergence movementof for negative than for positive ~ results in larger vertical the eyes. disparitiesfor negative ~. 3836 R. van EE and C. J. ERKELENS

In the introductory section we explained why the initiallyfrontal plane at a distanceof 100cm and fixated disparity fields of horizontal scale can be expected to at a latitudeof 20 deg is viewed after a forwardrotationof correspondto the disparityfieldscaused by head rotation the head (about the horizontal axis) over an angle of 20 about the vertical axis. Now we will calculate the deg. Since the distancebetween the plane and the head is disparity fields caused by horizontal scale between the shorterfor negativethan for positive(3,the disparityfield two parts of the theoretically correspondingstereogram. is more curved for negative 0. Figure 9(b) shows the In order to know which stereogramcorrespondsbest we vertical disparity for the same viewing conditions. The use a relationship between slant and the amount of vertical disparity is anti-symmetricalwith respect to the horizontal scale [see van Ee & Erkelens (1996a) for a @= O directionbut is not anti-symmetricalwith respectto derivation]: the 0 = O direction. This time, vertical disparities for negative 0 are larger than those for positive 0. (7) The relationship between slant and horizontal shear ‘l(angle P) is (see van Ee & Erkelens (1996a)afor a derivation)~: where M is the magnificationfactor of horizontalscale,Z the interocular distance and Z. the distance from the (8) stimulus.* According to this relation, a frontal plane at a ‘lant=arctan(tan~ ”?) distance of 100cm viewed after a head rotation over 20 This means that, theoretically, the disparity field of a deg about the vertical axis correspondstheoreticallyto a frontal plane at a distance of 100cm viewed with the stereogram at a distancet of 106cm with a horizontal head forwardly rotated over 20 deg corresponds to a scale of 2.2$Z0. horizontallyshearedstereogramwith magnitude1.26deg Considera stereogramat a distanceof 106cm in front at a distance of 106cm. Figure 9(c, d) shows the of the eyes. Figure 8(c) depicts the horizontal disparity horizontaldisparityand vertical disparityinduced by the caused by a horizontalscale of 2.270.Since the points of correspondinghorizontallysheared stereogram. [Figures intersection of the light rays which come from corre- 9(c) and 5(c) are similar to each other because the spondingpointsof both half-imagesof the stereogramare horizontal component of disparity induced by a hori- nearer for negative q5than for positive ~, the disparity zontal shear of 1.26 deg is similar to the horizontal field is more curved [as in Fig. 8(a)]. Figure 8(d) shows disparity caused by differential rotation of 1.26 deg the verticaldisparitythat is causedby thistransformation. within the stereogram.]The bottompanels of Fig. 9 show Note that the horizontalscale transformationalso has an that the horizontal(e) and vertical (f) disparitycaused by influenceon vertical disparitybecausethe left-handsides a forwardhead rotationresemblesthe horizontaldisparity of the half-images of the stereogram are translated in inducedby horizontalshear. The equivalenceis not very opposite directions, which alters the distance of these good. A possible reason is that the horizontal shear of a parts from the eyes (the same holds for the right-hand stereogram affects the x-component of the perceived sides).As shownin Fig. 8(b),the verticaldisparityis anti- plane, which is not the case with perceivedfrontalplanes symmetrical with respect to the line O= O. The bottom after a rotation of the head. A slanted plane, induced by panels of Fig. 8 show the differencein horizontal(e) and horizontalshear of a rectangularstereogram,is perceived vertical (~ disparity induced by head rotation about the as a trapezoid with the small side nearer than the large vertical axisand by the correspondinghorizontallyscaled side. stereogram. Rotation of the head about the horizontal axis vs horizontalshear within the stereogram Figure 9(a) shows the horizontal disparity when the In this paper we have studied the influenceof eye and head movements on the binocular disparity field. We have also investigatedwhat sortof disparityis inducedby differential rotation, horizontal lateral shift, horizontal *We adopt Ogle’s notation. Ogle (1950), who related slant to hori- scale and horizontal shear between half-images of a zontal magnificationusing an aniseikoniclens, found: stereogram.We have found that in the disparity domain: 1. Cyclovergence resembles differential rotation be- ‘tween the half-imagesof the stereogram; 1 The relationship between slant and magnificationin the case of a 2. Head translationin the primary direction resembles stereogram is slightly different from the relationshipin the case of horizontal lateral shift between the half-images of aniseikonic lenses. ~FromFig. 2 it can be inferredthat the geometryof a frontal plane at a the stereogram; distance of z. viewed after a head rotation over Mdeg is similar to 3. Rotationof,the head aboutthe vertical axis (side-to- that of a plane slanted over a deg but at a distance of z@os ct. side rotation) resembles horizontal scale between &4fter completing this paper, Prof. Collewijn remarked that Ogle & the half-imagesof the stereogram; Ellerbrock (1946) derived a similar equation in order to describe the relationship between the slant of one, in the medial plane- 4. Rotation of the head about the horizontal axis positioned,vertical line and the retinal orientationdisparityof this (forward rotation) resembles horizontal shear be- line. tween the half-imagesof the stereogram.

— STABILITYOF BINOCULARDEPTH PERCEPTION 3837

Head rotation about the horizontal axis a) b) ~d;9J40

-20 -20 -40 -40 Her. 100 Vert. 100 disp. disp. [arcmin]1.lo~ [arcmin]1.lo~ -40 -.-m -20 -20

Horizontal shear of stereogram c) 13[deg] 40 d)

-20 -20 -40 -40 Her. 100 Vert. 100 disp. disp. [arcmin]t-108 [arcmin]I-108 -40 -40 -20 -20

Difference between head rotation about horizontal axis and horizontal shear 13[deg] 4CI f) EI[deg] An e) I

-20 -20 -40 -40 20 20 Her. Verl disp. o dlsp [arcmin]1 [arcmm]t -2: -20 -40 -40 -20 -20

FIGURE9. The top two panels show horizontal(a) and vertical disparity(b) of the initially frontal plane after a head rotation over 20 deg about the horizontal axis. The middle panels show the horizontal (c) and vertical disparity (d) after horizontal shearingover 1.26degwithinthe correspondingstereogram.The bottomtwo panelsshowthe differencein disparityinducedby head rotation about the horizontal axis and stereogram-inducedhorizontalshear.

These numerical results lead to new interpretationsof acuity strongly degrades outside the foveal area (e.g. the results of earlier experiments. Fendick & Westheimer, 1983; Badcock & Schor, 1985; McKee et al., 1990a). Sensitivity of stereopsis Fendick & Westheimer(1983) found that stereoacuity In orderto interpretthe disparitiesof the bottompanels is about 1 arcmin at an eccentricity of 10 deg periph- of Figs 5 and 7, 8 and 9 (that is, the differencesbetween erally. According to Drasdo (1991) the stereoacuity (V) eye and head movement-induceddisparitiesand theore- found by Fendick & Westheimer (1983) can be extra- tically corresponding stereogram-induceddisparities) it polated with eccentricity < (in deg) by a linear function: is importantto realizethat sensitivityof humanstereopsis V= 0.1 + 0.12 L (in minarc). In this equation there is no varies with eccentricity.The relevantquestionis: Are the differencebetween horizontaland vertical eccentricities. disparities of the bottom panels small enough for the The idea of extrapolation is based on similar linear visual system not to perceive the difference between a extrapolation functions for several monocular domains disparity field induced by an eye or head movement and (vernier,LandoltC acuity, etc.) but has not been verified the disparity field induced by the above-mentioned for binocularvision. stereograms,correspondingto the eye or head movement. However, it should be realized that the experimental Not much is known about the sensitivity of peripheral data concerning stereoacuity have been obtained under binocular vision. Several studies showed that stereo- ideal and controlled laboratory conditions,usually with 3838 R. van EE and C. J. ERKELENS experienced subjects. The primary aim of such studies related to the well-knownhorizontal/verticalanisotropy* was to investigatethe limits of the visual system and not in depthperceptionwhich has been reportedby Rogers& the. sensitivity of binocular vision during natural Graham (1983). One might argue that the anisotropy is behaviour. Moreover, in view of the fact that in caused by the fact that the resemblance between stereoacuitymeasurementsdepthjudgments are always horizontal shear and forward rotation is less than the relative and invariant under whole-field transformations resemblance between horizontal scale and side-to-side due to eye and head movements, stereoacuity is not an rotation. Therefore, one could further argue that hori- indicator of the precision of all disparity information. zontalshearis lessambiguousthan horizontalscaleand is A useful experiment has been done by Schumer & consequentlyperceived better. Julesz (1984). They showed by using random-dot patterns that at a pedestral disparity of 0.5 deg, the Eye movements and the stability of depthperception threshold for detecting a corrugated plane from a flat Vergence movementsof the eyes lead to a translation plane was 33 minarc (theirFig. 12)almostirrespectiveof of the images over the retinae. A commonlyused way to the corrugationfrequenciesthey used. However, as far as induce translation of a stimulus over the retinae in an we know, the only paper on stereosensitivity for artificialway is to generate the images by a haploscope. relatively realistic stimuli is the study by McKee et al. In a haploscope the displays of the stimuli for the two (1990a). They showed that in comparison to lateral eyes can be independentlyrotated aboutthe centre of the judgments of distance, stereoscopicjudgments are not eyeball. However, except in the case of one unique precise. In addition, their study mentioned various combination of a fixation point and a location of the examples.and providesseveral referencesto demonstrate screens of the haploscopethere is in principle a conflict the insensitivityof stereopsis.They argue that in the first between oculomotor cues and disparity. Cyclovergence place stereopsis is for performing tasks at an arm’s leads to a rotation of the images over the retinae. distance or for breaking camouflage. Cyclovergencecan be mimicked by differentialrotation We still have to answer the question of whether the of the half-images of a stereogram. However, without disparitiesof the bottompanelsof Figs5,7,8 and 9 are so compensationalrotation of the eyes, differentialrotation small that the visual system cannot perceive the cannot mimic a real world stimulus which means that, difference between a disparity field induced by an eye again, there is a conflict between disparity cues and or head movement and the disparityfield induced by the oculomotorinformation.Thus,(cyclo)vergencegenerally above-mentioned stereograms corresponding to the eye cannotbe mimickedby a stereogramwithoutintroducing or head movement. Although stereoacuityis too precise this conflict.However, several reports mentioned in the to be an appropriate indicator for the sensitivity of Introductionshow that this conflictis not dominantwith stereopsis, we are more or less obliged to use it as an respect to depth perception. In situationswhere conflict- indicator, since no other indicators have been investi- ing eye muscle information is present, overall retinal gated in the literature on peripheral vision. We use displacements of a stimulus do not lead to changing Drasdo’s theoretical linear stereoacuity-thresholdfunc- binocularperceptionof depth in the case of large stimuli tion in order to interpretthe detectabilityof the disparity or they lead to only weak perceptionof depth in the case field of the bottompanels of Figs 5,7,8 and 9. Most (but of small stimuli. not all) of these disparity fields fall below Drasdo’s thresholds. A tolerance analysis which we conducted Head translation in the primary direction and the revealed that even for planes at a distance of only 40 cm stabili~ of depthperception (where the horizontaldisparityfieldis stronglycurved,as Erkelens & Collewijn(1985a) and Regan et al. (1986) can be inferred from Fig. 4) the disparity differences in showed that differential lateral translation of the entire most situationsfall below Drasdo’s(Drasdo, 1977,1991) dichoptically presented half-images does not elicit thresholds. perception of depth, even when the eyes pursue the Figure 8(e) shows the difference between horizontal lateral motion with a gain unequal to one (Erkelens & disparityinduced by head rotation about the vertical axis Collewijn, 1985b).They found that in the presence of a and horizontaldisparityinducedby a horizontallyscaled visual reference (which moved with a translational stereogram. Figure 9(e) shows the difference between velocity different from that of the stimulus) perception horizontal disparity induced by head rotation about the of depth was elicited vividly. horizontal axis and horizontal disparity induced by a In this report we show that disparity induced by a horizontally sheared stereogram. The latter difference translationof the head towards the stimuluscorresponds does not fall below stereoacuity thresholds for large to a disparity field caused by a lateral shift between the eccentricities. The results of Figs 8(e) and 9(e) can be entire half-images of a stereogram. On the basis of this insight we conclude that the experimental results of Erkelens & Collewijn (1985a) and Regan et al. (1986) *Thereport of Rogers& Graham(1983)showedthat subjectsare more implythatthe classof disparityinducedby translationsof sensitive to horizontal shear than to horizontal scale; perceived slants induced by horizontal shear demonstrate lower detection the head in the primary direction does not elicit depth thresholds and lower latencies than slant induced by horizontal perception. Of course we realize that cues other than scale. disparity are modifiedwhen the head is translated in the STABILITYOF BINOCULARDEP1’HPERCEPTION 3

TABLE 1. Relationship between depth perception and the disparity caused by an ego-movementor its corresponding stereogram

Ego-motion Stereogram Relation to depth perception

Change of fixation Haploscopicrotation Poor* Cyclovergence Differential rotation Poort

Head Forward translation Horizontal translation No$ Sidewardrotation Horizontalscale Poor~ Forward rotation Horizontalshear Poor~ Relevanteye and head movementsare presentedin the first column.The secondcolumnshowsthe transformationsbetween (the entire) stereogramhalf-imageswhich give rise to aboutthe same disparityfield as the movementslisted in column 1. Column3 gives psychophysicaldepthperceptionresults relating to experimentswhere disparityfieldscaused by the stereograms of column 2 are presented in isolation. *See text of the subsection “Eye movements and the stability of depth perception” and also the Introduction. By “haploscopicrotation” we mean a rotation of the displays about the centre of the eyeball. ~Howard& Zacher (1991), see also Introduction. $Erkelens & Collewijn (1985a). ~vsn Ee & Erkelens (1996a). primary direction. (When the distancebetween the head observationperiods lasting only a few seconds (see also and the stimulusis so large that the stimulusis effectively Gillam et al., 1988). Their experimental result implies at infinity, both retinal images are identical and thus that the class of disparitywhich is inducedby rotation of disparityhas vanished.During the translationtowardsthe the head elicits only poor depth perception. object, disparity developsbecause the retinal projections Jones & Lee (1981) reported on experimentsin which of the object become different and the retinal images human binocular performance and monocular perfor- become larger.) However, althoughchanging-sizestimu- mance were compared in a variety of visuomotortasks. lation and changing-disparity stimulation can both They found that stereopsis was not important in the produce a sensation of motion in depth (Regan & performance of visuomotor skills in three dimensions Beverley, 1979)and eye movements(Erkelens& Regan, when the subjects were free to move their heads. They 1986),they act largely independently.Both the motionin concluded that an important benefit of binocular frontal depth sensation and the eye movements produced by vision with movinghead is binocularconcordancerather changing-size stimulation can be cancelled by antag- than (changing)binocular disparity. In other words, the onisticchanging-disparitystimulation.In our analysiswe benefitof stereopsismay in fact be limitedto situationsin concentrate on the disparity domain. which the head is stationary(Jones & Lee, 1981).

Rotation of the head and the stabilityof depthperception The relationshipbetween ego-movement-induceddispar- Steinman & Collewijn (1980) measured eye move- ity and the stabili~ of depthperception ments of subjects while they actively rotated their head From the previous three subsectionswe conclude that about a vertical axis. They found that vergence velocity classesof disparitywhich can be inducedby eye and head errors of the order of 1 deglsec occurred. They also movements do not appear to be very relevant for obtained the impression that vision remained fused, stereopsis, at least if presented in isolation. In Table 1 stable and clear. In their 1985 study (Steinman et al., the resultsare summarized.We suggestthat the classesof 1985)they examined their impressionpsychophysically. disparitywhich can be inducedby ego-movementpoorly The study resulted in the conclusion that stereoacuity is elicit depth perceptionbecause they are ambiguous.The not disturbed by large fixation disparities or high classes of disparity which correspond to haploscopic vergence velocities.Patterson & Fox (1984) showed that rotation or to differential rotation of the entire half- the recognitionof a stereoscopicallypresentedLandoltC images of a stereogram are ambiguous because these was not impaired by active head rotations either. disparities could also be induced by vergence or Concerning tasks which require stereoscopic slant cyclovergence,respectively.Disparitycausedby a lateral perception, the stability of these tasks during head shift between the entire half-images of a stereogram is rotationsor in situationswhere neck muscle information ambiguous because instead of being caused by the and disparity information are decoupled, has still to be stimulus it could be caused by head translation in the deduced. primary direction. The classes of disparity associated van Ee & Erkelens (1996a) studied the temporal with horizontal scale and horizontal shear between the aspects of slant perception with large-field stimuli (no entire half-images of a stereogram are ambiguous visual reference was present).They found that horizontal because they could also be induced by head rotation. scale and horizontalshearbetween the entire half-images As explained in the Introduction, tasks based on of a stereogram elicit poor perception of depth for disparityprocessingcan be distinguishedinto relief tasks 3840 R. van EE and C. J. ERKELENS

and metrical tasks (Girding et al., 1995).Insensitivityof Temporalaspects stereopsis to disparity fields which result from eye and Taken together, we suggest that depth perception is head movements (for short observation periods) would invariant under eye and head movement-induced dis- mean that stereoscopicvision is, in principle,not able to parity. This formulationis probably too general because performmetricaltasks(for shortobservationperiods).On many authors have found that subjects are able to the other hand, relief characteristicsare preserved under perceive slant caused by whole-fieldtransformations(in eye and head movements.From the literatureit is known prolongedviewing). Generally, slant estimation induced that relief tasks can be done reliably in a couple of by whole-field transformations between the two half- milliseconds (e.g. Kumar & Glaser, 1993; Uttal et al., images of a stereogram depends on observation time. If 1994). The result of performing metrical tasks based on subjects are allowed to view the stereogram for more stereopsis alone is not veridical (Gogel, 1960; Foley, than, say, 10 see, then slant estimation is far more 1980; Gillam et al., 1984; Mitchison & McKee, 1990; veridicalthan if they view it for, say, 1 sec (Gillam Johnston, 1991;van Ee & Erkelens, 1996a).Visual tasks 1988; van Ee & Erkelens, 1996a). That subjects can that require a metric reconstruction of the three- perceivewhole-fieldslant after prolongedviewing could dimensionalvisual world are not very common. be caused by the integrationof either non-binocularcues The insensitivityof stereopsisto disparityfieldswhich or extra disparitysignals(for instancevertical disparity). result from eye and head movements(such as thosegiven Inspection of the stimulus by actively making eye in Table 1) means that stereopsisis insensitiveto global movements might also contribute to the enhancement zero and first order modifications between the half- of the slant perception over time (Enright, 1991). images of a stereogram. Stevens & Brookes (1988) Recently van Ee & Erkelens (1996b) have suggested reported that binocular 3D information is best acquired that Werner’s illusory depth contrast effect* (Werner, where a stereogram contains curvature features. Their 1938) may be explained by the idea that stereopsis is resultsaboutcurvaturefeaturesare related to stereograms relatively insensitiveto whole-fieldhorizontal scale and per se and not to the (retinal) disparity fields caused by shear.This insensitivity,in turn, resultsfrom the fact that these stereograms, since even a stereogram without a these transformationsinduce disparity fields similar to differencebetween the half-imagesgives rise to a curved those inducedby head rotationsas we have shown in this disparity field (see for instance Fig. 4). In this study we paper. The fact that slant estimation becomes more show why the zero and first order characteristicsof the veridical over time, makes their explanation consistent stereogram form a special class for which the visual with the fact thatthe illusoryslantof a stimuluscausedby system is relatively insensitive. Werner’s depth contrast effect decreases over time (e.g. In our view there are other relevant distinctions in Kumar & Glaser, 1993). additionto the distinctionof stereopsisinto metrical and Robot vision relief tasks. One of them is the distinctioninto short and Three-dimensionalimaging has various applications. long observationperiods (Gillam a 1988;van Ee & Erkelens, 1996a). A second one is the distinction into A possibleapplicationis the designof a binocularsystem conditions with and without a visual reference (e.g. (robot) which can produce information about places to Gogel, 1963;Shipley& Hyson, 1972;Gillamet al., 1984, which human beings cannot go or do not wish to go. However, in practice during movementsof the robot the 1988;Regan a 1986;Howard & Kaneko, 1994;van instability of camera images is such that disparity Ee & Erkelens, 1995). In the case of long observation processingunder practical circumstancesfails (Eklundh, periods there is not much of a difference between slant 1993). The idea that “ego-movement-induceddisparity” estimation (metrical task) with a visual reference and is irrelevantfor stereopsismay have interestingimplica- slant estimationwithout a visual reference. In both cases tions for the future of robotics. Shape perception by there is a large underestimation of slant. However, for means of two cameras could be greatly improved if the short observation periods, slant estimation without a types of disparities brought about by the robot’s own visual reference is generally extremely poor (van Ee & movements(which are classifiedin this report) could be Erkelens, 1996a).A third relevant distinctionof stereop- filtered out or ignored. sis is a distinction into small and large stimuli.A couple of reports (Rogers & Bradshaw, 1993, 1995;Howard & Kaneko, 1994) show that vertical disparities have a smaller influence on depth perception for small stimuli than for large stimuli. Oculomotorcues have a consider- We have calculated the binocular disparity field for a able influencefor small stimuli but hardly any for large wide range of possibleeye, head and stimuluspositions. stimuli (Rogers & Bradshaw, 1995; Regan et al., 1986). From the literature it is known that certain classes of disparity (such as whole-field horizontal lateral shift, differentialrotation,horizontalscale and horizontalshear *The slant of a surface is not only determinedby horizontal scale or between the half-images of the stereogram) induce shear between its own half-images, but also by the scale or shear between the half-imagesof a visual reference. A positive slant of a relatively poor perception of depth, at least if presented visual reference causes a negative slant of the object at hand (and in isolation. These classes of disparity turn out to be vice versa). similarto those caused by eye and head movements.Our STABILITYOF BINOCULARDEPTH PERCEPTION 3841 numerical calculations support the suggestion that Human gaze stability in the horizontal, vertical and torsional binocular 3D vision is based primarily on the classes of directionduringvoluntaryhead movements,evaluatedwith a three- disparity that are invariant under ego-movement. dimensionalscleral inductioncoil technique. V R 8 Fick, A. (1854). Die Bewegungen des menschlichen Augapfels. R Z r M 1 Foley, J. M. (1980). Binocular distance perception. P Alpern,M. (1962).Kinematicsof the eye. In Davson,H. (Ed.),Theeye R 4 (Vol. 3, pp. 15-27). New York: Academic Press. Friedman,R. B., Kaye,M. G. & Richards,W. (1978).Effect of vertical Backus,B. T., Banks,M. S. & Crowell,J. A. (1996).Eye positionand disparity upon stereoscopicdepth. V R 3 vertical disparity in stereoscopic surface perception. I Frisby, J. P., Buckley, D., Wishart, K. A., Porrill, J., G&ding, J. & OV S S Mayhew,J. E. W. (1995). Interaction of stereo and texture cuesp in Badcock, D. R. & Schor, C. M. (1985). Depth-increment detection the perception of three-dimensional steps. V R function for individual spatial channels. J O 1 S o 1 G&ding, J., Porrill, J., Mayhew, J. E. W. & Frisby, J. P. (1995). Berry, R. N. (1948). Quantitative relations amongvernier, real depth Stereopsis, vertical disparity and relief transformations. V and stereoscopic acuities. J E P R 7 7 Gillam, B., Chambers,D. & Rus$o,B. (1988).Postfusionallatency in Carpenter, R. H. S. (1988).M e 2nd edn. London: stereoscopic slant perception and the primitives of stereopsis. Pion. J E P H P Collett, T. S., Schwarz, U. & Sobel, E. C. (1991). The interaction of P 1 oculomotorcues and stimulus size in stereoscopic depth constancy. Gillam, B., Flagg, T. & Finlay, D. (1984). Evidence for disparity P 7 change as the primary stimulus for stereoscopic processing. Collewijn,H. & Erkelens, C. J. (1990).Binoculareye movementsand P & P 5 perception of depth. In Kowler, E. (Ed.), Eye m t Gogel,W. C. (1960).The perceptionof shapefrombinoculardisparity r v c p (pp. 216-261). Amsterdam: cues. J P 1 Elsevier. Gogel,W. C. (1963).The visualperceptionof size anddistance. V Collewijn,H., van der Steen,J. & van Rijn,L. J. (1991).Binoculareye R 1 movements and depth perception. In Gorea, A. (Ed.), R von Helmholtz, H. (1911). H t p t v t t a v r (pp. o Translated(1925)from the tliird Germaneditionof Southall, 165–183).Cambridge,U. K.: CambridgeUniversityPress. J. P. C. (Ed.),H P O York:The Cormack, R. & Fox, R. (1985). The computationof retinal disparity. Optical Society of America. P & P1 s Howard,I. P. (1982).H v o Toronto:John Wiley Cumming, B. G., Johnston, E. B. & Parker, A. J. (1991). Vertical & Sons. disparities and perception of three-dimensionalshape.N 3 Howard,L P. & Kaneko,H. (1994).Relative shear disparities and the 4 perception of surface inclination. V R 2 Donders, F. C. (1876). Versuch einer genetischen Erklarung der Howard, L P., Ohmi, M. & Sun, L. (1993). Cyclovergence: A Augenbewegmrgen,P A pp. 373421. (Cited in von comparisonof objective andpsychophysicalmeasurements.E Kries, 1911). m B R 3 Drasdo, N. (1977).The neural representationof visual space. N Howard, 1. P. & Zacher, J. E. (1991). Human cyclovergence as a 26 Drasdo, N. (1991). Neural substrates and threshold gradients of function of stimulus frequency and amplitude.E B peripheral vision. In Kulikowski,J. J., Walsh, V. & Murray, I. J. R 4 Johnston, E. B. (1991). Systematic distortions of shape from (Eds),L o (pp. 251-265). London:Macmillan Press. van Ee, R. & Erkelens, C. J. (1995). Binocular perception of slant stereopsis. V R 1 about oblique axes relative to a visual frame of reference. Jones, R. K. & Lee, D. N. (1981).Why two eyes are better than one: P 2 The two views of binocular vision. J E van Ee, R. & Erkelens, C. J. (1996a) Temporal aspects of binocular P H P P 3 slant perception. V R 4 Julesz, B. (1971). F c p Chicago: van Ee, R. & Erkelens,C. J. (1996b).Arrisotropyin Werner’sbinocrrlar Universityof ChicagoPress. depth contrast effect, V R 2 von Kries, J. (1911). Notes on $27 on the ocular movement. In Ekhrndh, J. O. (1993). Vergence control in machine vision. Oral H t o o Translated(1925)from presentation at the International Conference and Nato Advanced the third German edition of Southall,J. P. C. (Ed.), H Workshop on Binocular Stereopsis and Optic Flow, Toronto, P O (Vol. 3, pp. 127–154). New York: The Canada. Optical Society of America. Enright, J. T. (1990). Stereopsis, cyclotorsional “noise” and the Kumar,T. & Glaser, D. A. (1993).Temporalaspects of depthcontrast. apparent vertical. V R 1 V R 9 Enright, J. T. (1991). Exploring the third dimension with eye Listing, Unpublished.(Cited in Helmholtz, 1911,Vol 3,1854, p. 121). movements:Better than stereopsis.V R 1 Logvinenko,A. D. & Belopolskii,V. I. (1994).Convergenceas a cue Erkelens, C. J. & Collewijn, H. (1985a). Motion perception during for distance.P 2 dichoptic viewing of moving random-dot stereograms. V Mayhew,J. & L.onguet-Higgins,H. C. (1982).A computationalmodel R 5 of binoculardepth perception.N 2 3 Erkelens, C. J. & Collewijn, H. (1985b). Eye movements and McKee, S. P. (1983). The spatial requirements for fine stereoacuity. stereopsis during dichoptic viewing of moving random-dotstereo- V R 1 grams. V R 1 McKee, S. P., Levi, D. M. & Bowne, S. F. (1990a) The imprecision Erkelens, C. J. & Regan, D. (1986). Human ocular vergence stereopsis. V R 1 movements induced by changing size and disparity. J McKee, S. P., Welch, L., Taylor, D. G. & Bowne, S. F. (1990b) P 3 1 Findingthe commonbond:Stereoacuityand the other hyperacuities. Fendick, M. & Westheimer, G. (1983). Effects of practice and the V R 8 separation of test targets on foveal and peripheral stereoacuity. Mitchison, G. J. & McKee, S. P. (1990). Mechanismsunderlyingthe VisionR 1 anisotropy of stereoscopic tilt perception. V R Ferman,F., Collewijn,H., Jansen, T. C. & van den Berg,A. V. (1987). 1 3842 R. van EE and C. J. ERKELENS

Mitchison,G. J. & Westheimer, G. (1984).The perceptionof depth in Shipley, T. & Hyson, M. (1972). The stereoscopic sense of order-a simple figures. V R 1 classification of stereograms.A J O Mitchison, G. J. & Westheimer, G. (1990). Viewing geometry and A A A O 8 gradients of horizontal disparity. In Blakemore, C., (Ed.), V Steinman, R. M. & Collewijn, H. (1980). Binocular retinal image C e (pp. 302–309).Cambridge,U. K.: Cambridge motion during active head rotation. V R 4 UniversityPress. Steinman, R. M., Levinson,J. Z., Collewijn, H. & van der Steen, J. Nelson, J. (1977). The plasticity of correspondence: After-effects, (1985). Vision in the presence of known natural retinal image illusions and horopter shifts in depth perception. J motion.J O S A 2 T B 2 Stevens, K. A. & Brookes, A. (1988). Integrating stereopsis with Ogle, K. N. (1950). R b v Philadelphia: monocular interpretations of planar surfaces. V R Saunders. 3 Ogle, K. N. & Ellerbrock, V. J. (1946). Cyclofusional movements. Uttal, W. R., Davis, S. D. & Welke, C. (1994).Stereoscopicperception A O3 with brief exposures.P & P 5 p Patterson, R. & Fox, R. (1984). Stereopsis during continuous head Werner, H. (1938).Binoculardepth contrast and the conditionsof the motion. V R 2 binocularfield.A J P 4 Regan, D. & Beverley, K. I. (1979). Binocularand monocularstimuli Westheimer, G. (1979a) The spatial sense of the eye. I for motion in depth: Changing-disparityand changing-sizefeed the O V S 8 same motion-in-depthstage. V R 1 Westheimer, G. (1979b) Cooperative neural processes involved in Regan, D., Erkelens, C. J. & Collewijn, H. (1986). Necessary stereoscopic acuity.E B R 8 conditions for the perception of motion in depth. I Westheimer, G. & McKee, S. P. (1978). Stereoscopic acuity for OV S 2 moving retinal images.J O S A p van Rijn, L. J., van der Steen, J. & Collewijn,H. (1994).Instabilityof 6 ocular torsion during fixation: Cyclovergence is more stable than Westheimer, G. & McKee, S. P. (1979). What prior uniocular cycloversion. V R 1 processingis necessary for stereopsis.I O Rogers, B. J. & Bradshaw, M. F. (1993). Vertical disparities, V S 6 differential perspective and binocular stereopsis. N 3 Wheatstone,C. (1838).On some remarkable and hitherto unobserved 2 phenomenaof binocular vision. P T Rogers, B. J. & Bradshaw, M. F. (1995). Disparity scaling and the R S 1 3 perception of frontoparallel surfaces. P 1 Rogers, B. J. & Graham, M. E. (1983).Anisotropiesin the perception of three-dimensionalsurfaces. Science, 221, 1409–1411. Schor, R. H., Kearney, R. E. & Dieringer, N. (1988). Reflex stabilization of the head. In Peterson, B. W. & Richmond, F. J. A are indebted to Bert van den Berg and (Eds), C h m (pp. 141-166). Oxford: Oxford Hendrik-Jan van Veen for many inspiring discussions. We thank UniversityPress. Andrea van Doom and Jan Koenderinkfor the use of their computer Schumer, R. A. & Julesz, B. (1984). Binocular disparity modulation and various helpful remarks. We thank S. M. McNab for linguistic sensitivity to disparities offset from the plane of fixation. V advice. This work was supportedby the NetherlandsOrganizationfor R 5 Scientific Research (NWO),grant no. 805-33-702.

,., , !