SENSORIMOTOR PROCESSING OF VERTICAL DISPARITY

ROBERT SCOTT ALLISON

A thesis submitted to the Faculty of Graduate Studies in partial fulfilment of the requirements for the degree of

Doctor of Philosophy

Graduate Programme in Biology York University Toronto, Ontario

March 1998 Natioral Library Bibliothèque nationale 191 of Canada du Canada Acquisitions and Acquisitions et Bibliographie Services services bibliographiques 395 Wellington Street 395, rue Wellington OttawaON K1AON4 Ottawa ON K1A ON4 Canada Canada

The author has granted a non- L'auteur a accordé une licence non exclusive licence allowing the exclusive permettant à la National Library of Canada to Bibliothèque nationale du Canada de reproduce, loan, distribute or seIl reproduire, prêter, distribuer ou copies of ths thesis in microform, vendre des copies de cette thèse sous paper or electronic formats. la forme de microfiche/^, de reproduction sur papier ou sur format électronique.

The author retains ownership of the L'auteur conserve la propriété du copyright in this thesis. Neither the droit d'auteur qui protège cette thèse. thesis nor substantial extracts fiom it Ni la thèse ni des extraits substantiels may be printed or otheIUrise de celle-ci ne doivent être imprimés reproduced without the author' s ou autrement reproduits sans son permission. autorisation. SENSORIMOTOR PROCESSING OF VERTICAL DZSPARITY

by Robert S. Allison

a dissertation subrnitted to the Faculty of Graduate Studies of York University in partial fulfillment of the requirements for the degree of

DOCTOR OF PHILOSOPHY

Permission has been granted to the LIBRARY OF YORK UNIVERSITY to lend or sel1 copies of this dissertation. to the NATIONAL LIBRARY OF CANADA to microfilm this dissertation and to tend or seil copies of the film, and to UNIVERSITY MICROFILMS to publish an abstract of this dissertation.

The author reserves other publication rights. and neither the dissertation nor extensive extracts from it may be printed or otherwise reproduced without the author's written permission. Several recent computational theories of binocular vision have proposed that vertical disparities play a critical role in calibrating and scaling stereoscopic depth. In these theories. vertical disparities specify "viewing system" pararneters such as the viewing distance and the relative alignment and magnification of the images in the two eyes. 1hypothesised that vertical disparities are temporally integrated and therefore processed more slowly than horizontal disparities. This averaging would allow robust estimation of slowly changing pararneters. The temporal characteristics of vertical and horizontal disparity processing in human vision were investigated in three sets of experiments. First. scleral search coils were used to measure the gain and phase of vertical vergence as a function of stimulus disparity and temporal frequency. The half images in a large. textured. stereoscopic display were oscillated in counter-phase to evoke vertical vergence. At low frequencies and amplitudes. the gain of vertical vergence was near one and phase lag was srnall. Gain declined and phase lag increased with increasing stimulus frequency. The dynamics of vertical vergence tailor it ro compensate for disturbances in vertical eye alignment. Second, the temporal characteristics of the slant or inclination percepts evoked by linear transformations of disparity (shear or scale dispari ties) were studied. Stimuli contained combinations of horizontal and vertical size or shear disparities that were introduced stepwise or modulated sinusoidally. Subjects matched perceived slant or inclination with that of a visual cornparison surface. There were no clear differences in the effects of exposure time or temporal frequency on slant or inclination percepts induced by horizontal disparity and those induced by vertical disparity. Considerable individual differences were found and several subjects experienced slant reversal. particularly with oscillating stimuli. Perceived slant induced by oscillations of dilation disparity was in the direction of the vertical component. Percepts of slant and inclination also depend on monocular gradients of texture and form. In the third set of experiments, 1 measured the temporal dependencies of perspective-disparity cue integration. Observers viewed a dichoptic pattern undergoing smooth temporal modulations or step changes in simulated slant and inclination specified by gradients of disparity and/or by perspective. Perspective and disparity were cornbined in four ways: disparity alone, perspective aione, perspective concordant with disparity, or perspective in conflict with disparity. Large pattemed displays providing strong linear perspective were seen according to perspective. With a static irregular texture. conflict was typically resolved according to disparity. This dominance of disparity built over tirne with initial depth typically seen according to perspective. With moving displays. however. perspective dominated the percept of surface slant or inclination. The hypothesis that pattems of vertical disparities are processed more slowly than equivalent pattems of horizontal disparities was only partiy confirmed. Vertical vergence is more sluggish than horizontal vergence. However, perceived slant and inclination showed sirnilar temporal limitations for gradients of both horizontal and vertical disparity. These limitations were related to disparity-perspective conflict. The temporal characteristics of the resolution of disparity-perspective conflict indicates that kinetic perspective is especially salient for human vision. This conclusion has important implications for experiments studying temporal factors in stereopsis in the presence of unchanging monocular cues. 1 would like to thank my supervisor Dr. Ian Howard for his suppon and encouragement. 1 greatly appreciate having the opportunity to work under his expert supervision. It would also like to thank the members of rny supervisory cornmittee, Drs. Ono, Harris and Steel for their suppon and constructive advice throughout the course of this research. Dr. Brian Rogers was a frequent visitor to our lab throughout this period and his expertise and collaboration in these experiments were greatly appreciated. 1 appreciate the generous suppon of NSERC and CRESTECH and am especially grateful to Dr. Ron Kruk and CAE Ltd. for their partnership in a CRESTECH Co-operative Research Award.

1 would like to thank Xueping Fang, Holly Bridge and Jim Zacher for their assistance with these experiments and for their many helpful comments. 1 am also grateful to Drs. Masayuki Sato, Hirohiko Kaneko. Byron Pierce, Masahiro Ishii. Christine Portfors, Mn. Toni Howard and Mr. Rob Gray for many discussions of my work and their related experiments.

1 would also like to express rny appreciation to the subjects who volunteered their time and to Drs. Tirnney, Davey and Wilcox for taking the time to examine my thesis. 1 would also like to thank Heather. Eric. Alan, Stephen. Gang. and Dalia for making the lab an enjoyable and intellectually stimulating place to work. 1 would like to thank the students and faculty associated with the York Centre for Vision Research as well as Teresa Manini and Jeff Laurence for their friendship and assistance. I would like especially like to thank my wife Jovie for her unconditional suppon and love that made it possible for me to complete this study. ABSTRACT ...... IV

ACKNOWLEDGEMENTS ...... VI

TABLE OF CONTENTS ...... VII

LIST OF FIGURES ...... X

LIST OF TABLES ...... XII

LIST OF SYMBOLS AND ABBREVIATIONS ...... XJII

INTRODUCTION ...... 1 1 .1 Binocular Correspondence ...... I 1 -2 Patterns and Foms of Disparity ...... 7 1 2.1 Absolute and Zero-Order Disparities ...... 12 1 .2.2 First-Order Disparity Patterns ...... 13 1.2.2.1 Slant about a Vertical Axis ...... 16 1.2.2.2 Inclination about a Horizontai Axis ...... 23 1.2.3 Higher-order Dispariries ...... 29 1.3 Binocular Vision and Vertical Dispaity ...... 29 1.3.1 Fusion and Diplopia ...... 30 1.3.2 Binocular Visual Direction ...... 32 1.3.3 Dichoptic Stimulation by Dissimilar Images ...... 31 1 .3.4 Vergence ...... 35 1.3.5 Computational Theories of Vertical Disparity Processing in S tereopsis ...... 40 1.4 Objectives and Hypotheses ...... 50

2 DYNAMIC PROCESSING OF ZERO-ORDERVERTICAL DISPARITY: VERTICAL VERGENCE ...... 52 2.1 Introduction ...... 52 2.2 Methods ...... 54

vii 2.2.2 Stimulus ...... 57 2.2.7 Objective Eye Movement Recording ...... 61 2.2.3.1 The Magnetic Search Coi1 Technique ...... 61 2.2.3.2 Eye Movement Recording ...... 62 2.2.4 Procedure and Data Analysis...... 63 2.3 Results ...... 65 2.4 Discussion ...... 78

3 SPATIO-TEMPORAL PROCESSING OF GRADIENTS OF VERTICAL DISPARITY: SLANT AND INCLINATION PERCEPTION ...... 97 3.1 Introduction ...... 97 3.2 General Methods ...... 99 3.2.1 Image Presentation ...... 99 3.2.2 Stimulus...... 101 3.2.3 Measures of Surface Slant and Inclination ...... 106 3.3 Experiment 1 : Time Course of Slant and Inclination Perception ...... 107 3.3.1 Method ...... 107 3.3.2 Results ...... 3.3.2.1 Inclination ...... 3.3.2.2 Slant ...... 3.4 Experiment 2: Slant Thresholds as a Function of Modulation Durat 3.4.1 Methods ...... 117 3.4.2 Results ...... 118 3.5 Experiment 3: Efficiency of Slant and Inclination Perception as a Function of Temporal Frequency ...... 122 3.5.1 Method ...... 122 3.5.2 Results ...... 123 3.6 Discussion ...... 130 3.6.1 Processing of Horizontal venus Vertical Disparity Gradients ...... 130 3.6.2 Time Course of Slant and Inclination Percept ...... 131 3.6.3 Effects of Temporal Frequency ...... 133

... Vlll 3.6.4 CueConflict ...... 134

4 THE DYNAMIC INFLUENCE OF PERSPECTIVE AND "SLANT REVERSALS" 137 4.1 Methods ...... 137 4.2 Results ...... 140 4.2.1 Static Presentation ...... 140 . . 4.2.2 Oscillating Presentation...... 142 4.3 Discussion ...... 148

5 GENERAL DISCUSSION ...... 156

6 REFERENCES ...... 167

APPENDIX A - SYSTEMS THEORY AND VERGENCE ...... 183 LISTOF FIGURES

FIGURE 1. 1 : THE HOROPTER AND DISPARATE POINTS ...... 5

FIGURE 1 -2: EYE MOVEMENT AND RETINAL CO-ORDINATE SYSTEMS ...... ,,. ... 6 FIGURE 1-3: PATTERNS OF DISPARITY IN A FRONTAL SURFACE ...... 11 FIGURE 1-4: DISPARITY IN SLANTED AND INCLINED SURFACES ...... 15 FIGURE 1-5: SLANT FROM HORIZONTAL SlZE DISPARITY ...... 1 FIGURE 1-6: SLANT GEOMETRY OF ECCENTRIC SURFACES ...... 22 FIGURE 1-7: INCLINATION GEOMETRY AND DISPARITi ...... 27 FIGURE 1-8: OCULOMOTOR EXPLANATION OF THE IIWUCED-SHEAR EFFECT ...... 28 FIGURE 1-9: VERTICAL SIZE RATIOS (VSRS) ...... 48 FIGURE 1 .10: DEFORMATION DISPARITY ...... 49 FIGURE 2 .1 : WHEATSTONE STEREOSCOPE ...... -56 FIGURE 2-2: STIMULUS FOR VERTICAL VERGENCE ...... 59 FIGURE 2-3: SCHEMATIC DIAGRAM OF THE EXPERIMENTAL APPARATUS ...... 60 FIGURE 2-4: SAMPLE OF VERTICAL VERGENCE RECORDINGS ...... 71 FIGURE 2-5: MAGNITUDE OF VERTICAL VERGENCE AS A FUNCTION OF STIMULUS FREQUENCY ...... 72 FIGURE 2-6: VERTICAL VERGENCE GAIN AND PHASE ...... 73 FIGURE 2-7 : NON-LINEAR VERTICAL VERGENCE ...... 74 FIGURE 2-8:VERGENCE GAIN AS FUNCTION OF STIMULUS VELOCITY ...... 75 FIGURE 2-9: VERGENCE VELOCITY RESPONSE ...... 76 RGURE 2- 10: DIPLOPIA MEASLREMENTS ...... 77 FIGURE 2- 1 1 : CYCLO- AND HORIZONTAL VERGENCE GAIN ...... 91 FIGURE 2- 12: VERGENCE ERROR DUE TO PHASE ERROR ...... 92 FIGURE 2- 13: EFFECTS OF AREA ON DIFFERENT TYPES OF VERGENCE EYE MOVEMENTS ... 93 RGURE 2- 14: DEPENDENCE OF GAIN ON STIMULUS VELOCITY IN HORIZONTAL. VERTICAL AND CYCLOVERGENCE ...... 94 FIGURE 2- 15: COMPARISON OF VERGENCE AND VERSION FREQUENCY RESPONSES ...... 95 FIGURE 3- 1 : STEREOSCOPE APPARATUS ...... IO0 FIGURE 3-2: DISPLAY USED FOR LNDUCING SLANT AND INCLINATION ...... IO3 FIGURE 3-3: SHEAR AND SIZE DISPARITY COMBINATIONS ...... 104 FIGURE 3-4: PREDICTIONS ...... 105 FIGURE 3-5: INCLINATION MATCHES AS A FUNCTTON OF DISPARITY ...... 111 FIGURE 3-6: NOMI~ALISEDINCLII\TATION MATCHES ...... 112 FIGURE 3-7: SLANT MATCHES AS A FtTNCTION OF DISPARITY ...... ,...... 1 15 FIGURE 3-8: NORMALISED SLANT MATCHES ...... 1 16 FIGURE 3-9: SLANT THRESHOLDS ...... 120 FIGURE 3- 1 0: INCLINATION THRESHOLDS...... 1 21 FIGURE 3- 1 1 : INCLINATION MATCHES AS A FUNCTION OF TEMPORAL FREQUENCY ...... 126 FIGURE 3- 1 2: SLANT MATCHES AS FUNCTION OF TEMPORAL FREQUENCY ...... 127 FIGURE 3- 13: MATCHED SLANT AND INCLINATION IN OBSERVER PRONE TO SLANT REVERSALS ...... 128 FIGURE 3- 1 4: MATCHED SLANT AND INCLINATION IN PRESENCE OF REFERENCE ...... 129 FIGURE 4-1 : DISPLAYS USED TO INDUCE SLANT AND INCLINATION ...... 139 FIGURE 4-2: PERSPECTIVE AND STEREO INTERACTIONS .INCLINATION JUDGEMENTS ..... 144 FIGURE 4-31 PERSPECTIVE AND STEREO INTERACTIONS .SLANT JUDGEMENTS ...... 145 TABLE 4- 1: INCLUVATION ESTIMATES ...... 146 TABLE 4-2: SLANT ESI'IMATES ...... 147 LISTOF SYMBOLSAND ABBREVIATIONS

binocular disparity shear disparity (radians) two-dimensional three-dimensional interocular distance cpd- cycles per degree D- egocentric distance d- relative depth HSR- horizontal size ratio Hz- Hertz (cycles per second) i- angle of inclination LE- Left Eye M- magnification expressed as a ratio RE- Right Eye VSR- vertical size ratio Human visual perception relies on a variety of cues to reconstruct a three- dimensional world from the two-dimensional images projected on the retinae. These depth cues include monocular cues such as motion paralla, perspective, interposition and shading. In frontal eyed anirnals such as man, depth cm also arise from binocular depth cues such as vergence and stereopsis. Stereoscopic depth perception (stereopsis) is possible because of the separation of the eyes in the head. This separation results in different vantage points for the two eyes; hence, the perspective projections of a scene differ between the eyes. This difference between the images in the eyes is referred to as binocular parallm or binocular disparity. Since the eyes are separated laterally in the head. the principal disparities are in the horizontal dimension. Most treatments of stereopsis have been concemed with horizontal disparities since these code relative depth (Wheatstone 1838). Until relatively recently. vertical disparity has received little attention. Vertical disparity at a point is not unarnbiguously correlated with relative depth. However, the lateral separation of the eyes does produce vertical disparities which play a role in binocular vision. 1.1 Binocular Correspondence

in stereopsis, the perceived depth of objects depends on the disparity between the images in the two eyes. In order to describe these disparities precisely. we need to define corresponding points. Images of objects that fall on corresponding points in the two retinae have zero binocular disparity: other pairs of images are disparate. Since the retina is two dimensional, one may describe any disparity in terms of independent horizontal and vertical components. Corresponding points can be defined geometrically in ternis of retinal position on idealised sphencal retinae (see Tyler 199 1 and Howard and Rogers 1995 for review). The centres of the two foveae are taken as corresponding points and other points in the two retinae correspond if they have the same vertical and horizontal distances from the foveae. For any angle of convergence of the visual axes, there is a locus of positions in space where a point of light will stimulate corresponding points in the retinae. This is the theoretical or geometrical point horopter. For convergence in the horizon plane, the geometrical horopter consists of a circle through the fixation point and the nodal points of the two eyes, the Vieth-Müller circle, and a vertical line through the intersection of this circle and the median plane of the head (Figure 1-1). For oblique fixation, the horopter is a twisted cubic curve (Solornons 1975). For fixation at infinity, the horopter curve becomes a plane of infinite extent located at an infinite distance. AI1 objects that lie on the horopter project images to corresponding points; dl objects off the horopter project to disparate points. Objects that lie nearer than the horopter project with crossed disparity - since the visual lines of the two eyes through the point cross in front of the horopter circle. Objects that lie further than the horopter have uncrossed disparity. As well as this point horopter, which is the locus of object positions with zero vertical and horizontal disparity, line horopters cm be defined for which either vertical or horizontal disparity alone is zero (Clement 1987). Of special interest for this thesis is the horizontal line horopter for zero vertical disparity which, in the case of symmetric convergence. consists of the median plane and the horizon plane (Figure 1 - 1 ). A11 objects not on the horizontal line horopter have vertical disparity. Corresponding retinal points can also be defined physiologically or psychophysically. Corresponding points defined geometrically do not necessarily result in physiological or psychophysical correspondence. For example, the empirically determined vertical horopter deviates from a vertical line and is inclined top away. This has been interpreted as a relative outward tilt of the corresponding vertical meridia of the two eyes (Helmholtz 1909). Thus, the psychophysically determined corresponding points are sheared relarive to the geometricall y specified points. Helmholtz ( 1909) proposed that this is an adaptive deviation from geometric correspondence that tends to tilt the horopter into the ground plane. where most objects of interest lie. Each retinal image is a two-dimensional projection of the three-dimensional scene. Thus, a disparity is a difference between two. 2-D positions. A CO-ordinatesystem 2 is required to speciQ the size and direction of disparity. To descnbe the kinematics and dynamics of eye movements we must also define a frame of reference for eye rotations. Spherical CO-ordinatesystems are t;ic most convenient for both retinal positions and eye movements. The two systems described in the literature that specify eye kinematics in terms of orthogonal horizontal, vertical and torsional components are the Helmholtz md Fick systems (Howard 1982). Note that the frame of reference for describing eye movements is completely arbitrary admany alternative systems could be used. In Helmholtz's system for eye position, vertical eye movements are made about a head fixed horizontal ais and then horizontal movements are made ahout the resulting vertical axis (see Figure 1-2). Helmholtz' system results in a set of CO-ordinateswhich project ont0 a sphere as liiies of latitude for horizontal eye position and lines of longitude for vertical eye position. In Fick's system the order of rotation is reversed resulting in lines of longitude for honzontal eye position and lines of latitude for vertical eye position (see Figure 1-2). Similar systems can be used to define the CO-ordinatesof points projected on the retina or points in the optic arrays of each eye (Howard and Rogers 1995).Corresponding to the Helmholtz and Fick systems for eye movements. we can specify the latitudinal-azimutMongitudina1-elevationand longitudinal-azimuthI latitudioal-elevation retinal CO-ordinatesystems respectively. Each of these systems has advantages in studying binocular vision. The longitudinal-azimuthnatitudinal-elevatio representation of the Fick system has the advantage that changes in vergence do not effect the magnitudes of horizontal disparities in the optic arrays (Howard and Rogers 1995). Helmholtz's system has the advantage that with the visual axes aligned, epipolar planes through the n~dalpoints and any point in space correspond to longitudinal lines of elevation (i.e. planes of regard are represented in the system). Thus witn the eyes otherwise aligned, and in the absence of optical distortion, vertical disparity results solely from vertical image misalignment and is constant over the field. Helmholtz ( 1909) noted that in his system horizontal azirnuih (and hence vergence) position is independent of elevation. This has the advantage over Fick's system of not requiring calculation of trigonometric equations to determine 3 azimuth and elevation with shifis in origin of the primary plane of regard but rather a simple addition to the elevation angle. For modest angles of eye movement, the interactions between horizontal and vertical eye movements are small. In the expenments in this thesis, vertical vergence was driven in isolation while fixating a central target. Vertical eye position was obtained directly from the signal from the vertical search-coil. Thus, a Helmholtz system was implicitly used although 1 did not provide the negligible cosine correction to the horizontal movements. The vertical movements were so small that any error in horizontal eye position would be negligible (less than 0.1%). Figure 1-1: The horopter and disparate points a) The theoretical point horopter for symmetric fixation at a point, F, consists of a circle through the nodal points of both eyes and the fixation point, called the Vieth-Müller circle, and a vertical line through the fixation point. Al1 points off the horopter are disparate and project to non-corresponding points in the two eyes. b) Horizontal line horopter. Al1 points on the median plane of the head and the horizon plane have zero vertical disparity. Adapted fiom Howard and Rogers (1995).

Vieth-Müller Circle Figure 1-2: Eye movement and retinal CO-ordinatesystems Representatioii of two cornmon CO-ordinatesystems for measuring retinal position and eye movements, shown as parallel projections ont0 the page. a) In the longitudinal- azimuth, latitudinal-elevation system, the azimuth of retinal image is measured by the dihedral angle between the zero azimuth plane and a vertical plane through the point and the nodal point of the eye. Elevation is rneasured in the vertical plane containing the point. b) The latitudinal-azimuth, longitudinal-elevation system uses the reverse order. Elevation is measured by the dihedral angle between the zero elevation plane and a horizontal plane through the point. Azimuth is measured in the horizontal plane containing the point. c) In Fick's system for representing eye movements, the eye is conceptually rotated first about a vertical axis first (longitude), then about the horizontal axis that moves with the eye in a gimballed fashion (latitude). Fick's sysrem results in a longitudinal-azimuth, latitudinal-elevation representation of eye movements. d) In Helmholtz' system the order is reversed with elevation rotation proceeding azimuth rotation. This results in a laritudinal-azimuth, longitudinal- elevation representation. In both eye movement systems, the horizontal and vertical movements can be followed by a rotation about the resulting optic axis (torsion).

longitude T azimuth

latitude 1.2 Patterns and Forms of Disparity

A vergence correctable disparity is one that cm be created and nulled by vergence eye movements. For example. vertical misalignment of the visual axes results in a constant vertical-disparity across the entire field, which can be nulled by vertical vergence. Also, an overall rotational disparity can be created or nulled by opposite rotations of the eyes about the visual axes, or cyclovergence. An essential disparity cannot be nulled by disjunctive eye movements. An example is a vertical shear disparity, which results in a horizontal gradient of vertical disparity. Cyclovergence cannot nul1 this gradient; it simply transfoms it into a horizontal disparity gradient. An essentid size disparity exists when the images in the two eyes differ in size as in aniseikonia. A size disparity cm be an isotropic dilation or confined to a particular axis (meridional). The above disparity pattems are zero or first-order transformations of disparities in that they are constant shifts or gradients of vertical or horizontal disparities. A zero- order disparity is a simple linear disparity. A first-order disparity is differentiable once and corresponds to a constant gradient of disparity, or a constant change in disparity for a unit change in azimuth or elevation. Higher-order disparity pattems contain second or higher-order spatial derivatives of disparity or local discontinuities in disparity. The binocular subtense of an object is the angle subtended by the nodal points of the two eyes at the object. Foley (1980) referred to binocular subtense as absolute binocular parallax. Absolute binocular disparity is the retinal disparity of an object with respect to the horopter. When convergence is at infinity, the absolute retinal disparity of an object equals its binocular subtense. Relative binocular disparity is the disparity of one object relative to that of a second object. While the human visual system is relatively insensitive to absolute disparity. it is exquisitely sensitive to relative disparities. Most observers cm discriminate the direction of depth of a foveal stimulus, relative to a zero disparity fixation point, when the relative disparity is as small as severai seconds of arc (Howard 19 19). Depth between two objects (d) is related to their relative disparity (q), distance between the eyes (a) and viewing distance (D) according to the following formula

assurning that the targets are on or near the midline and depth is small compared to viewing distance (Howard and Rogers 1995). Thus. relative depth can be extracted from disparity up to a scaling factor related to distance. Unscaled depth (bas-relief) can be recovered directly from disparity but metric depth requires normalisation by am'. Unscaled depth information is sufficient for tasks such as breaking camouflage and perhaps object identification. but some visudly guided motor tasks require metric depth estimates (Glennerster et al. 1996). Use of relative disparity insulates depth perception against instability of gaze. St. Cyr and Fender (1969) reported that vergence oscillations of up to 1 O minutes of arc occur during steady fixation. Steinman et ai. (1982) reported that. during head movements. the instability of disjunctive and ccnjunctive eye position increases. They reported that vergence variations on the order of a degree were typical. Though the methodology and the magnitude of the vergence variability found by Steinman have bern criticised (Duwaer 1982 and Stark 1983), psychophysical measures of the motion of afterimages relative to a fixation target suggest vergence instability of at least several minutes of arc (Duwaer 1982). These results suggest that the conjugacy of eye movements is not tightly controlled. For many binocular tasks we hold Our heads relatively steady. With the head fixed on a bite board, Steinman et al. (1982) reported a standard deviation of vergence of 3 minutes of arc. Even this level of instability is much higher than stereoacuity and thus the visual system appears to be relatively immune to oculomotor instability. Westheimer and McKee (1978) measured the effects of retinal slip caused by lateral motion of a display. They found that stereoacuity was not affected by retinal image slip of up to ZO/s. Steinman et al. (1982) found no effect of head oscillations on stereoacuity or on the ability to fuse a random-dot stereogram even when these movements caused large vergence errors (on the order of minutes of arc). Calibration of depth from absolute disparity requires an estimate of egocentric distance and precise control or registration of eye position. Mathematicdly, relative disparity between the images of two objects is equivalent to the difference in lateral separation between them on the two retinae. The images of two objects move together on the retina as the eye moves and so maintain their relative separations. Thus. relative disparity is maintained when the eyes move. Relative disparity provides an estimate of the depth relation between two objects relatively independent of eye position (Westheimer 1979). An estimate of egocentnc distance is still required for the 1/D2 scaling to recover metric depth. In addition, control of eye position is required to ensure stable bifoveation of the target and thus high acuity. An absolute disparity (disparity pedestal) can impair the ability to discriminate small relative disparities between two points. Westheimer ( 1979) found that a dispari ty pedestal, imposed on the images of two objects relative to the fixation point, caused a decrease in acuity for detecting depth differences between the objects (see also Schurner and Julesz 1984).Collewijn et a!. (1991) played down any such role of absolute disparity. They suggested that the disparity pedestal effect is due to a large relative disparity between the test points and the fixation point rather than a large absolute standing disparity per se. This explanation seems unlikely since disparity sensitive cells in primate visual cortex tend to be either narrowly tuned near zero disparity or broadly tunrd to a wide range of either crossed or uncrossed disparity (Poggio and Fischer 1977). These cells are also sensitive to absolute disparity as reflected by difference in the displacement (or possibly differences in phase) of their monocular receptive fields. Any proposed mechanism based on labelled line detecton or opponent mechanisms would result in best stereoacuity near zero absolute disparity. On the other hand. a role for these disparity detectors in stereopsis remains to be established and their response to relative disparity is net known. It has not been detemiined whether a relative disparity pedestal rather than an absolute disparity pedestal results in decreased stereoacuity. 9 Relative horizontal disparity specifies relative depth at a given distance. Relative vertical disparity does not specify depth unambiguously. However, pattems of horizontal disparity combined with patterns of vertical disparity provide information for localising stimuli both laterally and in depth (Ogle 1964). For example, consider viewing a flat surface parallel to the frontal plane. Points on the surface to the right of the midline are closer to the right eye than to the left eye. Therefore, the right side of the surface subtends a larger angle in the right eye than in the left eye. Convenely, the left side of the surface subtends a larger angle in the left eye than in the right. In general, dl objects that are located away from the midline project retinal images that differ binocularly in size. In extended surfaces, the differential perspective becomes more apparent because it is systematically related to distance and eccentricity (Ogle 1964). Figure 1-3a shows the pattern of disparities that a frontal surface projects on flat converged retins. Horizontal disparity increases with horizontal eccentricity and vertical disparity increases with both horizontal and vertical eccentncity. These disparities are enhanced for near viewing and diminish to zero as distance increases to infinity. A flat retina is a convenient mode1 for studying pattems of disparity because it is easy to represent the resultant pattems on a flat sheet of paper and because typically one displays stereoscopic images on flat tangent screens. Howard and Rogers' (1995) attempt to represent the disparities on a more realistic spherical retina is shown in Figure 1-3b. For a givcn angle of elevation and azimuth, the horizontal and vertical disparities on a spherical retina decrease to zero as the distance to the frontal surface increases. Thus. the pattern of vertical and horizontal disparity depends on viewing distance, a fact that will be expanded on below. Figure 1-3: Patterns of disparity in a frontal surface

a) Pattern of dis~ar&ieson converged planar retinae. The left side shows the views of an image on the two retinae. The right side shows the pattern of disparity vecton which indicate the ciifference in position of elements in the two eyes. b) Pattern of disparities on spherical retinae. Adapted fiom Howard and Rogers (1995).

1 Left-eye image Right-eye image a 1.2.1 Absolute and Zero-Order Disparities

The egocentric distance of an object is its distance from the observer. Relative depth is the difference in distance between objects or the extent that one object is nearer or further than another. Egocentric distance perception is of interest in this thesis since it has been proposed that patterns of vertical disparity are used to judge egocentric distance (see Section 1.3.5). The egocentnc distance of a target could also be denved from the absolute binocular disparity and the convergence angle. The binocular subtense specifies the vergence posture of the eyes required to foveate an object binocularly and its tangent is inversely related to the egocentnc distance of the object (if the angle subtended by the centres of ocular rotation is used). Binocular subtense defines the target vergence angle for vergence eye movements. The idea that convergence angle could allow an observer to estimate egocentric distance dates from at least Kepler ( 1604 cited in Bishop 1989). Wheatstone ( 1852) tested this hypothesis in his early stereoscope expenments and concluded, somewhat tentatively. that increased convergence does result in the percept of decreased distance. In experimental tests of this idea. perceived distance is typically measured by verbal magnitude estimation or with a pointing or matching task. Foley (1980) has provided a critical review of the literature. In most studies requiring subjects to estimate egocentric distance using convergence information. convergence angle was not measured but it was assumed that subjects converged on the target. Foley ( 1980) concludes that egocentric distance estimates are unreliable but that there is some evidence for a vergence related distance signai. If proper controls are made for spurious distance cues such as accommodation, size and luminance the typical finding is that subjects can correctly judge the relative distance order of sequentially presented targets using convergence. but magnitude estimates are very crude (Momson and Whiteside 1984, Foley 1978. Komoda and Ono 1974). Typically, the range of perceived distances is compressed relative to the actual range of distances used. Gogel (1965) noted that near distances tend to be overestimated and far distances underestimated, an "equidistance" tendency attributed to setting a stimulus to a default distance (of about 2 m) in the absence of specific depth information. It should be noted that the tangent of the vergence angle changes inversely with distance and thus sensitivity of the vergence cue to distance decreases rapidly as distance increases. Although its direct role in depth perception appears to be minor, absolute retinal disparity plays an important role in driving vergence eye movements. Absolute retinal disparity serves as an error signal to drive vergence since it indicates the difference between target vergence (or binocular subtense) and the vergence state of the eyes. A constant absolute retinal disparity over the binocular visual field is a zero-order retinal disparity. A zero-order retinal disparity implies a vergence correctable misalignment of the eyes and drives horizontal vergence or vertical vergence appropriately. Absolute retinal disparity of a point can selectively drive horizontal vergence (Rashbass 198 1 ). Ideally. vergence acts to reduce the absolute retinal disparity of the target to zero.

1.2.2 First-Order Disparity Patterns

It has been argued that an overall disparity in the binocular field (a zero-order disparity) drives vergence but does not give rise to a strong percept of depth or egocentric distance. The next simplest type of disparity pattern in an extended surface is a first-order disparity or a constant gradient of disparity. With CO-planarflat retinae. a surface inclined about a horizontal mis gives nse to a gradient of horizontal disparity in the vertical direction (see Figure 1-4).Similarly, a surface slanted about a vertical axis gives rise to a gradient of horizontal disparity in the horizontal direction (see Figure 1-4). Tilt in depth about other axes gives rise to gradients of horizon ta1 disparity dong oblique directions intermediate between horizontal and vertical. The magnitude of these gradients is a function of surface slant or inclination and the viewing distance. The gradient of horizontal disparity is an approximately linear function of slant angle for small slant angles and varies inversely with the viewing distance. For converged planar retinae at any surface distance and for spherical retinae for near surfaces, these patterns are superirnposed upon the disparities present for a frontal surface. Thus. to a first approximation, gradients of horizontal disparity encode the tilt of a surface in depih. This approximation is qualified by the 1/Dscaling and by the deviation of the horopter from the fronto-parallel plane. The deviation of the horopter from the frontal plane results in the pattern of disparities shown in Figure 1-3. This pattern of disparities is inherently contained in the stereograrns presented in these experirnents. The projected disparity patterns for slanted or inclined surfaces correspond to a first-order transformation of disparity supenmposed on the frontal pattern. Figure 1-4: Disparity in slanted and inclined surfaces

Patterns of disparity of the projections of slanted (a) and inclined (b) surfaces onto coplanar flat retinae. Adapted from Howard and Rogers (1995).

Ci- - C 1.2.2.1 Sfant about a Vertical Axis A horizontal gradient of horizontal disparity is equivalent to a differential expansion dong the horizontid meridians or, in other words, a horizontal size disparity. Ogle ( 1964) has discussed the geometncally predicted slant induced in frontal-paral le surface when the image in one eye is magnified honzontally. He showed that the tangent of slant angle (v)varies with magnification (M expressed as a size ratio) in proportion to distance (D) and inversely with interpupillary distance (a) as follows:

The geornetry of this optical situation is illustrated in Figure 1-5. From Figure 1-5 it can be seen that a uniform rnagnification of one eye's image predicts distortions of visual space due to changes in the visual direction of points on the stereoscopically slanted surface. If one image is magnified and the other is symmetrically minified. the mean visual directions of points remain the sarne and the cyclopean angular extent of the surface is maintained. However, size-distance invariance still predicts large changes in apparent overall shape and texture size. Size-distance scaiing theoretically constrains the shape-slant relationship for the given retinal pattern. In a real slanted surface. dichoptic horizontal size differences exist for: 1) the overall image size. 2) the distance between images of pairs of elements and 3) for the horizontal size of each image feature. Horizontal magnification cm also cause orientation disparity in oblique line elements. A horizontal expansion increases the horizontal component of the line, orienting it more in the direction of the horizontal axis. Overall horizontal size disparities can be detected as point disparities providing depth relative to the fixation point. The percept of surface slant cm then arise from integrating over the point disparities. Dichoptic differences in overall length of relatively long line elements cm give rise to the percept of depth without any micro-texture in between (e.g. Wilde I %O). The accumulation point dispari ties across a display during scanning eye movements may be important for perception of surface slant from conventional point disparity (Howard and Rogers 1995). 16 It has been proposed that the local slant of a surface cm be detected directly frorn differences in the local size and scale of texture elements rather than from point disparities per SC.Blakemore (1970) proposed that surface slant cm be inferred from differences in spatial frequency in the two eyes. As with overall size, texture element size and density difTer in the two eyes. This is equivalent to a difference in mean spatial frequency in the two eyes. For penodic vertical gratings slanted about a vertical mis. differences exist in both mean grating line width and spacing. As proposed thus far, dichoptic spatial frequency differences are similar to any other size disparity and could be detected by registering point disparities. Blakemore (1970) suggested that special mechanisms exist that extract slant from spatial frequency differences independent of point disparity. Tyler and Sutter (1979) called this type of disparity dif-frequency disparity and called the proposed detectors dif-freq detectors. Blakemore's reasoning follows from the proposition made by Campbell and Robson (1968) that the visual system has channels tuned to various spatial frequencies which it uses as filter banks to perform a Fourier analysis of the visual scene. While a physiological bais for this type of operation remains to be discovered, a dif-frequency detector would be a binocular spatial frequency channel with different preferred spatial frequencies in the t wo eyes. Evidence for this proposa1 cornes from attempts to show that slant is perceived from dif-freq disparity even when the point disparity of surface elements does not correspond to a slanted surface. It is difficult to dissociate point disparity frorn spatial frequency in these expenments. The strongest test was made by Tyler and Sutter ( 1979) who presented uncorrelated dynamic textured patterns to the two eyes that differed in spatial frequency. This created a weak impression of depth that they argued was evidence for a pure dif-frequency mechanism. Since the textures were random point disparities they could not be properly matched and any spurious match would not give a consistent slant estimate. Although this does argue against a mechanism based solely on point disparities, these results could also be explained by dif-size mechanisrns sensitive to local differences in texture scale (dif-size disparity). Evidence against dif-frequency mechanisms comes from the finding that monocular spatial frequency adaptation after- effects do not transfer into perceived slant (Sloane and Blake 1987). Wilson !1976) criticised the idea that the visual system uses dif-frequency mechanisms. He argued that a real slanted surface presents a texture gradient to the two eyes. This makes dichoptic spatial frequency differences difficult to detect since even the image of a slanted grating contains a wide range of spatial frequencies. Wilson (1976) noted that consistent differences in local element size exist in a natural slanted surface. Slant could be detected by detecting local differences in size of texture elements (dif-size) rather than detecting spatial frequency differences. The concept of using dif-size disparity is equivalent to saying that the visual system uses relative disparity on a local scale. 1 have noted above that a surface slanted in depth gives rise to a gradient of disparity. It has been proposed that the visual system uses the gradient of retinal disparity directly in calculating surface slant. Rogers and Bradshaw

( 1993) suggested the use of the closely related horizontal size ratios (HSR) in surface perception. The dichoptic ratio of horizontal extent of elements in the images varies as a function of slant angle. The HSR is a first-order disparity and is equal to the disparity gradient minus one. It is conceivable that the HSR is a physiologically plausible means of implernenting a disparity gradient measure to estimate slant directly. As with disparity gradients. the size of the HSR varies inversely with viewing distance rather than the square of viewing distance. The closely relared dif-size measure is simply the difference between two zero-order disparity measures and as such is a zero-order disparity measure. Of course in the dif-size mechanism, consideration of the angular separation of elements would eventually have to occur in order to infer slant from depth estimates. None of the primitives discussed thus far can be used to directly estimate the slant of a surface since point disparities, HSRs, dif-freq and dif-size information are affected by distance and eccentricity as well as by slant angle. The zero- and first-order measures Vary inversely with the square of viewing distance and with the viewing distance respectively. The dependency on eccentricity aises because the horopter circle (the locus of zero disparity) is not an equidistant circle from the cyclopean eye but falls increasing 18 within this circle as eccentricity increases (see Figure 1-6). The cyclopean eye is a hypothetical single eye. located midway between the eyes, from wliich judgements of visual direction are made (see 1.3.2). A cyclopean line is a visual line referred to the cyclopean eye. Tangent lines to a circle, which passes through an eccentric point and is centred on the cyclopean eye, correspond to a line nomal to the cyclopean line through the point. Tangent lines to the horopter at a point correspond to the local slant of a zero- disparity surface at that point. Note that this slant will not correspond to a head-centric frontal surface since the horopter curves away from the frontal plane. It will not correspond to the normal to the cyclopean line either since the equidistant and horopter circles through an eccentnc point have different radii of curvature. It is easy to show that the slant of normal to the cyclopean line. relative to the frontal plane, is equal to the horizontal eccentncity of the cyclopean line. Sirnilarly. it can be shown that the slant of the tangent to the horopter, relative to the frontal plane. is equal to twice this angle. This theoretical argument was outlined by Ogle (1964) and revisited by Gillam and Lawergren

( 1983). Thus, interpretation of surface slant from horizontal disparity requires estimation of surface distance and eccentricity. 1 have argued in section 1.2.1 that vergence information alone does not generally give reliable estimates of egocentric distance. Sirnilarly, considerable doubt surrounds our ability to reliably judge surface eccentricity from oculomotor signals alone. Amigo ( 1967) and Ebenholtz and Paap (1973) have demonsuated that subjects can use oculomotor afference or efference signals to partially compensate for the rotation of the horopter in asymmetric convergence. However. in many situations subjects make large perceptual errors in tasks requiring judgements of perceived gaze direction (for review see Foley 199 1 ). To determine eccentricity from retinal information, various computational solutions that rely on vertical disparities have been proposed and will be discussed in section 1.3.5. Vertically oriented gradients of vertical disparity (vertical size disparity) also result in the percept of surface slant. Ogle (1938) refened to this as the induced-size effect since a real slanted surface does not have any overall vertical size differences 19 between the images in the two eyes. The induced-sire effect is approximately equal to the geometric effect for small magnifications but tends to saturxe at approximately 6% magnification (Ogle 1938). It has been proposed that the induced-size effect is a result of mechanisms designsd to deal with distortions of image size due to aniseikonia and eccentric viewing conditions. 1 will discuss the induced-size effect and its theoretical implications in deril in section 1 -3.5. Figure 1-5: Slant from horizontal size disparïty

Apparent slant fiom a stereo pair where the image of the right eye is horizontally magnified with respect to the left (dotted versus solid rectangle). ïhe disparity predicts a percept of surface slant away fiom the magnified eye as well as changes in image size and shape. Construction Iines are shown dashed. Adapted from Ogle (1964).

Apparent Surface

i Figure 1-6:Slant geometry of eccentric surfaces

Relative slant cf the frontal plane, the normal to the cyclopean line, and the tangent to the horopter (which is centred at H) at an eccenûic point P is shown. A frontal plane goes through P at a distance D. The normal to the cyclopean line to a point is equivalent to the tangent to the equidistant circle centred on the hypothetical cyclopean eye (C). C is located midway between the eyes, and is the centre from which judgements of visual direction are made. Judgements of surface slant relative to the normal to the cyclopean line are known as optical slant judgements.

Triangles DCP and PCS are similar. PCS and DPS are also similar. Therefore DCP and DPS are similar. Thus angles LDCP and LDPS are equal. LDPS is the slant of the normal to the cyclopean line to P relative to the frontal plane. This equal to the angle of eccentricity of P, that is angle 0.

LDHP is twice LDCP or 28. LDPH is the complement of this angle and LKPD is in tum its complement. Thus. the slant of the horopter to the frontal plane, LKPD. equals LDHP. This is twice the angle of eccentricity of P, that is 20. Tangent to Nonnal to the Horopter Cyclopean K..... Line to i?...... --.

Equidistant Circle Rirough P 1.2.2.2 Inclination about a Horizontal Axis Surface inclination about a horizontal axis also results in a gradient of honzontal disparity, this time in a vertical direction. This is equivalent to a horizontal shear distortion of the disparity field. This shearing was referred to by Ogle (1964) as vertical declination since it results in changes in orientation of vertical lines but not in horizontal lines. In this thesis, the honzontal shear tenninology will be used since it is more descriptive for large pattemed stimuli. The geometric relationship between the angle of inclination (i) of a line in the median plane of the head and the horizmal shear angle (0 in radians) for modest inclinations is:

where a is the interocular distance and D is the viewing distance. The geometry of the optical situation is shown in Figure 1-7. if the half images are syrnmetrically sheared in opposite directions. we get no change in perceived visual direction of texture elements. However, large size constancy effects are predicted if size-distance invariance holds. This is because the depth of the textured elements changes but their cyclopean angular extent does not. Note these changes predict increases in texture density for the near portion of the surface and decreases in texture density for the far portion of the surfaces. an effect opposite to the texture gradient presented by a real inclined surface. Horizontal shearing occurs for I ) the overall image outline. 2) for al1 points relative to each other and 3) for any texture element or line element in the display. Differences in relative disparity on an inclined surface can be detected as relative depth which increases with vertical separation and is zero for horizontally separated points. The percept of surface inclination can arise from integrating over these point disparities. The accumulation of these conventional point disparities across the display during scanning eye movements may be important for perception of surface inclination from conventional point disparity (Howard and Rogers 1995). It has been proposed that surface inclination may be detected directly by specialised mechanisms sensitive to orientation differences in the two eyes. Many cells in 23 primary visual cortex are known to be sensitive to the orientation of line elements presented within their receptive fields, firing rnaximally for a single preferred onentation (Hubel and Wiesel 1962). Blakemore (1972) found that some binocular cells in the cat differed in preferred orientation of inputs from the two eyes. These cells could be considered orientation disparity detecton since they are sensitive to differences in orientation. As such, they are tuned to particular orientations of line elements in depth. Hamy et al. (1980) found some cells in V2 of the alert monkey that are sensitive to the inclination of a surface in depth. Orientation disparity can also arise from torsional misalignment of the eyes. In this case both horizontal and vertical line elements in one eye are rotated with respect to those in the other eye. Orientation disparity detectors tuned to orientations near vertical are required for detecting surface inclination. Orientation disparity in horizontal line elernents does not arise from surface inclination but from cyclorotary misalignment of the eyes. Thus orientation disparity detectors tuned to horizontal line orientation can be used to signal cyclodisparity and drive compensatory cyclovergence. The cells discovered by Blakemore were sensitive to both orientation dispariry and position disparity and hence their role as orientation detectors is not clear (Howard and Rogers 1995). A psychophysical investigation into the use of orientation disparity in surface inclination perception is complicated by the fact that onentation and position are not separable features. Inclination can be seen in random-dot stereograms where there are no line elements to show differential orientation disparity in the two eyes. However. in most such displays there is non-zero power in Fourier components in al1 directions. Ninio (1985) and Devalois et al. (1975) have provided evidence for the use of orientation disparity but interpretation of their results is complicated by an inability to completely dissociate position and orientation disparities. Instead of attempting to dissociate position and orientation disparity. several investigators have manipulated the displays to Vary the arnount of orientation infomation present. For inclination, 1have already argued that the largest dichoptic orientation changes are for vertical lines, with less orientation disparity in oblique lines and none in 24 horizontal lines. Cagnello and Rogers (1993) found that the threshold for detection of the inclination of a circular patch of intersecting horizontal and vertical lines was about the same as for a grid patch composed of 45' lines. This appears to argue against the use of orientation disparity since the orientation disparity in the 45O lines is less than in the vertical lines. However, there are twice as many 4S0 lines as vertical lines in an equivalent size patch and this may explain their result. For surface slant, thresholds were significantly lower for the 45" Iine grids as predicted from orientation disparity considerations. Rogers and Graham (1983) and others have noted an anisotropy between inclination and slant thresholds for random-dot patterns with lower thresholds for inclination. Cagnello and Rogers (1993) found that this anisotropy disappeared when orientation disparity was equated by using a 45" grid patch. This result suggests that orientation disparity may piay a role in perception of surface tilt in depth and may underlie the anisotropy found by Rogers and Graham (1983). Similarly. the studies by Gillam et al. ( 1984, 1988) demonstrated that liitency for detection of surface slant was greater for slanted surfaces than for inclined surfaces but that this anisotropy was reduced when the slanted surface had orientation disparity.

On the other hand. supra-threshold data obtained by Mitchison and McKee ( 1990) and Gillam and Ryan ( 1992) demonstrate that a slant-inclination anisotropy still exists. even for stimuli cornposed of 45" lines. It has been proposed rhat cue conflict with perspective (which specifies a frontal surface) may be stronger for slant than inclination and rnay explain the supra-threshold results. In support of this claim. Mitchison and McKee found that addition of a square outline to the stereogram reduced the perceived depth more for slant than for inclination. Gillam and Ryan (1992) also found that perspective cue conflict had a stronger effect for slant than inclination. In keeping with this hypothesis, Cagnello and Rogers (1993) found that the effect of orientation disparity as reflected in differences between thresholds for 45' versus 0-90" grids had an effect for small displays but had little effect for larger displays where the perspective information is more salient. With srna11 displays near threshold, perspective information would be minimal. On the other hand. for displays smaller than 2O, thresholds decreased with 25 display size. Sincc orientation disparity is constant and position dispüri ty increases with display size. this suggcsts a possible role for positional disparity rather than oientation disparity for stimuli smaller than approximately 2". In keeping with an orientation detector mechanism. slant detection thresholds were constant with increasing display size for larger displays. Interpretations of results of manipulations of display size are complicated by the tact that disparity thresholds increase with retinal eccentricity (Siderov and Hanverth 1995)

Horizontal1y oriented gradients of vertical disparity ( vertical shear disparity) also result in the percrpr of surface inclination i Howard and Knneko 1 9% ). This is the shear disparity analogue of O_ole's ( 1938 induced-size effect and will be referred to as the induced-shear effm. Howard and Kaneko < 1994) proposed that the induced-shear effect is designed to deal u ith cyclodisparity. Vertical shear disparity is the stimulus for cyclovergence (Han ard 199 1 >.It has been proposed that cyzlover_oencraccounts for the induced-shear effect by transfonning vertical disparity gradients into horizontal disparity gradients (Figure 1-8 1. This is probably not the whole stor!. Cyclovergence gain is never unity but the inducd-shsar effect equals the geometric-shwr effect for modest shear angles. Funhermoit.. ihr pcrcsptual response does not shon the sarne idiosyncratic asymmetries as the oculomotor response (Howard and Kaneko 1994. 1 will discuss this effect and its theoi-eticül implications in greatrr detail in section 1.3.5. The induced effects rely on the ability of the brain to process local variations in vertical disparity since the shear and mapnif'ictition transformations result in gradients of w-tical disparity (although mschani~rii~bassd on comparing mean hemifirld averagrs of vertical disparity are also conceivabls i. Figure 1-7: Inclination geometry and disparity

An inclined surface projects images with horizontal shear disparity on the retinae. The figure shows an example illustrating this fact and shows the projection of a central vertical line onto the retinae. In the figure, D, a, i and 0 are defined as in the main text and x is the distance below eye level at which the inclined plane intersects the plane where the projections of the line on the two retinae intersect. From the diagram it is apparent that tan(%) = gxand also x = xi.Combining and rearranging terms leads to the relation t9 = .mm)/D for small values of 0. Figure adapted from Howard and Rogers (1995).

lnclined Planel

extension of retinal projection of the line

extension of a "vertical" line on the Figure 1-8: Oculomotor explanation of the induced-shear effect a) Vertical shear disparity pattern projected on the upright aligned retinae (crosses). b) With vertical shear disparity, the eyes will tend to cycloverge to nuil the vertical shear disparity. This tends to align the horizontal meridia of the eyes with the relative orientation of horizontal elements in the image. c) Resulting orientation of the image with respect to the retinae (part b re-drawn with the retinae aligned). It can be seen that cyclovergence converts a vertical shear dispanty into an equivalent retinal horizontal shear disparity of opposite sign. Adapted fiom Rogers (1992).

1 l Y 1 Retinal Mendia Higher-order changes in disparity can also be defined. A disparity curvature or second-order disparity defines, to a first approximation, a surface that is paraboiically curved in depth. As 1have noted above, point disparities on a surface scale with the inverse square of the viewing distance. Since disparity gradients are a first-order spatial derivative they scale with the inverse of distance. Disparity curvature is a second-order spatial denvative and is independent of viewing distance. Rogers and Cagne110 (1989) have suggested that this invariant could be used for surface curvature perception independent of distance. Mitchison and Westheimer (1990) cIaimed that the second spatial denvative of disparity is the basic primitive used in stereopsis and proposed special mechanisms, saiience detectors to detect it. When the stimulus contains step changes in disparity between two sets of elements or disparity between a single disparate element and a textured surface the situation is more complicated. This corresponds to a discontinuity of disparity, which is not differentiable, hence an infinite set of higher-order disparity tenns is required to represent the transition. The visual system seems to be especially sensitive to these discontinuities of disparity, and step changes in depth are well perceived (Miichison and McKee 1990. Brookes and Stevens 1989, Gillam et al. 1988a). 1.3 Binocular Vision and Vertical Disparity

In his review, Tyler ( 1983) subdivided the perceptual consequences of binocular parallax into five categories of phenornena. These categories are: sensory fusion and diplopia, binocular visual direction, dichoptic stimulation (by dissimilar images), vergence. and stereopsis. 1will use this categorisation as a convenient stariing point for review of these concepts in binocular vision and to discuss the behavioural consequences of vertical disparity. Following this, 1will outline the theoretical and psychophysical support for the use of vertical disparity in the human visual system. More general treatment of these topics can be found in the recent text by Howard and Rogers (1995) or the collection of reviews edited by Regan (1991). 1.3.1 Fusion and Diplopia

With thc concept of retinal correspondence, we can introduce the concepts of fusion and diplopia. Single vision of objects seen with both eyes is one of the most compelling aspects of binocular vision. In fact, the problem of how the visual world was seen single with two eyes was the principle preoccupation of students of binocular vision until Wheatstone's discovery of stereopsis in 1838 (Ono 199 1). In the discussion of correspondence earlier. it was impiicit that stimulation of corresponding points stimulated identical areas of the two retinae. A basic assumption of fusion theory is that stimulation of identical areas of the two retinae with identical images results in a percept of a single object. The two half images are said to be fused into a single object. Images that have a modest disparity can be also fused into the percept of a single object. The range of disparity for which an object appears fùsed is known as Panum's fusion area (after Panum 1858). Panum's area defines a region in space centred on the horopter, called the fusion horopter, within which objects will be seen in single vision. Still larger disparities produce diplopia. that is the perceptual separation of the two half images so that two objects are seen side by side. The transition region between diplopia and fusion results in more complex percepts such as perceptual thickening and rivalry that make the determination of fusional ranges complicated. The spatio-temporal properties of sensory fusion differ for horizontal and vertical disparities. Most reports have looked at the fusional range for static disparities in small targets at the fovea. Typically, these investigators have reported a smdler fusional area for vertical disparity than for horizontal disparity. Panum (1858 cited in Mitchell 1966a) reported that the fusional range was approximately k15 to t 25 minutes of arc for vertical lines and less for horizontal (vertical disparity). Schor et al. (1984) reported that the fusional area for a thin bar (2 minutes of arc) was approximately 15 minutes of arc for horizontal disparity and approximately half that for vertical disparity. In a larger display consisting of a 20' textured square, Ogle and Prangen (1953) found that vertical disparity (specifically fixation disparity) seldom exceeds 5-6 minutes of arc before diplopia occurs. which is much iess than for horizontal fixation disparity (Ogle 1964). Ogle and Prangen

( 1953) found that the size of Panum's area increased less rapidly with eccentricity for vertical disparity than for horizontal disparity. The snidy by Mitchell ( 1966b) was one of the few not to report a difference in fusional range between the vertical and horizontal disparity dimensions. His display consisted of a test spot located directly in front of or behind the fixation point. This would result in an infinite disparity gradient (change in disparity over change in direction) for horizontal disparity. Burt and Julesz (1980) found that subjects were unable to fuse laterdly separated targets if the disparity gradient between them was greater than one. This disparity gradient limit would make fusion of the horizontal disparities in Mitchell's experiment difficult. This may explain why the horizontal fusionai range was similar to the vertical disparity range (where a disparity gradient limit is not known and may not exist). The effects of spatial and temporal scale on fusion differ for horizontal and vertical disparities. Schor and Tyler (198 1) studied the temporal characteristics of sensory fusion using sinusoidal temporal modulations of disparity in line stimuli. They found that the size of Panum's area fell-off considerably with increasing temporal frequency for horizontal disparity but remained relatively constant for vertical disparities in three observers. Schor and Tyler ( 1981) dso looked for possible spatio-temporal interactions in the fusion of horizontal and vertical dispaities. For horizontal but not vertical disparity. Panum's area was increased for low temporal frequencies and at low spatial frequency and Panum's area was elliptical. Panum's area decreased in the horizontal dimension with increasing temporal frequency such that it becarne circular at high temporal frequency. The effect of temporal frequency was greatest at low spatial frequency and approached zero at 2.0 cpd. The diplopia limits for both vertical and horizontal disparity decreased with increasing spatial frequency (Tyler 1973). However, for vertical disparity, the spatial frequency dependence was much [ess pronounced. For horizontal disparity. Panurn's area decreased from more than 2" at 0.03 cpd to approximately 2 minutes of arc at 5 cpd. The fusional range for vertical disparity fell from about 20 minutes of arc to 6 minutes of arc over the same spatial frequency range. The disparity in a fused image cm be gradually increased beyond the disparity level at which an initially diplopic image can be fused (Fender and Julesz 1967). This hysteresis effect may differ for horizontal and vertical disparity. Fender and Julesz (1967 see also Hyson et al. 1983, Piantanida 1986. Erkelens 1988) found that for a retinally stabilised, large. random-dot stereogram, horizontal disparity in the image could be slowly increased to more than 2O before the depth percept was lost. Dispaiity then had to be reduced to a small value before the depth could be recaptured. For vertical disparity, a much smaller effect was found and disparity could be increased to only 20 arcminutes before the depth percept was Iost. Fender and Julesz argued that this illustrated a large hysteresis effect for horizontal disparity fusion in random-dot stereograms. This was based on their assumption that depth in a random-dot stereogram could not be seen if the images were not fused and thus depth and fusion limits were identicai. This is contrary to the fact that depth is appreciated in diplopic images of lines and dots (e-g. Richards 197 1 ) and no convincing evidence exists that this does not hold for random-dot stereograrns (see Duwaer 1982 and Piantanida 1986 for evidence against Fender and Julesz's assumption). The possible interaction between vertical and horizontal disparity in fusion has not been investigated. Vertical disparity beyond Panum's fusional area would of course preclude fusion of horizontally disparate stimuli. Ogle (1955) and Mitchell ( 1970) have demonstrated that presence of vertical disparity beyond Panum's range does not destroy stereopsis even though fusion is lost. It is not known if the presence of a fusible vertical disparity limits the fusional range for horizontal disparity.

1.3.2 Binocular Visual Direction

A topic intirnately related to the concept of corresponding points is the idea of binocular visual direction. Points with the same visual direction are said to have zero disparity. This is the basis of the determination of the space horopter by the nonius alignment method, which equates visual directions of points in the two eyes (Ames et al. 1932). The rules of visual direction were first set out in modem terms by Hering ( 1879) in the last century although Alhazen (ca 1ûûû AD cited in Howard 1995) and Wells (1792 cited in Ono 198 1) had produced earlier accounts. These rules have been reviewed and qualified by Ono ( 1979). People find it easy to make judgements of lateral position relative to the median plane of their head or body. Judgements of direction relative to the head or body are made relative to head-cenuic or body-centric frames of reference and the principal axes of these reference frames are adopted as noms in judgements of visual direction (Howard 1982). Since the eyes can move in the head. judgements of visual direction with respect to the head require an estimate of oculocentric visual direction and an estimate of eye position. In monocular viewing, ail objects falling dong a given visual line have a common unique head-centric visual direction and appear to point towards a point in the median plan of the head common for al1 visual lines, the visual egocentre. In binocular vision, objects have different directions with respect to the nodal points of the two eyes. Refemng visual direction to the egocentre, a "cyclopean eye" located approximately midway between the eyes, makes judgements of binocular visual direction (or cyclopean direction) of fused objects possible. Thus in binocular viewing a11 objects on visual lines of either eye appear to point to this cyclopean eye (Ono 199 1 ). The direction of these lines is specified from the observations that points on the horopter appear to lie in the same direction in the two eyes and that points lying on the same visuai line appear collinear. This leads to the law of cyclopean projection that defines the cyclopean direction of points in space. Points on a visual line of an eye appear to lie in the direction of a line between the cyclopean eye and the point where the visual line intersects the horopter. Thus, points on the horopter will be seen singly and in the correct direction while al1 other points will be mislocalisea. The above description describes the visual direction of corresponding points on the horopter and monocular or diplopic points. What about fused but disparate objects? One might expect that these objects have a visual direction corresponding to rnean direction of the two half images (Hering 1879). Ono et 33 al. (1977) showed that the apparent direction of a fused object is close to the mean visual direction of each of the half images. How does the concept of visuai direction relate to vertical disparity? Vertical visual direction differs from horizontal visual direction in that the right, left and cyclopean eye are not separated verticaily. In normal vision, with the eyes aligned, whether vertical disparity is present depends on the CO-ordinatesystem used represent the optic arrays. With a longitudinal system chosen for the vertical dimension and with the horizontal axis dong the interocular mis, the projection of al1 points lie on epipohr lines and no vertical disparity exists for the images of any object. Thus, in this system the venical visual direction of dl objects is the same in both eyes. Vertical disparity exists in other CO-ordinatesystems or with ocular misalignment. Furthermore. no matter which system is chosen we can introduce vertical disparity in a stereoscope. Sheedy and Fry (1979) have studied the apparent visual direction of fused images with vertical disparity. When studying visual direction with vertical disparity, the complication of the percept of depth is removed. Like Ono et al. (1977). these investigators found that the visual direction of the fused image is near the mean of the visuai directions of each half image. In this thesis. 1do not deal with the question of vertical cyclopean direction experimentally.

1.3.3 Dichoptic Stimulation by Dissirnilar Images

Sensory fusion and stereopsis result from similar images falling on corresponding or nearly corresponding points on the two retinae. Dichoptic stimulation of corresponding retinal areas by dissimilar stimuli gives nse to different binocular perceptual phenomena such as binocular rivdry, binocular lustre, binocular colour mixture, surnmation and masking. Binocular rivalry is probably the most commonly studied and robust of these phenomena. When the images falling on corresponding regions of the two retinae are so dissimilar that they fail to fuse, the percept is not a superposition of the two half images. What is perceived is a dominance of one half image and suppression of the other. The suppression tends to altemate between the two eyes over time and space. The degree to which a pattern predorninates is a function of a variety of monocular stimulus characteristics (Howard and Rogers 1995). Note that unlike many of the other binocular phenornenon 1 discuss in this introduction, these percepts require that the dichoptic stimuli stimulate corresponding rather than disparate points. Thus, vertical disparity has no role in relation to nvalry or binocular lustre except in detemining which stimulus features stimulate corresponding areas.

1.3.4 Vergence

Vergence is a disjunctive movernent of the eyes where the two eyes rnove in opposite directions. The eyes have three rotational degrees of freedorn and disjunctive movements cm be classified as horizontal, vertical or cyclovergence movements. This section will review basic concepts in horizontal vergence and introduce vertical vergence. More in depth discussion of vertical vergence cm be found in Chapter 2.

Maddox ( 1893) classified four different types of horizontal vergence movements: tonic. accommodative, proximal and disparity vergence. Tonic vergence refers to the tendency for the eyes to drift into the vergence state they assume in the dark. It is believed to be partly due to tonic efferent innervation since it differs from the anatomic position of rest assumed in deep sleep or anaesthesia. Tonic vergence is manifest in the presence of visual stimuli as the phenornena of phoria, fixation disparity and strabismus (see von Noorden 1990 for more details). Most people have a phoria, which is a rnisalignment of the eyes that occurs when the eyes are dissociated, that is presented with images that are not fusible. For example, when one eye is covered when viewing a fixation stimulus. the covered eye may tum either in or out - conditions known as esophoria and exophoria respectively. Under normal viewing conditions, the phoria results in a disparate image and disparity vergence acts to eliminate the disparity. This process is not perfect and typically a small vergence error remains, which is known as a fixation disparity. Ogle (1964) argued that the fixation disparity is a result of the eyes pulling towards the tonic vergence state and found a correlation between phoria and fixation disparity magnitude and direction. Some individuals cannot dign their eyes even in the presence of fusible stimuli. a condition known as strabismus. The concept of tonic vergence also applies to vertical and cyclovergence. Vertical phona is a common phenornenon in which one eye becomes elevated relative to the other when dissociated. Normally, vertical vergence would compensate for disparity resulting from the phoria in the presence of fusible stimuli (except in strabismus). As in the horizontal case, any residuai error results in a fixation disparity related to the phona (Ogle 1964). Viewing of a near target results in a synkinetic response of convergence, pupillary constriction and increase in dioptric power of the lens, or accommodation (Semmelow 198 1 ). The linkage between these responses is reflected in interactions between accommodation and vergence. Accommodative vergence (Semmelow and Hung 1983) refers to the tendency of the eyes to converge when accommodation is increased and can be observed wi th changes in accommodation under monocular viewing conditions. Vertical disparity is not unambiguousl y related to distance and thus accommodative vertical vergence is not required. An advantage of using vertical vergence to study oculomotor and sensory processes in binocular fusion is that the synergistic relationship between convergence. accommodation and pupil constriction (the near triad) is not present for vertical vergence movements. Although no longer covariates of the vergence response. these factors may still have an influence and must be controlled. Since vertical vergence is not influenced by accommodative state, disparity vergence can be studied without contamination from accommodation. Proximal vergence (see Hokoda and Ciuffreda 1983 for a review) refers to horizontal vergence resuiting from the "nearness" of an object. Maddox considered it a "psychic" phenomenon due to knowledge of the object's distance (Judge 199 1 ). As we have seen. a near object requires an increase in horizontal convergence in order to bring the images in the two eyes into correspondence. Changes in stimulus size (Erkelens and Regan 1986) and changes in simulated depth from perspective (Enright 1987) have been shown to elicit horizontal vergence responses. Changes in apparent depth arise from these pictorial cues. Thus, these effects could be considered proximal vergence. Voluntary vergence is proximal vergence where the subject converges or diverges his eyes, typically 36 by imagining a near or far stimulus. Voluntary vergence is cornrnonly used by trained observers in free fusion of stereograrns. Proximal vergence in the sense of changing vergence for a target perceived to be near or far is not relevant for vertical or cyclovergence since vertical and cyclo-disparity are not uniquely related to viewing distance. However, Balliet and Nakayarna (1978) have shown that subjects cm leam to control cyclovergence in the absence of visual stimuli but only after extensive biofeedback training. Presumably subjects could aiso be trained to control vertical vergence but this has not been reported. The lack of a voluntary component for vertical vergence is a potentiai advantage for studying disparity vergence. Disparity vergence is the type of vergence studied in this thesis. Westheimer and Mitchell (1956) demonstrated that robust horizontal vergence could be induced by changes in disparity in a stereoscope even though distance. size and other cues were kept constant. Vergence driven by image disparity is referred to as disparity vergence. Horizontal vergence compensates for horizontal disparity and shifts fixation between depth planes, vertical vergence compensates for vertical disparity, and cyclovergence compensates for cyclo-disparities. Constant disparities resulting from misalignment of the eyes can be eliminated by vergence. Vertical disparity drives vertical vergence and gradients of vertical disparity drive cyclovergence (Howard 199 1 ). Thus, processing of vertical disparity is critical for maintenance of eye alignment. In addition, disparity vergence rnay facilitate stereopsis by bringing stimuli into the disparity range of fine stereoscopic processing. It has been proposed that this precise depth information can be accumulated over a series of vergence movements to different depth planes in an image. This would allow construction of a detailed representation of depth in an extended scene (Mm 1982). Minimising vertical disparity at the fovea with vertical vergence presumably improves stereoacuity since vertical disparity has been found to decrease stereoacuity (Friedman et al. 1978). As discussed above (Section 1.1), the horizontal line horopter consists of the mid- sagittal plane of the head and the horizon plane. A movement to any point off the horizontal line horopter (Le. an oblique point) would require a vertical vergence 37 movement to maintain bifoveal fixation. The extent of the required vergence movement depends on the eccentricity and distance of the target and the CO-ordinatesystem used for defining eye movements (see Section 1.1). The amount of vertical vergence required for any given conjugate gaze direction increases with decreased distance but the sign and magnitude Vary with horizontal and vertical gaze angle and are thus not unambiguously correlated with distance. Fusion of a disparate image can be achieved by vergence andor by sensory fusion. In any given vergence state, only a subset of points on an extended surface can be in correspondence - those which lie on the horopter. Sensory fusion is limited to Panum's fusional area while stereoscopic depth processing has a somewhat larger disparity range. If vertical vergence were driven by mean vertical disparity, simply to keep the eyes in register, then much of the vertical disparity information could not be processed since it would lie outside the range of binocular visual mechanisms. For example, a point located 24" up and 24' to one side at a distance of 33 cm requires 1 S0of vertical vergence ro fixate (Ogle and Prangen 1953). This certainly exceeds the fusional range. Stereopsis would also be degraded but it is uncertain whether the point could sri11 be processed stereoscopically (see Ogle 1955, Mitchell 1970). If vertical vergence can be driven by local vertical disparity then vertical disparities could be sequentially compensated for by vergence as the eyes scan the surface and the local disparity patterns in each region registered. Thus we may expect that vertical vergence would be somewhat responsive to local variation in vertical dispaxity in a surface rather than responding only to whole field vertical disparity. The ability to maintain static alignment of the eyes and to compensate for prism induced vertical disparity has been known since the time of Helmholtz (1909). The ability to converge on a stimulus with a static absolute vertical disparity habeen addressed by several investigators (Ogle and Prangen 1953, Ellerbrock 1952, Allen 1974. Schor et al. 1983). Ogle (1964) has claimed that the largest forced divergences produced by a subject are S.5to 3 prism dioptres. Forced divergences were produced by placing a prism in front of one or both of the eyes to induce vertical disparity. Outside these Iimits. 38 subjects fail to fuse the images into a single object and experience doubling of the images or diplopia. This fusional range includes a motor component plus any sensory fusion of the remaining fixation disparity. Mitchell (1970) looked at the interaction of vertical and horizontal disparity in vergence. Even in the presence of up to 4" of vertical disparity, horizontal vergence movements could stiil be elicited by short presentations of horizontally disparate stimuli. The proportion of trials in which horizontal vergence was initiated declined if vertical disparity was increased. London (1 987) has demonstrated in clinical subjects that correction of a vertical eye misalignment (tropia) increased the ability of patients to compensate for horizontal prisrn disparity. This implies that vertical disparity disrupts horizontal vergence. The interaction of vertical vergence and horizontal vergence was also investigated by Bornan and Kertesz (1983). They found that vertical vergence latency increased and response magnitude declined when horizontai disparity was introduced into the display. Vertical disparity had no effect on horizontal vergence. However, their display contained a strong horizontal-vertical anisotropy, which may account for the anisotropy found in the results. In agreement with this hypothesis. data from Mitchell ( 1970 his Figure 8) suggest that vertical disparity in vertical Iines has a more detnmental effect on horizontal vergence than vertical disparity in horizontal lines. Some authors have distinguished between fusional and disparity vergence (tg. Jones 1983) with disparity vergence being the response to manifest (diplopic) disparity and fusional vergence the response to fusible disparities. Disparity vergence can be triggered by dissimilar images but the response is transient (Mitchell 1970. Jones 1977). For sustained vergence a fusible target must be presented. Erkelens (1987) found that the distinction between sustained and transient vergence was especially evident under open- loop conditions. Presentation of a normally fusible stimulus, with less than 2" of open- loop disparity, resulted in a ramp-like vergence response that saturated at a large sustained vergence. Larger open-loop disparity (2-5') resulted in a large initial vergence movement that was not maintained as the eyes gradually drifted back towards the initial vergence position. It appears that vergence may be effected by two systems (Westheimer 39 and Mitchell 1969). The first is selective to similar targets in the two eyes and operates over a small disparity range to maintain or lock on to fusion. The second is relatively less selective and operates over larger disparity ranges to initiate but not maintain changes in vergence. These postulated mechanisms have been investigated by several authors. Jones (1977) presented subjects with a target in the one eye that was matched to two targets in the other, one identical and the other dissimilar. Subjects tended to initiate vergence eye movements to whichever target pair had less disparity, regardless of similarity. From this.

Jones ( 1983) proposed that the vergence initiation mechanism was devoid of shape identification processes. He proposed that the initiation mechanism is dnven by coarse near or far-disparity detector pools (Richards 197 1) with feature selectivity precluded by pooling over a number of neurones and a coarse disparity range. It was proposed that these initiation neurones were of the transient type to account for the transient initiation response while the selective, fine disparity range cells used to maintain vergence were of the sustained type. It should be noted that the finding that vergence latency is much larger than one would predict from temporal frequency-response phase Iag also argues for different mechanisms for vergence initiation and maintenance. It is not known if there are any analogues of the transientkustained vergence distinction for vertical or c yclovergence.

1.3.5 Computational Theories of Vertical Disparit~Processing in Stereopsis

Many of the basic features of stereopsis relevant to this study have been reviewed earlier in this introduction. This section concentrates on computational consideration of the role of vertical disparity in binocular depth and distance perception. Computational methods of analysis and synthesis provide a description of possible algorithms the visual system could use given the information available in the optic arrays. In discussing these mechanisms, empirical evidence conceming their possible use in human vision is reviewed. The induced-size effect refers to the surface slant observed when the image in one eye is magnified vertically with respect to that in the other. This effect was systematically studied by Ogle (1938, 1964) aithough it had been known since the nineteenth century (Lippincott 1889). This size disparity, a verticaily oriented gradient of vertical disparity, induces a percept of surface slant opposite to that produced by a horizontal magnification of the image in the sarne eye. The percept of slant with horizontal size disparïty is predicted from projective geometry and Ogle (1938) cailed it the geometric effect (see 1.2.2.1). The vertical size disparity effect is not predicted from the projective geometry of real slanted surfaces. Ogle (1938) called it the induced-site effect because it is as though the vertical magnification of the image in one eye induces an equivalent horizontal magnification of the image in the other eye. Thus. the vertical size disparity is convened into an equivalent horizontal size disparity of opposite sign. Ogle (1938) found that the induced-size effect was equal to the geometric effect for small size disparities but saturated for rnagnifications above approximately 66. Banks and Backus (1998) have recently shown that perspective cue conflict may play an important role in limiting the range of the induced-size effect. Ogle (1 939) showed thai the effect could be induced by only two vertically separated black bars located in the median plane. Kaneko and Howard (1997) have found, however. that vertical size disparity is averaged over a wide retinal area (approximately 20") and the magnitude of the induced-size effect increases with stimulus size. Ogle noted that the vertical size ratio of the images of an object in the two eyes varied as a function of head-centric eccentricity. PIacing a surface eccentrically, normal to the line of sight. results in an isotropic rnagnification of the image in one eye relative to the other (aniseikonia). Ogle argued that the visual system uses vertical size disparity to judge the extent of this aniseikonia. This estirnate is then used to scale disparity to compensate for the size distortion. Thus, a vertical minification of the image in one eye results in a compensatory. neural. isotropic magnification of the image. This introduces a horizontal size disparity of opposite sign to the original vertical size disparity. This induced horizontal size disparity generates a percept of surface slant opposite to the geometric effect. Aniseikonia cm also result from differential growth or ageing in the two eyes and Ogle's explanation can explain compeiisation to these distortions as well. 4 1 Any theory of stereopsis that does not account for vertical disparity cannot explain the induced-sizc effect. Petrov ( 1980) has shown that, under normal circumstances. a properly fused image contains sufficient visud information for determination of binocular eye position. Unlike Ogle, Petrov proposed that, in the induced-size effect, the vertical disparity specifies that the viewed surface is eccentric. In Petrov's account, this eccentric location automaticaily results in the induced-size effect when the stereoscopic visual field is reconstmcted. Mayhew and Longuet-Higgins (1982) described a computational theory of stereopsis based on the processing of verticai disparities. They noted that horizontal disparities alone were insufficient to allow for the computation of depth. This is because horizontal disparity is also affected by the direction and distance of the disparate points (see Section 1.2). These authors proposed that this information could be obtained directly frorn vertical disparity rather tiian from extraretinai sources. This proposal is attractive since extraretinal sources of eye position information are nonnally regarded as unreliable and aiso because it provides a computational implementation of Ogle's theory of the induced-size effect. Under the strong assumption that the eyes are

vertically and torsionally aligned. Mayhew and Longuet-Higgins ( 1982) noted that knowing the horizontal and vertical position of as little as three points on both retinas was enough to specify the remaining degrees of oculomotor freedom: gaze eccentricity and convergence angle. Furthemore an approximate method of obtaining these parameters was outlined based on knowing the vertical disparities of only twc non-meridional points with djfferenr elevations. Theoretically, the gaze eccentricity and convergence parameters provide sufficient information for the recovery of metric depth information. Gillam and Lawergren (1983) noted that the vertical size ratio (VSR) of elements in a frontal surface increases with increasing eccentricity (Figure 1-9). We have seen in Section 1.2 that an extended frontal surface projects to the two retinae with a characteristic pattern of horizontal and vertical disparities (Figure 1-3). For increasing leftward eccentricity, the vertical angular subtense of the image in the left eye is increasingly larger relative to that in the right (and vice-versa for rightward eccentricity. Figure 1-9). These disparities increase with head-centnc eccentricity on the surface and 42 decrease with egocentric distance. Thus the rate of change of VSR with eccentricity is a function of egocentric distance of the surface from the observer. Howard (1 970) tumed this relation around and suggested that the pattern of vertical disparities in an image could be used to judge egocentric distance if eccentricity was known. This gradient of VSR is scaled by egocentnc distance but is relatively unaffected by local depth or surface slant. Gillam and Lawergren (1983) proposed that the visual system uses the VSR and the gradient of VSR to estimate the eccentricity and egocentric distance of a surface. The gradient of VSR provides distance information unconfounded with slant or eccentricity. Wi

( 1995) similarly rely on vertical disparity information pooled over regions of the visual field. In contrast to these 'global' theones, the theory of Koenderink and van Doom (1976) proposes a strictly local computation of surface slant from vertical and horizontal 44 disparity. Based on an analysis of the differential geometry of the disparity vector field they proposed that surface orientation in depth is coded by the deformation component of the vector field. The first-order disparity field can be decomposed into components of translation, rotation (curl), isotropie expansion (divergence), and deformation (Figure 1- 10). It can be shown that depth information is contained in the deformation component alone. Two types of deformation can be defined to simplifj discussion. In one type. the horizontal gradient of horizontai disparity is compared (ratiometrically) with the vertical gradient of vertical disparity. A deformation of this type implies an anisotropic size distortion of ones eye's image relative to the other (opposite horizontal and vertical size disparity). This type of deformation is interpreted as slant about a vertical axis regardless of whether it was caused by vertical or horizontal disparity and hence the induced-size effect follows directly. This type of disparity is immune to uniform expansion of ones eye's image since this results in changes to the divergence component but not to the deformation component. Thus the Koenderink and van Doorn ( 1976) theory predicts insensitivity to aniseikonia. In the second type of deformation, the horizontal gradient of vertical disparity is compared (ratiometncally) with the vertical gradient of horizontal disparity. A deformation of this type implies an anisotropic shear distortion of ones image relative to the other (opposite horizontal and vertical shear disparity). This type of disparity is immune to differential cyclorotation since this results in changes to the cur1 component but does not affect the deformation component. This type of deformation is interpreted as inclination about a horizontal ais. Pure horizontal shear and vertical shear disparity contain a rotation component plus a deformation of this type. The theory proposes that the rotation component is ignored and the geometric effect results from the deformation. Note that this theory predicts the existence of a shear disparity version of the induced-size effect. Gillam and Rogers (1 99 1 ) searched for this induced-shear effect of vertical shear disparity but did not find it. Also. they found that pure rotation disparity did result in a percept of surface inclination. Since the deformation component of the 45 disparity field is the same for both vertical and horizontal shear disparity (with a sign change) and is zero for rotation dispaity this appears to argue against the deformation disparity theory. Howard and Kaneko (1994) did however find a robust induced-shear effect with a large (65") isolated display. For small disparhies, this induced-shear effect was equal in magnitude to the geometric-shear effect. Furthemore, in such a large isolated display, rotation disparity did not result in any percept of surface inclination. These authors concluded that the negative results of Gillam and Rogers (1991) were due to the small size of the test display and the presence of zero-disparity reference stimuli in the dimly lit room. Howard and Kaneko (1994) tested this hypothesis by presenting various sized displays with shear disparity in isolation or in the presence of a zero disparity surround. The effect was weaker with smaller displays and inclination was absent in the presence of the surround. This was taken as evidence that the visual system averages vertical shear disparity over the entire visual field rather than extracting ii local ly. Since the Koenderink and van Doom (1976) mode1 relies on differential quanrities it is inherently local. Corrections for aniseikonia and differential cyclorotation are inherent and made on the same spatial scale as the slant and inclination judgements. This local action and the fact that deformation cm be easily estimated frorn local binocular differences in orientation of Iine elements suggests physiologically plausible implernentations. In contrast the global processing required in the other theories is computationally less attractive. On the other hand, psychophysicai experiments have suggested that vertical disparity is not processed locally. For exarnple, Kaneko and Howard (1997) found that depth comgations could not be detected for sinusoidal modulations of vertical-site disparity unless spatial frequency was extremely low. Corrugations were apparent for sinusoidal modulations of horizontal-size disparity at much higher spatial frequencies. This implies that vertical-size disparity is not processed on the same local scale as horizontal-size disparity. Similarly, the fact that different vertical shear disparity in two hdves of a display or in two overlapping transparent surfaces does not result in segregation of two inclined surfaces suggests that venical shear 46 disparity cannot be processed locally (Kaneko and Howard 1997). Adams et al. (1995) have dernonstraied that differential vertical disparities indicating different curvature in depth of two overlapping transparent surfaces could not be discriminated. These results indicate that deformation disparities are not processed locally as proposed by Koenderink and van Doorn (1976). Presumably this is because the viewing system parameters which effect deformation disparity are global and effect al1 points in the field. Thus local estimates are highly redundant since the invariant vertical part of the deformation disparity is evaluated repeatedly at each location. Howard and Kaneko (1994) proposed that a single global estimate of the vertical shear or size component of the deformation disparity is made for the whole binocular visual field or for large regions of it and then compared with the local horizontal component in calculating deformation. This is a modified type of deformation disparity processing which economises on the computation of the vertical disparity component but sacrifices the local action of the original proposai. For the slant case. if a single estimate of vertical size disparity is used for the entire visual field, this modified deformation disparity theory is analogous to

Ogle's ( 1964) explanation of the induced-size effect. For the inclination case. this modified form of the deformation disparity hypothesis is the only current theory capable of explaining the induced-shear effect and its global nature. Figure 1-9: Verticai Size Ratios (VSRs) a) The pattern of vertical disparity in a fkonto-parailel plane varies as a fiinction of horizontal eccentricity. Along the midline, all points are equidistant from both eyes and elements in the image subtend the same angle at the nodal points of each eye (PL0 equals PRO). For eccentric directions, the targets are closer to one eye than the other and thus for rightward eccentricity the image is larger in the right eye than the left (P'OLf is smaller than P'RO'). The ratio of the size of the lefi eye's image to that of the right eye's image is the vertical size ratio (VSR). b) VSRs increase with eccentricity. The rate of change of VSR with eccentncity decreases with increasing egocentric distance. Adapted fkom Rogers and Bradshaw (1993).

Midline

Horizon Figure 1-10: Deformation disparity

Koenderink and van Doorn (1976) proposed that processing of disparity could be simplified by the use of differential invariants. Local disparity structure cm be decomposed into translation, curl (or rotation), dilation and deformation components. a) Deformation disparity (an anisotropic expansion) is invariant under aniseikonia or cyclovergence and codes the slant or inclination of a surface. b) Dilation (expansion) and rotation are a function of eccentricity, aniseikonia and cyclovergence but independent of surface orientation and could be ignored by stereopsis. 1.4 Objectives and Hypotheses The objective of this work is to evaiuate the spatial and dynamic properties of the oculomotor response to venical disparity (vertical vergence) and of the perceptual response to venical disparity patterns as refl ected in sensory fusion and depth perception. It is hypothesised that the brain uses vertical disparities to estimate viewing system parameters (relative alignrnent of the eyes, absoiute viewing distance, eccentricity of gaze or viewing, relative image size in the two eyes). The visual system corrects for disparity distortions associated with these parameters by oculomotor or sensory mechanisms. 1 predict that vertical disparity processing mechanisms would be relatively slow and act over wide areas of the visual field. This is because the proposed mechanisms are sensitive to viewing system parameters that change slowly over time and affect the entire binocular visual fieId. Spatio-temporal averaging reduces the effects of spurious noise in arriving at a single estimate of siowly changing viewing system parameters. Vertical phoria and cyclophoria are typically slowly changing parameters. Vertical vergence and cyclovergence mechanisrns designed to compensate for these disturbances can be slow compensatory mechanisms. Over time. long-term adaptive measures can adjust to compensate for these phorias. Horizontal disparity changes rapidly with changes in surface relief. In contrast, vertical disparity is affected less by local depth changes. Changes in fixation distance require large changes in horizontal vergence that have no counterpart in vertical vergence. Thus 1predict that vertical vergence has a smaller linear range and a more sluggish response than horizontal vergence. The induced-size effect cm compensate for disparity distortions resulting from aniseikonia (Ogle 1964, Kaneko and Howard 1996). These dichoptic differences in image size can result from growth and ageing, or due to the optical demands of spectacle Wear. Similarly, the induced-shear effect can compensate for spatial distonions resulting from cyclodisparity. These types of long term changes in vertical disparity pattems are well compensated for by slow adaptive processes. Spatio-temporal averaging may not always be advantageous. For example, the human eye is hiphly specialised with a high acuity fovea The behavioural advantage of minimising vertical disparity at the fovea may tend to favour local action for vertical vergence control even though vertical disparity is typically slowly changing over the field. A type of aniseikonia arîses when a surface is viewed eccentncally (see 1.3.5). Aniseikonia in this case can potentially provide information about the eccentricity and egocentric distance of the surface. These viewing system parameters can change due to relative motion of the observer with respect to the viewed scene. Temporal averaging in this situation may compromise responsiveness to changes in the viewing system parame ters. The general objective of this work was to investigate the temporal properties of vertical disparity processing in the human visual system. Specifically. the following objectives were esrablished: 1. Evaluate the temporal efficiency of the oculomotor response to zero-order vertical disparity. The first experiments mesure the effects of temporal frequency and stimulus amplitude on vertical vergence. 2. Evaluate the temporal efficiency of the perceptual response to first-order vertical disparity. The second set of experiments studies the time course and temporal frequency dependence of percepts of stereoscopic depth based on horizontal and vertical disparity. 3. Evaluate the temporal characteristics of perspective-disparity interactions. The final experiment investigates the role of disparity-perspective conflict in the second set of expenments. 2.1 Introduction

The dynamics of vertical vergence have not been adequately investigated (see Appendix A for review of systems theory and control theory concepts used in this chapter). Perlmutter and Kertesz (1978, 1982) have been the most active in this area. Perlmutter and Kertesz (1978) were the frrst to study vertical vergence eye movements using an objective measurement technique. They used two double Purkinje eye trackers to monitor the movements of both eyes. Vertical disparities ranging from 33.6 to 53.6 minutes of arc were introduced into a dichoptic display of a horizontal line. Subjects could not compensate for disparity steps or gradual disparity exceeding la, a finding at odds with Ellerbrock (1949a). In addition, the amount of vergence was insufficient to compensate for the disparity. The display used by Perlmutter and Kertesz ( 1978) may explain the weak vergence response. They presented disparate lines in a fixed round aperture, perhaps the oscilloscope bezel. If visible, the aperture would provide a competing stimulus for vergence and may explain why the response was so poor. The vergence response to a disparity step took approximately 8 s to complete. This result suggests that the vertical vergence system is quite slow in comparison to horizontal vergence. In a later study, Perlmutter and Kertesz (1982) attempted to determine the temporal frequency-response of the vertical vergence system for open- and closed-loop conditions. The dichoptic display they used subtended 8.5" and consisted of a fixation square and a set of horizontal lines. The repetitive nature of the stimulus (apart from small discontinuities) would result in arnbiguity in stereoscopic matching and may have contaminated their findings. This may explain the unexpected result that vertical vergence had a 3.2 db gain and large phase lag (25-50") at low frequencies under closed-loop conditions. High-frequency response was limited to approximately 2 Hz. The combination of high gain with a large phase lag would result in poor oculomotor compensation for vertical disparity and would require sensory compensation. This remarkable result requires confirmation with a display that is less prone to correspondence ambiguity. Note that in their previous study the authors did not report any systematic elevation in vergence gain during the response to disparity steps (Perlmutter and Kertesz 1978). Under open-loop conditions the response to a sinusoidally changing disparity showed a 9 db gain at low frequency with an approximately second-order fall-off at 0.3 Hz (Perlmutter and Kertesz 1982). To rneasure the open-Ioop response. an eye-stabilised display was used to maintain a constant disparity. Rashbass and Westheimer ( 196 1) called this prccedure disparity clamping. Theoretically it is equivalent to breaking the feedback loop resulting from reafference of the eye movement itself. The latency of response to an open-loop step was between 130 and 220 ms. The initial velocity was proportional to the size of the disparity step but the overall response saturated at about 1 O regardless of the initial disparity. The linear relation between initial velocity and disparity was consistent with the open-loop system acting as an integraior in a manner similar to the horizontal vergence system. The 1 maximum vertical vergence was very small compared to the 4-6" of compensation to prism induced disparity seen by Ellerbrock ( 1949a,b) and Duwaer (1 982). About the tirne that Perlmutter and Kertesz (1978) reported their results, Houtman et al. (1 977) measured the temporal frequency-response of the vertical vergence system. These authors used the magnetic scleral search coi1 technique to record binocular eye movements. In this study. the displays were 1) a small dichoptic dot subtending 10 minutes of arc or 2) a 'complex' figure subtending 10". No information was provided about the 'complex' figure. They did however reference an earlier article that suggests it may have been a section of a Snellen eye chart although the reference is unclear. The vergence response to a 15' arc disparity step in the 10' figure had a latency of roughly 150 ms and took approximately 6 s to complete. The response was symmetric for onset and removal of the disparity step. A 0.02 Hz sinusoidal modulation of disparity produced 53 a response gain of 1 for 30 or 40 minutes of arc peak disparity while gain fell-off for frequencies above 0.2 Hz. Phase lag was near zero at low frequency and increased with stimulus frequency. The gain decreased with stimulus amplitude. Phase was independent of stimulus amplitude. The luminous dot gave lower gain, bandwidth and stability of response than the complex figure. Open-loop responses to step stimuli showed a latency of 150-200+ ms. As Perlmutter and Kertesz (1982) found, the response was in the form of a rarnp. However, the velocity appeared to be independent of stimulus disparity. In summary. the previous work done on the open-loop and closed-loop dynamics of vertical vergence has been contradictory and prone to artefact. This experiment was designed to measure the closed-loop frequency response of vertical vergence using a larger stimulus than used in previous experiments and for a wide range of frequencies and amplitudes of stimulus displacements. 2.2 Methods

2.2.1 Image Presentation

Wheatstone or mirror type stereoscopes (Wheatstone 1838) were used to present al1 of the stereo images used in this study and in the experiments described in Chapter 3. In any stereoscope the obvious requirement is to present a different image to the two eyes. In a mirror stereoscope this is achieved by presenting separate images to the two eyes through mirrors mounted in front of each eye (see Figure 2-1). In our apparatus, translucent mylar screens were mounted to the left and right of the subject on the sides of the field coi1 frame used for the eye movernent measurements. Dichoptic images were presented on the side screens and viewed through mirrors mounted at 90" in a standard Wheatstone stereoscope configuration. Semi-silvered mirrors were used to facilitate alignment and calibration of the instrument. Alignment and geometry correction were done with respect to plumb lines and to alignment marks on the display screens. This alignment was then checked both monocularly and binocularly against a reference pattern viewed directly through the mirrors. Absence of motion parallax between the half images and the reference surface ensured correct viewing distance, and absence of binocular parallax (with zcro disparity test pattems) ensured correct convergence angle. The rnirror stereoscope is regarded as the most precise and versatile type for research purposes (Howard and Rogers 1995). However, beyond the critical alignrnent several precautions must be undertaken. The use of mirrors to separate the images eliminates cross talk between the half images, thus avoiding a major problem of other systems. Displays of up to 90" cm be viewed in this configuration. However, large displays allow for viewing the monocular half images directly with peripheral vision or eccentric fixation. In addition. stray light reflected off objects in the visual field could provide spurious monocular or (less likely) binocular stimuli. To maintain a clean stimulus. care was taken to ensure that the screens were not directly visible by affixing blinders to the subject. Al1 objects in view were painted flat black or covered in black cloth. Consequently, only the stereoscopic stimulus combined through the mirron was visible. 1 used stimulus pattems that subtended up to 65' of visual angle; a necessary requirement since some phenornena studied are most apparent for whole field binocular stimulation. Disparity was specified in tems of degrees of visual angle projected on the tangent screen. Figure 2-1 : Wheatstone stereoscope

Wheatstone mirror stereoscope configuration used in al1 experimenis. The subject is presented with images dichoptically, that is separately to cach eye. In the cxperin~entsdesçribed in tliis thesis, the images were rear projected onto screens beside the subjcct. A pair of iiiii-rors rnounted at right angles and placed in front of the eyes direct the appropriate image to cadi eyc. Tlic visual system combiiics the Iialf-iiiiüges stereoscopicnlly to form a 3-D image. Adirpted frotn Ilowartl iind Kaneko ( lO94). The displays used in earlier studies of vertical vergence suffered from correspondence arnbiguity. were not described adequately, or did not provide a strong stimulus for vertical fusiond movements. 1 used a well-defined textured stimulus that filled the binocular field of view. Stimuli on photographic slides were projected ont0 the screens and viewed in the stereoscope. When viewed dichoptically, the two images formed a single binocular image subtending 65" in the frontal plane at a distance of 57 cm directly ahead of the subject. Care was taken to ensure that the screens were not visible directly. The pattem consisted of randomly positioned geometricd figures as shown in Figure 2-2. To prevent any suppression of vergence movements by surrounding features with zero disparity. the figures were light elements on a dark background and ail surrounding objects were matte black so that only the dichoptic textured pattem was visible. The stimulus elements in each monocular display had a mean luminance of 13 cd/m2 on a background of mean luminance of less 1 cd/m2. Luminance was reduced by approximately 506 after reflection off the semi-silvered mirrors. The stimulus had a wide range of spatial frequencies and a mixture of horizontal and vertical features to drive vertical vergence and control cyclovergence and horizontal vergence. A central vertical fixation line, which extended the length of the display. was also provided as a horizontal fusion lock. A horizontal line (1 .O0 long by 3 minutes of arc thick) intersected this line in the centre of the display to provide a fixation point. A regular pattern of line elements was avoided since the eyes have a tendency to misconverge on such a stimulus, as in the well-known wallpaper illusion (Howard and Rogers 1995). The size of the display elements increased outwards from the centre to compensate for decreased acuity in the visual periphery (Anstis 1974). I used an optically produced display since it has higher resolution than a cornputer-generated display (Figure 2-3). In addition, mechanical movement of the slides provided a smoother and more rapid movement than could be achieved in a computer- generated display. Identical left and right images were made into 35-mmslides. Each one was mounted in a custorn slide holder that could be oscillated sinusoidally up and down by a rocker m.The two rocker arms were driven by the same servomotor so that the two images oscillated in counterphase at a frequency determined by the speed of the motor. A microswitch on the motor shaft indicated the start of each cycle and allowed calibration of the oscillation frequency. A micrometer on the rocker am allowed the amplitude of oscillation of each image to be set with a resolution of 1 minute of arc. Since the two images oscillated symrnetrically in counterphase, the peak disparity was twice the individual peak image displacement. Figure 2-2: Stimulus for vertical vergence

Display used to induce vertical vergence. The display consisted of well spaced texture elements of various shapes so as to avoid false binocular matches. Texture elernent size was M-scaled in the penphery to compensate for decreasing visual acuity.

Pi- +O + '10+0. + 1 ot +. orW. + ,.", - 1 H .i, .O+ O + = Figure 2-3: Schematic diagram of the experimentai apparatus

Scleral search coils were used to record the movements of both eyes. The subject sat at the centre of the field coils with head supported by a bite bar. Translucent rnylar screens were mounted to the left and right of the subject on the sides of the field coi1 frame used for the eye movement measurement. Dichoptic images were presented on the side screens and viewed through mirrors mounted at 90" in a standard Wheatstone stereoscope configuration. Care was taken to ensure that the screens were not directly visible and that dl objects in view were painted flat black such that ody the stereoscopic stimulus combined through the mirrors was visible. The stimulus pattern can subtend up to 65" of visual angle at the viewing distance of 57 cm. Identical left and right images were made into 35 mm slides and projected ont0 the two screens. The slides were rnounted in a custom slide holder which could be translated up and down at the end of a servo-motor driven pivoted rod. The projected images were moved optically in counterphase to create a sinusoidal disparity stimulus. A microswitch marked the start of each stimulus cycle.

Driveshaft A Motor 1 Pivoted Rocker

0 \ @ \ 0 \ 0 % 0 0 \ C \ EY~ \ Position

Screen J Screen Ellx \ Projector

0 * Minors C \ 0 5 0 \ 0 * 0 5 0 \ 0 9 t L field Coils 2.2.3 Ohiective Eye Movement Recordinp;

2.2.3.1 The Mannetic Search Coi1 Technique The movements of the eyes cm be modelled as rigid body rotation about a hypothetical single centre fixed within the eye (Westheimer 1957). Under this assumption, the eyes have 3 independent degrees of freedom for rotational movements. Each eye can be moved horizontally (azimuth), vertically (elevation) and torsiondly (roll) with respect to the orbit. In reality. the eye does not have a well-defined centre of rotation and any rotation of the eye involves a small translation that, in most practical situations. can be ignored (Alpern 1962, Westheimer 198 1 ). For foveate animals such as man. horizontal and vertical movements of an eye redirect the line of sight to place the image of the desired target on the fovea or high acuity portion of the retina. Torsional movements are about the line of sight and play no role in directing the line of sight. The magnetic search coil system that 1 used is the standard research technique for both animal and human studies and is well suited to rneasurement of vertical vergence. This technique is based upon detection of the voltage induced in a coil of fine wire (called the search coil) worn on the eye and located in an extemal AC magnetic field (Robinson 1963). 'The field is generated by a pair of large extemal magnets, the field coils. arranged in spatial quadrature. The amplitude of the AC voltage induced in the search coi1 is proportional to the sine of the angula. orientation of the coil plane in the magnetic field (i.e. it is amplitude modulated by eye position). The search coil provides an amplitude- modulated signal that can be demodulated to give angular position. The linear range of the inherent sinusoidal nonlinearity comfortably exceeds the +3" eye movements measured in this study. The field coils are driven at different frequencies so that the angle between each driving plane and the coil be sensed independently. To record horizontal and vertical eye movements, a search coil wound in a frontal plane is embedded into a silicone annulus or contact lens. An additional coil wound in the sagittal plane can be used to transduce torsional movements (Robinson 1963, Ferman et ai. 1987). However. this requires a thicker more expensive annulus and is not required to study vertical eye movements. Since ordinary comeal contact lenses are not fitted tightly enough to prevent slippage during fast eye movements, the lens is designed to fit tightly over the sclera and to be held on with a slight suction. This vacuum is intrinsicdly developed from the annulus design and insertion technique (Collewijn, Van der Mark and Jansen 1975). The search-coi1 technique does not restrict the field of view. It is also insensitive to translation if the head is kept within the region of homogenous magnetic field, that is within the centre of the field coils, a requirement easily maintained by using a bite board. The magnetic search coi1 technique is inherently invasive due to the insertion of the annulus. Although the annulus is well tolerated by subjects for periods of approximately 30 minutes the contact lens is somewhat uncornfortable and must be inserted after application of anaesthetic eye drops. The effect on eye and lid movements of the very fine lead wires which leave the annulus is believed to be minimal (Collewijn et al. 1975).

2.2.3.2 Eve Muvernent Recording For these eye movement experiments. 1 m field coils wound on a cubical frame were used to generate AC magnetic fields at 60 and 90 kHz for the horizontal and vertical fields respectively. Wood and plastic was used for the construction and care was taken to eliminate or minimise al1 metallic objects and sources of electromagnetic radiation from the vicinity of the apparatus. since they could distort the field. Topical anaesthetic was applied to both eyes and standard search coi1 annuli (Skalar, Delft, the Netherlands) were inserted in each eye. The subject was seated with the midpoint between the two eyes located in the centre of the cube. The subject's head was stabilised with head supported on a custom fitted bite board. Scleral search coils were used to record the horizontal and vertical movements of both eyes of the subject. After insertion of the annuli, they were gently pressed ont0 the i:ye through the iids to ensure firm adhesion. This adhesion could be checked by noting drift during the recording and by the integrity of the seal when the contact lens was removed. The vertical and horizontal eye-rnovement signals for each eye were demodulated by phase sensitive detecton (CNC Engineering, Seattle WA) and the resulting analogue signals stored on digital tape. A marker signal indicating the start of a cycle was also stored ont0 the tape. The signals were digitised at 12-bit resolution with a 100 Hz sampling rate and read into a Macintosh computer. The instrument was calibrated at the beginning and end of each session by recording the demodulated coi1 voltage as the subject looked at fixation points between Godong horizontal and vertical axes. The relationship between eye position and coil voltage was defined by the best fitting line to the calibration data using a Ieast squares criterion. This formula was then used to convert raw coil voltages into angular eye . position. Resolution measured on a mechanical eye was better than 0.5 minutes of arc. Positive angles refer to downward and rightward positions of an eye relative to the primary position. Calibration of the coil could also be performed using a specially designed gimballed, mechanical alignment jig. After approxirnately 30 minutes of data collection. the annuli were removed from the eye using blunt forceps. There was no noticeable deviation from linearity of the coil signal and no noticeable crosstalk between horizontal and vertical channels. 1 measured the amplitude of vergence modulation in response to disjunctive vertical motion of the dichoptic displays. This measure is insensitive to slow drifts in eye position. The absence of any signifîcant drift in mean eye position indicated that the search coils effectively adhered to the eyes.

2.2.4 Procedure and Data Analysis

Four subjects, two males and two fernales. between the ages of 22 and 32 performed the experiment. Al1 had normal stereoscopic vision and gave their infomed consent. Three of the subjects were myopes who normally wore eyeglasses or contact lenses. One subject wore her spectacles during the experirnent. The other subjects did not Wear their glasses but reported that they could see the display clearly. The subjects were seated in the coi1 frame and asked to maintain fixation on the centre of the display. The two images were oscillated in counterphase at frequencies of 0.05, 0.1,O.S. 1 .O. and 2.0 Hz with 18'. 33', 1 .O0,2.0" and 4.0" of arc peak-to-peak disjunctive vertical displacement. The frequencies were presented in a random order for each amplitude and amplitudes were presented in a different order for each subject. Vergence eye movements were recorded for at least 5 complete cycles of vertical oscillation. Vergence responses were obtained from the difference between the responses of the left and right eyes. For a sinusoidal variation in stimulus disparity, vergence gain is defined as peak amplitude of vergence divided by peak amplitude of stimulus displacement. Gain and phase for each condition were estimated by averaging the response calculated from vergence amplitude and delay over al1 recorded cycles. For theoreticaI reasons and due tû trends found in the data, some of the analysis was performed on vergence velocity rather than vergence position. To calculate vergence velocity, a 5-tap finite-impulse-response filter was used to differentiate the position data. The form of the filter was a central difference differentiator with zero phase delay and a bandwidth exceeding 20 Hz (Matthews 1987). Since the noise level for a given data sequence is larger for velocity traces, I used spectral analysis methods to extract and estimate the parameters of the sinusoidal velocity response at the stimulus frequency. This is a more sensitive technique than the time domain analysis performed above. The amplitude of the first hmonic component of the response was used as a measure of response magnitude. In most records. spectral power was concentrated in this harmonic but occasionally some distortion terms (2"dand 3rd harmonies) were evident. The peak velocity of the response at the stimulus frequency divided by the peak velocity of the stimulus was defined as the velocity response gain at the stimulus frequency. 1 also attempted to measure the fusional range for vertical disparity as a function of temporal frequency using the sarne apparatus. Subjects fixated the centre of the dichoptic display in which the small fixation line through the vergence lock line was visible. This was the thinnest line element in the display (3') and was used to determine the presence of diplopia. Subjects were instructed to watch this cross and determine whether the horizontal line appeared to separate into two lines at any time during the oscillation. For consistency, subjects were instnicted to use a criterion of the percept of two simultaneously visible lines rather than intermediate percepts such as thickening or suppression. In our apparatus, adjustment of temporal frequency was more convenient 64 than that of amplitude. Thus the frequency at which diplopia could be detected was measured for fixed amplitudes of 18', 33', 1 .O0, 2.0" or 4.0° of arc as frequency was varied. Before each trial, the amplitude was set and the motor position adjusted to O" disparity (Le. start the modulation with zero disparity to promote fusion). The frequency was then set and the motor started to present a minimum of 30 sec or 5 cycles of oscillation to the subject. Following this the fusional response was taken, the display reset to zero phase and the next frequency set. The method of limits was used to determine when the pattern became diplopic with increasing frequency and the frequency at which fusion was restored when frequency was decreased. Eight measures were made for each subject at each amplitude, four in each direction. The temporal frequency was increased or decreased from random starting points between 0.01 7 and 5.0 Hz in increments of 0.1 on the potentiometer controlling the motor speed. The size of the steps in Hertz depended on frequency range studied and varied from 0.013 Hz in the low frequency range to 0.1 Hz in the highest frequency range. Subjects were also asked whether the displays appeared fused during static presentation of the various stimulus amplitudes as well. To estimate vergence error at the frequency at which diplopia occurred 1 used the following procedure. Estimates of peak response magnitude at the reported frequency were made from the vertical vergence magnitude vs. frequency curves (for each subject) detemined in the main experiment. Values for intemediate frequencies were obtained by linear interpolation between the gains at the nearest frequency above and below. The diplopia threshold estimate was then detennined by subtracting the peak stimulus amplitude from the calculated peak response magnitude (the diameter Panum's area would equal twice this limit).

A sample of vergence records of one subject is shown in Figure 2-4. Typically. the two eyes responded with similar magnitudes and 180" out of phase with each other. Thus. the response to disjunctive movement of the half images was primarily disjunctive vertical eye rotation, or vertical vergence. At low frequencies. the movements of the right and left eyes closely followed the movements of the left and right images respectiveiy. At higher frequencies, the response was attenuated and phase lag increased. Versional movements are not required by the stimulus motion and subjects tended to make only small versional movements that were not synchronous with the stimulus frequency. One subject (EG) had a tendency to make conjugate foll~wingmovements rather than pure vertical vergence at large amplitudes and temporal frequencies. This was rarely seen in the responses of the other subjects. Figure 2-5 shows the magnitude of vertical vergence as a function of the temporal frequency of stimulus oscillation for each of four subjects. Figure 2-6a shows the mean gain of vergence for the four subjects as a function of stimulus frequency for each of five stimulus amplitudes. At a stimulus frequency of O. 1 Hz or less and amplitude of 18 minutes of arc the gain of vergence was almost one and there was no appreciable phase lag (see Figure 2-6b). Although the amplitude of vergence increased with increasing stimulus amplitude, response gain (peak response amplitude over peak stimulus amplitude) declined. Thus the increase in vergence ampli tude did not match the increase in stimulus amplitude and hence vergence gain dropped with disparity amplitude. Both response amplitude and response gain declined with increasing stimulus frequency for al1 stimulus amplitudes. At the highest frequencies, however, vergence amplitude showed less dependence on stimulus amplitude as evidenced by the convergence of the response curves in Figure 2-5. At the highest temporal frequency (2 Hz) the vergence response sometimes ceased to be elicited in portions of the record but al1 subjects showed some vergence response at this frequency. At 2 Hz, the response tended to saturate at a peak-to- peak vergence amplitude of approximately 0.2". This response differed from the saturation noted for low frequency, large amplitude conditions in that the response remained sinusoidal in appearance and did not appear to be clipped. If this were a true ceiling effect, it would further restrain vertical vergence gain for large amplitudes at high frequencies. It is difficult to establish whether this is a ceiling effect with the current data since the gain Cumes converge at this point. Therefore, although vertical vergence amplitude fell more significantly with frequency for larger stimulus amplitudes. vergence gain did not show the same strong frequency-amplitude interaction. With a stimulus amplitude of 4". the response of some subjects was decidedly nonsinusoidal and significant clipping occurred (Figure 2-7). Larger stimulus amplitudes of 8 and 12' were tried for one subject at 0.1 and 1 .O Hz. The subject continued to respond with vertical vergence although magnitude did not increase beyond that seen for the 4" amplitude. As discussed above, one subject responded with conjugate as well as disjunctive eye rnovements to Iarge stimulus amplitudes, presumably because he suppressed one of the disparate images and followed the remaining image with both eyes. Figure 2-6b shows the mean phase iag of the four subjects as a function of stimulus frequency for each of five stimulus amplitudes. There was a small but significant increase in phase lag as stimulus amplitude increased but the increase was no1 always systematically related to stimulus amplitude. At the lowest stimulus frequency and amplitude. vergence gain was near one and phase lag was typically less than 10". Above a temporal frequency of O. 1 Hz, phase 1ag increased linearly with increasing stimulus frequency for al1 stimulus amplitudes. At 2 Hz, phase lag reached a value of 123" averaged across the five amplitudes and four subjects. Muitivariate and univariate repeated measures analysis of variance were run wirh sphericity tests indicating that univariate tests could be used. The univariate (and mdtivariate) tests of the various effects indicated a significant effect of stimulus amplitude (F(4.12) = 159.242, p < 0.01) and temporal frequency (F(4,12) = 103.088. pc 0.0 1 ) on the gain of vertical vergence. Analysis of variance also indicated a significant effect of stimulus frequency (F(4,12) = 489.617, p < 0.01) and a small but significant effect of amplitude (F(4.12) = 5.1 18. p< 0.05) on the phase lag of vertical vergence. The gain and phase appeared to change in a similar manner with increasing stimulus frequency for al1 stimulus amplitudes. However, a smdl but significant interaction between amplitude and frequency existed for both response gain (F(16,48) = 3.458, p < 0.01) and phase (F(16,48) = 2.186, p c 0.05) but these interaction terms accounted for less than 3% of the variance in the model. 67 The assumption of homogeneity of variance implicit in the general linear model used in the analysis was questionable. For example, inspection of the error bars in the gain data in Figure 2-6 shows that variability was somewhat greater at larger gains. Conversion of the gain data into decibels, a logarithmic transform. results in the variance being better equated across the data. When the andysis was repeated with the transformed data. there was no substantive difference either in the results or in the conclusions deduced from them. The results appeared to be quite robust and similar conclusions to those reached above were obtained frorn standard regression/ANOVA analysis (i.e. not repeated measures) with amplitude. frequency and their interaction as the independent variables. This analysis indicated a good fit of the model to the data with adjusted R' of 0.940 and 0.936 for the phase and gain respectively. Post-hoc multiple cornparison tests (Tukey. Boneferroni and Scheffé corrected F-tests) indicated that differences in phase could be grouped into two homogeneous subsets (a<0.05) with respect to stimulus amplitude effect. Thus the phase response to 1,2 or 4" stimulus amplitude was significantly different from the response to 18' or 33' amplitudes. Differences in gain were significant for al1 pairwise comparisons between the levels of stimulus amplitude. Thus. in pairwise comparisons, the effect of amplitude could be seen between al1 levels of the amplitude independent variable for the gain measure but was only significant between the smallest and largest amplitudes for the phase measure. Differences in both phase and gain could be grouped into 4 subsets with respect to frequency, with the responses to 0.05 and 0.1 Hz stimulation grouped together and the responses to 0.5, 1.O and 2.0 Hz stimulation in separate subsets. Thus, the effect of frequency resulted in significant differences between the levels used here except between the two lowest frequencies. In al1 of the above analysis, 1 used the transformed variable of log frequency as in Figure 2-6 rather than linear frequency. Regression analysis showed this transform had little effect on the conclusions reached except the model fit was worse without it. It can be seen by inspection that the log transform of the frequency variable resulted in a linearization of the gain data and the linearity of the phase data was slightly affected. Two possible outliers were identified from case-wise analysis of residuai variance. Inclusion or removal of these data points had no substantive effect on the results. Figure 2-8 shows that the gain of vertical vergence decreases approximately exponentially with increasing peak stimulus velocity in a similar fashion for al1 stimulus amplitudes. The peak stimulus velocity ranges for the different amplitudes overiapped each other. For these overlapping data points, the gain values seemed to be simila. regardless of stimulus amplitude. Thus, vergence gain appean to be more closely related to stimulus velocity than to stimulus frequency or amplitude. The vergence velocity gain as a function of stimulus velocity and an exarnple of a vergence velocity, frequency- domain record is shown in Figure 2-9. The fall-off in vergence velocity gain with peak target vergence velocity was similar to that found for amplitude gain. Peak vertical vergence velocity seemed to saturate with increased target vergence velocity and never exceeded 2"/s. The temporal frequency when manifest diplopia was reported in the central fixation line decreased with increasing stimulus amplitude for three of the four subjects. These three subjects varied in their dependence on amplitude. Two of these subjects showed a fairly strong dependency on amplitude with maximum fusible frequency declining from almost 1 SHz to less than 0.06 Hz for amplitudes varying from 18 minutes of arc to 2.0". The other subject in this group of three showed less dependence on amplitude with no fusible frequency limit (greater than 5 Hz) for the 18 minute of arc amplitude and a range of 0.33 Hz to 0.03 Hz for amplitudes from 33 minutes of arc to 2.0". The equivalent vergence error at which fusion was lost and regained (average of the two limifs) was estimated as outlined in the methods and plotted in Figure 2-10 as a function of stimulus frequency. These three subjects al1 showed a decline in diplopia threshold with increased temporal frequency. For example, subject HI3 showed a decline in diplopia threshold from an estimated 17.6 minutes of arc at a frequency of 0.03 Hz to 9.5 minutes of arc by 1.63 Hz. This decline was much less evident in subject XE Also shown in Figure 2-10 for comparative purposes is the data of Schor and Tyler (198 1) who measured Panum's fusionai area as a function of temporal frequency of disparity modulation in a line element display. Data from the fourth subject (EG) could not be obtained. This subject did not perceive the line going diplopic but noticed up and down motion of a single line as frequency was increased. This was the only subject to respond with significant conjugate eye movements to large, high frequency target vergence oscillations. The single appearance of the fixation line combined with the apparent movement suggest that this subject was suppressing one of the images and simply following the perceived image. The other subjects reported that, at the diplopia threshold, the centre Iine looked fused initially and becarne diplopic for a portion of the cycle (at the peak of the target vergence) then became re-fused. Stimulus Amplitude

Figure 2-7: Non-iinear vertical vergence

Record of vertical vergence to an amplitude of stimulus modulation of 4" at a frequency of 0.05 Hz in subject EG. In this condition, this subject showed the lowest gain and most pronounced nonlinearity for large-amplitude, low-frequency oscillation. The pulse train on the record indicates the peak of each disparity oscillation and provides an indication of time scale. Figure 2-8:Vergence gain as fmction of stimulus velocity

Amplitude gain of vertical vergence as a function of peak stimulus velocity averaged across the four subjects.

Peak Stimulus Velocity (deg/s)

Figure 2-10: Diplopia measurements

Calculated diplopia threshold (peak vergence error from gain and phase) as a function of temporal frequency in three observers. Also shown for cornparison purposes are the horizontal and vertical fusional ranges found by Schor and Tyler 198 1.

Frequency (Hz) 2.4 Discussion

There are two previous studies of vertical vergence dynarnics with which this study cm be cornpared. Houtman et al. (1 98 1) used 33 and 65 arc min amplitudes of stimulus disparity comparable with our 33 arc min and 1.O0 amplitudes for frequencies up to 1 Hz. For comparable conditions, our two studies report similar reductions in vergence gain with increasing stimulus amplitude and temporal frequency. For 33 arc min amplitude they report a somewhat higher gain at 0.05 Hz (near 1.0 versus 0.86) with a faster gain roll-off by 1.0 Hz (0.3 compared with 0.46). Their 65 arc min amplitude and our 1 .O0 amplitude resulted in a similar fall-off in response gain. Phase showed a similar weak decrement with stimulus amplitude in both studies. The display used by Houtman et al. ( 198 1) was considerably smaller than that used in our study. Their display was described as 'complex' which presumably meant that it contained a variety of distinct features and was difficult to misconverge. Unfortunately the incornplete description of their display does not allow cornparison of stimulus features which may be responsible for small differences in the results. In marked contrast to Our study and that of Houtman et al. (1 98 1). Perlmutter and

Kertesz ( 1982) reponed closed-loop gains much larger than 1 for dl stimulus amplitudes between 3.3 and 27.9 minutes of arc. Although gain declined with increasing temporal frequency, it remained above 1 even at a frequency of 1 Hz. This high gain would result in overcompensation for vertical stimulus misalignment. This led Perlmutter and Kertesz to conclude that vertical vergence does not cornpensate for vertical disparity and any compensation must be accomplished by sensory hision. In contrast. 1 found no large overshoots of vergence under any condition, including conditions with stimulus amplitudes and frequencies similar to those used by Perlmutter and Kertesz. The phase lags reported by Perlmutter and Kertesz were similar to those reported here. Unlike the results and conclusions reached by Perlmutter and Kertesz, our study indicates that vertical vergence is designed to compensate for vertical disparity at moderate amplitudes and frequencies of stimulus misalignrnent. It was argued earlier that the display used by Perlmutter and Kertesz was subject to artefact resulting from correspondence ambiguity between the mainly horizontal features of their stimulus. In an earlier study with line stimuli which did not suffer from this ambiguity. Perlmutter and Kertesz (1978) found no systematic overcompensation of vertical vergence. At low temporal frequencies the vertical vergence response should approximate the response to static disparity. The response of the vertical vergence system to static disparity has been most extensively studied by Ogle and Prangen (1953). They found that vergence eye movements cm compensate for optically imposed vertical disparities of 2.5 to 3.0 pnsm dioptres. after which diplopia occurs. Within this range the fixation disparity seldom exceeded 5-6 minutes of arc. With slow introduction of disparity and with time allowed for adaptation. vertical vergence movements could compensate for more than 3" of vertical disparity. Ogle used the subjective technique of nonius alignment and reported a sensitivity of 1 minute of arc with this procedure. Ellerbrock (1949a,b) measured fixation disparity using the nonius procedure for disparity slowly introduced in 0.25" steps. Subjects could compensate (i.e. fuse the stimulus with smail fixation disparity) for vertical disparity by vertical vergence over a range of approximately 4". In a later study Duwaer (1982) measured the relative displacement of a pair of aftenmages in the two eyes while viewing a disparate stimulus. Using this technique he found 8- 15 minutes of arc fixation disparity when fusing a 3-6" vertical dispuity. These studies indicate that response to static disparity is quite strong with a gain very near one. Even at Our lowest frequency 1 did not find this level of compensation especially for larger disparities. Gains were near one only for the smallest amplitudes, and vergence error for the 2.0 and 4.0" conditions was on the order of a degree or more. This indicates that the temporal frequency response may plateau at even lower frequencies than 1studied. Since I looked at oscillation with a period as long as 20 seconds this indicates that vertical vergence is quite sluggish. At a stimulus frequency of O. 1 Hz or less and amplitude of 18 minutes of arc the gain of vergence was near one and phase lag was small. This probably represents the normal operating range of the response. The range of vertical vergence has been reported 79 as up to several degrees for static vertical disparity (Ellerbrock 1949). The thesis has shown that considerable attenuation of vergence gain occurs with stimulus amplitudes as srnall as 33 minutes of arc. This amplitude dependent attenuation suggests that the linear range for dynamic vertical vergence is quite small. This nonlinekty would be problematic for modelling the behaviour of the vertical vergence system over the stimulus range studied, since any mode1 would have to include non-linea. mechanisms to describe this attenuation. This nonlinearity requires characterisation of the transfer function of the system under open-loop conditions prior to a realistic modelling effort. The maximum vergence disparity limit is the limit beyond which the stimulus no longer elicits vergence and is less than 2 to 4' of disparity under static conditions (Ogle and Prangen 1953,. Ellerbrock 1949). For sinusoidal oscillations 1have measured the vergence response in one subject for stimulus amplitudes of 8 and 12" at 0.1 and 1 .O Hz. The subject continued to respond to these oscillations with low gain vergence. This response to large disparity oscillations suggests that the vertical vergence system does not have a maximum disparity limit under these conditions. This may be because the stimulus contains a range of disparity values that are swept across and the systern may be responding prirnarily to these intermediate values. 1 did not attempt to determine the minimum disparity capable of driving vertical vergence. Dïwaer and van den Brink (1 98 1b) investigated whether a dead-band existed for vergence eye movements by attempting to determine the smallest disparity required to initiate vertical vergence. They asked subjects to fïxate a zero disparity dichoptic 10" by 10" cross composed of a horizontal and vertical line while aligning a pair of horizontal nonius lines. Next, a square with vertical disparity was presented for 2s and subjects detemined whether the right nonius line was displaced up, down or did not move (3 alternative forced choice). The authors assumed that the absence of a detectable shift was an indication that no vergence eye movements occurred. Using this criterion they found that vergence eye movements could be initiated by disparity as small as 1 minute of arc which they felt was the resolution of their technique. The vergence-initiation threshold was higher for stimuli presented for only 160 ms and at eccentricities greater ihan 4'. 80 These results are open to criticism on several fronts. First, in the forced choice method the subject may base his choice on a spurious feature of the stimulus. Since the subjects are presumably operating at the limits of their sensory capability it is likely that they would not be aware of using these stimulus covariates. More importantly, the authors included a nul1 choice of no movement presumably to prevent this use of spurious cues. However, this third alternative reintroduces a criterion into the psychophysical response since the subject's willingness to report a shift of the nonius line would influence the response. The third alternative in this instance has sacrificed one of the most important advantages of the forced choice paradigm - that it is a criterion free measure. The frequency response of vertical vergence bears a gross resemblance to both the horizontal and cyclovergence frequency responses. The results of Krishnan et al. (1973) are typical measurements of gain and phase lag of horizontal vergence. These authors studied the vergence response to a sinusoidally oscillating line target for a image disparity of 3.5". Low frequency gain was close to unity and flat with frequency but fell off sharply with frequency above 1.5 Hz with response eliminated by 2.0 Hz. Phase of near zero degrees at low frequency indicated that the horizontal vergence was compensating for the disparity. These data were typical of many early vergence studies in that low resolution monocular eye rnovement recordings (infrared technique) were made and the motion of the other eye inferred. Thus it was impossible to know if the subject had perhaps switched to suppression and monocular trackjng at the point where the response was lost. Zuber and Stark (1968) also measured the frequency response of horizontal vergence. Their data suggest that it is more robust than vertical vergence since they showed gain near one for frequencies less than 1 Hz with very small phase lag over this range. Erkelens and Collewijn ( 1985a) have made similar frequency response measurements using a large (30') random-dot display (see Figure 2-1 1). Gain was near one for low frequency (0.25 Hz) oscillations and fell with increasing stimulus amplitude in a manner similar to vertical vergence although saturation started at larger amplitudes. Bandwidth as reflected in the gain component of the frequency response was larger for horizontal vergence although phase responses were similar. This indicates that the vertical vergence response 8 1 is more sluggish than the horizontal vergence response (see also Houtman et al. 1977 and Rashbass and W estheimer 196 1). Both horizontal and vertical vergence show low pass behaviour. Vertical and horizontal vergence are sluggish compared with both saccadic and srnooth pursuit version (Carpenter 1988). Rjggs and Niehl (196 1) used an elegant contact lens eye-movement monitoring technique to show that horizontal vergence is considerably slower than the corresponding versions. Erkelens and Collewijn (1 985a) found that the gain of horizontal vergence is attenuated by increased stimulus amplitude but the phase is unafYected, rnuch like 1 have found for vertical vergence. Howard and Zacher (199 1) measured the frequency response of cyclovergence for disjunctive movement of a large (75") textured display and their data are also shown in Figure 2- 1 1. Both the fall-off in gain and the phase data were very similar to what was found for vertical vergence in this thesis. The gain showed a similar attenuation with stimulus amplitude. As with vertical vergence. stimulus amplitude had little effect on the phase response. Although vertical. horizontal and cyclovergence appear to have similar dynamic properties there are considerable differences between the responses. Although al1 the responses have a low pass characteristic. vertical and cyclovergence are more sluggish than horizontal vergence and give a lower gain at a given frequency. The range cf static disparity that can be compensated for by vertical vergence is only about 3" (Ogle and Prangen 1953) while the range of horizontal vergence is typically 10" (Jones and Stephens 1989). Horizontal vergence is also asymmetrical, with a larger range for convergence than divergence (Ogle 1964). Boltz et al. (1980) have found a range of static vertical fusion of t3 prism dioptres in a human subject and of d.5pnsm dioptres in the Rhesus monkey. The horizontal ranges were much larger, with 6 prism dioptres divergence and 22 prism dioptres convergence reported (Boltz and Harwenh 1979). Convergence movements also tend to be faster than divergent movements (Zuber and Stark 1968). Cyclovergence also appears to be asymmetric in many people, with in- cyclovergence greater than ex-cyclovergence (Howard and Kaneko 1994). 1 did not detect any evidence of a directional bias in vertical vergence. Work recently completed in Our 82 lab suggests the three types of vergence also differ considerably in their dependence on the spatial configuration of the stimulus (Fang 1997. Fang et al. 1997. Howard and Sun 1993). The gain of vertical vergence increases with stimulus area for displays less than 20" in diameter whereas cyclovergence requires very large displays and horizontal vergence cm be driven effectively by a point stimulus. Vertical and horizontal vergence are best dnven by central displays but the cyciovergence system responds best to peripheral stimulation (Figure 2-13). These results and the results descnbed in this chapter suggest that the vertical vergence system responds to slow changes in vertical disparity in a fairly large integration area surrounding the foveal region. Cyclovergence responds to slow changes in whole field cyclodisparity md parafoveal stimulation is not necessary. Horizontal vergence is capable of responding relatively rapidly to small disparities in foveal targets. 1 have found evidence that vergence magnitude falls-off approximately exponentially with peak stimulus velocity. Furthemore, the vergence gain versus peak velocity curves appear to overlap for the various amplitude conditions. Similar findings were obtained by Erkelens and Collewijn (1985a) for horizontal vergence and their data is reproduced along with ours in Figure 2-14. Note that horizontal vergence is much more robust and saturation occurs at larger stimulus velocities. Also shown is the equivalent cyclovergence data derived from Howard and Zacher (l991), although these authors did not analyse their data in this manner. That al1 three sets of data show an approximately inverse exponential relarionship to velocity for a given amplitude is of course expecred from the shape of their frequency responses. What is interesting is that the gain versus velocity curves overlap nearly perfectly for the different amplitude levels. This suggests that the nonlinearity is related to the velocity of vergence or the velocity of the stimulus and not to the position. Mays et al. (1 986) have identified cells in the midbrain just dorsal to the oculomotor nucleus that fire during horizontal vergence eye movements. Many of these cells are phasic and discharge is related to vergence velocity rather than simply vergence position and have been termed vergence burst neurones or vergence burst-tonic cells if they also have a tonic response. It has been suggested that these cells may indicate 83 a vergence velocity command that is integrated to form the vergence position signal at the motoneurone. This would be sirnilar to the velocity to position transformations that occur in other eye rnovement systems such as the saccadic and vestibulo-ocular systems where an early eye velocity command is transfomed into a position command via integration (Robinson 1981). Nonlinevities in the pathways forming and transmitting this velocity command could provide the substrate for velocity dependent saturation found in this study and by Erkelens and Collewijn ( 1%sa). Given this dependence on stimulus velocity, could vertical vergence sirnply be the result of two visual pursuit or optokinetic systems operating independently in the two eyes? Certainly this cannot be the whole story since vertical vergence can be driven most effectively by static stimuli. Optokinesis is a response to the motion of large stimuli (see Howard 1993) and cannot be dnven by static displacement. Smooth pursuit is also normally thought of as responding to retinal motion although smooth pursuit does appear to be dependent on position to some extent (Pola and Wyatt 1980, 1985, Kowler et al. 1984). The frequency responses of vertical and horizontal vergence are compared with those of vertical and horizontal optokinesis in Figure 2-15. Pursuit eye moments and optokinesis both tend to have sirnilar low pass characteristics to vergence but are much more robust. The eyes are capable of following constant stimulus velocities of up to 40°hec with high gain optokinetic responses for horizontal (Honmbia et al. 1967) and vertical stimulus motion (Ogino et al. 1996) and responses can be elicited for stimulus velocities to 100°/sec. Srnooth pursuit of a small target during sinusoidal stimulation is linear until about 80°/sec (Buizza et al. 1984, Meyer et al 1985). In contrat, we show in

Our results a strong vertical vergence saturation with peak stimulus velocities an order of magnitude lower than these (see Figure 2-15). The presence of cells in the midbrain which are associated only with versional or only with vergence eye movements argues against the idea that horizontal vergence movements are mediated by cells which respond to the motion of the stimulus in each eye (Mays et al. 1986). Furthemore clinical phenornenon such as intemuclear ophthalmoplegia which selectively disrupt conjugate adduction movements while sparing vergence suggesis that horizontal vergence 84 cornmands have a separate input to the motoneurones (Leigh and Zee 199 1). Sirnilar arguments cannot be put fonvard for venical vergence since the neurology of vertical vergence is unknown. A test of this idea would be to drive the eyes dichoptically with nonconesponding and nonfusible patterns which generate OKN when presented monocularl y. in these subjects 1 noted that the diplopia threshold calculated from predicted vergence error, given the measured gain, fell with increased frequency. The only study with which our diplopia measures cm be compared is that of Schor and Tyler (198 1). These investigators studied the fusionai range for sinusoidal temporal oscillations of disparity as a function of frequency using line stimuli. The fusional range decreased with temporal frequency for both horizontal and vertical disparity but to a larger extent for horizontal disparity. Their data are illustrated in Figure 2-10 for cornparison purposes. Our data seem to show a greater fall-off with frequency and a larger value at low frequencies. However the range of values for diplopia threshold reported in the literature is quite wide and the low frequency values 1 report are compatible with this range (Mitchell 1966, Duwaer and van den Brink 1981a). The fact that 1used a much larger display than Schor and Tyler ( 198 1 ) may partly explain the larger fusional area since fusional range is increased for large stimuli (Hebbard 1966, Erkelens and Collewijn 1985b).The rneasurement of diplopia thresholds is subject to the criterion chosen to define diplopia. Here 1 insisted on the lines appearing double. a conservative criterion that results in large diplopia thresholds. As temporal frequency increases. the amount of time that the stimulus appears diplopic decreases. However, this would tend to increase the fusional range rather than decrease it. Part of the difference between my results and those of Schor and Tyler (198 1 ) may be due to the indirect method used to calculate vergence emor in my diplopia measures. If 1 simply subtracted the peak vergence response from the peak stimulus level it would result in the minimum vergence error possible given the gain values at the interpolated frequency, and correspond to a phase lag of zero. However, at higher frequencies the phase lag was appreciable which results in a larger instantaneous vergence error than 85 expected from the gain alone (Figure 2-12). In the estimates of vergence error at the point of diplopia in Figure 2-10,I have corrected for phase error. This resulted in larger values of fusable vergence error at higher frequencies than caiculated from gain alone. These curves are not, however, significantly flatter than the uncorrected estimates and indicate a greater fail-off witb temporal frequency than found by Schor and Tyler (198 1). Whether this is a genuine difference or results from the indirect method of caiculating vergence error requires further investigation. Note that this phase error argument would also apply to the use of nonius line oscillation as a measure of vergence for sinusoidally oscillating stimuli. The motion of the nonius lines would be a measure of vergence error not vergence gain per se and if used as such would result in an overestimation of vergence gain. This may provide a partial explanation for the poor correspondence between search coi1 and nonius ljne measurements of vergence gain found by Howard et al. (1993). These considerations also imply that when using nulling of nonius line motion to measure vergence error a control must provided for adjusting oscillation phase as well as gain. One subject did not report seeing diplopia, regardless of stimulus frequency. Instead he saw sinusoidal motion of the whole surface suggesting suppression of one of the images. In the vergence trials, this subject tended to switch to monocular following of one half image at high amplitudes and frequencies, behaviour which also indicated suppression. This indicates that some subjects tend to switch from a vergence response to a monoculai response driven by a dominant eye when vergence demand becomes too severe. A similar observation has been made for horizontal disjunctive image oscillation at high frequencies (Erkelens and Collewijn l985a). What is the functional significance of the vertical vergence eye movements described in this thesis? At least three related roles can be postulated for vertical vergence: the maintenance of eye alignment; the bifoveation of local targets throughout the visual field and the calibration of conjugate vertical eye movements. A constant vertical disparity over the entire visual field is a vergence correctable vertical disparity that results from vertical misalignment of the eyes. This thesis has shown, and othen have demonstrared previously (e.g. Ogle 1964). that whole-field vertical disparity is an 86 adequate stimulus for vertical vergence. 1 have shown that vertical vergence movements for whole-field vertical disparity are relatively slow and show limited range. ui a related series of expenments we have shown that vertical vergence is dnven by a relatively large integration area in the centrai retina (Fang et al. 1997). These findings are in line with the hypothesis that vertical vergence is designed to deai with the sIowly changing whole field parameter of vertical eye alignment. As such it appean to integrate disparity information over time and space as a result of its large spatial integration area (20') and a relatively slow response to imposed disparity as compared to the horizontal vergence system. In a natural scene, constant vertical disparity over the whole field can only be due to eye misalignment. Furthermore. vertical disparity can only exist in a central fixation point when the two eyes are misaligned. In other parts of the visual field, however, there exist systematic patterns of vertical disparity as discussed in sections 1.2 and 1.3.5. For exarnple. changing gaze from a central position to targets in oblique positions requires vertical disjunctive movements. Ygge and Zee (1995) have shown that when a subject makes an eye movement to such a point most of the disjunctive motion is made through unequal sized saccades in the two eyes. These disjunctive movements are much faster than the vertical vergence reported in this study and in others (Houtman et al. 1977) and provide an adaptive mechanism to adjust vertical eye alignment during changes in gaze direction. These disjunctive saccades compensate for the major part of the required vertical disconjugacy but some slow vertical vergence following the saccade is evident (Lemij and Collewijn 199 1 ). However, these fast eye movements do not appear to be vergence movements in the classic sense in that they are not related to stimulus disparity. As evidence of this assertion, saccades remained unequal when made to merely remembered targets or when the natural disparity at the location was nulled by a prism (Ygge and Zee 1995). When the disparity was nulled by a pnsm, saccades identical to those made under natural conditions were made and slow vergence movements followed to compensate for the prism demand. Smooth pursuit eye movements cm also exhibit disconjugacy appropriate for the demands of position-specific vertical disparity (Schor et al. 1993a). Thus it appean that the majority of natural vertical disconjugacy required 87 during normal gaze changes is accomplished by pre-programmed disconjugacy in "versional" eye movements. The relatively slow vertical vergence movements act to compensate for any remaining vertical disparity in order to achieve bifoveal fixation. The finding that vertical vergence to pnsm disparity is enhanced for near viewing may be a result of its role in achieving bifoveal fixation (Lemij and Collewijn 1991). How does the relatively coarse spatial integration area for vertical vergence effect the ability of the system to achieve bifoveal fixation? In many natural environments smooth surfaces are present and vertical disparity is slowly changing. Averaging over space in these situations results in a low noise estimate of vertical disparity to drive vertical vergence. However, steep depth edges and transparency pose a potential problem. Vertical disparity changes with depth and large disparity differences can exist for points with similar visual direction that are separated in depth. A potential solution to this problem rnay be a selectivity to small horizontal disparities. Within a limited range of horizontal disparities, Say within Panum's area. vertical disparity changes relatively slowly over space making averaging for vertical vergence attractive. Jones and Stephens ( 1989) have provided evidence that horizontal vergence is affected by peripheral stimulation only over a small range of disparities. We may have to think of an integration volume for vertical vergence rather than an integration area with vertical disparity being averaged over a certain retinal area and for a certain range of horizontal disparity. We are currently investigating this idea further. A related role for vertical disparity vergence is in the calibration of disconjugacy in saccadic and pursuit eye rnovements. This adaptation can be regarded as a generalisation of phoria adaptation (Schor 1979). This adaptation can compensate for changes in the patterns of vertical disparities in the two eyes due to changes in the size of the globe, in terocular distance, oculomotor mechanics and retinal receptor distribution which occur with development and ageing. In man. these mechanisms are also important for dealing with the optical demands of spectacle Wear. Allen (1974) has reported complete adaptation to the prism demand of anisometropic spectacles. These glasses cause size differences in the two eyes leading to a requirement for position-specific 88 vertical vergence that increases with eccentricity. Lemij and Collewijn ( 199 1) have reponed that the major portion of this adaptation to anisometropic spectacles is in the fom of disconjugacy in saccadic and pursuit eye rnovements with some slow post- saccadic disconjugate drift evident. Kapoula et ai. (1996) attempted to adapt this drift specifically but found that it was not labile. Schor et al. (1993b) Getermined that this process is position specific and have measured the spatial spread of the phoria adaptation (tonic vergence). With a single adaptation point the change in phoria changed unifomly for fixation over the 18' test field and cornpensated for 60% of a 1.2S0 vertical disparity after 40 minutes adaptation. Adaptation to opposite vertical disparities at two points resulted in position specific phoria changes sirnilar to those seen following adaptation to aniseikonic lens (Lernij and Collewijn 1991). Phoria adapted differently and peaked appropriately at the two adaptation points with a gradua1 transition over the region of the visual field between them. If the points were placed too close together a resolution Iimit prevented them from being adapted independently and the peaks of adaptation were outside of the stimulated adaptation points (modelled as a Gaussian spread of adaptation with a sigma of approx. 6"). Similarly if the disparity gradient between the adaptation points was too large adaptation was reduced. Maxwell and Schor (1994) have shown that the phoria adaptation process first acts to lessen the disparity at one point in the field by an overall phoria shift and then phoria is adjusted at specific sites in the field. They have proposed a two stage process with a faster process to facilitate a global phona shift followed by a specific local process to adjust to position specific vergence demands. Phoria adaptation seems generalise to as a broadly as possible unless there is conflicting disparity information. It appears that vergence in the classical sense and as 1 have studied here provides the dnving force for horizontal phona adaptation (Schor 1979). Vertical vergence presurnably provides the motive force for vertical phoria adaptation as well. Adaptation of disjunctive saccadic and pursuit eye movements may also be driven by the stimulation of vertical vergence. These adaptive changes occur only if a binocularly disparate image is visible during or foliowing the eye movement (Schor et al. 1990). Vertical vergence presumably acts as a type of error signal which drives adaptive 89 mechanisms to adjust the conjugacy of "versional" eye movements as required to meet the demands of the changing optical environment. O v e O v

Amplitude of stimulus modulation (peak-peak) + 1" Horizontal Vergence * 3" i - 5" O 2" Cyclovergence - 0 6" r 12

O. 1 1 Frequency of stimulus modulation (Hz) Frequency of stimulus modulation (Hz)

Figure 2-1 1: Cyclo- and horizontal vergence gain (a) Gain and (b) phase of horizontal vergence and cyclovergence. Horizontal vergence was evoked by sinusoidül modulation of horizontal disparity of a 30" hy 30" textured püitern as a function of frequency of modulation for ttirce stimulus amplitudes. Adapied from Erkelcns and Collewijn ( 198%). Cyclovcrgeiice wüs evoked hy sinusaidal modulation of cyclodisparity of a 75" by 75" textured pattern as a funclioii of frcquciicy of iiiodulaiion for tlirce stiinulus üiiiplitudes. Adapted from Howard arid Ziicher ( 1 99 1 ). Figure 2-12: Vergence error due to phase error

Peak vergence enor is greater than predicted from vergence gain in the presence of non- zero phase lag. The example shows a sinusoidal stimulus with a unity gain response with 20" phase lag. Peak vergence error is 34% of the stimulus amplinide. Figure 2-13: Effects of area on different types of vergence eye movements

Effects of stimulus size on vergence gain vertical vergence, horizontal vergence and cyclovergence. a) Effects of size of a central stimulus. b) effects of occlusion of the central portion of a 65" textured stimulus. Data from Fang et al (1998).

+.O...... O-...... O ~.'"....

0-' +------O-- - /--- / / +Horizontal Vergence (0.1 Hz) / .-*o.-.Vertical Vergence (0.1 Hz) 4' -r - Cyclovergence (0.05 Hz)

0.0 1 1 1 1 I O 20 40 60 80 Size of Display (deg) (a>

Size of Occlusion (deg) Figure 2-14: Dependence of gain on stimulus velocity in horizontal, vertical and cyclovergence

Gain of vergence as a function of peak stimulus velocity for: 1) vertical vergence, 2) horizontal vergence, adapted from Erkelens and Collewijn (1985a) 3) cyclovergence. data calculated from Howard and Zacher (199 1).

A Cyclovergence a Horizontal Vergence r Vertical Vergence

Peak Stimulus Velocity + Horizontal Vergence ,.o.,Vertical Vergence O . , * Horizontal OKN -+ Horizontal Pursuit

. . o , Vertical Pursuit

111111 I 1 IIlia11 -< -200 0.1 1 o. 1 1 Frequency of Stimulus Modulation (Hz) Frequency of Stimulus Modulation (Hz)

Figure 2-15: Cornparison of vergence and version frequency responses

a) Gain and phase of horizontal and vertical vergence and version as a funciion of stimulus temporal frequency. Vertical vergence data is for the 1 .O0 pcak-peak condition. Horizontal vergence data is from Erkelens and Collewijn ( 1985a) for 1 .O0 peak-peak oscillation of a 30" hy 30" texturcd display. Horizontal OKN data is from Yüsui and Young (1984) for sinusoidal oscillation of ü 45" diameter optokineiiç stripe pattern (penk velocity of motion was less than 3S0/s). Horixontel and vertical pursuits gains arc froin Bnloli ct ;il ( 1988) for 20" pcak-peak oscillation of a siiiall Iüser spot. Figure 2-15 (conünued) b) Effects of stimulus velocity on vertical version and vergence gain. Vertical version data from Ogino (1991) for the constant velocity motion of an OKN pattern.

Vertical Vergence + - Vertical Upwards OKN -O Vertical Downwards OKN

Stimulus Velocity (de#) 3.1 Introduction

In Chapter 1, we noted that a horizontal size disparity produces the impression of a surface slanted in depth about a vertical axis (a right-wail or left-wail plane). A horizontal shear disparity produces the impression of a surface inclined in depth about a horizontal axis (a sky or ground plane). These effects are predicted from the geometry of binocular vision and thus Ogle (1938) called them geometric effects. Vertical size or shear disparity in an isolated textured surface also creates an impression of a surface slanted or inclined in depth (Ogle 1938, Howard and Kaneko 1994). These effects are not predicted from the projective geometry of real slanted or inclined surfaces. After Ogle (1938). we cal1 them induced effects because it is as though the gradient of vertical disparity in the image in one eye induces an equivalent gradient of horizontal disparity in the image in the other eye. Thus, the vertical size or shear disparity is converted into an equivalent horizontal size or shear disparity of opposite sign. It has been proposed that these vertical disparity mechanisms protect against anisei konia. differences in size due to eccentricity and cyclodisparity (see Section 1.35). Because vertical disparities change gradually over space. one would expect them to be averaged over wide areas of the visual field. This averaging should reduce the effects of local noise in estimating global parameters such aniseikonia or cyclorotary fixation disparity. Kaneko and Howard ( 1996, 1997) demonstrated that vertical shear and size disparities are averaged over large portions of the visual field. Similarly, one may predict that temporal averaging is employed to arrive at a stable estimate of parameters that change slowly over time. Whole-field vertical shear disparity results from cyclotorsional misalignment of the eyes. Rogers ( 1992) proposed that the inclination perceived in a display with vertical shear disparity could be due to cyclovergence transfonning the vertical shear disparity into a horizontal shear disparity. Rogers and Howard (199 1 ) have shown that vertical shear disparity is a strong stimulus for cyclovergence eye movements. If cyclovergence plays a role in the induced-shear effect, one may expect the temporal characteristics of the induced-shear effect to be determined by the temporal properties of cyclovergence. Cyclovergence is a slow eye movement system with little response at high temporal frequencies or short durations (Howard and Zacher 1991). Thus, one would expect the induced-shear effect to be limited at high temporal frequencies and at short durations. Although this is an attractive explanation, subsequent evidence has shown that cyclovergence does not provide a complete explanation for the induced-shear effect (Howard and Kaneko 1994). Even if cyclovergence does not play a major roie, one might expect the sensory mechanism that mediates the induced-shear effect to be relatively slow-acting if its role is to deal with the slowly changing parameter of cyclodisparity. Similarly, if the induced-size effect results from mechanisms designed to deal with aniseikonia - a slowly changing pararneter of the optical system - one would expect its response to be sluggish. It has been proposed (see Section 1.3.5) that vertical disparity patterns provide estimates of viewing system parameters such as gaze direction and vergence angle (or alternatively eccentricity and viewing distance). Averaging over time may allow for more robust estimates of these parameters. Thus. 1 hypothesised that vertical disparities are processed more slowly than horizontal disparities. According to this hypothesis. the contribution of vertical disparity to perceived slant and inclination should be weakened relative to that of horizontal disparity as temporal frequency is increased or presentation time shortened. The experiments described in this chapter investigate the percept of inclination and slant in depth induced by vertical and horizontal shear and size disparities as a function of temporal frequency and exposure time. 3.2 General Methods

3.2.1 Image Presentation

A Wheatstone mirror stereoscope sirnilar to that descnbed in section 2.2.1 was used to present dichoptic images. Translucent mylar screens were mounted to the left and right of the subject on the sides of the field coi1 frame and viewed through mirrors mounted at 90° from a distance of 93 cm. Images were rear projected ont0 these screens using two Electrohome EDP-58 monochrome projection monitors, one for each eye (Figure 3- 1 ).These are high resolution monitors with excellent geometry controls. This is important since calibration of the monitors is critical to maintaining a clean stereoscopic display. Alignment of the half images was achieved by superimposing a projected gnd pattem upon a real grid pattem which could be mounted on the side screens. Alignment was also verified against a physical grid pattem display located directly in front of the subject at 93 cm and viewed through the semi-silvered mirrors. The mirrors and projected images were aligned with spirit levels and plumb lines. That the half images were coplanar with, and at the sarne distance as, the alignment surface was verified by the absence r>f parallax between either the half images or the fused dichoptic image and the cornparison surface. Distortions and nonlineanties in the display were small. The screen subtended 65 by 75 degrees at each eye. The display was composed of 640 by 480 pixels (width by height). Sub-pixel positioning techniques were used to improve the positioning of stereoscopic elements. The luminance of the display set to range from 6 cd/m2 for white to less than 0.5 cdlm' for black. Luminance was reduced by approximately 50% after reflection off the semi-silvered mirrors. Stereoscopic stimuli were presented in a dark room and al1 surfaces were covered with matte black cloth or paint. In order to maintain a clean stimulus, care was taken to mask the monocular half images from being directly viewed, so that only the fused stimulus was visible through the mirrors. Figure 3-1: Stereoscope apparatus

Stereoscope used for the expenments in Chapters 3 and 4. The subject dichoptically viewed two large projection rear screens through rnirrors in a standard Wheatstone stereoscope configuration. Two computer driven projection moniiors (Electrohome EDP- 58) were used to display the half images. Viewing distance was 93 cm.

Eye Position Screen C

EDP-58 Projection Minors CRT . CRT

L

Frarne The image for most of the experiments was an irregularly textured black and white circula pattern subtending 60" of visual angle (see Figure 3-2). An irregular texture was chosen to minimise the effects of conflicting texture gradient cues. The image pairs were pre-computed and the base image transformed to produce various combinations of vertical and horizontal shear and size disparities. The percept was of a flat textured plane tilted out of the frontal plane. Size disparity produced a plane slanted about a vertical ais and shear disparities produced a surface inclined about a horizontal ais. In experirnent 1, the images were presented statically for various durations. In experiments 2 and 3. a sequence of frarnes (frame rate 15 Hz) produced modulation of shear or size disparities. Images were generated using custom C code on a Macintosh Quadra 900 from base image files. The base files were created using Adobe Photoshop on a IBM Penrium computer. Each type of transformation and each level of stimulus was generated separately and constituted a separate animation. Each frame of a given animation was generated off-line and stored on disk. Transformation mappings between the base image and each frame were calculated as exactly as possible using floating point arithmetic to sub-pixel precision. Standard (Foley et al. 199 1) antjaliasing techniques were then used to render a smooth image with sub-pixel accuracy and to reduce the aliasing effects of a finite pixel count. Shear disparities were of four types: horizontal shear. vertical shear, rotation and deformation (see Figure 3-3 and Gillam and Rogers 199 1, Howard and Kaneko 1994). Note that rotation cm be interpreted as horizontal and vertical shear in the same direction and deformation can be interpreted as horizontal and vertical shear in opposite directions. Four types of scale, or size, disparity were used: vertical magnification, horizontal magnification, deformation, and dilation (overall magnification). Dilation can be interpreted as horizontal and vertical size disparity in the same direction and deformation can be interpreted as horizontal and vertical size disparity in opposite directions. Since induced effects and geometric effects produce opposite depth, deformation disparities should produce a larger effect than generated by either component alone. The component effects should tend to cancel in the rotation and dilation conditions. The predictions of the hypothesis that vertical disparity is processed more slowly than horizontal disparity are illustrated in Figure 3-4. Vertical shear and size disparities are predicted to result in reduced slant at high temporal frequency. Consequently, with increasing temporal frequency, the vertical disparity component should decrease relative to the horizontal disparity component. This should reduce perceived depth in deformation-disparity conditions but increase perceived depth in rotation and dilation disparity conditions. Figure 3-2: Display used for inducing slant and inclination.

Irregularly textured display used for these experiments. Actual image is white on black rather than black on white. Figure 3-3: Shear and size disparity corn binations

The transformed half-images are shown siiperiniposed and represented by heavy and light grid patterns. Equal and opposite transfomiations were applied to cncli Iialf-image. Shear disparities: a) horizontal shear, b) vertical shear, c) rotation and d) shear-deformation dispnrity. Sizc disparities: e) Iiarizoiital sizc, f) vertical size, g) dilation and h) sizc-dcfonnation disparity. Figure 3-4: Predictions

Predictions of the hypothesis that gradients of vertical image disparity are processed more slowly than gradients of horizontal disparity for surface slant and inclination.

Relative Rediction Transformation Shear -Size

No hypothesiz- Horizontal ShearEize ed effect of frequency

Less depth at Vertical Shear/Size hig h frequency or short durations

More depth at high frequency or short durations with slant in the direction of the horizontal component

Less depth at ShearfSize Deformation high frequency (Horizontal -Vertical) or short durations 3.2.3 Measures of Surface SIant and Inclination

Subjects matched the perceived slant or inclination of the disparity surface with that of the subsequently displayed real surface. This red surface was textured with the same pattern as that of the test surface and subtended 32". The comparison surface contained a variety of depth cues to its tme orientation. Although a surface defined only by disparity could be used, I used a real surface with al1 depth cues present in order to achieve the most reliable and repeatable percept of surface slant. The surface was supported on a gimbal mounting and could be rotated about either a horizontal or vertical axis by the subject. using a long steel rod. Following each presentation of a test surface. the real surface was illuminated and subjects adjusted its slant or inclination to match the perceived slant or inclination of the test surface. After the subject indicared the surface was appropriately adjusted, calibrated voltages from potentiorneters attached to the slant and inclination axes of the test surface were read into a computer. If the comparison surface tilt was defined only by simulated slant in the computer dispiay. this procedure could easily be modified for two alternative forced choice measurements. For Our threshold measurements 1 did not present a alternative choice for a forced choice rneasure but relied on the fact that the subjects could compare the surface slant with a strong nom, a fronto-parallel surface. Thus. in the threshold measurements. subjects were forced to choose whether the test surface appeared slanted right side towards them or away from them. In the nomenclature developed in the introduction. the subjects chose between classifying the surface as a right wall plane or a left wall plane. Theoretically this is considered a one interval (yes-no) discrimination or recognition task (McMillan and Creelman 1991). The matching task allows for a response following presentation of the stimulus but not during the presentation. A visible matching surface present during stimulus presentation (Gillam et al. 1988a) may interfere with perception of the test stimuli: in the case of vertical disparity patterns it rnay destroy the percept (Howard and Kaneko 1994: Kaneko and Howard 1996). Nulling of perceived inclination by injecting a compensatory disparity (e.g. trading honzontal disparity for vertical. Ogle 1964) is an option for a simultaneous measure of inclination but changes stimulus disparity during the measurement. 3.3 Experiment 1: Time Course of Slant and Inclination Perception

The purpose of this experiment was to measure the tirne course of the build-up of the percept of surface slant and inclination. The relative contributions of vertical and horizontal size- and shear-disparity mechanisrns were evaluated by studying the response to combinations of honzontal and vertical gradients of disparity. According to the hypothesis outlined above, the effects of the horizontal disparity component should become evident sooner than those of the vertical disparity component.

3.3.1 Method

The irregularly textured display described above was presented initially with zero disparity. This caused it to appear as a flat, frontal surface. A constant gradient of disparity was dien added to the display to cause it to rotate in depth out of the frontal plane. Horizontal shear. vertical shear. rotation and deformation disparities were used to induce inclination. Vertical magni fication. horizontal magni fication, deformation and dilation disparities were used to induce slant. Two levels (0.73" and 1-46" of shear disparity or 1.28% and 2.56% of size disparity) and both directions of disparity were used for each of the transforms. These levels were used for both the horizontal and vertical components of the rotation. dilation and deformation disparity transformations. The test stimulus was presented for 0.1, 1 .O, 10.0 or 30.0 S. after which the comparison surface was illuminated. The subject matched the slant or inclination of the comparison display to the final perceived slant or inclination of the test surface. Four subjects with normal binocular vision were studied. Each stimu1us. level and duration combination (8x4~4) was presented eight times over four sessions in randomised order. Presentation order was counterbalanced across sessions for each subject. 3.3.2 Results

Figure 3-5 and Figure 3-7 show perceived inclination and slant. respectively. as a function of disparity for various exposure durations averaged across the four subjects. For each subject, responses were normalised by dividing the judged slant or inclination by the theoretical slant or inclination predicted from the horizontal size or shear disparity component (10" and 20" for shear or size disparity of 0.73" or 1.288 and 1.46" or 2.56% respectively). Figure 3-6 and Figure 3-8 show these normalised responses collapsed across disparity level and plotted as a function of duration for each subject. Note that this procedure introduces some additional variability. This is because it ignores idiosyncratic differences between sky and ground responses and the differences in response gain between the two stimulus levels. For example. for inclined surfaces the subjects tended to respond more strongly when disparity specified a ground plane than a sky plane, leading to a somewhat asymmetric response. Results for slant and inclination are discussed separately below.

3.3.2.1 inclination For each horizontal shear disparity, perceived inclination increased significantly with exposure duration (Figure 3-5). Inclination was underestimated at al1 exposure durations, but more so at short durations. Perceived inclination increased with increased horizontal shear disparity as expected. Figure 3-6 shows that subjects HJ and XF typically saw some inclination even at the shortest O. 1 s duration. Subject JZ perceived no inclination and subject RA perceived inclination in a direction opposite to that predicted for 0.1 or 1.O s presentations of horizontal shear disparity. At longer durations. al1 subjects saw inclination in the predicted direction. At 30 s exposure time. inclination was typically underestimated for horizontal shear disparity and was less than the predicted values of 10 and 20" for the 0.73 and 1.46" shear conditions. For vertical shear disparity, only subject XF perceived any depth in the shortest

0.1 s presentation. The other three subjects perceived no inclination at the 0.1 S. and JZ reported no inclination at the 1 .O s presentation. Perceived depth increased with exposure time and disparity level as in the horizontal shear disparity conditions. In subject HJ, there was little difference between the 10 and 30 s presentation durations which suggests that responses saturare. In the other three subjects, however. depth continued to build over the 30 s exposure. There is little suggestion in the slopes of the duration functions that the response to vertical shear is appreciably slower than the response to horizontal shear (except perhaps for subject HJ). At the shonest durations. however. subjects were more apt to report inclination for horizontal shear than for vertical shear. This perhaps indicates a shorter latency for horizontal shear disparity. For rotation disparity al1 subjects reponed little depth. regardless of exposure time or stimulus level. The predicted transient response in the direction of the horizontal shear component was not observed. For shear deformation. perceived inclination was typicall y larger than for ei ther horizontal or vertical shear disparity. At exposures of 10 and 30 S. the response io deformation was close to the sum of responses to the horizontal and vertical disparity components. At the shortest duration, the response to deformation disparity was similar in size to the response to horizontal shear disparity. Occasionally. subjects responded with a depth match in the opposite direction to that predicted by disparity - a so-called slant revend (Gillam 1967. see also Stevens and Brookes 1988). 1 counted a response as a slant or inclination reversal if the normalised response exceeded 0.05 and was opposite to the predicted direction. 1excluded rotations since the predicted inclination is zero. Across the other three conditions. reversals occumed significantly more often for shon durations (O. 1 and 1.0 s) than for long durations (10 and 30 s), occumng in 41 of 384 trials and in 6 of 384 trials respectively

(XZ = 27.76, df = 1, p < 0.001). Specifically, reversals occurred significantly more often for the two short durations for horizontal shear (X' = 20.57 1, df = 1. p < 0.00 1) and shear deformation cases (X' = 8.258. df = 1. p < 0.0 1 ) but not for vertical shear. However. the low frequency of reversals for deformation (8 of 256 trials) makes frequency analysis problematic. Reversals were also significantly more common for horizontal shear disparity than for vertical and deformation shear disparity. occurring in 32 of 256 trials for horizontal shear disparity cornpared with 7 and 8 of 256 trials for vertical (X' =

17.347. df = 1. p < 0.001) and deformation (X' = 15.62. df = 1, p < 0.001) disparities. In summary. for the four subjects. the percept of inclination built up slowly over durations up to 30 s for horizontal. vertical and deformation shear disparities. Analysis of variance indicated a significant effect of exposure duration, disparity magnitude and their interaction on the response for horizontal, vertical and deformation shear disparity conditions (p < 0.01). The nature of the interaction was that perceived depth built up more slowly with larger disparities even after normalisation by disparity magnitude. None of these parameters had a significant effect on the response to rotation disparity. Regression analysis did not demonstrate a significant difference in the effect of exposure duration for horizontal versus vertical shear disparities. Little depth was reported at 0.1 s presentation time. For rotation. little depth was seen even at 30 s duration. titi 3.3.2.2 SZanr The trends for slanted surfaces were generally similar to those for inclined surfaces. On average (Figure 3-7) subjects perceived more slant for each horizontal size disparity as exposure duration increased. As expected, perceived slant increased with increased horizontal size disparity especially for longer durations. Figure 3-8 shows that three subjects tended to see slant in the correct direction even at O. 1 s durations, although it was greatly underestimated. One subject (RA) tended to perceive the slant in the wrong direction for these very shon durations. A11 subjects saw slant in the correct direction for longer presentations. For 30 s exposure time, slant matches were the largest but still fell short of theoreticai values. I did not find the well known anisotropy between slant about a vertical axis and inclination about a horizontal axis; perhaps because of the small sarnple size. For vertical size disparity. perceived slant increased with exposure time and disparity level as in the horizontal size disparity data. Only subject XF perceived depth in the shortest 0.1 s presentation. Slant built up over the 30 s exposure of the vertical size disparity stimulus for al1 4 subjects. The slopes of the horizontal and vertical size disparity versus duration curves do not suggest a difference in the temporal propenies of the iwo responses. There was some evidence that the latency for vertical disparity is somewhat longer since subjects were more apt to report slant at short durations when the stimulus contained horizontal rather than vertical size disparity. However, there is also a suggestion in the data that the vertical size disparity mechanism is faster since the curves for dilation disparity go in the direction of the venical component for brief durations in three of the four subjects. Otherwise the dilation disparity results are generally Bat and show little depth for al1 exposure times and disparity levels. The predicted transient response in the direction of the horizontal size component was not observed. For size deformation, the response was typicaily larger than that for either the horizontal or vertical size disparities and was close to their sum at 10 and 30 s exposure durations. Excluding dilations. where predicted slant is zero, depth reversals occurred more often in the 0.1 and 1 .O s durations than in the IO and 30 s durations, occumng in 26 and Il3 7 of 384 trials respectively = 1 1.38. df = 1, p c 0.001). Specifically, the frequency of depth reversals was significantly higher for the two short durations than for the longer

durations for horizontal size disparity (X2 = 16.43, df = 1, p c 0.01) but not for vertical and deformation size disparities. In the size deformation disparity case, the low frequency of reversals makes frequency analysis have little statistical power. Slant reversals were rarest for size deformation, occumng in only 2 of 256 trials. significantly less than for

horizontal disparity (X2 = 14.35, df = 1, p 4.00 1 ). Reversals were slightly more cornmon in horizontal size disparity trials than in the vertical disparity trials occurring in 19 versus 12 of 256 trials respectively but this difference was not significant = 1.654, df = 1. p >O. 1). Most of the reversals for slant were reponed by one subject (RA). As 1reported above, dilation disparity tended to result in matched depth corresponding to the vertical component for short durations and to the horizontal component for long durations if slant was perceived. Frequency analysis confirmed that depth in the vertical direction was

reported more often for short durations than for long durations (X' = 3 1.352. df = 1, p c

0.00 1 ). In surnrnary. for al1 subjects. the percept of slant built up over durations up to 30 s for horizontal. vertical and deformation size disparities. Analysis of variance indicated a significant effect of exposure duration. disparity magnitude and their interaction on the perceived slant produced by horizontal. vertical and deformation size disparity (p < 0.01). When the responses were nonnalised, a significant magnitude by duration interaction still existed indicating a somewhat slower build-up of perceived slant for larger disparities (p < 0.05).Regression analysis did not demonstrate a significant difference in the effect of exposure duration for horizontal versus vertical shear disparities. For dilation, little depth was seen at long durations. A srnaIl transient response in the vertical direction was found in three subjects. This was reflected in a significant effect of exposure duration on the

response to dilation disparity (p ~0.01 ).

3.4 Experiment 2: Slant Thresholds as a Function of Modulation Duration

This experiment measured the effects of oscillation period on the ability to discriminate the direction of surface slant. The prediction of Our hypothesis is that sensitivity to slant will decline more with temporal frequency for vertical disparities than for horizontal disparities.

3.4.1 Methods

Vertical and horizontal size disparity were temporally modulated. Four types of scale disparity were used: vertical magnification. horizontal magnification, deformation (differential horizontal and vertical magnification) and dilation. The disparity profile was such that the surface slanted in one direction only. to a maximum disparity, before returning to the zero position. The temporal disparity profile of the animation was a single cycle raised cosine:

Y,,-(1-cosf2Igl)) O

The slant thresholds obtained are illustrated in Figure 3-9. As peak disparity is increased, subjects responded with more certainty. Typically this resulted in subjects correctly discriminating the direction of slant. However, for short horizontal shear disparity presentations, one subject had a tendency to respond that slant appeared opposi te to the expected direction (slant reversal). There was no support for the hypothesis that sensitivity to slant from vertical size disparity declines with temporal frequency relative to that from horizontal size disparity. The subjects had relatively constant deformation and vertical disparity thresholds versus temporal frequency. For horizontal disparity. thresholds were constant or the subject became confused about the direction of slant and certainty decreased (threshold rose) with temporal frequency. Both subjects showed a consistent decrease in threshold for slant from dilation disparity. 1predicted that the contribution of the vertical size disparity would decline with increasing temporal frequency which would result in slant from the horizontal dispai ty becoming more detectable. However. although the subjects responded with more certainty to dilation disparity at higher temporal frequencies, responses were in the direction of the vertical component rather than the horizontai. Specifically, for each subject. the results are as follows. Thresholds for the subject not prone to slant reversals were relatively constant over frequency for horizontal size, vertical size and deformation disparities at about 0.2%. This corresponds to a disparity gradient of approximately 0.12 minutes of arc per degree. Thresholds for vertical size disparity were higher than thresholds for horizontal size disparity. Thresholds for deformation disparity were slightly lower than the thresholds for both vertical and horizontal size disparity. The decrease in threshold for deformation disparity was greater than expected from probability summation. However it was Iess than expected for slant from vertical and horizontal size disparity being additive. For dilation disparity. thresholds were large at low frequencies (about 2%)and fell with increasing temporal frequency to approach levels found for the other disparity patterns (see Figure 3-9). The results for the second subject were similar except for horizontal size disparity. As temporal frequency increased the subject became less certain about her responses and tended to respond opposite to the predicted direction of slant. One subject prone to slant reversals in Experiment 3 was also shown these stimuli. However this subject tended to perceive surface slant in the wrong direction for horizontal disparity and meaningful thresholds could not be obtained. In Figure 3-10, I have shown similar inclination threshold data from Bridge et al. ( 1996). Inclination thresholds in two subjects were relatively constant over temporal frequency for horizontal shear. vertical shear. and shear-deformation disparities. For rotation, one of the subjects showed a decline in threshold for rotation disparity for the middle two frequencies studied. Rotation thresholds could not be achieved for the other subject. The results did not demonstrate a significant difference in the effect of temporal frequency on inclination thresholds between horizontal and vertical shear disparities. Figure 3-9: Slant thresholds

Slant thresholds as a function of the temporal fiequency of the raised cosine (carrier fiequency of single cycle raised cosine) for two subjects.

4 Horizontal Expansion ...*-. Vertical Expansion q- Dilation Size Deformation

Frequency (Hz)

0.0 0.2 0.4 0.6 0.8 1 .O Frequency (Hz)

120 Figure 3-10: Inclination thresholds

Inclination thresholds as a function of the temporal frequency of the raised cosine (carrier frequency of single cycle raised cosine) after Bridge et al. (1996).

4 Horizontal Shear ...... Vertical Shear *-Rotation 4.Shear Deformation

Frequency (Hz)

No rotation thresholds achieved (Subject was insensitive to rotation)

Frequency (Hz) 3.5 Experiment 3: Efficiency of Slant and Inclination Perception as a Function of Temporal Frequency

The purpose of this experiment was to use supra-threshold matching to map the relative temporal frequency sensitivity of visual mechanisms responding to vertical and horizontal size and shear disparities. I also studied the response to combinations of horizontal and vertical size disparities (size deformation and dilation disparity) and to combinations of shear disparities (shear deformation and rotation disparity). This allowed for evduation of the relative potency of the horizontal and vertical components as temporal frequency was varied.

3.5.1 Method

For these measurements. five cycles of sinusoidal oscillation were presented at each of five frequencies in random order (O. 1 12.0.225.0.45.0.90, 1.8 Hz). Subjects perceived a flat textured plane which sinusoidally changed its inclination or slant with respect to the frontal plane. The eight size and shear disparity transformations were the same as those used as in Experimenr 1. The magnitude of the disparity oscillation was 1.46" peak shear disparity or 2.589 peak size disparity. Subjects indicated the sign of inclination (or slant) by verbally reporting when the surface was a sky or ground plane (or right wallAeft wall plane) a task which could be performed only for the lowest three frequencies. Following the presentation of each frequency, the subject matched the slant or inclination of a visible comparison display (not seen during the test display) to the funhest and nearest extent of the depth oscillation of the test surface. The subjects saw each stimulus combination twice over two sessions. The perceived surface slant or inclination as a function of temporal frequency of disparity gradient modulation was obtained. Eleven subjects with normal binocular vision were studied. In three subjects. an extended frequenc y range was studied with stimuli presented at lower temporal frequencies of 0.0 187.0.037, and 0.075 Hz. In these three subjects, 1 also looked at the response to stimulus oscillation of horizontal size and shear disparity in the presence or absence of a zero dispanty sunound. In this stimulus, the irregularly 122 textured disk was divided into a central disk subtending 32" visual angle and an annular surround separated by a 5" black region. The central disk was subject to the disparity oscillation while the outer annulus had a constant zero disparity.

3.5.2 Results

The results required treating the subjects as two separate groups. Eight of the 1 I subjects comprised the first group. They perceived the surface oscillating in phase with the horizontal and vertical disparity transformations. Since there was no significant differences between the two directions of slant or inclination estimates, the data were collapsed across direction. Repeated measures analysis of variance found a significant interaction between frequency and transform type for the pooled inclination data (F(12.84) = 8.46, p < 0.0 1 ) as well as main effects of frequency (F(4.28) = 14.40. p <

0.0 1) and transform type (F(3.2 1 ) = 15.64. p < 0.0 1 ). Regression analysis showed that in these subjects perceived inclination declined significantly (p < 0.01) with increased frequency for horizontal. vertical and shear-defornation disparity transformations (see Figure 3-1 1). There was no discemible difference between the sensitivity to temporal frequency for the horizontal and vertical shear transformations either in the ensemble or individual data sets (Le. the slopes were not significantly different). Perceived inclination for rotation disparity was small and tended to peak slightly at mid-frequencies. Individuals in this group also showed trends consistent with the group averaged data. Regression analysis on individual data indicated a negative slope versus frequency for horizontal, vertical and deformation shear disparities although the trends did not reach significance in some individual data sets. Results for slant in these eight subjects were similar but the effects were not as strong. Since there was no significant difference between the right wall and left wall estimates the data were collapsed across direction. Repeated measures analysis revealed a significant interaction between frequency and transform type for the ensemble slant data

(F(12,84) = 3.33, p < 0.0 1 ) as well as main effects of frequency (F(4,28) = 4.70, p < 0.0 1 ) and transform type (F(3.21) = 8.24. p < 0.01). Regression analysis showed that in these subjects. perceived slant declined significantly (p < 0.05) with increased frequency for horizontal, vertical and defonnation size disparities (see Figure 3- 12). There was no discemible difference between the sensitivity to temporal frequency for the horizontal and venical size components (the slopes were not significantly different). Perceived slant for dilation disparity was small at high and low frequencies and tended to peak at mid- frequencies and was in the direction of the vertical size disparity cornponent. Individuais in this group also showed trends consistent with the pooled data. Regession analysis on individual data typically indicated a negative dope versus frequency for horizontal. vertical and defonnation shear disparities. However. the weaker effects for the slant case resulted in many of these trends not being significant in the individual data. The three subjects comprising the second group perceived the inclination or slant of the surfüce opposite to the predicted direction (slant reversa1 effects) for oscillating horizontal gradients of disparity. Thus when the horizontal size disparity corresponded to a surface slanted right side nearer they reported that it appeared left side nearer. Similarly. they reversed the sign of the perceived inclination in oscillating horizontal shear disparity patterns. These subjects had a response that differed markedly from that of the subjects who saw the slant direction veridically. They did not tend to see slant or inclination falling with frequency: instead they saw very little depth at the lowest frequency. which stayed the same or increased slightly with frequency. I tested one of these subjects (along with two of the other group) over an extended low frequency range. This subject was not prone to depth reversais at very low frequencies of oscillation (less than 0.01 Hz) and saw depth at these very low frequencies which declined with increased frequency (Figure 3-13). Thus, this subject appeared to show a similar response to the other subjects but shifted to a much lower frequency range. None of the three subjects saw depth reversals for oscillations of horizontal size or shear disparity in a central stimulus in the presence of a zero disparity annular visual surround (Figure 3- 14). The matched depth for the subject prone to depth reversais (as well as for the two individuals from the other group) was higher than observed for the isolated stimulus in the main experiment and declined with increased temporal frequency in the presence of the surround. Note that the centra1 disk in 1 24 this stimulus was considerably smaller than that used in the main expenment. With this smaller disk presented in isolation this subject still experienced depth reversals and did not have a strong impression of depth. With oscillating dilation disparity. al1 subjects in both groups saw slant in the direction corresponding to the vertical size disparity component. Note that this is opposite to the typically small slant seen with static dilation disparity and opposite to the direction predicted with Our hypothesis. With static or oscillating rotation disparity, al1 subjects who saw inclination saw it in the direction of the horizontal component. Inclination or slant seen in the oscillating displays with horizontal, vertical or defornation shear or size disparity was smaller than that seen in static presentations and less than the theoretical values (20" for horizontal disparity). Average levels of slant and inclination for static presentations are typically larger than the slants and inclinations observed in the dynamic displays. Figure 3-11: Inclination matches as a fwiction of temporal frequency

Inclination matches as a function of temporal frequency for horizontal shear disparity, vertical shear disparity, shear-deformation disparity and rotation disparity. The curves represent the averaged response (I SE of mean) of eight observers.

Shear Disparity

-0- Horizontal Shear Vertical Shear -- Rotation + Deformation

I Frequency (Hz) Figure 3-12: Siant matches as function of temporal frequency

Slant matches as function of temporal frequency for horizontal size disparity, vertical size disparity, size-deformatiori disparity and clilation disparity. The curves represent the averaged response (A SE of mean) of eight observers.

Size Dispanty Horizontal Expansion Vertical Expansion + Dilation + Defornation

0.1 Frequency (Hz) Figure 3-13: blatched slant and inclination in observer prone to slant reversals

Slant (a) and inclination @) matches in subject who was prone to perceiving slant in the opposite direction to bat predicted by disparity (slant or inclination revends). Reversed slant is represented in the figure by negative slant or inclination.

Size Disparity + Horizontal Magnification -P Vertical Mâgnification + Dilation + Deformation

-1 0 1 Frequency of Disparity Modulation (Hz)

Shear Disparity

-O- Horizontal Shear Vertical Shear + Rotation + Deformation

-10 J Frequency of Disparity Modulation (Hz)

(b)

128 Figure 3-14: Matched slant and inclination in presence of reference

Slant (a) and inclination @) matches as a function of temporal frequency for the subject in Figure 3-13. The stimulus was oscillation of a central 30" disk in the presence or absence of an annular stationary surround. Reversed depth was seen by this observer for the isolated stimulus.

20 1 Horizontal Sire Disparity +With Background 2.56% * No Background 2.56% -With Background 5.1 2% -No Background 5.12%

- Horizontal Shear Disparity +With Background 1.46" -O- No Background 1.46" -With Background 2.92" +F No Background 2.92"

l 0.1 1 Frequency of Dispanty Modulation (Hz) 3.6 Discussion

3.6.1 Processin2 of Horizontal versus Vertical Disparity Gradients

These experiments failed to support the hypothesis that gradients of vertical disparity are processed more slowly than gradients of horizontal disparity. In experimenr 1, the build-up of the percept for vertical shear and size disparity was not appreciably slower than the build-up for horizontal shear and size disparity. For inclination. there is a suggestion that the vertical shear-disparity mechanism is slower since, at the shonest durations, subjects were more apt to report inclination for horizontal shear than for vertical shear. In experiments 2 and 3, I could find no evidence of a difference in sensitivity to temporal frequency in direct cornparisons of the vertical and horizontal frequency responses. A potentially more sensitive method of comparing the temporal characteristics of the horizontal and vertical disparity gradient mechanisms is to study the response to stimuli that contain combinations of the two components. The hypothesis predicts that the response should become dominated more by the horizontal component as temporal frequency increases or exposure duration is shonened. The most straight-fonvard tests are the dilation or rotation cases in which the horizontal and vertical components indicate opposite directions of slant or inclination. As predicted, the depth response to dilation disparity tended to increase with increased temporal frequency in experiment 3, typically peaking at mid-frequencies. However, the perceived slant was in the direction of the vertical rather than the horizontal cornponent. Similarl y, dilation disparity tended to evoke slant responses in the direction of the vertical component for short exposure durations in experiment 1. These results do not support the proposal that the horizontal size disparity mechanism has a higher temporal bandwidth than the vertical size disparity mechanism. Rather they suggest that the vertical size disparity mechanism has a higher temporal bandwidth. This provides some evidence that the response to vertical size disparity is faster than the response to horizontal size disparity. Altematively the vertical size disparity may potentiate the slant reversa1 effect for horizontal size disparity. For inclination. there was a small increase in response in the direction of the horizontal component for rotation disparity with increased temporal frequency or decreased presentation time. This suggests that the vertical mechanisrn may be slower in the shear disparity case, although the effect is weak. Slant discrimination thresholds as a function of frequency were relatively flat for both horizontal and vertical disparïties. The decline in suprathreshold depth with temporal frequency observed for horizontal and vertical shear and size disparities in Experiment 3 was not reflected in decreased sensitivity at threshold in Experiment 2. Detection thresholds may not be generalisable to suprathreshold depth perception in a straight-forward manner. h agreement with this interpretation. there appear to be significant differences in the temporal frequency dependence of thresholds for motion in depth and in the perceived depth of suprathreshold disparity oscillation. Tyler (1 97 1) found that the threshold for motion in depth was minimal ai about 0.5 to 1 .O Hz and was degraded for higher or lower temporal frequencies. Regan and Beverley (1 973) found that threshold for motion in depth declined with increased temporal frequency from O. 1 to 3.3 Hz. Regan and Beverley (1973) found that suprathreshold depth does not show the response attenuation at low temporal frequencies suggested by the threshold data. In summary, 1found no strong evidence to support the hypothesis that vertical disparity gradients are processed more slowly than horizontal disparity gradients either in time course or frequency response measures at threshold or suprathreshold disparities. The response to dilation disparity suggests that vertical size disparity may be processed more rapidly than horizontal size disparity. 1 will discuss this result further in a later section.

3.6.2 Time Course of SIant and Inclination Percept

Gillarn et al. (1984) measured the latency of slant perception for horizontal size disparity using a monocular rnatching task. Their operational definition of latency was the time at which the rnatched slant exceeded 50% of the final value. Latencies were 15 and 25 s for their two observers for 5% horizontal magnification. This result suggests a relatively slow development of the slant percept. Van Ee and Erkelens (1996) measured the time course of slant and inclination perception using methods similar to those used here for horizontal shear and size disparities in large isolated displays. 1have confimed their findings that weak slant and inclination are perceived for presentation durations of less than 1 S. with the percepts eventually developing over a period of up to 30 S. 1 have extended these observations for various combinations of vertical and horizontal size and shear disparity. 1 have found a similar slow development of the percepts of slant and inclination for stimuli with vertical size or shear disparity. When horizontal afid vertical disparity gradients are combined and specify the same direction of slant or inclination (defonnation dispanty) the build-up of the percept is similarly slow. Gillarn et al. (1988a) have shown that the post-fusional latency for identification of one of seven slant or inclination configurations was longest when the horizontal shear or size disparity transformation was applied to the entire stimulus. particularly for slant. Latencies were reduced and perceived slant and inclination were larger when the test surface was presented along with a reference stimulus containing a different gradient of horizontal disparity. Presence of a reference surface is not expected to aid in the processing of vertical disparity since vertical shear and vertical size disparities are averaged over large portions of the visual field (Kaneko and Howard 1996. 1997). Somewhat surprising is the slow build-up of depth for horizontal shear and size disparity. Severai studies have demonstrated that the percept of slant or inclination is weak for horizontal size and shear disparity in the absence of a visual reference (Brookes and Stevens 1989, Gillam et al. 1988a. van Ee 1995). This has been interpreted as an insensitivity of the visual system to low spatial frequency changes in disparity. This interpretation is supported by the existence of an analogue of the Craik-O'Brien- Cornsweet illusion in the disparity domain (Anstis et al, 1978). In contrast. in these studies, discontinuities in disparity were well perceived. The visual system appears to be especially sensitive to abrupt changes in relative horizontal disparity and relatively 132 insensitive to absolute disparities or constant gradients of absolute disparity (Gillam et al. 1988a). In this experiment, 1 have shown that an abrupt temporal change in horizontal size or shear disparity does not result in a rapid percept of surface slant or inclination. Thus, it appears that a temporal disparity change cannot substitute for spatial change in disparity gradient in enhancing the slant response at short exposure durations. One caveat is that the disparity cue in Our experiments, as well as those of Gillam et al. (1988a), was in conflict with other depth cues, especially perspective, which were consistent with a frontal surface regardless of disparity. Stevens et al. (1991) have provided anecdotal evidence that under conditions of cue conflict, gradients of disparity are relied on more as viewing time increases. The possible effects of disparity-perspective conflict are investigated funher below and in Chapter 4. The response to deformation disparity is ideally the sum of the response to horizontal and vertical disparity components. This additive relationship holds approximately at long exposure durations but fails at short durations where the response is close to the values found for horizontal disparities. This non-additivity could be the result of longer time being required to process larger slants or inclinations. In agreement with this interpretation, 1 found that the build-up of perceived slant and inclination was slower with larger disparity although the effect was not large.

3.6.3 Effects of Temporal Frequencv

Amplitudes of temporally oscillating inclination and slant were underestimated at the frequencies used. In addition, perceived slant and inclination declined with increased temporal frequency for horizontal, vertical and deformation disparities in 8 of 1 1 subjects. This suggests that stereoscopic processing is Iimited at high temporal frequencies. Richards (1972) found that the apparent depth of a test bar oscillating in depth, as measured with a depth probe. declined with increasing temporal frequency of disparity oscillation over a range similar to that studied here, for disparities less than 0.5". For larger disparities apparent depth was maximal for frequencies of approximately 1.O Hz. Regan and Beverley (1 973) measured the amplitude of disparity oscillation required to match the depth of a simultaneously presented static disparity. They found that. for frequencies up to 2-3 Hz, the depth produced by a given amplitude of sinusoidal oscillation of disparity was nearly equal to that produced by static disparity of the same amplitude. Above this frequency. perceived depth fell steeply with increasing frequency. This suggests that attenuation of depth in stereopsis occurs at much lower frequencies than temporal limits for luminance flicker. However, Regan and Beverley's results predict that slant perception is relatively unaffected by temporal frequency over the low range used in Our experiment. What is the explanation for the relatively low temporal frequency limit for slant and inclination perception found in this study? It may be that a constant gradient of disparity is treated much like an absolute disparity (absolute disparity gradient) to which the visual system is comparativeiy insensitive (Gillam et al. 1984, Mitchison and Westheimer 1984,1990). h the two studies discussed above, targets were presented against a textured background (Regan and Beverley 1973) or in the presence of fixation targets (Richards 1972). Our results support the proposa1 that relative point disparity between two targets or a target and the background is well perceived. while a constant gradient of disparity is not. However, as we shali see. the high frequency roll-off may result from cue conflict rather than intrinsic limits in stereoscopic processes.

3.6.4 Cue Conflict

The hypothesis put fonvard in Section 3.1 proposed that vertical disparity patterns are processed more slowly than horizontal disparity patterns. In studying the effects of both viewing time (Experiment 1) and temporal frequency (Expenment 2 and 3) 1found no clear difference between the processing of horizontal and vertical disparity. However, perceived inclination for modulations of dilation disparity was in the direction of the vertical component. This suggests that the vertical size disparity mechanism has a higher temporal bandwidth. An alternative explanation may be that cue-conflict is more potent for horizontal size disparity than vertical size disparity. Little is known about the mechanisms of size constancy and cue conflict under the induced effects. Cue conflict may also be significant in interpreting some of the other findings. I found that amplitudes of temporally oscillating inclination and slant are underestimated at the frequencies used and for short presentations. Perceived slant and inclination declined with increased temporal frequency for horizontal, vertical and deformation disparities in the majority of our subjects. However, considerable individual differences existed. The most striking of these differences is the tendency for some subjects to report slant and inclination in the opposite direction to that predicted by disparity. The experiment described in the next chapter suggests that much of the variability and apparently paradoxical results may be explained by cue conflict. The fact that 1used a large display may have contributed to the strong perspective cue conflict. Blake et al. (1993) have recently shown. using an ideal observer model. that texture cues to surface inclination (compression and density) become more reliable with larger stimuli. Buckley et al. (1996) found that four of six observers experienced increased surface inclination as field of view increased from IO to 30" when texture specified an inclined surface and dis pari ty speci fied a fron ta1 surface. When both texture and disparity specified an inclined surface, increase in stimulus size also resulted in an increase in perceived inclination. In the present study 1 required large displays to achievs robust induce-size and induced-shear effects (Kaneko and Howard 1996, 1997). Several subjects experienced the percept of slant in the reversed direction for brief or high frequency presentations of horizontal and vertical shear and size disparity. This suggests that perspective-disparity conflict resolution has a temporal component. As GiIlarn (1967, 1993) has pointed out, a possible explanation for the slant reversal effect is that it is an example of a size-distance paradox (or more descriptively a shape-slant paradox) resulting from size constancy. Nomally if a homogeneously textured disk is inclined or slanted in depth, there is a gradient of image size from near to far. In the present experiments, objective average texture size, density and overall pattem shape were constant over the pattem and the same for al1 disparities. Size constancy predicts thai if the visual subtense of an object on a retina (or with respect to the cyclopean eye) is constant, apparent size will be scaled by apparent distance. If size constancy mechanisms 135 operate, texture elements and the pattern as a whole should appear smaller in the near part of the disk and larger in the far part (Figure 1-5). Anecdotally, we find that if subjects perceive slant or inclination in the direction predicted by disparity then they also perceive this apparent gradient of texture size and density. Note that the apparent texture gradient is opposite in direction to the texture gradient resulting from a homogeneously textured surface slanting or inclining in the direction specified by disparity. Thus, the apparent texture gradient and disparity give conflicting information about the direction, or sign, of slant or inclination. Some subjects see the surface according to disparity with the shape disrorted. Others seem to use the apparent perspective and see the surface sloping in the opposite direction. The findings that at least one subject prone to reversals was able to resolve this conflict wi th extremely low temporal frequencies and that al1 subjects showed an appropriate stereoscopic response with 30 s static presentations suggest that the conflict has a motion or temporal cornponent. We will look at this issue in more detail in the next Chapter. In Experiment 3, a zero-disparity reference enhanced the perception of disparity gradients with more slant or inclination reponed. Furthemore, a reference stimulus seemed to help disambiguate perspective-disparity conflict and slant reversals were not observed. Cue conflict is presumably less important at threshald where the size changes predicted by constancy would be small. This may explain why slant thresholds were relatively constant over temporal frequency in Experiment 2. However, subjects still appeared to be prone to depth reversa1 effects, even near threshold. 4 THEDYNAMIC INFLUENCE OF PERSPECTIVEAND ''SLANT

The variability in the data and the presence of subjects who saw slant reversals suggest that conflicts between perspective and stereopsis played an important role in the experiments described in Chapter 3. Perhaps the subjects who see depth reversals put more weight on perspective cues than those subjects who do not see reversals. 1 studied slant and inclination perception with various combinations of disparity and perspective information under static and kinetic conditions. 4.1 Methods

Two stimulus pattems were used: a portion of the irregular textured pattem used in Chapter 3 and a rectangular grid pattem (Figure 4-1). Both pattems subtended 32' of visual angle. These stimuli were chosen to provide different levels of disparity- perspective conflict. The grid was composed of regularly spaced lines and provided strong linear perspective, aspect ratio and foreshonening cues to surface slant. in contrast. the irregular texture provided mainly weak texture gradient information (due to the irregularity) and some overall outline aspect ratio cues. These stimuli were transformed to present various combinations of perspective and disparity cues to slant or inclination. The perspective projections were computed for the viewing distance and were constmcted as projections through the cyclopean eye (mid- point of the two eyes) of a slanted or inclined plane ont0 the screen. Horizontal shear and size disparity transformations were applied to the pattems to induce slant and inclination as descri bed in the previous experiments. Perspective and disparity were combined in four ways: 1) disparity appropriate for the tilt in depth and no perspective (disparity-alone transformation), 2) perspective appropriate for the tilt and no disparity (perspective-alone transformation), 3) disparity and perspective concordant and appropriate for the tilt, and 4) disparity and perspective in conflict specifying the same magnitude of tilt but in opposite directions. For these measurements, stimuli were presented either statically for 30 s or for five cycles of sinusoidal oscillation at 0.45 Hz. For sinusoidal stimulation the peak disparity and/or perspective corresponded to a peak predicted slant or inclination of 20". Disparity or perspective for static presentations dso corresponded to theoretical slant or inclination of 20". For the moving stimuli. subjects reported when the surface appeared to be a sky or ground plane (or right wall/left wali plane) and the timing of these reports was related to the sign of the stimulus disparity. Following the presentation of each frequency, the subject matched the slant or inclination of the visible comparison display (not seen during the test display) to the furthest and nearest extent of the depth oscillation of the test surface. For static presentations, subjects matched the final slant or inclination of the test surface following each stimulus presentation. I also asked the subjects to report if the percept changed in any way over the 30s observation time. Ten of the eleven subjects who participated in Experiment 3 of Chapter 3 were available for this study. Two of these subjects could only participate in the trials with the irregularly textured displays (including one of the subjects who was prone to slant reversals). For fair comparison between the grid and the irregularly textured display, these results were not included in cornparisons of rneans for various conditions or in the averagd data in Figures 1.2 and 4.3. The other eigh~subjects were presented each stimulus condition in random order over two sessions. Figure 4-1: Displays used to induce slant and inclination a) Irregularly textured display used for these experiments. b) Grid pattern stimulus. Actual images were white on black rather than black on white. 4.2.1 S tatic Presentation

For both inclination and slant, there were qualitative differences between the responses for irregularly textured and grid patterns. For static presentation, repeated measures analysis of variance indicated significant (p c 0.01) effect of type of disparity- perspective transformation and a significant interaction between transformation condition and pattern type for both slant and inclination. The slant and inclination estimates for the subjects are tabulated in Tables 4- 1 and 4-2. Magnitude estimates were averaged across the two directions of slant and inclination for each of the subjects. Concordant Displuys. Mean perceived slant and inclination were stable and largest when disparity and perspective were concordant (see Figures 1.2 and 4.3). Average slant estimates were 17.78" + 2-13' (mean values shown here with 195% confidence intervals) and 16-76" r 3.04O for the grid and textured pattern respectively. Average inclination estirnates were 16.41 r 2.86" for the grid pattern and 15-33" r 2.85' for the textured pattern. Both slant and inclination were close to but significantly less than the veridical value of 30". Perspective Alone Disphys. When the irregularl y tex tured surface had on1 y perspective specifying depth, subjects reponed that the depth was transient and fadrd away over the 30 s observation period. This was the case for ten of ten and eight of ten subjects for inclination and slant respectively. The three subjects who were prone to depth reversals perceived slants averaged over the two directions ranging from 1.5 to 12.8" and inclinations ranging from 1.9 to 1 1.3". In contrast. al1 subjects who did not report depth reversals in the previous experiment reported small inclinations (with six of the seven subjects reporting less than 2"). For slant, these latter subjects showed more variation with a range of zero to 10.4" and with only two subjects reporting no slant. For the grid pattern, the effects of perspective appeared stronger and more consistent. Al1 eight subjects saw slant or inclination in th2 direction specified by perspective with a range of 3.4 to 19.4" for inclination and 7.9 to 19.9' for slant. The 140 percept appeared much more stable than for the irregularly textured surface. Inclination faded for only two subjects and slant faded for only one subject. Figures 4.2 and 4.3 show that. on average, static perspective of the gnd pattem produced significantly greater slant and inclination than did static perspective of the irregular texture. DisparipAlone Displays. Under the disparity-alone transformation of the irregular pattern, a few subjects (two and three for inclination and slant respectively) reported that the depth built up over the 30 s observation period. The others subjects did nor remark on any changes over tirne. Two of the three subjects who were prone to depth reversals perceived inclinations in the correct directions (at the end of the 30 s) averaging 6.1 and 10.4" and slants averaging 5.1 and 10.0". The other subject gave inclination and slant responses in the opposite direction to that expected from disparity. The subjects who did not report depth reversals in the previous experiment tended to report strong depth for horizontal shear and size disparity, typically more than 10' of inclination or slant. With the grid pattem, little depth was seen when horizontal disparity alone specified slant or inclination (Figures 4.2 and 4.3). The two subjects prone to slant reversals who viewrd this stimulus saw small slants in the reversed direction to that expected from the disparit),. Figures 4.2 and 4.3 show that, on average, perceived slant and inclination were significantly smaller for the static disparity-alone transformation in the grid pattem than in the irregular texture. Conflicting Displays. When the perspective and disparity cues specified conflicting depth, subjects showed increased inter-subject variation in their responsrs. particularly for inclination. For slant, nine of the ten subjects reported that the irregularly textured pattem ini tiall y looked slanted in the direction indicated by perspective but that the percept gradually switched over to slant in the direction indicated by disparity. Two of the three subjects who saw slant reversals earlier had this percept but perceived only a small (less than 2") slant after 30 S. The other subject prone to depth reversals continued to see the surface slanted in the direction of the perspective component but noted that the slant faded somewhat. For the grid pattem, however, the results were different. All subjects except one (who perceived no depth in this stimulus) had a stable percept of slant in the direction specified by perspective. The results for inclination with conflicting cues were similar. Six of the ten subjects reported a reversa1 of inclination from the direction specified by perspective to that specified by disparity as time progressed when viewing the irregularly textured pattern- One subject reported that the depth was initially in the direction of the perspective component and faded over time. The effect of the perspective cue in the irregularl y tex tured display appeared stronger for slan t than inclination since three of the subjects saw the depth in the direction of the disparity component immediately for inclination whereas none did for slant. Two of the subjects who were prone to depth reversals saw a srnaIl amount of depth in the disparity direction at the end, after initially seeing inclination in the perspective direction. The other subject prone to reversals saw strong (approx. 20°), stable inclination in the perspective direction throughout the 30 s period. For the grid pattern. results were quite varied, with one subject not seeing any inclination, five (including the two subjects prone to depth reversals) seeing it according to perspective and two seeing it according to disparity.

4.2.2 Oscillatinp; Presentation

When the stimulus was a 0.45 Hz sinusoidal oscillation of disparity andor perspective, the results were similar for both the grid and textured surfaces. For the oscillating presentation of slant and inclination, repeated rneasures analysis of variance indicated a significant (p < 0.0 1) effect of only type of disparity-perspective transformation and non-significant effects of pattern type or interaction between transformation condition and pattern type. The slant and inclination estimates for the 10 subjects are tabulated in Tables 1 and 2. Since there was no evidence of directional bias. the magnitude estimates shown for each subject correspond to the average of the slant or inchation over both directions. When disparity and perspective specified the same slant, al1 subjects perceived slant and inclination that was nearly veridical on average (Figures 4.2 and 4.3). Thus. when kinetic disparity and perspective agreed a strong percept of changing depth was attained. When the two cues were in conflict, kinetic perspective cues dominated perceived depth for al1 subjects. When equal and opposite slant oscillations were specified by disparity and perspective, the subjects saw slant in the direction of the perspective cue which was on average close to the theoretical value of slant from the perspective transformation. For the perspective-alone transformation, similar results were found, with the subjects perceiving nearly veridical slant or inclination in phase with the perspective oscillation. The average slant and inclination perceived in the oscillating perspective-alone condition was significantly higher (p ~0.05)than in the static perspective-alone condition. For an oscillating stimulus. al1 subjects perceived less slant or inclination under the disparity-alone transformation than under the perspective-alone transformation. The three subjects prone to depth reversals perceived slant of the opposite sign to the disparity component while al1 the other subjects perceived slant according to the disparity. One of the latter group of subjects saw slant in the direction of the disparity for the irregularly textured pattern but in the opposite direction for the grid pattern. Figure 4-2: Peispective and stereo interactions - inclination judgements

The chart shows mean inclination estirnates for the imgularly texnired stimulus or the regular grid stimulus. Perspective information was concordant with, in conflict with, or specified no inclination when disparity information specified an inclined surface or perspective specified inclination when disparity was zero. Positive inclination indicates perceived inclination in the direction specified by the perspective or disparity cues. For the conflict situation positive values of inclination indicate perceived depth in the direction specified by disparity (and opposite to perspective).

r//l GRd Static 0llrtegular Texture Static 5Grid 0.45 Hz trregular Texture 0.45 Hz

Disparity

w- c 0 .-c. O Cu -.E -5 - Shear Concordant Perspective cO Disparity Perspective+ Alone AIone Disparity -1 5 Figure 4-3: Perspective and stereo interactions - slant judgements The chart shows mean slant estimates for the irregularly textured stimulus or the regular grid stimulus. Perspective information was concordant with, in conflict with. or specified no slant when disparity information specified a slanted surface or perspective specified slant when disparity was zero. Positive slant indicates perceived slant in the direction specified by the perspective or disparity cues. For the conflict situation positive values of slant or inclination indicate perceived depth in the direction specified by disparity (and opposite to perspective).

t- t- Grid Static 17lrregular Texture Static BGrid 0.45 Hz Irregular Texture 0.45 Hz Conflicting 20 15

10 5

O -5 Perspective Disparity Perspective+ Alone Table 4-1: Inchation estimates

Inclination estimates for static and oscillating presentations of concordant, conflicting, disparity-alone and perspective-alone stimulus in 8 observers. For the conflict situation positive values of inclination indicate perceived depth in the direction specified by disparity (and opposite to perspective). Table 4-2: Siant estirnates

Slant estimates for static and oscillating presentations of concordant, conflicting, disparity-done and perspective-alone stimulus in 8 observers. For the conflict situation positive values of slant indicate perceived depth in the direction specified by disparity (and opposite to perspective). 4.3 Discussion

In al1 subjects, kinetic perspective cues dominated perceived depth. This was true whether disparity supported the changing slant indicated by perspective, confl icted w i th perspective, or was held constant. In al1 cases, perceived depth was relatively unaffected by disparity when perspective was changing. Kinetic disparity did induce oscillation in depth of the irregular pattem with perspective held constant (disparity-alone condition). However, not al1 subjects perceived depth according to disparity under this condition. This suggests that conflict with perspective occurs even here. Cue conflict was not elirninated by the perspective-alone and disparity-alone conditions since the unchanging cue was set at zero but not eliminated. In the disparity-alone condition the unchanging perspective would indicate unchanging slant. If a size constancy mechanism were operating, the resultant apparent change in texture gradient would indicate changing slünt in the opposite direction. As in Experiment 3 of Chapter 3. several subjects respondcd in this manner. In the perspective-disparity conflict situation. the conflict may be accentuated by apparent changes in the texture gradient induced by the changing disparity, which would support the objective change in the perspective gradient. This may explain why large responses were obtained in the conflict trials with moving stimuli. The manner in which perspective-disparity conflict was resolved under static conditions was affected by the type of pattem, axis of rotation and individual differences. Subjects saw weak, transient slant and inclination under the perspective-alone transformation of the irregularly textured stimulus. When strong perspective information was present (grid pattem) the percept of depth in the direction of the perspective componeni was stronger. Perspective appears to be given a higher weighting for slant than for inclination, as indicated by the persistent slant in the direction of perspectise in the grid pattern under conflict conditions. Subjects tend to perceive more depth for surfaces defined by horizontal shear disparity than by horizontal size disparity (Rogers and Graham 1982, Mitchison and McKee 1990, Mitchison and Westheimer 1984) al though considerable intersubject variability exists (Mitchison and Westheimer 1990). This insensitivity to size disparit y could explain why the perspective cue was relatively more important for slant than for inclination judgernents. On the other hand, it is possible that perspective information is more salient for slanted surfaces and that this factor plays a role in the anisotropy. Ryan and Gillam (1994) noted that the effects of adding static perspective cues were much stronger for slant than inclination. This anisotropy was also found by Buckley and Frisby (1993) for texture-disparity conflict in the depth of ridge stnictures. Our results confirm that conflicting static perspective information has more influence on slant than on inclination. Whether this anisotropy is due wholly to the relative ineffectiveness of size disparity (Wallach and Bacon 1970, Rogers and Graham 1982, Gillarn et al. 1988a) or in part to a particular saliency of perspective in slant perception is presently unclear. In this study, when disparity conflicted with kinetic perspective. perspective was dominant for both slant and inclination and no anisotropy was found. Presumably. kinetic perspective completely ovenvhelmed both types of disparity. Subjects occasionally reported slant or inclination in the opposite direction to thar predicted by disparity. Gillam ( 1967. 1993) hainvestigated this phenornenon most thoroughly. She has proposed two explanations. One is based upon size constancy effects as described in section 3.6.4. In agreement with this proposal, Gillam (1993) has show and 1have confirmed in this study that slant reversa1 effects are most pronounced when the stimulus is configured to provide strong perspective information. This size constancl. explanation is apparently paradoxical. The apparent texture gradient is only a consequence of the stereoscopic relation between depth and disparity but the depth predicted by disparity is not perceived by the subject. That the disparity-induced changes in apparent size can be perceived in the absence of corresponding changes in perceived depth suggests that depth and size judgements are both driven by registered disparity but are not causaliy Iinked. Similar explanations have been proposed for the moon illusion (Kaufman and Rock, 1962) and the paradoxical size distance effects seen with convergence micropsia (Ono et al, 1974). Erkelens and Regan (1986) have shown that subjects shown open-loop absolute disparity changes over the whole field do not perceiw 149 changes in egocentric distance of the target surface. However, subjects do perceive the target becoming apparently smaller for increased crossed disparity and larger for uncrossed dispanty. Regan et al. (1986) have reported that these size changes do not tend to cause the surface to oscillate in depth in the opposite direction to that predicted by disparity. However, the conflict with apparent size changes may inhibit the percept of motion in depth caused by changing disparity. These size-distance paradox effects result from zero-order (constant) size-disparity conflicts. Our data and those of Gillam (1967) suggest that sirnilar slant-shape paradox effects can occur for first-order (gradient) size- disparity conflicts. The differences in this experiment between the responses of observers who experienced slant reversals (see Section 3.4) and those who did not suggest a bias towards perspective cues for the former. However. this distinction is subtle. These subjects tended to see more depth in the direction of the perspective component or relatively small amounts of depth in the direction of the disparity cornponent under conditions of conflict. As well, they generally tended to see relatively less depth in the irregularly textured stimulus when slant was defined by disparity-alone and more depth when it was defined by perspective-alone. These results suggest that these subjects are especiall y sensitive to perspective. The slant-shape paradox explanation of slant reversals requires that these subjects are also more likely to interpret the apparent perspective arising from disparity as depth. Subjects experiencing reversed slant typically do not notice any size distortions. a finding consistent with their attributing the size changes to depth (Gillam 1967). Gillam proposed a second explanation of slant reversals based on the idea that the horizontal size disparity of eccentric displays is calibrated by perspective (Gillam 1993. see also Frisby et al. 1995). The theoretical locus of zero disparity. the horopter, is the Vieth-Muller circle passing through the nodal points of the two eyes and the fixation point. Horizontal size disparity specifies the local slant of a surface relative to the horopter. The perspective gradient specifies stant with respect to the normal to the cyclopean line of sight; a measure Gibson called optical slant (Gibson and Comsweet 1952). This corresponds to slant relative to the tangent to a circle centred on the I5O cyclopean eye. Both this circle and the Vieth-Muller circle curve inward with increasing eccentricity. Thus, if the frontal plane acts as a nom for the subject. the tangents to both the horopter and the equidistant circle (the normal to the line of sight) have non-zero geographical slant (Gibson and Comsweet 1952) for al1 eccentric positions. The radius of curvature is smaller for the horopter and thus the slant relative to the frontal plane for the tangent to the horopter is nearly twice that of the slant of the normal to the cyclopean line of sight (Figure 1-6). This relation implies that the eccentricity of a surface can be uniquely specified by the optical slant indicated by perspective in combination with the slant relative to the horopter indicated by horizontal size disparity. In a sense, Gillam's theory proposes that the optical slant frorn perspective constrains the possible slant- eccentricity combinations. A non-zero horizontal-size disparity in the presence of a zero perspective :radient signifies a surface with an angle of eccentricity equal to the angle by which the surface is slanted with respect to the horopter. Under these conditions, the slant relative to the horopter, specified by the size disparity. is equal to the slant of the normal to the cyclopean line of sight relative to the horopter for the indicated eccentricity. Up to this point. this explanation is similar to theories of space perception used to explain the induced-size effect (eg. Ogle 1964, Petrov 1980, Mayhew and Longuet-Hi_g_oins1982. Gillam and Lawergren 1983). Gillam (1993) further proposes that, since the normal to the line of sight is slanted with respect to the frontal plane, the end percept is that the surface is tilted in the opposite direction to its slant relative to the horopter - a slant reversal. This theory of slant reversa1 is quite appealing although it relies on the sornewhat unconventional use of slant relative to the frontal plane rather than optical slant as the percept. This makes the theory inconsistent with most theories for the induced-size effect based on processing the surface âs if it were eccentrically placed (c.f. Ogle 1964 however see GiHam et al. 1988b). Whatever the merit of this theory for static slant reversals it cannot explain Our dynamic resuits. I have noted reversal effects for both horizontally and vertically orirnted gradients of horizontal disparity. The Gillam (1 993) theory is incapable of explaining 151 reversed depth for inclination. In static conditions, Gillam found reversals were more common for slant than inclination which led her to reject the slant-shape paradox explanation for the static case (Gillam 1993). However, our results show that reversals can occur for inclination as well, especially under dynamic conditions. An extension of the Gillam (1993) theory to the inclination case does not seem to be possible. For inclination, the theoretical vertical point horopter is a vertical line in the frontal plane. Slant with respect to the frontal plane is thus always specified by the shear disparity of the surface regardless of vertical eccentricity (after correction for the Helmholtz shear, see Nakayama 1977). While the Vieth-Muller circle lies within the equidistant circle. the vertical horopter lies outside it. Thus the predictions for a Gillam type theory are opposite for slant and inclination, and geographic inclination is in the same direction as inclination to the horopter. While this theory may help to explain the anisotropy found in the incidence of slant reversals under static conditions, an analogous argument cannot account for inclination reversais under dynamic conditions. The differences in the resolution of disparity-perspective conflict between the moving and static conditions in the present experiment are striking. This suggests that changing perspective (motion or kinetic perspective) is much more compelling than static perspective. This may account for the fact that Gillam's (1968) subjects (see also Ames 1946) had great difficulty in nulling the slant introduced by aniseikonic lenses by adjusting the actual physical slant of the stimulus. The resulrs were unreliable because of "the odd appearance of the moving contours viewed through the lens". This forced her to choose an alternative matching measure. The lenses introduced apparent size distortion appropriate for the disparity (Gillam 1967) that caused the apparent perspective changes to conflict with the disparity. If kinetic perspective dominates kinetic disparity, as 1 have found, it is little wonder that Gillarn's subjects had difficulty performing the nulling tasEr. A few studies have exarnined the effect of kinetic perspective on depth perception directly. Hillebrand (1 894 cited in Ittleson and Ames 1950) showed that viewing a continuously expanding aperture in othenvise dark surrounds results in the percept of an approaching object of constant size. Regan and Beverley (1978) have shown that a two- 153 dimensional image of an object undergoing continuous isotropie expansion resulis in a strong percept of an object moving in depth. In contrast, differences in the size of static geometnc stimuli give only vague inconsistent impressions of depth differences (Epstein et al. 196 1). Pong et al. (1990) presented a random-dot display in which a square defined by a subset of the dots was made to move in depth. In one condition, the motion in depth indicaied by disparity was opposite that indicated by the monocular motion of the dots (radial expansion). With the lowest temporal frequency of 0.0437 Hz oscillation, disparity prevailed and determined the direction of motion in depth: at a higher temporal frequency of 0.175 Hz, the motion in depth corresponded to the optical flow. Regan and Beverley

( 1979). have shown that the weighting of change-in-disparity as cue to motion in depth relative to a conflicting change-in-size cue increases with inspection time. These results suggest that. for motion in depth, monocular cues are increasingly relied on as duration is decreased or temporal frequency increased. 1 have found that the resolution of cue conflict in slant perception has a similar temporal dependence. When the stimulus moves, the velocity gradients of changing perspective provide sufficient information to allow for determination of the surface slant given the assumption of rigidity (Braunstein 1968). Gibson and Gibson (1957) used cast shadows to present changing perspective in the absence of changes in other depth cues and found that changes in perspective resulted in a much stronger percept of slant than static perspective. Our data support the conclusion that motion perspective is a powerful cue to changing surface slant and inclination. The information provided by changing perspective is related to motion parallax. Parallax (both motion and binocular) arises frorn a change in vantas point over space or time. The optic flow field ansing in motion parallax is a result of changing perspective due to relative motion between the observer and the 3-D scene. In our stimulus the motion was in the depth dimension rather than orthogonal to it as is typical in motion parallax studies (e.g. Rogers and Graham 1982; Ono et al. 1986). Our results indicatr ihat this type of change in perspective (whether referred to as motion parallax or kinetic perspective) is particularl y effective in determining change in surface orientation in depth. 153 This distinguishes it from lateral motion parallax which has variously been found to be roughly as effective as binocular disparity ( Rogers and Graham 1979 ) or relatively ineffective (Gibson et al. 1959. Rock 1984) as binocular depth cue. The kinetic depth effect is another phenornenon which is related to the changing perspective effects 1have found here. In the kinetic depth effect, relative motion in the frontal plane results in a percept of a three-dimensional rotating object (Wallach and O'Connel1 1953). Both the kinetic depth effect and motion parallax are instances of 3-D structure from motion (Ullman 1979). In the stimulus used in this experiment the motion is a perspective transformation while this is not necessarily the case for generalised structure from motion. For example, the silhouette of a rotating bent piece of wire appears as a three-dimensional rotating object even if viewed under parallel projection (Wallach and O'Connel1 1953). Simple relative lateral motion between two sets of dots is sufficient for segregation of the dots into separate depth planes (Mace and Shaw 1974). Under parallel projection. structure from motion is ambiguous and a unique solution for the depth and rotation relationship does not exist even under the assumption of rigidity (cf. the Necker cube illusion). Recently, Eagle and Hogervorst (1 997) have described some of these issues and have exarnined the role perspective projection can play in disambiguating structure from motion. The irregular and grid patterns elicited significantly different responses. indicating that Our attempts to change the degree of perspective-disparity conflict were successful. Several earlier studies have demonstrated that stimulus configuration can have a profound effect on the resolution of dispari ty-perspective conflict. Gibson ( 1950) reported that subjects consistently underestimate the slant of surfaces defined by a texture gradient in the absence of other cues. He noted that this regression to the frontal plane was much stronger for irregular textures than for regular textures. One effect of texture irregulari ty is to add noise to estimates of texture gradient. Young et al. (1 993) have provided evidence that under cue conflict, percepts shift to the more reliable cue when noise degrades information from the other. Gillam (1968). Gillam and Ryan (1992). and Ryan and Gillam (1994) studied disparity-perspective cue conflict in slant perception using 154 static pattemed stimuli. The stimuli were horizontal andfor vertical lines with perspective specifying a frontal surface and disparity specifying slant or inclination. The patterns containing real or irnplicit contours perpendicular to the aisof stereoscopic rotation in depth showed a stronger attenuation effect of perspective than those containing contours parallel to the ais of rotation. Slant responses in monocular viewing were also stronger when lines with the appropriate perspective were perpendicular to the axis of rotation than when they were parallel to it (Gillarn 1968). They concluded thai linear perspective (convergence of parallels) is a more powerful cue to surface orientation than compression or foreshortening. We are currently looking at the effects of differences in the types and ainount perspective information prwided in pattemed and textured displays under dynamic and static conflict conditions. 5 GENERALDISCUSSION

Many investigators have emphasised the importance of spatial changes in disparity as the primitives of stereopsis. In this thesis 1 studied the temporal characteristics of the oculomotor and perceptual responses to various spatial patterns of disparity. 1 hypothesised that the responses to vertical disparity would be slower than the response to horizontal disparity. This hypothesis was only partly confirmed. Vergence is driven by the absolute disparity of a target. In this thesis it was show that vertical vergence can be driven effectively by oscillations of absolute vertical disparity. At low frequencies and amplitudes, vertical vergence gain is near one and phase lag is near zero. The frequency response has a low pass charactenstic; gain falls and phase lag increases with increasing temporal frequency. Gain also falls with increased stimulus velocity, which is evidence of a nonlinearity. Most eye movement rnodels have representations of the input and motor output signals coded in terms of retinal or eye velocity (Robinson 198 1). The velocity dependent non-linearity reported in this thesis suggesrs that nonlinearities in the processing or representation of these signals may be a limiting factor in vergence. Further study of the open- and closed-loop dynamics of vertical vergence may help to illuminate this issue. The characteristics of vertical vergence make it well suited to cornpensate for modest ampli tude, slowly changing disturbances in verticai eye alignment. Slower plastic mechanisms of phoria adaptation (tonic vergence) can compensate for even larger static changes in the demands on vertical vergence over the period of minutes to days (Ellerbrock 1949a, Schor 1983). Rapid vertical vergence associated with shifts in gaze to targets away from the horopter appears to be pre-programmed and independent of disparity. These fast and slow mechanisms of maintaining vertical eye alignmeni ma} be related to the vertical disparity vergence studied here. Schor (1 979) has proposed that horizontal phoria adaptation occurs when there exists a persistent demand for vergence correction. Similarly, vergence appears to drive the adaptation of the disconjugacy of saccadic and pursuit eye movements made to points off the horopter. Thus, at least three mechanisms maintain the vertical alignment of the eyes. Disconjugacy of "versional" eye movements provides pre-programmed correction for the disparity introduced during changing fixation. Vertical disparity vergence calibrates this process and corrects for disturbances and errors in eye alignment. The disparity vergence studied here is well suited for dealing with modest errors in vertical eye alignment. Persistent demand for these corrections results in phoria adaptation to compensate for disturbances in venical eye alignment due to growth, ageing, injury or the demands of optical devices. Horizontal disparity vergence can be driven by the local absolute disparity of a small target. In contrast verticai vergence is driven by disparities averaged over a large (20") portion of the central retina (Fang et al. 1997). The large integration area and the modest amplitude range for vertical disparity vergence are compatible with vertical disparity in natural scenes. Vertical disparity changes in a natural scene are smaller than the horizontal disparity changes. Furthemore. the vertical disparities in a surface are relatively unaffected by local surface structure (Mayhew 1982). However, vertical disparity 1s affected by changes in distance and eccentricity. At surface discontinuities and in transparency the distance between adjacent elements can be large. Averaging dispcirity over the elements in these cases would lead to an intermediate estimate of dispority rather than the disparity of the fixated surface. In our laboratory. we have found that a small target spot cannot be fused when it is embedded in a textured background containing vertical disparity. We are currently studying the spatial properties of the vertical vergence system using cornpetitive stimuli such as these. With large differences in distance between two transparent surfaces. the vertical disparities in elements comprising the surface that is not fixated would exceed the range that can be detected and processed. Thus. vertical disparities could be integrated over space to drive vergence without averaging over elements with large differences in vertical disparity. However, another consideration may prevent disparity averaging of even more modest vertical disparities. In a natural environment, differences in vertical disparity between adjacent elements in distinct depth planes would be accompanied by larger changes in horizontal disparity. Vertical disparity may be averaged over a large retinal 157 area but only over a range of horizontal disparities near that of the fixation plane. Thus. it may be more appropriate to speak of an integration volume for vertical vergence rather than an integration area. This selectivity for horizontal disparity would allow spatial integration of disparity for robust vertical vergence combined with insensitivity to vertical disparity in targets located at djfferent depths. This is also physiologically plausible since disparity detector neurones respond only to a limited range of disparity (Poggio and Fischer 1977). Thus, vertical disparities of targets with large horizontal disparity cannot be registered. Vertical vergence dynamics may also be affected by eccentricity of fixation. The situation studied in this thesis - vertical disparity in images during central fixation - can occur only with misalignment of the eyes. Oblique fixation requires vertical versence to compensate for vertical disparity. For oblique fixation, vertical vergence demand is also affected by fixation distance. The differences in the normal demands on vertical vergence may cause differences in dynamics of vertical vergence in oblique fixation. Another interesting question is whether an analogue of the transient and sustained components of horizontal disparity vergence exists for vertical vergence. 1 would like to study the open loop properties of vertical disparity vergence to investigate this issue further. The human visual system is not especially sensitive to absolute retinal disparity or to vergence mediated distance information. Westheimer ( 1979) found chat depth discrimination was approximately two orders of magnitude worse for sequenrially presented absolute disparity than for simultaneously presented relative disparity. Recently. Rogers (1997) has demonstrated thar discrimination is similarly poor for absolute retinal disparities presented under open-loop vergence conditions, Le. when the disparities are held constant regardless of vergence. It is uncenain in these experiments whether it was the disparity itself or the vergence movement initiated or suppressed that gave rise to the weak depth sensation. Absolute retinal disparity can be created or eliminated by vergence eye movements. Relative disparities are essential disparities and are affected less by vergence. The reliance on relative disparity in binocular depth perception leads us to consider whether there are classes of disparity change that are particularly salient. We have seen that zero-order disparity transformations have little effect on perceived depth although they do effect perceived size and depth scaling. In this thesis. 1 studied the perceptual response to first-order transformations of disparity. Gradients of horizontal disparity occur when the subject views a slanted or inclined surface. We have seen that the perceptual response to these types of patterns develops slowly and the sign of the response is sometimes inverted when they are viewed in isolation. Gradients of horizontal disparity. like absolute disparities, do not appear to be particularly salient for stereoscopic depth perception. When a reference surface is in view, change in slant or inclination is well perceived. Higher-order changes in disparity appear to be particularly salient. Westheimer and Mitchison ( IWO) have argued that disparity curvature or second-order disparity changes may be particularly salient for stereopsis. Also discontinuities in the disparity field tend to be well perceived. Man (1982) has argued on theoretical grounds that discontinuities in the representation of an image are particularly relevant for vision. Discontinuities in disparity are useful in defining and segregating surfaces and objects. It would be interesting to study systematically how the temporal sensitivity of the perceptual response to disparity curvature or disparity discontinuity compares with the response to zero- or first-order disparity transformations. Based on the work of Gillam cr al. ( 1 %Ba), 1 predict that the stereoscopic response to disparity discontinuity would be more immediate than the response to first-order disparity transformations. Texture discontinuities should result in a strong percept of surface segregation. However. 1 think that the depth response to monocular texture discontinuities would be less robust than the response to disparity discontinuities under cue conflict conditions. Both zero- and first-order disparity transformations result in weak depth percepts but rather strong percepts of changing size. A strong percept of a texture gradient is typically induced into an inclined or slanted surface defined by horizontal shear or size disparity. These size changes are sirnilar to and may refieci a first-order analogue of 159 convergence micropsia. In Chapter 4.1 found that a texture gradient is a more salient depth cue for moving stimuli or for short presentations of a step change in slant or inclination. Depth from perspective is also enhanced under cue-conflict conditions with moving stimuli and short durations. Thus, we may expect that the conflicting perspective bas the strongest modulating effect on perceived depth at higher temporal frequencies or for shorter presentations. The dependence of stereoscopic slant and inclination perception on temporal frequency and viewing time (see Chapter 3) may reflect this temporal sensitivity to perspective, at least in part. Thus we should qualify the conclusion that depth from first-order disparity is weak by the noting that it is weak relative to the conflicting depth from perspective. I am currently investigating to what extent the temporal sensitivity of slant perception can be attributed to cue conflict. Sirnilarly. extra-retinal vergence and absolute disparity give poor estimates of epocentric distance but a role for vergence in size and depth constancy has more support (see Section 1.1.1). The size of the image that a target projects on the retina is related to its objective size scaled by distance (for review see Sedgewick 1986). Size constancy refers to the ability of an observer to use this invariant relationship to maintain a veridical estimate of the objective size of an object at various distances. When a subject increases convergence when viewing a target of fixed angular size and constant zero disparity the target appears to decrease in size (e-g. - Heinemann et al. 1959. Hetmans 1954). afi effect known as micropsia. This is consistent with the increased con\.ergence indicatine a nearrr target: however, the size changes are typically iess than predicted from size-constancy (Heinemann et al. 1959, Komoda and Ono 1974). The finding that convergence influences perceived size suggests that a vergence related distance signal is available to perception. Erkelens and Collewijn (1985a) found no percept of motion in depth when the absoluts horizontal disparity of a large stereogram was varied. This was true whether disparity was controlled with open-loop vergence or when the subject made vergence movements to compensate (Regan et al. 1986). However, the stereograrn did appear to change in size. becoming smaller with increased convergence. The changes in apparent size introduces a potential cue conflict into these experiments since the changing size indicates an opposite 160 change in distance to that indicated by the changing vergence. These size constancy effects may have contributed to the non-veridical depth from disparity. in some cases, the contradictory apparent size information may result in the paradoxical result of perceived distance being reported in the reversed order to that expected from the vergence information. This is an example of a size-distance paradox (Gruber 1954, Komoda and Ono 1974). Kinetic or motion perspective appears to be a more salient depth cue than zero- or first-order disparity. The reliance on perspective for short durations may reflect the presence of apparent motion between the frontal stimulus and the slanted or inclined surface. Under kinetic conditions, an assumption of rigid motion constrains the shapr- slant arnbiguity inherent in depth from static perspective. This makes kinetic perspective information more reliable than static perspective. Depth perception is complicated and rnany cues to depth are potentially available. It would be interesting to study the temporal factors in the integration of other pictorial cues to depth with stereopsis. Perspective cue conflict is stronger for slanted surfaces defined by size disparity than for inclined surfaces defined by shear disparity. The typical finding that horizontal shear disparity results in more robust depth than horizontal size disparity (Rogers and Graham 1983) may reflect, in pan. this difference in sensitiviiy to perspective. For surfaces with headcentric azimuth eccentricity, we have seen that a zero horizontal size disparity does not correspond to either a frontal surface or a surface with zero optical slant. Perspective provides an indication of the optical slant of the surface. This indication could potentially be used to resolve the slant-eccentricity ambiguity inherent in slant judgements from horizontal size disparity. In this scenario, disparity does not provide rneaningful slant information and thus the dominance of perspective over disparity in slant judgements is accounted for. For inclination, the perspective gradient indicates optical inclination. As in the slant case, this does not correspond to the inclination indicated by horizontal shear disparity. For a surface with elevation eccentricity, the horizontal shear disparity indjcates the inclination of the surface with respect to the frontal plane. This is true whether the 16 1 surface Is fixated or whether the eyes remain in the horizon plane. This is because the theoretical horopter is invariant with elevation of gaze (Howard and Rogers 1995). In addition, with the head erect, the disparity also provides inclination with respect to the gravito-inertial vertical - Gibson's geographical slant (Gibson and Cornsweet 1952). Thus, horizontal shear disparity provides an indication of inclination with respect to a strong headcentric norm and often with respect to a strong exocentric norm. In the inclination case, perspective and disparity can both provide reliable estimates of inclination, albeit in different frarnes of reference. To reconcile the two slant judgements under conflict conditions, the visuai system must interpret the surface as being elevated. If the surface is fixated this implies that gaze is elevated since local sign information is presumably more reliable than extraretinal eye position registration. Thus. for both slan t and inclination cue conflict. reconciliation of disparity and perspective based depth judgements requires localising gaze eccentrically. This eccentricity estimate derived frorn retinal cues is in conflict with oculomotor signals. Under dynamic cue conflict conditions this conflict between oculomotor and retinal indicators of eye position may be increased. Recent evidence suggests that change in eye position is registered more veridically than static eye position (Brenner and van Damrne 1998). Headcentric and exocentric judgements of surface inclination are possible frorn horizontal shear disparity alone. In contrat, slant from horizontal size disparity is ambiguous without information about eccentricity. This ambiguity leads to a stronger dependence on perspective cues in slani perception than in inclination perspective. Horizontal shear information becomes unreliable in the presence of cyclotorsion. Cyclophoria can result in cyclorotary fixation disparities that cause the theoretical horopter to become inclined (Howard and Roger 1995). Cyclotorsion associated with Listing's law and the elevation of gaze can also introduce cyclodisparity. In practice. al1 these cyclodispanties are small for modest gaze angles and spatial distortion should be compensated for by the induced-shear effect and cyclovergence. In Chapter 3 I found that first-order transformations of vertical disparity were not processed more slowly than first-order transformations of horizontal disparity. This result 162 did not support the hypothesis that vertical disparities are processed by relatively slow mechanisms. This failure may be related to the finding thar the response to horizontal disparity transformations are also slow. 1have argued above that the temporal sensitivity of the horizontal disparity mechanisms cm be explained by cue conflict enhanced by size- constancy induced texture gradients. For rotation disparity there was some evidence of a weak tendency to perceive increased depth in the direction of the horizontal component with increased temporal frequency. This finding suggests that the response to vertical shear disparity rnay be slower than that to horizontal shear disparity, as hypothesised. However. the low-pass nature of the horizontal shear disparity response interferes with the ability to interpret differences in the temporal frequency responses. The effects of size constancy mechanisms in the induced-shear effect are not known. Cue conflict may play a role in limiting the frequency response of both the horizontal and vertical shear disparity effects. Vertical shear disparity arises in natural scenes solely from cyclo-rotary fixation disparity associated with cyclophoria. Cyclodisparity also contains horizontal shear disparity. Because of the actions of cyclovergence and the induced-shear effect. the horizontal shear disparity associated with cyclophoria does not result in distortions of stereoscopic space. Neural and oculornotor compensation to cyclophoria rnay also be required to solve the correspondence problem. Properly matched dichoptic elements lie along corresponding epipolar lines in the two eyes. If this constraint is used. it reduces the rnatching task to a one dimensional search along epipolar lines (Prazdny 1983). However. the ability to correctly label epipolar lines requires that the eyes be torsionally aligned or that the misalignment be accounted for. Cyclophoria is a slowly changing parameter of the visual system and is not directly affected by movements of the body or target in space. Cyclophoria can be corrected for by cyclovergence and the induced-shear effect. Cyclovergence is a sluggish eye movement system but is well suited to deal with disturbances in rotational eye alignment. Perceived slant from dilation disparity increased in the direction of the vertical component with increased temporal frequency or decreased presentation tirne. This result 163 was contrary to the predictions of the hypothesis and suggests that vertical size disparity may be processed more quickly than horizontal size disparity. Aniseikonia due to changing optical conditions in the eye tends to change slowly with ageing and growth. Thus we predicted that vertical size disparity would be processed relatively slow if used as an indicator of aniseikonia. However, a type of aniseikonia arises when viewing eccentricaily located objects (see 1.3.5). The interpretation of optical slant from horizontal size disparity is arnbiguous due to this aniseikonia. To resolve this arnbiguity. an estirnate of headcentric eccentricity is required before the horizontal size disparity is interpreted. We have seen that the pattern of vertical disparity is a possible cue for eccentricity (see Section 1.3.5). Wben the subject or target is mobile, the headcentric eccentricity can change. Thus a mobile observer requires that vertical size disparity be processed in order to estimate the current headcentric direction of a target prior to the interpretation of slant from horizontal disparity. Thus, if one assumes that vertical size disparity is used to estimate the eccentricity of a surface. then it is not unreasonable that vertical size disparity be processed more rapidly than horizontal size disparity. There is another possible explanation for the fact that dilation disparity results in depth in the direction of the vertical component for higher frequency oscillation. It may be that vertical size disparity enhances and biases the perspective-disparity conflict responsible for slant reversals. The induced-size effect produces slant in the opposire direction to the geometric-size effect. For dilation disparity, the slant indicated by the vertical size disparity is in conflict with that indicated by the horizontal size disparity. However, the slant indicated by the vertical size disparity is in the same direction as the apparent texture gradient induced by the horizontal size disparity. Perhaps, under dynamic dilatioii conditions, the change in slant due to vertical disparity change biases the disparity-perspective conflict. This change would favour the consistent change in apparent texture gradient and oppose the conflicting and ambiguous slant from horizontal size disparity. One puzzling aspect is the Iack of slant reversa1 for the induced effect. Zero perspective in the presence of vertical size disparity indicates an eccentrically located surface normal to the cyclopean line to the surface. For example, a relative vertical 164 magnification in the right eye indicates that the surface is located to the right of the median plane of the head and the absence of texture gradient indicates that it has zero opticai slant. Thus perspective and vertical size disparity information are only weakly in conflict since vertical size disparity does not change significantly with changes in slant (Gillam and Lawergren 1983). Vertical size disparity and perspective are not directly in conflict in the induced effect. However, the vertical disparity causes the zero horizontal size disparity to produce a stereoscopic slant and one would predict a secondary conflict with perspective. Little is known about size constancy under the induced effects. Perhaps the lack of reversal effects in the induced-size effect is an indication of the level of stereoscopic processing at which the compensation for vertical disparity is introduced. Higher order vertical disparity patterns are also important in the perception of depth. Theoretically, the pattern of vertical disparity, specifically the horizontally oriented gradient of vertical size disparity. can be used to determine the epocentric distance of a surface (see Section 1.3.5). This estimate of egocentric distance could be reflected directly in perceived distance or indirectly in the scaling of size and depth with distance. Rogers and Bradshaw (1993) have shown that the scaling of vertical disparities in an image can effect the perceived distance of a surface. Johnston ( 199 1 ) and Curnming et si.

( 199 1) have found that constancy for equating the apparent depth and height of a horizontal cylindrical image was affected by vertical disparity scaling but constancy wns only about 25%. Bradshaw et al (1996), using tactile matching of apparent depth in a comgated surface, have reported similar levels of constancy. A larger level of depth constancy (about 756) was found by Glennerster et ai. (1996) when additional rnonocular cues to distance were made available. Rogers ar~dBradshaw (1995) have studied the effects of vertical disparity and vergence on depth scaling. When these cues were consistent, constancy, as reflected in settings of horizontal disparity patterns for an apparent frontal parallel plane, was close to 100%. In small 10' textured displays, constancy was close to 90% for distance specified by convergence. even in the presencr of contradictory vertical disparity information. With larger displays. vertical disparity played the most significant role in the judgernents. Constancy exhibited in the apparent 165 frontal pardlel task was quite high compared to constancy exhibited in relative depth judgements. FoIey (1980) has noted that the apparent frontal parallel plane task may be a particularly good psychophysical task with relatively srnall intersubject variability. In Chapter 3, the phenornenon of slant reversals suggested that the perceived slant and size constancy responses to gradients of disparity had different temporal properties. In lighr of this conclusion one could study the temporal sensitivity of the response to changes in the scaling of these higher order patterns of vertical disparity. It would be interesting to see if differences in the temporal sensitivity of vertical disparity scaling exist in egocentric distance estimates compared to size and depth constancy rneasures. Adams W., Frisby JP, Buckley D, Ghding J, Hippisley-Cox SD, Pomll J. 1996 "Pooling of vertical disparities by the human visual system" Perception 25 165- 176 Allen, D. C., 1974 "Vertical prism adaptation in anisornetropes" American Jolmial of Optontetv and Physiological Optics 51 252-259. Allison R S, Howard 1 P, Rogers BJ, Bridge H, 1997 "Efficiency of Slant and Inclination Perception as a Function of Temporal Frequency" Investigarive Ophrhalmology and Visual Science 38 S907 Alpem, M, 1962 "Types of eye movement" In The eye, ed. H. Davson (Academic Press. New York) Vol. 3, pp 7-27. Ames A, 1946 "Binocular vision as affected by reIations between uniocuIar stimulus patterns in cornmonplace environments" American Journal of Psyhology 59 333-357 Ames A, Ogle KN, Glidden GH, 1932 "Corresponding retinal points, the horopter and size and shape of ocular images" Jor~nznlof the Optical Socie~of Anierica 22 538-63 1 Amigo G, 1967 "The stereoscopic framr of reference in asyrnmetric convergence of the eyes" Vision Research 7 785-799 Anstis SM. Howard P.Rogers B, 1978 "A Craik-Comsweet illusion for visual depth" Visioti Resenrch 18 2 1 3-2 17 Anstis SM. 1974 "Chart demonstrating variations in acuity with retinal position" Visiou Resenrch 14 589-592 Balliet R. Nakayama K, 1978 "Training of voluntary torsion." Investigati\.r Ophthnlrnology and Visual Science 17 303-3 14. Baloh RW, Yee RD. Honrubia V, Jacobson K. 1988 "A cornparison of the dynamics of horizontal and vertical smooth pursui t in nomal human subjects" Ai*intion.Spnce nd Envit-orzr~ze~ztalMedicine 59 12 1- 124 Banks MS, Backus BT, 1998 "Extra-retinal and perspective cues cause the small range of the induced effect" Visiorz Research in press. Bishop, P. O., 1989 "Vertical disparity, egocentric distance and stereoscopic depth constancy: a new interpretation" Proceedings of the Royal Socieiy (Londoji)B 237 445- 469. Blake A, Bülthoff H, Sheinberg D, 1993 "Shape from texture: ideal observers and human psychophysics" Vision Research 33 1723- 1737 Blakemore C, 1970 "A new kind of stereoscopic vision" Visiolz Research 10 1 18 1-1 199 Blakemore C, Fiorentini A, Maffei L, 1972 "A second neural rnechanism of binocular depth discrimination" Jounlal of Physiology 226 725-749 Boltz RL, Harwerth RS, 1979 "Fusional vergence ranges of the monkey: a behavioural study" Experimntal Brain Research 37 87-9 1 Boltz RL, Smith EL, Bennett MJ, Harwerth RS, 1980 "Vertical fusional vergence ranges of the rhesus monkey" Vision Research 20 83-85. Boman DK, Kertesz AE, 1983 "Interaction between horizontal and venica1 fusionai responses" Perception and Psychophysics 33 565-570 Bradshaw MF, Glennerster A, Rogers BJ, 1996 "The effect of display size on disparity scaling from differential perspective and vergence cues" Vision Research 36 1255-1264 Braunstein ML, 1968 "Motion and texture as sources of slant information" Jo~inzalof Experirnental Psychology 78 247-253 Brenner E. van Damme JM, 1998 "Judging distance from ocular convergence" Visiorl Research 38 493-498. Bridge H. Rogers BJ, Bradshaw M. 1996 "Thresholds for detecting change of inclination in stereoscopic surfaces as a function of temporal frequency" Investignrive Ophrlzalr~zologyand Visrial Science 37 S285 Brookes A, Stevens KA. 1989 "The analogy between stereo depth and brightness" Perceprion 18 60 1-6 14 Buckley D,Frisby JP. Blake A, 1996 "Does the human visual system implement an idral observer theory of slant from texture" Visiorz Research 36 1 163- 1 176. Buckley D, Frisby JP. 1993 "Interaction of stereo, texture and outline cues in the shape perception of three-dimensional ridges" Vision Research 33 9 19-933 Buizza A. Schmid R, Gig MR, 1984 "The range of linearity of the smooth pursuit control systern", in Theoretical and Applied Aspects of Eye Movement Researclt Eds. A Gale. F Johnston (Amsterdam: Elsevier) pp 5 17-524 Burt P. JuIesz B, 1980 "Modifications of the classical notion of Panum's fusional area" Perceptio~i9 67 1-682 Cagenello R, Rogers BR, 1993 "Anisotropies in the perception of stereoscopic surfaces: the role of orientation disparity" Vision Research 33 2 189-220 1 Campbell FW, Robson JG, 1968 "Application of Fourier analysis to the visibiiity of gratings" Jorimal of Physiology (London) 197 55 1-566 Carpenter RHS, 1988 Movements of the eyes, London: Pion Clernent, R. A. 1987 "Line correspondence in binocular vision" Perception 16 193- 199 Collewijn H, 1977 "Eye and head movements in freely moving rabbits" Joitnml of Plryiolog~(London) 266 47 1-498 Collewijn H, van der Mark F, Jansen TC, 1975 "Precise recording of human eye movements" Vision Research 15 447450 Collewijn H, Steinman RM, Erkelens CJ, Regan D, 1991 "Binocular fusion. stereopsis and stereoacuity with a moving head" In Vision and visuaZ dysfimction. Vol. 9, Birzocrdnr vision ed. D. Regan (London: Macmillan) pp. 12 1-1 36 Comsweet TN,Crane HD, 1973 "Accurate two-dimensional eye tracker using first and founh Purkinje Images" Journal of the Optical Sociev of America 63 92 1-928 Cumming BG. Johnston EB, Parker AJ, 199 1 "Vertical disparities and the perception of three-dimensional shape" Nature 349 4 1 1413 Devalois KK, Heydt R von der, Ado rjani CS, Devalois RL, 1975 "A tilt aftereffect in depth" Investigative Opkthalmology and Visual Science, 15 ( Abs trac ts). 90 Duwaer AL, 1982 "Nonmotor component of fusional response to vertical disparity: a second look using an afterimage method" Joimal of the Optical Sociec of Anierica 72 87 1-877. Duwaer AL, van den Brink G. 198 1 a "Foveal diplopia thresholds and fixation disparities" Perception and Psychopkysics 30 32 1-329 Duwaer AL, van den Brink G, 198 1 b "Diplopia thresholds and the initiation of ver,Dence eye-movements" Vision Research 21 1727- 1737 Eagle R. Hogervorst M (forthcoming) 'The recovery of structure from motion under perspective and onhographic projection" Perceptiorl

Ebenholtz S. Paap K, 1973 "The constancy of object orientation: compensation for ocular rotation" Perception md Ps?-chophjsics 14 458-470 Ellerbrock VJ, 1949a "Experimental investigation of vertical fusional movements Pan 1" Antericm Jolrntal of Optomet~and Archives of the Anzerican Acadeniy of Opromen-. 26 327-337 Ellerbrock VJ, 1949b "Experimentai investigation of vertical fusional movements Pan II" American Journal of Optornetry and Archives of the American Academy of Optornerp 26 388-399 Ellerbrock VJ, 1952 "Effect of aniseikonia on the amplitude of vertical divergence" American Journal of Optornetry and Archives of American Academ of Optor?tern29 403-4 15 Enright JT, 1987 "Perspective vergence: oculomotor responses to line drawings" Visioil Research 27 I 5 13- 1526 Epstein W, Park J, Casey A, 1961 'The current status of the size-distance hypothesis" Psycholog ical Bulletin 58 49 1-5 14 Erkelens CJ, 1987 "Adaptation of ocular vergence to stimulation with large disparities" Experimental Brain Research 66 507-5 16 Erkelens CJ 1988 "Fusionai limits for a large random-dot stereogram" Vision Research 28 345-353 Erkelens CJ, Collewijn H, 1985a "Eye movements and stereopsis during dichoptic viewing of moving random4ot stereograms" Vision Research 25 1689-1 700 Erkelens CJ, Collewijn H. 1985b "Eye movements in relation to loss and regaining of fusion of disjunctively moving random-dot stereograrns" Human Neurobiology 4 18 1- 188. Erkelens CJ, Regan D. 1986 "Human ocular vergence movements induced by changing size and disparity" Journal of Physiology 379 145- 169 Fang XP, Howard IP, Allison RA, Zacher JE, 1997 "Effects of field size on verticai vergence" lnvestigative Ophthalmology and Visual Science 38 5986. Fang XP 1997 The Effects of Stiinrdus Size and Eccentricit). on Horizontal and Vertical Verge~icesYork University. Toronto, unpublished M.Sc. Thesis. Fang XP, Howard P.Allison RA, Zacher JE, 1998 "The effects of stimulus size and eccentricity on horizontal vergence" lnvestigative Ophthalvtology and Visual Sciem-e Srlpp lent en f . Fender D. Julesz B, 1967 "Extension of Panum's fusional area in binocularly stabilized vision" Joicnzal of the Optical Sociee of America 57 8 19-830 Ferman L, Collewijn H, Jansen TC, van den Berg AV, 1987 "Human gaze stability in horizontal, vertical and torsional direction during voluntary head movements, evaluatcd with a three-dimensional scleral induction coi1 technique" Vision Researcli 27 8 1 1-818 Finney DJ. 197 1 Probit analysis London: Cambridge University Press Foley JM. 1978 "Primary distance perception" In Handbook of Semon Physiology ivol. VIII Perceptio~ieds. R. Held, H. Leibowitz, H Teuber (Berlin: Springer Verlag) pp. 18 1 - 214 Foley JM, 1980 "Binocular distance perception" Psychological Review 87 4 1 1-434 Foley JM, 199 1 "Binocular space perception" In Vision and visual dysfunction. Vol. 9, Binocular vision ed. D. Regan (London: Macmillan) pp. 75-92 Foley J, van Dam A, Feiner S, Hughes J, 199 1 Coinputer Graphics: Principles and Practice New York: Addison-Wesley. Friedman RB, Kaye MG, Richards W. 1978 "Effect of vertical disparity upon stereoscopic depth" Vision Researclr 18 35 1-352 Fnsby JP, Buckley D, Wishart KA, Porrill, J, Giirding J, Mayhew JEW, 1995 "Interaction of stereo and texture cues in the perception of three-dimensional steps" Vision Research 35 1463-1472 Gkding J, Pomll J, Mayhew JEW, Frisby JP, 1995 "Stereopsis, vertical disparity and relief transformations" Vision Research 35 703-722 Gibson J. 1950 "The perception of visual surfaces" Arnerican JownaZ of psycho log^ 63 367-384 Gibson J, Cornsweet J, 1952 ''The perceived slant of visual surfaces - optical and geographical" Journal of Experimental Psychology 44 1 1- 15 Gibson J, Gibson E, 1957 "Continuous perspective transformation and the perception of rigid motion" Journal of Experimental Psychology 54 129- 138 Gibson EJ, Gibson JJ, Smith OW, Rock A, 1959 "Motion parallax as a determinant of perceived depth" Journal of Experimental Psycl~ology54 40-5 1 GilIam B, 1967 "Changes in the direction of induced aniseikonic slant as a function of distance" Vision Research 7 777-783 Gillam B. 1968 "Perception of slant when perspective and stereopsis conflict: experiments with aniseikonic lenses" Joumal of Experimental Psy9zology 78 799-305 Gillam B, 1993 "Stereoscopic slant reversais: a new kind of 'induced' effect" Perceptiori 22 1OB- 1 O36 Gillam B, Chambers D, Russo T, 1988a "Postfusional latency in slant perception and the primi rives of stereopsis" Jounial of Experinzentnl Psyhology: Hui?tmz Perception mid Perfon?initce 14 163-1 75 Gillam B. Chamoers D, Lawergren B, 1988b "The role of vertical disparity in the scaling of stereoscopic depth perception: an empincal and theoretical study." Perceptioii niid Ps~chophpics40 477483. Gillam B, Flagg T, Finlay D, 1984 "Evidence for disparity change as the primary siimulus for stereoscopic processing" Perception and Ps\.clzopllpics 36 559-564 Gillam B, Lawergren B, 1983 "The induced effect, vertical disparity, and stereoscopic theocy" Perception and Psychophysics 34 12 1-1 30 Gillam B. Rogers B, 1991 "Orientation disparity, defomation, and stereoscopic slant perception" Perception 20 44 148 Gillam B. Ryan C, 1992 "Perspective, orientation disparity, and anisotropy in stereoscopic slant perception" Perception 21 427-439 Glennerster A, Rogers BJ, Bradshaw MF. 1996 "Stereoscopic depth constancy depends on the subject's task" Vision Research 36 344 1-3456 Gogel WC, 1965 "Equidistance tendency and its consequences" Psychological Bulletin 64 153-163 Gruber HE, 1954 "The relation of perceived size to perceived distance" The American Jmmal of Psychology 67 4 11-426 Hanny P. von der Heydt R. Poggio GF. 1980 "Binocular neuron responses to tilt in the monkey visual cortex. Evidence for orientation disparity processing" Experimental Brain Research 41 A26 Heinemann EG, Tulving E, Nachmias J, 1959 "The effect of oculomotor adjustments on apparent size" American Journal of Psychology 72 32-45 Helmholtz H von, 1909 Physiological optics English translation by J. P. C. Southall (New York: Dover 1962) from the 3rd Geman edition of Handbuch der Physiologischen Optik. Vos, Hamburg Hering E, i 879 "Spatial sense and movements of the eye" (English Trans. by C. A. Radde). American Academy of Optometry, Baltimore. 1942 Hermans TG, 1954 "The relationship of convergence and elevation changes to judgmenra of size" Joiunal of Experimerital Psychology 48 204-208 Hokoda SC. Ciuffreda KJ 1983 '"Theoretical and ciinical significance of proximal vergence and accommodation" Ln Vergence eye movements: Basic and clitiical. aspec~s. eds. M.C Schor and K. J. Ciuffreda (Boston: Buttenvonh) pp. 75-97 Honrubia V, Downey WL, Mitchell DP. Ward PH, 1968 "Experimental studies on optokinetic nystagmus II normal humans" Acta Otola~gologica65 11 1-448 Houtman WA. Roze JH, Scheper W. 1977 "Vertical rnotor Fusion" Doc~mertta Oplitl~alnzologica44 1 79- 1 85 Houtman WA, Roze JH, Scheper W, 198 1 "Vertical vergence movements" Doc~ir?zenta Oplttl~ali>iologica51 199-207 Houtman WA, van der Pol BAE, 1982 "Fusional movements by peripheral retinal stimulation ('peripheral motor fusion')." Graefe's Archives Clinical Eiperinierital Opltthulmology 218 2 1 8-220 Howard HJ. 1 9 19 "A test for the judgment of distance" American Journal of Psvcholog~ 2 656-675 Howard IP, 1970 "Vergence, eye signature, and stereopsis" Psyhonomic Morlogrnpli Supplements 3 20 1-204 Howard IP, 1982 Human visual orientation Chichester: Wi ley Howard PT199 1 "Image cyclorotation. cyclovergence and perceived slant" SAE Techizical Paper Series 911392 1-8 Howard IP, 1993 'The optokinetic system" In The vestibirlo-ocular reflex. nystagmiis and vertigo eds. J. A. Sharpe and H. O. Barber (New York: Raven Press) pp. 163-181 Howard IP, 1996 "Alhazen's Neglected discovenes of visual phenornena" Perception 25 1203-1217 Howard IP, Ohmi M. Sun L, 1993 "Cyclovergence: a cornparison of objective and psychophysical measurements" Experimental Brain Research 97 349-355 Howard IP, Kaneko H, 1991 "Relative shear disparities and the perception of surface inclination" Vision Research 34 2505-25 17 Howard IP, Rogers BJ, 1995 Binocular vision and stereopsis New York: Oxford University Press Howard IP, Sun L, 1994 "Cycloversion and cyclovergence: the effects of the area and position of the visual display" Experimental Brain Research 100 509-5 14 Howard IP, Zacher JE, 1991 "Human cyclovergence as a function of stimulus frequency and amplitude" Experirnental Brain Research 85 445450 Hubel DH, Wiesel TN, 1962 "Receptive fields, binocular interaction and functional architecture in the cat's visual cortex" Journal of Physiolog~160 106-1 54 Hyson MT. Julesz B, Fender DH, 1983 "Eye movements and neural remapping during fusion of misaligned random-dot stereograms" Jormal of the Optical Sociee of Anrerica 73 1665-1673. Ittelson WH, Ames A, 1950 "Accommodation. convergence and their relation to apparent distance" Jorirnnl of Psyclrolog~30 43-62 Johnston EB, 199 1 "Systematic distortions of shape from stereopsis" Vision Resenrcit 31 1351-1360 Jones R, 1983 "Horizontal disparity vergence" In Vergence eye rnoveineïirs: Basic ami c[inicnl. aspects, eds. M.C Schor and K. J. Ciuffreda (Boston: Butterwonh) pp. 297-3 16 Jones R. Kerr KEY1972 "Vergence eye movements ro pairs of disparity stimuli wirh shape selection cues" Vision Research 12 1425-1430 Jones R, Stephens GL, 1989 "Horizontal fusional amplitudes" Investigative Ophthalniology and Visual Science 30 1638- 1612 Judge SJ, 199 1 "Vergence" In Vision and visual dys$oiction. Vol. 8, Eye »tovet~~eiitsed. R. H. S. Carpenter (London: Macmillan) pp. 157-172 Kaneko H, Howard IP, 1996 "Relative size disparities and the perception of surface slant" Vision Research 36 19 1 9- 1 930 Kaneko H, Howard LP, 1997 "Spatial properties of shear disparity processing" Visioir Resetrrch 37 3 15-323 Kapoula 2,Eggert T. Bucci MP, 1996 "Disconjugate adaptation of the vertical oculomotor system" Vision Research 36 2735-2745 Kaufman L, Rock 1, 1962 "The moon illusion" Science 136 953-961 Kepler J, 1604 Ad vitellionem paralipomena Frankfin: Marnium and Haer Kenesz AE, 198 1 "Effect of stimulus size on fusion and vergence" Journal of the Optical Society of America 71 289-293 Koenderink JJ, van Doom AJ, 1976 "Geometry of binocular vision and a mode1 for stereopsis" Biological Cybemetics 21 29-35 Kowler ETMartins kl,Pave1 M, 1984 "The effect of expectations on slow oculomotor control. TV. Anticipatory smooth eye rnovernents depend on pior target motions" Vision Research 24 197-2 10 Knshnan VV, Phillips S. Stark L, 1973 "Frequency analysis of accommodation, accommodative vergence and disparity vergence" Vision Research 13 1545- 1554 Leigh R.J. Zee DS, 199 1 The Neurology of Eye Movements. 2nd Edition Philadelphia: FA Davis Lemij HG, Collewijn H, 199 1 "Short-tem nonconjugate adaptation of hurnan saccades to anisometropic spectacles" Vision Research 31 1955-1 966 Lippincott JA, 1889 "On the binocular metamorphopsia produced by correcting glasses" AMA Archives of Ophtltal~nology18 1 8-30 London RF, Wick B, 1987 "Vertical fixation disparity correction: Effect on the horizontal forced-vergence fixation disparity curve" Anterican Journal of Optomet-. md Plr~siologicnlOptics 64 653-656 Komoda MK and Ono H, 1974 "Oculomotor adjustments and size-distance perception" Perception &Psychophysics 15 35 3-3 60 Mace W. Shaw R. 1974 "Simple kinetic information for transparent depth" Perceptiori ami Psylzopltysics 15 20 1-209 Maddox EE, 1893 The chical use of prisnts aiid the decentering of Zertses Bristol: John Wright & Sons MmD, 1982 Vision San Francisco: Freernan Matthews JH, 1 987 Numericol methods for cornputer science, engineering and mathenlatics Englewood Cl i ffs NJ: Preritice-Hall Maxwell JS, Schor CM, 1994 "Mechanisms of vertical phoria adaptation revealed by time-course and two-dimensional spatiotopic maps" Vision Research 34 244 1-245 1 Mayhew J, 1982 "The interpretation of stereo-disparity infornation: the computation of surface orientation and depth" Perception 11 387-404 Mayhew J, Longuet-Higgins HC. 1982 "A computational mode1 of binocular depth perception" Nature 297 376-378 Mays LE. Porter JD, Garnlin PDR. Tello CA. 1986 "Neural control of vergence eye movements: neurons encoding vergence velocity" Journal of Neurophysiology 56 1007- 1021 McMillan NA, Creelman CD, 199 1 Detection Theop: A User's Guide Cambridge: Cambridge University Press Mitchell DE. l966a "Retinal disparity and diplopia" Vision Research 6 44 1451 Mitchell DE. l966b "A review of the concept of 'Panum's fusional areas"' American Journal of Optometry 43 387401 Mitchell DE, 1969 "Qualitative depth localization with diplopic images of dissimilar shape" Vision Research 9 99 1-994. Mitchell DE, 1970 "Properties of stimuli eliciting vergence eye movements and stereopsis" Vision Research 10 145- 162 Mitchison GJ. McKee SP, 1990 "Mechanisms underlying the anisotropy of stereoscopic tilt perception" Vision Research 30 1 78 1- 1 79 1 Mitchison GJ, Westheimer G. 1981 "The perception of depth in simple figures" Visim Researclz 23 1063- 1073 Mitchison GJ. Westheimer G, 1990 "Viewing peometry and gradients of horizontal disparity", in Vision: Coding and Elfiency. Ed. C Blakemore (Cambridge: Cambridge University Press) pp 303-309 Morrison JD. Whiteside T'CD, 1984 "Binocular cues in the perception of distance of a point source of lightT'Perception 13 555-566 Meyer CH, Lasker AG. Robinson DA, 1985 "The upper Iimit of human smooth pursuir velocity" Visio~zResearch 25 56 1-563 Nakayama K, 1977 "Geometric and physiological aspects of depth perception" Proceedings of Society of Photo-Optical Instrument Engineers 120 2-9 Ninio J. 1985 "Orientational versus horizontal disparity in the stereoscopic appreciation of slant" Perception 14 305-3 14 Ogino S, Kato 1, Sakuma A, Takahashi K. Takeyama 1. 1996 "Vertical optokinetic nystagmus in normal individuals" Acta Otolaryzogolica (SrocMi),Suppl. 522 38-11. Ogle KN. 1938 "Induced size effect. 1. A new phenomenon in binocular space-perception associated with the relative sizes of the images of the two eyes" AMA Archives of Oplzthalmology 20 604-623 Ogle KN, 1939 "Induced size effect. II. An expenmental study of the phenomenon with restricted fusion stimuli" AMA Archives of Oplrthalmology 21 604625 Ogle KN, 1955 "Stereopsis and vertical disparity" AMA Archives of Ophfhalmology 53 495-504 Ogle KN, 1964 Researches in Binocular Vision New York: Hafner Ogle KN, Prangen A de H, 1953 "Observations on vertical divergences and hyperphorias" AMA Archives of Ophfhalmology49 3 13-334 Ono H, 1979 "Axiomatic surnrnary and deductions from Hering's principles of visual direction" Perception and Psychophysics 25 473-477 Ono H, 198 1 "On Well's (1792) Iaw of visual direction" Perception and Psychophysics 30 403-406 Ono H, 199 1 "Binocular visual directions of an object when seen as single or double" In Vision and visual dysifrtnction. Vol. 9. Binocular vision ed. D. Regan (London: Macmillan) pp. 1- 18 Ono H, Muter P. Mitson L, 1974 "Size-distance paradox with accommodative convergence" Perception and Psychophysics 15 30 1 -307 Ono H, Angus R. Gregor P, 1977 "Binocular single vision achieved by fusion and suppression'' Perception and Psychophysics 21 5 13-52 1 Ono ME, Rivest J, Ono H, 1986 "Depth perception as a function of motion parallax and absolute-distance information" Journal of Experimental Ps~cltology-Human Percepriori arid Pe$ormance 12 33 1-337 Panum PL, 1858 Physiologische Untersrrchingen iiber dm Sellen mit zwei Alrgen Keil.: Schwers Perlmutter A, Kenesz AE, 1978 "Measurement of human vertical fusional response" Visiorz Research 18 2 19-223 Perlmutter A, Kertesz AE, 1982 "Human vertical fusional response under open and closed loop stimulation to predictable and unpredictable disparity presentations" IEEE Traitsactions on Bionzedicnl Engineering 29 57-6 1 Petrov AP, 1980 "A geometrical explanation of the induced size effect" Visioiz Resenrcli 20 409-4 13 Piantanida TP, 1986 "Stereo hysteresis revisited" Vision Resenrch 26 43 1437 Poggio GF, Fischer B, 1977 "Binocular interaction and depth sensitivity in striate and prestriate cortex of behaving rhesus monkey" Journal of Neurophysiology 40 1392- 1105 Pola J, Wyatt HJ 1980 'Target position and velocity: the stimuli for smooth pursuit eye movements" Vision Research 20 523-534 Pola J, Wyatt HJ 1985 "Active and passive smooth eye movements: effects of stimulus size and eccentricity" Vision Research 25 1063- 1O76 Pong TC, Kenner MA, Otis J, 1990 "Stereo and motion cues in preattentive vision-some expenments with random-dot stereographic image sequences" Perception 19 16 1- 170 Prazdny K. 1983 "Stereoscopic matching eye position and absolute depth" Perception 12 151-160. Rashbass C. 198 1 "Reflections on the control of vergence" In Models of oculoinotor behavior and control ed. B. L. Zuber (Boca Raton: CRC Press) pp. 139-148. Rashbass C, Westheimer G, 1961 "Disjunctive eye movements" Journal of Physiologv 159339-360 Regan D, 199 1 Vision and visual dyfiunction. Vol. 9, Binocular vision ed. D. Regan, London: Macmillan Regan D, Beverley KI. 1973 "Some dynarnic features of depth perception" Visio~t Researclr 13 2369-2379 Regan D, Beverley KI, 1978 "Looming detectors in the human visual pathway" Visiori Researclz 18 4 1 5-42 1 Regan D, Beverley KI, 1979 "Binocular and monocular stimuli for in depth: changing- disparity and changing-depth feed the same motion-in-depth stage" Vision Research 19 1331-1342 Regan D, Erkelens CJ, Collewijn H, 1986 "Necessary conditions for the perception of motion in depth" investigative Ophtlzol»iolog~and Visita1 Science 27 584-597 Richards W. 197 1 "Anornalous stereoscopic depth perception" Joitnzol of the Opticnl Socie~of Aïrierica 61 3 10-2 14 Richards W. 1972 "Response functions for sine- and square-wave modulations of disparity" Journal of the Optical Society of America 62 907-9 1 1 Riggs LA, Niehl EW, 1960 "Eye movements recorded during convergence and divergence" Joltmal of the Optical Society of Anzerica 50 9 13-920 Robinson DA, 1963 "A method of measuring eye movement using a scleral search coi1 in a magnetic field." IEEE Transactions on Biomedical Engineering 10 137-145 Robinson DA, 198 1 "The use of control systems analysis in the neurophysiology of e ye movements" Annual Review of Neuroscience 4 463-503 Rock 1, 1984 Perception New York: Scientific American Library Rogers BJ, 1992 "The perception and representation of depth and slant in stereoscopic surfaces", in Artificial and Biological Vision Systems, Ed GAOrban and H Nagel (Berlin: Springer-Verlag) pp 24 1-266 Rogers B J, 1997. "Eye Movements and Binocular Stereopsis" Paper presented ai International Conference on Vision and Action. York University. North York, Ontario June 1997. Rogers BJ, Bradshaw MF, 1993 "Vertical disparities, differential perspective and binocular stereopsis" Nature 361 253-255 Rogers BJ. Bradshaw MF, 1995 "Disparity scaling and the perception of frontoparallel surfaces" Perception 24 155- 179. Rogers BJ, Cagenello R, 1989 "Disparity curvature and the perception of three- dimensional surfaces" Nature 339 135-1 37. Rogers BJ, Graham ME, 1979 "Motion parallax as an independent cue for depth perception" Perception 8 125- 134 Rogers BJ, Graham ME. 1982 "Similarities between motion parallax and stereopsis in human depth perception" Vision Research 22 2 16-270 Rogers B J, Graham ME. 1983 "Anisotropies in the perception of three-dimensional surfaces" Science 221 f 409-1 4 1 1 Ryan C, Gillam B, 1994 Tueconflict and stereoscopic surface slant about horizontal and vertical axes" Perception 23 645-658 Schor CM. 1979 'The relationship between fusional vergence eye movements and fixation disparity" Vision Research 19 1359-1367 Schor CM. 1983 "Fixation disparity and vergence adaptation" In Vergertce E>e Movements: Basic and Clinical Aspects eds. MC Schor, KI Ciuffreda (Boston: Butterworth) pp. 465-5 16 Schor CM. Tyler CW, 198 1 "Spatio-temporal properties of Panurn's fusional area" Vision Resenrch 21 683-692 Schor CM. Wood IC, Ogawa J, 1984"Binocular sensory fusion is limited by spatial resolution" Vision Research 24 66 1-665 Schor CM, Gleason G, Homer D, 1990 "Selective nonconjugate binocular adaptation of vertical saccades and pursuits" Vision Research 30 1 827- 1844 Schor CM, Gleason G, Maxwell J, Lunn R. 1993a "Spatial aspects of vertical phoria adaptation" Vision Research 33 73-84 Schor CM. Gleason G, Lunn R, 1993b "Interactions between short-term vertical phoria adaptation and nonconjugate adaptation of vertical pursuits" Vision Researcli 33 55-64 Schumer RA, Julesz B. 1984 "Binocular disparity modulation sensitivity to disparities offset from the plane of fixation" Vision Research 24 533-542 Sedgewick HA, 1986 "Space perception" In Handbook of Hrtnian Perceptiou alid Pe$onnance ed. K. R. Boff, L Kaufman, and J. P. Thomas (New York: Wiley) Chapt. 3 1 1-57 Semmelow JL, 198 1 "Oculomotor responses to near stimuli: the near triad" In Models of Oculonlotor Behavionr and Control ed. B. Zuber (Boca Raton: CRC Press) pp 16 1 - 191 Semmelow JL, Hung G, 1983 "The near response: theories of control" In Vergence eye movements: Basic and clinical, aspects. eds. M.C Schor and K. J. Ciuffreda (Boston: Buttenvonh) pp. 175- 195 Sheedy JE. Fry GA, 1979 "The perceived direction of the binocular image" Vision Research 19 20 1-2 1 1 Siderov J, Harwerth RS, 1995 "Stereopsis, spatial frequency and retinal eccentncity" Vision Research 35 2329-2337 Sloane ME, Blake R, 1987 "Perceptually unequal spatial frequencies do not yield stereoscopic tilt." Perception and Psychophysics 42 569-575 Sobel EC, Collett TS, 1991 "Does vertical disparity scale the perception of stereoscopic depth?" Proceedings of the Royal Socie~(London) B 244 87-90 Solomons H, 1975 "Properties of the space horopter" British Jo~tmalof PIz~ioiogicd Qptics 30 8 1-100 St. Cyr GF, Fender DH, 1969 'The interplay of drifts and flicks in binocular fixation" Vision Researc*h9 245-265 Stark L, 1983 "Normal and abnormal vergence" In Vergence Eye Movemetzts: Brisic nid ClNzicol Aspects eds. MC Schor, KJ Ciuffreda (Boston: Buttenvonh) pp 3- 13

S teinman R M. Cushman WB, Martins A J, 1982. "The precision of gaze" Hirninii Nertrobiology l 97- 109 Stevens KA. Brookes A. 1988 "Integrating stereopsis with monocular interpretations of planar surfaces" Visioit Research 28 37 1-386 Stevens KA, Lees M. Brookes A, 199 1 "Cornbining binocular and monocular curvaturr features" Perception 20 425-440 Tyler CW, 197 1 "Stereoscopic depth movement: two eyes less sensitive than one" Science 174 958-96 1 Tyler CW, 1973 "Stereoscopic vision: cortical limitations and a disparity scaling effect'. Scierzce 181 276-278 Tyler CW, 1983 "Sensory processing of binocular disparity" In Vergence eve movements: Basic and clinical, aspects. eds. M.C Schor and K. J. Ciuffreda (Boston: Buttenvonh) pp. 199-296. Tyler CW, Sutter EE, 1979 "Depth from spatial frequency difference: an old kind of stereopsis?" Vision Research 19 859-865 Ullman S. 1979 The irtterpretation of visual motion Cambridge: MIT Press Van Ee R, 1995 Stabiliv of Binocular Deptlt Perception PhD thesis, University of Utrecht, Utrecht, the Netherlands Van Ee R, Erkelens CJ, 1996 "Temporal aspects of binocular slant perception" Vision Resea rclt 36 45- 5 1 Van Rijn U,van der Steen J, Collewijn H, 1994 "Eye torsion elicited by oscillating gratings: effects of orientation, wavelength and stationary contours" Vision Researcli 34 533-540 Von Noorden GK Binocular Vision and Ocular Motility 4" edition St Louis: Mosby Wallach H, Bacon J, 1976 "Two forms of retinal disparity" Perception and Psychophysics 19 375-382 Wallach H, OTConnellD, 1953 "The kinetic depth effect" Journal of Experirnental Psycltology 45 205-2 17 Westheimer G. 1957 "Kinematics of the Eye" Joiirnal of the Optical Socieg of Anierico 47 967-973. Westheimer G, 1979 "Cooperative neural processes involved in stereoscopic acuity" Ekperirnental Brain Research 36 585-597 Westheimer G, 198 1 "Donder's, Listing's and Hering's Laws and Their Implications" in Models of Ocidoinotor Behaviour and Control ed. B. Zuber (CRC Press: Boca Raton) pp. 149-159 Westheimer G,. McKee SP, 1978 "S tereoscopic acuity for moving retinal images" Joimral of the Optical Sociep of America 68 450155 Westheimer G, Mitchell AM, 1956 "Eye movement responses to convergence stimuli" AMA A rclr ives of Oplitl~alniology55 848-856 Westheimer G, Mitchell DE, 1969 "The sensory stimulus for disjunctive eye movements" Vision Research 9 749-755 Wheatstone C, 1838 "Contributions to the physiology of vision - Part the first. On some remarkable and hitherto unobserved phenomena of binocular vision" Philosophical Transactiorts of the Royal Society (hndon)128 37 1-394 Wheatstone C, 1852 "Contributions to the physiology of vision - Part the second. On some remarkable and hitheno unobserved phenomena of binocular vision" Philosophicd Transactions of the Royal Society (London)142 1- 17 Wilde K, 1950 "Der Punktreiheneffekt und die Rolle der binocularen Querdisparation beim Tiefensehen" Psychologische Forschung 23 223-262 Wilson HR, 1976 'The significance of frequency gradients in binocular grating perception" Vision Research 16 983-989 Yasui S, Young LR, 1984 "On the predictive control of foveal eye tracking and slow phases of optokinetic and vestibular nystagmus" Journal of Physiology 347 17-33. Ygge J, Zee DS, 1995 "Control of vertical eye alignment in three-dimensionai space" Vision Research 35 3 169-3 18 1 Young LR, Sheena D, 1975 "Survey of eye rnovement recording methods" Behavior Research Methuds and Instrumentation 7 397429 Young MJ, Landy MS, Maloney LT. 1993 "A perturbation analysis of depth perception from combinations of texture and motion cues" Vision Research 33 2685-2696 Zuber BL, Stark L, 1968 "Dynarnical characteristics of the fusionai vergence eye- movement system" IEEE Transactions Systems Science and Cybernetics 4 72-79 APPENDIXA - SYSTEMSTHEORY AND VERGENCE Oculomotor and perceptual systems cm often be conveniently described and analysed in tems of system theory or automatic control theory. Oculomotor respomes such as vergence are often goal directed and cm be considered the response of a neural automatic control system. Automatic control is a branch of applied mathematics that deals with generalised inputs, system States and charactenstics. However. with certain assumptions the analysis of many linear and non-linear systems is greatly simplified. This section describes some definitions and concepts related to automatic control and systems theory germane to this thesis and the study of vertical vergence control. In Chapter 3. we found many of these concepts are also useful in studying stereopsis. The vergence system is the oculomotor system to which 1 applied automatic control system analysis. This approach was used by Rashbass and Westheimer (196 1 ) in one of the first studies using modem quantitative eye movement recording techniques. In control theory terms, stimulus disparity is the vergence system input and vergence eye position is the system output. In studying vergence. it is useful to disringuish between dispariry in the stimulus and disparity in the retinal images. Stimulus disparity or target vergence is a physical misalignment of two dichoptic (or dioptic) stimuli and can be described in terms of the optic arrays of the two eyes. For horizontal vergence in a natural environment, stimulus disparity is equivalent to the binocular subtense of a target. Venical target vergence is defined with respect to a canonical zero vergence position where the optic axes of the two eyes lie in the same plane of elevation. Image disparity is the misalignment of the images between the two retinae and is the stimulus disparity minus any vergence correction. Image disparity is thus the absolute retinal disparity of the images of the target point. The vergence response can be described in terms of conrrol theory and characterised by a number of static and dynamic parameters including: fixation disparity - is the residual image disparity following a vergence movement and can be regarded as a static vergence error. In servo feedback models, the output or a correlate is subtractec! from the input to obtain an error signal used to drive the response to reduce the error. In this sense. image disparity is an error signal and fixation disparity is the steady-state error that remains after a change in vergence is complete. 1will reserve the term fixation disparity for the static vergence error following a change in vergence. 1will use the term vergence error more generally. An exarnple is the residual steady state image disparity during sinusoidal stimulation if gain is not unity. vergence gain - is the ratio of the amplitude of vergence movement to the size of stimulus disparity input. Any deviation from a vergence gain of unity will result in vergence error. Attenuation is the inverse of gain. Open-loop gain is the gain of the system v~henthe feedback loop has been disabled or opened. In the vergence system the feedback is visual - the image disparity error signal changes during the movement. To measure open-loop vergence. one must break this feedback loop by controlling image disparity rather than stimuius disparity. Closed-loop gain is the gain of the system with the feedback loop operational. phase shift- of a linear system is the phase difference between a sinusoidal input and the resulting sinusoidal output of the same frequency. The phase shift reflects the phase response of the system at the specified frequency. A phase lead is an advance of the output sinusoid relative to the input. A phase lag is a delay of the output relative to the input. transfer function - a mathematical expression that characterises the relation between the input and response of a system. By applying the transfer function to the input one obtains the output. linear range - of a system is the range of input amplitudes over which the output is a linear function of the input. Many systems are non-linear in general but can be can be approximated by linear systems over a relatively small range of inputs (small signal linearity assumption). Apart from the linear range. there may be a range of disparity values outside of which the vergence response breaks down completely that is characterised by the maximum vergence dispari ty limit. threshold - an important nonlinearity present in sensory systems. A threshoid is a limit below which a system will not respond. Since sensory systems have noisy, or stochastic inputs, thresholds rnust be defined as the Ievel below which the probability of response is below a defined level. In terms of automatic control theory one often refers to a deadband which is a range of inputs to which a system will not respond and is thus similar to a threshold. Determination of a thresholdldeadband for vertical vergence ha.been controversial due to the coarseness of eye movement recordings and potential artefacts in the more sensitive psychophysical techniques (Kenesz 198 1 vs. Duwaer and van den Brink 198 1b). frequency - the number of times a periodic function repeats itself for one unit variation in the independent variable. Temporal frequency describes periodicity in temporally varying functions. Spatial frequency describes periodicity in spatially varying functions. frequency response - the gain and phase of the vergence system measured over a range of frequencies. In a non-linear system the frequency response is dependent on amplitude and frequency content and mav not be meaninofui. IMAGE EVALUATION TEST TARGET (QA-3)

APPLIED - IWGE. lnc -= 1653 East Main Street , -. Rochester. NY 14609 USA --s -- Phone: 71 W482-0300 --=-- Fax: 71 6/288-5989

O 1993. Applied image. inc.. NI Rights Reserved