Perception &: Psychophysics 1991, 49 (3), 230-244 The interplay between stereopsis and structure from motion MARK NAWROT and RANDOLPH BLAKE Vanderbilt University, Nashville, Tennessee In a series of psychophysical experiments, an adaptation paradigm was employed to study the influence of stereopsis on perception of rotation in an ambiguous kinetic depth (KD) display. Without prior adaptation or stereopsis, a rotating globe undergoes spontaneous reversals in per­ ceived direction of rotation, with successive durations of perceived rotation being random vari­ ables. Following 90 sec of viewing a stereoscopic globe undergoing unambiguous rotation, the KD globe appeared to rotate in a direction opposite that experienced during the stereoscopic adap­ tation period. This adaptation aftereffect was short-lived, and it occurred only when the adapta­ tion and test figures stimulated the same retinal areas, and only when the adaptation and test figures rotated about the same axis. The aftereffect was just as strong when the test and adapta­ tion figures had different shapes, as long as the adaptation figure contained multiple directions of motion imaged at different retinal disparities. Nonstereoscopic adaptation figures had no ef­ fect on the perceived direction of rotation of the ambiguous KD figure. These results imply that stereopsis and motion strongly interact in the specification of structure from motion, a result that complements earlier work on this problem. This paper documents the interplay between stereop­ space; in the other instance, information is integrated over sis and motion information in the generation of three­ time. Besides their geometric similarities, stereopsis and dimensional (3-D) surface perception in human vision. motion parallax also yield comparable levels of perfor­ It has long been known that retinal disparity can create mance on tasks involving the measurement ofdepth sen­ a robust sensation of depth and solidity (Wheatstone, sitivity (Rogers & Graham, 1982). Moreover, both stere­ 1838), and its effectiveness is most dramatically revealed opsis and motion parallax yield equivalent variations in in the case of texture stereograms devoid of monocular perceived depth that are dependent on the retinal orienta­ information about shape (Julesz, 1971). Equally impres­ tion of the depth gradient (Rogers & Graham, 1983). sive is the recovery of shape information when an ob­ Stereopsis and motion are also similar in that both are ject is viewed under conditions of motion (Wallach & degraded at equiluminance. Lu and Fender (1972) found O'Connell, 1953). Motion information can even reveal that stereopsis with random-dot stereograms was markedly the surface structure ofan object that is literally invisible impaired when the chromatic stereopair was equiluminant. when stationary. This well-known effect, sometimes called Similarly, Ramachandran and Gregory (1978) reported the kinetic depth effect (KDE), has received much atten­ that moving contours at equiluminance yield very feeble tion in recent years (Braunstein & Andersen, 1984; Lappin motion perception. The exact extent to which stereopsis & Fuqua, 1983; Sperling, Landy, Dosher, & Perkins, and motion perception are impaired at equiluminance re­ 1989; Todd, 1984, 1985). mains unsettled (Shapley, 1990). As numerous authors, including Helmholtz (1909/ Are these similarities between motion parallax and ste­ 1962), have pointed out, stereopsis and motion parallax reopsis coincidental or do they reveal important linkages are comparable geometrically. In the case of stereopsis, in the underlying processing architecture? Several lines of the brain utilizes two views ofa scene from slightly differ­ evidence suggest that the similarities are more than coin­ ent perspectives (i.e., the difference in location ofthe two cidental. For example, a significant correlation has been eyes) to derive a description of an object's depth and reported between an observer's level of competence on a shape. Similarly, from a single vantage point, slightly stereopsis task and thatperson's ability to judge depth when different views ofan object over time form the basis for given motion information. Specifically, Richards and the KDE. In one instance, information is integrated over Leiberman (1985) tested observers with varying degrees ofimpairment in stereopsis, as indexed by a deficiency to judge depthfrom disparity information. Next, Richards and Preliminaryresults from some of these experiments have been reported Leiberman measured the ability of these observers to iden­ in Nawrot and Blake, 1989. This work was supported by NIH Grant tify and judge the apparent 3-D size of a dynamic 2-D EY07760 and NIH Vision Core Grant P3D-EY08126. We thankMyron Braunstein for helpful discussion. Correspondence should be addressed display. The correlation between stereo performance and to Randolph Blake, Department of Psychology, Vanderbilt University, kinetic depth was 0.72. Richards and Leiberman also ob­ Nashville, TN 37240. served that people who performed particularly well on Copyright 1991 Psychonomic Society, Inc. 230 STEREOPSIS AND STRUCTURE FROM MOTION 231 stereo displays containing crossed disparity information study, Wallach et al. used a mirror stereoscope to exag­ also tended to exhibit better performance on the KD task, gerate the retinal disparities associated with viewing a whereas stereo performance with uncrossed disparities rotating wire figure. Following prolonged inspection of was less predictive of KD performance. Richards and this exaggerated stereoscopic depth, observers perceived Leiberman concluded that the outputs from kinetic depth a stationary version of the figure to be flattened, or com­ mechanisms and from convergent stereoscopic depth pressed in depth, even though it contained the same ex­ mechanisms are combined prior to an object's assignment aggerated disparities. This so-called satiation effect was to a given depth. While not quarreling with this conclu­ not observed, however, when the test figure was a mon­ sion, Bradshaw, Frisby, and Mayhew (1987) found that ocularly viewed KD version of the figure, implying that observers were adept at detecting structure from motion stereoscopic satiation did not generalize to motion. In a in displays imaged entirely at uncrossed disparities; this later paper, Wallach and Karsh (1963) proposed that the led them to question the tightness of the linkage between initial effect ofadaptation to exaggerated disparity results crossed disparities and KD. from the discrepancy between the depth signaled by KD Quite recently, Howard and Simpson (1989) reported and the depth signaled by the exaggerated disparities. that the gain of optokinetic nystagmus varies inversely According to this view, then, kinetic depth information with binocular disparity, which they attributed to neurons plays an essential role in the modification ofstereoscopic sensitive to both disparity and direction of motion. As depth perception. Richards (1985) pointed out, linkage between stereopsis In the present paper, we give the results of a series of and motion processing could serve to resolve ambigui­ psychophysical experiments in which we examined in ties inherent when disparity information or motion paral­ closer detail the interactions between stereopsis and mo­ lax information is available on its own. tion information in the specification of structure from mo­ Cross-adaptation studies also imply some form of in­ tion. We utilized a version of the cross-adaptation para­ teraction in the processing of stereoscopic information and digm, whereby adaptation to a stereoscopically defined, kinetic depth information. Smith (1976) found what he rotating object subsequently biases the perception of ro­ termed a "contingent depth aftereffect" with Lissajous tation ofan object defined solely by motion information. figures (a type of KD display easily generated on an os­ In particular, we have assessed the necessary and suffi­ cilloscope by applying the same signal to the horizontal cient stimulus conditions for adaptation by various 3-D and vertical amplifiers; see Braunstein, 1962). Viewed figures to affect the perception of subsequently presented normally, a Lissajous figure undergoes spontaneous rever­ 2-D figures. sals in the direction ofapparent rotation. However, place­ ment of a neutral density (ND) filter in front of one eye MEmOD produces a strong bias in the perceived direction of rota­ tion, presumably by generating a retinal disparity cue from Stimuli and Displays interocular delay (i.e., the Pulfrich effect). Smith found TIlestimuli for these experiments consisted of computer-generated that when the ND filter was removed following a 30-sec random-dot cinematograms depicting objects rotating in depth. Each observation period ofunambiguous motion, the Lissajous frame in a given cinematogram was a 2-D, parallel projection "snap­ shot" of the object at some degree of rotation around a stationary figure appeared to rotate in the direction opposite that ex­ axis in the mathematicallydefined3-D coordinate system. Presenting perienced with the filter in place. The stereo cue, in other the cinematogram frames quickly in succession on the face of the words, disambiguated the motion cue, and this disambig­ video monitor produced the KDE of a rotating object. Although uation in tum yielded a pronounced visual aftereffect. A the KD figure presented on a single monitor appears vividly to be rather similar cross-adaptation result was described by 3-D, it is devoid of retinal