Perception & Psychophysics /993. 54 (2). /57-J(B Recovery of 3-D shape from binocular disparity and structure from motion JAMES S. TI'ITLE The Ohio State University, Columbus, Ohio and MYRON L. BRAUNSTEIN University of California, Irvine, California Four experiments were conducted to examine the integration of depth information from binocular stereopsis and structure from motion (SFM), using stereograms simulating transparent cylindri­ cal objects. We found that the judged depth increased when either rotational or translational motion was added to a display, but the increase was greater for rotating (SFM) displays. Judged depth decreased as texture element density increased for static and translating stereo displays, but it stayed relatively constant for rotating displays. This result indicates that SFM may facili­ tate stereo processing by helping to resolve the stereo correspondence problem. Overall, the re­ sults from these experiments provide evidence for a cooperative relationship betweel\..SFM and binocular disparity in the recovery of 3-D relationships from 2-D images. These findings indicate that the processing of depth information from SFM and binocular disparity is not strictly modu­ lar, and thus theories of combining visual information that assume strong modularity or indepen­ dence cannot accurately characterize all instances of depth perception from multiple sources. Human observers can perceive the 3-D shape and orien­ tion, both sources of information (along with many others) tation in depth of objects in a natural environment with im­ occur together in the natural environment. Thus, it is im­ pressive accuracy. Prior work demonstrates that informa­ portant to understand what interaction (if any) exists in the tion about shape and orientation can be recovered from visual processing of binocular disparity and SFM. binocular disparity (e.g., Ono & Comerford, 1977) and Recent research has considered how the human visual structure from motion (SFM) (e.g., Braunstein, 1976). system combines depth information from multiple sources However, neither SFM nor binocular disparity in isola­ (Braunstein, Andersen, Rouse, & Tittle, 1986; Bruno & tion specifies completely both an object's shape and orien­ Cutting, 1988; Biilthoff & Mallott, 1988; Dosher, Sper­ tation relative to an observer. Structure from motion can ling, & Wurst, 1986; Rogers & Collett, 1989; Rouse, Tit­ provide information about object shape, but near/far rela­ tle, & Braunstein, 1989). Specifically, Biilthoff and Mal­ tions between the object and the observer are not speci­ lott discuss four types of interactions between different fied (Wallach & O'Connell, 1953).1 Binocular disparity, sources of depth information: accumulation, disambigu­ on the other hand, unambiguously specifies near/far rela­ ation, veto, and cooperation. Accumulation refers to an tions, but the shape of the object is only determined up additive relationship between the various processes that to a scale factor: perceived fixation distance (Gogel, 1960; is consistent with modularity in visual information pro­ Ogle, 1953; Wallach & Zuckerman, 1962). Because each cessing (Fodor, 1983). The perceived depth from this type of these sources of information specifies what the other of interaction is a weighted summation or integration of lacks, the combination of SFM and binocular disparity the depth specified by each separate process. Empirical could provide a more robust representation of an object evidence for an accumulative relationship has been pro­ than would either source alone (Richards, 1985). In addi- vided for stereo and relative brightness (Dosher et al., 1986), motion parallax and pictorial depth cues (Bruno & Cutting, 1988), and motion parallax and binocular dis­ Thisresearchwas supportedby National ScienceFoondationGrant BNS­ parity (Rogers & Collett, 1989). A veto relationship be­ 8819565. Portions of this research were presented at the SPIE Sensor tween visual processes is one in which one source of Fusion III Conference (Tittle & Braunstein, 1991) and at the Annual information completely dominates others. It is also con­ Meeting of the Association for Research in Vision and Ophthalmology (Tittle & Braunstein. 1989). This research was completed as partial ful­ sistent with a modular conceptualization of visual depth fillment of the requirements for the Ph.D. at the University of Califor­ processing. An example ofa disambiguating relationship nia, Irvine by the first author. We would like to thank Asad Saidpour between two sources of information comes from a study for assistance in developing the stimulus displays and Joseph Lappin by Andersen and Braunstein (1983), who used dynamic for helpful comments on an earlier draft of this paper. Correspondence should be addressed to J. S. Tittle, Department of Psychology, Ohio occlusion to disambiguate the order of perceived depth State University, 140D Lazenby Hall, 1827 Neil Avenue, Columbus, in orthographic SFM displays, Disambiguation ofan SFM Ohio 43210 (e-mail: [email protected]). display can also result from adaptation to a stereo dis- 157 Copyright 1993 Psychonomic Society, Inc. 158 TITTLE AND BRAUNSTEIN play (Nawrot & Blake, 1991). A cooperative interaction still able to make accurate curvature discriminations on between visual processes is one in which strict modular­ the basis of SFM. ity does not hold. Instead, information from one process In a study examining the contributions of relative bright­ facilitates the functioning of another process. ness and stereopsis to the perception of sign of depth, Richards (1985) has proposed a specific relationship be­ Dosher et al. (1986) presented observers with perspec­ tween stereopsis and SFM that eliminates the ambiguities tive projections of rotating wire-frame Necker cubes. inherent in each source ofinformation. According to this They found that direction-of-rotation judgments were analysis, SFM provides information for 3-D shape and primarily determined by binocular disparity. However, stereopsis disambiguates depth order. The derivation of the relative brightness cue did have some influence on the 3-D shape from motion assumes parallel projection and rotation judgments, and the results were fit closely by a planar motion of the points. Using this approach, Richards model in which the strength of evidence from each source shows that with two stereo views of three points, the depth of information was algebraically added. Their weighted coordinates of those points are uniquely determined. Such linear summation model was similar to a more general a relationship between stereo and SFM would achieve approach for describing information integration developed shape constancywhile still preserving unambiguousviewer­ by Anderson (1981, 1982). relative depth relations. However, one assumption of this Braunstein et al. (1986) showed that binocular dispar­ analysis is that the magnitude of perceived depth from ity can disambiguate the sign of depth for computer­ stereopsisand SFM depends solelyon the SFM information. generated displays consisting of orthographic projections Early empirical research on conflicting stereo and SFM of texture elements on the surface of rotating spheres. information is consistent with Richards's (1985) theoreti­ They also found that, for opaque stimuli, dynamic occlu­ cal analysis. Specifically, Wallach, Moore, and Davidson sion information usually vetoed conflicting binocular dis­ (1963), and Epstein (1968) found that stereopsis is recali­ parity information in determining perceived sign ofdepth brated by prolonged adaptation to conflicting SFM infor­ (measured by direction-of-rotationjudgments). However, mation. In these studies, wire-frame objects were viewed for transparent stimuli, direction-of-rotation judgments through a telestereoscope that exaggerated the binocular were primarily determined by binocular disparities. disparities projected onto the retinae. Thus, when the ob­ Braunstein et al. also found that 6 subjects who could not jects were rotated, the binocular disparity information was detect any targets on a static stereo test (Randot circles) consistent with an object much deeper than that indicated were able to make accurate direction-of-rotation judg­ by the SFM information. Observers made depth judgments ments for stereo-SFM displays on over 90% of the trials. of static stereo displays before and after a lO-min adap­ Because the direction of rotation is inherently ambiguous tation period. Epstein found that the same amount ofdis­ in an orthographic projection, these static stereo-deficient parity was judged to have less depth after the adaptation observers must have been able to use the binocular dis­ period than before it. These findings seem to provide evi­ parity information in the stereo-SFM displays, even dence that SFM dominates stereopsis, and may indeed though they did not respond correctly to binocular dis­ solely determine the magnitude of perceived depth when parities on a standard static stereo test. combined with binocular disparities, as suggested by Rouse et aI. (1989) examined this effect further, and Richards. However, Fisher and Ebenholtz (1986) showed found that 7 out of 11 static stereo-deficient observers that depth aftereffects, of the type reported in the previous could judge, with perfect accuracy, direction of rotation research, occurred even when stereo and SFM were not with combined stereo-SFM displays similar to those used presented together in the adapting stimulus. Their results by Braunstein