Qualitative Aspects of Chromo-Stereoscopy for Depth Perception * Thierry Toutin Abstract viewing a color-encoded composite image using chromo-ste- The display of three-dimensional (30) quantitative data sets reoscopy, the stereo effect is based on physiological and psy- is a basic topic of research in cartography, image processing, chological factors of color and depth. and applications related to spatial information. A new appli- The first attempt to produce a chromo-stereoscopic im- cation for data visualization and analysis, which combines age from remote sensing data (Toutin and Rivard, 1995) re- color vision and depth perception, has been developed using sulted in an interesting 3-D product with some artifacts. the effectknown as chromo-stereoscopy based on Eintho- Published as a highlight article in the October 1995 issue of ven 's -theory. PEbRS, it briefly described the process of depth perception, It enables the generation of flat color composite images the chromo-stereoscopic method, and the generation of the from multisource data in which depth information is coded cover page image. There was no detailed development of into colors. When viewed with double prism refraction these different aspects, nor any analysis of the relationships ChromaDepthTMglasses, a "dramatic" 30 effect is produced. between the processing steps and the parameters. Following a description of the method, the geometric and ra- In order to better understand the different components diometric processing parameters are qualitatively analyzed to (eyes, glasses, images) of the chromo-stereoscopic method, assess their impact on the quality of the chromo-stereoscopic this paper will expand on the earlier article and address the images and depth perception. different factors (physical, physiological, psychological) which influence the generation and the perception of chromo-stereoscopic~mages.This method examines depth Introduction perception qualitatively; it does not extract quantitative Through the normal process of vision, we unconsciously eval- depth information which can be directly obtained from the uate the forms, shapes, distances, and colors of a vast number input data. Therefore, only the qualitative aspects of the pro- of objects around us. However, to successfully represent three- cess are discussed. Because the eye and the brain are part of dimensional (3D) information on a flat surface such as an im- the "sensor" which acquires and perceives information, age or painting, both the perceptual and conceptual levels of some key aspects of their important role in this method will understanding spatial relationships must be combined. Illu- be examined. After reviewing stereo-viewing processes ap- sions of perceptual space can be generated by use of the linear plied in remote sensing and describing in detail the eye as a perspective system, taking advantage of the viewer's concep- sensor, the chromo-stereoscopy method and the Chroma- tual knowledge of the perspective phenomena. DepthTMglasses, developed by Steenblik (1986), are exam- Psychological research has indicated that performance at ined. Finally, the technique to control the input parameters searching a display is much improved if one knows something of the geometric and radiometric processing is analyzed and beforehand about what is to be looked at (Smith, 1962). Be- tested on different images. cause psychological factors play a major role in perception, the remote sensing expert can "go beyond the information Review of Stereo-Viewing Methods given" in the display of an image. Thus, a viewer with some a Different methods have been developed to recreate depth priori knowledge of the data and of the terrain and with a perception. This paper is not intended to fully address all good understanding of the processing has a more qualitative three-dimensional imagery techniques. Only those most com- experience. It has been suggested (Hoffman, 1990) that re- monly used, especially in remote sensing, are described. searchers might devote more time to studying and integrating Okoshi (1976) is a good reference for more details. Depth these qualitative aspects of the remote sensing process. perception can be "natural," with two images taken from dif- Chromo-stereoscopy offers a tool to qualitatively per- ferent view points, or "synthetic," to generate a stereo-pair ceive depth. Differently colored objects at the same viewing or a perspective view. For the latter, depth perception can be distance can often appear to lie at different depths (Eintho- related to any theme such as terrain elevation, magnetic or ven, 1885). This apparent depth difference can then be en- gravimetric fields, etc. hanced by a refraction process. Therefore, these two known In natural depth perception, stereoscopic pictures are phenomena can be used to perform fusion of multi-source re- viewed separately by each eye: the right viewpoint image to mote sensing data, and to qualitatively perceive depth from the right eye and the left viewpoint image to the left eye. the single stereoscopic fused image. The method is straight- An experienced observer of the stereo-pair may be able to forward: depth is coded into colors, and then decoded by achieve the proper focus and convergence without special means of basic optics to produce depth perception. When *The ChromaDepthT"glasses, provided in the October 1995 issue of Photogrammetric Engineering & Remote Sensing, PEbRS,can be used to perceive the 3D effect on the chromo-stereo- Vol. 63, No. 2, February 1997, pp. 193-203. scopic images. 0099-1112/97/6302-193$3.00/0 Canada Centre for Remote Sensing, 588 Booth Street, Ottawa, O 1997 American Society for Photogrammetry Ontario KIA OY7, Canada. and Remote Sensing PE&RS February 1997 equipment; however, some devices used ordinarily allow configuration, information corresponding to both the ampli- each eye to see the appropriate picture of the pair. The de- tude and the phase of the wave scattered by the object. vice used depends on how the stereo pair was generated. Because holography is the only true 3D photography, a Most commonly, they are conventional stereoscope, ana- hologram can be viewed from different perspectives just as is glyph, or polarized glasses. Recently, 3D shutter glasses are a possible with the objects. For example, if a magnifying glass more sophisticated device used with a computer screen, is recorded on a hologram, the magnifying glass in the holo- where shutter glasses are synchronized with a screen alterna- gram magnifies different objects as we change our point of tively displaying left and right images. view (Friedhoff and Benzon, 1991). As reported by Okoshi (1976), the first attempt at a stere- But, there are practical disadvantages mostly due to the oscopic drawing was a technique devised by ~io;anni Bat- use of coherent light in the recording and reconstruction tista della Porta around the year 1600. Today, stereo- steps: objects must be still and recorded only in a darkroom paintings have been created by depicting left and right per- on high resolution expensive plate, and the result is mono- spectives, as Magritte with his "Man with a Newspaper" chromatic (black and white). However, since 1964 the use of (The Tate Gallery, London, England, 1928), or Dali with his white-light illumination for multicolor reconstruction has "Christ of Gala" (Museum Ludwig, Cologne, Germany, 1978). overcome these drawbacks (Leith and Upatnieks, 1964; Pen- In the same way, many proposals for 3D movies have ap- nington and Lin, 1965). Holography applied to remote sens- peared since the first stereoscopic polarized movie in 1939: some ing data has been also realized (Benton et al., 1985). They relied upon parallax effects with simultaneous projections on the generated white-light viewable holographic stereograms ob- screen; others, like "Cinerama" in the 1950s, relied upon psycho- tained by digital and optical processing of a Landsat-MSS logical illusion to give a strong, realistic 3D sensation by sur- stereo pair which produced a black-and-white 3D image of rounding the spectators through a wide angle projection. the Earth's surface. More recentlv in 1995, Sharu Laboratories researchers in Europe claimed a* major advancein 3D moving image tech- The Eye as a Sensor nology (Toutin and Rivard, 1995). To produce the 3D image, The eye is the organ through which humans acquire knowl- "twin-LCD (Liquid Crystal Displays)" are placed at right edge of their environment by virtue of the light reflected angles to each other. The images are then combined by a from, or emitted by, the objects within the environment. For proprietary optical filter that transmits the image from one humans, the information provided by the eyes undoubtedly display, and reflects the image from the other, producing a plays the dominant role in our interpretation of the environ- 3D image. A sensor on top of the device detects the position ment. But the power to integrate the viewed image to recog- of the viewer's head on which a tiny silver spot has been nize its contour, its color, and its relation with other objects placed. The images are then updated on the screen in such a indicates that the process of vision does not merely consist way that they appear to be at a constant angle of 2 20". Be- of "seeing," but also of "perceiving and understanding" tween these limits, the system provides the ability to "look" through the central nervous system. The eye, considered as around the objects without image flipping (Walko, 1995). part of the brain, is fundamentally an organizer. The eye/ But most of these stereoscopic devices have a common brain, starting with the activity of the retina, is actively feature: the presentation of a separate view to each eye with building a world of objects. This suggests that a priori horizontal binocular parallax. Jones et a1. (1984) developed a knowledge is useful for a better interpretation and under- new 3D imaging technique which employs a modified alter- standing of the image: to have a clear idea of what to look nating-frame system.