SPATIAL INPUT/DISPLAY CORRESPONDENCE IN A STEREOSCOPIC COMPUTER GRAPHIC WORK STATION Christopher Schmandt Architecture Machine Group Massachusetts Institute of Technology ABSTRACT The fields with the most interest in three-dimensional display are those with a An interactive stereoscopic computer need to analyze complex spatially struc- graphic workspace is described. A conven- tured real world data, gathered by or part tional frame store is used for three- of a process requiring expensive data dimensional display, with left/right eye acquisition hardware (hence the willing- views interlaced in video and viewed ness to experiment with relatively expen- through PLZT shutter glasses. The video sive displays). Although a variety of nonitor is seen reflected from a half techniques exist for stereoscopic display, silvered mirror which projects the graphics there has been little work done in inter- into a workspace, into which one can reach acting with 3-D data by 3-D modes: using and manipulate the image directly with a motion parallax to look around objects "magic wand". The wand uses a magnetic [Fisher, Littlefield], peeling away inter- six degree-of-freedom digitizer. In an fering data layers, or constructing addi- alternative configuration, a graphics tional data types in image space itself. tablet was placed within the workspace for input intensive tasks. CR catagories: 1.3.2 [Computer Graphics]: Graphics Systems; 1.3.6 [Computer Graphics]: Methodology and Techniques; 1.4.8 [Image Processing]: Scene Analysis- Stereo. Key Words: Stereoscopic display, half silvered mirror, three-dimensional digi- tization. INTRODUCTION The last decade has seen a resurgence of interest in three-dimensional imaging, such as holography, commercial 3-D movies, experimental anaglyphic television broad- casts, and lenticular 35mm cameras. Some of this enthusiasm has overflowed into computer graphic applications, most note- ably computer aided design, medical imag- ing, and seismic exploration, but wide- spread appreciation of the utility of stereoscopic displays has yet to happen. This may be in part due to the lack of convincing interactive capabilities of such displays. Permission to copy without fee all or part of this material is granted Figure i. A cutaway view of the work provided that the copies are not made or distributed for direct station, showing the view through the half commercial advantage, the ACM copyright notice and the title of the silvered mirror of the video monitor and publication and its date appear, and notice is given that copying is by user's hand. permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission. © ACM 0-89791-109-1/83/007/0253 $00.75 253 In short, the blossoming of conventional two-dimensional interactive computer graphics has yet to penetrate the three- dimensional world. Such graphic inter- activity requires three components: display technology (e.g., digital frame store), input technology (e.g., tablet, mouse) and algorithms or techniques link- ing the two. This paper will briefly survey existing three-dimensional display and input technologies. It will then focus on a style of interaction characterized by spatial correspondence between the display and input devices; we desire to build a stereoscopic workspace which will allow a user to reach into the midst of a three- dimensional image and interact with it at exactly that point in space which he is Figure 3. The PLZT glasses and electronic touching. controller box. The workstation display is a conventional color frame buffer using television inter- conditions [Stover, Fuchs]. Two CRTs may lace and special opto-electric shutter be used in a head mounted display, using glasses for stereoscopic viewing. The optics to deliver each image to the monitor is viewed through a half-silvered respective eye [Sutherland, Callahan]. mirror, which projects its image into an Two CRTs may be viewed through orthogonal illuminated workspace (figure i). The pairs of polarizers, as is commonly done user views his hand and the graphics, in movie theater projection, although mixed by the mirror, overlapping in three- crosstalk may be a problem and the viewer dimensional space. Input technology has must keep his head fairly erect. Two included a six degree-of-freedom magnetic images may be mixed on a single CRT as a digitizer and a conventional magneto- color anaglyph and viewed through red and strictive tablet. green/blue filter glasses. Finally, two images may be time-multiplexed on the CRT A series of experiments will be discussed. and viewed through electrically controlled Early experiments exercised the work glasses [Roese]. station, using the magnetic digitizer, in various applications; despite early optimism, numerous limitations on inter- The latter technique was used in our work. action were noticed. After discussing Conventional interlaced video actually these, we will explore alternative appli- displays two fields sequentially; one cations, with more clearly defined inter- field consists of the even scan lines, the action goals, as well as a second work- other of the odd lines. Using an ordinary station configuration which uses two- frame buffer, the right eye view is drawn dimensional tablet input to build 2½-D on even lines, with the left eye view on databases. odd lines (figure 2). This mixed image is viewed through wafers of a lead lanthanum DISPLAY TECHNOLOGY zirconate titanium (PLZT) ceramic, a very fast electrically triggered shutter Stereopsis is the perception of depth from (figure 3). Electronics detect the video the binocular fusion of the slightly off- vertical interval, alternately opening set views of our left and right eyes. and closing each eye's shutter while the Stereo display systems must somehow cor- corresponding field is being displayed. rectly generate left and right eye views, Our controller and PLZT glasses were and present them separately, avoiding obtained from Honeywell, with additional crosstalk, to the respective eyes. As glasses from Matsushita. this paper focuses on real-time inter- active computer generated display of In terms of the computer graphics algo- information, we will limit this section to rithms, the depth of an object is a func- various forms of computer controllable tion of the distance between the even and video display [Okoshi]. odd line portions of the image (figure 4). Zero disparity makes the graphic appear on A number of techniques for display of the surface of the screen, as we are three-dimensional video have been devel- accustomed to viewing. When the horizontal oped. Points may be displayed on a CRT position of left and right eye portions in synchronization with a vibrating mir- of the object are different, eye conver- ror, which generates a true three- gence will make the object appear behind dimensional image under ordinary viewing or in front of the screen. This relation- ship can be expressed as: 254 interfaced to our laboratory's machines (Perkin-Elmer 3230's). f~ dx = IO (D~) Three mutually orthogonal dipole coils in the radiator generate a rotating magnetic where dx is image disparity, d the apparent field, which induces currents in the distance of the object from the screen, sensor, a plastic cube about 1.5 cm on Ds the viewer's distance from the screen, edge which also contains a similar array and IO the interocular distance (about of receiver coils (figure 5). The sensor 65mm). is small and light, connected to nearby preamplifiers by a thin, flexible cable. As position is sensed magnetically, there screen are no awkward linkages, and a user's body plane does not interfere with the device (although, as will be discussed below, color monitors do). 4 ..... --_ For the purposes of this work station, a sensor has been mounted at the end of a small wooden "magic wand" which also has 4 .......... ;-x.I a momentary contact bush button switch I I placed within the handle. The wand, switch, and cable are black, while a white D~ a spot on the sensor aids visibility and indicates the "active" portion of the wand. Figure 4. Ocular convergence and depth The radiator is mounted out of sight perception. The left eye sees an object behind the work area. In later versions at Xl on the screen, the right eye sees it of the workspace, a conventional magneto- at Xr, with disparity dx. The object will strictive graphics tablet, also painted be perceived to lie at distance d behind black with a white spot on the puck, was the screen. Note that if dx were negative used for input. the object would appear in front of the screen. INPUT TECHNOLOGY Digitizing three dimensional data points for system input has been a more difficult problem than stereo display. One approach uses various forms of mechanical linkages (such as fishing line) connected to measuring devices [Roberts, Clark]. Although high accuracy may be obtained, these linkages limit freedom of movement and are somewhat awkward. Other techniques use infrared light sources, worn on the body, as transmitters and light sensing hardware as recievers tQ track multiple points in three space Figure 5. The Polhemus radiator and wand, [Burton, Woltring]. Although the commer- with the sensor mounted near its end. cial product (Selspot) is quite expensive, recent work indicates lower cost tracking may be possible in simplified configura- THE WORKSPACE tions [Ginsberg]. The main problem with light transmission is obscuration by the The goal in workspace design was to allow user's body. a style of interaction in which spatial correspondence between the particular We chose a magnetic six degree-of-freedom input and output devices could be main- digitizer developed by Polhemus Naviga- tained. We wished to be able to project tional Sciences [Polhemus, Rabb] ; while the stereoscopic image into an unobscured still fairly expensive, it avoids most space which would allow one to reach into problems of previous technologies and is the projection with the wand and modify very accurate under ordinary circumstances.