An Overview of Autostereoscopy as Used in Augmented and Virtual Reality Systems Brian Valerius University of Minnesota, Morris 600 East 4th Street [email protected] ABSTRACT The technology of today has allowed companies to prototype Displaying images in 3-dimensional form can prove useful virtual models through 3D displays, thus saving time and for performing certain tasks, and also for entertainment pur- money on physically building them. Electronic communica- poses. In this paper we discuss 3-dimensional imaging with- tion is widely used throughout the world, and 3D imaging out the use of external hardware for the user. This type of can help provide a more realistic feel to human interactions 3-dimensional imaging is becoming more and more popular over a distance. as related technologies continue to develop. We also look We describe how autostereoscopy can be used for sev- at how 3-dimensional imaging can be used with augmented eral practical purposes. Background on stereoscopy and au- and virtual realities. Topics of commercial and professional tostereoscopy are given, as well as an example of a failed use are discussed, along with some knowledge of how these attempt at creating a stereoscopic video game console. We uses work for some 3-dimensional displays. also describe how autostereoscopy can be used for virtual prototyping and live display. How autostereoscopy can be used in augmented and virtual realities is also discussed. Categories and Subject Descriptors I.3.1 [Computer Graphics]: Hardware architecture|Three- 2. BACKGROUND dimensional displays; I.3.7 [Computer Graphics]: Three- Dimensional Graphics and Realism 2.1 Stereoscopy Stereoscopy creates an illusion of three-dimensional depth General Terms from images on a two-dimensional plane. It works by pre- Design, Human Factors senting a slightly different image to each eye. It was first invented by Sir Charles Wheatstone in 1838 [8]. Many 3D Keywords displays use this method to display images. Stereoscopy is used in many places today. It can be used Stereoscopy, Autostereoscopy, 3-Dimensional Imaging, Par- to view images rendered from large multi-dimensional data allax Barrier, Lenticular Lenses, Augmented Reality, Virtual sets such as ones produced by experimental data. Stere- Reality oscopy is used in photogrammetry, the practice of determin- ing the geometric properties of objects from photographic 1. INTRODUCTION images. Stereoscopy is often used for entertainment pur- poses, such as through stereograms. A stereogram is an Technology for displaying images in 3-dimensional (3D) optical illusion of depth created from flat, two-dimensional form is becoming more widely used. Although movie the- image or images. Stereograms were originally two stereo im- atres have been showing 3D movies for many years, the pro- ages that were able to be viewed through a stereoscope. A gression of technology for displaying 3D images has grown user would look through the stereoscope and view one image rapidly throughout recent years. There is a trend to want to through the left eye and one image through the right eye, display 3D images without the use of external utilities worn thus creating the 3D effect. Today there are several other by the user. There are multiple ways of achieving this type ways to display stereograms that are not described in this of 3D imaging, some of which are described in this paper. paper, such as rapidly switching between images to achieve Beyond entertainment, there are many practical uses for a similar effect. displaying 3D images without the use of external hardware. 2.2 Failed Stereoscopic System: Virtual Boy The Nintendo Virtual Boy was first introduced in July of 1995. It was the first dedicated stereoscopic video game con- sole released to the public. It is considered the most famous This work is licensed under the Creative Commons Attribution- market failure by Nintendo, a large video gaming company. Noncommercial-Share Alike 3.0 United States License. To view a copy The Virtual Boy console did not last a year on the market of this license, visit http://creativecommons.org/licenses/by-nc-sa/3.0/us/ or before it was pulled. Its number of unique videogames was send a letter to Creative Commons, 171 Second Street, Suite 300, San Fran- cisco, California, 94105, USA. limited to 22, with 14 games released in North America and UMM CSci Senior Seminar Conference Morris, MN. 19 released in Japan. Challenges for Success in Stereo Gaming: A Virtual Boy Case Study Matt Zachara José P. Zagal College of Computing and Digital Media College of Computing and Digital Media DePaul University DePaul University 243 South Wabash Ave. 243 South Wabash Ave. Chicago, IL 60604, USA Chicago, IL 60604, USA [email protected] [email protected] A BST R A C T computers and consoles rendering increasingly sophisticated and Stereo video stands to revolutionize the medium of videogames in life-like images. Very little progress seems to have been made, the same way that stereo sound revolutionized the audio however, in how we actually see virtual worlds. experience. It is a pending revolution. Despite years of research, This apparent lack of progress is dissociated with an upsurge of development, and hype, 3D stereo video ubiquity in videogames interest in stereoscopic video gaming. Popular magazine Game has yet to be observed. 8VLQJ 1LQWHQGR¶V 9LUWXDO %R\ (VB) Informer recently discussed some of the newer technologies gaming platform as a case-study, we explore why the revolution reaching the mass market [1] while Edge magazine explored how KDVQ¶W yet happened. We identify six factors that played a independent game designers are looking at what novel game VLJQLILFDQW UROH LQ 9%¶V IDLOXUH its lack of defined identity as a experiences can be designed with stereoscopic video [2]. product, a comparatively weak display, its socially isolating game However, is the current excitement simply the result of unbridled experience, purported negative effects, the challenges in optimism? Perhaps, but despite the continuous evolution in explaining and demonstrating stereoscopic gaming, and its lack of improving 3D technology, the adoption of stereoscopic a must-have game. We note that the factors we identify DUHQ¶WMXst technologies for gaming purposes has been preceded with a technological and that they interact in confounding ways. Nearly caution: videogame history seems littered with devices, fifteen years after the introduction of the VB, new technologies contraptions, and games with unfulfilled promises to deliver the may address the original 9%¶V technical shortcomings, but not ultimate game immersion experience. necessarily the others. There is currently still an issue with raising WKHEDURQFRQVXPHU¶VH[SHFWDWLRQVDVZHOODVencouraging game Headfinder Controls (a) (b) Prism Mask designers to explore the design space offered by stereoscopic Movable video. Prism Mask 1 Pixel Keywords Right Virtual Boy, stereoscopic, case study, virtual reality, head- Eye mounted display, depth perception, 3D, LCD shutter glasses 1. IN T R O DU C T I O N (c) (d) In interactive entertainment, 3D graphics usually refers to a rendered illusion of a three-dimensional space; the player understands objects are not flat because the environment rotates, Standard TFT Panel giving a sense of tangible perspective. However, these graphics Left are not really tridimensional in their representation since the eyes Eye are constantly focused on a single surface, such as a television display. In stereo video displays, however, each eye is presented with an independent image source such that the brain is able to Figure Figure1 ± Virtu 1:al B Theoy un Virtualit with at Boytache consoled controll [9].er & three Figure 9: Principle of an autostereoscopic display [29]. interpret the perceived depth of field based on the subtle offsets game cartridges featured Figure 2: Principle of an autostereoscopic display between the two visual inputs. Similarly to how we use our eyes In this article we explore the multiple challenges faced by stereo [5]. in the real world, there is a need to refocus depending on layer technologiesVirtual Boy forworked interactive based entertainment on oscillating by mirrorsanalyzing that what dis- is tributed among the columns by rendering textured lines. Again, that is being visually examined ± essentially stimulating natural on certain architectures it may perform better if for the first im- perhapsplayed athe linear Figurehighest-profile array 8: Finite of elements lines, commercial powered from Fig. videogame 5 by have light been emitting fixed:platform (a) edge is eye reflexes to visual cues. Thus, the illusion of depth is achieved. age a different strategy is chosen, e.g. distributing the columns via failurediodes(LEDs), 1LQWHQGR¶Vcollapsed one 9LUWXDO to for common each %R\ node;eye[3] [9].( (b)Figure triangle The 1). user andThe adjacent placed Virtual quadrilateral their Boy isway thatglCopyPixels alternating.Fig.10(c)showsthecorrespondingresultusing columns can be seen either with the left storyface againstprovidesdivided; the insight eyepiece, (c) to stretched the largely which and adjacent blockedignored elements genre out externalof are stereoscopic divided; light, (d) adjacent or withthis the method right and eye. in Fig. One 11 approach and 1(c) one isto can use seea autostereoscopic prism mask Curiously, advancements in entertainment displays have primarily triangles are merged if possible. addressed two issues: how to display an image and how to render JDPLQJ1LQWHQGR¶VHDUO\PLVWDNHVVHWWKHVWDJHIRUGLVFXVVLQJWKHand