Introduction to Augmented Reality R. Silva, J. C. Oliveira, G. A. Giraldi National Laboratory for Scientific Computation, Av. Getulio Vargas, 333 - Quitandinha - Petropolis-RJ Brazil rodrigo,jauvane,[email protected] ABSTRACT This paper presents an overview of basic aspects of Augmented Reality (AR) and the main concepts of this technology. It describes the main fields in which AR is applied nowadays and important AR devices. Some characteristics of Augmented Reality systems will be discussed and this paper will provide an overview of them. Future directions are discussed. Keywords: Augmented Reality, Virtual Reality, Scientific Visualization 1 INTRODUCTION Augmented Reality (AR) is a new technology that involves the overlay of computer graph- ics on the real world (Figure 1). One of the best overviews of the technology is [4], that defined the field, described many problems, and summarized the developments up to that point. That paper provides a starting point Figure 1: AR example with virtual for anyone interested in researching or using chairs and a virtual lamp. Augmented Reality. In telepresence, the fundamental purpose is AR is within a more general context termed to extend operator’s sensory-motor facilities Mixed Reality (MR) [20], which refers to and problem solving abilities to a remote en- a multi-axis spectrum of areas that cover vironment. In this sense, telepresence can be Virtual Reality (VR), AR, telepresence, and defined as a human/machine system in which other related technologies. the human operator receives sufficient infor- mation about the teleoperator and the task Virtual Reality is a term used for computer- environment, displayed in a sufficiently nat- generated 3D environments that allow the ural way, that the operator feels physically user to enter and interact with synthetic en- present at the remote site [26]. Very similar vironments [9][27][28]. The users are able to to virtual reality, in which we aim to achieve “immerse” themselves to varying degrees in the illusion of presence within a computer the computers artificial world which may ei- simulation, telepresence aims to achieve the ther be a simulation of some form of real- illusion of presence at a remote location. ity [10] or the simulation of a complex phe- nomenon [33][9]. AR can be considered a tecnology between 1 VR and telepresence. While in VR the envi- 2 AR Components ronment is completely synthetic and in telep- resence it is completely real, in AR the user 2.1 Scene Generator sees the real world augmented with virtual objects. The scene generator is the device or software responsible for rendering the scene. Render- ing is not currently one of the major problems When designing an AR system, three aspects in AR, because a few virtual objects need to must be in mind: (1) Combination of real and be drawn, and they often do not necessarily virtual worlds; (2) Interactivity in real time; have to be realistically rendered in order to (3) Registration in 3D. serve the purposes of the application [4]. Wearable devices, like Head-Mounted- 2.2 Tracking System Displays (HMD) [28], could be used to show The tracking system is one of the most impor- the augmented scene, but other technologies tant problems on AR systems mostly because are also available [4]. of the registration problem [3]. The objects in the real and virtual worlds must be prop- erly aligned with respect to each other, or Besides the mentioned three aspects, another the illusion that the two worlds coexist will one could be incorporated: Portability. In be compromised. For the industry, many ap- almost all virtual environment systems, the plications demand accurate registration, spe- user is not allowed to go around much due to cially on medical systems [16][4]. devices limitations. However, some AR ap- plications will need that the user really walks through a large environment. Thus, portabil- 2.3 Display ity becomes an important issue. The tecnology for AR is still in development and solutions depend on design decisions. For such applications, the 3D registration be- Most of the Displays devices for AR are HMD comes even more complex. Wearable com- (Head Mounted Display), but other solutions puting applications generally provide unreg- can be found (see section 3). istered, text/graphics information using a monocular HMD. These systems are more When combining the real and virtual world of a ”see-around” setup and not an Aug- two basic choices are available: optical and mented Reality system by the narrow defini- video technology. Each of them has some tion. Henceforth, computing platforms and tradeoffs depending on factors like resolution, wearable display devices used in AR must be flexibility, field-of-view, registration strate- often developed for more general applications gies, among others [4]. (see section 3). Display technology continues to be a limit- ing factor in the development of AR systems. The field of Augmented Reality has existed There are still no see-through displays that for just over one decade, but the growth and have sufficient brightness, resolution, field of progress in the past few years has been re- view, and contrast to seamlessly blend a wide markable [12]. Since [4], the field has grown range of real and virtual imagery. Further- rapidly. Several conferences specialized in more, many technologies that begin to ap- this area were started, including the Inter- proach these goals are not yet sufficiently national Workshop and Symposium on Aug- small, lightweight, and low-cost. Neverthe- mented Reality, the International Sympo- less, the past few years have seen a number sium on Mixed Reality, and the Designing of advances in see-through display technol- Augmented Reality Environments workshop. ogy, as we shall see next. 2 3 AR Devices Four major classes of AR can be distin- guished by their display type: Optical See- Through, Virtual Retinal Systems, Video See-Through, Monitor Based AR and Pro- jector Based AR. The following sections show the correspond- ing devices and present their main features. Figure 2: Optical See-Through HMD. 3.1 Optical See-Through HMD Optical See-Through AR uses a transparent Head Mounted Display to show the virtual environment directly over the real world (Fig- ures 2 and 3). It works by placing optical combiners in front of the user’s eyes. These combiners are partially transmissive, so that Figure 3: Optical See-Through Scheme. the user can look directly through them to see the real world. The combiners are also Recent Optical See-Through HMD’s are be- partially reflective, so that the user sees vir- ing built for well-known companies like Sony tual images bounced off the combiners from and Olympus and have support for occlusion, head-mounted monitors. varying accommodation (process of focusing the eyes on objects at a particular distance). Prime examples of an Optical See-through There are very small prototypes that can be AR system are the various augmented medi- attached to conventional eyeglasses (Figure cal systems. The MIT Image Guided Surgery 4). has concentrated on brain surgery [15]. UNC has been working with an AR enhanced ultra- sound system and other ways to superimpose radiographic images on a patient [23]. There are many other Optical See-through systems, as it seems to be the main direction for AR. Despite of these specific examples, there is still a lack of general purpose see-through Figure 4: Eyeglass display with holo- HMDs. One issue for Optical See-through graphic element. AR is the alignment of the HMD optics with the real world. A good HMD allows adjust- ments to fit the eye position and comfort of 3.2 Virtual Retinal Systems individual users. It should also be easy to move it out of the way when not needed. The VRD (Virtual Retinal Display) was in- However, these movements will alter the reg- vented at the University of Washington in istration of the VE over the real world and the Human Interface Technology Lab (HIT) require re-calibration of the system. An ex- in 1991. The aim was to produce a full pensive solution would be to instrument the color, wide field-of-view, high resolution, high adjustments, so the system could automagi- brightness, low cost virtual display. Microvi- cally compensate for the motion. Such de- sion Inc. has the exclusive license to com- vices are not reported in the literature. mercialize the VRD technology (Figure 5). 3 This technology has many potential applica- worlds is much easier. There are a variety of tions, from head-mounted displays (HMDs) solutions available including chroma-key and for military/aerospace applications to medi- depth mapping. Mixed Reality Systems Lab cal purposes. (MRSL) of Japan presented a stereo video see-through HMD at ISAR 2000. This de- The VRD projects a modulated beam of light vice addresses some of the parallax related (from an electronic source) directly onto the to location of the cameras vs eyes. retina of the eye producing a rasterized im- age (Figure 6). The viewer has the illusion of seeing the source image as if he/she stands two feet away in front of a 14-inch monitor. In reality, the image is on the retina of its eye and not on a screen. The quality of the im- age he/she sees is excellent with stereo view, full color, wide field of view and no flickering characteristics [13][24]. Figure 7: Video See-Through HMD. Figure 5: Virtual Retinal System HMD. Figure 8: Video See-Through Scheme. 3.4 Monitor Based Monitor Based AR also uses merged video streams but the display is a more conven- tional desktop monitor or a hand held dis- play. It is perhaps the least difficult AR Figure 6: Virtual Retinal System Scheme.