Augmented Reality Glasses State of the art and perspectives Quentin BODINIER1, Alois WOLFF2, 1(Affiliation): Supelec SERI student 2(Affiliation): Supelec SERI student Abstract—This paper aims at delivering a comprehensive and detailled outlook on the emerging world of augmented reality glasses. Through the study of diverse technical fields involved in the conception of augmented reality glasses, it will analyze the perspectives offered by this new technology and try to answer to the question : gadget or watershed ? Index Terms—augmented reality, glasses, embedded electron- ics, optics. I. INTRODUCTION Google has recently brought the attention of consumers on a topic that has interested scientists for thirty years : wearable technology, and more precisely ”smart glasses”. Howewer, this commercial term does not fully take account of the diversity and complexity of existing technologies. Therefore, in these lines, we wil try to give a comprehensive view of the state of the art in different technological fields involved in this topic, Fig. 1. Different kinds of Mediated Reality for example optics and elbedded electronics. Moreover, by presenting some commercial products that will begin to be released in 2014, we will try to foresee the future of smart augmented reality devices and the technical challenges they glasses and their possible uses. must face, which include optics, electronics, real time image processing and integration. II. AUGMENTED REALITY : A CLARIFICATION There is a common misunderstanding about what ”Aug- III. OPTICS mented Reality” means. Let us quote a generally accepted defi- Optics are the core challenge of augmented reality glasses, nition of the concept : ”Augmented reality (AR) is a live, copy, as they need displaying information on the widest Field Of view of a physical, real-world environment whose elements are View (FOV) possible, very close to the user’s eyes and in a augmented (or supplemented) by computer-generated sensory very compact device. Moreover, the images must be displayed input”. so as to appear far from the user, in order for the eye not It must not be confused with Virtual Reality, which con- to be forced to accommodate, which would be a source of sists in presenting to the user a completely alternate reality, discomfort and ocular fatigue. Therefore, all devices rely on an simulated from scratch. optical system which is in charge of forming images at infinity. The concept of augmented reality is part of a wider one, A number of systems have been designed by industrials to which is mediated reality[1]. It defines not only the fact of achieve this goal. Here are three of them which give a good superposing computer-based information on real images, but outlook on the main technologies used so far. also the modification of those said images. For example, Steve Mann, ”father of wearable computing”, imagined a smart A. Heads Up Diplay welder mask which could shadow the welding arc.[2] Heads Up Displays (HUD) aim at superimposing computer- Therefore, devices such as Google Glass can not be defined created images on user’s FOV. Today, they rely on two main as ARG (”Augmented Reality Glasses”), but should be refered different opical systems which are presented in the next lines. to as ”ubiquitous computers” as they consist more of a 1) Classical systems: Though not fully accurate, the figure complementary screen than of a device capable of altering 2 shows the principle of most basic HUD which corresponds reality before presentig it to the user. This difference being to the system used in Google Glass. It consists of several core now clear in the reader’s mind, following lines will focus on components : allowing the system to display information, it can not alter the reality seen by the user. This can be done by chosing a more complex optical system, know as EyeTap. B. EyeTap EyeTap is the name of a system imagined and designed by Steve Mann’s team which is based on the same principles that HUD.[4] However, this system includes a complementary component named ”amerac” as well as a camera so that the Fig. 2. Google Glass optical system principle system sees what the user sees, as one can see on figure 4. These two added components, linked by some processing blocks that allow focus and zoom control, allow the super- • A compact, very dense source display position of perfectly synchronised artificial rays of light on • A magnifying system in charge of giving the image a natural ones, which enables a much better mixing of added satisfying size informations and reality. • A system charged of placing the image at infinity. Here Moreover, by replacing the beamsplitter by a two-sided both these operations are processed by the parabolic mirror, user gets isolated from direct ambient light and reality mirror placed on the left can be mediated before being presented to them; e.g., a very • A beamsplitter (an optical component that lets a propor- lighted zone of FOV may be shadowed artificially. tion of light go trough and that reflects the other part) Therefore EyeTap is much more powerful than simple HUD which allows the superposition of light coming from the and can fully answer the challenges of mediated (including display on the user’s field of view. augmented) reality. However, it seems much more complex However, this system uses classical optical components as far as technology is concerned. Besides, both HUD and that are difficult to integrate in a limited space. Besides, the EyeTap use a physical display which is both energy-consuming unavoidable use of a beamsplitter at an angle of 45 degrees and difficult to integrate in compact devices. Hence, some does not allow a wide FOV with limited dimensions. Hence new technologies draw the interest of scientists and industrials, the interest of another system, used for example by a French among them Virtual Retina Displays (VRD) which appear to company, Optinvent, in their ARG : diffraction based see- be a very promising alternative to the use of a physical display. through video glasses. 2) Diffraction based systems: Diffraction based systems use a light guide which is in charge of carrying the image from the output of the collimator to the user’s FOV. Until recently, it was almost impossible to use for commercial purpose as technology was both expensive and fragile. An innovation called ”Clear VU” has recently made it wearable and affordable.[3] Fig. 3. Optical guide used in Clear VU Fig. 4. Eyetap functionnal diagram The principle of this technology is depicted on figure 3. The image created on the microdisplay is collimated and the ray beam is then guided through the device by Total C. Virtual Retina Display Internal Reflection, before being impressed on the eye through Functionnal diagram of the system is presented on figure 5. a set of micro-mirrors. The turning point achieved by Clear The principle of VRD relies on a system which uses a VU is the realization of this system in molded plastic. It photon source - typically a laser - to impress what could be allows the microdisplay and the collimator, which remain the called pixels directly on the user’s retina. It therefore involves biggest unavoidable components to be placed more discreetly. a scanning system which is in charge of pointing the laser to Therefore, it partial answers the challenge of integration, each point of the retina at high speed.[5] compactness and design. Therefore, VRD is likely to be able to answer the prob- However, both these systems do not fully answer to the lematics of compactness, optical aberrations and even power ambitions of mediated (and augmented) reality as, though consumption. As such, VRD seems to be the future of ARG, as Fig. 5. Virtual Retina Display functionnal diagram Fig. 6. Basic Electronic Schematic of a typical interactive ARG far as optics are concerned. However, this is so far an unready technology, but it is very likely that ARG will soar when VRD B. Fully embedded solution : GOOGLE GLASS will be perfectly mastered.[6] Google is currently building their Glasses as a standalone Android device. Most of the computing will be done on-board, IV. ELECTRONICS AND INTEGRATION and the hardware will be built accordingly.[7] Until the announcement of the Google Glass Project [7], Their architecture resembles closely to the one of a common in 2011, the different Augmented Reality Glasses (ARG) smartphone : a TI System-on-Chip (SoC), the OMAP 4430, products could easily be divided into two categories : HEAD- based on a 1.2GHz dualcore ARM processor with a dedicated MOUNTED DISPLAYS and HEADS-UP DISPLAYS. The former multimedia hardware accelerator, drives the display. That chip regroups the different devices able to display a 2D or a 3D was even used for a 2011 Google/Samsung phone, the Galaxy image on the whole user’s field of view, while obstructing Nexus. the wearer’s vision, whereas the latter regroups the devices Connectivity is ensured by WiFi and Bluetooth to maximize displaying images while still allowing the user to view his compatibility. On-board sensors include a 9-axis sensor, the surroundings. MPU9150 by INVENSENSE, comprising an accelerometer, The first group could achieve a better screen resolution, gyroscope and magnetometer and accounting for a total orien- while severely impairing the view of the wearer’s whereabouts. tation awareness of the device, but also a GPS chip, a touch However, both device types had something in common : they pad and microphone for user interaction, and headphones. The were not designed to interact with the wearer. Simply put, whole system is powered by a 570mAh Li-Poly battery, giving they were output devices with no responsiveness to stimuli, a 5 hour ”normal utilization” time between charging.