ISWC '17, SEPTEMBER 11–15, 2017, MAUI, HAWAII, USA Open Source EyeTap: Empowering Every Maker with Phenomenal Augmented Reality and Wearable Computing Sarang Nerkar Max Hao Lu Abstract University of Toronto University of Toronto Augmented reality and wearable computing development Toronto, ON M5S 3G4, Canada Toronto, ON M5S 3G4, Canada has skyrocketed in the consumer product domain in the [email protected] [email protected] past few decades, while the open source domain remained rather neglected. We believe that this is due to the existing development platforms being expensive, closed-source and Sen Yang Alex Papanicolaou due to the lack of a strong open source augmented reality University of Toronto University of Toronto community. Toronto, ON M5S 3G4, Canada Toronto, ON M5S 3G4, Canada [email protected] [email protected] We present a hardware and software open source wearable augmented reality platform which enables users to make augmented reality glasses that cost less than $250 and re- Cindy Jinhee Park Steve Mann alize new applications of augmented reality which can then University of Toronto Stanford University be added to the platform for others to use and contribute to. Toronto, ON M5S 3G4, Canada Stanford, CA 94305, USA [email protected] [email protected] Author Keywords Mediated Reality; Augmented Reality; Wearable Comput- ing; EyeTap; Open Source, Open Innovation; Maker; Devel- oper; Tinkerer; Open Source Community Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed Introduction for profit or commercial advantage and that copies bear this notice and the full citation Augmented reality technology enhances the natural percep- on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, tion of humans by enabling them to sense and interact with to post on servers or to redistribute to lists, requires prior specific permission and/or a things that they normally wouldn‘t be able to, thus augment- fee. Request permissions from [email protected]. ing to the natural capabilities of humans[17]. In his child- ISWC '17, September 11–15, 2017, Maui, HI, USA © 2017 Association for Computing Machinery. hood in the 1970s, Mann invented phenomenological aug- ACM ISBN 978-1-4503-5188-1/17/09...$15.00 mented reality[10], wearable computing, and quantimetric https://doi.org/10.1145/3123021.3123075 203 ISWC '17, SEPTEMBER 11–15, 2017, MAUI, HAWAII, USA self-sensing (later called “quantified self”) which he brought open source framework that is not only open source in the to MIT in the early 1990s to found the MIT wearable com- software end but also at the hardware end, while also being puting project (http://wearcam.org/nn). The miniaturization affordable [13][7]. We are not aware of any existing solu- of the overall system enabled users to wear this system tions that enable the developer community to develop and outside the laboratory environment and use it on a daily ba- contribute to a platform. In this paper, we present a hard- sis[18]. Mann later used this work to co-found InteraXon[4] ware and software open source wearable augmented reality and Metavision (Meta Company)[6]. platform which enables the users to make augmented re- ality glasses that cost less than $250 and realize new ap- Recent implementations of augmented reality are focused plications of augmented reality which can then be added to mostly on overlaying images onto reality. However, the ac- the platform for others to use and contribute to. tual essence of augmented reality isn’t limited to just that, it can also be performed in forms of various seeing or hear- Motivation ing aids [12]. One example is using real time high dynamic Wearable computing and augmented reality have grown ex- range video to enhance the overall dynamic range of the ponentially in the past few decades, in the consumer mar- human eye [16][1][14]. Thus enabling humans to see ob- ket domain. Although, there are various augmented reality jects that are a lot brighter than what their eyes can handle. development glasses available in the market today, these Mann developed one of the first augmented reality seeing products enable developers to develop games and apps for aids, called EyeTap [11]. Since then various augmented the consumer market only. We believe that the real potential Figure 1: Top: One of world‘s first reality headsets have come into place such as Google of augmented reality hasn’t been realized. Current devel- wearable augmented reality Glass, Microsoft Hololens and Meta glasses. There has opment has happened for people living in cities that can af- computer systems developed by been exponential development in the consumer applications ford to spend $1000 or more on augmented reality glasses Mann. Middle: The EyeTap of augmented reality glasses since Mann’s EyeTap [20] . to replace their desktop computers, and this development system. Bottom: Mann‘s recent However, there does not seem to be significant growth in EyeTap, completed in 1998 (19 is mostly closed-source. The applications of augmented the open source augmented reality domain. There are a years ago), which he wears on a reality in rural areas, underprivileged areas and minority daily basis. lot of open source augmented reality software frameworks communities have not been realized yet. We believe that that enable hackers, tinkerers and developers to develop this is because the available technology is expensive and augmented reality applications. But the way these frame- doesn‘t focus on the needs of the billions of people in the works are designed only enables development of con- world. The technology, right now, focuses on the few thou- sumer applications of augmented reality such as games sands who can buy these products. One way of realizing and apps., while prohibiting enthusiasts from having the these applications is by enabling the billions of people to freedom to modify or design hardware systems tailored to find the solutions to their problems with augmented reality their unique applications. Furthermore, these frameworks and wearable computing on their own [5][2][19][3]. This can require the developers to buy the consumer augmented re- be done by making the technology open source and provid- Figure 2: EyeTap principle as in [9] ality glass developer kits that cost over $1000, a cost that ing a platform where people can use and contribute to the most early-stage developers cannot afford. To embody the technology, similar to what Linux did with operating systems true essence of augmented reality our society needs an [15]. This thought of having an open source platform for 204 DESIGN EXHIBITION augmented reality and wearable computing has been the motivation for this project. Modularity: A New Approach in Wearable Eyeglasses We approached various maker communities to realize the requirements for augmented reality development. While conducting this survey we recognized certain attributes that make a device suitable for augmented reality development. We realized that in order for the maker community to real- ize applications in augmented reality that solve problems in their localities, the device needs to be modular and open source, not only in software but also in hardware. Open source and modularity together eliminate the problem of being limited by the given technology, because the makers can just add the missing technology pieces for the develop- ment of their application. This enables open innovation, an approach that has proven to be successful in enabling new specialized application for various professions[3][2][19]. We Figure 5: Modularity enables multiple functionalities in the Open have already seen makers develop different modules with Source EyeTap our system for various applications such as eye tracking, clear vision at a distance and thermal vision(See figure 5). We envision that our system will enable a lot more makers Design Approach to solve problems and enhance the capabilities of people in The existing EyeTaps were made from various materials in- Figure 3: Open Source EyeTap their localities with augmented reality. cluding off-the-shelf safety helmet, CNC milled stainless design as part of Mobile World steel, and injection moulded plastic as seen in figure 1. Congress special fashion show in We tested different wearable eyeglasses and head up dis- We performed an evaluation of the existing manufactur- Barcelona, 2017. plays to see how they compare with our system based on ing methods and witnessed the benefits and drawbacks of the attributes that we realized are suitable for augmented using these methods. We then compared them with our reality development(See figure 6). We found out that cost proposed manufacturing method: 3D printing. Although 3D is a major reason why modularity is not a favorable choice printing is still at the early stages of development and adop- for commercial products. The customization makes it less tion, it shows a promising future in the ergonomic related, Figure 4: Base frame design with effective for mass production. Our system does not face highly customized applications like smart wearables. With separated frame and nose piece this problem because advanced tabletop manufacturing an emphasis on design for everyday use, this Open Source techniques such as 3D printing allow for production of cus- EyeTap design was also used in a fashion show at Mobile tomized personal apparel such as eyeglasses. World Congress 2017, as shown in figure 3. 205 ISWC '17, SEPTEMBER 11–15, 2017, MAUI, HAWAII, USA "# $%! #$ & & & ! Figure 6: Comparison of different wearable eyeglasses and head up displays based on attributes suitable for augmented reality development We have designed spaces to mount external mechanical modules to meet the functional needs of various applica- Design Description tions.