Peripheral Vision: A New Killer App for Smart Glasses Isha Chaturvedi Farshid Hassani Bijarbooneh The Hong Kong University of Science and Technology The Hong Kong University of Science and Technology Hong Kong Hong Kong [email protected] [email protected] Tristan Braud Pan Hui The Hong Kong University of Science and Technology University of Helsinki Hong Kong Helsinki, Finland [email protected] The Hong Kong University of Science and Technology Hong Kong [email protected] ABSTRACT KEYWORDS Most smart glasses have a small and limited field of view. Human perception; field of view; peripheral vision; smart The head-mounted display often spreads between the human glasses head-mounted display; information input central and peripheral vision. In this paper, we exploit this ACM Reference Format: characteristic to display information in the peripheral vision Isha Chaturvedi, Farshid Hassani Bijarbooneh, Tristan Braud, and Pan of the user. We introduce a mobile peripheral vision model, Hui. 2019. Peripheral Vision: A New Killer App for Smart Glasses. which can be used on any smart glasses with a head-mounted In 24th International Conference on Intelligent User Interfaces (IUI display without any additional hardware requirement. This ’19), March 17–20, 2019, Marina del Ray, CA, USA. ACM, New York, model taps into the blocked peripheral vision of a user and NY, USA, 14 pages. https://doi.org/10.1145/3301275.3302263 simplifies multi-tasking when using smart glasses. To display the potential applications of this model, we implement an 1 INTRODUCTION application for indoor and outdoor navigation. We conduct Smartglasses have become increasingly popular in recent an experiment on 20 people on both smartphone and smart years. They provide various applications in information visu- glass to evaluate our model on indoor and outdoor conditions. alization [49], education [16], gaming [41], medical [36] and Users report to have spent at least 50% less time looking at the other commercial industries [2, 15]. Nowadays, most smart- screen by exploiting their peripheral vision with smart glass. glasses embed a small head-mounted screen which spreads 90% of the users Agree that using the model for navigation over the eye of the user. The Angular field of view (AFOV or is more practical than standard navigation applications. AOV) measures the angular extent of a 360-degree circle that is visible by the human eye [6]. Figure 1 shows the AFOV CCS CONCEPTS of the human eye. The foveal system, responsible for the foveal vision, lies within the central and para-central area. • Human-centered computing → User studies; Empiri- The area outside the foveal system is responsible for the pe- cal studies in HCI; • Computing methodologies → Per- ripheral vision [39, 42]. The term Field of view (FOV) is often ception. used interchangeably with AFOV. Most smartglasses have small and limited FOV which restricts their potential applica- tions [34, 46]. The AFOV of Google Glass1 is approximately 30 degrees (as represented Figure 2), which is significantly Permission to make digital or hard copies of all or part of this work for smaller than the AFOV of the human eye. This is the case for personal or classroom use is granted without fee provided that copies are not most smartglasses including MadGaze Glass2. This limited made or distributed for profit or commercial advantage and that copies bear FOV forces the user to direct his central eye gaze towards the this notice and the full citation on the first page. Copyrights for components small screen of the glass to extract meaningful information. of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to Additionally, focusing the eyes on a display screen at close fo- redistribute to lists, requires prior specific permission and/or a fee. Request cal distances causes visual fatigue [37, 43], which immensely permissions from [email protected]. affects the usability of smartglasses. As the user focuses his IUI ’19, March 17–20, 2019, Marina del Ray, CA, USA central eye gaze on the screen of the smartglass at a close © 2019 Association for Computing Machinery. ACM ISBN 978-1-4503-6272-6/19/03...$15.00 1Google Inc, https://en.wikipedia.org/wiki/Google_Glass https://doi.org/10.1145/3301275.3302263 2MadGaze Group, http://madgaze.com/x5/specs IUI ’19, March 17–20, 2019, Marina del Ray, CA, USA I. Chaturvedi et al. by removing the need to focus on the screen of smartglasses. mid-peripheral This paper contributes to the state-of-the-art by develop- far far peripheral peripheral ing a model that combines two theories: motion detection near-peripheral through peripheral vision [8] and color sensitivity of human 30° 60° 90° eye [26] and demonstrates its application for navigation on paracentral central smartglasses with a head-mounted display. Existing works mainly focus on exploring peripheral vision by changing the hardware of the smartglasses while we propose in this page a pure software solution. Using our model, we develop a high-fidelity peripheral vision-based navigation application for both indoor and outdoor environment scenarios. To the Figure 1: Angular Field of View of the Human Eye best of our knowledge, this paper presents the first use of peripheral vision in a mobile context, using standard smart- glasses in both indoor and outdoor environment without additional hardware. This paper presents the following contributions: AFOV~30° • We present an MPV Model using color and motion to display visual cues in the peripheral vision of the user. • We implement the MPV Model within a navigation application. This application is then compared to a standard navigation application on smartglasses, as Figure 2: Angular Field of View of Google Glass well as the same application on smartphone. As such, we are able to isolate both the impact of peripheral focal point, his multitasking ability is strongly affected. This vision and use of smartglasses. Thanks to our model, temporary shift of focus may have deadly consequences. For users spend on average 50% less time looking at the instance, a user driving a car on the highway at 100km/h screen of the smartglasses. Furthermore, 90% Agree who takes his eyes off the road for one second to look at that the smartphone application was beneficial. a map screen is actually blind for 28 meters. Using mobile • We further discuss two specific cases, namely strabis- devices also limits cognitive ability and restricts peripheral mus and color-blindness, for which our MPV model vision [20]. There have been about 5,984 pedestrian traffic does not apply. Indeed, color-blindness changes the fatalities in 2017. One of the main causes of these accidents color sensitivity of the eye, while strabismus impacts is the extensive use of mobile devices3. the eye mobility. We propose modifications to our Smartglasses with a head-mounted display like Google model to account for these specific cases. Glass or even Microsoft HoloLens partially cover the user’s 4 peripheral vision . The peripheral visual field is an important The rest of this paper is organized as follows: We first part of the human vision and is useful for daily locomotive ac- discuss research studies related to ways of increasing field tivities such as walking, driving, and sports [40]. Visual cues of view, use of peripheral vision in providing notifications to from the periphery can help to detect obstacles, avoid acci- the user, and navigation using smartglasses. In Section 2, we dents and ensure proper foot placement while walking [21]. explain our MPV model and its applications for the mobile In this paper, we present a Mobile Peripheral Vision (MPV) users. In Section 3, we discuss our demo application and the model. Any smartglass with a head-mounted display over- user study built around the application. Finally, we discuss lapping with the peripheral vision can run this model, which the results of the experiments to evaluate the applicability does not require any additional hardware. Our model taps of our model. into the peripheral vision of the user by using the screen of the head-mounted display of the smartglass to present visual cues. The model simplifies multi-tasking for the mobile user Related Work In this section, we present the main related studies. These 3Pedestrian Traffic Fatalities by State, https://www.ghsa.org/resources/ spotlight-pedestrians18 studies spread around three main fields: enhancing the FOV 4Google Glass Blocks Peripheral Vision, https://www.livescience.com/ of smartglasses, displaying information on peripheral vision, 48608-google-glass-blocks-peripheral-vision.html and navigation on smartglasses. Peripheral Vision: A New Killer App for Smart Glasses IUI ’19, March 17–20, 2019, Marina del Ray, CA, USA Enhancing the FOV of smartglasses non-exhaustive list of studies, gaze detection and guiding has Augmenting the field of view has previously been studied by been a very active field to target the user’s attention towards changing the hardware of the smartglasses [7, 28]. Sparse- specific details of a scene and improve global recollection. LightAR increases the field of view of head-mounted displays However, none of these studies exploit peripheral vision to by adding an array of Light Emitting Diodes (LEDs) around send subtle cues to the user without altering his focus on the the central display [47]. Similarly, AmbiGlasses illuminates main task. the periphery of the human visual field by adding 12 LEDs Few studies have explored the possibility of using ani- in the frame of the glasses [31]. Matviienko et al. [22] dis- mations in peripheral vision displays for enhancing visual cuss the possibility of employing ambient light in the car to interest, without distracting the user [30]. The study in [18] keep the focus of the user on the road.