Smart Glasses: Interaction, Privacy and Social Implications
Total Page:16
File Type:pdf, Size:1020Kb
Smart glasses: Interaction, privacy and social implications Ubiquitous Computing Seminar FS2014 Student report Marica Bertarini ETH Zurich Department of Computer Science Zurich, Switzerland [email protected] ABSTRACT INTERACTION Smart glasses are wearable devices that display real-time The main purpose of smart glasses is to provide users with information directly in front of users’ field of vision by us- information and services relevant for their contexts and ing Augmented Reality (AR) techniques. Generally, they useful for the users to perform their tasks; in other words, can also perform more complex tasks, run some applica- such devices augment users’ senses. In addition, they allow tions, and support Internet connectivity. This paper pro- users to do basic operations available on today common vides an overview of some methods that can be adopted to mobile devices such as reading, writing e-mails, writing allow gesture-based interaction with smart glasses, as well text messages, making notes, and answering calls. as of some interaction design considerations. Additionally, Therefore, although most of the usage of smart glasses is it discusses some social effects induced by a wide-spread passive for the users, i.e. reading content on the little screen deployment of smart glasses as well as possible privacy of the device, active interaction with such devices is fun- concerns. damental to control them and supply inputs. In fact, users Keywords: Smart glasses, Head-worn displays, input tech- need ways to ask smart glasses for instance to open a par- niques, interaction, body interaction, in-air interaction, pri- ticular application, answer something they need to know, vacy, social implications insert content for emails, messages or input fields, or to control games. INTRODUCTION Head-worn displays (HWD) have recently gained signifi- Designing interaction techniques cant attention, in particular thanks to the release of a tem- Before presenting how users can interact with smart glass- porary version of Google Glass. Moreover, the anticipation es, it is worth mentioning the main aspects that have to be of the commercial launch of Google Glass1 in the upcom- taken into account during the design process of such tech- ing months and the fresh news that Facebook, Inc. acquired niques, which is summarized in [2]. As for technical char- Oculus Rift2 increased the popularity of such devices even acteristics, a gesture recognition system for HWD should further. ideally be very accurate, i.e. able to distinguish fine shape- based gestures, insensitive to daylight, as small as possible, The trend of wearable device purchases is importantly consume low power, and be robust in noisy and cluttered growing and some business analysts [1] forecast more than environment that are typical conditions in everyday life 20 million annual sales of Google Glass in 2018. Further- scenarios. As far as user experience is concerned, the phys- more, researchers have been already studying and investi- ical effort required to users to interact with the devices is gating HWD for several years. As a consequence, it is im- relevant, as well as easiness of use and encumbrance of the portant to give an overview of different methods that could device. be used to interact with smart glasses and, above all, ana- lysing privacy concerns and identifying the current and po- Interaction approaches tential social implications related to these devices has a Two different categories of interaction methods can be dis- great significance at this point. tinguished for smart glasses: free form and others. The former includes for example eye tracking, wink detection, voice commands, and gestures performed with fingers or 1 http://www.google.com/glass/start/ hands. On the other hand, the latter comprises for instance 2 http://www.oculusvr.com/ the use of hand-held devices, e.g. point-and-click control- lers, joysticks, one-hand keyboards and smartphones, or smart watches to control the HWD. The aforementioned examples should help to understand long pressing on the surface can be used to invoke new ges- the difference between the two kinds of interaction. How- tures. ever, it can be expressed by saying that the free form does The need to find an alternative to performing gestures di- not require any extra device other than the smart glass to be rectly on the smart glass itself arises from the fact that the performed and detected; on the contrary, it is obvious how touchable area available for interaction on such devices is the control of the smart glass that happens via smartphones, usually only a narrow strip in correspondence to the user’s pointer, etc. cannot satisfy the free form criterion. temple; in fact, this means that many gestures are needed to GESTURE-BASED INTERACTION browse some pages or applications that require a lot of Thereinafter, we will only focus on gesture-basted interac- panning and zooming due to the very small size of the in- tion, because it is preferable to assure a great user experi- ence. Gesture-based interaction has been researched more than eye tracking and wink detection so far, and voice recognition has already reached a huge diffusion on today mobile devices. It is relevant to note that several different techniques to de- tect gestures exist; analysing them in detail would lead us to stray from the purpose of this paper. Some of them make use of devices and sensors that have to be tied to users’ body, e.g. to wrists, hands or fingers, whereas others ex- Figure 1: Main areas identified by participants as ploit cameras that are external to the smart glass itself and suitable for input on the face (left) and on the HWD are located around the user. In addition, some of these (right). The size of the circles is proportional to the methods are used together with markers, e.g. reflective, in- percentage of gestures. Source: [3] frared, coloured, in order to identify the position of users’ hands. teraction surface. As a result, the time taken to reach a tar- Alternatively, gestures can be recognized by using cameras get in a page is negatively affected as well as the arm- or sensors, such as RGB or 3D cameras and depth sensors, shoulder fatigue is significant because of the prolonged lift- that can be embedded into smart glasses. It is essential to ing. stress that real free forms of interaction are realized only by The first relevant result of this study is that most of the par- using cameras or sensors embedded into the HWD as they ticipants preferred the temple to all the other parts of the nullify the need of external components and reduce the en- smart glass and voted the cheek as the best part of the face cumbrance. As a consequence, these approaches are ideal for interaction, as it can be observed in figure 1. Moreover, for the commercial version of the smart glasses that will be some participants found interacting on the cheek more suit- launched on the market. Differently, the other recognition able than performing gestures on the device thanks to its techniques that have been classified as non-free forms are usually used in research for several studies in this context. Gestures on body/smart glass Another significant distinction applicable to gestures is based on where the gesture is performed. A common solu- tion is doing gestures very close to or directly on some sur- faces such as some parts of the smart glass itself or the us- er’s body. The following paragraphs present two different studies that have investigated this approach. Hand-to-face input. In [3], Serrano et al. describe their study aimed at identifying which gestures and surfaces al- low fast interaction with as little effort as possible. As re- gards the surface, they considered and compared the results about some parts of both the smart glass and the user’s face. In particular, the face has been chosen since it is a part of the body we interact with very often, i.e. 15.7 times/hour in a working set; consequently, this means that gestures performed on some parts of the face could be less intrusive than others and may reach a good level of acceptance. On the other hand, the high frequency of hand-face contacts leads to the need of gesture delimiters used by the users to inform the system that a new gesture is starting and avoid Figure 2: Mean time in seconds (left) and mean unintentional triggers; for instance, voice commands or Borg value (right) for technique and interaction area. Source: [3] modified bigger size; also, they stated that this part of the face can be Figure 4 shows the four study conditions investigated in considered somehow similar to a touchpad. this study. The study shows that when not blindfolded As far as the performances are concerned, figure 2 illus- (Figure 4a), the grid drawn on the fake phone (Figure 4c) trates that cheek obtained better results in terms of both orients users on the screen and helps them to find the tar- time taken and exertion required to perform the interaction. gets that were reached faster than on the palm (Figure 4d). In addition, using the cheek took the participants roughly Additionally, this experiment proved that, in contrast, 20 seconds to perform three different zooming and they touching the palm was faster than touching the fake phone considered it the easiest method compared to the over-sized when blindfolded; this has a great importance because it temple and regular temple. The mean Borg referenced in demonstrates that the tactile cues received by the users the plot is a scale for ratings of perceived exertion; in par- ticular, it allows to estimated fatigue and breathlessness during physical work [5].