STUDY OF THE USE OF WEARABLE DEVICES FOR PEOPLE WITH DIFFERENT TYPES OF CAPABILITY Alejandro Rigueira Cabrera Master’s Thesis presented to the Telecommunicatons Engineering School Master’s Degree in Telecommunications Engineering Supervisors Prof. Enrique Costa Montenegro Prof. Milagros Fernández Gavilanes 2017 What we think, we become. — Buddha To my family and friends. Abstract Nowadays, in some territories, a large number of people possess a smartphone, and even in some cases they have more than one. Not only do they have some but they use them several times a day for different activities and applications. The smartphone penetration rate is really high, but other devices cataloged as intelligent have also appeared as in the case of the smartwatches. The goal is to use one of these wearable devices, which can be purchased from any regular electronic devices distributor on the market, as input element that allow interaction with the smartphone as a joystick or other external appliance. Our target audience is mainly those with possible conditions in hands and fingers that imply lack of mobility or limited one in many of them. Facilitate access to anyone, anywhere, anytime. To bring this about, it will be developed an Android app as proposed solution which be able to detect arm movements using a smartwatch which has the necessary sensors to it. Although not all apps and aspects on the smartphone are reachable due to certain limitations, the proposed solution fulfills to a large degree the proposed objective, including easy access to a custom keyboard. Key words: Accessibility service, Android, Android wear, Gesture sensing, Input device, Wearable computer. i Contents Abstract i List of figuresv List of tables vii 1 Introduction1 1.1 Introduction...................................1 1.2 Objective....................................2 1.3 Target audiences................................3 1.4 State of the art.................................3 1.5 Universal Design................................6 1.5.1 What is it?...............................6 1.5.2 Principles and considerations.....................7 1.5.3 User interface, usability........................8 1.6 Proposal.....................................8 2 Technologies 11 2.1 Android..................................... 11 2.1.1 Why Android.............................. 11 2.1.2 Android fragmentation and devices.................. 12 2.1.3 Accessibility on Android........................ 14 2.1.4 Android limitations........................... 15 2.2 Wearable device................................. 17 2.3 Android wear.................................. 18 2.4 Development tools............................... 19 3 Development 21 3.1 Mobile...................................... 21 3.1.1 Keyboard................................ 21 3.1.2 Mouse cursor.............................. 25 3.1.3 Communication service......................... 25 3.1.4 Preferences............................... 26 3.2 Wear....................................... 26 iii Contents 3.2.1 Movements detection.......................... 26 3.2.2 Movements assignment......................... 28 3.2.3 User interface.............................. 30 3.3 Tests and debugging.............................. 30 4 Use manual 33 4.1 Mobile...................................... 33 4.1.1 Keyboard................................ 36 4.1.2 Mouse cursor.............................. 38 4.1.3 Settings preferences.......................... 39 4.2 SmartWatch................................... 43 5 Conclusions 47 5.1 Possible improvements and future study lines................ 47 5.2 Conclusions................................... 48 Bibliography 53 iv List of Figures 1.1 Word-wide smartphone sales. Units sold, in thousands...........1 1.2 Smartwatch sales forecast according Gartner. Millions of unit sales....2 1.3 Wristband-type input device..........................5 1.4 Sensing arm-shape change based on capacitive sensing............6 2.1 Android fragmentation............................. 12 2.2 Screen sizes and densities........................... 14 2.3 Asus ZenWatch 2................................ 17 2.4 Smartwatches using Android Wear...................... 19 3.1 Application keyboard.............................. 22 3.2 Activity lifecycle................................ 23 3.3 IME lifecycle.................................. 24 3.4 Mouse pointer.................................. 25 3.5 Reference smartwatch............................. 29 3.6 Smartwatch on a left hand (left figure) and on a right hand (right figure). 29 3.7 Interaction modes on smartwatch....................... 30 4.1 App icon..................................... 33 4.2 Application main screen............................ 33 4.3 About application message........................... 34 4.4 Enabling keyboard............................... 34 4.5 Selecting keyboard............................... 35 4.6 Notification application running........................ 35 4.7 Usage test screen................................ 36 4.8 Application keyboard.............................. 36 4.9 Keyboard rightward movement selection................... 37 4.10 Keyboard leftward movement selection.................... 37 4.11 Keyboard upward movement selection.................... 37 4.12 Keyboard downward movement selection................... 37 4.13 Enabling mouse cursor............................. 38 4.14 Mouse cursor.................................. 39 4.15 Application settings preferences........................ 39 4.16 Application hand settings........................... 40 v List of Figures 4.17 Application leftward sensors sensibility preference.............. 41 4.18 Application back button preference...................... 41 4.19 Select directions on smartwatch........................ 42 4.20 Show directions assigned on smartwatch................... 42 4.21 Application keyboard scanning time settings................. 43 4.22 Main, buttons, swipe and sensors screens on smartwatch app....... 44 4.23 Get out of swipping screen on smartwatch app............... 44 4.24 Select and back buttons activation using sensors............... 45 4.25 Sensor activation screen............................ 45 vi List of Tables 2.1 Worldwide smartphone sales to end users by operating system in 4Q16 (thousands of units).............................. 11 2.2 Platform versions................................ 12 2.3 Screen sizes and densities........................... 14 2.4 Android wear versions............................. 18 vii 1 Introduction 1.1 Introduction Nowadays, in some territories, a large number of people possess a smartphone, and even in some cases they have more than one. Not only do they have some but they use them several times a day for different activities and applications. According to the Deloitte Mobile Consumer Report 2015 [1], 88% of Spaniards have a smartphone. Only Singapore has a higher penetration rate of smartphone devices, 92%. The average penetration rate in the EU countries participating in the study is 78%. The report shows that more than 50% of respondents claimed to check their mobile in the first 15 minutes of the day. They also claim to do a daily average of 41 times. This, according to Deloitte, gives "a very clear idea of the high degree of digitalization of Spanish society at present". Figure 1.1: Word-wide smartphone sales. Units sold, in thousands 1 Chapter 1. Introduction It is clear the great penetration in the society and the great deployment of smartphones. Increasingly they used more and more as it can be seen in Figure 1.1. There are more than 700,000 apps on Google Play [2], Google’s official Android app store. Applica- tions of messaging, management, buying and selling, education, games, photography, information. There are also movies, music, books, magazines and more. In addition to the well-known smartphones, other devices have appeared also cataloged as intelligent devices. Some of these, in turn, fall within the category of wearable devices. That is, they can be worn on the body as implant or accessories. Among these devices are those known as smartwatch, which have grown in popularity and sales. In Figure 1.2 you can see the sales forecast for smartwatches according to Gartner [3, 4] (consulting and research company of information technologies). Figure 1.2: Smartwatch sales forecast according Gartner. Millions of unit sales 1.2 Objective Being able to access all these services and products can be very useful for any user. However not everyone can achieve it for several reasons. We will talk later about people who have difficulties making use of some of these devices. We will also talk about a design model that aims to get the environments, products, services and systems to be employed by as many people as possible. Currently, the typical application of a smartwatch is as a notifying device, although there are many applications that are usable directly on it doing more than just notify. Example of this are simple actions that interact with the mobile device or the acquisition of data such as heart rate. The goal is to use these wearable devices, such as smartwatches, as input element that allow interaction with the smartphone as a joystick or other external appliance. Although theire use are possible for a large number of people, and so it is intended, the target audience, for which the study is mainly focused, are people with motor conditions that are difficult to a normal use of a smartphone or tablet. 2 1.3. Target audiences 1.3 Target audiences Alterations in