Detecting User Activities using the Accelerometer on Android Smartphones Sauvik Das —Georgia Institute of Technology LaToya Green —University of Houston Beatrice Perez —University of Puerto Rico,Mayaguez Michael Murphy —Franklin W. Olin College of Engineering Supervisor: Dr.Adrian Perrig July 30, 2010 Contents even have the phone learn new activities. Mul- tiple approaches were implemented to attempt to 1 Abstract 1 find the best results. With individual calibra- tion we obtained accuracy of 93%, which could 2 Introduction 1 be improved with future work. 3 Background Information 2 4 Related Work 3 2 Introduction Smartphone security is becoming increas- 5 Methodology 3 ingly important as consumers rely more 5.1 Background . 3 and more on their smartphones to store 5.2 Data Acquisition . 4 personal information. With the devel- 5.3 Signal Processing . 5 opment of smartphone operating systems, 5.3.1 Preprocessing . 5 such as Apple’s iOs, Google’s Android and 5.3.2 Noise Reduction . 5 Blakcberry’s RIM operating systems on the 5.3.3 Linearization . 6 rise, it is imperative that security measures 5.3.4 Smoothing . 6 are in place to protect the privacy of the user. 5.4 Feature Extraction . 7 This is especially true for Google’s Android 5.5 Classification . 8 platform, being that it has an open develop- ment environment. 6 Results 9 What is Android? 7 Conclusions and Future Work 9 The Android platform is an open platform for mobile devices consisting of an operat- 8 References 10 ing system, applications and middleware1. Android gives users the opportunity to build and publish their own applications by 1 Abstract providing an open development environ- ment. Android treats all applications (na- The purpose of this study is to identify whether tive and third-party) as equals2. Therefore, smartphones pose a security threat to the user. having such an open development environ- The accelerometer and other sensors within the ment requires security measures to be taken device can be used without the users consent. in order to protect the integrity of the An- Our intent in this is to show that the accelerom- droid platform and the privacy of its users. eter can be used to obtain sensitive information Android Security and Permissions about the user. Using the magnitude of the ac- The Android Platform takes advantage celerometer data we found that we could iden- of several mechanisms designed to protect tify general activities preformed by the user, and the privacy and security of Android users, 1 as well as the operating system. These for an application to collect user informa- methods include the Android security ar- tion from these sensors without the user’s chitecture, application certificates, and ap- knowledge. plication permissions. The purpose of the Accelerometers Android security architecture is to prevent The accelerometer in Android phones applications from being able to automati- measures the acceleration of the device on cally perform operations that could jeopar- the x (lateral), y (longitudinal), and z (ver- dize the security of other applications, the tical) axes. Accelerometers can be used to operating system or the user. Certificates detect movement and the rate of change of are used to identify the author of a spe- the speed of movement. As stated above, cific application and to prevent users from the use of accelerometers in Android ap- installing fraudulent software on their de- plications does not require the application vices. Android will not install an applica- to have permission to use it. Therefore, it tion that has not been signed with a certifi- is possible for an application to collect a cate. Therefore, the origin of all published user’s accelerometer data without the user’s applications is traceable. knowledge. With accelerometer data and Android security permissions are han- the use of a server to collect the informa- dled by the AndroidManifest.xml file tion, it is a fairly simple task for someone to present within all application files. When gain a user’s personal information, their lo- a user downloads an application onto their cation, or to figure out what a user is doing device, they are automatically notified of or typing. the permissions the application has access to. This informs the user of what type of information an application is able to collect 3 Background Information from the device as well as the hardware the application can use. Accelerometers have been used for a variety The AndroidManifest.xml file takes care of uses throughout the world today, from of both software and hardware permissions. medical to research, from car performance But while Android does require permis- to robotics. However, with the advent of sions for the use of hardware devices such the iPhone and Android, accelerometers are as the camera and vibrator, it does not re- much more commonplace in the world of quire permissions to be set in place for the today. The most commonly used accelerom- use of any available sensors, including the eter within our phones is the LIS311DLH, at accelerometer, orientation, and gyroscope 3x3x1 millimeters, it is a tiny, low power, sensors3. But we have found that these sen- high performance linear accelerometer. It sors, when used alongside other tools such senses the forces of acceleration in the X Y as the internet and GPS, can also pose as se- and Z planes to a precision of six decimal curity threat to the user. And it is possible places. In order to investigate the possible 2 link between cyber security and physical se- but not implemented, such as a path track- curity, we set out to see how these sensors in ing device using only accelerometer data, smart phones, specifically this accelerome- or the use of accelerometer to transmit data ter could be used. In our research we wish other than the data of acceleration. Such to see if this device can turn what seems to phones could then be used as tracking de- be a harmless application on the Android vices for the creators, able to see their user’s Marketplace into a potential spying device, daily lives through accelerometer data. able to detect what actions the user is taking Current location tracking systems offer at the moment. high accuracy using GPS and other such methods; however, those require consent from the user. In this paper we attempt to 4 Related Work show such things as location and activity can be detected and recorded without the Many groups have developed sensor knowledge of the user, using simply the ac- recording and processing devices, such as celerometer within their phone. the eWatch, a wearable sensor that is in- tended for use of monitoring elderly and sick, identify non-responsiveness and keep 5 Methodology taps on their position5. A Japanese com- pany, KDDI, has developed something sim- 5.1 Background ilar to this, a system to keep track of em- ployee’s actions throughout the day, and The data received from the accelerometer send it back to a central mother server. was in the form of a three-valued vector This device can detect such activities such of floating point numbers that represented as walking, climbing stairs, or even clean- the individual accelerations of the smart- ing. However, little thought from the com- phone device in the X, Y, and Z axes sub- pany has gone into the potential security tracted by the gravity vector G. The ac- violations6. Many other universities have celeration values were recorded in meters m explored activity recognition using video per second squared( s2 ). Thus, layed flat instead of accelerometer data7. While this on a level surface, the expected reading of is not exactly related, it also explores the the accelerometer would be approximately idea of using a system we don’t have on our [0,0,-9.81]. The process of converting these minds as being a security threat, in fact, the acceleration vectors into known activities opposite, a security tool - cameras - as some- occured in four broad phases: data acqui- thing to invade privacy. Indeed, some uni- sition,signal processing, feature extraction and versities also attempted activity recognition classification. using cell phones as well8. Other systems During the preprocessing stage, each sam- have been suggested by the world today, pled acceleration vector was combined into 3 a single magnitude; in the noise reduction stage, the received signal was linearized and smoothed to reduce noise; in the feature ex- traction stage, features were extracted for each sample window of a predetermined number of 512 samples; in the classification stage, unknown patterns of data were clas- sified via feature comparison of known pat- terns of data, using the Nearest Neighbor and Naive Bayes classifiers. Two versions of the classification stage were implemented, online one –which was done on the fly as data Figure 1: The data acquisition application running was being received–and one offline–which on the Motorola Droid was done on a server after all relevant data had been collected. Prior to implementation, we had decided . Following reasonable success with these on attempting to classify the following five additions, users would be allowed to enter activities: customized gesture patterns. Phone Detached: The user is not cur- • rently holding the phone Idle: The user is holding the phone in • an idle state 5.2 Data Acquisition Walking: The user is walking • Running: The user is running • The device used was the Motorola Droid. Jumping: The user is jumping To acquire the accelerometer data, we cre- • ated an application for the Droid using the . Having achieved significant accuracy Android SDK. Training data was acquired in classifying the aforementioned activities, by having users perform an instructed task two additional activities were added to the (e.g., walking, running, jumping) while the list: data was either being exported to a server or Descending Stairs: The user is de- written to the Droid’s SD card.