ASYSTEMATIC SURVEY ON ANDROID API USAGE FOR DATA-DRIVEN ANALYTICS WITH SMARTPHONES APREPRINT Hansoo Lee Joonyoung Park Uichin Lee∗ School of Computing Graduate School of Knowledge Service Engineering School of Computing KAIST KAIST KAIST Daejeon, Republic of Korea Daejeon, Republic of Korea Daejeon, Republic of Korea [email protected] [email protected] [email protected] April 26, 2021 ABSTRACT Recently, there has been an increase in industrial and academic research on data-driven analytics with smartphones based on the collection of app usage patterns and surrounding context data. The Android mobile operating system utilizes Usage Statistics API (US API) and Accessibility Service API (AS API) as representative APIs to passively collect app usage data. These APIs are used for various research purposes as they can collect app usage patterns (e.g., app status, usage time, app name, user interaction state, and smartphone use state) and fine-grained data (e.g., user interface elements & hierarchy and user interaction type & target & time) of each application. In addition, other sensing APIs help to collect the user’s surroundings context (location, network, ambient environment) and device state data, along with AS/US API. In this review, we provide insights on the types of mobile usage and sensor data that can be collected for each research purpose by considering Android built-in APIs and sensors (AS/US API, and other sensing APIs). Moreover, we classify the research purposes of the surveyed papers into four categories and 17 sub-categories, and create a hierarchical structure for data classification, comprising three layers. We present the important trends in the usage of Android’s built-in APIs and sensors, including AS/US API, the types of data collected using the presented APIs, and discuss the utilization of mobile usage and sensor data in future research. Keywords Smartphone sensing, User interaction data, data analysis, android accessibility, Android API, data collection, sensor, mobile device, smartphone, data driven analytics 1 Introduction arXiv:2104.11271v1 [cs.HC] 22 Apr 2021 Currently, many people regard smartphones as an extension of their bodies. According to research of smartphone usage statistics, 79% of adults carry their mobile phone for 22 hours a day [1], and smartphone users spend time with their devices for an average of 3 hours and 15 minutes a day [2]. During smartphone usage, fine-grained contextual information (e.g., ambient environment, physical activity, app usage pattern) can be collected from built-in sensors and application programming interfaces (API). Thus, smartphones provide new opportunities to collect and analyze a user’s everyday lifelog data (i.e., smartphone/mobile usage and sensor data) for enabling digital health services and user experience research. For example, mobile usage and sensor data serves as a valuable resource for the diagnosis and prognosis of mental and physical health [3, 4]. As the number of smartphone users worldwide in 2021 is expected to increase to more than 3.8 billion people [5], it has become increasingly important to explore novel opportunities for data-driven analytics using mobile usage and sensor data. This review focuses on the Android mobile operating system (OS) that dominates 85% of the global smartphone market [6]. Android OS allows developers to customize these built-in APIs to build applications for smartphone data collection. ∗The corresponding author A PREPRINT -APRIL 26, 2021 For this reason, Android has been actively used in the research community. As shown in Figure 1, Android APIs1 can identify a user’s life patterns by collecting various mobile usage and sensor data through human and physical related context aware. In addition, users touch their phone an average of 2,617 times a day [7]. Thus, we can collect app usage pattern data (e.g., application status/name/time) on frequent digital interactions between users and smartphones, including fine-grained interaction details (e.g., user interaction type and time, user interface (UI) hierarchy and elements, notification) for each application. In addition, we can collect the user’s external contextual data, such as physical activities and environmental states. Among Android’s built-in APIs, researchers commonly use the Accessibility Service API (AS API) which is also called Accessibility API. The AS API includes the classes such as AccessibilityService2, AccessibilityEvent3, Accessibili- tyRecord4, AccessibilityManager5 [8]. The AS API is operated to generate various AccessibilityEvent classes (e.g., transition type, view type, exploration type, and notification type) for app usage, by recognizing the state change of a smartphone which is the event based system [9]. AccessibilityEvent can be used to collect fine-grained datasets for app usage pattern data as depicted in Figure 1. Another way to collect app usage pattern data is using the Android Usage Statistics APIs (US APIs) such as UsageS- tatManager6, UsageStats7, UsageEvents8, UsageEvents.Event9. US API can be used to collect the app and device usage history and statistics as shown in Figure 1 [10]. US API can access device usage history and statistics to obtain the currently running app information in third-party applications (as of Android API level 21 in 2014). The US API provides functions to query app usage logs by specifying time units of day, month, and year, thereby retrieving the results specified by the interval which is a more granular and explicit query system [11]. Since 2014, the US API has been mainly used to collect app usage pattern data in many studies (e.g., [12, 13, 14, 15]). However, the AS API is still exclusively used to track the user interface (UI) status information (e.g., UI changed status, interaction type, UI properties & hierarchy, and notification) in the present studies (e.g., [16, 17, 18, 19]). Along with the AS/US APIs, other Android built-in sensors and APIs are also used to collect mobile usage and sensor data for the research. These data can be collected using Android built-in hardware (HW) sensors (e.g., accelerometer, gyro, GPS, Wi-Fi, Bluetooth, proximity, light, compass, pressure) and APIs (e.g., Sensor APIs11, Wi-Fi APIs12, Google Location Services APIs13, Bluetooth APIs14). In other words, we can use these Android built-in sensors and APIs to track a user’s physical activity, physiological state, and surrounding context information (e.g., location, ambient environment) for the research as shown in Figure 1. Furthermore, Android system APIs (e.g., BroadcastReceiver15, BatteryManager16) can be used to collect device status information (e.g., battery information, screen status, device information setting, ringer mode) as depicted in Figure 1. Leveraging data-driven analytics with smartphone data is widely performed in various research fields (e.g., sensors, human-computer interaction, ubiquitous computing, and mobile computing). These include: • Understanding a user’s smartphone usage pattern [17, 20, 21, 22, 18] • Identifying a user’s notification checking factors, how the user is affected by the notification, and message management and scheduling [23, 24, 16, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36]. • Understanding a user’s psychological & health states and personal traits in digital phenotype and digital health research fields [37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48]. • Validating permission, improve authentication & authorization, and improve protection from malware in privacy & security research fields [49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60]. 1https://developer.android.com/reference 2https://developer.android.com/reference/android/accessibilityservice/AccessibilityService 3https://developer.android.com/reference/android/view/accessibility/AccessibilityEvent 4https://developer.android.com/reference/android/view/accessibility/AccessibilityRecord 5https://developer.android.com/reference/android/view/accessibility/AccessibilityManager 6https://developer.android.com/reference/android/app/usage/UsageStatsManager 7https://developer.android.com/reference/android/app/usage/UsageStats 8https://developer.android.com/reference/android/app/usage/UsageEvents 9https://developer.android.com/reference/android/app/usage/UsageEvents.Event 11https://developer.android.com/reference/android/hardware/package-summary 12https://developer.android.com/reference/android/net/wifi/package-summary 13https://developers.google.com/android/reference/com/google/android/gms/location/package-summary 14https://developer.android.com/reference/android/bluetooth/package-summary 15https://developer.android.com/reference/android/content/BroadcastReceiver 16https://developer.android.com/reference/android/os/BatteryManager 2 A PREPRINT -APRIL 26, 2021 Figure 1: Mobile usage and sensor data collection through Android APIs • Improving user’s interaction, accessibility, and optimization [61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72]. • Testing and programming support fields, such as graphical user interface (GUI) automated testing, crowdsourced system testing, programming support for developer, and app performance measurement [73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86]. • Development of frameworks and platforms to collect smartphone data [87, 88]. Therefore, it is important to understand the nature of the collected smartphone data and its research purpose. Existing studies have reviewed mobile usage and sensor data collection, but they have mainly focused on one specific purpose or type of data related to their