Automated Concolic Testing of Smartphone Apps Saswat Anand Mayur Naik Hongseok Yang Georgia Tech Georgia Tech University of Oxford [email protected] [email protected] [email protected] Mary Jean Harrold Georgia Tech [email protected] ABSTRACT tap on the device’s touch screen, a key press on the device’s We present an algorithm and a system for generating in- keyboard, and an 0,0 message are all instances of events. put events to exercise smartphone apps. Our approach is This paper presents an algorithm and a system for gener- based on concolic testing and generates sequences of events ating input events to exercise apps. pps can have inputs automatically and systematically. It alleviates the path- besides events! such as files on disk and secure web content. explosion problem by checking a condition on program exe- Our work is orthogonal and complementary to approaches cutions that identifies subsumption between different event that provide such inputs. sequences. We also describe our implementation of the ap- Apps are instances of a class of programs we call event- proach for Android, the most popular smartphone app plat- driven programs' programs embodying computation that form! and the results of an evaluation that demonstrates its is architected to react to a possibly unbounded sequence effectiveness on five ndroid apps. of events. 7vent-driven programs are ubiquitous and, be- sides apps! include stream-processing programs, web servers! GUIs, and embedded systems. 8ormally! we address the fol- Categories and Subject Descriptors lowing problem in the setting of event-driven programs in D.2.$ %Software Engineering&' (esting and Debugging) general, and apps in particular. Symbolic execution, Testing tools Branch-coverage Problem: *iven a constant bound k 5! e9ciently compute a set of event sequences that Keywords execute≥ each branch of an event-driven program that can *+I testing, testing event-driven programs, Android be executed by some event sequence of length up tok. The above problem poses two separate challenges: /52 how 1. INTRODUCTION to generate single events and /#2 how to extend them to Mobile devices with advanced computing ability and con- sequences of events. We next look at each of these in turn. nectivity! such as smartphones and tablets! are becoming Generating Single Events. 7xisting approaches for increasingly prevalent. t the same time there has been a generating all events of a particular kind use either capture- surge in the development and adoption of specialized pro- replay techniques to automatically infer a model of the app’s grams, called apps! that run on such devices. Apps per- GUI %##! :5& or model-based techniques that require users to vade virtually all activities ranging from leisurely to mission- provide the model %:;! :6]. (hese approaches have limita- critical. (hus, there is a growing need for software-quality tions. =apture-replay approaches are tailored to a particular tools in all stages of an app’s life-cycle, including develop- platform’s event-dispatching mechanism but many apps use ment! testing, auditing, and deployment. a combination of the platform’s logic and their own custom Apps have many features that make static analysis chal- logic for event dispatching. 8or example, where the plat- lenging: a vast software development kit /0"12! asynchrony! form sees a single physical widget, an app might interpret inter-process communication, databases! and graphical user events dispatched to different logical parts of that widget interfaces (GUIs2. (hus! many approaches for analyzing differently. In contrast! model-based approaches are general- apps are based on dynamic analysis (e.g.! %3! 4! 10]). purpose, but they can require considerable manual effort. question central to the effectiveness of any dynamic In this paper, we present a new approach, which is both analysis is how to obtain relevant program inputs. (he most general and automatic, to address this problem. The ap- indivisible and routine kind of inputs to an app are events. proach builds on a systematic test-input generation tech- nique called concolic testing %<! 5$! :6& /also known as dy- namic symbolic execution) which has made significant strides in recent years. Our approach symbolically tracks events Permission to make digital or hard copies of all or part of this work for from the point where they originate to the point where they personal or classroom use is granted without fee provided that copies are are ultimately handled. (he approach is thus oblivious to not made or distributed for profit or commercial advantage and that copies where and how events are dispatched. bear this notice and the full citation on the first page. To copy otherwise, to Generating Event Sequences. Our concolic-testing republish, to post on servers or to redistribute to lists, requires prior specific approach for generating single events can be extended nat- permission and/or a fee. SIGSOFT’12/FSE-20, November 11–16, "#12, Cary, North Carolina, USA. urally to iteratively compute sets of increasingly longer se- Copyright 2012 '$( 978-1-4503-1614-9/12/11 ...$15.00. quences of events. n algorithm, hereafter called AllSeqs! can generate all event sequences of length up tok such that tion %5#&! generational search %5<&! and Satisfiability Modulo each event sequence executes a unique program path. How- Theories /0,(2 solving %?&. We show the effectiveness of ever, AllSeqs does not scale as the value ofk is increased our system on five Android apps. because typical programs have a large number of program The primary contributions of this work are as follows. paths. 8or instance, for a simple music player app writ- ten for the Android mobile app platform, AllSeqs gener- 1. novel approach to systematically generate events to ates 55 one-event sequences, 5#? two-event sequences, 5!$46 exercise apps. Our approach, based on concolic test- three-event sequences! and #51 four-event sequences. This ing, is fully automatic and general! in contrast to exist- problem is known as the path-explosion problem %5#&! and, ing capture-replay-based or model-based approaches. in practice! AllSeqs may fail to solve the aforementioned 2. An efficient algorithm! ACTEve! to solve the branch- branch-coverage problem within a time budget for a high coverage problem. Our key insight is a subsumption value ofk. condition between event sequences. Checking the con- Although several techniques have been developed in recent dition enables ACTEve to prune redundant event se- years to address the path-explosion problem /e.g.! %5! ;! 5#@ quences, and thereby alleviate path explosion, while 5;! 53! 54! #6! #4&2! the problem is still intractable for real- being complete with respect to AllSeqs. world software. In this paper, we present a new technique! 3. An implementation and evaluation of ACTEve in a ACTEve /stands for utomated =oncolic (esting of Event- system for ndroid apps. Our system is portable! ex- driven programs2! to alleviate the path-explosion problem. ploits available parallelism, and leverages recent ad- ACTEve is tailored to event-driven programs such as smart- vances in concolic testing. phone apps. (he key insight underlying ACTEve is a no- tion of subsumption between two event sequences. If an event sequenceπ is subsumed by another event sequence 2. OVERVIEW O$ OUR APPROACH π�! then ACTEve avoids generating sequences that are ex- In this section! we illustrate our approach using an ex- tensions ofπ! and instead generates extensions ofπ � only. ample music player app from the Android distribution. We Such pruning ofπ results in compounded savings as longer first describe the app by discussing its source code shown in event sequences are generated. We show that ACTEve is Figure 5 (Section 2.5). We then describe how we generate relatively complete with respect to AllSeqs. Specifically! events to exercise this app (Section 2.#2 and how we extend ACTEve covers the same set of branches as AllSeqs for a them to sequences of events /Section 2.:2. given upper bound on the length of event sequences.1 ACTEve computes subsumption between event sequences 2.1 The Music Player App by checking for simple data- and control-flow facts of each Android apps are incomplete programs: they implement program execution in isolation. Compared to existing tech- parts of the Android 0"K’s API and lack a main method. niques for the path-explosion problem! ACTEve has at least When the music player app is started, the 0"1 creates an in- four distinctive characteristics that greatly improves its ap- stance of the app’s main activity MainActivity! and calls its plicability to real-world software such as Android apps. First, onCreate() method. This method displays the main screen, it does not impose any unrealistic assumption (e.g.! meth- depicted in Figure #/a), which contains six buttons: rewind! ods are free of side effects) about the program. Second, it play! pause! skip! stop! and eject. (he method also sets the does not store and match program traces or program states, main activity as the handler of clicks to each of these but- which can dramatically increase memory overhead. Third, tons. The app waits for events once onCreate() finishes. it does not require powerful /e.g., support for quantifiers) When any of the six buttons is clicked, the 0"1 calls or specialized constraint solvers to reason about path con- the main activity’s onClick() method, because the main straints of multiple program paths simultaneously. 8ourth, activity was set to handle these clicks. If the eject button it does not require static analysis of real-world code that is clicked, this method displays a dialog, depicted in Fig- often contain parts that are beyond the scope of static anal- ure 2(b)! that prompts for a music file URL. If any of the ysis.