Dynamic Analyses for Privacy and Performance in Mobile Applications Mingyuan Xia Doctor of Philosophy School of Computer Science McGill University Montreal, Quebec 2016-08-14 A Thesis Submitted to the Faculty of Graduate Studies and Research in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy Copyright c 2016 Mingyuan Xia DEDICATION To my beloved family ii ACKNOWLEDGMENTS First and foremost, I deeply appreciate my supervisor Dr. Xue Liu for his patience and advice during my graduate study. I am also very fortunate to have Dr. Laurie Hendren and Dr. David Lie provide their invaluable feedbacks to improve my thesis work. I want to thank Dr. Zhengwei Qi and Dr. Yi Gao for collaboration in various research projects. At McGill, I would like to thank all members of CPSLab, staff of School of Computer Science, and Ron Simpson. And I enjoy the fun days with friends from MTSA and SJTU alumnus. I appreciate Howard Wang's great French skills and Nos Th´esfor brewing the best milk tea. At IBM Almaden, Dr. Pin Zhou and Dr. Mohit Saxena have provided the greatest mentorship during my internship. Finally I want to acknowledge the IBM Ph.D. fellowship, McGill Lorne Trottier Fellowship and NSERC for financially supporting my graduate career. iii ABSTRACT Mobile applications (also called apps) have greatly extended and inno- vated users' daily tasks. The mobile programming model features event-driven execution, rapid changing APIs (about three generations per year) and ubiqui- tous accesses to user's personal data. These features enrich app functionalities but also give rise to many new software problems that impact performance or damage user privacy, many of which are not occasional programming mistakes. In this thesis, we systematically study these problems and develop dynamic program analyses to effectively detect, diagnose and fix these new problems. We start by researching the sensitive data leakage problem in apps. Since mobile apps can access various sensitive user data stored on the device, data leaks become a great concern for both end users and app market operators. Ex- isting leak detecting approaches rely on static analysis that does not perform well on real-world apps with growing complexity, further limiting their adop- tion for real usage. We propose AppAudit, which embodies a novel dynamic analysis that can execute part of the app code while tracking the dissemination of sensitive data. AppAudit also has a static analysis to shrink analysis scope and boost analysis performance. The synergy of two analyses achieves higher detection accuracy, runs 8.3× faster and uses 90% less memory on real-world Android apps as compared to previous approaches. Based on the analysis building blocks from AppAudit, we further develop binary instrumentation to profile and improve app performance. We study 115 thousand apps and common performance anti-patterns from existing lit- erature. Based on these understandings, we propose AppInspector, which instruments apps to profile a small set of methods while collecting various app runtime diagnostic data. These profiling data is transformed into a graph iv structure, where AppInspector programmatically diagnoses three common per- formance anti-patterns from this graph. We also develop AppSwift based on AppInspector, which transforms app code to automatically fix some perfor- mance anti-patterns and improve app performance. Both tools instrument app code automatically. Instrumented apps can run on unmodified Android OSes and thus being readily deployable to existing test environments. With extensive tests on real-world apps, AppInspector uncovers 22 performance is- sues per app, with detailed analysis results to guide developers to fix them; AppSwift automatically eliminates about 5 of such issues without any code modification from the app developer. We believe that the analysis method- ologies, frameworks and tools developed in this thesis can assist developers in debugging various performance problems and better protecting user privacy. v ABREG´ E´ Les applications mobiles (´egalement appel´esapps) ont consid´erablement ´etenduet innov´eeles t^aches quotidiennes des utilisateurs. Le mod`elede pro- grammation mobile dispose d'ex´ecution´ev´enementielle, API ´evolution rapide (environ trois g´en´erationspar an ) et omnipr´esente des acc`esaux donn´eesper- sonnelles de l'utilisateur. Ces fonctionnalit´esenrichissent app fonctionnalit´es, mais aussi donner lieu `ade nombreux probl`emesnouveaux logiciels que la per- formance de l'impact ou de dommages utilisateur vie priv´ee,dont beaucoup ne sont pas des erreurs de programmation occasionnelles. Dans cette th`ese,nous ´etudions syst´ematiquement ces probl`emeset d´eveloppons le programme dy- namique des analyses pour d´etecterefficacement, diagnostiquer et r´esoudreces nouveaux probl`emes. Nous commen¸conspar rechercher le probl`emede fuite de donn´eessensibles dans des apps. Comme les applications mobiles peuvent acc´eder`adiverses donn´eessensibles de l'utilisateur stock´essur l'appareil, les fuites de donn´eesdevient une grande pr´eoccupation pour les utilisateurs fin- aux et les op´erateursdu march´ede l'app. Les m´ethodes de d´etectionde fuites existantes s'appuient sur l'analyse statique qui ne fonctionne pas bien sur les applications dans le monde rel avec une complexit´ecroissante. Nous proposons AppAudit, qui incarne une nouvelle analyse dynamique qui peut ex´ecuterla partie de l'app code tout en effectuant le suivi de la diffusion des donn´eessensi- bles. AppAuditposs`ede´egalement une analyse statique pour r´etr´ecirl'analyse des performances de l'analyse et boost scopie. La synergie des deux analyses permet d'obtenir une plus grande pr´ecisionde d´etection,8.3x plus rapide et utilise ex´ecute90% moins de m´emoiresur les applications Android dans le monde r´eelpar rapport aux approches pr´ec´edentes. Sur la base des blocs de vi construction de l'analyse d'AppAudit, nous d´eveloppons l'instrumentation bi- naire au profil et am´elioronsles performances des applications. Nous ´etudions 115 mille applications et performance communs anti-mod`eles `apartir de la litt´eratureexistante. Sur la base de ces accords, nous proposons AppInspector, qui instrumente applications au profil d'un petit ensemble de m´ethodes tout en recueillant des donn´eesde diagnostic diff´erentes application d'ex´ecution. Ces donn´eesde profilage se transforme en une structure de graphe, o`uAppIn- spector diagnostique trois performances commune anti-mod`eles`apartir de ce graphique. Nous d´eveloppons ´egalement AppSwift bas´esur AppInspector, qui transforme le code de l'application pour corriger automatiquement certaines performances anti-mod`eleset d'am´eliorer les performances des applications. Les deux outils instrument code de l'application automatiquement. Les appli- cations instrument´eespeuvent fonctionner sur les syst`emesd'exploitation An- droid non modifi´eset donc ^etrefacilement d´eployable `ades environnements de test existants. Avec des tests approfondis sur les applications du monde r´eel, AppInspector d´ecouvre22 probl`emesde performance par application, avec des r´esultatsd'analyse d´etaill´espour guider les d´eveloppeurs de les corriger; AppSwift ´elimine automatiquement environ 5 de ces questions sans aucune modification de code `apartir du d´eveloppeur de l'application. Nous croyons que les m´ethodes d'analyse, les cadres et les outils d´evelopp´esdans cette th`ese peuvent aider les d´eveloppeurs `ad´eboguer divers probl`emesde performance et une meilleure protection de la vie priv´eedes utilisateurs. vii TABLE OF CONTENTS DEDICATION............................... ii ACKNOWLEDGMENTS......................... iii ABSTRACT................................ iv ABREG´ E´.................................. vi LIST OF TABLES............................. xi LIST OF FIGURES............................ xiii 1 Introduction..............................1 1.1 Contributions.........................3 1.2 Thesis Organization......................5 2 Background..............................6 2.1 Android System Hierarchy..................6 2.2 Android Applications.....................7 2.2.1 Code, Manifest, and Resources............8 2.2.2 Execution Model and Performance.......... 10 2.2.3 Permission and Privacy................ 11 3 AppAudit: Analyzing and Detecting Data Leaks.......... 14 3.1 The Information Flow Problem Revisited.......... 14 3.2 Related Work......................... 17 3.2.1 Static Analysis..................... 17 3.2.2 Dynamic Analysis................... 19 3.2.3 Compiler Techniques................. 19 3.3 The Synergy of Two Analyses................ 19 3.4 API Usage Analysis...................... 21 3.4.1 Call Graph Extensions................ 21 3.4.2 API Usage Analysis.................. 24 3.5 Approximated Execution................... 25 3.5.1 Object and Taint Representation........... 26 3.5.2 Basic Execution Flow................. 27 3.5.3 Complete Execution Rules.............. 30 3.5.4 Tainting Rules..................... 31 viii 3.5.5 Execution Extensions and Optimizations...... 32 3.5.6 Approximation Mode................. 32 3.5.7 False Positive Analysis: Execution Path Validation. 35 3.5.8 False Negative Analysis: Tainting Validation.... 38 3.5.9 Infinity Avoidance................... 40 3.6 Evaluation........................... 41 3.6.1 Implementation.................... 42 3.6.2 Evaluation Methodology............... 44 3.6.3 Completeness of Static API Analysis......... 45 3.6.4 Detection Accuracy.................. 46 3.6.5 Usability.......................