An Experimental Study of the Performance, Energy, and Programming Effort Trade-offs of Android Persistence Frameworks Jing Pu Thesis submitted to the Faculty of the Virginia Polytechnic Institute and State University in partial fulfillment of the requirements for the degree of Master of Science in Computer Science and Applications Eli Tilevich, Chair Barbara G. Ryder Francisco Servant July 1, 2016 Blacksburg, Virginia Keywords: Energy Efficiency; Performance; Programming Effort; Orthogonal Persistence; Android; Copyright 2016, Jing Pu An Experimental Study of the Performance, Energy, and Programming Effort Trade-offs of Android Persistence Frameworks Jing Pu (ABSTRACT) One of the fundamental building blocks of a mobile application is the ability to persist program data between different invocations. Referred to as persistence, this functionality is commonly implemented by means of persistence frameworks. When choosing a particular framework, Android|the most popular mobile platform—offers a wide variety of options to developers. Unfortunately, the energy, performance, and programming effort trade-offs of these frameworks are poorly understood, leaving the Android developer in the dark trying to select the most appropriate option for their applications. To address this problem, this thesis reports on the results of the first systematic study of six Android persistence frameworks (i.e., ActiveAndroid, greenDAO, Orm- Lite, Sugar ORM, Android SQLite, and Realm Java) in their application to and performance with popular benchmarks, such as DaCapo. Having measured and ana- lyzed the energy, performance, and programming effort trade-offs for each framework, we present a set of practical guidelines for the developer to choose between Android persistence frameworks. Our findings can also help the framework developers to optimize their products to meet the desired design objectives. This thesis is based on research accepted for publication in the proceedings of the IEEE Modeling, Analysis, and Simulation of Computer and Telecommunication Sys- tems Conference (MASCOTS 2016). This research is supported by the National Science Foundation through the Grant CCF-1116565. Dedication To My Family For Their Love And Support iii Acknowledgments I would like to thank all of those who have contributed to this work. In particular I am most grateful to Dr. Eli Tilevich, my supervisor, for his help and support which made this thesis possible. I am indebted to him not just for his scientific contribution but also for his motivating words, day after day, and his friendship. I would like to thank the members of my thesis committee: Drs. Barbara G. Ryder, Francisco Servant and Eli Tilevich for their scientific and emotional support. I would like to give special thanks to Zheng Song for his help and suggestion with my research project. I wish to express my sincere appreciation to Sharon Kinder-Potter in the office of Computer Science Department for her help and assistance. My thanks goes to all my colleagues in the Computer Science Department for their help and advice which made my stay at Virginia Polytechnic Institute and State University a most pleasant experience. I would like to thank Newman Library for the job position provided to me and Anne Lawrence, Amanda French, Keith Gilbertson and other colleagues there for their assistance. Finally, I would like to pay tribute to the constant support of my family, my relatives and my friends, without their love over the many years none of this would have been possible and whose sacrifice I can never repay. I want to thank my lovely mother for always being there. A very special thanks goes to my husband, Sai, for helping and encouraging me all the time. iv Contents 1 Introduction 1 1.1 Persistence, Android Programming, and Energy Efficiency . .1 1.2 Analyzing the Trade-offs of Android Persistence Frameworks . .2 1.3 Research Questions . .3 1.4 Research Findings . .3 1.5 Research Contributions . .4 1.6 Thesis Roadmap . .5 2 Terminology 6 3 Motivation 9 4 Related Work 11 4.1 Persistence Framework . 11 4.2 Performance and Energy Efficiency . 12 4.3 Programming Effort . 13 5 Android Persistence Frameworks 14 5.1 The Evolution of Persistence Frameworks . 14 5.2 Android Persistence Framework Overview . 15 v 5.3 Android Persistence Framework Feature Comparison . 16 6 Experiment Design 21 6.1 Android Persistence Framework Experiment Architecture . 21 6.2 Android Persistence Framework Selection . 22 6.3 Benchmark Selection . 23 6.4 Parameters and Dependent Variables . 24 6.4.1 Performance . 25 6.4.2 Energy . 25 6.4.3 Programming Effort . 26 6.4.4 Benchmark Input Parameters . 26 6.5 Experimental Hardware and Tools . 26 7 Study Results 30 7.1 Programming Effort Analysis . 30 7.1.1 Uncommented Lines of Code Analysis . 30 7.1.2 McCabe Cyclomatic Complexity Analysis . 31 7.1.3 Overall Programming Effort Analysis . 33 7.2 Experiments with the Android ORM Benchmark . 34 7.2.1 Initialization Analysis . 34 7.2.2 Insert Analysis . 36 7.2.3 Update Analysis . 38 7.2.4 Select and Delete Analysis . 40 7.2.5 Overall Analysis . 44 7.3 Experiments with the DaCapo benchmark . 45 8 Recommendation Model 52 vi 8.1 Recommendation Model . 52 8.1.1 Design of PEP Model . 52 8.1.2 Benchmark Evaluation . 54 9 Threats to Validity 56 9.1 External Threats . 56 9.2 Internal Threats . 56 10 Future Work 58 10.1 Expanding Platform and Case Study . 58 10.2 Understanding the Concurrency Behavior of Android Persistence Frame- work.................................... 59 10.3 Designing Energy Efficient Persistence Frameworks . 59 10.4 Expanding the Study of Android Middleware . 60 11 Conclusions 61 Bibliography 61 A Persistence Framework Libraries 70 B Initialization Core Code 71 C Insert Core Code 74 D Update Core Code 79 E Select Core Code 88 F Delete Core Code 94 vii List of Figures 6.1 Android Persistence Framework Experiment Architecture . 22 6.2 Monsoon Mobile Device Power Monitor's main channel measurement connection . 27 6.3 Monsoon Mobile Device Power Monitor's Software UI . 28 6.4 Traceview Profiling UI . 29 7.1 Comparison of initialization energy consumption among different per- sistence frameworks in Android ORM benchmark. The x axis indi- cates six Android persistence frameworks and the y axis is the energy consumption of benchmark with individual framework. 35 7.2 Comparison of initialization execution time among different persis- tence frameworks in the Android ORM Benchmark. The x axis indi- cates six Android persistence frameworks and the y axis is the execu- tion time of benchmark with individual framework. 35 7.3 Comparison of initialization read/write operations among SQLite-based Android persistence frameworks in the Android ORM Benchmark. The x axis indicates five SQLite-based Android persistence frame- works, and the y axis is the operation number of benchmark with individual framework: specially, black bar shows the read operation number and red bar shows the write operation number. the read and write number for Sugar ORM are 0. 36 viii 7.4 Comparison of insert energy consumption among different persistence frameworks in the Android ORM Benchmark. Six Android persistence frameworks are represented by different colors, the x axis indicates number of transaction, and the y axis is the energy consumption of benchmark with individual framework. 37 7.5 Comparison of insert execution time among different persistence frame- works in the Android ORM Benchmark. Six Android persistence frameworks are represented by different colors, the x axis indicates number of transaction, and the y axis is the execution time of bench- mark with individual framework. 37 7.6 Comparison of insert read/write operations among SQLite-based An- droid persistence frameworks in the Android ORM Benchmark. The x axis indicates five SQLite-based Android persistence frameworks, and the y axis is the operation number of benchmark with individ- ual framework when number of transaction = 125: specially, black bar shows the read operation number and red bar shows the write operation number. 38 7.7 Comparison of update energy consumption among different persis- tence frameworks in the Android ORM Benchmark. Six Android per- sistence frameworks are represented by different colors, the x axis indi- cates number of transaction, and the y axis is the energy consumption of benchmark with individual framework. 39 7.8 Comparison of update execution time among different persistence frame- works in the Android ORM Benchmark. Six Android persistence frameworks are represented by different colors, the x axis indicates number of transactions, and the y axis is the execution time of bench- mark with individual framework. 39 7.9 Comparison of update read/write operations among different persis- tence frameworks in the Android ORM Benchmark. The x axis in- dicates five SQLite-based Android persistence frameworks, and the y axis is the operation number of benchmark with individual framework when number of transactions = 125: specially, black bar shows the read operation number and red bar shows the write operation number. 40 ix 7.10 Comparison of select energy consumption among different persistence frameworks in the Android ORM Benchmark. Six Android persistence frameworks are represented by different colors, the x axis indicates number of transactions, and the y axis is the energy consumption of benchmark with individual framework. 41 7.11 Comparison of select execution time among different persistence frame- works in the Android ORM Benchmark. Six Android persistence frameworks are represented by different colors, the x axis indicates number of transactions, and the y axis is the execution time of bench- mark with individual framework. 41 7.12 Comparison of select read/write operations among SQLite-based An- droid persistence frameworks in the Android ORM Benchmark. The x axis indicates five SQLite-based Android persistence frameworks, and the y axis is the operation number of benchmark with individ- ual framework when number of transactions = 125: specially, black bar shows the read operation number and red bar shows the write operation number.