DOCSLIB.ORG

Advancing Energy Testing of Mobile Applications

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Advancing Energy Testing of Mobile Applications UC Irvine UC Irvine Electronic Theses and Dissertations Title Advancing Energy Testing of Mobile Applications Permalink https://escholarship.org/uc/item/4vs7z6f2 Author Jabbarvand Behrouz, Reyhaneh Publication Date 2020 License https://creativecommons.org/licenses/by-nc/4.0/ 4.0 Peer reviewed|Thesis/dissertation eScholarship.org Powered by the California Digital Library University of California UNIVERSITY OF CALIFORNIA, IRVINE Advancing Energy Testing of Mobile Applications DISSERTATION submitted in partial satisfaction of the requirements for the degree of DOCTOR OF PHILOSOPHY in Software Engineering by Reyhaneh Jabbarvand Dissertation Committee: Professor Sam Malek, Chair Professor Cristina Videira Lopes Associate Professor James A. Jones 2020 © 2020 Reyhaneh Jabbarvand DEDICATION To my beloved husband and daughter, Alireza and Nora ii TABLE OF CONTENTS Page LIST OF FIGURES vii LIST OF TABLES ix ACKNOWLEDGMENTS x CURRICULUM VITAE xi ABSTRACT OF THE DISSERTATION xiii 1 Introduction 1 1.1 Dissertation Overview . .4 1.2 Dissertation Structure . .6 2 Background 8 2.1 Methodology . .9 2.1.1 Search Protocol . .9 2.1.2 Search Process . 12 2.1.3 Threats To Validity . 13 2.2 Defect Model . 14 2.2.1 Connectivity . 14 2.2.2 Display . 18 2.2.3 Location . 19 2.2.4 Recurring Constructs . 20 2.2.5 Sensor . 21 2.2.6 Wakelock . 21 2.3 Analysis . 23 2.3.1 RQ1: Distribution of Energy Defects . 23 2.3.2 RQ2: Misused Resource Type . 26 2.3.3 RQ4: Consequences of Energy Defects . 28 2.4 Discussion . 31 3 Related Work and Research Gap 32 3.1 Related Work . 32 3.1.1 Regression Testing . 32 iii 3.1.2 Test Adequacy Criterion . 34 3.1.3 Mutation Testing . 35 3.1.4 Android Testing . 36 3.1.5 Test Oracle . 37 3.1.6 Green Software Engineering . 38 3.2 Research Gap . 40 4 Research Problem 44 4.1 Problem Statement . 49 4.2 Research Hypotheses . 50 5 Energy-Aware Mutation Testing 53 5.1 Introduction . 53 5.2 Framework Overview . 56 5.3 Mutation Operators . 58 5.3.1 Location Mutation Operators . 60 5.3.2 Connectivity Mutation Operators . 61 5.3.3 Wakelock Mutation Operators . 65 5.3.4 Display Mutation Operators . 65 5.3.5 Recurring Callback and Loop Mutation Operators . 66 5.3.6 Sensor Mutation Operators . 66 5.4 Analyzing Mutants . 68 5.4.1 Killed Mutants . 68 5.4.2 Equivalent and Stillborn Mutants . 71 5.5 Evaluation . 72 5.5.1 Experimental Setup and Implementation . 73 5.5.2 RQ1: Prevalence, Quality, and Contribution . 76 5.5.3 RQ2: Effectiveness . 77 5.5.4 RQ3: Association to Real Faults . 80 5.5.5 RQ4: Accuracy of Oracle . 81 5.5.6 RQ5: Performance . 82 5.6 Discussion . 83 6 Energy-Aware Test Input Generation 85 6.1 Introduction . 85 6.2 Illustrative Example . 88 6.3 Approach Overview and Challenges . 90 6.4 Model Extractor . 93 6.4.1 Component Transition Graph (CTG) . 93 6.4.2 Lifecycle State Machine (LSM) . 97 6.4.3 Hardware State Machine (HSM) . 98 6.5 Test Generator . 101 6.5.1 Genetic Algorithm . 101 6.5.2 Genetic Operators . 103 6.5.3 Fitness Evaluation . 108 iv 6.5.4 Test-Suite Minimization . 110 6.6 Evaluation . 110 6.6.1 Experimental Setup . 112 6.6.2 RQ1: API and Execution Context Coverage . 113 6.6.3 RQ2: Effectiveness . 114 6.6.4 RQ3: Necessity of the Models . 115 6.6.5 RQ4: Energy Defects Coverage . 117 6.6.6 RQ5: Performance . 118 6.7 Discussion . 118 7 Energy-Aware Test Oracle 120 7.1 Introduction . 121 7.2 Motivating Example . 123 7.3 Approach Overview . 125 7.4 Sequence Collector . 127 7.4.1 State Vector (SV) . 127 7.4.2 Collecting Sequences . 129 7.5 Learning Engine . 130 7.5.1 Model Selection . 131 7.5.2 Long Short-Term Memory (LSTM) . 132 7.5.3 Dataset Curation . 134 7.5.4 Attention Mechanism . 135 7.6 Attention Analysis . 136 7.7 Evaluation . 140 7.7.1 Experimental Setup . 140 7.7.2 RQ1: Effectiveness . 144 7.7.3 RQ2: Usage of Attention Mechanism . 146 7.7.4 RQ3: Detecting Unseen Defect Types . 148 7.7.5 RQ4: Reusability of the Oracle . 149 7.7.6 RQ5: Performance . 151 7.8 Discussion . 152 8 Energy-Aware Test-Suite Minimization 153 8.1 Introduction . 154 8.2 Motivation . 156 8.3 Energy-Aware test-suite Minimization . 159 8.4 Approach Overview . 161 8.5 Energy-aware Coverage calculator . 162 8.6 energy-aware test-suite minimization . 165 8.6.1 Integer Non-linear Programming . 165 8.6.2 Integer Linear Programming . 168 8.6.3 Greedy algorithm . 170 8.7 Experimental Evaluation . 173 8.7.1 Experiment Setup . 173 8.7.2 RQ1: Effectiveness . 174 v 8.7.3 RQ2: Correlations . 177 8.7.4 RQ3: Performance . 178 8.8 Discussion . 181 9 Conclusion 183 9.1 Research Contributions . 184 9.2 Future Work . 188 Bibliography 192 vi LIST OF FIGURES Page 2.1 Proposed taxonomy of energy defects.
Load more

Recommended publications
  • Comparison of GUI Testing Tools for Android Applications
    Comparison of GUI testing tools for Android applications University of Oulu Department of Information Processing Science Master’s Thesis Tomi Lämsä Date 22.5.2017 2 Abstract Test automation is an intriguing area of software engineering, especially in Android development. This is since Android applications must be able to run in many different permutations of operating system versions and hardware choices. Comparison of different tools for automated UI testing of Android applications is done in this thesis. In a literature review several different tools available and their popularity is researched and the structure of the most popular tools is looked at. The two tools identified to be the most popular are Appium and Espresso. In an empirical study the two tools along with Robotium, UiAutomator and Tau are compared against each other in test execution speed, maintainability of the test code, reliability of the test tools and in general issues. An empirical study was carried out by selecting three Android applications for which an identical suite of tests was developed with each tool. The test suites were then run and the execution speed and reliability was analysed based on these results. The test code written is also analysed for maintainability by calculating the lines of code and the number of method calls needed to handle asynchrony related to UI updates. The issues faced by the test developer with the different tools are also analysed. This thesis aims to help industry users of these kinds of applications in two ways. First, it could be used as a source on what tools are over all available for UI testing of Android applications.
    [Show full text]
  • Mobile Developer's Guide to the Galaxy
    Don’t Panic MOBILE DEVELOPER’S GUIDE TO THE GALAXY U PD A TE D & EX TE ND 12th ED EDITION published by: Services and Tools for All Mobile Platforms Enough Software GmbH + Co. KG Sögestrasse 70 28195 Bremen Germany www.enough.de Please send your feedback, questions or sponsorship requests to: [email protected] Follow us on Twitter: @enoughsoftware 12th Edition February 2013 This Developer Guide is licensed under the Creative Commons Some Rights Reserved License. Editors: Marco Tabor (Enough Software) Julian Harty Izabella Balce Art Direction and Design by Andrej Balaz (Enough Software) Mobile Developer’s Guide Contents I Prologue 1 The Galaxy of Mobile: An Introduction 1 Topology: Form Factors and Usage Patterns 2 Star Formation: Creating a Mobile Service 6 The Universe of Mobile Operating Systems 12 About Time and Space 12 Lost in Space 14 Conceptional Design For Mobile 14 Capturing The Idea 16 Designing User Experience 22 Android 22 The Ecosystem 24 Prerequisites 25 Implementation 28 Testing 30 Building 30 Signing 31 Distribution 32 Monetization 34 BlackBerry Java Apps 34 The Ecosystem 35 Prerequisites 36 Implementation 38 Testing 39 Signing 39 Distribution 40 Learn More 42 BlackBerry 10 42 The Ecosystem 43 Development 51 Testing 51 Signing 52 Distribution 54 iOS 54 The Ecosystem 55 Technology Overview 57 Testing & Debugging 59 Learn More 62 Java ME (J2ME) 62 The Ecosystem 63 Prerequisites 64 Implementation 67 Testing 68 Porting 70 Signing 71 Distribution 72 Learn More 4 75 Windows Phone 75 The Ecosystem 76 Implementation 82 Testing
    [Show full text]
  • Behavioral Analysis of Android Applications Using Automated Instrumentation
    2013 Seventh International Conference on Software Security and Reliability Companion Behavioral Analysis of Android Applications Using Automated Instrumentation Mohammad Karami, Mohamed Elsabagh, Parnian Najafiborazjani, and Angelos Stavrou Computer Science Department, George Mason University, Fairfax, VA 22030 { mkarami, melsabag, pnajafib, astavrou}@gmu.edu Abstract—Google’s Android operating system has become one application is not a straight forward task due to variety of the most popular operating system for hand-held devices. Due inputs and heterogeneity of the technologies [12]. to its ubiquitous use, open source nature and wide-spread Two primary methods are being employed for mobile appli- popularity, it has become the target of recent mobile malware. In this paper, we present our efforts on effective security cation analysis: white-box approach and black-box approach. inspection mechanisms for identification of malicious applications In black-box testing only the inputs and outputs of the appli- for Android mobile applications. To achieve that, we devel- cation are being exercised. On the other hand, for white box oped a comprehensive software inspection framework. Moreover, approach the source code need to be analyzed. Since the source to identify potential software reliability flaws and to trigger code of the malicious applications that we get from Google malware, we develop a transparent instrumentation system for automating user interactions with an Android application that Play is not available we cannot analyze the internal structure does not require source code. Additionally, for run-time behavior of the malicious applications to figure out what they exactly analysis of an application, we monitor the I/O system calls gener- do, but we can utilize the black-box testing to define their ated the by application under monitoring to the underlying Linux functionality.
    [Show full text]
  • Unit Testing, Integration Testing and Continuous Builds for Android
    Unit Testing, Integration Testing and Continuous Builds Manfred Moser simpligility technologies inc. http://www.simpligility.com @simpligility Agenda Get an overview about testing and continuous integration for Android app development Why testing? What can we test? How can we do it? 2 Manfred Moser simpligility.com Apache Maven See previous presentation Maven used to control build and more Good library reuse and dependency use – makes testing easier out of the box Strong tool support But its all possible without Maven too... Why (automated) testing? Find problem early and you ● Can fix it quickly ● Safe money on QA testing ● Do not get negative customer feedback ● Deal with feature requests instead of bugs ● Avoid production problems ● Can refactor (and change) without breaking old stuff 4 Manfred Moser simpligility.com What are we testing? Plain java code Android dependent code Configuration User interface Look and feel 5 Manfred Moser simpligility.com JVM vs Dalvik/Android stack JVM based: ● Faster ● More tools available ● More mature tooling Dalvik based: ● Necessary for integration tests ● Reproduce actual behaviour ● Full stack testing (beyond VM, to native..) 6 Manfred Moser simpligility.com JVM testing tools ● JUnit ● TestNG ● EasyMock ● Unitils ● Cobertura ● Emma ● and many more 7 Android SDK Test Tools ● Integrated Junit ● use on emulator/device though ● Instrumentation Test Tools ● rich set of classes for testing ● now well documented ● MonkeyRunner ● control device/emulator running tests ● take screenshots ● jython 8 Dalvik/Android
    [Show full text]
  • Large-Scale Android Dynamic Analysis
    Andlantis: Large-scale Android Dynamic Analysis Michael Biermayz, Eric Gustafsonz, Jeremy Ericksony, David Fritzy, Yung Ryn Choey ∗ySandia National Laboratories fmbierma, jericks, djfritz, [email protected] zUniversity of California, Davis fmhbierma, [email protected] Abstract— In this paper, we present Andlantis: a highly scalable Analyzing Android applications for malicious behavior is an dynamic analysis framework for analyzing applications on important area of research, and is made difficult, in part, by the Android operating system. Andlantis runs the Android the increasingly large number of applications available for the operating system in a virtualized environment and is able platform. While techniques exist to perform static analysis on a to provide the virtual device with artificial network data in large number of applications, dynamic analysis techniques are order to provide an environment which closely replicates that relatively limited in scale due to the computational resources of a physical device. Andlantis is able to schedule and run required to emulate the full Android system to achieve accurate thousands of Android instances in parallel, enabling us to execution. We present Andlantis, a scalable dynamic analysis investigate the behavior of mobile malware at scale. system capable of processing over 3000 Android applications per hour. During this processing, the system is able to collect Andlantis employs a scalable high-performance emulytics valuable forensic data, which helps reverse-engineers and mal- framework, minimega, to parallelize this expensive task as ware researchers identify and understand anomalous application much as possible and achieve a level of throughput un- behavior. We discuss the results of running 1261 malware samples precedented in Android dynamic analysis.
    [Show full text]
  • Guide to Test Automation Tools 2017 - 2018
    Guide to Test Automation Tools 2017 - 2018 WHITEPAPER QATestlab 2017 Copyright 2017 ©QATestLab. All Rights Reserved Table of Contents Summary 3 Introduction 3 1. Test Automation Tools. Market review 1.1. Selenium WebDriver Framework 4 1.2. Appium Framework 5 1.3. Robotium Framework 7 1.4. Serenity Framework 9 1.5. Robot Framework 10 1.6. Galen Framework 12 1.7. HP Unified Functional Testing (UFT) 14 1.8. Ranorex Studio 16 1.9. TestComplete 19 1.10. Telerik Test Studio 20 1.11. Applitools Eyes 22 1.12. Test Automation Tools and Frameworks: Comparison of 23 Technical Aspects 2. Test Automation Tools Approved by QATestLab 2.1. Selenium WebDriver 26 2.2. Appium 28 2.3. TestComplete 29 2.4. Ranorex Studio 31 3. Summary 32 Contact Information 33 2 Copyright 2017 ©QATestLab. All Rights Reserved Summary Table of Contents Click the section to jump This whitepaper aims at providing the comprehensive data on the most ahead popular test automation tools in 2017 - 2018 including the description of Summary their parameters which can be considered when selecting a tool / framework for test automation. The document also provides the Introduction comparison of the leading test automation tools highlighting both 1. Test Automation advantages and disadvantages, and also main objectives, technical Tools. Market review characteristics and the information about a provider. 1.1. Selenium WebDriver Framework The whitepaper is aimed to assist in selecting a proper test automation 1.2 Appium Framework tool avoiding time and money losses. Besides, it includes the 1.3 Robotium recommendations on the most effective test automation tools, Framework 1.4 Serenity Framework information about their effectiveness and maintainability, which were 1.5 Robot Framework prepared by QATestLab on the ground of successful execution of 50 test 1.6 Galen Framework automation projects.
    [Show full text]
  • Profiling the Responsiveness of Android Applications Via Automated
    Profiling the Responsiveness of Android Applications via Automated Resource Amplification Yan Wang Atanas Rountev Ohio State University Ohio State University ABSTRACT Android run-time|the user may decide to uninstall the ap- The responsiveness of the GUI in an Android application is plication and/or rate it negatively in the app market. an important component of the user experience. Android Android guidelines [9, 6] are very clear on the importance guidelines recommend that potentially-expensive operations of designing responsive applications. The general rule is the should not be performed in the GUI thread, but rather in following: \In any situation in which your app performs a separate threads. The responsiveness of existing code can potentially lengthy operation, you should not perform the be improved by introducing such asynchronous processing, work on the UI thread, but instead create a worker thread either manually or automatically. and do most of the work there." [9]. One simple view is that all potentially-expensive opera- There are various mechanisms for achieving this goal. Typ- tions should be removed from the GUI thread. We demon- ical examples include user-managed threads, AsynchTask, strate that this view is too simplistic, because run-time cost and IntentService. The responsiveness of existing code under reasonable conditions may often be below the thresh- can be improved by introducing these mechanisms either old for poor responsiveness. We propose a profiling ap- through manual refactoring or by using automated transfor- proach to characterize response times as a function of the mations (e.g., [12, 11]). A natural question that arises in size of a potentially-expensive resource (e.g., shared prefer- this context is the following: which operations should be re- ences store, bitmap, or SQLite database).
    [Show full text]
  • Automation for Mobile Apps
    automation for mobile apps SeConf 13 Workshop http://appium.io/seconf.pdf Jonathan Lipps | @jlipps | Sauce Labs @TheDanCuellar | @maudineormsby appium is the cross-platform solution for native and hybrid mobile automation appium introduction 1 2 3 4 5 6 iOS Android calabash-ios calabash-android Frank MonkeyTalk UIAutomation Robotium ios-driver UiAutomator KeepItFunctional selendroid Philosophy R1. Test the same app you submit to the marketplace R2. Write your tests in any language, using any framework R3. Use a standard automation specification and API R4. Build a large and thriving open-source community effort Platforms • Real devices (iOS, Android) • Simulators (iOS, Android, FirefoxOS) • Hybrid apps (iOS, Android, FirefoxOS) • Safari on iOS • Chrome on Android • Robot-controlled devices Architecture • Apple Instruments & UIAutomation for iOS • Google UiAutomator for Android (4.2.1 up) • Selendroid for older Android & hybrid • Selenium WebDriver interface Selenium WebDriver? • this is SeConf, isn’t it? appium.app 1 2 3 4 5 6 Appium.app • GUI for launching Appium server • Monitor status • Set preferences Appium.app • Inspector for probing your app • Create hooks for UI elements in app • Try out actions • Record / playback actions • Convert UIAutomation JS to Appium code Appium.app • Mac: stable • Windows: under development Monitor Preferences Inspector Recorder Robot support • Bitbeambot Delta-2 • http://www.bitbeam.org • Tapster • https://www.tindie.com/products/hugs/robot-that- plays-angry-birds/ Robot support • Redirects touch actions
    [Show full text]
  • GUI-Guided Test Script Repair for Mobile Apps
    This is the author's version of an article that has been published in this journal. Changes were made to this version by the publisher prior to publication. The final version of record is available at http://dx.doi.org/10.1109/TSE.2020.3007664 1 GUI-Guided Test Script Repair for Mobile Apps Minxue Pan, Tongtong Xu, Yu Pei, Zhong Li, Tian Zhang, Xuandong Li Abstract—Graphical User Interface (GUI) testing is widely used to test mobile apps. As mobile apps are frequently updated and need repeated testing, to reduce the test cost, their test cases are often coded as scripts to enable automated execution using test harnesses/tools. When those mobile apps evolve, many of the test scripts, however, may become broken due to changes made to the app GUIs. While it is desirable that the broken scripts get repaired, doing it manually can be preventively expensive if the number of tests need repairing is large. We propose in this paper a novel approach named METER to repairing broken GUI test scripts automatically when mobile apps evolve. METER leverages computer vision techniques to infer GUI changes between two versions of a mobile app and uses the inferred changes to guide the repair of GUI test scripts. Since METER only relies on screenshots to repair GUI tests, it is applicable to apps targeting open or closed source mobile platforms. In experiments conducted on 22 Android apps and 6 iOS apps, repairs produced by METER helped preserve 63.7% and 38.8% of all the test actions broken by the GUI changes, respectively.
    [Show full text]
  • Code Coverage Measurement Framework for Android Devices∗
    Acta Cybernetica 21 (2014) 439–458. Code Coverage Measurement Framework for Android Devices∗ Ferenc Horváth,y Szabolcs Bognár,y Tamás Gergely,y Róbert Rácz,y Árpád Beszédes,y and Vladimir Marinkovicz Abstract Software testing is a very important activity in the software development life cycle. Numerous general black- and white-box techniques exist to achieve different goals and there are a lot of practices for different kinds of soft- ware. The testing of embedded systems, however, raises some very special constraints and requirements in software testing. Special solutions exist in this field, but there is no general testing methodology for embedded systems. One of the goals of the CIRENE project was to fill this gap and define a general testing methodology for embedded systems that could be specialized to different environments. The project included a pilot implementation of this methodology in a specific environment: an Android-based Digital TV receiver (Set-Top-Box). In this pilot, we implemented method level code coverage measurement of Android applications. This was done by instrumenting the applications and creating a framework for the Android device that collected basic infor- mation from the instrumented applications and communicated it through the network towards a server where the data was finally processed. The result- ing code coverage information was used for many purposes according to the methodology: test case selection and prioritization, traceability computation, dead code detection, etc. The resulting methodology and toolset were reused in another project where we investigated whether the coverage information can be used to de- termine locations to be instrumented in order to collect relevant information about software usability.
    [Show full text]
  • How to Make a Successful App Appfutura Annual Report 2016
    HOW TO MAKE A SUCCESSFUL APP APPFUTURA ANNUAL REPORT 2016 Copyright © 2016 by AppFutura appfutura INDEX 01 AppFutura 02 Introduction 04 Time and cost of a mobile app 21 Steps to hire the best mobile app development company 27 Benchmarking an app idea 39 Wireframes, the key to usability 48 Designing mobile apps 64 Apps for kids 85 Game apps 97 Health apps 121 Social apps 136 Wearable apps 146 Mobile app testing 152 Marketing your mobile app 167 Future mobile app development trends 196 Final Considerations 198 Collaborators Copyright © 2016 by AppFutura All rights reserved. This book or any portion thereof may not be reproduced, uploaded, transmitted or used in any manner whatsoever without the prior written permission of the publisher. For permission requests, write to the publisher, addressed “AppFutura Annual Report” at [email protected] Follow us on AppFutura AppFutura Annual Report 2016 appfutura APPFUTURA AppFutura started as a project in the Mobile World Congress of 2013, born out of the needs of mobile app development companies to find projects and the needs of clients to find experts to develop their apps. The platform has now become a worldwide community for mobile app developers where they can meet people or companies looking for the best firms to develop an app project. We offer help so clients reach their best partner. We have more than 15,000 developers and mobile app development companies listed worldwide. On AppFutura, they can find everything they are looking for: verified leads and qualified traffic. AppFutura has so far published over 4,500 mobile app projects in its history.
    [Show full text]
  • An Automated Virtual Security Testing Platform for Android Mobile Apps
    An Automated Virtual Security Testing Platform for Android Mobile Apps Yong Wang College of Business and Information Systems Dakota State University Madison, SD 57042 [email protected] Abstract—This paper proposes an automated virtual multiple of the following techniques, privilege escalation, security testing platform for Android mobile apps. The testing remote control, financial charge, and information collection, platform includes three key components: customizing Android etc. The previous stated techniques provide a malicious OS to include mobile app trace information, creating a virtual attacker with a variety of options to utilize a compromised testing platform using the customized OS, and developing static mobile device. and dynamic analyzing techniques for mobile malware detection. The proposed testing platform is a server-side malware detection Most client-side malware detection tools are based on solution. It can utilize both static and dynamic analysis and is a signatures. However, the signature-based approach can only good compensation to the client-side mobile security software. be used to detect known malware. Google has introduced a Keywords—Android, mobile app, security, malware, detection server-side approach, Bouncer, to detect malicious apps before they hit the Google Play Store [4]. This technique is great for I. INTRODUCTION apps that are downloaded through the Google Play Store, but Mobile devices such as smartphones and tablets have been is disadvantageous for the users who use third party app widely adopted for personal and business purposes. However, stores. A cloud-based mobile malware detection framework is the popularity of the mobile devices also raises many security introduced in [5]. The proposed testing platform in this paper issues and challenges.
    [Show full text]