NO CODE TEST RECORDING for Ios APPLICATIONS
Total Page：16
File Type：pdf, Size：1020Kb
NO CODE TEST RECORDING FOR iOS APPLICATIONS A Thesis Presented to The Academic Faculty by Anushk Mittal In Partial Fulfillment of the Requirements for the Degree Bachelor of Science with the Research Option in the School of Computer Science Georgia Institute of Technology July 2020 Copyright © Anushk Mittal 2020 NO CODE TEST RECORDING FOR iOS APPLICATIONS Approved by: Dr. Alessandro Orso, Advisor School of Computer Science Georgia Institute of Technology Dr. Qirun Zhang School of Computer Science Georgia Institute of Technology Date Approved: July 27, 2020 2 ACKNOWLEDGMENTS I wish to thank Dr. Alessandro Orso without whom none of this would have been possible. I also wish to thank my mentor Dr. Shauvik Roy Choudhary, whom I worked under and with throughout my years of undergraduate research. He inspired me with the initial idea and helped develop the skills necessary to bring it to life. He guided me and supported me throughout the process, from ideation to completion. I would also like to thank my parents, my professors throughout my undergraduate studies at Georgia Tech, and the College of Computing, who believed in me, supported me and guided me through the path of learning new things and venturing into the unknown. If you want to change the world, you're at Georgia Tech! You can do that! 3 TABLE OF CONTENTS Page ACKNOWLEDGMENTS................................................................................................................... 3 LIST OF TABLES............................................................................................................................... 5 LIST OF FIGURES............................................................................................................................. 6 SUMMARY......................................................................................................................................... 7 CHAPTER I. Introduction...................................................................................................................................... 8 II. Literature Review............................................................................................................................ 17 III. Tool Design.................................................................................................................................... 24 IV. Experiment & Results..................................................................................................................... 47 V. Conclusion........................................................................................................................................ 54 VI. Future Work.................................................................................................................................... 55 REFERENCES...................................................................................................................................... 57 4 LIST OF TABLES Page Table 1: Assertable properties for the test case recording phase....................................................... 38 Table 2: List of open-source apps & NLTCs used for experimentation............................................ 49 Table 3: List of devices and recording / execution status.................................................................. 52 5 LIST OF FIGURES Page Figure 1a: Calculator Test Application UI....................................................................................... 12 Figure 1b: Calculator Test Application UI Hierarchy...................................................................... 12 Figure 2: iOS main function............................................................................................................. 20 Figure 3: iOS App Architecture....................................................................................................... 20 Figure 4: High-level overview of the adopted Barista approach [4] ............................................... 24 Figure 5: SDK Use Flow................................................................................................................... 25 Figure 6: Flowchart representing a high-level overview of recording phase.................................... 31 Figure 7: Abstract syntax of the record trace.................................................................................... 32 Figure 8: Recorded trace for “2+9=11 test case”.............................................................................. 33 Figure 9: Importing iOSTestSDK in a sample open-source Calculator app..................................... 34 Figure 10a: iOSTestSDK float button in the Calculator open-source app......................................... 36 Figure 10b: Float button presented with options............................................................................... 36 Figure 11a: Assertion check grid presented after selecting assertion check...................................... 37 Figure 11b: Button “1” selected in the assertion mode, single tap..................................................... 37 Figure 11c: Double-tapping presents an assertion table, where the receiver subview has been selected to assert property.................................................................................................................. 37 Figure 12: Add comments screen presented with user typed in comment........................................ 39 Figure 13: Automatically generated XCTestCase for “2+9=11 test case”........................................ 42 Figure 14: Pseudocode for encoder service that reads record trace JSON to write XCTestCase swift file............................................................................................................................................. 43 Figure 15: Adding UI Testing Bundle target to an existing Xcode project....................................... 45 Figure 16: UI widgets handled by iOSTestSDK............................................................................... 53 6 SUMMARY Over the past decade, mobile apps have touched every sphere of life with ~4.5M applications available to download through Apple’s App Store and Google’s Play Store [1] that are expected to generate ~$1T in revenue by 2023 [2]. Today, an average American checks their phone once every 12 minutes [3], but testing these mobile apps is mostly unreliable and too resource expensive with current state-of-the-art technology. Specifically, testing iOS apps requires writing code using Xcode IDE that requires a development environment setup for all testers. These testers must also be familiar with coding for iOS apps as they need to interface with XCUITest API to write UI tests or verify the automatically generated code through Xcode’s XCUITest Recorder. To address this issue and to make iOS testing accessible to everyone, we adopt the Barista technique [4] to passively record user interactions and build device-independent test scripts using any iOS device. We describe a three-part technique of recording user interactions through Objective-C swizzling, encoding generated test cases using a separately hosted server, and generating XCUITest files to run encoded test cases. We conclude with experimental results and discussions that demonstrate the effectiveness of our solution on a host of sample open-source applications that represent the most common and popular app categories and functionalities along with future directions on how this collected big data could be leveraged for intelligent insights. The goal of this research is to make testing approachable and easy for large corporations and indie developers alike. The presented tool has been made open-source at https://github.com/anushkmittal/iOSTestSDK. 7 CHAPTER I INTRODUCTION Over the past decade, mobile apps have touched every sphere of life from making new friends online to finding one’s significant other, to banking, shopping and playing games. They have replaced physical objects on our desks, such as calculators, torches, notepads, stickies, and more. In 2019 alone, the App Store ecosystem supported $519 billion in billings and sales globally [5]. No doubt, mobile apps play a significant role in our lives and present a multi-billion dollar industry [6]. Like all traditional software, these apps must be tested to check for correct behavior under different input conditions and scenarios, especially when failures in an app can result in loss of reputation, and ultimately customers, for all online businesses. These concerns are largely unanswered for iOS smartphones created and developed by Apple, Inc. iOS is the operating system that presently powers many of the company's mobile devices, including the iPhone, iPad, and iPod Touch. Initially unveiled in 2007 for the iPhone, iOS has been extended to support other Apple devices such as the iPod Touch (September 2007) and the iPad (January 2010). As of the 1st quarter of 2020, Apple's App Store contains more than 1.8 million iOS apps [1], and in 2019, apps available on the App Store were downloaded over 31 billion times [7]. Today, approximately 50% of all smartphone devices in the US are iPhones [8]. However, testing iOS apps is particularly challenging as major new iOS releases are announced yearly during Apple’s Worldwide Developers Conference (WWDC) with support for the majority of iPhones and iPads. Between 2017 to 2020, in the last three years, Apple has released 10 new iPhones and 11 new iPad models. iPhones introduced new screens of sizes 5.8-inch, 8 6.5-inch,