MASARYK UNIVERSITY FACULTY OF INFORMATICS Testing techniques for mobile device applications DIPLOMA THESIS Bc. Radim Göth Brno, 2015 Declaration Hereby I declare, that this paper is my original authorial work, which I have worked out by my own. All sources, references and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source. Brno, _______________ Bc. Radim Göth Advisor: Mgr. Petr Neugebauer, MBA II Acknowledgement I would like to express my gratitude to my supervisor Mgr. Petr Neugebauer, MBA for the opportunity to develop this master thesis within the Y Soft Corporation and for useful comments and help during the thesis elaboration. Furthermore, I would like to thank Tomáš Kuba for consultations, support and advices. I am also very thankful to my family and close friends for their patience and support. III Abstract This thesis investigates approaches for validation and verification of mobile applications. There is a description of mobile operating systems, application types and mobile test environment, followed by various techniques, tools and guidelines for mobile applications testing. Practical part of the thesis describes developing and testing mobile application SafeQ Terminal Demo for Y Soft Corporation. This application serves for demonstrating behavior of YSoft SafeQ on mobile devices. IV Keywords Mobile application, testing, Android, iOS, Windows Store, software quality, YSoft SafeQ. V Contents 1 Introduction ............................................................................................................ 3 2 Mobile application development options .......................................................... 5 2.1 Native applications ........................................................................................ 5 2.2 Web applications ............................................................................................ 5 2.3 Hybrid applications ....................................................................................... 6 3 Mobile operating systems .................................................................................... 7 3.1 Android ............................................................................................................ 7 3.2 iOS .................................................................................................................... 8 3.3 Windows & Windows Phone ....................................................................... 9 4 Mobile testing approaches and techniques ..................................................... 11 4.1 Test equipment ............................................................................................. 11 4.1.1 Real devices ............................................................................................ 11 4.1.2 Simulators and emulators .................................................................... 11 4.1.3 Mobile device cloud .............................................................................. 12 4.1.4 Summary................................................................................................. 14 4.2 Test levels ...................................................................................................... 15 4.2.1 Component testing ................................................................................ 15 4.2.2 Integration testing ................................................................................. 16 4.2.3 System testing ........................................................................................ 16 4.2.4 Acceptance testing ................................................................................. 17 4.3 Test automation ............................................................................................ 18 4.4 Continuous integration ............................................................................... 19 5 Mobile application qualities............................................................................... 20 5.1 Reliability ....................................................................................................... 21 5.1.1 Interruptions .......................................................................................... 21 5.1.2 Networks ................................................................................................ 22 5.2 Transferability ............................................................................................... 23 5.2.1 Installation .............................................................................................. 23 5.2.2 OS updates ............................................................................................. 23 5.3 Security .......................................................................................................... 24 5.3.1 Data storage ............................................................................................ 24 5.3.2 Transport security ................................................................................. 24 5.3.3 Binary protection ................................................................................... 25 5.4 Usability ......................................................................................................... 26 1 5.5 Performance .................................................................................................. 26 5.5.1 Launch time ............................................................................................ 27 5.5.2 Responsiveness ...................................................................................... 27 5.5.3 Battery life ............................................................................................... 27 5.5.4 Network usage efficiency ..................................................................... 27 5.6 Application store compliance ..................................................................... 28 6 Testing tools ......................................................................................................... 29 6.1 Experitest ....................................................................................................... 29 6.2 Ranorex .......................................................................................................... 30 6.3 Calabash & Xamarin Test Cloud ................................................................ 31 6.4 Perfecto mobile ............................................................................................. 32 6.5 IBM Mobile Quality Assurance .................................................................. 32 6.6 Testing tools overview ................................................................................. 33 6.7 Automated validation tools ........................................................................ 34 7 Testing guidelines ............................................................................................... 36 8 SafeQ Terminal Demo ......................................................................................... 39 8.1 Motivation ..................................................................................................... 39 8.2 Analysis ......................................................................................................... 39 8.2.1 Stakeholders ........................................................................................... 39 8.2.2 Requirements ......................................................................................... 39 8.2.3 Solution design ...................................................................................... 40 8.2.4 Functional requirements specification ............................................... 40 8.2.5 Non-functional requirements specification ....................................... 41 8.3 Architecture ................................................................................................... 42 8.4 Functional testing ......................................................................................... 45 8.5 Non-functional testing ................................................................................. 45 8.5.1 Performance ........................................................................................... 45 8.5.2 Transferability ........................................................................................ 52 8.5.3 Security ................................................................................................... 52 8.5.4 Reliability ................................................................................................ 52 8.5.5 Usability .................................................................................................. 53 8.5.6 Windows Store compliance ................................................................. 53 9 Conclusion ............................................................................................................ 54 10 Bibliography ....................................................................................................... 55 Appendix A – User stories .................................................................................... 62 Appendix B – SafeQ Terminal Demo screenshots ............................................. 66 2 1 Introduction Mobile application testers and developers have to adapt to ever-changing mobile environment and market trends. Reported by Critercism