Software Quality Characteristics Tested for Mobile Application Development
Total Page:16
File Type:pdf, Size:1020Kb
Thesis no: MGSE-2015-02 Software Quality Characteristics Tested For Mobile Application Development Literature Review and Empirical Survey WALEED ANWAR Faculty of Computing Blekinge Institute of Technology SE-371 79 Karlskrona Sweden This thesis is submitted to the Faculty of Computing at Blekinge Institute of Technology in partial fulfillment of the requirements for the degree of Master of Science in Software Engineering. The thesis is equivalent to 10 weeks of full time studies. Contact Information: Author(s): WALEED ANWAR E-mail: [email protected] University advisor: Dr. Simon Poulding Department of Software Engineering Faculty of Computing Internet : www.bth.se Blekinge Institute of Technology Phone : +46 455 38 50 00 SE-371 79 Karlskrona, Sweden Fax : +46 455 38 50 57 ABSTRACT Context. Smart phones use is increasing day by day as there is large number of app users. Due to more use of apps, the testing of mobile application should be done correctly and flawlessly to ensure the effectiveness of mobile applications. Objectives. The objective of this research is to find out the important mobile application quality characteristics from developer’s perspective and how developers actually test for them. Apart from that how the developers test their mobile applications are also addressed. Methods. Two methodologies were used: the literature survey and the empirical survey. The literature survey was used to get familiar with the most commonly known mobile application quality characteristics for which mobile applications are tested for. The empirical survey was used to get data from developers by sending an online questionnaire link to the Google Play store developers and their response was recorded and further evaluated to present results. I categorized apps as top rated apps and low rated apps. Results. I got 43 responses as a whole from 1000 developers. I got 20 responses out of 500 from the top rated apps developers and 23 responses out of 500 from the low rated apps developers. The results were used to compare the quality characteristics importance to testing. According to the responses from both top rated and low rated apps developer, functional testing and usability testing is considered to be the most important and are mostly tested for “Android” mobile applications. Load testing and energy testing is considered to be not that important from developer’s point of view but energy testing and memory testing is not so often tested for “Android” mobile applications by developers. When it comes to testing process, top rated and low rated apps developer perform testing by creating test cases that are predefined in scripts or documents. They prefer to execute their tests on emulator rather than on mobile and run their tests automatically via tool or service as a cloud. Conclusion. My study reports the quality characteristics importance for which the mobile applications are tested for from developer’s perspective, actual testing done by developers and the testing process used by them. Functional testing and usability testing are considered to be most important and mostly tested quality characteristics according to developer’s perspective. Testing is done by running their tests on emulator automatically through some tool. They prefer to create their test cases that are predefined in scripts. Keywords: Mobile Application Testing, Quality Characteristics 3 ACKNOWLEDGEMENT I would like to extend my gratitude to my supervisor Simon Poulding for his vital support and feedback till end. This thesis would not have been possible without Simon Poulding guidance throughout the research. I really feel honored working with him. I would like to thank my mom, siblings and friends who helped me and supported me. Finally, I sincerely appreciate all the participants of the web questionnaire from all over the world because without their response my thesis would not be accomplished. 4 TABLE OF CONTENTS ABSTRACT ................................................................................................................................... 3 ACKNOWLEDGEMENT ............................................................................................................ 4 TABLE OF CONTENTS .............................................................................................................. 5 LIST OF FIGURES ....................................................................................................................... 7 LIST OF TABLES ......................................................................................................................... 8 1 INTRODUCTION ................................................................................................................. 9 1.1 AIMS AND OBJECTIVES ........................................................................................... 10 1.2 RELATED WORK ........................................................................................................ 10 2 METHODOLOGY: ............................................................................................................. 12 2.1 RESEARCH QUESTIONS ............................................................................................. 12 2.2 LITERATURE REVIEW ................................................................................................. 12 2.2.1 EXTENDED DESIGN AND DERIVING THE START SET OF PAPERS: ............ 13 2.3 EMPIRICAL SURVEY ................................................................................................... 14 2.4 PREPARATION OF QUESTIONNAIRE ....................................................................... 15 2.5 RESEARCH DESIGN AND DATA COLLECTION: ...................................................... 15 3 RESULTS ............................................................................................................................. 17 3.1 TOP RATED APPS DEVELOPER’S FEEDBACK ..................................................... 17 3.1.1 IMPORTANCE OF QUALITY CHARACTERISTICS ............................................ 17 3.1.2 TESTING OF QUALITY CHARACTERISTICS ..................................................... 18 3.1.3 TESTING PROCESS OF TOP RATED APPLICATIONS ..................................... 20 3.2 LOW RATED APPS DEVELOPER’S FEEDBACK ................................................... 20 3.2.1 IMPORTANCE OF QUALITY CHARACTERISTICS ............................................ 21 3.2.2 TESTING OF QUALITY CHARACTERISTICS ..................................................... 22 3.2.3 TESTING PROCESS OF LOW RATED APPLICATIONS ..................................... 23 4 ANALYSIS ........................................................................................................................... 25 4.1 DATA ANALYSIS ....................................................................................................... 25 4.2 TOP RATED APPS ....................................................................................................... 25 4.2.1 IMPORTANCE BY TOP RATED DEVELOPERS ................................................. 25 4.2.2 TESTING BY TOP RATED DEVELOPERS .......................................................... 27 4.2.3 TESTING PROCESS OF TOP RATED APPS ....................................................... 28 4.3 LOW RATED APPS ..................................................................................................... 29 5 4.3.1 IMPORTANCE BY LOW RATED DEVELOPERS ................................................ 29 4.3.2 TESTING LEVEL BY LOW RATED DEVELOPERS ............................................. 30 4.3.3 TESTING PROCESS BY LOW RATED DEVELOPERS ........................................ 32 4.4 COMPARISON OF QUALITY CHARACTERISTICS IMPORTANCE, TESTING AND THEIR TESTING PROCESS .......................................................................................... 33 5 DISCUSSION ....................................................................................................................... 34 6 CONCLUSION .................................................................................................................... 36 6.1 VALIDITY .................................................................................................................... 36 6.2 FUTURE WORK........................................................................................................... 37 7 REFERENCES .................................................................................................................... 38 8 APPENDIX A: QUESTIONNAIRE................................................................................... 40 9 APPENDIX B: TOP RATED DEVELOPER’S INFORMATION: ................................ 44 10 APPENDIX C: LOW RATED DEVELOPER’S INFORMATION: .............................. 60 6 LIST OF FIGURES Figure 1: Comparison of important quality characteristics from top rated developer’s perspective………………………………………………………………………………………..26 Figure 2: Comparison of testing level of quality characteristics from top rated developer’s perspective. .................................................................................................................................... 28 Figure 3: Comparison of important quality characteristics from low rated developer’s perspective……………………………………………………………………………………… 30 Figure 4: Comparison of testing level of quality characteristics from low rated developer’s perspective. ...................................................................................................................................