Thesis no: MSSE-2014-06 Automated Debugging and Bug Fixing Solutions: A Systematic Literature Review and Classification Hafiz Adnan Shafiq Zaki Arshad Faculty of Computing Blekinge Institute of Technology SE-371 79 Karlskrona Sweden This thesis is submitted to the School of Engineering at Blekinge Institute of Technology in partial fulfillment of the requirements for the degree of Master of Science in Software Engineering. The thesis is equivalent to 20 weeks of full time studies. Contact Information: Author(s): Hafiz Adnan Shafiq E-mail: [email protected] Zaki Arshad E-mail: [email protected] University advisor: Dr. Wasif Afzal School of Computing, BTH School of Computing Internet : www.bth.se Blekinge Institute of Technology Phone : +46 455 38 50 00 SE-371 79 Karlskrona Fax : +46 455 38 50 57 Sweden ABSTRACT Context: Bug fixing is the process of ensuring correct source code and is done by developer. Automated debugging and bug fixing solutions minimize human intervention and hence minimize the chance of producing new bugs in the corrected program. Scope and Objectives: In this study we performed a detailed systematic literature review. The scope of work is to identify all those solutions that correct software automatically or semi-automatically. Solutions for automatic correction of software do not need human intervention while semi-automatic solutions facilitate a developer in fixing a bug. We aim to gather all such solutions to fix bugs in design, i.e., code, UML design, algorithms and software architecture. Automated detection, isolation and localization of bug are not in our scope. Moreover, we are only concerned with software bugs and excluding hardware and networking domains. Methods: A detailed systematic literature review (SLR) has been performed. A number of bibliographic sources are searched, including Inspec, IEEE Xplore, ACM digital library, Scopus, Springer Link and Google Scholar. Inclusion/exclusion, study quality assessment, data extraction and synthesis have been performed in depth according to guidelines provided for performing SLR. Grounded theory is used to analyze literature data. To check agreement level between two researchers, Kappa analysis is used. Results: Through SLR we identified 46 techniques. These techniques are classified in automated/semi- automated debugging and bug fixing. Strengths and weaknesses of each of them are identified, along with which types of bugs each can fix and in which language they can be implement. In the end, classification is performed which generate a list of approaches, techniques, tools, frameworks, methods and systems. Along, this classification and categorization we separated bug fixing and debugging on the bases of search algorithms. Conclusion: In conclusion achieved results are all automated/semi-automated debugging and bug fixing solutions that are available in literature. The strengths/benefits and weaknesses/limitations of these solutions are identified. We also recognize type of bugs that can be fixed using these solutions. And those programming languages in which these solutions can be implemented are discovered as well. In the end a detail classification is performed. Key Words: Automated and semi-automated, bug fixing techniques, debugging techniques, solutions, strengths and weaknesses, search algorithms, search based software engineering, systematic literature Review, classification 3 Table of Content Abstract ......................................................................................................................................................... 3 List of Figures ................................................................................................................................................ 9 List of Tables ............................................................................................................................................... 10 1 INTRODUCTION ................................................................................................................................... 11 1.1 Background ................................................................................................................................. 11 1.2 Problem Definition ...................................................................................................................... 12 1.3 Aims and objectives .................................................................................................................... 12 1.4 Research Questions .................................................................................................................... 13 1.5 Related Work .............................................................................................................................. 13 1.6 Thesis Structure .......................................................................................................................... 14 2 RESEARCH METHODOLOGY ................................................................................................................ 15 2.1 Research Design .......................................................................................................................... 15 2.2 Exploratory Study ........................................................................................................................ 15 2.3 Systematic Literature Reviews .................................................................................................... 16 2.3.1 Planning the review ............................................................................................................ 16 2.3.2 Conducting the review ........................................................................................................ 16 2.3.3 Reporting the review .......................................................................................................... 16 2.4 Grounded Theory ........................................................................................................................ 16 3 SYSTEMATIC LITERATURE REVIEW ...................................................................................................... 17 3.1 Planning the review .................................................................................................................... 17 3.1.1 Purpose of Systematic Review ............................................................................................ 17 3.1.2 Development of Review Protocol ....................................................................................... 17 3.1.3 Search Strategy ................................................................................................................... 17 3.1.4 Keywords and Search String................................................................................................ 18 3.1.5 Search Databases ................................................................................................................ 18 3.1.6 Study Selection Criteria (Inclusion and Exclusion Criteria) ................................................. 19 3.1.6.1 Inclusion and Exclusion Criteria .......................................................................................... 19 3.1.7 Study Selection Procedure .................................................................................................. 20 3.1.8 Data Extraction Strategy ..................................................................................................... 20 4 3.1.9 Data Synthesis ..................................................................................................................... 20 3.1.10 Validation of Review Protocol ............................................................................................. 20 3.1.11 Quality Assessment Criteria ................................................................................................ 21 3.1.12 Pilot Study ........................................................................................................................... 21 3.2 Conducting the Review ............................................................................................................... 21 3.2.1 Identification of Research ................................................................................................... 21 3.2.2 Primary Study Selection ...................................................................................................... 22 3.2.3 Data Extraction Strategy ..................................................................................................... 24 3.2.4 Data Analysis ....................................................................................................................... 24 3.2.4.1 Open Coding ........................................................................................................................ 24 3.2.4.2 Axial Coding ......................................................................................................................... 24 3.2.4.3 Selective Coding .................................................................................................................. 24 3.2.5 Study Quality Assessment ................................................................................................... 25 3.2.5.1 Data Extraction .................................................................................................................... 25 3.2.5.2 Primary Study Selection .....................................................................................................