Effectively Integrating Unit Testing and Code Coverage Into Continuous Integration
Total Page:16
File Type:pdf, Size:1020Kb
M ASARYKOVAUNIVERZITA FAKULTAINFORMATIKY}w¡¢£¤¥¦§¨ !"#$%&'()+,-./012345<yA| Effectively Integrating Unit Testing and Code Coverage into Continuous Integration D IPLOMATHESIS Tom´aˇsSvrˇcek Brno, spring 2015 Declaration Hereby I declare, that this paper is my original authorial work, which I have worked out by my own. All sources, references and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source. Advisor: Ing. RNDr. Barbora Buhnov¨ a,´ Ph.D. ii Acknowledgement I would like to thank my supervisor, Barbora Buhnov¨ a,´ for all valuable ad- vice during the writing of this master thesis. I would like to thank Ronny Kolb as well for the patience, guidance and feedback during the whole process of implementation and writing part. I cannot forget to thank all my colleagues in ACT team for the great cooperation and willingness to help. Without these people the work would not be completed. iii Abstract These days, automated testing is a fundamental part of the software devel- opment process. It helps the developers to avoid the basic mistakes imme- diately and therefore it saves the development time. Thanks to the rapid errors discovering, the quality of the developed product is increasing. This master thesis investigates how to effectively integrate Unit Testing and Code Coverage to the Continuous Integration process. The idea is ana- lyzed in the State of the Art chapter and the current situation is inspected in the State of the Practice in Honeywell chapter. Based on the results of the analysis, the plugin for the Atlassian Bamboo server is designed and implemented. Evaluation of the developed product is done and its improvements are proposed for the future work. iv Keywords Atlassian, Bamboo, Code Coverage, Continuous Integration, Unit Testing, xUnit v Contents 1 Introduction ...............................3 1.1 Context ...............................3 1.2 Goals ................................4 1.3 Thesis Structure ..........................4 2 State of the Art .............................6 2.1 Software Development Process .................6 2.1.1 Rational Unified Process . .7 2.1.2 Other Approaches . .8 2.2 Capability Maturity Model Integration .............9 2.3 Continuous Integration ...................... 10 2.3.1 Continuous Integration Servers . 11 2.3.2 Summary . 12 2.4 Automated Software Testing ................... 12 2.4.1 Reasons and Strategies for Test Automation . 14 2.4.2 Unit Testing . 15 2.4.3 xUnit Framework Testing . 16 2.5 Code Coverage .......................... 17 2.6 Unit Testing in Continuous Integration ............. 19 2.7 Summary .............................. 19 3 State of the Practice in Honeywell .................. 20 3.1 ACS Software Development Process .............. 20 3.2 Software Development Process in ACT ............. 22 3.3 Continuous Integration in Honeywell ............. 22 3.4 Bamboo ............................... 23 3.5 Unit Testing ............................ 24 3.6 Summary .............................. 25 4 Feasibility Study ............................ 26 4.1 Overview .............................. 26 4.2 Bamboo Architecture ....................... 26 4.2.1 Project Structure . 27 4.2.2 Agents and Server . 27 4.2.3 Capabilities and Requirements . 29 1 4.3 Bamboo Extensibility ....................... 30 4.4 xUnit Frameworks ......................... 31 4.5 Code Coverage Tools ....................... 33 4.6 Tools Integration ......................... 34 4.7 Summary .............................. 35 5 Implementation ............................. 36 5.1 Architecture Design ........................ 36 5.2 Bamboo Plugin Composition .................. 37 5.3 Used Technologies ........................ 37 5.3.1 Java . 37 5.3.2 JavaScript . 37 5.3.3 XML, XSD and XSLT . 38 5.3.4 Active Objects . 38 5.3.5 FreeMarker Template Language . 38 5.4 Data Flow ............................. 39 5.5 Analysis of Potential Problems revealed during the Imple- mentation .............................. 41 5.6 Presentation Layer ......................... 41 5.7 Summary .............................. 43 6 Evaluation ................................ 45 6.1 First Round ............................ 46 6.1.1 Evaluation Method . 46 6.1.2 Scenarios . 46 6.1.3 Questionnaire Results . 47 6.1.4 Proposed Improvements . 48 6.2 Second Round ........................... 49 6.2.1 Evaluation Method . 49 6.3 Summary .............................. 50 7 Conclusion ................................. 51 7.1 Future Work ............................. 51 7.2 Obtained Benefits ......................... 52 7.3 Summary .............................. 53 Appendices . 57 A Terms and Abbreviations ....................... 58 B Package Diagram ............................ 59 C Installation Instructions ........................ 60 2 Chapter 1 Introduction 1.1 Context Honeywell is an international company with headquarters in the USA. It employs over 132,000 people all over the world. The company develops and manufactures a wide range of products from systems for residential homes and commercial buildings such as thermostats and security devices, through chemicals and materials up to turbochargers and equipment for airplanes. The company is divided into four strategic business groups: • Aerospace • Automation and Control Solutions (ACS) • Performance Materials and Technologies (PMT) • Transportation Systems (TS) This master thesis has been designed as improvement of the current state for the ACS Center of Technologies (ACT) team. ACT was founded to drive soft- ware excellence such as development of Best Practices, training, mentoring and improvement of the software development tools used in Automation and Control Solutions (ACS). For the purposes of software improvement, Honeywell ACS follows ACS Software Development Process (ASDP) which is compliant with Capability Maturity Model Integration (CMMI) as de- scribed in Section 3.1. 3 1 . I NTRODUCTION 1.2 Goals Bamboo from Atlassian is a continuous integration, deployment, and release management solution that is used by Honeywell ACS. The latest version of the Bamboo Continuous Integration (CI) server offers only a few plugins for xUnit Testing and does not provide any Code Coverage functionality by default. Honeywell needs better support of mentioned tools and therefore this master thesis has been designed. The objective is to adjust Bamboo functionality for Honeywell users by implementing the plugin for this tool. The integration xUnit framework family and Code Coverage analysis into Continuous Integration (CI) process is chosen to fulfill the assignment. The goal of this master thesis is to improve part of the development process and save the development cycle time. The result should be saving develop- ers‘ time during their activities. Based on the information from the State of the Art and also from the State of the Practice in Honeywell studies the exten- sion for this Continuous Integration server is developed. 1.3 Thesis Structure This master thesis is further divided into the following six chapters. Chapter 2 State of the Art provides an overview of Unit Testing automation and Code Coverage as important parts of software development process. It also describes Continuous Integration process, tools and their usability. Chapter 3 State of the Practice in Honeywell describes how Unit Testing, Code Coverage and Continuous Integration are used in Honeywell ACS (Appendix A). Moreover, it discusses possibilities for improvement. Chapter 4 Feasibility Study discusses the possibility of the Unit Testing tools integration into the Bamboo CI (Appendix A) server. It describes Bam- boo architecture, its extensibility and possible integration with testing and Code Coverage tools. Chapter 5 Implementation describes the implementation process. It con- tains a section about the architecture, code sharing and used technologies. The end of this chapter outlines the potential problems revealed during the implementation. 4 1 . I NTRODUCTION Chapter 6 Evaluation describes the methodology of thesis evaluation and summarizes gathered results. Measurements and findings are also provided. Chapter 7 Conclusion summarizes the results of this master thesis and its fulfillment. It also contains suggestions for the future improvements of the extension. 5 Chapter 2 State of the Art This chapter describes Continuous Integration and its importance in the software development process. It describes Unit Testing, Code Coverage and other parts of the Continuous Integration process. The chapter also mentions Automated Software Testing. 2.1 Software Development Process Software Development Process (SDP) also known as a Software Develop- ment Life-Cycle (SDLC) [1] is division of software development work into distinct phases. In other words it is a set of steps that a developed software program goes through. First mentioned in 1960, SDLC was the first frame- work for building information systems. This framework is meant to develop software in a structured and methodical way from the first birth of an idea to the deployment and support. There are more approaches to define the SDP (Appendix A). Figure 2.1 describes one of the possible approaches [2]. Figure 2.1: Phases of Software Development Process • Plan represents planning of the whole project and obtaining users‘ requirements. 6 2 . S TATEOFTHE A RT • Analysis of the problem and design suggestions are the second step of the SDP. The output is usually expressed in the graphical notation such as the Unified Modeling