Introducing Automated Unit Testing in a Legacy PHP System
Total Page：16
File Type：pdf, Size：1020Kb
Introducing Automated Unit Testing in a Legacy PHP System Alexander Olsson February 22, 2010 Abstract Introducing unit testing in a previously untested system is problematic. De- velopers may lack experience from extensive unit testing, and many work hours is likely needed before the system has high test coverage. It is therefore important to early on choose a framework and a strategy for implementing tests which suits both the developers and the system. The framework is selected from a survey based on requirements specified by the company and an initial guess of important testing criteria. The survey is complemented with test implementations and interviews to capture group- specific viewpoints. A strategy for implementing tests is decided using a code analysis tool, which estimates implementation effort and testing need on a per-unit basis. A test implementation order is suggested by ranking units using a diagram-based method. This approach allows for fitting the strategy to the testing skills of the team. In this application, the most suitable framework from a selection of frame- works is PHPUnit. The framework and strategy is experimentally verified on a PHP web system with over 40k SLOC and a development team of 8 people. As the team is fairly inexperienced with unit testing, a test implementation strategy which prioritize low-effort units was chosen. Furthermore, a small li- brary of unit tests is implemented and serves as templates for system-specific tests and strategy validation. i ii Acknowledgement Iwouldliketothankthemanypeoplewhohelpedmeduringmythesis.First and foremost, Richard Kronf¨alt, my supervisor at Axis Communications who has come up with countless of ideas and suggestions. Secondly, Mathias Haage, my supervisor from Lund Institute of Technol- ogy has helped me with anything concerning my thesis and answered all my questions. Thirdly, the people at Axis Communications, and especially the ones at VHS. In particular Hans M˚ansson, who have come with many good insights of how to further improve what I’ve been doing. Also from VHS, Michael Rengbrandt helped particularly with tests needed for the thesis to be complete. iii iv Contents 1 Introduction 1 1.1 Background ............................ 1 1.2 Problem description ........................ 2 1.3 Problem approach ......................... 2 2 Theory 3 2.1 Software Testing ......................... 3 2.1.1 Unit Testing ........................ 4 2.1.2 Integration Testing .................... 7 2.1.3 System Testing ...................... 7 2.2 Frameworks ............................ 8 2.2.1 The xUnit Family of Frameworks ............ 8 2.2.2 FIT ............................ 8 3Framework 11 3.1 Selection Criteria ......................... 11 3.1.1 Quantification of Selection Criteria ........... 12 3.2 Benchmark test suite ....................... 12 3.3 Nominated frameworks ...................... 15 3.4 Framework Case Study ...................... 16 3.4.1 Overview ......................... 16 3.4.2 Survey Interpretation ................... 17 3.5 Framework evaluation ...................... 18 3.5.1 PHPUnit ......................... 18 3.5.2 SimpleTest ......................... 20 3.5.3 Testilence ......................... 22 3.5.4 SnapTest ......................... 24 3.5.5 PHPSpec ......................... 27 3.5.6 PHPT ........................... 29 3.6 Conclusion ............................. 31 v 4Integration 33 4.1 Where to start .......................... 33 4.1.1 An example ........................ 33 4.1.2 Approach ......................... 34 4.1.3 Summary ......................... 36 4.2 Making the analysis ........................ 36 4.3 Creating an analysis tool ..................... 37 4.3.1 PHPA: List of units .................... 37 4.3.2 PHPA: Cyclomatic complexity .............. 38 4.3.3 PHPA: Dependency .................... 41 4.3.4 PHPA: Frequency ..................... 42 4.3.5 Analysis Results ..................... 43 4.4 Choosing the entry point ..................... 46 4.4.1 Creating the ranked list ................. 47 4.5 Writing tests ........................... 49 4.5.1 Implementation ...................... 50 4.6 Putting it all together ...................... 53 4.7 Validation ............................. 54 5 Conclusion 57 5.1 Discussion ............................. 58 6 Summary 63 A Source Code 65 A.1 Complex .............................. 65 A.1.1 Complex class ....................... 65 A.1.2 Filehandler ........................ 70 A.1.3 Config ........................... 71 A.1.4 Complex slim ....................... 72 B Case study 75 B.1 Introduction ............................ 75 B.2 Task ................................ 76 B.2.1 Purpose .......................... 76 B.2.2 Task ............................ 77 B.3 Survey ............................... 86 Bibliography 86 vi Chapter 1 Introduction 1.1 Background Automated software unit testing may be performed in several different ways. It is often difficult to proclaim one school as superior. Coding standards, languages, requirements and personal taste will influence the choices made when introducing automated testing. Automated testing is a quick and easy way to swiftly verify the func- tionality of a given system. It can be used after implementing changes to the system to verify that parts not touched by the new implementation still works in the same manner. Creating the test cases itself is also a way to force the developer to find a solution to the problem at hand before implementing, resulting in better code[16]. Unfortunately, testing is too often cut from a project to save time and money. It may be argued that enough testing is performed by the developers as they write the source code. This is, however, prone to result in faulty code. Implementing new features may break some previously working code and without an automated test suite there is no way to verify this, aside from checking manually. Bugs will be discovered later in the project and will both delay the time and increase the costs of the project. In many cases the delay and increased cost is a substantial amount of the project’s initial values. One mean to overcome these issues is to introduce automated testing. Different approaches to accomplish this in a legacy system will be discussed and evaluated in this thesis. 1 1.2. PROBLEM DESCRIPTION CHAPTER 1. INTRODUCTION 1.2 Problem description In a system which have been existing for several years a need for systematic testing has risen. As the system have increased not only in complexity but also size it has become difficult to manually verify and validate the system. Manual testing has also proved to be time consuming and mistakes are more commonly made. 1.3 Problem approach There are mainly two tasks which will be carried out in this thesis. First, aframeworkforunittestingisneeded.Thebestone,accordingtoselected criteria, for this specific application will be chosen. Second, the framework should be integrated into the existing system. In this phase, how to imple- ment test cases and overcome difficulties when doing so will be discussed. Finding a suitable framework poses several difficulties. Some questions we need to answer before we can make a choice is; What do the current legacy system require from the framework in order • for it to be applicable to the system? What functionality would improve the quality of the resulting tests? • What do the developers need, and want? • Answering these questions is not trivial. No questions holds an objective answer, but rather depends on humans and personal taste. Different people may also have different opinions on, for example, what functionality is neces- sary. In order to get a fair understanding of the answers to these questions a case study with two developers of the system were performed and is further discussed in section 3.4.Furtherdiscussionoftheanswerstothesequestions is the criteria for selecting a framework, covered by section 3.1. Once a fitting framework has been found, a way to integrate the frame- work along with methods for implementing unit test cases and best practices is needed. As there are many test cases which needs to be implemented, the test cases for a unit will be implemented according to priority. The priority of a unit will be decided by examining the effort for bringing the unit under test and comparing it to the value the unit provides from being under test. The effort is measured in terms of cyclomatic complexity and dependencies. The value is measured in number of invocations during a certain timespan. The results are illustrated in a diagram where each of the two quantities (ef- fort and value) will be on each axis of the diagram. This is further discussed in section 4.3.5. 2 Chapter 2 Theory 2.1 Software Testing In software testing, there are usually three parts of the hierarchy: Unit test- ing, integration testing and system testing. Unit testing is performed by the developer to ensure his newly developed unit is working as intended. This is further described in section 2.1.1.Integrationtestingisperformedtoas- sure that all current units, including the newly developed ones, are working together as a whole, known as a module. This is further described in sec- tion 2.1.2.Finally,asystemtestiscarriedouttoensuretheend-to-end system is working satisfactory. This is further described in section 2.1.3. The procedure of testing is illustrated in figure 2.1.[19] Figure 2.1: Testing methodology in a typical software development situation. 3 2.1. SOFTWARE TESTING CHAPTER 2. THEORY Sometimes, integration and system