The Development and Evaluation of a Unit Testing Methodology
Total Page：16
File Type：pdf, Size：1020Kb
Computer Science Stefan Lindberg and Fredrik Strandberg The Development and Evaluation of a Unit Testing Methodology Master’s thesis 2006:3 The Development and Evaluation of a Unit Testing Methodology Stefan Lindberg and Fredrik Strandberg ii This report is submitted in partial fulfillment of the requirements for the Master of Science degree in Information technology. All material in this report which is not my own work has been identified and no material is included for which a degree has previously been conferred. Stefan Lindberg Fredrik Strandberg Approved, 2006-03-07 Opponent: Per Hurtig Advisor: Donald Ross Examiner: Tim Heyer iii iv Abstract Westinghouse Fuel Manufacturing in Västerås, Sweden, manufactures fuel rods for nuclear plants. Manufacturing-IT is a software development section at Westinghouse Fuel Manufacturing. This thesis involves the development of a unit testing methodology (UTM) for the Manufacturing-IT section, which currently does not follow a well-defined software test process. By evaluating different unit testing best practices and UTM design issues collected from literature, articles, papers and the Internet, a UTM document was developed. The UTM document was developed according to requirements from Manufacturing-IT and as an extension to existing documents within the Westinghouse organization. The UTM was evaluated by applying the methodology in a case study. A single unit within a production control system in the rod manufacturing workshop at the Westinghouse fuel factory in Västerås was tested. Asides from evaluating the UTM, the case study was intended to find software tools that could simplify the unit testing process, and to test the production control system unit thoroughly. The 182 test cases designed and implemented revealed 28 faults in the tested unit. NUnit was chosen to be the tool for automated unit testing in the UTM. The results from the case study indicate that the methods and other unit testing process related activities included in the UTM document developed are applicable to unit testing. However, adjustments and further evaluation will be needed in order to enhance the UTM. The UTM developed in this thesis is a first step towards a structured testing process for the Manufacturing-IT section and the UTM document will be used at the Manufacturing-IT section. By using the methods and other unit testing process related activities in the UTM developed in this thesis, any company or individual with similar requirements for a UTM as Manufacturing-IT, and that performs unit testing in an unstructured way, may benefit in that a more structured unit testing process is achieved. v vi Contents 1 Introduction...................................................................................................................1 1.1 What is a unit testing methodology?.......................................................................1 1.2 Thesis summary.....................................................................................................1 1.3 Reading guidelines.................................................................................................2 2 Background...................................................................................................................5 2.1 Introduction............................................................................................................5 2.2 Software engineering..............................................................................................6 2.2.1 What is software engineering?.............................................................................................6 2.2.2 Principles of software engineering.......................................................................................7 2.3 Verification and validation.....................................................................................8 2.4 Software Testing....................................................................................................9 2.5 Unit testing...........................................................................................................11 2.6 Unit testing at ManIT...........................................................................................11 2.6.1 Existing standards and documents.....................................................................................12 2.7 Summary..............................................................................................................15 3 Unit testing methodology development.......................................................................17 3.1 Introduction..........................................................................................................17 3.2 Methodology design issues...................................................................................20 3.3 ManIT requirements.............................................................................................23 3.4 Unit testing best practices.....................................................................................25 3.4.1 Definition of a unit............................................................................................................25 3.4.2 When to design and execute unit test cases........................................................................26 3.4.3 Unit testing roles...............................................................................................................27 3.4.4 Traditional unit testing methods.........................................................................................29 3.4.5 Formal specification and verification.................................................................................29 3.4.6 Black-box unit testing.......................................................................................................30 3.4.7 Equivalence partitioning and boundary value analysis........................................................30 3.4.8 The all-pairs method.........................................................................................................32 3.4.9 White-box unit testing.......................................................................................................34 3.4.10 Path testing.......................................................................................................................34 3.4.11 Data flow testing...............................................................................................................36 3.4.12 Multiple condition testing..................................................................................................37 3.4.13 Loop testing......................................................................................................................37 3.4.14 Testing exceptions............................................................................................................38 3.4.15 Unit test methods for object-oriented code.........................................................................39 vii 3.4.16 State-based unit testing......................................................................................................39 3.4.17 Selecting test cases for state-based unit testing...................................................................40 3.4.18 Inheritance........................................................................................................................42 3.4.19 Polymorphism...................................................................................................................46 3.4.20 Information hiding............................................................................................................46 3.4.21 Order of testing within a unit.............................................................................................48 3.4.22 Documentation of unit test cases........................................................................................48 3.4.23 Unit test drivers.................................................................................................................49 3.4.24 Stubs.................................................................................................................................50 3.4.25 Prioritizing unit tests.........................................................................................................52 3.4.26 Criteria for ending unit testing...........................................................................................52 3.4.27 Redundant test cases.........................................................................................................57 3.4.28 Reporting unit test results..................................................................................................57 3.5 Summary..............................................................................................................61 4 Case study....................................................................................................................63 4.1 Introduction..........................................................................................................63 4.2 Case study expectations........................................................................................64 4.3 Case study setup...................................................................................................64 4.3.1 Tool research....................................................................................................................64