BIO PRESENTATION T14 November 18, 2004 1:30 PM AUTOMATED DATABASE TESTING WITH NUNIT Alan Corwin Process Builder, Inc. International Conference On Software Testing Analysis & Review November 15-19, 2004 Anaheim, CA USA Alan Corwin Alan Corwin has been a software developer, trainer, and course designer for nearly thirty years. He is president and founder of Process Builder, CIO of OptionBots, and teaches High Performance Data Integration in the University of Washington’s Advanced Web Development program. Automated Database Testing with NUnit Planned Test-Driven Development 1 Planned? z Isn’t test-driven development planned? Yes. Although planning is often decried as a waste of time by the XP community, simply writing your tests before you write your code is a form of planning. It’s the next level of planning that few XPers are willing to take – planning to do a series of standard tests for an entire application. 2 Test Goals z Test the Generic Database Requirements z Provide a Framework for More Complex Tests z Support Rapid Development (Daily Deliverables!) z Support Test-Driven Development z Allow Tests to be run both by developers and testers. z Test engineers and developers should be able to quickly and easily add test cases as needed. z Eliminate unnecessary communication cycles. 3 Test Design Features z Tests are fully automated. z Tests are black box (ignorant of the underlying data structure) z Tests are infinitely expandable. z Tests remain valid even when the implementation changes. 4 Tools Required z VisualStudio.NET (Enterprise Architect Edition) z NUnit z XML Spy (Optional) 5 The Test Process z Developer Builds the Test DataSet Classes z Developer Creates the Test Data File z The Test Data Files are placed in a common directory. z Developer constructs and executes the basic test classes. z Test Engineers develop test cases and add them to the test file. z Tests are continually added. z Each time the product is changed or a test case is added, all of the tests are rerun. z Developers run all tests when the code changes. z Test engineers run all tests when new test cases are added. 6 Test Data z Create the Test Data Dataset z Create the Test Data Data File 7 What is the Purpose of the test DataSet? z To hold the values necessary to robustly test the Create, Read, Update, and Delete functions. z To provide a strongly typed DataSet to increase reliability and reduce coding. z To simply the generation and initial population of the test data file. 8 Why do you need a DataSet? z First, you don’t “need” a DataSet. They are just a good idea. z DataSets reduce the amount of code required to construct the tests. z DataSets increase the readability of code. z DataSets eliminate many kinds of errors in the test code itself. 9 Create the Test DataSet z Each DataSet is based on an underlying table, view, or stored procedure. z Start the DataSet Wizard by adding a new DataSet to your test project. z Drag the underlying object to the designer. z Switch to the XML view. z Modify the primary key field. z Duplicate the data fields. z Add “Update” to the end of each new field name. z Save. Demo Here! 10 What Just Happened? z The format for the test data file was defined. z An XSD (XML schema definition file) was just created. z A Strongly Typed DataSet class was generated (3000 or more lines of highly reliable code) – requires that Generate DataSet be turned on (it is by default). 11 Generate the Data File z Here’s where XML Spy comes in handy. z Open the Schema file you just created in XML Spy z Choose the ‘Generate Sample XML File’ command from the DTD/Schema menu. z When the dialog appears, select your options including the number of rows to generate, and click OK. z Save the file in your public TestData directory as an XML file. z Edit the data rows as needed. Demo Here! 12 The Test Data File z XML Format Means z Editable with a wide variety of tools (including Notepad) z Built to a known standard supported by every software vendor z Easy to transfer to any other data format z Easy to read and understand. z .NET-ready z For proper testing, you need both a green bar data file and a red bar data file. 13 NUnit Basics z What is NUnit? z NUnit Limitations z Obtaining and Installing the Product z Constructing A Simple Test z Running Tests 14 What Is NUnit? z A Public Domain Test Harness for .NET z A Set of Classes to Facilitate Test Development and Execution z A Command Line Interface z A GUI to Run the Tests from Windows z A Tool For Developer-Centered Testing z Developed as JUnit, Ported to .NET, then Rewritten in C# 15 NUnit Limitations z You can’t test exception propagation easily. z Hard Code Tests Unlike user interfaces, database and business objects should throw z Red Bar Tests rather than handle exceptions. z Can’t debug when you run from the UI (This was a much longer list until version 2.1 which corrected most of the problems.) 16 Obtaining and Installing NUnit z http://www.nunit.org z Go to the Downloads Page z Run the Executable (That’s all there is to installing the program. Note, however that each project only knows about the test harness when it contains a reference to the NUnit libraries.) 17 Constructing a Simple Test z Adding the Harness Reference z Creating a Test Class z Adding Test Fixture Decorations z Write the Test Logic 18 Adding the Harness Reference z Right-click the References icon for the test project in the Solution Explorer and select Add Reference. z When the Open dialogue appears, navigate to the NUnit Installation directory. z Select all of the DLLs and click Open. (I’m not sure they are all needed, but the documentation does not specify which are needed.) 19 Creating a Test Class z Add a new class to the project. z Import the NUnit Framework z Imports NUnit.Framework (VB.NET) z using NUnit.Framework; (C#) z Add a TestFixture Decoration to the class declaration. z <TestFixture()> (VB -- must be on the same line as the class declaration) z [TestFixture] (C# -- Must be on the declaration line or the line before) 20 Write the Test Logic The test logic is often in a z Get the Test Data different class than the test execution. z For Each Test Case That is the approach that z Create the Record we use because it makes debugging both the test z Verify the Record and the application easier. z Update the Record z Verify the Record z Delete the Record z Verify the Deletion 21 Running Tests z From the GUI z Green Bar Tests z Red Bar Tests z From the Code 22 Process Change Results z As a result of adding NUnit and the basic database test process demonstrated here, we were able to shrink delivery times significantly while testing our database applications hundreds of times more thoroughly than we could before. z To our credit and embarrassment, our prototypes are now more robust and bug-free than our finished applications used to be. (And our finished applications were always more robust than the majority of what you see.) 23 Current Deliverables for One Programmer for One Four Hour Work Period z One DataSet Class z One Test Class for Each z One Command Helper Business Object Class Class z One Test Data File z Five Stored Procedures containing at least 3 test cases z One Business Object Class for Individual Objects z A Data Entry form for the individual object class. z One Business Object Class for the DataSet z A List form for the DataSet Class 24 Summary z NUnit is free, flexible, and easy to use. z XML test data files are easy to create and use. z NUnit supports a simple, standard approach to database testing that addresses the needs of both the test team and the development team. z The result of this approach is an automated test suite that creates significant gains in quality and productivity. 25 Technical Appendix z Test Logic z Set Tests z Additional Tests z Standardizing Test Classes z Resources z Automation of Test Automation 26 Test Logic: Get the Data z Three Details Make getting the Data Easy z Our test data is in an XML file. z We have a strongly typed DataSet class that matches that XML file. z Microsoft provides the ReadXML method. z VB Dim myTestData As New MyTestDataSetName() myTestData.ReadXML(fullPathOfTestFile) z C# MyTestDataSetName myTestData = New MyTestDataSetName(); myTestData.ReadXML(fullPathOfTestFile); 27 Test Logic: Get Each Test Case z A Strongly Typed DataSet consists of Strongly Typed DataRows. z VB For Each rowInstance _ As BackSpreadsParameterSetsTestData.SampleBusinessObjectsRow _ in _testSet.Tables[0].Rows _testRow = rowInstance ‘Assign to class variable for common use. Next rowInstance z C# These Row classes were generated when foreach (BackSpreadsParameterSetsTestData.SampleBusinessObjectsRowthe Test Data DataSet was created. rowInstance in _testSet.Tables[0].Rows) {_testRow = rowInstance;} // assign to a class variable for common use. 28 Test Logic: The CRUD Tests z The Logic of your CRUD tests will depend on the structure of your application, but the set of tests will always have to be done within the test data loop. z VB For Each rowInstance _ As BackSpreadsParameterSetsTestData.SampleBusinessObjectsRow _ in _testSet.Tables[0].Rows _testRow = rowInstance ‘Assign to class variable for common use.