BIO PRESENTATION T14
November 18, 2004 1:30 PM
AUTOMATED DATABASE TESTING WITH NUNIT
Alan Corwin Process Builder, Inc.
International Conference On Software Testing Analysis & Review November 15-19, 2004 Anaheim, CA USA Alan Corwin Alan Corwin has been a software developer, trainer, and course designer for nearly thirty years. He is president and founder of Process Builder, CIO of OptionBots, and teaches High Performance Data Integration in the University of Washington’s Advanced Web Development program.
Automated Database Testing with NUnit
Planned Test-Driven Development
1 Planned? z Isn’t test-driven development planned? Yes. Although planning is often decried as a waste of time by the XP community, simply writing your tests before you write your code is a form of planning. It’s the next level of planning that few XPers are willing to take – planning to do a series of standard tests for an entire application.
2 Test Goals z Test the Generic Database Requirements z Provide a Framework for More Complex Tests z Support Rapid Development (Daily Deliverables!) z Support Test-Driven Development z Allow Tests to be run both by developers and testers. z Test engineers and developers should be able to quickly and easily add test cases as needed. z Eliminate unnecessary communication cycles.
3 Test Design Features z Tests are fully automated. z Tests are black box (ignorant of the underlying data structure) z Tests are infinitely expandable. z Tests remain valid even when the implementation changes.
4 Tools Required z VisualStudio.NET (Enterprise Architect Edition) z NUnit z XML Spy (Optional)
5 The Test Process z Developer Builds the Test DataSet Classes z Developer Creates the Test Data File z The Test Data Files are placed in a common directory. z Developer constructs and executes the basic test classes. z Test Engineers develop test cases and add them to the test file. z Tests are continually added. z Each time the product is changed or a test case is added, all of the tests are rerun. z Developers run all tests when the code changes. z Test engineers run all tests when new test cases are added.
6 Test Data z Create the Test Data Dataset z Create the Test Data Data File
7 What is the Purpose of the test DataSet? z To hold the values necessary to robustly test the Create, Read, Update, and Delete functions. z To provide a strongly typed DataSet to increase reliability and reduce coding. z To simply the generation and initial population of the test data file.
8 Why do you need a DataSet? z First, you don’t “need” a DataSet. They are just a good idea. z DataSets reduce the amount of code required to construct the tests. z DataSets increase the readability of code. z DataSets eliminate many kinds of errors in the test code itself.
9 Create the Test DataSet z Each DataSet is based on an underlying table, view, or stored procedure. z Start the DataSet Wizard by adding a new DataSet to your test project. z Drag the underlying object to the designer. z Switch to the XML view. z Modify the primary key field. z Duplicate the data fields. z Add “Update” to the end of each new field name. z Save.
Demo Here! 10 What Just Happened? z The format for the test data file was defined. z An XSD (XML schema definition file) was just created. z A Strongly Typed DataSet class was generated (3000 or more lines of highly reliable code) – requires that Generate DataSet be turned on (it is by default).
11 Generate the Data File z Here’s where XML Spy comes in handy. z Open the Schema file you just created in XML Spy z Choose the ‘Generate Sample XML File’ command from the DTD/Schema menu. z When the dialog appears, select your options including the number of rows to generate, and click OK. z Save the file in your public TestData directory as an XML file. z Edit the data rows as needed.
Demo Here! 12 The Test Data File z XML Format Means z Editable with a wide variety of tools (including Notepad) z Built to a known standard supported by every software vendor z Easy to transfer to any other data format z Easy to read and understand. z .NET-ready z For proper testing, you need both a green bar data file and a red bar data file.
13 NUnit Basics z What is NUnit? z NUnit Limitations z Obtaining and Installing the Product z Constructing A Simple Test z Running Tests
14 What Is NUnit? z A Public Domain Test Harness for .NET z A Set of Classes to Facilitate Test Development and Execution z A Command Line Interface z A GUI to Run the Tests from Windows z A Tool For Developer-Centered Testing z Developed as JUnit, Ported to .NET, then Rewritten in C#
15 NUnit Limitations z You can’t test exception propagation easily. z Hard Code Tests Unlike user interfaces, database and business objects should throw z Red Bar Tests rather than handle exceptions. z Can’t debug when you run from the UI (This was a much longer list until version 2.1 which corrected most of the problems.)
16 Obtaining and Installing NUnit z http://www.nunit.org z Go to the Downloads Page z Run the Executable (That’s all there is to installing the program. Note, however that each project only knows about the test harness when it contains a reference to the NUnit libraries.)
17 Constructing a Simple Test z Adding the Harness Reference z Creating a Test Class z Adding Test Fixture Decorations z Write the Test Logic
18 Adding the Harness Reference z Right-click the References icon for the test project in the Solution Explorer and select Add Reference. z When the Open dialogue appears, navigate to the NUnit Installation directory. z Select all of the DLLs and click Open. (I’m not sure they are all needed, but the documentation does not specify which are needed.)
19 Creating a Test Class z Add a new class to the project. z Import the NUnit Framework z Imports NUnit.Framework (VB.NET) z using NUnit.Framework; (C#) z Add a TestFixture Decoration to the class declaration. z
20 Write the Test Logic
The test logic is often in a z Get the Test Data different class than the test execution. z For Each Test Case That is the approach that z Create the Record we use because it makes debugging both the test z Verify the Record and the application easier. z Update the Record z Verify the Record z Delete the Record z Verify the Deletion
21 Running Tests z From the GUI z Green Bar Tests z Red Bar Tests z From the Code
22 Process Change Results z As a result of adding NUnit and the basic database test process demonstrated here, we were able to shrink delivery times significantly while testing our database applications hundreds of times more thoroughly than we could before. z To our credit and embarrassment, our prototypes are now more robust and bug-free than our finished applications used to be. (And our finished applications were always more robust than the majority of what you see.)
23 Current Deliverables for One Programmer for One Four Hour Work Period z One DataSet Class z One Test Class for Each z One Command Helper Business Object Class Class z One Test Data File z Five Stored Procedures containing at least 3 test cases z One Business Object Class for Individual Objects z A Data Entry form for the individual object class. z One Business Object Class for the DataSet z A List form for the DataSet Class
24 Summary z NUnit is free, flexible, and easy to use. z XML test data files are easy to create and use. z NUnit supports a simple, standard approach to database testing that addresses the needs of both the test team and the development team. z The result of this approach is an automated test suite that creates significant gains in quality and productivity.
25 Technical Appendix z Test Logic z Set Tests z Additional Tests z Standardizing Test Classes z Resources z Automation of Test Automation
26 Test Logic: Get the Data z Three Details Make getting the Data Easy z Our test data is in an XML file. z We have a strongly typed DataSet class that matches that XML file. z Microsoft provides the ReadXML method. z VB Dim myTestData As New MyTestDataSetName() myTestData.ReadXML(fullPathOfTestFile) z C# MyTestDataSetName myTestData = New MyTestDataSetName(); myTestData.ReadXML(fullPathOfTestFile);
27 Test Logic: Get Each Test Case z A Strongly Typed DataSet consists of Strongly Typed DataRows. z VB For Each rowInstance _ As BackSpreadsParameterSetsTestData.SampleBusinessObjectsRow _ in _testSet.Tables[0].Rows _testRow = rowInstance ‘Assign to class variable for common use. Next rowInstance z C# These Row classes were generated when foreach (BackSpreadsParameterSetsTestData.SampleBusinessObjectsRowthe Test Data DataSet was created. rowInstance in _testSet.Tables[0].Rows) {_testRow = rowInstance;} // assign to a class variable for common use.
28 Test Logic: The CRUD Tests z The Logic of your CRUD tests will depend on the structure of your application, but the set of tests will always have to be done within the test data loop. z VB For Each rowInstance _ As BackSpreadsParameterSetsTestData.SampleBusinessObjectsRow _ in _testSet.Tables[0].Rows _testRow = rowInstance ‘Assign to class variable for common use. RunCrudTestSuite() Next rowInstance A better design would pass the test row to z C# the CRUD test routine.. foreach (BackSpreadsParameterSetsTestData.SampleBusinessObjectsRow rowInstance in _testSet.Tables[0].Rows) { _testRow = rowInstance; // assign to a class variable for common use. RunCrudTestSuite(); }
29 Our Basic CRUD Tests
private void RunCrudTestSuite() { int recordID = theNewID(); TestInitialValues(recordID); UpdateRecord(recordID); TestUpdateValues(recordID); The recordID ties the tests to the DeleteRecord(recordID); same database TestDeletion (recordID); object. }
30 theNewID() Note the Assertion. protected override int theNewID() This is an NUnit { SampleBusinessObject testObject = new SampleBusinessObject(); Assertion. testObject.Name = _testRow.name; testObject.OptionBotID = _testRow.optionBotID; testObject.Save(); Assert.AreEqual(testObject.Exceptions.Count, 0, "An object loading error occurred."); return testObject.ID; }
z Access to the database is allowed only through the business objects. z A new business object is created. z The properies are set from the test data row. z The business object is saved. z A check is made to see that no exceptions occurred. z The id of the new record is returned.
31 TestInitialValues() protected override void TestInitialValues(int idOfObject) { SampleBusinessObject testObject = new SampleBusinessObject(idOfObject); Assert.AreEqual(testObject.Name, _testRow.name, “Wrong name."); Assert.AreEqual(testObject.OptionBotID, _testRow.optionBotID, “Wrong OptionBot ID!"); Assert.AreEqual(testObject.Exceptions.Count, 0, "An object loading error occurred."); }
z Tests are very simple. z You put a value into the field. z You check to make sure that the same value was recorded and retrieved.
32 UpdateRecord() protected override void UpdateRecord(int idOfObject) { SampleBusinessObject testObject = new SampleBusinessObject(idOfObject); testObject.Name = _testRow.nameUpdate; testObject.OptionBotID = _testRow.optionBotIDUpdate; testObject.Save(); Assert.AreEqual(testObject.Exceptions.Count, 0, "An object loading error occurred."); return testObject.ID; }
z This is very similar to creating the record except that we start with an existing object. z Note that we are populating the business object with the update values from the test data row.
33 TestUpdateValues() protected override void TestUpdateValues(int idOfObject) { SampleBusinessObject testObject = new SampleBusinessObject(idOfObject); Assert.AreEqual(testObject.Exceptions.Count, 0, "An object loading error occurred."); Assert.AreEqual(testObject.Name, _testRow.nameUpdate, “Wrong name."); Assert.AreEqual(testObject.OptionBotID, _testRow.optionBotIDUpdate, “Wrong OptionBot ID!"); }
z Update test is almost identical to the initial values test. z Note that we are testing against the update values.
34 DeleteRecord()
protected override void DeleteRecord(int idOfObject) { SampleBusinessObject testObject = new SampleBusinessObject(idOfObject); testObject.Delete(); } z DeleteRecord is very simple. The logic for the deletion is in the business object so our test code simply calls the objects Delete Method.
35 TestDeletion() protected override void TestDeletion(int idOfObject) { SampleBusinessObject testObject = new SampleBusinessObject(idOfObject); Assert.AreEqual(testObject.Exceptions.Count, 0, "An object loading error occurred."); Assert.AreEqual(testObject.IsNew, true, “The object was not deleted."); }
z Update test is almost identical to the initial values test. z Note that we are testing against the update values.
36 The Test Decoration z Note that although we had to add a TestFixture decoration to the class declaration, none of the individual tests have had any additional decorations. These cannot be run from the NUnit GUI. That is intentional; the CRUD regression tests are designed to be run as a unit. z Add a test in your Test Fixture class to run the whole set and add the declaration as shown on the next screen.
37 Test Decoration Example
z The test decoration allows the NUnit GUI to recognize a method as a test.
In VB.NET, the correct decoration is ‘
[Test] public void TestBackSpreadParameterSets() { SampleObjectTest _thisTest = new SampleObjectTest(@"H:\TestData\OptionBot\SampleObjects.xml"); _thisTest.RunAllTests(); }
38 Test Initialization Example z NUnit allows you to specify a set of actions that are performed when the Test Fixture class is initialized.
39 The Basic Persistent Set Tests z Get the Set z Select Subsets z Count the Set
40 Additional Tests z Are Driven by Customer Requirements z Are added at the specification of the test engineer. z Are added by the developer. z Additional Test Example
41 Standardizing the Test Classes z Test Class Interfaces are used to standardize the language used for the test methods. z Abstract Test Classes capture the common elements of test classes. The latest versions of our test classes are currently available at http://www.ProcessBuilder.com for the following languages z VB.NET z C#
42 NUnit Resources z NUnit Home Page, http://www.nunit.org z NUnit Resources http://www.testdriven.com/modules/news/
43 Automating Test Automation z Note that this presentation is about how to automate the execution of tests, not the creation of those tests. The creation of those tests is clearly automatable. z Extracting MetaData z Code Generation Alternatives
44