Build Automation

• Supports frequent integration • Supports regression testing • Supports frequent customer releases • Saves time from the day you do it – so why not day 1? • Makes visible what was hidden

March 2005 Atomic Object LLC Challenges

• Consistent environment across – computers – developers • Platform dependencies – shell script vs batch file – Ant, Nant • Databases – creation, population

March 2005 Atomic Object LLC Atomic Standards

• Environment script in repository – shell or batch • Standardized project directory structure • Self-contained projects, versus – c:\windows\system32 – /usr/local/lib • Ant build files • Cygwin • Integration with source control

March 2005 Atomic Object LLC Example Ant Build

March 2005 Atomic Object LLC Test Driven Development

Unit testing… is done by developers, as a form of specification, while they write source code, to know when they are done, to document what they have done, to extend and maintain code fearlessly.

March 2005 Atomic Object LLC Unit & Integration Testing

• Unit – single method or function – easy to see test cases – easy to automate • Integration – multiple methods within or between classes – subsystems – less obvious structural guidance for testing – composition complicates testing

March 2005 Atomic Object LLC System Testing

• Limited to events visible at the boundaries – what a user can do – correspond to features, use cases • Harder to automate – GUI problem – Isolation of subsystems • Test design – brittle tests – redundancy

March 2005 Atomic Object LLC Test Automation

• Supports regression testing – run all the tests every time – absolutely necessary to automate • Lets anyone work on any code – moves naturally into maintenance • Test suites and xUnit framework – Composite pattern of tests – Setup, execution, tear down, results

March 2005 Atomic Object LLC JUnit main loop

suite(); suiteSetUp(); for (each test method X) { testObj = new TestClass(); testObj.setUp(); testObj.X(); testObj.tearDown(); } suiteTearDown();

March 2005 Atomic Object LLC public void testConstructor() throws Exception { WheelData wD = new WheelData();

assertNotNull("dataLabels not initialized", wD.dataLabels); assertNotNull("pointLabels not initialized", wD.pointLabels); assertEquals("wrong size", 0, wD.dataLabels.size()); }

public void testLoadDataNull() throws Exception { try { WheelData data = new WheelData(); data.loadData(null); fail("should have thrown"); } catch (IllegalArgumentException ex) { // should throw } }

March 2005 Atomic Object LLC Design for testability

March 2005 Atomic Object LLC Easier test, better design

March 2005 Atomic Object LLC What do we test?

• Test “everything that can break” – White-box, experienced-based, intuitive – Cardinal sin of missing tests • Subsystem tests are more methodical – All possible combinations of subsystem state – Distributed file system example Usage X Ownership X Storage X Connectivity

March 2005 Atomic Object LLC Automotive testing app

Source code classes 310 code bulk (statements) 19,700 average statements / class 63.5

Unit and integration tests test methods 2,500 test bulk (statements) 35,000 assertions 6,500 average tests/class 8.1 average asserts/method 2.6

March 2005 Atomic Object LLC March 2005 Atomic Object LLC Testing and Integration

• All tests run 100% correct before you commit • Limits the scope of problems, speeds up the process of finding them • Less onerous to maintain continuously • The system is always in a working state – Growing working complex systems – Great sales tool • Size of project determines implementation – 10 minute rule

March 2005 Atomic Object LLC Subtle Benefits of TDD

• Requirements for a class are resolved earlier and more thoroughly • Classes are more loosely coupled since they are designed to be testable in isolation • Developers have tests as a form of class documentation • Pace of development is smoother, tests never fall behind code base, avoiding technical debt • Small, positive psychological lift from passing tests • Minimize the anxiety of "releasing" incomplete or buggy code to your fellow developers

March 2005 Atomic Object LLC Pragmatics

• “Program to an interface” pays off big – Example: MVP pattern and GUI app development • Concurrency is hard in tests, too • Poorly written tests may run standalone • Tests must be independent of each other • Tests should leave the system as they find it • Tests that do too much are easy to think of, hard to maintain • Using the file system directly for test resources is bad

March 2005 Atomic Object LLC Test infection

Developer morale

Confident integration

Better design

Maintainability

March 2005 Atomic Object LLC