Contents Foreword xiii Preface xv Acknowledgments xxi About the Authors xxiii Chapter 1 Getting Started 1 Beyond JUnit 3 3 Stateful Classes 3 Parameters 4 Base Classes 4 Exceptions Are Not That Exceptional 4 Running Tests 5 Real-World Testing 6 Configuration Methods 6 Dependencies 6 Epiphanies 7 JUnit 4 7 Designing for Testability 8 Object-Oriented Programming and Encapsulation 8 The Design Patterns Revolution 9 Identifying the Enemy 10 Recommendations 16 TestNG 17 Annotations 17 Tests, Suites, and Configuration Annotations 18 Groups 20 testng.xml 21 Conclusion 21 Chapter 2 Testing Design Patterns 23 Testing for Failures 23 Reporting Errors 24 Runtime and Checked Exceptions 25 v vi Contents Testing Whether Your Code Handles Failures Gracefully 27 When Not to Use expectedExceptions 31 testng-failed.xml 32 Factories 34 @Factory 35 org.testng.ITest 38 Data-Driven Testing 39 Parameters and Test Methods 42 Passing Parameters with testng.xml 44 Passing Parameters with @DataProvider 47 Parameters for Data Providers 50 The Method Parameter 50 The ITestContext Parameter 52 Lazy Data Providers 54 Pros and Cons of Both Approaches 59 Supplying the Data 60 Data Provider or Factory? 62 Tying It All Together 63 Asynchronous Testing 67 Testing Multithreaded Code 71 Concurrent Testing 72 threadPoolSize, invocationCount, and timeOut 75 Concurrent Running 79 Turning on the Parallel Bit 82 Performance Testing 83 Algorithm Complexity 84 Testing Complexity 87 Mocks and Stubs 90 Mocks versus Stubs 90 Designing for Mockability 95 Mock Libraries 96 Selecting the Right Strategy 99 Mock Pitfalls 100 Dependent Testing 103 Dependent Code 104 Dependent Testing with TestNG 105 Deciding Whether to Depend on Groups or on Methods 106 Dependent Testing and Threads 110 Failures of Configuration Methods 110 Inheritance and Annotation Scopes 113 The Problem 113 Pitfalls of Inheritance 116 Test Groups 119 Syntax 120 Groups and Runtime 122 Running Groups 125 Using Groups Effectively 127 Contents vii Code Coverage 132 A Coverage Example 133 Coverage Metrics 134 Coverage Tools 136 Implementation 146 Beware! 147 A Guide to Successful Coverage 147 Conclusion 150 Chapter 3 Enterprise Testing 153 A Typical Enterprise Scenario 154 Participants 155 Testing Methodology 155 Issues with the Current Approach 156 A Concrete Example 157 Goals 159 Nongoals 160 Test Implementation 160 Testing for Success 161 Building Test Data 163 Test Setup Issues 166 Error Handling 172 Emerging Unit Tests 175 Coping with In-Container Components 177 Putting It All Together 178 Exploring the Competing Consumers Pattern 182 The Pattern 182 The Test 184 The Role of Refactoring 186 A Concrete Example 187 An In-Container Approach 193 Conclusion 194 Chapter 4 Java EE Testing 197 In-Container versus Out-of-Container Testing 198 In-Container Testing 200 Creating a Test Environment 200 Identifying Tests 201 Registering Tests 203 Registering a Results Listener 204 Java Naming and Directory Interface (JNDI) 207 Understanding JNDI’s Bootstrapping 207 Spring’s SimpleNamingContextBuilder 209 Avoiding JNDI 210 viii Contents Java Database Connectivity (JDBC) 210 c3p0 212 Commons DBCP 213 Spring 213 Java Transaction API (JTA) 215 Java Open Transaction Manager (JOTM) 217 Atomikos TransactionEssentials 218 Java Messaging Service (JMS) 219 Creating a Sender/Receiver Test 219 Using ActiveMQ for Tests 221 Java Persistence API (JPA) 225 Configuring the Database 227 Configuring the JPA Provider 227 Writing the Test 229 Simulating a Container 230 Using Spring as the Container 231 Enterprise Java Beans 3.0 (EJB3) 236 Message-Driven Beans 237 Session Beans 240 Another Spring Container 243 Disadvantages of a Full Container 244 Java API for XML Web Services (JAX-WS) 246 Recording Requests 248 Setting Up the Test Environment 248 Creating the Service Test 251 XPath Testing 253 Testing Remote Services 254 Servlets 255 In-Container Testing 255 Mock/Stub Objects 255 Refactoring 257 Embedded Container 257 In-Memory Invocation 260 XML 262 Using dom4j 263 Using XMLUnit 264 Conclusion 266 Chapter 5 Integration 269 Spring 270 Spring’s Test Package Features 271 Test Class Hierarchy 272 Guice 280 The Issue with Spring 280 Enter Guice 281 A Typical Dependency Scenario 282 Contents ix The Object Factory 284 Guice Configuration 286 Guice-Based Test 290 Grouping Test Dependencies 291 Injecting Configuration 293 DbUnit 295 Configuration 295 Usage 297 Verifying Results 299 HtmlUnit 303 Configuration 304 Usage 305 Selenium 310 Swing UI Testing 312 Testing Approach 312 Configuration 313 Usage 314 Tests for Painting Code 316 Continuous Integration 320 Why Bother? 320 CI Server Features 320 TestNG Integration 321 Conclusion 322 Chapter 6 Extending TestNG 325 The TestNG API 325 org.testng.TestNG, ITestResult, ITestListener, ITestNGMethod 325 A Concrete Example 328 The XML API 331 Synthetic XML Files 333 BeanShell 335 BeanShell Overview 335 TestNG and BeanShell 337 Interactive Execution 339 Method Selectors 341 Annotation Transformers 346 Annotation History 346 Pros and Cons 348 Using TestNG Annotation Transformers 348 Possible Uses of Annotation Transformers 353 Reports 355 Default Reports 355 The Reporter API 360 The Report Plug-in API 360 x Contents Writing Custom Annotations 366 Implementation 367 Testing 371 Conclusion 375 Chapter 7 Digressions 377 Motivation 377 The TestNG Philosophy 378 The Care and Feeding of Exceptions 378 Stateful Tests 382 Immutable State 382 Mutable State 383 The Pitfalls of Test-Driven Development 385 TDD Promotes Microdesign over Macrodesign 385 TDD Is Hard to Apply 386 Extracting the Good from Test-Driven Development 388 Testing Private Methods 388 Testing versus Encapsulation 391 The Power of Debuggers 392 Logging Best Practices 394 The Value of Time 397 Conclusion 399 Appendix A IDE Integration 401 Eclipse 401 Installing the Plug-in 401 Verifying the Installation 404 Creating a Launch Configuration 404 Configuring Preferences 410 Converting JUnit Tests 410 IntelliJ IDEA 411 Installing the Plug-in 411 Running Tests 412 Running Shortcuts 417 Viewing Test Results 418 Running Plug-in Refactorings 419 Appendix B TestNG Javadocs 421 JDK 1.4 and JDK 5 421 Shortcut Syntax for JDK 5 Annotations 423 Contents xi Annotation Javadocs 423 @DataProvider/@testng.data-provider 425 @Factory/@testng.factory 426 @Parameters/@testng.parameters 426 @Test/@testng.test 427 The org.testng.TestNG Class 428 The XML API 432 Appendix C testng.xml 435 Overview 436 Scopes 437 XML Tags 437 <suite> 437 <packages> and <package> 440 <parameter> 441 <suite-files> and <suite-file> 442 <method-selectors>, <method-selector>, <selector-class>, and <script> 443 <test> 444 <groups>, <define>, and <run> 446 <classes> and <class> 446 <methods> 447 Appendix D Migrating from JUnit 449 JUnitConverter 449 From the Command Line 449 From ant 452 Integrated Development Environments 453 Eclipse 453 IDEA 454 Incremental Migration and JUnit Mode 455 Converting JUnit Code 456 Assertions 457 Running a Single Test 458 Maintaining State between Invocations 461 Suite-wide Initialization 463 Class-wide Initialization 463 The AllTests Pattern 463 Testing Exceptions 467 The Parameterized Test Case Pattern 469 Index 471 Beust.book Page 90 Monday, September 17, 2007 12:36 PM 90 Chapter 2 Testing Design Patterns Mocks and Stubs In any software project, there are likely numerous subsystems and compo- nents. These components often have contractual boundaries that specify the contact points between them, as well as define the external interface to each component or subsystem. Take any standard Web application, for example. There’s likely to be a component that manages users, one that handles emails, another that man- ages presentation, and another that manages persistence. There are rela- tionships between all of these, with components relying on others to perform their roles and in turn providing functionality for other compo- nents to fulfill their roles. When it comes to testing, especially unit testing, it’s crucial that we’re able to isolate these components as much as possible. How our components are written can make this more or less difficult, due to how dependencies between components are managed. Are components looked up from a cen- tral registry? Do we have singletons? Are dependencies injected? Regardless of which approach is used, our unit tests wouldn’t be very good unit tests if they had to cart in the entire system to test any given part of it. So very shortly after the ideas around unit testing started to solidify, there was a recognized need to be able to provide the bare minimum of dependencies required in order to test effectively. If we wanted to test our user manager component in the example Web application we just described, we should be able to do so without necessarily having to provide every other component as well, just because our user man- ager happens to use a Mailer object and a UserDAO object to perform its roles. The solution in this case is to use stubs or mocks, depending on the situ- ation. In this section we’ll cover both, with examples highlighting the differ- ences, as well as advice on when you should choose one over the other. Mocks versus Stubs Before we delve into the differences, let’s first identify the common design pattern that these two approaches share in a trivial example, as illustrated in Figure 2–1. In Figure 2–1, we have a UserManager object that creates a user through a UserDAO helper and, when that’s done, emails the user with his or her login information. A Data Access Object (DAO) is a pattern in which we encapsulate com- munication with an underlying data store in an object, which usually han- dles create/read/update/delete operations for a given entity. Beust.book Page 91 Monday, September 17, 2007 12:36 PM Mocks and Stubs 91 Figure 2–1 Sequence diagram for a component We already have unit tests in place to ensure that emailing works and that the UserDAO object behaves as expected and saves users correctly in our data store.