Contents
Foreword xiii Preface xv Acknowledgments xxi About the Authors xxiii
Chapter 1 Getting Started 1 Beyond JUnit 3 3 Stateful Classes 3 Parameters 4 Base Classes 4 Exceptions Are Not That Exceptional 4 Running Tests 5 Real-World Testing 6 Configuration Methods 6 Dependencies 6 Epiphanies 7 JUnit 4 7 Designing for Testability 8 Object-Oriented Programming and Encapsulation 8 The Design Patterns Revolution 9 Identifying the Enemy 10 Recommendations 16 TestNG 17 Annotations 17 Tests, Suites, and Configuration Annotations 18 Groups 20 testng.xml 21 Conclusion 21
Chapter 2 Testing Design Patterns 23 Testing for Failures 23 Reporting Errors 24 Runtime and Checked Exceptions 25
v vi Contents
Testing Whether Your Code Handles Failures Gracefully 27 When Not to Use expectedExceptions 31 testng-failed.xml 32 Factories 34 @Factory 35 org.testng.ITest 38 Data-Driven Testing 39 Parameters and Test Methods 42 Passing Parameters with testng.xml 44 Passing Parameters with @DataProvider 47 Parameters for Data Providers 50 The Method Parameter 50 The ITestContext Parameter 52 Lazy Data Providers 54 Pros and Cons of Both Approaches 59 Supplying the Data 60 Data Provider or Factory? 62 Tying It All Together 63 Asynchronous Testing 67 Testing Multithreaded Code 71 Concurrent Testing 72 threadPoolSize, invocationCount, and timeOut 75 Concurrent Running 79 Turning on the Parallel Bit 82 Performance Testing 83 Algorithm Complexity 84 Testing Complexity 87 Mocks and Stubs 90 Mocks versus Stubs 90 Designing for Mockability 95 Mock Libraries 96 Selecting the Right Strategy 99 Mock Pitfalls 100 Dependent Testing 103 Dependent Code 104 Dependent Testing with TestNG 105 Deciding Whether to Depend on Groups or on Methods 106 Dependent Testing and Threads 110 Failures of Configuration Methods 110 Inheritance and Annotation Scopes 113 The Problem 113 Pitfalls of Inheritance 116 Test Groups 119 Syntax 120 Groups and Runtime 122 Running Groups 125 Using Groups Effectively 127 Contents vii
Code Coverage 132 A Coverage Example 133 Coverage Metrics 134 Coverage Tools 136 Implementation 146 Beware! 147 A Guide to Successful Coverage 147 Conclusion 150
Chapter 3 Enterprise Testing 153 A Typical Enterprise Scenario 154 Participants 155 Testing Methodology 155 Issues with the Current Approach 156 A Concrete Example 157 Goals 159 Nongoals 160 Test Implementation 160 Testing for Success 161 Building Test Data 163 Test Setup Issues 166 Error Handling 172 Emerging Unit Tests 175 Coping with In-Container Components 177 Putting It All Together 178 Exploring the Competing Consumers Pattern 182 The Pattern 182 The Test 184 The Role of Refactoring 186 A Concrete Example 187 An In-Container Approach 193 Conclusion 194
Chapter 4 Java EE Testing 197 In-Container versus Out-of-Container Testing 198 In-Container Testing 200 Creating a Test Environment 200 Identifying Tests 201 Registering Tests 203 Registering a Results Listener 204 Java Naming and Directory Interface (JNDI) 207 Understanding JNDI’s Bootstrapping 207 Spring’s SimpleNamingContextBuilder 209 Avoiding JNDI 210 viii Contents
Java Database Connectivity (JDBC) 210 c3p0 212 Commons DBCP 213 Spring 213 Java Transaction API (JTA) 215 Java Open Transaction Manager (JOTM) 217 Atomikos TransactionEssentials 218 Java Messaging Service (JMS) 219 Creating a Sender/Receiver Test 219 Using ActiveMQ for Tests 221 Java Persistence API (JPA) 225 Configuring the Database 227 Configuring the JPA Provider 227 Writing the Test 229 Simulating a Container 230 Using Spring as the Container 231 Enterprise Java Beans 3.0 (EJB3) 236 Message-Driven Beans 237 Session Beans 240 Another Spring Container 243 Disadvantages of a Full Container 244 Java API for XML Web Services (JAX-WS) 246 Recording Requests 248 Setting Up the Test Environment 248 Creating the Service Test 251 XPath Testing 253 Testing Remote Services 254 Servlets 255 In-Container Testing 255 Mock/Stub Objects 255 Refactoring 257 Embedded Container 257 In-Memory Invocation 260 XML 262 Using dom4j 263 Using XMLUnit 264 Conclusion 266
Chapter 5 Integration 269 Spring 270 Spring’s Test Package Features 271 Test Class Hierarchy 272 Guice 280 The Issue with Spring 280 Enter Guice 281 A Typical Dependency Scenario 282 Contents ix
The Object Factory 284 Guice Configuration 286 Guice-Based Test 290 Grouping Test Dependencies 291 Injecting Configuration 293 DbUnit 295 Configuration 295 Usage 297 Verifying Results 299 HtmlUnit 303 Configuration 304 Usage 305 Selenium 310 Swing UI Testing 312 Testing Approach 312 Configuration 313 Usage 314 Tests for Painting Code 316 Continuous Integration 320 Why Bother? 320 CI Server Features 320 TestNG Integration 321 Conclusion 322
Chapter 6 Extending TestNG 325 The TestNG API 325 org.testng.TestNG, ITestResult, ITestListener, ITestNGMethod 325 A Concrete Example 328 The XML API 331 Synthetic XML Files 333 BeanShell 335 BeanShell Overview 335 TestNG and BeanShell 337 Interactive Execution 339 Method Selectors 341 Annotation Transformers 346 Annotation History 346 Pros and Cons 348 Using TestNG Annotation Transformers 348 Possible Uses of Annotation Transformers 353 Reports 355 Default Reports 355 The Reporter API 360 The Report Plug-in API 360 x Contents
Writing Custom Annotations 366 Implementation 367 Testing 371 Conclusion 375
Chapter 7 Digressions 377 Motivation 377 The TestNG Philosophy 378 The Care and Feeding of Exceptions 378 Stateful Tests 382 Immutable State 382 Mutable State 383 The Pitfalls of Test-Driven Development 385 TDD Promotes Microdesign over Macrodesign 385 TDD Is Hard to Apply 386 Extracting the Good from Test-Driven Development 388 Testing Private Methods 388 Testing versus Encapsulation 391 The Power of Debuggers 392 Logging Best Practices 394 The Value of Time 397 Conclusion 399
Appendix A IDE Integration 401 Eclipse 401 Installing the Plug-in 401 Verifying the Installation 404 Creating a Launch Configuration 404 Configuring Preferences 410 Converting JUnit Tests 410 IntelliJ IDEA 411 Installing the Plug-in 411 Running Tests 412 Running Shortcuts 417 Viewing Test Results 418 Running Plug-in Refactorings 419
Appendix B TestNG Javadocs 421 JDK 1.4 and JDK 5 421 Shortcut Syntax for JDK 5 Annotations 423 Contents xi
Annotation Javadocs 423 @DataProvider/@testng.data-provider 425 @Factory/@testng.factory 426 @Parameters/@testng.parameters 426 @Test/@testng.test 427 The org.testng.TestNG Class 428 The XML API 432
Appendix C testng.xml 435 Overview 436 Scopes 437 XML Tags 437