ABAP Object Unit, 747 ABAP Unit, 747 Abstract Setup Decorator Defined, 449 Example, 453 Acceptance Tests. See Also Customer Test

Index

A Hard-Coded Test Data solution, 196 ABAP Object Unit, 747 preface, xxi ABAP Unit, 747 anonymous inner class Abstract Setup Decorator defi ned, 786 defi ned, 449 Test Stub examples, 535–536 example, 453 Ant, 753 acceptance tests. See also AntHill, 753 customer tests anti-pattern (AP) defi ned, 785 defi ned, 786 why test?, 19 test smells, xxxv accessor methods, 785 AOP (aspect-oriented programming) ACID, 785 defi ned, 786 acknowledgements, xxvii–xxviii Dependency Injection, 681 action components, 280 retrofi tting testability, 148 agile method API (application programming inter- defi ned, 785–786 face) property tests, 52 Creation Methods, 416 AllTests Suite database as SUT, 336 example, 594–595 defi ned, 786 introduction, 13 Test Utility Method, 600 when to use, 593 architecture, design for testability. annotation See design-for-testability defi ned, 786 arguments Test Methods, 351 messages describing, 371–372 Anonymous Creation Method as parameters (Dummy defi ned, 417 Arguments), 729 example, 420 role-describing, 725

835 836 Index

Arguments, Dummy, 729 improperly coded in Neverfail Ariane 5 rocket, 218 Tests, 274 aspect-oriented programming (AOP) introduction, 77 defi ned, 786 Missing Assertion Messages, Dependency Injection, 681 226–227 retrofi tting testability, 148 reducing Test Code Duplication, Assertion Message 114–119 of Assertion Method, 364 refactoring, xlvi–xlix pattern description, 370–372 Self-Checking Tests, 107–108 Assertion Method unit testing, 6 Assertion Messages, 364 Verify One Condition per Test, calling built-in, 363–364 46–47 choosing right, 364–365 assumptions, xxxix–xl Equality Assertions, 365 Astels, Dave, 110 examples, 368–369 asynchronous tests Expected Exception defi ned, 787 Assertions, 366 Hard-To-Test Code, 210–211 Fuzzy Equality Assertions, Humble Object, 696–697 365–366 Slow Tests, 255–256 implementation, 363 testability, 70–71 as macros, 364 Attachment Method motivating example, 367–368 defi ned, 418 overview, 362–363 example, 421 refactoring, 368 attributes Single-Outcome Assertions, defi ned, 787 366–367 dummy, 729 Stated Outcome Assertions, 366 hiding unnecessary, 303–304 Assertion Roulette One Bad Attribute. See One Eager Tests, 224–226 Bad Attribute impact, 224 parameters as, 608 introduction, 14 Suite Fixture Setup, 442–443 Missing Assertion Message, Test Discovery using, 397 226–227 Test Selection, 403–405 symptoms, 224 Automated Exercise Teardown assertions defi ned, 505 Built-in, 110–111 example, 508 custom. See Custom Assertion Automated Fixture Teardown, defi ned, 786 504–505 diagramming notation, xlii Automated Teardown Domain Assertions, 476, ensuring Repeatable Tests, 27 481–482 examples, 507–508 Index 837

implementation, 504–505 Beck, Kent, xxii Interacting Test Suites, 232 sniff test, xxxviii Interacting Tests solution, 231 Test Automation Frameworks, motivating example, 505–506 301 overview, 503–504 test smells, 9 of persistent fi xtures, 99–100 Testcase Class per Class, 618 refactoring, 506–507 xUnit, 57 resource leakage solution, 233 Behavior Sensitivity when to use, 504 cause of Fragile Tests, 242–243 automated unit testing caused by Overspecifi ed author’s motivation, xxiv–xxv Software, 246 fragile test problem, xxxi–xxxii defi ned, xxxi introduction, xxx–xxxii smells, 14 behavior smells, 223–247 B Assertion Roulette. See Assertion Roulette back door, defi ned, 787 defi ned, 10–11, 788 Back Door Manipulation Erratic Tests. See Erratic Test control/observation points, 66–67 Fragile Tests. See Fragile Test database as SUT API, 336 Frequent Debugging. See Expected State Specifi cation, 464 Frequent Debugging fi xture setup, 333–335 Manual Intervention. See implementation, 330–332 Manual Intervention motivating example, 332 overview, 13–15 overview, 327–328 Slow Tests. See Slow Tests refactoring, 333 Behavior Verifi cation setup, 329 approach to Self-Checking teardown, 330 Tests, 108 verifi cation, 329–330 examples, 472–473 verifi cation using Test Spy, 333 implementation, 469–471 when to use, 328 indirect outputs, 179–180 Back Door Setup motivating example, 471–472 controlling indirect inputs, 128 overview, 468–469 fi xture design, 59 refactoring, 472 Prebuilt Fixtures, 430–431 vs. state, 36 transient fi xtures, 86 test results, 112–114 Back Door Verifi cation, 130–133 using Mock Objects. See BDUF (big design upfront) Mock Object defi ned, 787 using Test Spies. See Test Spy design for testability, 65 using Use the Front Door test automation strategy, 49 First, 40 838 Index

verifying indirect outputs, BPT (Business Process Testing) 130–133 defi ned, 753 when to use, 469 Recorded Tests, 280 behavior-driven development Test Automation defi ned, 787–788 Frameworks, 301 Testcase Class per Fixture Bug Repellent, 22 usage, 632 Buggy Test Behavior-Exposing Subclass introduction, 12–13 Test-Specifi c Subclass reducing risk, 181 example, 587 symptoms, 260–262 when to use, 580 Built-in Assertion Behavior-Modifying Subclass calling, 363–364 Defi ning Test-Specifi c Equality, introduction, 110–111 588–589 built-in self-tests Substituted Singleton, defi ned, 788 586–587 test fi le organization, 164 Test Stub, 584–585 built-in test recording when to use, 580 defi ned, 281 Bespoke Assertion. See Custom example, 281–282 Assertion business logic bimodal tests, 687 defi ned, 789 binding, static developer testing, xxx defi ned, 809 development process, 4–5 Dependency Injection, 678–679 Layer Tests example, 344–345 black box testing without databases, defi ned, 788 169–171 Remoted Stored Procedure Business Process Testing (BPT). Tests, 656 See BPT (Business Process Testing) block closures defi ned, 788 Expected Exception Tests, C 354–355 Calculated Value. See also Derived blocks Value cleaning up fi xture teardown Loop-Driven Tests, 615 logic, l–liv Production Logic in Test defi ned, 788 solution, 205 try/fi nally. See try/fi nally block Canoo WebTest boundary values defi ned, 753 defi ned, 788 Scripted Tests, 286 erratic tests, 238 Test Automation Minimal Fixtures, 303 Frameworks, 301 result verifi cation patterns, 478 test automation tools, 53 Index 839 capacity tests, 52 samples, xli–xlii Capture/Playback Test. writing tests, 27–29 See Recorded Test code smells Chained Test Conditional Test Logic. See customer testing, 6 Conditional Test Logic examples, 459–460 defi ned, 10–11, 789 implementation, 456–457 Hard-To-Test Code. See motivating example, 457–458 Hard-To-Test Code overview, 454–455 obscure tests. See Obscure Test refactoring, 458 Test Code Duplication. See Test Shared Fixture strategies, 64–65 Code Duplication Shared Fixtures, 104–105, 322 Test Logic in Production. See when to use, 455–456 Test Logic in Production xUnit introduction, 57 types of, 16–17 class attributes coding idioms defi ned, 789 defi ned, xxxv Test Discovery using, 397 design patterns, 792 Testcase Class Selection using, collisions 404–405 Interacting Tests, 229–231 class methods Shared Fixtures, 318 defi ned, 789 Command object with Test Helper, 645, 646 introduction, 82 class variables Testcase Object as, 382 defi ned, 789 Command-Line Test Runner Suite Fixture Setup, 442 Assertion Message, 371 classes defi ned, 379–380 diagramming notation, xlii introduction, 79 as fi xtures, 59 Missing Assertion Message, Test Double, 569–570, 572–573 226–227 Testcase. See Testcase Class commercial recorded tests class-instance duality, 374 refactored, 283–284 Cleanup Method, 602 tools, 282–283 closure, block common location, Test Discovery, defi ned, 788 397–398 Expected Exception Tests, Communicate Intent 354–355 defi ned, 41 Cockburn, Alistair refactoring Recorded Tests to, pattern naming, 578 283–284 service layer tests, 339 compiler macro, Test Method code Discovery, 395–396 inside-out development, 34–36 Complex Teardown, 206–207 organization. See test Complex Test. See Dependency organization Lookup 840 Index

Component Broker. See Dependency Confi gurable Test Double Lookup examples, 564–567 Component Registry, 688 implementation, 559–562 component tests installing, 141–142 defi ned, 790 as kind of Test Double, 528 layer-crossing tests, 69 motivating example, 562–563 per-functionality, 52 overview, 558 test automation philosophies, refactoring, 563 34–36 when to use, 559 test strategy patterns, 340 Confi gurable Test Stub. See also components Confi gurable Test Double defi ned, 790 implementation, 532 depended-on component. See indirect input control, 179 DOC (depended-on Confi guration Interface component) examples, 564–566 Composite object, defi ned, 82 implementation, 560 Concerns, Separation of, 28–29 Confi guration Mode concrete classes, 581 example, 566–567 Condition Verifi cation Logic, 203–204 implementation, 560 Conditional Test Logic Constant Value. See Literal Value vs. Assertion Method, 363 constants in Derived Value, avoidance, 119–121 718–722 avoiding via Custom constructing Mock Object, 546 Assertion, 475 Constructor Injection avoiding via Guard Assertion, example, 683–684 490–493 implementation, 680–681 causes, 201–202 installing Test Doubles, 144 Complex Teardown, 206–207 Constructor Test Condition Verifi cation Logic, defi ned, 351 203–204 example, 355–357 Flexible Tests, 202–203 introduction, 77 impact, 201 constructors introduction, 16 defi ned, 790 Multiple Test Conditions, problems with, 419 207–208 containers, Humble Container Production Logic in Test, Adapter, 698 204–205 Context Sensitivity symptoms, 200 avoiding via Isolate the SUT, Test Methods, 155 43–44 Confi gurable Mock Object, 546–547. defi ned, 245–246 See also Confi gurable Test Double introduction, xxxii, 14 Confi gurable Registry, 691–692 continuous design, xxxiii Index 841 continuous integration CSV fi les, xUnit Data-Driven avoiding Lost Tests, 270 Test, 296 defi ned, 791 CUnit, 748 impact of Data-Driven Tests, 290 Cunningham, Ward, xxv, 290 steps, 14 Custom Assertion control points as Conditional Verifi cation defi ned, 791 Logic solution, 204 testability, 66–67 examples, 480–484 Coplien, Jim, 576 implementation, 477–478 CORBA standards, 744 Indirect Testing solution, cost effectiveness, Self-Checking 198–199 Tests, 107–108 Irrelevant Information costs, test automation, 20–21 solution, 193 Covey, Stephen, 121 motivating example, 478–480 CppUnit overview, 474–475 defi ned, 748 reducing Test Code Duplication, Test Automation Frameworks, 116–117 300 refactoring, 480 Test Method enumeration, 401 Test Utility Methods, 602 Creation Method when to use, 475–477 Delegated Setup, 89–91, writing simple tests, 28 411–414 Custom Assertion test eliminating unnecessary example, 483–484 objects/attributes, 303–304 implementation, 477–478 examples, 420–423 Custom Equality Assertion, 476 as Hard-Coded Test Data customer tests solution, 196 defi ned, 791 hybrid setup, 93 Eager Tests cause, 225 implementation, 418–419 Missing Unit Test, 271 motivating example, 419 overview, 5–6 overview, 415–416 per-functionality, 51 persistent fi xtures as Scripted Test, 285–287 teardown, 100 Cut and Paste code reuse, preface, xxiii 214–215 refactoring, 420 as Test Utility Method, 600 when to use, 416–418 D writing simple tests, 28 data access layer cross-functional tests, 52–53 database testing, 172–173 cross-thread failure assertion, 274 defi ned, 791 Cruise Control, 754 Slow Tests with Shared CsUnit, 748 Fixtures, 319 842 Index

data leaks database testing, 167–174 avoiding with Delta Assertions, overview, 167–169 486–487 persistent fi xtures, 313 Complex Teardown, 206 testing without databases, Data Loader, Back Door 169–171 Manipulation, 330–331 types of, 171–174 data minimization, 738–739 Database Transaction Rollback Tear- data population script, 434 down, 674–675 Data Retriever, 331 databases Data Sensitivity fake. See Fake Database defi ned, 243–245 as SUT API, 336 introduction, xxxii, 14 teardown, 100 Data Transfer Object (DTO) Data-Driven Test defi ned, 793 customer testing, 5 result verifi cation, 116 Fit framework example, Database Extraction Script, 331 296–297 Database Partitioning Scheme frameworks, 300 Data Sensitivity solution, implementation, 290 244–245 implemented as Recorded developer independence, 173 Test, 281 example, 653 introduction, 83 Global Fixtures, 430 motivating example, 293–294 implementation, 652 overview, 288–289 database patterns, 649–675 principles, 48 Database Sandbox, reducing Test Code Duplication, 650–653 118–119 Stored Procedure Test, refactoring notes, 294 654–660 Test Suite Object Simulator, 293 Table Truncation Teardown, using Fit framework, 661–667 290–292 Transaction Rollback via Naive xUnit Test Interpreter, Teardown, 668–675 292–293 Database Population Script, 330 via Test Suite Object Database Sandbox Generator, 293 database testing, 168 when to use, 289–290 design for testability, 7 xUnit with CSV input fi le, 296 pattern description, 650–653 xUnit with XML data fi le, as Test Run Wars solution, 294–295 236–237 DB Schema per Test Runner Unrepeatable Tests cause, 235 developer independence, 173 when to use, 650 implementation, 651–652 Index 843

DbUnit Delta Assertion Back Door Manipulation, 335 avoiding fi xture collisions, 101 defi ned, 748 as Data Sensitivity solution, 245 Expected State Specifi cation, 464 detecting data leakage with, 487 DDSteps, 754 examples, 488–489 Decorated Lazy Setup, 449–450 introduction, 111 Decorator pattern description, 485–486 Abstract Setup Decorator, depended-on component (DOC). See 449, 453 DOC (depended-on component) Parameterized Setup Decorator, dependencies 452–453 Interacting Tests, 230–231 Pushdown Decorator, 450 replacement with Test Setup. See Setup Decorator Doubles, 739 Test Hook as, 710 replacing using Test Hooks, Dedicated Database Sandbox, 651 709–712 Defect Localization retrofi tting testability, 148 customer testing, 5 test automation philosophies, 34 defi ned, 22–23 Test Dependency in Production, Frequent Debugging, 248 220–221 Keep Tests Independent Tests, 43 test fi le organization, 165 right-sizing Test Methods, 154 Dependency Initialization Test, 352 test automation philosophies, 34 Dependency Injection unit testing, 6 design for testability, 7 Verify One Condition per Test, 45 examples, 683–685 defi ning tests implementation, 679–681 introduction, 76–78 installing Test Doubles via, suites of, 78–79 143–144 delays. See Slow Tests Isolate the SUT, 44 Delegated Setup motivating example, 682 example, 413–414 overview, 678 introduction, 77 Persistent Fresh Fixtures matching with teardown code, avoidance, 62–63 98–99 refactoring, 682 overview, 411–414 testability improvement, 70 of transient fi xtures, 89–91 when database testing, 171 when to use, 412 when to use, 678–679 Delegated Teardown Dependency Lookup example, 514–515 design for testability, 7 overview, 511 examples, 691–693 of persistent fi xtures, 98–99 implementation, 688–689 Table Truncation Teardown, 665 installing Test Doubles, 144–145 844 Index

Isolate the SUT, 44 deterministic values, 238 motivating example, 690 developer independence, 173 names, 693–694 developer testing overview, 686 defi ned, 792 Persistent Fresh Fixtures, introduction, xxx 62–63 Developers Not Writing Tests, 13 refactoring, 690–691 development when database testing, 171 agile, 239 when to use, 687–688 behavior driven, 632, 787–788 Derived Expectation document-driven, 793 example, 720 EDD. See EDD (example-driven when to use, 719 development) Derived Input, 719 incremental, 33–34, 799–800 Derived Value inside-out, 463 examples, 719–722 inside-out vs. outside in, 34–36 overview, 718 need-driven. See need-driven when to use, 718–719 development design patterns, xxxv, 792 outside-in, 469 design-for-testability process, 4–5 control points and observation TDD. See TDD (test-driven points, 66–67 development) defi ned, 792 test-fi rst. See test-fi rst divide and test, 71–72 development ensuring testability, 65 test-last. See test-last development interaction styles and testability Diagnostic Assertion, 476–477 patterns, 67–71 diagramming notation, xlii overview, 7 Dialog, Humble. See Humble Dialog Separation of Concerns, 28–29 direct output test automation philosophies. defi ned, 792–793 See test automation verifi cation, 178 philosophies Direct Test Method Invocation, 401 test automation principles, 40 disambiguation, test fi xtures, 814 test-driven testability, 66 Discovery, Test. See Test Discovery design-for-testability patterns, Distinct Generated Values 677–712 Anonymous Creation Dependency Injection. See Methods, 417 Dependency Injection Delegated Setup, 90 Dependency Lookup. See example, 725–726 Dependency Lookup Hard-Coded Test Data Humble Object. See Humble solution, 196 Object implementation, 724 Test Hooks, 709–712 Unrepeatable Tests solution, 235 Index 845

Distinct Values, 717 dynamic binding Do No Harm, 24–25 defi ned, 793 DOC (depended-on component) use in Dependency Injection, 679 Behavior Verifi cation, 469 Dynamically Generated Mock control points and observation Object, 550 points, 66–67 Dynamically Generated Test Double defi ned, 791–792 implementation, 561–562 outside-in development, 35 providing, 140–141 replacing with Test Double. Dynamically Generated Test Stub, See Test Double 534–535 retrieving. See Dependency Lookup terminology, xl–xli E Test Hook in, 712 Eager Test Documentation, Tests as. Assertion Roulette, 224–226 See Tests as Documentation Fragile Tests, 240 document-driven development, Obscure Tests, 187–188 793 right-sizing Test Methods, 154 Domain Assertion EasyMock defi ned, 476 defi ned, 754 example, 481–482 Test Doubles, 140 domain layer eCATT defi ned, 793 defi ned, 754 test strategy patterns, 337 Test Automation Frameworks, domain model, 793 301 Don’t Modify the SUT, 41–42 Eclipse drivers, test Debugger, 110 defi ned, 813 defi ned, 754 lack of Assertion Messages, economics of test automation, 20–21 370 EDD (example-driven development) DRY (don’t repeat yourself), 28 defi ned, 794 DTO (Data Transfer Object) tests as examples, 33 defi ned, 793 effi ciency, 11 result verifi cation, 116 emergent design Dummy Argument, 729 vs. BDUF, 65 Dummy Attribute, 729 defi ned, xxxiii, 794 Dummy Object encapsulation confi guring, 141–142 Creation Method. See Creation defi ned, 133 Method as Test Double, 134–135, 526 Dependency Lookup as value pattern, 728–732 implementation, 688–689 xUnit terminology, 741–744 indirect outputs and, 126 846 Index

Indirect Testing solution, 198 equivalence class SUT API. See SUT API Behavior Smells, 238 Encapsulation defi ned, 794 using Test Utility Methods. Untested Code, 272 See Test Utility Method Erratic Test endoscopic testing (ET) Automated Teardown and, 27 defi ned, 794 customer testing, 5 Mock Objects, 545 database testing, 168–169 Test Doubles, 149 impact, 228 Ensure Commensurate Effort and Interacting Test Suites, 231–232 Responsibility, 47–48 Interacting Tests, 229–231 Entity Chain Snipping introduction, 14–16 example, 536–537 Lonely Tests, 232 testing with doubles, 149 Nondeterministic Tests, when to use, 531 237–238 entity object, 794 Resource Leakage, 233 enumeration Resource Optimism, 233–234 customer testing, 5 symptoms, 228 Suite of Suites built using, 389–391 Test Run Wars, 235–237 test conditions in Loop-Driven troubleshooting, 228–229 Tests, 614–615 Unrepeatable Tests, 234–235 Test Enumeration, 399–402 essential but irrelevant fi xture Test Suite Object built using, 388 setup, 425 xUnit organization ET (endoscopic testing) mechanisms, 153 defi ned, 794 Equality, Sensitivity Mock Object use for, 149, 545 Fragile Tests, 246 example-driven development (EDD) test-fi rst development, 32 defi ned, 794 Equality Assertion tests as examples, 33 Assertion Methods, 365 examples, tests as, 33 Custom, 476 exclamation marks, xlii example, 368 Executable, Humble. See Humble Guard Assertion as, 491 Executable introduction, 110 Executable Specifi cation, 51 reducing Test Code execution optimization, 180–181 Duplication, 115 exercise SUT unit testing, 6 defi ned, 794 Equality Pollution, 221–222 test phases, 359 equals method expectations Equality Pollution, 221–222 defi ned, 795 Expected State Specifi cation, 464 Derived Expectations, 719, 720 reducing Test Code Duplication, messages describing, 371–372 115–116 naming conventions, 159 Index 847

Expected Behavior Specifi cation in persistent fi xture teardown, 98 defi ned, 470–471 refactoring Recorded Tests, 283 example, 473 Extract Testable Component, 197, Expected Behavior Verifi cation 735–736 defi ned, 112 eXtreme Programming indirect outputs, 131–132 defi ned, 795 Expected Exception Assertion projects affected by Slow Tests, defi ned as Assertion Method, 319–321 365–366 eXtreme Programming Explained example, 369 (Beck), xxii Expected Exception Test Conditional Verifi cation Logic solution, 204 F introduction, 77 factories as Test Method, 350–351 defi ned, 795 using block closure, 354–355 Factory Method, 592–593 using method attributes, 354 Object Factories, 145, 688 using try/catch, 353–354 failed tests Expected Object due to Unfi nished Test reducing Test Code Duplication, Assertions, 494–497 115–116 implementation, 80 refactoring tests, xlv–xlviii “Fail-Pass-Pass”, 234–235 State Verifi cations, 109, 466–467 failure messages unit testing, 6 Assertion Messages, 370–372 expected outcome, 795 Built-in Assertions, 110–111 Expected State Specifi cation, removing “if” statements, 120 464–465 Single-Outcome Assertions, expected values, 546–547 366–367 exploratory testing Fake Database cross-functionality, 53 avoiding persistence, 101 defi ned, 795 database testing, 170 Scripted Tests, 287 example, 556–557 Expression Builders, 564–566 Slow Component Usage expressiveness gaps, 27–28 solution, 254 external resource setup, 740 Slow Tests with Shared external result verifi cation, 111–112 Fixtures, 319 external test recording, 280 when to use, 553 Extract Method Fake Object Creation Methods, 418 confi guring, 141–142 Custom Assertions, 117 customer testing, 6 Delegated Setup, 89 defi ned, 134 as Eager Tests solution, 225 examples, 556–557 example, xlvii implementation, 553–554 848 Index

motivating example, 554–555 Fit optimizing test execution, 180 Data-Driven Test example, overview, 551–552 296–297 refactoring, 555–556 Data-Driven Test as Test Double, 139, 525 implementation, 290–292 when to use, 552–553 defi ned, 754–755, 796 xUnit terminology, 741–744 Expected State Specifi cation, 464 Fake Service Layer, 553 fi xture defi nition, 59, 86 Fake Web Services, 553 fi xture vs. Testcase Class, 376 false negative, 795 Scripted Tests false positive, 795–796 implementation, 286 fault insertion tests Test Automation defi ned, 796 Framework, 301 per-functionality, 52 test automation tools, 54 Feathers, Michael, 40 tests as examples, 33 Highly Coupled Code vs. xUnit, 57 solution, 210 Fitnesse Humble Object, 708 Data-Driven Test pattern naming, 576 implementation, 290 retrofi tting testability, 148 defi ned, 755 Self Shunt, 578 Scripted Test test automation roadmap, 176 implementation, 286 Unit Test Rulz, 307 “Five Whys”, 11 features fi xture design defi ned, 796 upfront or test-by-test, 36 right-sizing Test Methods, Verify One Condition per 156–157 Test, 46 Testcase Class per. See Testcase xUnit sweet spot, 58 Class per Feature fi xture holding class variables, 797 visibility/granularity in fi xture holding instance Test-Specifi c Subclass, variables, 797 581–582 fi xture setup feedback in test automation, xxix Back Door Manipulation, 329, fi le contention. See Test Run War 333–335 File System Test Runner, 380 cleaning up, liv–lvii Finder Method defi ned, 797 accessing Shared Fixtures, Delegated Setup, 89–91 103–104 external resources, 740 Mystery Guests solution, 190 Four-Phase Test, 358–361 when to use, 600–601 Fresh Fixtures, 313–314 fi ne-grained testing, 33–34 hybrid setup, 93 Index 849

Implicit Setup, 91–93 introduction, 77 In-Line Setup, 88–89 Lazy Setup problems, 439 introduction, 77 persistent fi xtures, 97–100 matching with teardown code, Persistent Fresh Fixtures, 314 98–99 refactoring, l–liv Shared Fixtures, 104–105 Shared Fixtures, 105 speeding up with doubles, transient fi xtures, 93–94 149–150 Verify One Condition per strategies, 60 Test, 46 fi xture setup patterns, 407–459 fi xture teardown patterns, 499–519 Chained Test. See Chained Test Automated Teardown, Creation Method. See Creation 503–508 Method Garbage-Collected Teardown, Delegated Setup, 411–414 500–502 Implicit Setup, 424–428. See also Implicit Teardown, 516–519. Implicit Setup See also Implicit Teardown In-line Setup, 408–410. See also In-line Teardown, 509–515. In-line Setup See also In-line Teardown Lazy Setup. See Lazy Setup Table Truncation Teardown, Prebuilt Fixture. See Prebuilt 661–667 Fixture Transaction Rollback Setup Decorator. See Setup Teardown. See Transaction Decorator Rollback Teardown Suite Fixture Setup. See Suite fi xtures Fixture Setup collisions, 100–101 Fixture Setup Testcase, 456 database testing, 168–169 fi xture strategies defi ned, 796, 814 overview, 58–61 Four-Phase Test, 358–361 persistent fresh fi xtures, 62–63 fresh. See Fresh Fixture shared fi xture strategies, 63–65 introduction, 78 fi xture teardown Minimal. See Minimal Fixture avoiding in persistent fi xtures, right-sizing Test Methods, 100–101 156–157 Back Door Manipulation, 330 Shared. See Shared Fixture cleaning up, l–liv speeding up setup with doubles, Complex Teardown, 206–207 149–150 data access layer testing, 173 Standard. See Standard Fixture defi ned, 797 Testcase Class as, 376 fi xture strategies, 60 Testcase Class per Fixture. Four-Phase Test, 358–361 See Testcase Class per Fixture Implicit Setup, 426 transient. See transient fi xtures 850 Index

Flexible Test, 202–203 Data Sensitivity, 243–245 fl uent interface, 797 Fragile Fixture, 246–247 For Tests Only, 219–220 High Test Maintenance foreign-key constraints, 663 Cost, 266 forms, pattern, xxxiv–xxxv impact, 239 Four-Phase Test Interface Sensitivity, 241–242 Custom Assertions, 478 introduction, xxiii, xxxi–xxxii, fi xture design, 59 13–14 introduction, 76–78 Overspecifi ed Software, 246 Mock Object patterns, 546 Sensitivity Equality, 246 pattern description, 358–361 symptoms, 239 unit testing, 6 troubleshooting, 239–240 Verify One Condition per Test, 46 frameworks Fowler, Martin, xxvi Fit. See Fit code smells, 16 Test Automation Framework, 75, Creation Methods, 418 298–301 Custom Assertions, 117 Frequent Debugging Cut and Paste code reuse, 215 avoidance with Custom Delegated Setup, 89, 413 Assertion, 475 Eager Tests solution, 225 causes, 248–249 Multiple Test Conditions impact, 249 solution, 208 introduction, 15 pattern forms, xxxvi solution patterns, 249 refactoring, xxxix symptoms, 248 refactoring Recorded Tests, 283 Fresh Fixture reusable test logic, 123 Creation Method. See Creation self-testing code, xxi Method Standard Fixtures, 306 Data Sensitivity solution, state vs. behavior 244–245 verifi cation, 36 Delegated Setup, 411–414 test smells, 9 example, 316 Testcase Object exception, 385 fi xture strategies, 60–61 Fragile Fixture implementation, 312 defi ned, 246–247 Implicit Setup, 424–428 introduction, 14, 16 Interacting Tests solution, 231 setUp method misuse, 93 motivating example, 315 Fragile Test Mystery Guests solution, 190 Behavior Sensitivity, 242–243 overview, 311 Buggy Tests, 260 persistent, 62–63, 313–314. causes, 240–241 See also persistent fi xtures Context Sensitivity, 245–246 refactoring, 315 Index 851

setup, 313–314 misuse of setUp method, test automation philosophies, 36 92–93 Test Run Wars solution, 236–237 Obscure Tests, 190–192 transient, 61–62. See also Slow Tests, 255 transient fi xtures Generated Value, 723–727 Transient Fresh Fixture, 314 Geras, Adam, 280 when to use, 312 Global Fixture, 430 front door, 797 global variables Front Door First defi ned, 798 defi ned, 40–41 instance variables as, 92 Overspecifi ed Software goals, test automation. avoidance, 246 See test automation goals Fully Automated Test Gorts, Sven, 537 behavior smells and, 15 granularity Communicate Intent and, 41 test automation tools and, Manual Fixture Setup 53–54 solution, 251 Test-Specifi c Subclass, minimizing untested code, 44–45 581–582 running, 25–26 Graphical Test Runner unit testing, 6 clicking through to test code, functional tests 226–227 defi ned, 798 defi ned, 378–379 per-functionality, 50–52 green bar, 26 Fuzzy Equality Assertion introduction, 79, 300 defi ned, 365–366 graphical user interface (GUI). example, 368–369 See GUI (graphical user interface) external result verifi cation, green bar, defi ned, 798 111–112 Guaranteed In-Line Teardown, introduction, 110 233 Guard Assertion G Conditional Verifi cation Logic solution, 203–204 Gamma, Erich, 57 introduction, 80 garbage collection, 798 pattern description, 490–493 Garbage-Collected Teardown removing “if” statements in design-for-testability, 7 Test Method, 120 pattern description, 500–502 GUI (graphical user interface) persistent fi xtures, 97 defi ned, 799 transient fi xtures, 87–88 design for testability, 7 General Fixture Interface Sensitivity, xxxii database testing, 169 testing with Humble defi ned, 187 Dialogs, 696 852 Index

H testing with, 140–142 when to use, 569 Hand-Built Test Double. See also Hard-Coded Test Spy. See Hard-Coded Test Double Hard-Coded Test Double Confi gurable Test Double, Hard-Coded Test Stub. See also 560–561 Hard-Coded Test Double providing, 140–141 implementation, 531–532 Hand-Coded Mock Object, 548–550 indirect input control, 179 hand-coded teardown, 97–98 Hard-Coded Value, 103 Hand-Coded Test Stub, 533–534 Hard-To-Test Code Hand-Scripted Test. See also Asynchronous Code, 210–211 Scripted Test Buggy Tests, 261 introduction, 75 code smells, 16 tools for automating, 53–54 Developers Not Writing Hand-Written Test. See Scripted Test Tests, 264 happy path divide and test, 71–72 defi ned, 799 High Test Maintenance Cost, Responder use, 530 266–267 Simple Success Tests, 349–350 Highly Coupled Code, 210 test automation roadmap, impact, 209 177–178 solution patterns, 209 Hard-Coded Mock Object. See Hard- symptoms, 209 Coded Test Double Untestable Test Code, 211–212 Hard-Coded Setup Decorator hierarchy of test automation needs, defi ned, 449 176–177 example, 451–452 High Test Maintenance Cost Hard-Coded Test Data Conditional Test Logic, 200 causing Obscure Tests, 194–196 In-Line Setup, 89 defi ned, 187 introduction, 12–13 introduction, lv–lvii, 16 smell description, 265–267 Hard-Coded Test Double Higher Level Language confi guring, 141–142 Custom Assertion, 117 implementation, 527, 569–571 Interface Sensitivity solution, 241 motivating example, 571 xUnit sweet spot, 58 naming patterns, 576–578 Highly Coupled Code, 210 overview, 568 historical patterns and smells, xxxviii refactoring, 572 Hollywood principle Self Shunt/Loopback, 573 defi ned, 56, 799 Subclassed Inner Test Double, test results, 79 573–575, 578 Hook, Test. See Test Hook Test Double Class, 572–573 HTML user interface sensitivity, xxxii Index 853

HttpUnit, 755 Idea, 755 Humble Container Adapter, 698 IeUnit Humble Dialog defi ned, 748 design-for-testability, 7 Graphical Test Runner, 378 example, 706–708 “if” statements Hard-To-Test Code, 72 Conditional Test Logic, 201 minimizing untested code, 45 Guard Assertions, 490–491 when to use, 696–697 removing, 120 Humble Executable IFixtureFrame, 442 asynchronous tests, 70–71 ignoring tests, 270 minimizing untested code, 44 Immutable Shared Fixture motivating example, 700–702 defi ned, 323 Neverfail Test solution, 274 example, 326 when to use, 697 Interacting Tests solution, 231 Humble Object introduction, 61, 65 Asynchronous Code solution, 211 vs. Irrelevant Information, 192 Humble Dialog, 706–708 Test Run Wars solution, 237 Humble Transaction impact Controller, 708 Assertion Roulette, 224 implementation, 698–700 Asynchronous Code, 211 motivating example, 700–702 Buggy Tests, 260 overview, 695–696 Conditional Test Logic, 201 Poor Manís Humble Developers Not Writing Executable, 703 Tests, 263 refactoring, 702 Equality Pollution, 221 True Humble Executable, Erratic Tests, 228 703–706 Flexible Tests, 203 when to use, 696–698 Fragile Tests, 239 Humble Transaction Controller Frequent Debugging, 249 data access layer testing, 173 General Fixtures, 191–192 example, 708 Hard-Coded Test Data, 195 when to use, 697–698 Hard-To-Test Code, 209 Hurst, John, 670–671 High Test Maintenance hybrid setup, 93 Cost, 265 Highly Coupled Code, 210 I Indirect Testing, 197 IDE (integrated development Irrelevant Information, 193 environment) Manual Intervention, 250 defi ned, 799 Mystery Guests, 189 introduction, 78 Neverfail Tests, 274 refactoring, xxxix Nondeterministic Tests, 237 854 Index

Obscure Tests, 186 incremental tests, 322 Production Bugs, 268 In-Database Stored Procedure Test Slow Tests, 253 database testing, 172 Test Code Duplication, 214 example, 658–659 Test Dependency in implementation, 655–656 Production, 221 Independent Tabular Test, 612–613 Test Hooks, 218–219 independent testing. See Keep Tests Test Logic in Production, 217 Independent Test Run Wars, 236 indirect input For Tests Only, 220 alternative path verifi cation, 179 Untestable Test Code, 211 controlling, 128–129 Untested Requirements, 273 controlling in Layer Tests, 341 Implicit Setup defi ned, 800 vs. Four-Phase Test, 360–361 importance of, 126 introduction, 7, 77 Test Doubles, 125–126 matching with teardown code, indirect output 98–99 Behavior Verifi cation. pattern description, 424–428 See Behavior Verifi cation pattern naming, 577 defi ned, 800 reusing test code with, 162 importance of, 126–127 transient fi xtures, 91–93 registries, 541 Implicit Teardown Test Doubles, 125–126 Complex Teardown solution, verifi cation, 130–133, 178–180 206–207 verifying in Layer Tests, 341 database, 100 Indirect Testing vs. Four-Phase Test, 360–361 defi ned, 187 pattern description, 516–519 Fragile Tests cause, 240 persistent fi xtures, 98–99 Obscure Tests cause, 196–199 Self-Checking Tests with, 108 testability, 70–71 Imposter. See Test Double Infrequently Run Test incremental delivery Frequent Debugging cause, agile development, 239 248–249 defi ned, 799 Production Bugs cause, 268–269 incremental development inheritance defi ned, 799–800 reusing test code, 164 test automation philosophies, reusing test fi xtures, 62 33–34 injected values, Test Stub. Incremental Tabular Test See Test Stub implementation, 609–610 Injection, Parameter. See Parameter Parameterized Test patterns, Injection 613–614 in-line Four Phase Test, 360 Index 855 in-line resources, 736–737 Mock Object, 547 In-line Setup retrofi tting testability, introduction, 77 146–148 matching with teardown code, instance methods 98–99 defi ned, 800–801 Mystery Guest solution, 190 with Test Helper, 645, 647 pattern description, 408–410 instance variables transient fi xtures, 88–89 converting for Implicit Setup, 427 In-line Teardown Data-Driven Tests using Fit examples, 512–515 Framework, 297 implementation, 510–511 defi ned, 801 motivating example, 511 Fresh Fixtures, 313 Naive In-Line Teardown, 512 as global variables, 92 overview, 509 Reuse Tests for Fixture Setup, of persistent fi xtures, 98–99 418–419 refactoring, 512 with Test Specifi c Subclass, 558 when to use, 510 Testcase Class per Fixture, 632 In-Memory Database, 553 instances inner class reusing, 63 anonymous, 535–536, 786 Testcase Object exception, defi ned, 800 384–385 Inner Test Double integrated development environment example, 573–574 (IDE). See IDE (integrated Hard-Coded Test Double development environment) implementation, 570–571 Integration Build, 4 Subclassed from Pseudo-Class, Intent-Revealing Name 574–575, 578 Custom Assertion, 474–475 Test Spy implementation, 541 Implicit Setup, 92 input Parameterized Test, 608 derived, 719 Test Utility Method, 602–603 indirect. See indirect input Interacting Test Suites, naming conventions, 158–159 231–232 inside-out development Interacting Tests vs. outside-in development, 34–36 avoiding with Database State Verifi cation, 463 Sandbox, 650–653 installing Test Doubles, 528 avoiding with Delta Assertion, Dependency Injection, 143–144, 111, 486 679–680 caused by Shared Fixture, 63 Dependency Lookup, 144–145 Chained Tests, 455 Fake Object, 554 customer testing, 5–6 introduction, 143 database testing, 169 856 Index

Erratic Test cause, 229–231 JBehave introduction, 15 defi ned, 748 Keep Tests Independent, 43 tests as examples, 33 interaction point, 801 JFCUnit, 755 interaction styles, 67–71 JMock Interaction Testing. See Behavior Confi guration Interface, 560 Verifi cation defi ned, 755 Interface Sensitivity Test Double implementation, defi ned, 241–242 140 introduction, xxxii, 13 Johnson, Rod, 670 interfaces JUnit Confi guration Interface, 560 defi ned, 748 defi ned, 801 Expected Exception Test GUI. See GUI (graphical user expression, 351 interface) fi xture design, 59 outgoing interface, 804–805 language-specifi c terminology, xl standard test, 378 Suite Fixture Setup support, Test Runner. See Test Runner 442–443 Use the Front Door First, 40–41 Test Automation internal recording tools, 56 Framework, 300 interpreters in Data-Driven Tests. test automation tools, 55 See Data-Driven Test Testcase Object exception, Intervention, Manual. See Manual 384–385 Intervention testing stored procedures, 657 Introduce Explaining Variable refactoring, lvii–lviii K IoC (inversion of control) framework defi ned, 801 Keep Test Logic Out of Production for Dependency Injection, 680 Code irrelevant information minimizing risk, 24 defi ned, 187 principle, 45 Obscure Test, 192–194 test code organization, 164–165 Isolate the SUT, 43–44 Keep Tests Independent iterative development, 802 running, 26 test automation principles, 42–43 J using Fake Object. See Fake Java Object language-specifi c xUnit Kerievsky, Joshua, xxxix terminology, xl keys, Literal Values as, 714 test code packaging, 165 King, Joseph, 319–321 Index 857

L leakage, resource Erratic Tests, 233 languages persistent fi xtures, 99 terminology, xl–xli learning styles, xxxix–xl variations in Built-in Assertions, legacy software 110–111 Buggy Tests, 261–262 xUnit implementations, 76 defi ned, 802 language-specifi c xUnit terminology, tests as safety net, 24 xl–xli lenient Mock Object “Law of Raspberry Jam”, xxv defi ned, 138 Layer Test when to use, 545 Business Layer Tests, 344–345 lightweight implementation using database testing, 169–171 Fake Object. See Fake Object implementation, 340–341 Literal Value motivating example, 341–342 Hard-Coded Test Data, 195 overview, 337–338 pattern description, 714–717 Presentation Layer Tests, 343 local variables refactoring, 342 converting in Implicit Subcutaneous Tests, 343–344 Setup, 427 when to use, 338–340 defi ned, 802 layer-crossing tests Fresh Fixtures, 313 defi ned, 802 Lonely Test testability, 67–69 caused by Chained Test. See Layered Architecture Chained Test design-for-testability, 7 Erratic Tests, 232 layer-crossing tests, 67–69 Interacting Tests. Lazy Initialization, 435 See Interacting Tests Lazy Setup Long Tests. See Obscure Test Decorated, 449–450 Loopback. See Self Shunt examples, 439–440 Loop-Driven Test implementation, 436–437 implementation, 610 Interacting Tests solution, 231 Parameterized Test, 614–615 motivating example, 437–438 loops overview, 435 as Conditional Test Logic, 201 vs. Prebuilt Fixtures, 431–432 eliminating, 121 refactoring, 439 Production Logic in Test cause, Shared Fixture, 64, 105 204–205 when to use, 436 Lost Tests Lazy Teardown avoiding, 597 example, 665–666 Production Bugs cause, implementation, 663–664 269–271 858 Index

M metatests, 803 method attributes Mackinnon, Tim, 149 defi ned, 803 macros, Assertion Methods as, 364 Expected Exception Tests, 354 maintenance Test Discovery using, 397 High Test Maintenance Cost. See Test Method Selection High Test Maintenance Cost using, 405 optimizing, 180–181 method names test automation goals, 27–29 language-specifi c xUnit Manual Event Injection, 251–252 terminology, xl–xli Manual Fixture Setup, 250–251 Test Method Discovery, 395–396 Manual Intervention methods impact, 250 diagramming notation, xlii introduction, 15 instance. See instance methods Manual Event Injection, setUp. See setUp method 251–252 static, 809 Manual Fixture Setup, 250–251 suite, 399 Manual Result Verifi cation, 251 tearDown. See tearDown method symptoms, 250 Template Method, 164 Manual Result Verifi cation, 251 test commands, 82 manual testing verifi cation. See result defi ned, 802 verifi cation right-sizing Test Methods, 154 Miller, Jeremy, 687 Marrick, Brian Minimal Fixture purpose of tests, 51 external result verifi cation, 112 right-sizing Test Methods, 155 General Fixtures solution, 192 tests as examples, 33 minimizing data, 738–739 Maslow, 176 misuse of setUp method, 93 MbUnit pattern description, 302–304 defi ned, 749 strategy, 62–63 Parameterized Test test automation philosophies, 36 implementation, 608–609 Minimize Test Overlap, 44 Tabular Test with framework Minimize Untestable Code, 44–45 support, 614 Missing Assertion Message, 226–227 Message, Assertion. See Assertion Missing Unit Test Message Defect Localization, 23 messages, failure. See failure Production Bugs, 271 messages mixins meta objects defi ned, 803 Data-Driven Tests, 290 Test Helper Mixins, 639, defi ned, 803 641–642 Index 859

Mock Object N Confi gurable. See Confi gurable Naive In-line Teardown Test Double defi ned, 511 confi guring, 141–142 example, 512 defi ned, 133 of persistent fi xtures, 97 examples, 548–550 Naive xUnit Test Interpreter, Expected Behavior Specifi cation, 292–293 470–471 Named State Reaching Method, implementation, 546–548 417–418 motivating example, 548 Named Test Suite Overspecifi ed Software examples, 594–598 cause, 246 implementation, 594 overview, 544–545 introduction, 160–161 refactoring, 548 overview, 592–593 Test Double patterns, 525 refactoring, 594 Test Doubles, 137–139 Test Enumeration, 400 unit testing, 6 when to use, 593–594 vs. Use the Front Door First, 40 names verifying indirect output, Dependency Lookup, 693–694 131–133 intent-revealing. See when to use, 545 Intent-Revealing Name xUnit terminology, 741–744 referring to patterns and smells, MockMaker, 560 xxxviii modules, 803–804 Scripted Test, 287 Move Method, 413 Suite Fixture Setup, 446 MSTest, 749 naming conventions Mugridge, Rick, xxiv assertion-identifying multimodal tests, 687 messages, 371 multiple-condition tests making resources unique, Conditional Test Logic, 737–738 207–208 patterns, 576–578 defi ned, 45–47 vs. test code organization, Multiresource In-line Teardown, 158–159 513–514 Test Method Discovery, MySql, 651 395–396 Mystery Guest Testcase Class per Class, 618 defi ned, 187 Testcase Class per Feature, 626 Obscure Test cause, 188–190 Testcase Class per Fixture, 632 For Tests Only solution, 220 860 Index

need-driven development Object Mother Behavior Verifi cation, 469 in Delegated Setup, 90–91 defi ned, 804 when to use, 644–645 testing with doubles, 149 object technology, xxxix–xl using Mock Objects, 545 Object Transaction Rollback Neverfail Test, 274 Teardown, 673–674 New River Gorge bridge, xxvi object-oriented programming Newkirk, James, 384–385 language (OOPL), 76 NMock, 756 object-relational mapping (ORM). No Test Risk, 24–25 See ORM (object-relational Nondeterministic Test mapping) dangers of, 26–27 objects Erratic Test, 237–238 Creation Method. See Creation Generated Values cause, 723–724 Method notation, diagramming, xlii determining necessary, Null Object vs. Dummy Object, 730 303–304 null values in Dummy Objects, diagramming notation, xlii 729–732 fake. See Fake Object NUnit Test Suite Objects. See Test Suite defi ned, 749 Object Expected Exception Test Testcase. See Testcase Object expression, 351 Obscure Test fi xture design, 59 avoiding with Custom Assertion, Interacting Test Suites, 232 475 Suite Fixture Setup support, avoiding with Separation of Con- 442–443 cerns, 28–29 Test Automation Frameworks, Buggy Test, 261 300 causes, 186–187 test automation ways and vs. Communicate Intent, 41 means, 55 customer testing, 5 test fi xtures, 814 database testing, 169 Testcase Classes, 376 Eager Test, 187–188 Testcase Object exception, General Fixture, 190–192 384–385 Hard-Coded Test Data, 194–196 High Test Maintenance Cost, O 266 Object Attribute Equality Assertion, impact, 186 476 Indirect Testing, 196–199 Object Factory introduction, xlvi, 12–13, 16 Dependency Lookup, 688 Irrelevant Information, 192–194 installing Test Double, 145 Mystery Guests, 188–190 Index 861

optimizing test execution/ overlapping tests maintenance, 180 minimizing, 44 smells, 10 Too Many Tests, 256–257 solution patterns, 199 Overspecifi ed Software symptoms, 186 avoiding with Fake Objects, observation points 552 defi ned, 804 Fragile Tests, 246 test automation strategy, 66–67 testing with doubles, 150 O’Grady, Ted, 319–321 Use the Front Door First, 40 One Bad Attribute example, 721–722 P introduction, xxiii, 90 Minimal Fixtures, 304 Parameter Injection when to use, 719 example, 683 OOPL (object-oriented implementation, 680 programming language), 76 installing Test Doubles, 144 optimism, resource, 189, 233–234 Parameterized Anonymous Creation order of tests, 456 Method, 417 organization, test. See test Parameterized Creation Method organization; test organization defi ned, 417 patterns Delegated Setup, 90 ORM (object-relational mapping) example, xxiii, 420–421 defi ned, 804 Irrelevant Information Table Truncation Teardown, 663 solution, 193 Table Truncation Teardown Parameterized Setup Decorator using, 667 defi ned, 449 Transaction Rollback example, 452–453 Teardown, 671 Parameterized Test Outcome Assertions, Stated. See example, 611–612 Stated Outcome Assertion extracting. See Data-Driven Test outcome verifi cation patterns. See further reading, 615–616 result verifi cation patterns implementation, 608–610 outcome-describing Verifi cation Incremental Tabular Test, Method, 117 613–614 outgoing interface, 804–805 Independent Tabular Test, out-of-order calls, 138 612–613 output, indirect. See indirect output Loop-Driven Tests, 614–615 outside-in development motivating example, 610–611 Behavior Verifi cation, 469 overview, 607–608 vs. inside-out development, reducing Test Code Duplication, 34–36 118–119 Overcoupled Software, 40 refactoring, 611 862 Index

Tabular Test with framework managing, 103–105 support, 614 overview, 95–96 Test Utility Method, 602 Slow Tests cause, 102 when to use, 608 Table Truncation Teardown. See parameters, arguments as, 729 Table Truncation Teardown “Pass-Fail-Fail”, 234–235 teardown avoidance, 100–101 pattern language tearing down, 97–100 defi ned, xxxv–xxxvi, 805 test strategy patterns, 313–314 pattern naming, 577 what’s next, 106 Pattern Languages of Programming Persistent Fresh Fixture (PLoP), 576 building, 88 patterns defi ned, 60–61 aliases and variations, 767–784 strategies, 62–63 database. See database patterns Personal Oracle, 651 defi ned, 805 philosophy, test automation. See test design-for-testability. See automation philosophies design-for-testability patterns PHPUnit, 749 fi xture setup. See fi xture setup PLoP (Pattern Languages of patterns Programming), 576 result verifi cation. See result Pluggable Behavior verifi cation patterns in Named Test Suites, 597 test automation introduction, Testcase Object xxxiv–xxxviii implementation, 383 Test Double. See Test Double pollution test organization. See test Equality Pollution, 221–222 organization patterns Shared Fixture, 326 test strategy. See test strategy polymorphism, 805 patterns Poor Manís Humble testability, 67–71 Executable, 703 value. See value patterns Poor Man’s Humble Object xUnit basics. See xUnit basics implementation, 699 patterns Transaction Rollback peeling the onion, 11 Teardown, 671 per-functionality test, 50–52 Poppendieck, Mary, 51 Perrotta, Paolo, 537 Pragmatic Unit Testing, 743 Per-Run Fixtures, 323 Prebuilt Fixture persistence layer, 339–340 examples, 432–434 persistence resources, 504 implementation, 430–431 persistent fi xtures, 95–106 motivating example, database testing, 168–169 431–432 issues caused by, 96 overview, 429–430 Index 863

refactoring, 432 Missing Unit Tests, 271 Shared Fixture strategies, 64 Neverfail Tests, 274 Shared Fixtures, 104–105 overview, 268 Unrepeatable Tests cause, 235 reducing risk, 181 presentation layer Untested Code, 271–272 defi ned, 805 Untested Requirements, 272–274 Layer Tests example, 343 production code testing, 338–339 defi ned, 806 presentation logic, 805 keeping test logic out of, 45 Preserve Whole Object refactoring, Production Logic in Test, 204–205 xlviii–xlix profi ling tools, 254 principles Programmatic Test. See Scripted Test list of, 757–759 programmer tests, 806 patterns vs., xxxv–xxxvi project smells, 259–274 test automation. See test Buggy Tests, 260–262 automation principles defi ned, 806 Private Fixture. See Fresh Fixture Developers Not Writing Tests, private methods, 586 263–264 problem statements, xxxvi–xxxvii High Test Maintenance Cost, Procedural Behavior Verifi cation 265–267 defi ned, 470 overview, 12–13 example, 472–473 Production Bugs. See Production indirect outputs, 131 Bugs introduction, 112–113 property tests, 52 Test Spy usage, 137 Pseudo-Object Procedural State Verifi cation Hard-Coded Test Double defi ned, 463–464 implementation, 570–571 example, 466 Inner Test Double Subclassed introduction, 109 from Pseudo-Class, 574–575, Procedural Test Stub 578 defi ned, 526 testing with doubles, 140–141 introduction, 135–136 pull system, 806–807 when to use, 531 Pull-Up Method refactoring Procedure Test, Stored. See Stored Delegated Setup, 413 Procedure Test moving reusable test logic, 123 procedure variables, 805–806 Testcase Superclass, 640 production, 806 Pushdown Decorator, 450 Production Bugs PyUnit Infrequently Run Tests, 268–269 defi ned, 749 introduction, 12–13 Test Automation Framework, Lost Tests, 269–271 300 864 Index

Q red bar, 807 Refactored Recorded Tests QA (quality assurance), 22–23 commercial, 283–284 QaRun, 244 overview, 280 QTP (QuickTest Professional) refactoring. See also test refactorings Data-Driven Tests, 290 Assertion Message, 372 defi ned, 756 Assertion Method, 368 record and playback tools, 282 Automated Teardown, Test Automation 506–507 Framework, 301 Back Door Manipulation, 333 quality assurance (QA), 22–23 Chained Test, 458 QuickTest Professional (QTP). Confi gurable Test Double, 463 See QTP (QuickTest Professional) Creation Method, 420 Custom Assertion, 480 R Database Sandbox, 653 random values Data-Driven Test, 294 Nondeterministic Tests, 238 defi ned, 807 Random Generated Values, 724 Delegated Setup, 413 Record and Playback Test, 13 Delta Assertion, 488 record and playback tools Dependency Injection, 682 introduction, xxxi Dependency Lookup, 690–691 Recorded Tests, 282–283 Derived Value, 720 xUnit sweet spot, 58 Dummy Object, 731 Recorded Test Fake Object, 555–556 built-in test recording, Fresh Fixture, 315–316 281–282 Garbage-Collected commercial record and Teardown, 502 playback tool, 282–283 Generated Value, 725 customer testing, 5 Guard Assertion, 492 Data-Driven Tests and, 289 Hard-Coded Test Double, 572 implementation, 280–281 Humble Object, 702 Interface Sensitivity, 241 Implicit Setup, 427 overview, 278–279 Implicit Teardown, 518–519 refactored commercial recorded In-line Setup, 410 tests, 283–284 In-line Teardown, 512 vs. Scripted Tests, 286 Layer Test, 342 smells, 10 Lazy Setup, 439 tools, 56 Literal Value, 716 tools for automating, 53–54 Mock Object, 548 when to use, 279–280 Named Test Suite, 594 Recording Test Stub. See Test Spy Parameterized Test, 611 Index 865

Prebuilt Fixture, 432 Related Generated Values Setup Decorator, 451 example, 726–727 Shared Fixture, 324 implementation, 725 Standard Fixture, 309–310 Remoted Stored Procedure Test State Verifi cation, 465–466 example, 659–660 Stored Procedure Test, 658 implementation, 656–658 Suite Fixture Setup, 444 introduction, 172 Table Truncation Teardown, Repeatable Test 664–665 defi ned, 26–27 Test Discovery, 395 indirect inputs control, 179 Test Helper, 646 Replace Dependency with Test Test Spy, 541–542 Double refactoring Test Stub, 533 Behavior Verifi cation, 472 Test Utility Method, 605 defi ned, 739 Testcase Class per Feature, Repository 627–628 Data-Driven Test fi les, 290 Testcase Class per Fixture, persistent objects, 90 634–635 source code, 24, 79, 234, Testcase Superclass, 640 561, 656 Test-Specifi c Subclass, 584 test code, 164, 561 Transaction Rollback Requirement, Untested. Teardown, 672 See Untested Requirement Unfi nished Test Assertion, 496 ReSharper, 756 Refactoring: Improving the Resource Leakage Design of Existing Code (Fowler), Erratic Tests, 233 9, 16 persistent fi xtures, 99 references, 819–832 Resource Optimism, 189, 233–234 refl ection resources defi ned, 807 external, 740 Test Discovery, 393 in-line, 736–737 Testcase Object unique, 737–738 implementation, 383 Responder Registry defi ned, 524 confi gurable, 691–692 examples, 533–535 in Dependency Lookup, 688–689 indirect input control, 179 Interacting Tests, 230 introduction, 135 Test Fixture, 644 when to use, 530 regression tests response time tests, 52 defi ned, 807 result verifi cation, 107–123 Recorded Tests. See Recorded Behavior Verifi cation, 112–114 Test Conditional Test Logic Scripted Tests, 285–287 avoidance, 119–121 866 Index

Data Sensitivity, 243–245 Robust Tests defi ned, 807 defi ned, 29 Four-Phase Test, 358–361 indirect inputs control, 179 Mock Object, 547–548 role-describing arguments, 725 other techniques, 121–122 root cause analysis reducing Test Code Duplication, defi ned, 808 114–119 smells, 11 reusable test logic, 123 round-trip tests Self-Checking Tests, 107–108 defi ned, 808 State Verifi cation, 109–112 introduction, 67–69 result verifi cation patterns, 461–497 Layer Tests, 340–341 Behavior Verifi cation. See row tests. See Tabular Test Behavior Verifi cation RSpec Custom Assertion. See Custom defi ned, 750 Assertion fi xture design, 59 Delta Assertion, 485–489 tests as examples, 33 Guard Assertion, 490–493 runit State Verifi cation. See State Veri- defi ned, 750 fi cation Test Automation Unfi nished Test Assertion, Frameworks, 300 494–497 running tests results, test introduction, 79 defi ned, 815 structure, 81 introduction, 79–80 test automation goals, 25–27 Retrieval Interface, 137, 540 runtime refl ection, 393 retrospective, 807–808 reusable test logic S Creation Method, 418–419 Saboteur fi xture setup patterns, 422–423 defi ned, 135 organization, 162–164 example, 535–536 result verifi cation, 123 inside-out development, 35 Test Code Duplication, 214–215 Test Double patterns, 524 Test Utility Method. See Test when to use, 530 Utility Method Safety Net Reuse Tests for Fixture Setup, 90 Buggy Tests, 260 Robot User Test. See Recorded Test tests as, 24 robot user tools sample code, xli–xlii defi ned, 55–56 screen scraping, 241 introduction, xxxi Scripted Test Test Automation Framework, Communicate Intent, 41 299 customer testing, 5 Index 867

Data-Driven Tests and, 289 data. See Data Sensitivity introduction, 75 interface. See Interface pattern description, 285–287 Sensitivity vs. Recorded Tests, 279 Separation of Concerns, 28–29 smells, 10 Service Facade, 71–72 UI, 55 service layers Verify One Condition per fake, 553 Test, 46 tests, 7, 339 Self Shunt Service Locator Behavior Verifi cations, 113 in Dependency Lookup. example, 573 See Dependency Lookup Hard-Coded Test Double installing Test Doubles, 145 implementation, 570 service objects, 808 pattern naming, 576 Setter Injection Test Spy implementation, Confi guration Interface 540–541 using, 564 Self-Call, 582 example, 684–685 Self-Checking Test implementation, 681 Assertion Method usage, 362 installing Test Doubles, 143 Conditional Test Logic setters, 808 solution, 201 setup, fi xtures. See fi xture setup defi ned, 80 Setup Decorator happy path code, 178 examples, 451–453 introduction, 107–108 implementation, 448–450 running, 26 Implicit Setup, 426 Self-Describing Value motivating example, 450–451 example, 717 overview, 447–448 Literal Value patterns, 715 refactoring, 451 self-testing code, xxi Shared Fixture strategies, 64, self-tests, built-in 104–105 defi ned, 788 when to use, 448 test fi le organization, 164 setUp method Sensitive Equality Implicit Setup, 91–92, 424–428 Fragile Tests, 246 misuse of, 92–93 test-fi rst development, 32 pattern naming, 577 sensitivities Setup Decorator. See Setup automated unit testing, Decorator xxxi–xxxii Suite Fixture Setup. See Suite behavior. See Behavior Sensitivity Fixture Setup Buggy Tests cause, 260 shadows, diagramming notation, xlii context. See Context Sensitivity Shank, Clint, 457–458, 613, 616 868 Index

Shared Fixture. See also Standard introduction, 77 Fixture pattern description, 349–350 Behavior Verifi cation, 108 The simplest thing that could Chained Test. See Chained Test possibly work (STTCPW), 810 customer testing, 5 Single Glance Readable. Data Sensitivity cause, 243 See Communicate Intent database testing, 169 Single Layer Test. See Layer Test defi ned, 60–61 Single Test Suite Delta Assertions, 111 example, 596–597 example, 324–325 Lost Tests solution, 270 Immutable. See Immutable when to use, 593–594 Shared Fixture single tests, 161–162 Immutable Shared Fixtures, 326 Single-Condition Test implementation, 322–323 Eager Tests solution, 225–226 incremental tests, 322 Obscure Tests solution, 188 Interacting Tests cause, 229–231 principles. See Verify One introduction, 15, 63–65 Condition per Test Lazy Setup. See Lazy Setup unit testing, 6 managing, 103–105 Single-Outcome Assertion motivating example, 323–324 Assertion Method, 366–367 in Nondeterministic Tests, 27 defi ned, 365 overview, 317 example, 369 Prebuilt Fixture. See Prebuilt Singleton Fixture in Dependency Lookup, refactoring, 324 688–689 Setup Decorator. See Setup Interacting Tests, 230 Decorator retrofi tting testability, 146–147 Slow Tests cause, 318–321 Singleton, Substituted Suite Fixture Setup. See Suite example, 586–587 Fixture Setup when to use, 581 Table Truncation Teardown. skeletons, 744 See Table Truncation Teardown Slow Component Usage, 254 Test Run Wars cause, 236 Slow Tests Unrepeatable Tests cause, 235 Asynchronous Tests, 255–256 using Finder Methods, 600–601 avoiding with Shared Fixture, when to use, 318 318–321 Shared Fixture Guard Assertion, database testing, 168 492–493 design for testability, 7 Shared Fixture State Assertion, 491 due to Transaction Rollback Simple Success Test Teardown, 669 example, 352–353 General Fixtures, 255 happy path code, 177 impact, 253 Index 869

introduction, 15 solution patterns, code smells optimizing execution, 180 Asynchronous Code, 211 persistent fi xtures, 102 Conditional Verifi cation Logic, preventing with Fake Object. 203–204 See Fake Object Cut and Paste code reuse, 215 preventing with Test Eager Test, 188 Double, 523 Equality Pollution, 222 Slow Component Usage, 254 Flexible Test, 203 symptoms, 253 General Fixture, 192 Too Many Tests, 256–257 Hard-Coded Test Data, 196 troubleshooting, 253–254 Hard-To-Test Code, 209 smells, test. See test smells Highly Coupled Code, 210 Smith, Shaun, 39 Indirect Testing, 197–199 Smoke Test Irrelevant Information, 193 development process, 4 Multiple Test Conditions, suites, 597–598 207–208 Test Discovery, 394 Mystery Guests, 190 sniff test Obscure Tests, 199 defi ned, xxxviii Production Logic in Test, 205 test smells, 10 Test Code Duplication, 115–216 solution patterns, behavior smells Test Dependency in Asynchronous Tests, 256 Production, 221 Behavior Sensitivity, 242–243 Test Hook, 219 Context Sensitivity, 246 For Tests Only, 220 Data Sensitivity, 243–245 Untestable Test Code, 212 Eager Tests, 225–226 solution patterns, project smells Frequent Debugging, 249 Buggy Test, 261–262 General Fixture, 255 Infrequently Run Test, 269 Interacting Test Suites, 232 Lost Test, 270–271 Interacting Tests, 231 Missing Unit Test, 271 Interface Sensitivity, 241–242 Neverfail Test, 274 Manual Intervention, Untested Code, 272 250–252 Untested Requirements, 274 Missing Assertion Messages, Special-Purpose Suite, 595–596 226–227 specifi cation Resource Leakage, 233 Expected Behavior, 470–471 Resource Optimism, 234 Expected Behavior Slow Component Usage, 254 example, 473 Test Run War, 236–237 Expected Object example, 466 Too Many Tests, 257 Expected State, 464–465 Unrepeatable Tests, 235 tests as, xxxiii, 22 870 Index

spikes, 809 static binding Spy, Test. See Test Spy defi ned, 809 SQL, Table Truncation Teardown Dependency Injection, using, 666–667 678–679 Standard Fixture static methods, 809 implementation, 307–308 static variables, 809 motivating example, 308 Statically Generated Test overview, 305–306 Doubles, 561 refactoring, 309–310 STDD (storytest-driven when to use, 306–307 development), 4, 810 standard test interface, 378 stop on fi rst failure starbursts, diagramming Naive xUnit Test Interpreter, notation, xlii 292–293 state, initializing via xUnit introduction, 57 Back Door Manipulation. Stored Procedure Test See Back Door Manipulation database testing, 172 Named State Reaching Method, examples, 658–660 417–418 implementation, 655–658 State Verifi cation motivating example, 658 vs. behavior, 36 overview, 654 examples, 466–467 refactoring, 658 implementation, 463–465 when to use, 654–655 indirect outputs, 179–180 storytest, 810 introduction, 109–112 storytest-driven development motivating example, 465 (STDD), 4, 810 overview, 462–463 strategies, test automation. See test refactoring, 465–466 automation strategies Self-Checking Tests, 108 stress tests, cross-functionality, 52 Use the Front Door First, 41 strict Mock Object when to use, 463 defi ned, 138 Stated Outcome Assertion when to use, 545 Assertion Methods, 366 STTCPW (The simplest thing that defi ned, 365 could possibly work), 810 example, 369 Stub, Test. See Test Stub Guard Assertions as, 491 Subclass, Test-Specifi c. See introduction, 110–111 Test-Specifi c Subclass State-Exposing Subclass Subclassed Humble Object, 700 Test-Specifi c Subclass, 289–590 Subclassed Inner Test Double, when to use, 580 573–574 stateless, 809 Subclassed Singleton, 7 statements, “if”. See “if” statements Subclassed Test Double, 146–147 Index 871

Subcutaneous Test Suites of Suites customer testing, 5 building with Test enumeration, database testing, 174 400 design for testability, 7 defi ned, 388 Layer Tests, 343–344 example, 389–391 Subset Suite Interacting Test Suites, 231–232 example, 594–598 introduction, 7, 15, 78 implementation, 594 SUnit introduction, 160–161 defi ned, 750 overview, 592 Test Automation Too Many Tests solution, 257 Frameworks, 300 when to use, 593 Superclass, Testcase. See Testcase substitutable dependencies Superclass defi ned, 810 SUT (system under test) Dependency Initialization control points and observation Test, 352 points, 66–67 using Test Spy, 540 dangers of modifying, 41–42 Substitutable Singleton defi ned, 810–811 in Dependency Lookup, 689 Four-Phase Test, 358–361 example, 586–587, 692–693 interface sensitivity, xxxii retrofi tting testability, 146–147 isolation principle, 43–44 when to use, 581 minimizing risk, 24–25 substitution mechanisms, preface, xxii–xxiii 688–689 replacing in Parameterized Suite Fixture Setup Test, 609 example, 444–446 result verifi cation. See result implementation, 442–443 verifi cation implicit, 426 state vs. behavior motivating example, 443–444 verifi cation, 36 overview, 441–442 terminology, xl–xli refactoring, 444 test automation tools, 53–54 Shared Fixture strategies, 64 Test Hook in, 711–712 Shared Fixtures, 104–105 understanding with test when to use, 442 automation, 23 suite method, 399 SUT API Encapsulation suites Chained Tests as, 455 Named Test Suite. See Named Indirect Testing solution, 198 Test Suite Interface Sensitivity test organization, 160–162 solution, 241 Test Suite Object. See Test Suite SUT Encapsulation Method, Object 601–602 872 Index

Symbolic Constants Mystery Guests, 188–189 example, 716 Obscure Tests, 186 Literal Value, 715 Production Logic in Test, symptoms, behavior smells 204–205 Assertion Roulette, 224 Test Code Duplication, 213–214 Asynchronous Tests, 255 Test Dependency in Behavior Sensitivity, 242 Production, 220 Context Sensitivity, 245 Test Logic in Production, 217 Data Sensitivity, 243 test smells, 10 Eager Tests, 224–225 For Tests Only, 219 Erratic Tests, 228 Untestable Test Code, 211 Fragile Tests, 239 symptoms, project smells Frequent Debugging, 248 Buggy Tests, 260 General Fixtures, 255 Developers Not Writing Tests, Interacting Test Suites, 231 263 Interacting Tests, 229 High Test Maintenance Interface Sensitivity, 241 Cost, 265 Manual Intervention, 250–252 Infrequently Run Tests, 268–269 Missing Assertion Messages, 226 Lost Tests, 269 Nondeterministic Tests, 237 Missing Unit Tests, 271 Resource Leakage, 233 Neverfail Tests, 274 Resource Optimism, 233 Production Bugs, 268 Slow Tests, 253 Untested Code, 271–272 Test Run Wars, 236 Untested Requirements, 272–273 Too Many Tests, 256 symptoms, test smells, 10 Unrepeatable Tests, 234–235 synchronous tests symptoms, code smells avoiding with Humble Object, Asynchronous Code, 210 696–697 Complex Teardown, 206 deﬁ ned, 810 Conditional Test Logic, 200 system under test (SUT). See SUT Eager Tests, 187–188 (system under test) Equality Pollution, 221 Flexible Tests, 202 General Fixtures, 190–191 T Hard-Coded Test Data, Table Truncation Teardown 194–195 data access layer testing, 173 Hard-To-Test Code, 209 deﬁ ned, 100 Highly Coupled Code, 210 examples, 665–667 Indirect Testing, 196–197 implementation, 662–664 Irrelevant Information, 192–193 motivating example, 664 Multiple Test Conditions, 207 overview, 661–662 Index 873

refactoring, 664–665 test automater, 811 when to use, 662 test automation, xxix–xliii tabular data, 291 assumptions, xxxix–xl Tabular Test automated unit testing, xxx–xxxii Chained Tests, 457–458 brief tour, 3–8 with framework support, 614 code samples, xli–xlii implementation, 609–610 developer testing, xxx Incremental, 613–614 diagramming notation, xlii Independent, 612–613 feedback, xxix tasks, 811 fragile test problem, xxxi–xxxii TDD (test-driven development) limitations, xliii defi ned, 813 overview, xxix implementing utility methods, patterns, xxxiv–xxxviii 122 refactoring, xxxviii–xxxix introduction, xxxiii–xxxiv terminology, xl–xli Missing Unit Tests, 271 testing, xxx need-driven development, 149 uses of, xxxiii–xxxiv process, 4–5 Test Automation Framework Test Automation introduction, 75 Frameworks, 301 pattern description, 298–301 test automation principles, 40 test automation goals, 19–29 teardown, fi xture. See fi xture ease of running, 25–27 teardown improving quality, 22–23 Teardown Guard Clause list of, 757–759 example, 513 objectives, 21–22 Implicit Teardown, 517–518 reducing risk, 23–25 In-line Teardown, 511 system evolution, 29 tearDown method understanding SUT, 23 Implicit Teardown, 516–519 why test?, 19–21 persistent fi xtures, 98 writing and maintaining, 27–29 Setup Decorator. See Setup Test Automation Manifesto, 39 Decorator test automation philosophies, 31–37 Template Method, 164 author’s, 37 Temporary Test Stub differences, 32–36 when to use, 530–531 importance of, 31–32 xUnit terminology, 741–744 test automation principles, 39–48 terminology Communicate Intent, 41 test automation introduction, Design for Testability, 40 xl–xli Don’t Modify the SUT, 41–42 transient fi xtures, 86–88 Ensure Commensurate Effort xUnit. See xUnit basics and Responsibility, 47–48 874 Index

Isolate the SUT, 43–44 tools for, 53–58 Keep Test Logic Out of transient fresh fi xtures, 61–62 Production Code, 45 what’s next, 73 Keep Tests Independent, wrong, 264 42–43 Test Bed. See Prebuilt Fixture Minimize Test Overlap, 44 test cases, 811 Minimize Untestable Code, test code, 811 44–45 Test Code Duplication overview, 39–40 causes, 214–215 Test Concerns Separately, 47 Custom Assertions, 475 Use the Front Door First, Delegated Setup, 412 40–41 High Test Maintenance Verify One Condition per Test, Cost, 266 45–47 impact, 214 Write the Tests First, 40 In-Line Setup, 89 test automation roadmap, 175–181 introduction, 16 alternative path verifi cation, possible solution, 216 178–179 reducing, 114–119 diffi culties, 175–176 reducing with Confi gurable direct output verifi cation, 178 Test Doubles. See Confi gurable execution and maintenance Test Double optimization, 180–181 reducing with Parameterized happy path code, 177–178 Tests. See Parameterized Test indirect outputs verifi cation, reducing with Test Utility 178–180 Methods. See Test Utility maintainability, 176–177 Method test automation strategies, 49–73 removing with Testcase Class per brief tour, 3–8 Fixture. See Testcase Class per control points and observation Fixture points, 66–67 reusing test code, 162 cross-functional tests, 52–53 symptoms, 213–214 divide and test, 71–72 Test Commands, 82 ensuring testability, 65 Test Concerns Separately, 47 fi xture strategies overview, 58–61 test conditions, 154, 811–812 interaction styles and testability test database, 812 patterns, 67–71 test debt, 812 overview, 49–50 Test Dependency in Production, per-functionality tests, 50–52 220–221 persistent fresh fi xtures, 62–63 Test Discovery shared fi xture strategies, 63–65 introduction, 78 test-driven testability, 66 Lost Tests solution, 271 Index 875

pattern description, 393–398 providing, 140–141 Test Suite Object Generator, 293 retrofi tting testability, Test Suite Objects, 388 146–148 Test Double, 125–151, 521–590 reusing test code, 162 Back Door Manipulation, 332 terminology, 741–744 Behavior Verifi cation, 112 vs. Test Hook, 709–712 Confi gurable Test Double. Test Spy, 137, 538–543 See Confi gurable Test Double Test Stub. See Test Stub confi guring, 141–142 Test-Specifi c Subclass. considerations, 150 See Test-Specifi c Subclass customer testing, 5 types of, 133–134 database testing, 169–171 when to use, 523–526 Dependency Injection. Test Double Class See Dependency Injection example, 572–573 Dependency Lookup, 144–145 implementation, 569–570 dependency replacement, 739 Test Double Subclass design for testability, 7 implementation, 570 Don’t Modify the SUT, 41–42 when to use, 580–581 Dummy Object, 134–135 test drivers example, 526–528 Assertion Messages, 370 Fake Object. See Fake Object defi ned, 813 Fragile Test, 240 test driving, 813 Hard-Coded Test Double. Test Enumeration See Hard-Coded Test Double introduction, 153 Highly Coupled Code pattern description, 399–402 solution, 210 test errors, 80, 813 indirect input and output, test failure, 80, 813 125–126 test fi rst development indirect input control, 128–129 defi ned, 813–814 indirect input, importance process, 4–5 of, 126 test automation philosophy, indirect output, importance of, 32–33 126–127 vs. test-last development, xxxiv indirect output verifi cation, Test Fixture Registry 130–133 accessing Shared Fixtures, 104 installing, 143 Test Helper use, 644 minimizing risk, 25 test fi xtures. See fi xtures Mock Object. See Mock Object Test Helper other uses, 148–150 Automated Teardown, 505 outside-in development, 35–36 introduction, xxiii overview, 522–523 pattern description, 643–647 876 Index

Test Helper Mixin implementation, 349 example, 641–642 invocation, 402 vs. Testcase Superclass, 639 Lost Tests, 269–270 Test Hook minimizing untested code, pattern description, 709–712 44–45 in Procedural Test Stub, organization, 7, 155–158. See 135–136 also test organization patterns retrofi tting testability, 148 overview, 348–349 Test Logic in Production, persistent fi xtures. See persistent 217–219 fi xtures testability, 70 right-sizing, 154–155 Test Logic, Conditional. running, 81 See Conditional Test Logic selection, 404–405 Test Logic in Production Simple Success Test, 349–350 Equality Pollution, 221–222 Simple Success Test example, impact, 217 352–353 introduction, 17 test automation philosophies, 34 symptoms, 217 Test Commands, 82 Test Dependency in Production, Test Concerns Separately, 47 220–221 Test Suite Objects, 82 Test Hooks, 148, 217–219 Testcase Object implementation, For Tests Only, 219–220 384–385 test maintainer, 815 transient fi xture management. Test Method See transient fi xtures calling Assertion. See Assertion unit testing, 6 Method Verify One Condition per Test, Constructor Test example, 46–47 355–357 writing simple tests, 28 Constructor Tests, 351 Test Method Discovery Dependency Initialization defi ned, 394–395 Tests, 352 examples, 395–397 enumeration, 401 Test Object Registry. See Automated Expected Exception Test, Teardown 350–351 test organization, 153–165 Expected Exception Test using code reuse, 162–164 block closure, 354–355 introduction, 153 Expected Exception Test using naming conventions, 158–159 method attributes, 354 overview, 7 Expected Exception Test using right-sizing Test Methods, try/catch, 353–354 154–155 fi xture design, 59 test fi les, 164–165 Index 877

Test Methods and Testcase Test Runner Classes, 155–158 Graphical. See Graphical Test test suites, 160–162 Runner test organization patterns, 591–647 implementation, 378–381 Named Test Suite. See Named introduction, 79 Test Suite Missing Assertion Messages, Parameterized Test. 226–227 See Parameterized Test overview, 377–378 Test Helper, 643–647 Test Automation Frameworks, Test Utility Method. See Test 300 Utility Method test runs, 815 Testcase Class per Class. Test Selection See Testcase Class per Class pattern description, 403–405 Testcase Class per Feature. Test Suite Object, 388 See Testcase Class per Feature test smells, 9–17 Testcase Class per Fixture. aliases and causes, 761–765 See Testcase Class per Fixture behavior. See behavior smells Testcase Superclass, 638–642 catalog of, 12–17 test packages code smells. See code smells defi ned, 815 database testing. See database test fi le organization, 164–165 testing test readers, 815 defi ned, 808, 816 test refactorings. See also refactoring introduction, xxxvi Extractable Test Component, overview, 9–11 735–736 patterns and principles vs., In-line Resource, 736–737 xxxv–xxxvi Make Resources Unique, project smells. See project smells 737–738 reducing Test Code Duplication, Minimize Data, 738–739 114–119 Replace Dependency with Test Test Spy Double, 739 Back Door Verifi cation, 333 Set Up External Resource, 740 Behavior Verifi cation, 113 test results Confi gurable. See Confi gurable defi ned, 815 Test Double introduction, 79–80 examples, 542–543 verifi cation. See result verifi cation implementation, 540–541 Test Run War indirect outputs verifi cation, database testing, 169 179–180 Erratic Tests cause, 235–237 introduction, 131–133, introduction, 15 137, 525 vs. Shared Fixture strategy, 64 motivating example, 541 878 Index

overview, 538–539 when to use, 530–531 Procedural Behavior xUnit terminology, 741–744 Verifi cation, 470 test success, 816 refactoring, 541–542 Test Suite Enumeration when to use, 539–540 defi ned, 400 xUnit terminology, 741–744 example, 402 test strategy patterns, 277–345 Test Suite Factory, 232 Data-Driven Test. See Data- Test Suite Object Driven Test enumeration, 400 Fresh Fixture. See Fresh Fixture Interacting Test Suites, 231–232 Layer Test. See Layer Test introduction, 7, 82 Minimal Fixture, 302–304 pattern description, 387–392 Recorded Test. See Recorded Test Suite Object Generator, 293 Test Test Suite Object Simulator, 293 Scripted Test, 285–287 Test Suite Procedure Shared Fixture. See Shared defi ned, 388–389 Fixture example, 391–392 Standard Fixture. See Standard test suites Fixture defi ned, 816 Test Automation Framework, Lost Tests, 269–270 298–301 Named Test Suites. See Named test strippers, 816 Test Suite Test Stub Test Tree Explorer, 161–162, Behavior-Modifying Subclass, 380–381 584–585 Test Utility Method Confi gurable. See Confi gurable Communicate Intent, 41 Test Double eliminating loops, 121 confi guring, 141–142 example, 605–606 Context Sensitivity solution, 246 implementation, 602–603 controlling indirect inputs, 129 introduction, xxiii, 16–17, 23, creating in-line resources, 737 162–163 examples, 533–537 motivating example, 603–604 implementation, 531–532 Obscure Tests solution, 199 indirect inputs control, 179 overview, 599 inside-out development, 34–35 reducing risk of bugs, 181 introduction, 133, 135–136, 524 refactoring, 605 motivating example, 532–533 reusing, lviii–lix overview, 529–530 reusing via Test Helper, 643–647 refactoring, 533 reusing via Testcase Superclass, unit testing, 6 638–642 Index 879

using TDD to write, 122 Testcase Superclass when to use, 600–602 pattern description, 638–642 Test Utility Test, 603 reusing test code, 163–164 testability, design for. See design- Test Discovery using, 397–398 for-testability test-driven bug fi xing, 812 Testcase Class test-driven development (TDD). introduction, 78 See TDD (test-driven development) organization, 7, 155–158 Test-Driven Development: By pattern description, 373–376 Example (Beck), 301 reusable test logic, 123 test-driven testability, 66 selection, 404–405 Testing by Layers. See Layer Test Testcase Class Discovery testing terminology. See terminology defi ned, 394 test-last development example, 397–398 defi ned, 815 Testcase Class per Class strategy, 65 example, 618–623 test automation philosophy, implementation, 618 32–33 overview, 617 vs. test-fi rst development, xxxiv when to use, 618 TestNG Testcase Class per Feature defi ned, 750 example, 628–630 Interacting Tests, 231 implementation, 626 Testcase Object exception, motivating example, 626–627 384–385 overview, 624 vs. xUnit, 57 refactoring, 627–628 Tests as Documentation when to use, 625 Communicate Intent, 41 Testcase Class per Fixture customer testing, 5 example, 635–637 defi ned, 23 implementation, 632–633 reusing test code, 162 motivating example, unit testing, 6 633–634 Tests as Safety Net, 24, 260 overview, 631 Tests as Specifi cation, xxxiii, 22 refactoring, 634–635 test-specifi c equality, 588–589, 816 Verify One Condition per Test, Test-Specifi c Extension. 46–47 See Test-Specifi c Subclass when to use, 632 Test-Specifi c Subclass Testcase Class per Method, 625 Behavior-Exposing Subclass, Testcase Class per User Story, 625 587 Testcase Object Behavior-Modifying Subclass introduction, 81 (Substituted Singleton), pattern description, 382–386 586–587 880 Index

Behavior-Modifying Subclass transient fi xtures, 85–94 (Test Stub), 584–585 Delegated Setup, 89–91 defi ning Test-Specifi c Equality, hybrid setup, 93 588–589 Implicit Setup, 91–93 Don’t Modify the SUT, 42 In-Line Setup, 88–89 implementation, 581–582 overview, 85–86 Isolate the SUT, 44 vs. persistent fi xtures, 96 motivating example, 582–584 tearing down, 93–94 overview, 579–580 terminology, 86–88 refactoring, 584 what’s next, 94 retrofi tting testability, 146–147 Transient Fresh Fixture State-Exposing Subclass, database testing, 170 289–590 defi ned, 60–61, 314 For Tests Only solution, 220 vs. Shared Fixture, 61–62 when to use, 580–581 troubleshooting Test::Unit, 750 Buggy Tests, 261 Thread-Specifi c Storage, 688–689 Developers Not Writing Tests, Too Many Tests, 256–257 264 tools Erratic Tests, 228–229 automated unit testing, Fragile Tests, 239–240 xxx–xxxi High Test Maintenance Cost, commercial record and playback, 267 282–283 Slow Tests, 253–254 QTP. See QTP (QuickTest True Humble Executable, 703–706 Professional) True Humble Objects, 699–700 robot user. See robot user tools TRUNCATE command. See Table for test automation strategy, Truncation Teardown 53–58 try/catch types of, 753–756 Expected Exception Tests, Transaction Controller, Humble. 353–354 See Humble Transaction Single-Outcome Assertions, 367 Controller try/fi nally block Transaction Rollback Teardown cleaning up fi xture teardown data access layer testing, 173 logic, l–liv defi ned, 100 Implicit Teardown, 519 examples, 673–675 In-line Teardown, 512–513 implementation, 671 type compatibility, 679 motivating example, 672 type visibility overview, 668–669 Test Helper use, 644 refactoring, 672 Test Utility Methods, 603 when to use, 669–671 Testcase Superclass use, 639 Index 881

U Untested Code alternative path verifi cation, UAT (user acceptance tests) 178–179 defi ned, 817 indirect inputs and, 126 principles, 42 Isolate the SUT, 43 UI (User Interface) tests minimizing, 44–45 asynchronous tests, 70–71 preventing with Test Doubles, Hard-To-Test Code, 71–72 523 tools, 55 Production Bugs, 271–272 UML (Unifi ed Modeling unit testing, 6 Language), 816 Untested Requirement Unconfi gurable Test Doubles, 527 Frequent Debugging cause, unexpected exceptions, 352 249 Unfi nished Test Assertion, 494–497 indirect output testing, 127 Unfi nished Test Method from preventing with Test Template, 496–497 Doubles, 523 Unifi ed Modeling Language Production Bugs cause, (UML), 816 272–274 unique resources, 737–738 reducing via Isolate the Unit Testing with Java (Link), 743 SUT, 43 unit tests usability tests, 53 defi ned, 817 use cases, 817 introduction, 6 Use the Front Door First per-functionality, 51 defi ned, 40–41 rules, 307 Overspecifi ed Software Scripted Tests, 285–287 avoidance, 246 xUnit vs. Fit, 290–292 user acceptance tests (UAT) unnecessary object elimination, defi ned, 817 303–304 principles, 42 Unrepeatable Test User Interface (UI) tests database testing, 169 asynchronous tests, 70–71 Erratic Test cause, 234–235 Hard-To-Test Code, 71–72 introduction, 15, 64 tools, 55 persistent fresh fi xtures, 96 user story vs. Repeatable Test, 26–27 defi ned, 817 Untestable Test Code Testcase Class per, 625 avoiding Conditional Logic, utility methods. See Test Utility 119–121 Method Hard-To-Test Code, 211–212 utPLSQL, 750 882 Index

V visibility of SUT features from Test- value patterns, 713–732 Specifi c Subclass, 581–582 Derived Values, 718–722 test fi le organization, 165 Dummy Objects, 728–732 type. See type visibility Generated Values, 723–727 visual objects, Humble Dialog Literal Values, 714–717 use, 706 variables Visual Studio, 756 in Derived Values, 718–722 global, 92, 798 W instance. See instance variables local. See local variables waterfall design, 65 procedure variables, 805–806 Watir static, 809 defi ned, 756 VB Lite Unit, 751 Test Automation Frameworks, VbUnit 301 defi ned, 751 test automation tools, 53 Suite Fixture Setup support, 442 Weinberg, Gerry, xxiv–xxv, 61–62 Testcase Class terminology, 376 widgets xUnit terminology, 300 Humble Dialog use, 706 Verbose Tests. See Obscure Test recognizers, 299 verifi cation Wikipedia, 729 alternative path, 178–179 Working Effectively with Legacy Back Door Manipulation, Code (Feathers), 210 329–330 Write the Tests First, 40 Back Door using Test Spy, 333 writing tests cleaning up logic, xlvi–l Developers Not Writing Tests direct output, 178 project smells, 263–264 indirect outputs, 130–133, development process, 4–5 178–180 goals, 27–29 state vs. behavior, 36 philosophies. See test automation test results. See result verifi cation philosophies Verify One Condition per Test, principles. See test automation 45–47 principles Verifi cation Method defi ned, 477, 602 X example, 482–483 XML data fi les, Data-Driven Tests, Verify One Condition per Test 294–295 defi ned, 40, 45–47 xUnit right-sizing Test Methods, Data-Driven Tests with CSV 154–155 input fi le, 296 verify outcome, 817 Data-Driven Tests with XML Virtual Clock, 246 data fi le, 294–295 Index 883

defi ned, 751 fi xtures, 78 family members, 747–751 overview, 75–76 vs. Fit, 291–292 procedural world, 82–83 fi xture defi nitions, 86 running Test Methods, 81 Interacting Test Suites, 232 running tests, 79 introduction, 56–57 Test Commands, 82 language-specifi c terminology, test results, 79–80 xl–xli Test Suite Object, 82 modern, 55 xUnit basics patterns, 347–405 Naive xUnit Test Interpreter, Assertion Message, 370–372 292–293 Assertion Method. profi ling tools, 254 See Assertion Method Suite Fixture Setup support, Four-Phase Test, 358–361 442–443 Test Discovery, 393–398 sweet spot, 58 Test Enumeration, 399–402 terminology, 741–746 Test Method. Test Automation Frameworks, See Test Method 300 Test Runner. test fi xtures, 814 See Test Runner test organization mechanisms, Test Selection, 403–405 153 Test Suite Object, 82, xUnit basics, 75–83 387–392 defi ning suites of tests, 78–79 Testcase Class, 373–376 defi ning tests, 76–78 Testcase Object, 382–386