Story Test Driven Development

Home , Cowboy coding, Make (software), Ward Cunningham

Test andMeasure SOFTWARE SOFTWARE 18 IS AN EXTREME PROGRAMMING PRACTICE. (STDD) DEVELOPMENT STORYTEST-DRIVEN ETRSFWR JULY/AUGUST 2004 BETTER SOFTWARE “cowboy coding,” or “snake oil,” or is STDD that say Opponents test.” “story- executable an producing ment) and fleshes out that story by require-a of idea rough (or story a takes team a written, is code any before that is premise basic The BREAK DON’T MAKE JUST www.stickyminds.com

PRESTON MACK/REDUX PLUS . . work. developers and customers, QA, way the changing is development driven storytest- How by Tracy Reppert STDD rates high with the development team at Nielsen Media Research. Standing (l to r): Thomas Hund, Praseed Thapparambil, and Pam Smoot. Seated (l to r): Cheryl Redding, Rhoda Foxworthy, and Merrilee Albright Test and Measure

specific contract for acceptance, so dis- crepancies are quickly resolved. There’s a “Creating the expected results clear context for customers and develop- was quite challenging, but it ers to have a conversation and weed out was rewarding when they ran.” misunderstandings. The risk of building —MERRILEE ALBRIGHT the wrong system is much lower.” Ken Auer, president of RoleModel Software and one of the earliest practi- just “a hindrance to real work.” Propo- expectation of the customer into an exe- tioners of STDD, agrees. His team, along nents argue that STDD produces the sim- cutable and readable contract that pro- with Ward Cunningham, first used plest system possible and is the wave of grammers have to obey if they want to STDD to produce a software system in the future, the new frontier. claim a working system. 1999. “STDD made it easier to nail “STDD gets the right people collabo- down what our end-users wanted,” re- EVERYONE GETS A SAY rating at the right time,” says Joshua calls Auer. “By having executable re- With traditional acceptance testing, de- Kerievsky, XP coach and founder of In- quirements for each story, we were able velopers would create an entire system, dustrial Logic, Inc. STDD brings together to know when we had completed stories, declare it finished, and then submit it for the customer, developers, and testers be- which made project tracking a whole lot acceptance testing. This mostly manual fore any code has been written. They col- easier and better.” testing tended to get bogged down in pa- laborate to identify a specific piece of perwork and occurred so late in the de- functionality, or “story,” to be worked NO ONE HAS TO BE velopment lifecycle that any mistakes on. Customers and testers then specify THE BAD GUY were very expensive to fix. Even if the criteria to validate that the story works Raha explains that not only does STDD system met all of the stated requirements and create an executable document that relieve stress; all the involved parties love and passed through acceptance testing can be accessed by anyone on the team. the process. “Customers love it because with flying colors, the customer, who “When testing is done after the fact, of the new level of confidence they get— hadn’t seen the system since the require- customers simply say that the system is instead of just hoping the development ments meeting, often was left feeling like broken when it doesn’t meet their require- team has understood the requirements he had not gotten what he wanted. ments,” says Somik Raha, XP coach for correctly. Developers love it because STDD makes it possible to formalize the Industrial Logic. “With STDD, there is a there’s no way they can be accused of

from being lost in abstract discussions and vague generalities. A GLIMPSE To illustrate, let’s suppose a business has a rule that, for ac- counting purposes, all months are treated as having thirty days. OF STDD (Some bonds are valued under a similar rule.) One of the stories might be “Implement the 30-day rule.” The business expert IN ACTION. might write something like this on the whiteboard: by 1 April to 1 May is 30 (as normal) Brian 1 Feb to 1 March is 30 (even though 28) Marick 1 Jan to 1 Feb is 30 (really 31)

QUESTIONS should follow. A programmer or tester might ask if STORYTESTS AS PROPS FOR CONVERSATION it’s really true that the number of days from 1 January to both In order to estimate a story, programmers have to understand 1 February and 31 January is thirty. The business expert would what the business expert wants. They learn this through face- answer that it is. Then someone else might ask, “So, it’s twen- to-face conversation. The business expert describes the feature, ty-seven days from 1 February to 11:59 PM on 28 February, but explains why it’s useful, and gives examples, perhaps scribbling the next minute it’s thirty days?” The business expert might them on a whiteboard. The programmers ask questions, split start to answer “Yes,” only to be interrupted by someone say- the feature into constituent parts (smaller tasks), ask more ing, “Not if it’s a leap year,” which would lead to a discussion questions about those tasks, and sometimes propose variations of the relevance of leap years, and then, perhaps, to whether on the feature that might be simpler to implement. time zones matter. That discussion might well be cut short, because the point EXAMPLES are particularly useful because they ground the con- here is for the programmers to have enough information to es- versation in something concrete and keep important details timate accurately, not for them to know every last detail.

20 BETTER SOFTWARE JULY/AUGUST 2004 www.stickyminds.com Test and Measure building a system that is not acceptable. THE FIRST STEP CAN helps teams obtain the necessary amount And QA loves it because programmers BE A BIG ONE of story details when they need it—before aren’t viewing them as the bad guys,” If STDD is so great, why do so many peo- writing code. says Raha. ple distrust it? Perhaps some of the skep- Even now that most of the quirks STDD challenges the notion that QA tics of STDD haven’t experienced it since have been worked out, people may still has to play a rival role to a programming its inception. XP and STDD definitely had be wary of such a big change. “It was dif- team if they expect to be able to spot de- some growing pains. Joshua Kerievsky ex- ficult to trust the process in the begin- fects. In practice, teams find that they plains, “In the early days of XP, we did a ning,” says Pam Smoot, senior project are saving a lot of time by having QA lot of unit test-driven development and manager at Nielsen Media Research, help produce storytests from the begin- would automate storytests only after we’d which began implementing STDD for a ning. “A tester is sometimes viewed as written our code. The trouble with this major data-warehousing project, “but it’s an adversary, when all they are trying to approach was that it was often hard to get so much better than what we used to do. do is ensure that the delivered product subject matter experts to define storytests, Having expected results created upfront functions well,” says Merrilee Albright, lead SQA analyst at Nielsen Media Re- search and member of an STDD team. “It was difficult to trust “I’ve been testing a long time, and have the process in the beginning, found that my role on this team definite- but it’s so much better than ly has more respect than it might in the normal software development process. I what we used to do.” —PAM SMOOT can utilize the expertise of other team members to help me create tests. Having the customer available to review the test which meant that with successive itera- by a business/QA person helped us find results and make suggestions immediate- tions we would get further and further be- miscalculations, misinterpretations, and ly has proved invaluable. Creating the hind on the number of storytests we still miscommunications right away rather expected results was quite challenging, needed. That produced higher defects, than late in the project.” but it was rewarding when they ran and since the lack of sufficient story details led Skeptics begin to see the value of ensured that the system was functioning to code that failed to fully satisfy customer STDD in just a couple of weeks, accord- as expected.” expectations.” He adds that STDD now ing to Smoot, though it can take a few

CONVERSATIONAL EXAMPLES getting those tests to pass, other people can contin- BECOME STORYTESTS ue writing tests. The test-driven style is to make tests

Once the iteration is planned, it’s the job of the programmers pass one at a time; while working on one test, the continued to produce features that satisfy the business expert, and it’s programmer doesn’t worry overmuch about what > the job of the business expert to support them—mainly by be- tests remain. So it doesn’t matter if some of those ing available for further conversation. For example, a program- tests are not yet known. These storytests do not necessarily mer might realize that she doesn’t know what the program completely replace conventional testing. They’re designed to should do in some special case, so the business expert has to build quality into the product. Someone else still needs to cre- tell her. Or a programmer might realize that the code is so ate tests to detect where the storytests fell short of the goal structured that it would be trivial to add a new wrinkle to the of preventing bugs. feature, so the business expert should tell her if the new wrinkle is a good idea. The business expert should also help with the tests. The CalendarFixture table at right shows a starting set of tests that came up in the from to elapsed() actual() imaginary planning meeting we mentioned earlier. (Ignore the 1 Apr 2003 1 May 2003 30 30 first row for now.) Who writes that table? It’s entirely up to the team. Some 1 Feb 2003 1 Mar 2003 30 28 business experts might be happy writing such tables them- 1 Jan 2003 1 Feb 2003 30 31 selves. In other cases, it might be dedicated testers or the pro- 1 Jan 2003 31 Jan 2003 30 30 grammers. What’s important is that the tests get written effi- ciently and that the business expert believes that seeing them 1 Feb 2003 28 Feb 2003 27 27 pass will increase his confidence in the program. 1 Feb 2004 29 Feb 2004 28 28 This set of tests needn’t be complete. It need only be 1 Feb 2004 1 Mar 2004 30 29 enough for the programmers to get started. As they work on

www.stickyminds.com JULY/AUGUST 2004 BETTER SOFTWARE 21 Test and Measure

compromised if, for example, new developers don’t want to learn a method, and “There’s really no situation their managers—not wanting to upset in which I can see STDD not them—don’t want to enforce it.” being the best process to use.” The key to making the change is just —THOMAS HUND getting people to try things, explains Cun- ningham. Developers who do try STDD tend to love the influence it has on a soft- months to fully absorb it, explains her fel- continues, “It’s just one of those quirks ware design. “When I started doing low team member, product manager Rho- of human organization. Attitude is based STDD, I was amazed at how much simpler da Foxworthy. “Now everyone on the on experience, and eventually people my code became, versus the code I’d writ- team really sees its value and is excited shift their attitudes and realize this ten using either upfront design or minimal about taking it outside the team, although method is to their advantage.” design with unit test-driven development,” we’re not sure how,” says Foxworthy. Considering such human quirks is key says Kerievsky. “Following STDD closely “Our next big challenge, when our plat- to successfully integrating STDD. “Peo- enabled me to produce designs I simply form team is retired, is continuing its spir- ple always resist paradigm shifts,” says would not have anticipated.” it and use in a production environment Raha. “So it’s especially important to in- “There’s really no situation in which where there are walls and naysayers.” troduce STDD when a project has just I can see STDD not being the best begun and people are still motivated.” process to use,” says Thomas Hund, se- CHANGE IS HARD— As with any XP practice, keeping the nior SQA analyst at Nielsen Media Re- BUT WORTHWHILE STDD process going can be challenging search. “As well as it’s worked for us, Many STDD practitioners encounter an if it’s considered to be merely an option I’m a believer.” {end} ironic reaction to their success: resent- and not a rule. “STDD should be fol- ment from the rest of the organization. lowed as a policy from the top down,” Tracy Reppert is a freelance journalist “It’s weird that that happens, but it says Narkeeran Narasimhan, a testing who has written about technology for does,” says Ward Cunningham, whose specialist for Ansys Corp., which devel- the New York Times, the San Francisco FIT tool is typically used for STDD be- ops engineering simulation software and Chronicle, PC Magazine, and Wired. She cause team members with diverse back- has been doing XP since 2002. “Over a lives in Berkeley and can be reached at grounds can understand it. Cunningham period of time, these processes can get [email protected].

STORYTESTS BECOME CODE gaining the reassurance of seeing a new test pass and all pre- Making a table of storytests pass requires two steps. vious tests continue to pass. So they will break their coding

continued task into smaller pieces. They’ll determine a chunk of code > THE FIRST STEP is to write a fixture that translates the that needs to change, write a programmer test that describes table into executions of the program. (In the case of the change (using a programmer testing tool like JUnit or the example in the sidebar on page 21, the fixture is named by NUnit), make the change, and check it. They’ll proceed that the first row of the table: CalendarFixture.) Usually, the fixtures way, test by test and chunk by chunk, until the larger story- do not drive the graphical user interface. Instead, they go “be- test passes. low the GUI” and make calls to the business logic of the pro- This process continues until all the tests for the story pass. gram in the same way that the GUI does. This makes tests Notice that these tests may change at any time before the end. more maintainable, and it also helps to align the language of Any one of the business experts, testers, or programmers may the program with the language of the business. (If business add or change a test if it helps complete the story more surely, people talk of buying bonds or negotiating leases, it makes quickly, or smoothly. However, the business expert is the final sense for the code to use words like “bond” and “lease” and arbiter over whether a test is a correct example of the feature. “buy”. This style of programming is called “domain-driven de- When all the tests pass, the story is done, and a new story can sign,” after Eric Evans’s book of the same name.) be started.

THE SECOND STEP is to make the test pass. In some cases (like Brian Marick has worked in testing since 1981. He is the au- our calendar example), that’s straightforward. The programmer thor of The Craft of Software Testing, and is the technical edi- runs the table, sees that a row fails, changes the code, runs tor for Better Software magazine. Contact Brian at marick@ the table again to see that the row now passes. testing.com. Read other writings at testing.com. (Author’s In other cases, the changes needed to make the tests run note: Cheryl Redding and Praseed Thapparambil of Nielsen are too large for that. Fans of test-driven design like to run Media Research, and Chris Wheeler of Sciex contributed to tests often. They don’t like to make many changes before this series of sidebars.)

22 BETTER SOFTWARE JULY/AUGUST 2004 www.stickyminds.com Test and Measure

such as sentences to describe a story or a description of a GETTING formula that is being exercised by a storytest. It takes some time to learn how to best model a story- STARTED test in a tabular form using Ward Cunningham’s FIT framework (http://fit.c2.com). “You need a day or two, WITH STDD. unless you’ve never done Java,” says Cunningham. Ward’s site offers a number of open source sample applications that let you practice without getting confused with your is- So how does a team get started doing STDD? For Nielsen sues at work. Cunningham recommends trying to under- Media Research, changing to STDD was part of the shift stand the music player, use the tables there to make it do to XP. First, the main players and decision makers got to- something new, and then apply it directly to your work. gether and held a two-day overview boot camp. Once Once you have enough experience with that, decide they had senior management’s approval, they formed the whether to use a Column Fixture, Row Fixture, or Ac- group and moved into an open workspace soon after- tion Fixture. In most cases, whatever tabular form allows wards. Next, they bought books on XP for all team you to present the storytest in the most readable format members and held a five-day intensive XP boot camp. is probably best, because the main audience for a story- They retained XP experts as trainers and coaches for the test is nontechnical people. In practice, the easier it is to first four months. Their QA manager continuously found read a storytest, the more work programmers will have and fed them pertinent articles. Finally, they held a two- to do behind the scenes to map the storytest to the classes day intensive retrospective that delved into the pluses and in the production code. That’s okay because once pro- minuses of their first thirteen weeks. grammers begin to write such mapping code, it will be- XP coach Joshua Kerievsky offers these specific steps come increasingly easier to do so for future storytests. for getting started with STDD: 3 1 Check In Storytests Define Stories It is best to check in storytests to your version control soft- Work with subject matter experts and analysts to define ware as soon as they’ve been created. We do this as soon as stories: roughly defined chunks of functionality. We typi- a storytest has been defined. When developers integrate cally use index cards for stories. Good story headings are their code, they ought to see that new storytests have been no longer than five words and use the active voice. For ex- checked in. This informs them that they can begin working ample: “Book a Room,” “Calculate Capital for Term on the story(ies) associated with the checked in storytest(s). Loan,” or “Export Risk Report into Excel” are good story In general, programmers should not begin programming a headings. The details of a story give enough information story until they have a storytest that is failing. for programmers to make decent time estimates for implementing stories. For the story “Book a Room,” the details might be “Allow user to make a reservation for a single 4 room, assuming no special rate discounts or late check- outs.” For more details on how to write good stories, see Program the Storytests the new book User Stories Applied by Mike Cohn. Once developers have a storytest, they will begin writing test code to show that the storytest is failing. If the team is using FIT, this involves defining the appropriate sub- 2 class of FIT fixture class and then delivering production code to make the storytest pass. During this time, it is Define Storytests typical and optimal to do unit test-driven development in Once stories have been defined, you can begin writing conjunction with STDD. This helps you to evolve pro- storytests. This is best done through the collaboration of duction code that has sufficient test coverage. The combi- subject matter experts, testers or QA persons, and pro- nation of these testing techniques leads to fine-grained grammers. A storytest helps verify that a story works cor- unit tests for classes and medium- to large-grained story- rectly by documenting what inputs are supplied to a sys- tests for the features of the system. Throughout this tem and what outputs are expected. A document that process, programmers check in code, unit tests, and contains a storytest can include other useful information, storytests when they are passing.