COVER FEATURE Test-Driven Development: Concepts, Taxonomy, and Future Direction

Test-driven development creates software in very short iterations with minimal upfront design. Poised for widespread adoption, TDD has become the focus of an increasing number of researchers and developers.

David Janzen he test-driven development strategy method, a close examination of the term reveals a Simex LLC requires writing automated tests prior to more complex picture. developing functional code in small, rapid Hossein iterations. Although developers have been The test aspect Saiedian T applying TDD in various forms for several In addition to testing, TDD involves writing auto­ University decades,1 this software development strategy has mated tests of a program’s individual units. A unit of Kansas continued to gain increased attention as one of the is the smallest possible testable software component. core extreme programming practices. There is some debate about what exactly constitutes XP is an agile method that develops object-ori­ a unit in software. Even within the realm of object- ented software in very short iterations with little oriented programming, both the class and method upfront design. Although not originally given this have been suggested as the appropriate unit. name, TDD was described as an integral XP prac­ Generally, however, the method or procedure is the tice necessary for analysis, design, and testing that smallest possible testable software component. also enables design through refactoring, collective Developers frequently implement test drivers and ownership, continuous integration, and programmer function stubs to support the execution of unit tests. courage. Test execution can be either a manual or automated Along with pair programming and refactoring, process and can be performed by developers or des­ TDD has received considerable individual attention ignated testers. Automated testing involves writing since XP’s introduction. Developers have created unit tests as code and placing this code in a test har­ tools specifically to support TDD across a range of ness or framework such as JUnit. Automated unit languages, and they have written numerous books testing frameworks minimize the effort of testing, explaining how to apply TDD concepts. Re­ reducing a large number of tests to a click of a but­ searchers have begun to examine TDD’s effects on ton. In contrast, during manual test execution devel­ defect reduction and quality improvements in aca­ opers and testers must expend effort proportional demic and professional practitioner environments, to the number of tests executed. and educators have started to examine how to inte­ Traditionally, unit testing occurred after devel­ grate TDD into computer science and software opers coded the unit. This can take anywhere from engineering pedagogy. Some of these efforts have a few minutes to a few months. The unit tests might been implemented in the context of XP projects, be written by the same programmer or by a desig­ while others are independent of them. nated tester. With TDD, the programmer writes the unit tests prior to the code under test. As a result, TEST-DRIVEN DEVELOPMENT DEFINED the programmer can immediately execute the tests Although its name implies that TDD is a testing after they are written.

0018-9162/05/$20.00 © 2005 IEEE P u b l i s h e d b y t h e I E E E C o m p u t e r S o c i e t y September 2005 43 The driven aspect and what that program’s interface should look like Some definitions of TDD imply that it is is a radical concept for most software developers. TDD assumes primarily a testing strategy. For example, in that the software JUnit in Action (Manning Publications, The development aspect design is either 2003), Vincent Massol and Ted Husted stated Intended to aid in constructing software, TDD incomplete that is not in itself a software development methodol­ ogy or process model. It is a practice, a way of or pliable Test-driven development (TDD) is a program­ developing software to be used with other prac­ and open ming practice that instructs developers to write tices, in a particular order and frequency and in the to changes. new code only if an automated test has failed, context of a process model. TDD has emerged and to eliminate duplication. The goal of TDD within a particular set of process models. It can be is ‘clean code that works.’ applied as a microprocess within the context of many different process models. However, according to XP and TDD pioneer TDD produces a set of automated unit tests that Ward Cunningham, “Test-first coding is not a test­ provide some side effects in the development ing technique.” TDD is known by various names process. The practice assumes the automated tests including test-first programming, test-driven will not be thrown away once a design decision is design, and test-first design. The driven in test-dri­ made. Instead, the tests become a vital component ven development focuses on how TDD leads analy­ of the development process, providing quick feed­ sis, design, and programming decisions. TDD back to any changes to the system. If a change assumes that the software design is either incom­ causes a test to fail, the developer knows immedi­ plete or pliable and open to changes. In the context ately after making the change, while the test is still of XP, TDD subsumes many analysis decisions. The fresh in the developer’s mind. Among the draw­ customer should be “on-site” in XP. Test writing is backs, that developer must now maintain both the one of the first steps in deciding what the program production code and the automated tests. should do, which is essentially an analysis step. The Agile Alliance offers another definition that cap­ TDD’S HISTORICAL AND MODERN CONTEXTS tures this idea (www.agilealliance.org/programs/ Despite the lack of attention in undergraduate roadmaps/Roadmap/tdd/tdd_index.htm): curriculum and inconclusive reports of usage in industry, a wide range of software tools exist to sup­ Test-driven development (TDD) is the craft of pro­ port testing, making TDD’s emergence possible. ducing automated tests for production code, and using that process to drive design and program­ Software development methodologies ming. For every tiny bit of functionality in the pro­ A software development process or methodol­ duction code, you first develop a test that specifies ogy defines the order, control, and evaluation of the and validates what the code will do. You then pro­ basic tasks involved in creating software. Software duce exactly as much code as will enable that test process methodologies range in complexity and to pass. Then you refactor (simplify and clarify) control from largely informal to highly structured. both the production code and the test code. Developers classify these methodologies as pre­ scriptive or agile and label the specific types as To promote testing to an analysis and design step waterfall, spiral, incremental, or evolutionary. the practice of refactoring must be introduced. When an organization states that it uses a par­ Refactoring changes the structure of an existing ticular methodology, it often applies a combination body of code without changing its external behav­ of smaller, finer-grained methodologies on a pro­ ior. A test may pass, but the code may be inflexible ject scale instead. For example, an organization or overly complex. By refactoring the code, the test might apply an incremental development model, should still pass but the code will be improved. building small, cumulative slices of a project’s fea­ Understanding that TDD is more about analysis tures. In each increment, the developers could apply and design than it is about testing is one of the most a waterfall or linear method of determining require­ challenging conceptual shifts for new adopters of ments, designing a solution, coding, testing, and the practice. Program testing has traditionally then integrating. Depending on the size of the incre­ assumed the existence of a program. The TDD idea ments and the waterfall’s time frame, the process that a test can be written before the program or that could be labeled differently, with potentially very test can aid in deciding what program code to write different results in terms of quality and developer

44 Computer satisfaction. If we break a software project into N can vary widely. Some testing can be con­ increments where Ii represents each increment, then ducted early—concurrent with other coding The order the equation ΣN I can represent the entire project. activities. Test-driven development reorders i = 1 i in which If N is reasonably large, we can label this as an these steps to an advantage. By placing fine- incremental project. However, if N ≤ 2, we would grained unit tests before just enough code to construction label this as a waterfall project. satisfy that test, TDD can affect many aspects tasks occur If the increments require modifying a significant of a software development methodology. influences a amount of overlapping software, we can say that project’s label our methodology is more iterative in nature. TDD’s historical context and its quality. Specifically, for project P consisting of code C and Test-driven development has emerged in ΣN iterations I= i = 1Ii, if Ci is the code affected by iter­ conjunction with the rise of agile process ∩ ation Ii, and if project P is iterative, then Ci Ci + 1 models. Both have roots in the iterative, incre­ ≠ Θ for most i such that 1 < i < N. mental, and evolutionary process models used as Similarly, with incremental and waterfall ap­ early as the 1950s. In addition, tools have evolved proaches, we expect a formal artifact, such as a spec­ to play a significant role in supporting TDD. ification document, for documenting the increment’s Early test, early examples. Research on testing has requirements. If the artifact is informal—say, some generally assumed the existence of a program to be whiteboard drawings or an incomplete set of UML tested, implying a test-last approach. Moving tests diagrams and was generated quickly—we would be from the end of coding to the beginning, however, working in the context of an agile process. The is nothing new. Software and test teams commonly approach and perspective of the architecture or the develop tests early in the software development design would cause us to label the process aspect- process, often with the program logic. The evalu­ oriented, component-based, or feature-driven. ation and prevention life-cycle models integrated Some individual software developers and smaller testing early in the software development process teams apply even finer-grained models such as the nearly two decades ago. Introduced in the 1980s, personal software process or the collaborative soft­ the Cleanroom approach to software engineering ware process. The time, formality, and intersection included formal verification of design elements of the steps in software construction can determine early in the development process. Some claim that the way developers categorize a process method­ NASA’s Project Mercury applied a form of TDD as ology. early as the 1950s.1 The order in which construction tasks occur influ­ Prior to the introduction of XP in 1998, little had ences a project’s label and its quality. Natural and been written about the concept of letting small logical, the traditional ordering is requirements elic­ incremental automated unit tests drive software itation, analysis, design, code, test, integration, development and design processes. Despite the lack deployment, and maintenance. We could, however, of published documentation, many developers have consider some possible reorderings even though probably used a test-first approach informally. Kent most do not make sense. For example, we would Beck claims he “learned test-first programming as never maintain a system that hasn’t been coded. a kid while reading a book on programming. It said Similarly, we would never code something for which that you program by taking the input tape ... and we have no requirements. typing in the output tape you expect. Then you pro­ Requirements do not necessarily imply formal gram until you get the output tape you expect.”2 requirements. A requirement can be as simple as an Some argue that TDD merely gives a name and idea in a programmer’s head. Programmers have definition to a practice that has been sporadically applied the prototyping approach when require­ and informally applied for some time. TDD is more ments are fuzzy or incomplete. With this approach, than this. As Beck states, XP takes the known best we may do very little analysis and design before cod­ practices and “turns the knobs all the way up to ing, but ultimately might have to discard the proto­ ten.”2 Many developers might have been thinking type even though it was a useful tool in determining and coding in a test-first manner, but TDD does requirements and evaluating design options. this in an extreme way: always writing tests before When we closely examine the design, code, and code, making tests as small as possible, and never test phases, we see many finer-grained activities. letting code degrade. TDD fits within a process For example, various types of testing take place, model, and the development of incremental, itera­ including unit, integration, and regression testing. tive, and evolutionary process models has been vital The timing, frequency, and granularity of these tests to its emergence.

September 2005 45 Iterative, evolutionary, and incremental develop­ Essential for implementing TDD with Java, JUnit ment. Iterative development involves repeat- is arguably responsible for much of TDD and XP’s TDD developed ing a set of development tasks, generally wide popularity. JUnit-like frameworks have been within the on an expanding set of requirements.1 implemented for several different languages, cre­ context of Evolutionary approaches involve adaptive ating a family of frameworks referred to as xUnit. iterative, and lightweight iterative development. Generally, xUnit lets a programmer write sets of incremental, Being adaptive refers to using feedback from automated unit tests that initialize, execute, and previous iterations to improve the software. make assertions about the code under test. and evolutionary Being lightweight refers to the lack of complete Individual tests are independent of one another so models. specifications at the beginning of development, test order does not matter. The programmer reports thus allowing feedback from previous itera­ the total number of successes and failures. xUnit tions and from customers to guide future iter­ tests are written in the same language as the code ations. Lightweight can also refer to other under test and thus serve as first-class clients of the aspects such as a process’s level of formality and code, while tests can actually serve as documenta­ degree of documentation. The spiral model is an evo­ tion for it. lutionary approach that incorporates prototyping On the other hand, because developers imple­ and the cyclic nature of iterative development with ment xUnit in the target language, that language risk-driven-iterations and anchor point milestones. determines the tool’s relative simplicity and flexi­ The incremental model produces a series of releases, bility. For example, JUnit is simple and portable called increments, that provide more functionality partly because it takes advantage of Java’s porta­ with each increment. bility through the bytecode/virtual machine archi­ TDD developed within the context of such itera­ tecture. It uses Java’s ability to load classes tive, incremental, and evolutionary models. TDD dynamically and exploits Java’s reflection mecha­ works because these approaches provide the pre­ nism to automatically discover tests. In addition, requisite process models. Beck claims that to imple­ JUnit provides a portable graphical user interface ment XP, developers must apply all of the that has been integrated into popular integrated incumbent practices—leaving some out weakens the development environments such as Eclipse. model and can cause it to fail.3 TDD requires that A wide range of additional tools have emerged to design decisions be delayed and flexible to influence support automated testing, particularly in Java. software design. Each new test might require refac­ Some tools simplify the creation of mock objects, toring and a design change. Automated tests give or stubs. The stubs replace the needed collaborat­ programmers the courage to change any code and ing objects so that developers can test a particular the information they need to know quickly if some­ object. They can use other tools such as Cactus and thing has broken, enabling collective ownership. Derby with JUnit to automate tests that involve J2EE components or databases. Automated testing The proliferation of software tools that support Software tools have become important factors in TDD shows that it has widespread support and will the development of modern software systems. Tools likely become an established approach. JUnit’s sim­ ranging from compilers, debuggers, and integrated plicity and elegance have been a significant factor development environments to modeling and com­ in TDD’s use, particularly in the Java community. puter-aided software engineering tools have Programmers can develop unit tests easily and exe­ improved and increased developer productivity. cute large test suites with a single button click, Tools have played an important role in the emer­ yielding quick results about the system’s state. gence of TDD, which assumes the existence of an automated unit testing framework. Such a frame­ Early testing in academia work simplifies both the creation and execution of The undergraduate computer science and soft­ software unit tests. Test harnesses, basically auto­ ware engineering curriculum provides one indica­ mated testing frameworks, provide a combination tor of a software practice’s widespread acceptance. of test drivers, stubs, and interfaces to other sub­ Sometimes academia has led practice in the field, systems. Often such harnesses are custom-built, sometimes it has followed. Software engineering, although commercial tools do exist to assist with iterative development, and TDD all seem to follow test harness preparation. this latter model. Erich Gamma and Kent Beck developed JUnit, Much software engineering research originated an automated unit testing framework for Java. in academia and then found its way into common

46 Computer practice. The undergraduate computer science and methods. Pekka Abrahamsson and col­ software engineering curriculum, however, tends leagues4 provide an evolutionary map of to reflect or even lag behind common practice in nine agile methods and describe how they XP proposes industry. The choice of programming language has focus on simplicity and speed while empha­ using TDD as commonly followed business needs. Process mod­ sizing people over processes. an integral els developed in practice later become reflected in Probably the most well-known agile component curricula. method, XP is often used in combination for developing The 1991 ACM Curriculum Guidelines recom­ with other agile methods such as Scrum. XP mended giving fewer than eight hours each of lec­ proposes using TDD as an integral compo­ high-quality ture and lab time to iterative development processes nent for developing high-quality software. software. and verification and validation. The 2001 guide­ The highly disciplined practice of TDD and lines recommended giving an even smaller amount the simple, lightweight nature of agile of time to development processes and software val­ processes give rise to an interesting conflict. idation—two and three hours each, respectively. Potential TDD adopters often express concern Undergraduate texts give little attention to com­ regarding the time and cost of writing and main­ parative process models. Texts provide limited cov­ taining unit tests. Although he concedes that auto­ erage of software design and testing techniques. mated unit tests are not necessary for absolutely The topics of software design and testing are often everything, Beck insists that XP cannot work with­ relegated to a software engineering course that may out TDD because it provides the glue that holds not be mandatory for all students. the process together.3 Extreme programming’s place in undergraduate Adoption measures. Measuring the use of a partic­ education has been the topic of much debate. Some ular software development methodology is hard. argue strongly for using XP to introduce software Many organizations might be using the method­ engineering to undergraduates. Others argue that XP ology without talking about it. Others might claim and agile methods offer only limited benefits. Given to be using a methodology when in fact they are the different opinions on using XP in the undergrad­ misapplying it. Worse yet, they might be advertis­ uate curriculum, TDD has received limited exposure ing its use falsely. Surveys might be conducted to at this level. Some educators have called for increased gauge a method’s usage, but often only those who design and testing coverage. Others see TDD as an are much in favor or much opposed to the method­ opportunity to incorporate testing in the curriculum, ology will respond. rather than relegating it to an individual course. A 2002 survey reported that out of 32 survey TDD tools have, however, found their way into respondents across 10 industry segments, 14 firms early programming education. BlueJ, a popular used an agile process.5 Of these, five were in the environment for learning Java, incorporates JUnit e-business industry. Most of the projects using agile and adds help for building test cases at an early processes were small, involving 10 or fewer partic­ stage in a programmer’s learning cycle. Proponents ipants and lasting one year or less. A 2003 survey have advocated using JUnit for early learning of reported that 131 respondents claimed they used an Java because it abstracts the bootstrapping mech­ agile method.6 Of these, 59 percent claimed to be anism of main(), allowing the student to concen­ using XP and implied they were using TDD. Both trate on the use of objects early. surveys revealed positive results from applying agile TDD has yet to achieve widespread acceptance in methods, with increases in productivity and qual­ academia, at least partly because faculty who do ity and reduced or minimal changes in costs. not specialize in software engineering are not likely XP has accumulated a substantial body of litera­ to be familiar with it. TDD instructional materials ture. Most of this involves the promotion of XP or that target undergraduate courses remain basically explains how to implement it. Many experience nonexistent. reports present only anecdotal evidence of XP’s ben­ efits and drawbacks. Although the existence of these Recent context reports indicates that XP is being adopted in many XP and agile methods have received much atten­ organizations, it remains unclear if these same orga­ tion in the past few years. Even though conclusive nizations will continue to use XP over time or, if documentation is lacking, anecdotal evidence indi­ they have, if they will move on to other methods. cates that TDD usage is rising. Although XP’s popularity implies a growing Agile methods. These methods clearly have roots adoption of TDD, we have no idea how widely it is in the incremental, iterative, and evolutionary being used. Organizations may be using XP with-

September 2005 47 Table 1. Summary of TDD research in industry.

Number of Number of Study Type companies programmers Quality effects Productivity effects

George8 Controlled experiment 3 24 TDD passed 18% more tests TDD took 16% longer Maximilien9 Case study 1 9 50% reduction in defect density Minimal impact Williams10 Case study 1 9 40% reduction in defect density No change

out adopting all of its practices or they may be TDD in industry applying the practices inconsistently. On a project at A few evaluative research studies have been con­ ThoughtWorks, Jonathan Rasmusson, an early XP ducted on TDD with professional practitioners. adopter, estimates one-third of the code was devel­ North Carolina State University seems to be the only oped using TDD.7 In the same report, Rasmusson source of such a study to date. Researchers at NCSU stated, “If I could only recommend one coding prac­ have performed at least three empirical studies on tice to software developers, those who use XP or TDD in industry settings involving fairly small otherwise, it would be to write unit tests.” groups in at least four different companies.8-10 These In this ThoughtWorks project, developers used studies examined defect density as a measure of soft­ 16,000 lines of automated unit tests on 21,000 lines ware quality, although some survey data indicated of production code. Many tests were written in that programmers thought TDD promoted simpler both test-first and test-last iterations. designs. In one study, programmers’ experience with Despite the possibility of adopting XP without TDD varied from novice to expert, while program­ it, TDD seems to be a core XP practice. Anecdotal mers new to TDD participated in the other studies. evidence indicates that TDD is commonly included These studies showed that programmers using when only a subset of XP is adopted. TDD produced code that passed 18 percent to 50 The use of xUnit testing frameworks provides percent more external test cases than code pro­ another possible indicator of TDD’s use. JUnit, the duced by corresponding control groups. The stud­ first such framework, has enjoyed widespread pop­ ies also reported less time spent debugging code ularity (www.junit.org). No JUnit adoption statis­ developed with TDD. Further, they reported that tics are directly available. The Eclipse core applying TDD had an impact that ranged from distribution, a popular integrated development minimal to a 16 percent decrease in programmer environment primarily used for Java development productivity—which shows that applying TDD includes JUnit. A press release issued in February sometimes took longer. In the case that took 16 per­ 2004 on the Eclipse Web site states that the Eclipse cent more time, researchers noted that the control platform has recorded more than 18 million down­ group wrote far fewer tests than the TDD group. load requests since its inception. Although dupli­ Table 1 summarizes these studies and labels each cate requests from the same developer can occur, experiment as either a case study or a controlled the figure is still substantial. Certainly not all experiment. Eclipse developers use JUnit, nor do all JUnit adopters use TDD, but it is likely that the popu­ TDD in academia larity of XP, JUnit, and Eclipse combined implies a Several academic studies have examined XP as a certain degree of TDD adoption. whole, but a few focused on TDD. Although many of the TDD studies published in academic settings EVALUATIVE TDD RESEARCH are anecdotal, the five studies shown in Table 2 Since the introduction of XP, many practitioner specifically report on empirical results. When refer­ articles and books on applying TDD have been ring to software quality, all but one11 study focused written. There has been relatively little evaluative on the ability of TDD to detect defects early. Two research on the benefits and effects of TDD, how­ of the five studies reported significant improvement ever. in software quality and programmer productiv­ Research on TDD can be categorized broadly by ity.11,12 One reported a correlation between the context. In particular, TDD research is classified as number of tests written and productivity.13 In this industry if the study or research was conducted pri­ particular study, students using test-first methods marily with professional software practitioners. It wrote more tests and were significantly more pro­ is classified as academia if the practitioners are pri­ ductive. The remaining two studies14,15 reported marily students and the work takes place in the con­ no significant improvement in either defect density text of an academic setting. Academic research also or productivity. includes studies in which students work on a pro­ All five of these relatively small studies lasted a ject for a company but in the context of an acade­ semester or less and involved programmers who had mic course. little or no previous experience with TDD.

48 Computer Table 2. Summary of TDD research in academia.

Number of Controlled experiment programmers Quality effects Productivity effects

Kaufmann11 8 Improved information flow 50% improvement Edwards12 59 54% fewer defects n/a Erdogmus13 35 No change Improved productivity Müller14 19 No change, but better reuse No change Pancˇur15 38 No change No change

FACTORS IN SOFTWARE PRACTICE ADOPTION Additional research and the availability of train­ A variety of factors play into the widespread ing and instructional materials will likely play an adoption of a software practice. These include important role in determining how widespread motivation for change, economics, availability of TDD will become. tools, training and instructional materials, a sound theoretical basis, empirical and anecdotal evidence UNDERSTANDING TDD’S EFFECTS of success, time, and even endorsements of the prac­ Further research must be done to determine and tice by highly regarded individuals or groups. understand TDD’s effects. To date, research has These factors complicate TDD’s current state. focused on TDD as a testing technique to lower Current software development practice provides a defect density. Empirical studies should be con­ clear motivation for change and thus TDD seems ducted to evaluate TDD’s effect on software design poised for growth. Software development involves quality and to examine characteristics such as a complex mix of people, processes, technology, extensibility, reusability, and maintainability. and tools that struggle to find consistency and pre­ Even with the focus on defect density, there have dictability. Projects continue to go over schedule been only a small number of studies conducted, and budget, which makes practitioners eager to find and those on only small samples. One industry improved methods. study with more than 10 participants involved a Tool support for TDD is strong and improving small application that took only one day to com­ for most modern languages. Tools such as JUnit, plete. The results were suspect because the control MockObjects, and Cactus are mature and widely group wrote a minimal number of tests. available. Much of this tool development has tar­ The few academic studies that have examined geted Java, an increasingly popular language in defect density produced inconsistent results. The both commercial applications and academia. largest study reported a 54 percent reduction in Economic models have noted the potential for defect density with beginning programmers. Two positive improvements for XP and TDD, but rec­ other reasonably large studies with advanced pro­ ognized that additional research is needed. This is grammers did not provide any significant reduc­ especially true regarding speed and defects and tion in defect density. One study hinted at better when TDD is combined with pair programming. designs. The interplay between academic and industry prac­ Future studies should consider the effectiveness titioners for acceptance is an interesting one. of TDD at varying levels in the curriculum and the Research indicates that it takes five to 15 years for programmer’s maturity. The studies can also exam­ academic development to succeed in commercial ine how TDD compares to test-last methods that practice—and the reverse holds true. Research fix the design ahead of time, as well as iterative test- shows how TDD can improve programming ped­ last methods that build an emergent design. agogy, yet few instructional resources exist. The XP-EF,16 a framework for consistently conducting JUnit incorporation into BlueJ and the corre­ and assessing case studies on XP projects, currently sponding programming textbook indicates under development, seems appropriate for adapta­ improvement may be on its way, however. tion into a TDD case studies framework. Given such The adoption of TDD faces many challenges. a framework, researchers can conduct multiple case First, TDD requires a good deal of discipline on the studies and controlled studies. Programmer produc­ programmer’s part. Hence, programmers may tivity and software quality should be examined. require compelling reasons before they try it. Adoption issues such as learning curves, suitability Second, TDD is still widely misunderstood, per­ and fit, and motivation must be addressed. haps because of its name, but many still think erro­ Additionally, research is needed to examine the neously that TDD addresses testing only, and not effects of combining TDD with other practices such design. Third, TDD doesn’t fit every situation. as pair programming and code inspection. Developers and managers must determine when to Both industry and academic settings can benefit apply TDD and when to do something else. from more research. In particular, academic stud-

September 2005 49 ies need to examine whether TDD improves or at the 18th Ann. ACM Sigplan Conf. Object-Oriented least does not hinder learning. Can TDD be incor­ Programming, Systems, Languages, and Applica­ porated into the undergraduate curriculum in a tions, ACM Press, 2003, pp. 298-299. way that improves students’ ability to design and 12. S.H. Edwards, “Using Test-Driven Development in test? If so, then TDD must be written into appro­ the Classroom: Providing Students with Automatic, priate student texts and lab materials. Concrete Feedback on Performance.” Proc. Int’l Conf. Education and Information Systems: Tech­ nologies and Applications (EISTA 03), Aug. 2003; ven if XP fades in popularity, TDD may persist. http://web-cat.cs.vt.edu/grader/Edwards-EISTA03. Additional research is needed on TDD’s ability pdf. E to improve software quality and on its place in 13. H. Erdogmus, M. Morisio, and M. Torchiano, “On undergraduate computer science and software engi­ the Effectiveness of Test-First Approach to Program­ neering curricula. If TDD finds its way into acade­ ming,” IEEE Trans. Software Eng., Mar. 2005, pp. mia, students could enter software organizations 226-237. with increased discipline and improved software 14. M.M. Müller and O. Hagner, “Experiment about design and testing skills, increasing the software Test-First Programming,” IEE Proc. Software, IEE engineering community’s ability to reliably produce, Publications, 2002, pp. 131-136. reuse, and maintain quality software. ■ 15. M. Pancˇur et al., “Towards Empirical Evaluation of Test-Driven Development in a University Environ­ ment,” Proc. Eurocon 2003: The Computer as a References Tool, The IEEE Region 8, vol. 2, 2003, pp. 83-86. 1. C. Larman and V.R. Basili, “Iterative and Incremen­ 16. L. Williams, L. Layman, and W. Krebs, Extreme Pro­ tal Development: A Brief History,” Computer, June gramming Evaluation Framework for Object-Ori­ 2003, pp. 47-56. ented Languages, v. 1.4, tech. report TR-2004-18, 2. K. Beck, “Aim, Fire,” IEEE Software, Sept./Oct. North Carolina State Univ., 2004. 2001, pp. 87-89. 3. K. Beck, Extreme Programming Explained: Embrace Change, Addison-Wesley, 1999. 4. P. Abrahamsson et al., “New Directions on Agile Methods: A Comparative Analysis,” Proc. 25th Int’l Conf. Software Eng. (ICSE 03), IEEE CS Press, 2003, pp. 244-254. David Janzen is the president of Simex LLC, a con­ 5. D.J. Reifer, “How Good are Agile Methods?” IEEE sulting and training firm, and an associate profes­ Software, July/Aug. 2002, pp. 16-18. sor of computer science at Bethel College. He is 6. Shine Technologies, “Agile Methodologies Survey currently a PhD candidate at the University of Results,” 17 Jan. 2003; www.shinetech.com/ Kansas, where he received an MS in computer sci­ download/attachments/98/ShineTechAgileSurvey2003­ ence. Janzen is a member of the IEEE Computer 01-17.pdf?version=1. Society and the ACM. Contact him at david@ 7. J. Rasmusson, “Introducing XP into Greenfield Pro­ simexusa.com. jects: Lessons Learned,” IEEE Software, May/June, 2003, pp. 21-28. 8. B. George and L. Williams, “A Structured Experiment of Test-Driven Development,” Information and Soft­ ware Technology, vol. 46, no. 5, 2004, pp. 337-342. Hossein Saiedian is a professor and associate chair 9. E.M. Maximilien and L. Williams, “Assessing Test- in the Department of Electrical Engineering and Driven Development at IBM,” Proc. 25th Int’l Conf. Computer Science, and a member of the Informa­ Software Eng. (ICSE 03), IEEE CS Press, 2003, pp. tion and Telecommunication Technology Center at 564-569. the University of Kansas. His research interests 10. L. Williams, E.M. Maximilien, and M. Vouk, “Test- include software process improvement, object tech­ Driven Development as a Defect-Reduction Prac­ nology, and software architecture. Saiedian received tice,” Proc. 14th Int’l Symp. Software Reliability Eng. a PhD in computer science from Kansas State Uni­ (ISSRE 03), IEEE Press, 2003, pp. 34-45. versity. He is a senior member of the IEEE, and a 11. R. Kaufmann and D. Janzen, “Implications of Test- member of the IEEE Computer Society and the Driven Development: A Pilot Study,” Companion of ACM. Contact him at [email protected].

50 Computer