focusreports from the field
Launching Extreme Programming at a Process- Intensive Company
James Grenning, Object Mentor
his is a story about starting a project using an adaptation of XP in a company with a large formal software development process. De- fined Process Systems is a big company (the name is fictitious). The T division I worked with was developing safety-critical systems and A company that was building a new system to replace an existing legacy product. The project has traditional was an embedded-systems application running on Windows NT and was part formal of a network of machines that had to collaborate to provide services. processes launched a Through a systems engineering effort, DPS What we were facing project using divided the system into subsystems, each of The standard practice in the division was many Extreme which was later designed by a development created in response to past and current Programming team. I was brought in to help one of the problems. Most of the developers had one practices. The teams start its part of the project using iter- to three years’ experience. The more senior ative development, use cases, and object- developers had the role of reviewing and author covers oriented design techniques. approving the work of the less experienced how XP was Shortly after starting this project, I spent developers. To their credit, the review team proposed to a week in XP Immersion I, a one-week in- took the formal review process very seri- management, tensive training class in the techniques and ously. They were good at it: they captured philosophy of Extreme Programming.1 En- issues and defects on the Web, prepared re- how the project thused by XP, I talked to the DPS team viewers, and held crisp review meetings. seed began and about applying some XP practices to our In the world of Big Design Up Front and grew, and some work. The company had already decided to phase containment, these guys were good. I of the issues try iterative development and OO tech- could end this article right now except for niques, departing from the division’s stan- one problem: all this process added over- the team faced dard process. They were game for some- head to the development of software. To de- during its first thing different, but how different? We sign something, they needed to figure out six months. decided to take the idea of incorporating the design, document it in Rose, schedule a XP practices to the director of engineering. review meeting, and distribute materials to
0740-7459/01/$10.00 © 2001 IEEE November/December 2001 IEEE SOFTWARE 27 review. Then reviewers had to review the dex card. We decided to choose our battles. materials and enter issues on the Web, have We needed to get some of the beneficial The need for a review meeting, document the meeting practices into the project and not get hurt outcome, repair the defects and then close by leaving other practices behind. We did documentation them on the Web, fix documents, and have not omit practices that we didn’t feel like was ingrained the changes reviewed again. doing; we tried to do as many as we could. in the culture, However, all this process work was not We used the practices and their interactions keeping bugs out of the product. Unrealistic as ways to sell around the objections. so we expected deadlines and surprises late in the project We started by identifying the project’s concern over were taking their toll. Products were deliv- goals—to build a working product with re- ered late. Engineers were just getting their liable operation and timely delivery, with XP’s lack skills to a decent technical depth, but they enough documentation to enable effective of formal were also burning out and heading for sys- maintenance (no more, no less), and with documentation. tems engineering or management. understandable source code. This was as The team struggled with how to begin the objectionable as motherhood and apple pie. new project. Its requirements were in prose The standard process would identify the format and fully understood only by the per- same objectives. son who wrote them. We all agreed that a reliable working To summarize: the existing process had a product was a critical output of the project. lot of overhead, deadlines were tight, engi- This was particularly important, as this was neers were running away, requirements were a safety-critical system. A tougher question partially defined, and they had to get a project was, what was enough documentation? This started. With all these issues, something had to was where it got interesting. This applica- change. Did they need something extreme like tion was not your typical XP target applica- XP? I believed and hoped XP would help. tion—it was part of a larger system that mul- tiple groups were developing at multiple Choosing your battles sites. These other groups were using the The DPS culture values up-front require- standard, waterfall-style DPS process, not ments documents, up-front designs, reviews, XP or short-iteration development. We had and approvals. Writing the product features a potential impedance mismatch between on note cards, not doing any up-front de- the XP team and the rest of the project. sign, and jumping into coding were not go- ing to be popular ideas. To people unfamil- How much documentation? iar with XP, this sounded a lot like hacking. Proposing no documentation would end How did we get by these objections? the conversation. We decided to keep the Having been introduced to XP, the group conversation going and answer a question understood what the main objections would with a question. What did we want from be as we tried to sell XP to the management our documentation? We needed team. Like good lawyers, we prepared an- ticipated questions along with their answers enough documentation to define the for our presentation. We expected that the product requirements, sustain technical decision makers would consider some of the reviews, and support the system’s main- practices dangerous and unworkable at tainers; DPS. The need for documentation was in- clean and understandable source code; grained in the culture, so we expected con- and cern over XP’s lack of formal documenta- some form of interface documentation, tion. Can the code be the design? Can we due to the impedance mismatch be- really build a product without up-front de- tween groups. sign? What if there is thrashing while refac- toring? What about design reviews? These answers did not all align with XP out of To paraphrase Kent Beck, one of XP’s the book, but they kept the conversation go- originators, “do all of XP before trying to ing. XP is not antidocumentation; it recog- customize it.” I think that is great advice, nizes that documentation has a cost and that but for this environment we would never not creating it might be more cost-effective. have gotten the okay to mark up the first in- This, of course, violates conventional wisdom.
28 IEEE SOFTWARE November/December 2001 After acknowledging and addressing the given to the maintainers describes the state project’s objectives, I led the team through of the software at the time it was shelved or the cost-of-change pitch from Beck’s Ex- transferred. This document can and should Documentation treme Programming Explained.1 The direc- be written in a way that avoids needing fre- tor, the manager, and some senior technol- quent modification. As maintenance pro- and reviews ogists agreed that XP addressed many of ceeds, the document will likely be neglected. were going to their current development problems. They Make the document high-level enough so be the big also thought XP right out of the book that the usual maintenance changes and bug would not work for them. What did we fixes do not affect it. Let the documentation roadblocks. want to do differently? guide the future developers to the right part Documentation and reviews were going of the code—then they can use the high- to be the big roadblocks. We heard, “Re- quality, readable, simple source code to quirements on note cards!?!” “I can’t give a work out the details. stack of note cards to the test team.” “Bob Following this strategy will not result in a in firmware needs the cards too.” “Someone huge document. Remember, you have some will lose the cards.” I noticed that the com- of the best detailed documentation available pany’s “standard” process allowed use cases in the form of automated unit tests—work- in the form described by Alistair Cock- ing code examples of exactly how to use burn.2 This is a text-based method, similar each object in the system. XP does not pro- to user stories but with more details. Other hibit documentation; just realize it has a cost DPS groups needed to look at the use cases, and make sure it is worth it. You can plan so we decided not to fight that battle—we documentation tasks into any iteration. The had enough lined up already. We decided to advice here is to document what you have use use cases. built, not what you anticipate building. Another objection was “We need docu- The next follow-on objection was that mentation for the future generations of en- we’d never do the document at the end of gineers that will maintain the product we the project. My reply: So, you would rather are building. We need our senior people to do a little bit at a time, and have to keep look at the design to make sure your design changing and rewriting it? Doesn’t that will work.” Our answer was that a good sound like it would take a lot of time? It way to protect future software maintainers does! So the management team must stick to is to provide them with clean and simple its guns and do the high-level documenta- source code, not binders full of out-of-date tion task at the end of the project. Pay for it paper. Maintainers always go to the source with less time wasted during development. code; it cannot lie, as documents can. XP re- lies on the source code being simple and ex- Reviews pressive and uses refactoring to keep it that The director did not completely buy into way.3 The source code is the most important documenting only at the end of the project part of the design. At some point, the main- and hence made a final objection: “I still tainers will need a high-level document to have my concerns. What if the design is no navigate the system. good? Do I have to wait until the end of the A follow-on objection was that what one project to find out?” Pair programming was person thinks is readable source code is not not reason enough for the management team to another. XP addresses this through pair to give up its review process. The big prob- programming. If a pair works hard at mak- lem with DPS’s development process was that ing the source readable, there is a really it guaranteed that development would go good chance that a third programmer who slowly. The process went something like this: sees the code will find it readable, too. Plus, create a design, document it in Rose, sched- with collective code ownership, anyone can ule a review meeting, distribute the materials, change the source code if need be. have everyone review the materials, have the Another expected objection was that code review meeting, collect the issues, fix the is- was not enough—we needed a documenta- sues, maybe do another review, then finally tion or transition strategy for whenever a write some code to see if the design works. project is put on the shelf or transferred to This made sure the cost of change was high. another team. Ideally, the documentation Because of the safety-critical nature of the
November/December 2001 IEEE SOFTWARE 29 application, the management team was not build a high-quality product. XP’s iterative na- willing to give up on reviews. ture could help the team determine how fast it At the beginning I proposed a more efficient way to do the could go and give management the feedback it reviews: the ask-for-forgiveness (rather than needed. So, it’s about getting the job done. XP of a project, ask-for-permission) design process. We let is a set of techniques that seemed promising. you need to the development team work for a month at Projects can get stalled in the fuzzy front believe that the a time on the system. At the end of the end.4 This is a problem, especially in water- month, they assembled and reviewed a de- fall projects, where you must define all the design can and sign-as-built review package. This took the requirements prior to starting the design will evolve. review off the critical path, so the review process. In XP, as soon as you have defined process did not slow down the team. We a couple weeks’ worth of user stories, devel- agreed to document the significant designs opment can start. Think of how hard it is to within the iteration and review them with shave a month off the end of a project. Now the review team. We gave the remaining is- think of how easy it would be to save that sues that the reviewers found to the next it- month just by starting development as soon eration as stories. The idea here was to as you have identified a month’s worth of spend a small amount of time in the itera- stories. Throughout the rest of the project, tion documenting the design decisions that story development occurs concurrently with month. As it turned out, we really did not story implementation. have to ask for forgiveness at all. The first iteration It’s not about XP The team had three people—a customer It’s about building better software pre- and two developers, including me. We started dictably and faster. We compromised on a by getting the unit test tool CppUnit set up number of issues, but we did have agreement and integrated with our development envi- to use most of the XP practices: test-first pro- ronment, VC++. This did not take long—the gramming, pair programming, short itera- tools are relatively easy to use. tions, continuous integration, refactoring, The project’s customer (our systems engi- planning, and team membership for the cus- neer) gave us a requirements document. As tomer. Table 1 describes the XP practices we we identified a functional requirement, we used and how we modified them to suit our wrote it on a card. Each card named a use needs. We added some process and formality: case. We did not bother to elaborate the use use cases, monthly design reviews, and some cases, just name them; in a few days, we had documentation. This adaptation of XP was a identified 125 use cases. Picking the most im- significant change in how DPS developed portant ones was relatively easy using this list. software, and we hoped to prove it was an In XP, the customer chooses the most improvement. The director thought XP of- valuable user stories and discusses them fered a lot of promise for a better way to with the programmers. We were using use work that could lead to improved quality, cases, a similar idea; our customer chose the faster development, better predictability, and most valuable use cases and elaborated more on-the-job satisfaction (“smiles per them. For the early iterations, we decided to hour”). He said we were “making footprints ignore use case extensions (which hold the in the sand.” If we could improve one devel- special cases or variations) and keep the opment factor—quality, job satisfaction, pro- product definition simple.2 We just assumed ductivity, or predictability—the director there were no special cases or error cases; thought it might be worth doing XP. If we because we were using XP, we believed we could improve any two of those factors, he could ignore the details and not be penal- thought there would be a big payoff. ized later. We also did not bother using use The problems of poor quality, delivery de- case diagrams, because they did not add any lays, long delivery cycles, and burned-out en- value to the development team. Our main gineers plague the software industry. XP goal in the first iteration was to build a lit- struck a cord with our team leaders. The tech- tle bit of the product, get some experience, niques appeared to address some of the prob- and build some skill and confidence. lems the team was facing, and the focus on At the beginning of a project, you need to testing and pair programming could help it believe that the design can and will evolve.
30 IEEE SOFTWARE November/December 2001 Table 1 Summary of XP practices used XP practice Adoption status Our experience Planning game Partially adopted The team practiced scope limiting, task breakdown, and task sign-up techniques. We used use cases rather than user stories. We wrote the use cases from an existing requirements document and stored them in a database. Small releases Adopted Iterations were one month long each. Metaphor Not adopted A metaphor had not yet evolved, and we didn’t develop one. Instead, a high-level design evolved and was recorded in a small set of UML diagrams and explanatory text. It played the role of our metaphor. Simple design Adopted The design did not have anticipatory elements. A monthly design-as-built review let the senior people monitoring the project see the team’s design decisions. Functional testing Adopted We developed functional tests in a custom scripting language. The tests demonstrated that the application logic met the customer’s need. However, the team got behind on automated acceptance tests. This is not recommended. Test-first design Adopted We wrote the first line of production code using test-first design. We wrote the program in C++ and used CppUnit 1.5 as our unit test tool. Refactoring Adopted We refactored regularly, and the design evolved smoothly. Pair programming Adopted We could develop tests, interfaces, and simulations on our own but used pair programming to create production code. Collective ownership Adopted We collectively owned the code. On one occasion, new code that required special knowledge resulted in a module owned by one programmer. Development slowed on that part of the system. Continuous integration Adopted During our first iteration, continuous integration was no problem. As soon as we added a second pair to the project, the team had integration problems. We quickly learned how to avoid collisions and do merges. 40-hour week (also known as Adopted The team worked at a sustainable pace. A couple times, we put in overtime to meet the sustainable pace) iteration goals. On-site customer Partially adopted A systems engineer acted as the on-site customer. We derived our acceptance tests from the use cases. The customer was not directly responsible for these tests. The team scared the customer a couple times by delivering his use cases in less time than it took him to define them. Coding standards Adopted The main coding standard was “make the code look like the code that is already there.” The team used header and C++ file source templates to provide the company-required comment blocks. The comment blocks were mainly noise that hid the code. Open workspace Not adopted There was no open workspace. Workstations were in the corners, making pair programming awkward. The roadblocks to building a team workspace were political.
Otherwise, the desire to do up-front specifica- about half a day with a whiteboard looking tion work will put the team into analysis at design ideas. Because we were unsure of paralysis. Knowing that the design can evolve XP, we were not sure if we could really start sets the team free to start building the system coding without doing some design. We did a as soon as some functionality is identified. little bit of design, or as we called it a Little Simplifying assumptions keep complexity out Design Up Front (LDUF). Ron Jeffries calls of the code, at least temporarily. John Gall this a Quick design session.6 wrote, “A complex system that works is in- In our LDUF session, we found a group variably found to have evolved from a simple of collaborating objects we thought would systems that works.”5 Thus, it really helps to meet the needs of our first stories. We make these simplifying and scope-limiting de- worked from hand-drawn copies, not both- cisions in each iteration. This lets the core fea- ering with a diagramming or CASE tool. We tures drive the design. knew things would change, and we did not We had a planning meeting and discussed want to spend our time making it look the features that were to be included in the pretty. It gave us confidence and a vision of first iteration. My partner and I had OOD where we were going. experience but no real XP experience (except During the iteration planning meeting for the week I spent in class). We wanted a and our LDUF session, we identified some of guide to our first iteration, so we spent the interfaces we needed to support the iter-
November/December 2001 IEEE SOFTWARE 31 Figure 1. Simulating the interface by creating a Mock Application Service Interface logic class Object, shown in Unified Modeling working on the project. Meetings were a Language. <
32 IEEE SOFTWARE November/December 2001 Object-oriented design is an important impressed with our results in terms of pro- About the Author supporting practice of XP. OO program- ductivity and quality. Because of this project, ming languages let you build software in two other pilot projects were started. James Grenning small independent pieces, a practice that is the director test-first programming promotes.8 of consulting n my experience, when the engineers at Object Project manager surprises Mentor and want XP, the management doesn’t, and is currently Not only were the programmers happy I if management wants XP, the engineers practicing with their creation, but the project manager don’t. Where is the trust between manage- and coaching Extreme Programming. Areas of interest was as well. After the fourth iteration, he said, ment and engineering? are organizational change, software proc- “I’d only have three documents by now! In- To managers: Try XP on a team with ess improvement, and the business impact stead I have a piece of the system that works!” open-minded leaders. Make it okay to try of agile software development. He helped create the Manifesto for Agile Software The manager discovered another benefit. new things. Encourage the XP practices. Pro- Development (http://AgileAlliance.org) Usually a project manager coordinating a vide good coaching. Challenge your team to and is a signatory. He is also a member of team’s work with other teams spends a lot of go against the status quo. Recruit a team that the ACM. He earned a BS in electrical en- gineering and computer science from the time juggling priorities and figuring out task wants to try XP rather than force a team to University of Illinois, Chicago. Contact dependencies. On the XP team, dependencies use XP. Make sure the organization sees that him at Object Mentor, 565 Lakeview Park- between features were almost nonexistent. no one will be punished for trying something way, Ste. 135, Vernon Hills, IL 60061; [email protected]. We built features in the order of customer different. When hand-picking XP practices, priority, not internal software framework or- you might compromise the self-supporting der dictated by a BDUF (Big Design Up Front). nature of XP. Try as much of XP as you can. The team was agile and able to adapt to the Iterations are short. Feedback comes often. other subsystems’ changing needs. To engineers: Develop a sales pitch. Iden- tify problems that you might solve. Identify Building the team the benefits, identify the risks. Do a pilot We built the team slowly, while we were project. Iterations are short. Feedback developing skills. We felt we could absorb one comes often. or two people per iteration. We did not let newcomers take tasks right away, but used Acknowledgments them mainly as pair partners during their first I thank the real client that provided the experience iteration. Then, as they got to know the sys- to write this article (who wished to remain anony- mous). I also want to thank Chris Biegay and Jennifer tem and our practices, they started to take on Kohnke for a job well done in helping me prepare this tasks at the iteration planning meetings. We article. I presented a earlier version of this article at XP didn’t assume that team velocity would in- Universe in July 2001. crease when we added a new person to the team—we measure velocity, not predict it. References The DPS way of developing software 1. K. Beck, Extreme Programming Explained, Addison- made up for programmer inexperience by Wesley, Reading, Mass., 1999. 2. A. Cockburn, Writing Effective Use Cases, the Crystal having senior engineers review the less ex- Collection for Software Professionals, Addison-Wesley, perienced engineers’ work. In XP projects, Reading, Mass., 2000. you must still address the spread of exper- 3. M. Fowler et al., Refactoring: Improving the Design of Existing Code, Addison-Wesley, Reading, Mass., tise; for instance, it is critical to have at least 1999. one senior engineer on the team. We don’t 4. S. McConnell, Rapid Development, Microsoft Press, Red- give senior people a big title or special role, mond, Wash., 1996. 5. J. Gall, Systemantics: How Systems Really Work and but we need them. They help spread the How They Fail, 2nd ed., General Systemantics Press, wealth of knowledge, and both they and Ann Arbor, Mich., 1986. their pair partners learn. 6. R.E. Jeffries, Extreme Programming Installed, Addison- Wesley, Reading, Mass., 2001. 7. T. Mackinnon, S. Freeman, and P. Craig, Endo-Testing: The end of the story Unit Testing with Mock Objects, XP Examined, Addi- Unfortunately, I cannot present the story son-Wesley, Reading, Mass., 2001. 8. R. Martin, “Design Principles and Design Patterns,” of how the project completed, because it was www.objectmentor.com/publications/ mothballed due to changing market needs. Principles%20and%20Patterns.PDF (current 12 Oct. This is in the spirit of one of XP’s mantras: 2001). Work on the most important thing first. Nev- For more information on this or any other computing topic, please visit our ertheless, the team and the managers were Digital Library at http://computer.org/publications/dlib.
November/December 2001 IEEE SOFTWARE 33