COVER FEATURE Introducing an Agile Process to an Organization

The transition from a plan-driven to an agile process affects not only the development team members, but also other teams, departments, and man- agement. The authors describe common pitfalls and effective approaches to making this change.

Mike Cohn ince the publication of Kent Beck’s Extreme simple projects reached across many departments Mountain Goat Programming Explained,1 agile processes or functional areas. A failure to sell the process Software have grown increasingly popular. Agile change to any one area can negatively impact the processes allow for changing require- project’s outcome. Doris Ford S ments throughout the development cycle Through trial and error, we have discovered sev- Precision Projects and stress collaboration between software devel- eral approaches for successfully introducing agile opers and customers and early product delivery. processes to organizations. The “Agile Manifesto” establishes a common framework for these processes: Value individuals DEVELOPERS and interactions over processes and tools, working Most developers respond to the proposed intro- software over comprehensive documentation, cus- duction of an agile process with the appropriate tomer collaboration over contract negotiation, and combination of skepticism, enthusiasm, and cau- responding to change over following a plan.2 The tious optimism. Other developers, however, either processes most commonly considered agile include resist the change or overzealously jump into the pro- Extreme Programming (XP),1 Lean Development,3 ject without enough forethought. Either reaction Crystal,4 and Scrum.5-6 can cause problems. In Scrum, projects progress in a series of month- long iterations called sprints. Development teams Resistance meet with the client, or product owner, before each In general, agile processes value code production sprint to prioritize the work to be done and select the more than plan-driven processes do. In a plan-dri- tasks the team can complete in the upcoming sprint. ven process, developers might treat Unified During the sprint, the team stays on track by Modeling Language designs and other noncode holding brief daily meetings. At the end of each items as first-class artifacts. In an agile process, sprint, the team delivers a potentially shippable however, these items usually exist only to support product increment. Scrum is ideally suited for pro- the coding activity. jects with rapidly changing or highly emergent While introducing Scrum to various project requirements such as Web projects or product devel- teams, we invariably found programmers who opment for new markets. enjoy producing noncode artifacts far more than Over the past four years, we have introduced they are willing to admit. We also encountered pro- Scrum to seven organizations in four states. Some grammers who measure their contribution to a pro- of the projects were complex and involved distrib- ject by the number of meetings they attend. These uted teams. Others were straightforward and programmers go beyond “analysis paralysis” and involved small, colocated teams. However, even the actively seek opportunities to add formalized tasks

74 Computer Published by the IEEE Computer Society 0018-9162/03/$17.00 © 2003 IEEE back into an agile process. One programmer, for sprint types introduces just enough formality example, created a formal document type and that the teams can more clearly see their way attempted to coerce his peers into generating the through the project. As the teams become Teams using document for hundreds of cases when it was called more adept with the informality of the agile agile processes for in perhaps a dozen. process, they gradually drop the sprint types tend to make In such situations, it is best not to intervene. The concept. decisions more other team members can quickly assess the value quickly than of these activities and will not adopt them if they do Distributed development not help the overall development effort. Teams using agile processes tend to make plan-driven decisions more quickly than plan-driven teams. Micromanagement teams, relying on more frequent (and usually A surprising number of developers view using informal) communication to support this agile processes as an attempt to micromanage. pace. Because approaches like Scrum and XP accelerate A failed attempt to use an agile process in a pro- project cycles, developers typically interact with their ject with sites 2,000 miles apart taught us that orga- managers more frequently but for shorter periods. nizations should resist distributed development for In a plan-driven process, a manager might go a full at least the first two or three months after initiat- week without talking with a particular developer, ing an agile process. The companies involved in the but daily contact is the norm in most agile processes. merger that initiated the project needed to resolve Developers who view agile processes as micro- their political and cultural issues before developers management tend to perceive interactions with could succeed with a project shared across multi- their project managers as being about due dates and ple cities. missed deadlines. To avoid this problem, project If distributed teams must be combined, bringing managers should constantly demonstrate their as many people as possible together for the first desire to remove obstacles as quickly as possible week or two of the project can increase the likeli- and not complain if a task takes too long. hood of success. We have successfully used this Managers can be surprised, but should not be judg- approach on multiple distributed projects. mental, when told that a task will take longer than originally thought. The need for top talent Barry Boehm’s principle of top talent, “use bet- Transitioning from a heavyweight process ter and fewer people,”7 is central to an agile Some developers we encountered preferred process. Agile processes strip nonessential activi- heavyweight plan-driven processes because they ties from projects, leaving developers more time to believe they looked better on a resume. Because develop. these individuals do not have deeply held convic- Although the difference in productivity between tions about the process’s value, they can eventually the best and worst programmers on a team may be swayed by their coworkers’ opinions and actions. exceed the documented ratio of 10:1,8 the produc- A gradual transition from a heavyweight to an tivity difference matters most when the program- agile process can make the change easier on the mers are working on tasks essential to software development team. Some teams, when first intro- delivery. Productivity differences are irrelevant duced to Scrum, are overwhelmed to the point of when the programmers are engaged in nonessen- inaction by the freedom of not having a day-by-day tial activities. Gantt chart directing their work. When fully engaged and comfortable with an We have helped teams ease into Scrum by defin- agile process, a development team moves very ing a set of sprint types: quickly. Too many slow workers either slow the entire team or end up left behind by their faster col- • prototyping, leagues. • requirements capture, • analysis and design, Overzealous teams • implementation, and One team we worked with was overly enthusi- • stabilization. astic about a move to XP. At the project’s onset, team members aggressively began documenting We then work with the teams to define the arti- requirements by writing user stories, planning iter- facts that will result from each sprint type. Using ations, and pairing up for programming tasks.

June 2003 75 They even moved out of their existing space receive on most agile process projects. As with pro- Agile processes and into an adjacent abandoned building bet- grammers, testers will see over time that an agile do not imply ter suited for XP. Unfortunately, they did all process is not synonymous with micromanagement. of this so quickly that the rest of the organi- We have encountered testers who secretly want making zation had no idea what they were doing, to be programmers and use a project’s early itera- decisions resulting in a number of problems. tions to sneak in some programming. We also have without Although agile processes promise greater worked with testers who either volunteer or are forethought. productivity once in place, productivity may coerced into writing unit tests for programmers. decrease during the transition while the team Especially during a project’s earliest iterations, you learns new techniques. Not having antici- should discourage testers from such inappropriate pated this decreased productivity, the team activities. First, if the tester knows enough about pro- chose to redouble its efforts when it occurred. gramming to program and you need another pro- Agile processes do not imply making decisions grammer, hire the tester. Second, writing a unit test without forethought, a distinction this overzealous for someone else may result in a useful test, but it team missed in its quest for speed and agility. To almost certainly loses some of the white-box advan- this team, Beck’s recommendation to “put in what tages inherent in self-written unit tests. you need when you need it”1 meant they needed to think only about an hour ahead. They didn’t have UPPER MANAGEMENT the discipline XP requires, and, while paying lip Upper management often presents unique chal- service to XP, they were actually doing nothing lenges to organizations wishing to move toward an more than hacking.9 agile process. Upper-management concerns gener- Overzealousness also caused the team to split ally fall into four categories: into two camps: the overzealous team members and a group that knew decisions were being made too • How can we promise new features to cus- quickly and often incorrectly. Much like the tor- tomers? toise and the hare, these two subteams pursued the • How can we track progress? project differently. After the project’s failure, it took • How will the agile process impact other a while to convince both groups to jointly pursue groups? an agile process, albeit one that was “less agile” • When does the project end? than the overzealous members initially wanted. Many managers, particularly those at higher lev- Testers els, are reluctant to surrender the feeling of control Writing source code is a primary activity in any that Gantt charts and other plan-driven process arti- development process, but it is especially important facts give them. Similarly, they may be comforted in agile processes: “Code is the one artifact that by the development group’s promise to deliver an development absolutely cannot live without.”1 exact amount of functionality on a specified date, Agile processes do not have separate coding and even if they know the group won’t be able to do so. testing phases; rather, code written during an iter- ation must be tested and debugged during that iter- Customer commitments ation. Testers and programmers work more closely Managers who worry that an agile process will earlier in an agile process than in other processes. mean they can’t make product commitments must Thus, testers and other nonprogrammers must be understand that any past project plan that implied carefully integrated into any agile project in which guarantees about delivery date, cost, and func- they participate. tionality was either wrong, heavily padded, or both. Initially, testers, even more than programmers, In organizations with a history of incorrect pro- tend to view an agile process as micromanagement. ject estimates, it might not be difficult to convince Before the adoption of an agile process, many testers upper management that an agile process is worth (especially those in organizations without a sepa- trying. If a software group has a record of on-time rate and dedicated testing group) do not receive delivery, however, you must convince upper man- much attention from managers. Test activities are agement that using an agile process could have often relegated to a single line item on a project plan. resulted in projects being completed either sooner In plan-driven projects, testing tends to occur or with fewer resources.10-12 without explicit notice from a project manager, so Drawing a typical cost, date, and feature triangle testers are not used to the extra attention they usually can persuade upper management that

76 Computer such commitments are an illusion. Once upper the other group could write new require- management realizes that past commitments were ments. Model status mostly combinations of good luck and padded esti- When introducing an agile process into an mates, they become very interested in any process organization, upper management must under- reports can help that promises greater efficiency. stand and agree on how this will impact groups convince upper outside the development group. If they don’t, management Tracking progress and if they can’t agree on how to resolve dif- that agile Plan-driven processes appeal to many upper ferences of opinion, most efforts will likely fail. processes allow managers because they facilitate progress tracking. A manager can track a process that results in Project completion adequate project numerous documents by simply asking if the nec- Finally, upper management commonly tracking. essary documents have been produced. fears that a project will go on forever. Most If a Software Test Plan is called for, a first level of managers are comfortable with a model in tracking can occur when the manager verifies its which project budgets are approved and the existence. A second level of tracking can occur project remains within the budget confines. They when the manager weighs the document, and a are less comfortable when told that project itera- third if the manager reads it. tions will persist as long as the customer or a cus- To persuade upper managers that agile processes tomer proxy continues to identify high-priority, allow adequate project tracking, we usually create high-value work. two or more model status reports based entirely on Wrapping budgeting and strategic-planning activ- fictional data about the project an organization is ities around the project can address these concerns. considering for an agile process. The reports depict For example, we estimated one commercial project a fictional two- to four-week project cycle. would take from nine to 15 months—a fairly impre- A typical status report for a Scrum process pro- cise estimate because we didn’t know exactly what ject includes a list of key dates, a two- to five-para- features the completed system would include. graph commentary on the project’s state, a burn- Regardless of how many bells and whistles the client down chart comparing progress to planned work, desired, however, we felt reasonably sure that a suit- key metrics (defect inflow, percentage of tests able initial release would be available in nine passed, and so on) appropriate to the project’s months. We therefore convinced upper manage- current state, and a list of key risks. The upper- ment to fund a nine-month development effort with management decision makers we have worked with the agreement that additional funding would be dis- have found this type of status reporting satisfactory. cussed near the end of that period.

Impact on other groups HUMAN RESOURCES Some upper managers have expressed concern You might think that a human resources depart- that although an agile process might benefit the ment would have no interest in one group’s transi- development group, it can adversely affect one or tion to an agile development process, but this is not more other groups. This concern becomes the case. On a project that was transitioning to XP, unhealthy when another group’s processes nega- two programmers approached the HR department tively impact the development team’s work. For with complaints about pair programming—two pro- example, one software engineering group wanted grammers sharing a keyboard and monitor and writ- to use Scrum, while the product management group ing code in tandem—which, of course, sounded odd that provided all specifications and requirements and unproductive to HR. Because we hadn’t told HR wanted to continue with a heavyweight waterfall about the process change, we were immediately in approach. The CEO saw no problem in letting each the difficult position of having to explain why pair group pursue its independent strategies. The result programming made sense. was 2,000 pages of detailed product specifications Another time, a small subset of programmers fed into a development process that needed work complained to HR that they didn’t like how a pro- prioritized in monthlong units. ject was being managed. The complaints stemmed The software engineering group had to guess pri- from their unwillingness to consider or try anything orities from the product management group’s three- new or different. The programmers were familiar to four-month task lists. Once the software engi- with plan-driven processes, and anything else neering group was accustomed to Scrum, however, seemed too chaotic. This was in 1999, before XP they moved through the requirements faster than had become an intriguing buzzword, before the

June 2003 77 Agile Alliance2 had been formed, and when Scrum Agile processes have continued to evolve over the was only beginning to be documented. past four years, and approaches that worked in one The HR department must be told when a group case have not worked in another. As experience in is transitioning to an agile process. When told, the introduction of object technology into compa- however, the HR department may raise its own nies in the late 1980s and early 1990s led to the dis- concerns, centering on an agile process’s imprecise covery of best practices in introducing that deliverables and dynamic goals. Many HR depart- technology, we expect an understanding to arise ments require corrective action plans, which cite over the next few years for agile processes. specific deliverables and deadlines that can result in the employee’s termination if not met. It is difficult to shoehorn tasks from an XP iter- References ation or Scrum sprint into a deterministic correc- 1. K. Beck, Extreme Programming Explained, Addison- tive action plan. However, we have found that by Wesley, 2000. proactively working with HR, we can sufficiently 2. K. Beck et al., “Manifesto for Agile Software Devel- define tasks and deadlines that both satisfy HR and opment,” Feb. 2001; www.agilemanifesto.org. let the project follow an agile process. 3. M. Poppendieck and T. Poppendieck, Lean Software Development, Addison-Wesley, 2003. ow an agile process is introduced into an orga- 4. A. Cockburn, Agile Software Development, Addi- nization will significantly impact the ultimate son-Wesley, 2002. H success of the process change. Any new 5. M. Cohn, “The Scrum Development Process”; process is likely to appeal to some developers, who www.mountaingoatsoftware.com/scrum. are excited to be among the first to try it. Similarly, 6. K. Schwaber and M. Beedle, Agile Software Devel- this newness is an obstacle to developers who opment with Scrum, Prentice Hall, 2002. oppose change. 7. B. Boehm, Software Engineering Economics, Pren- tice Hall, 1981. 8. F. Brooks Jr., The Mythical Man-Month, Addison- Wesley, 1975. 9. B. Boehm, “Get Ready for Agile Methods, with Care,” Computer, Jan. 2002, pp. 64-69. 10. A. Cockburn and J. Highsmith, “Agile Software Development: The People Factor,” Computer, Nov. JOIN A 2001, pp. 131-133. 11. M. Cohn, “The Upside of Downsizing,” Software Test and Quality Eng., Jan. 2003, pp. 18-21. 12. Shine Technologies, “Agile Methodologies Survey”; THINK www.shinetech.com/agile_survey_results.jsp, Jan. 2003.

Mike Cohn is the founder of Mountain Goat Soft- TANK ware. His current research interests include agile ooking for a community targeted to your project management, communicating requirements area of expertise? IEEE Computer Society through user stories, and metrics for agile projects. L Technical Committees explore a variety Cohn received an MS in computer science from the of computing niches and provide forums for University of Idaho. He is a member of the ACM dialogue among peers. These groups influence and the Agile Alliance. Contact him at mike@ our standards development and offer leading mountaingoatsoftware.com. conferences in their fields. Doris Ford is the president of Precision Projects. Her current research interests include software pro- Join a community that targets your discipline. ject management, project metric development, and tracking methodologies. Ford received an MBA from Regis University. She is a member of the Pro- In our Technical Committees, you’re in good company. ject Management Institute. Contact her at [email protected]. computer.org/TCsignup/