Lean Software Development

Home , Kanban (development), Specification by example

Sloan Management Review, I sum- Lean Software marized my findings while Krafcik attempted to generalize Toyota’s management philosophy and presented his own data showing Japanese superior- Development: ity. Krafcik coined the term “lean” as a contrast to a Ford-style “buffered” A Tutorial mass production system.2 Toyota’s techniques dramatically reduced in-process and final invento- Mary Poppendieck, Poppendieck.LLC ries, expanding worker responsibilities to operate more machines, as well Michael A. Cusumano, Massachusetts Institute of Technology as enabling more assembly line workers to build in quality. A key element was that Toyota reversed the flow of information signals (then in the form // This tutorial describes where lean software of Kanban cards) controlling production operations. This change gave To development comes from, what it means, how it yota factories the ability to make small relates to well-known agile development practices, batches of components “just in time” (JIT) for final assembly and shipment and how it is likely to evolve in the future. // to dealers, while eliminating most of the “just in case” in-process and final inventories. The basic philosophy was to “pull” materials and components through the production system as needed in a continuous flow, rather than “push” them through and stock- pile using fully predetermined production plans. The terms “lean” and “JIT” were later popularized by MIT’s global best-seller The Machine That Changed The term “lean” was first applied and thesis under my (Michael’s) super- the World (Rawson, 1990). The au- publicly to a production management vision, after I had just published The thors used the term “lean” to describe process and then to product develop- Japanese Automobile Industry in 1985. any efficient management practice that ment at MIT during the mid-1980s I had reported that Toyota, using tech- minimized waste, including in prod- (for a history of lean in the operations niques that had evolved since the late uct development, where Japanese au- management literature, see Matthias 1940s, was producing automobiles tomakers had achieved shorter leader Holweg’s “The Genealogy of Lean Pro- with roughly half the number of labor times (by about one-third) and fewer duction”1). John Krafcik, currently the hours as the Big Three US automakers engineering hours (by about half) com- CEO of Hyundai Motor America, was with much faster inventory turns and pared to US and European projects. the first American engineer hired by higher quality. Nissan was not far be- (The more detailed research was done Toyota to work in the NUMMI (New hind Toyota. Encouraged by MIT’s at Harvard Business School by IMVP United Motor Manufacturing, Inc.) as- International Motor Vehicle Program research affiliates Kim Clark and Taka- sembly plant in California, a joint ven- (IMVP) research director, James Wom- hiro Fujimoto, published as Product ture with General Motors. He came ack, Krafcik launched a survey to com- Development Performance [Harvard to the MIT Sloan School of Manage- pare auto assembly plants around the Business School, 1991].) ment in 1986 to do a master’s degree world. In 1988 companion articles in Some similarities between Japanese

26 IEEE Software | published by the IEEE computer society 0740-7459/12/$31.00 © 2012 IEEE management and PC-style software Process comparison of Toyota and Microsoft. development were becoming apparent by the mid-1990s. For example, in Mi- Toyota-style “lean” production 1990s Microsoft-style “agile” development (manual demand-pull with Kanban cards) (daily builds with evolving features) crosoft Secrets (Free Press, 1995), we

(Michael and coauthor Richard Selby) 1 able JIT “small lot” production Development by small-scale features noted a similarity in the philosophy T Minimal in-process inventories Short cycles and milestone intervals behind Microsoft’s daily builds, where engineers had to stop and fix bugs on a Geographic concentration—production Geographic concentration—development daily basis (we dubbed this the “synch Production leveling Scheduling by features and milestones and stabilize” process), and Toyota’s JIT production philosophy, where Rapid setup Automated build tools and quick tests workers stopped assembly lines when- Machine and line rationalization Focus on small, multifunctional teams ever they detected problems to fix them immediately. This book did not use Work standardization Design, coding, and testing standards the term “lean” for software develop- Foolproof automation devices Builds and continuous integration testing ment; we were thinking mainly about the common application of a JIT engi- Multiskilled workers Overlapping responsibilities neering and quality-control practice, Selective use of automation Computer-aided tools, but no code generators as well as the reduced levels of bureaucracy and staffing in Microsoft com- Continuous improvement Postmortems, process evolution pared to companies such as IBM. To Source: M. Cusumano, Staying Power, Oxford, 2010, p. 197. dive deeper into product development in the auto industry, along with an MIT doctoral student (Kentaro Nobeoka) I (Michael) launched another major re- book, Lean Software Development or agile development (before the Win- search effort, summarized in the book (Addison-Wesley, 2003). In common dows teams became inordinately large Thinking beyond Lean (Free Press, with the MIT and IMVP researchers, in the late 1990s and early 2000s), 1998). This book also built on a pub- we also emphasized eliminating waste even when we look at specific prac- lication by James Womack and Daniel and bureaucracy in product develop- tices. My (Michael’s) 2010 book sum- Jones, Lean Thinking (Simon & Schus- ment, encouraged learning through marized some of these similarities un- ter, 1996), which attempted to gener- short cycles and frequent builds, and der the principle of “pull, don’t just alize lean practices and applications. promoted late changes and fast itera- push” (see Table 1). Thinking beyond Lean focused on tions, with feedback pulling changes Early use of the term “lean” no newer approaches to product develop- into a product, rather than require- doubt leveraged the popularity of Jap- ment, emphasizing the systematic reuse ments documents and rigid plans push- anese management techniques, but the of product platforms and major com- ing development work forward. emphasis has always been on reducing ponents, as well as other techniques, Authors who have used the term waste in terms of time and staffing, fo- such as short, overlapping phases (con- “lean” with reference to software de- cusing on value for the customer, the current engineering) to reduce calendar velopment all seem to have stressed product, and the enterprise, as well as time and engineering hours, but it only the difference between older and more stressing the benefits of a more flex- included a brief discussion of software labor-intensive, bureaucratic, push- ible, iterative, lightweight development development. style methods, initially associated with process. We also see this orientation The popularization of the term the mainframe computer business, in XP and Scrum.3 These approaches, “lean” and its association with “ag- and newer, less bureaucratic, iterative referred to as iterative, incremen- ile” for software product develop- or incremental, and flexible methods. tal, or agile, encompass a set of tech- ment seems to have come mainly from There clearly are many common ele- niques for software development that later efforts, such as those of one of us ments between Toyota-style lean pro- are less sequential or “waterfall-ish” (Mary) and Tom Poppendieck in the duction and Microsoft-style iterative as well as bureaucratic and slow, and

september/october 2012 | IEEE Software 27 FOCUS: Lean software development

have become commonplace around the on how to use IT to create value for the process, and so on. Moreover, the value world owing to the benefits they offer.4 customers…. Thus lean development is of software isn’t derived solely from not a software engineering methodol- the development phase; design and de- Lean Software ogy in the conventional sense. It’s re- ployment are fundamental to its value. Development ally a synthesis of a system of practices, Finally, the value of software is rarely This leads to the more general issue principles, and philosophy for building limited to a single time-bound effort; of whether or not it is even appropri- software systems for a customer’s use.” the capability to modify a code base ate to apply the principles behind lean In the book, Lean Software De- over time tends to be a dominant factor production to product development velopment (Addison-Wesley, 2003) in its overall value. or software engineering. In fact, we we (Mary and coauthor Tom) also Eliminate Waste In lean terms, “waste” is anything that If lean is thought of as a set of practices, doesn’t either add customer value di- rectly or add knowledge about how it doesn’t translate well from operational to deliver that value more effectively. to development environments. Some of the biggest causes of waste in software development are unnecessary features, lost knowledge, partially done work, handovers, and multitasking, not have frequently encountered nega- emphasized seven principles of lean to mention the 40 to 50 percent of de- tive reactions to this with comments software development (which we later velopment time spent finding and fix- such as, “Development is not at all modified slightly, based on subse- ing defects. Many of these wastes have like manufacturing.” And, indeed, if quent experience): their roots in the large batches of par- lean is thought of as a set of practices, tially done work created in sequential it doesn’t translate well from opera- • optimize the whole, development processes, in the bound- tional to development environments. • eliminate waste, aries between different functions, and However, if lean is thought of as a set • build quality in, in the delays and knowledge lost when of principles rather than practices, then • learn constantly, work crosses these boundaries. When applying lean concepts to product de- • deliver fast, organizations look at the flow of value velopment and software engineering • engage everyone, and across the entire value stream, causes of makes more sense and can lead to pro- • keep getting better. waste are more easily exposed and ad- cess and quality improvements. dressed, much as in a JIT pull system In fact, the idea of applying lean These principles are used to frame for manufacturing. principles to software development is the discussion about what product de- almost as old as the term “lean” itself. velopment practices might be appropri- Build Quality In In the 1990s, Robert Charette used the ate for, depending on the unique situa- Microsoft’s synch and stabilize pro- term “lean development” to refer to a tion of each organization. cess seemed to be a dramatic departure risk management strategy that brings from the prevailing process of waiting about dynamic stability in organiza- Optimize the Whole until the end of a development cycle tions by making them more agile, re- Lean software development should be before integrating small units of soft- silient, and change tolerant. In 2002, founded on a deep understanding of ware into a large system. But, in fact, Charette wrote “Challenging the Fun- a job that customers would like done continuously integrating small units of damental Notions of Software Develop- and how this job might be mediated by software into larger systems has long ment,” a piece that is as wise today as it software. Discovering what customers been held to be best practice, even was a decade ago.5 In it, he provides an care about and what they will value is though it might not have been com- excellent summary of the main points not a simple process, especially because mon practice. As long ago as 1970, of this article: “As Drucker pointed out software rarely has value in and of it- IBM’s Harlan Mills successfully devel- long ago, a business’s sole purpose is to self. The value of software is delivered oped an approach he called “top-down create and serve the customer. In this in the context of a larger system: an au- programming”—a process whereby spirit, [lean development]’s focus is not tomated car, a website for buying prod- modules are integrated into the overall on the development process per se, but ucts, an automated order fulfillment system as they are written, rather than

28 IEEE Software | www.computer.org/software at the end of development. In his 1988 frequent deployment have dramatically a flow process, a fundamentally differ- book Software Productivity (Dorset improved quality while eliminating the ent organizational structure is often House), he noted, “My principal crite- large regression overhead formerly asso- required. Instead of placing software rion for judging whether top down pro- ciated with releases. Of course, existing development in a separate department gramming was actually used is [the] ab- code bases can be riddled with defects, called “IT,” software development sence of any difficulty at integration.” and small perturbations can have serious tends to be thought of as product de- unintended consequences, but many or- velopment and located in line business Learn Constantly ganizations have discovered how to cre- units. Responsibility for discovering, In the end, development is all about ate test harnesses that can automatically creating, and delivering value falls to creating knowledge and embedding detect most defects introduced by small a team that encompasses the complete that knowledge in a product. Lean de- changes. Thus as an industry, we have value stream. Thus a value stream velopment recommends approaching moved much closer to Harlan Mills’s team that includes software develop- this in two different ways, depending ideal. ment will almost certainly include ad- on the context. When software is delivered quickly, ditional functions; in fact, it will prob- The first is to explore multiple op- thinking about software development ably look like a miniature version of tions for expensive-to-change deci- as a project is an inappropriate meta- a business. It will include people who sions such as fundamental architecture, phor. It’s much better to think of soft- understand customers, designers, de- choice of language, design language for ware as a flow system where software velopers, testers, operations, support, user interaction, and so on. Delay criti- is designed, developed, and delivered in and perhaps finance. cal decisions to the last responsible mo- a steady flow of small changes. This is Even when software development ment, and then make decisions based fundamentally different from thinking occurs in a separate organization, on the best available knowledge at the about software development as a proj- lean practices encourage teamwork time. Because multiple options have ect to be completed, or even thinking among engaged people who are em- been explored, there will always be an about software as a series of annual or powered to make decisions appropri- alternative that will work, and the op- semiannual releases. ate to their level. Although some so- tion that best optimizes the overall sys- Rapid delivery should not be isolated called lean implementations appear tem can be chosen. This is often called to software development; flow should to emphasize processes over people, a “learn first” approach. happen within the overall product de- these represent a misunderstand- The second way is to build a mini- velopment cycle, of which software is ing of lean principles. Empowering mum set of capabilities to get started, one aspect. Embedded software devel- people, encouraging teamwork, and followed by frequent delivery, while using feedback from real customer experience to make product content decisions. This continuous learning Even when software development process will minimize the effort spent occurs in a separate organization, developing features that customers lean practices encourage teamwork. don’t find valuable. Although the “learn first” approach might seem diametrically opposed to the “learn constantly” approach, they opment, for example, usually takes on moving decision-making to the low- can be used together effectively. The key the flow characteristics of the product est possible level are fundamental to is to use options and constraints to drive it’s embedded in. A good place to look any lean implementation. critical decisions while recognizing that for more information on product devel- the content of most software systems opment flow is in Principles of Product Keep Getting Better will change constantly over time. Development Flow by Don Reinertsen In their September 1999 Harvard Busi- (Celeritas Publishing, 2009). ness Review article “Decoding the Deliver Fast DNA of the Toyota Production Sys- In many lean development environments, Engage Everyone tem,” Steven Spear and Kent Bowen production releases occur frequently— When software is thought of as some- point out that at Toyota, every work weekly, daily, even continuously. Mistake- thing that grows continually over time, system is improved constantly, us- proofing mechanisms necessary for such and its development is conceived of as ing the scientific method, under the

september/october 2012 | IEEE Software 29 FOCUS: Lean software development

guidance of a teacher, at the lowest appropriate for individual contexts and (Prentice Hall, 2001) by Ken Schwa- possible level of the corporation. Lean situations that expand beyond soft- ber and Mike Beedle. Over the next thinking holds that specific practices, ware development. few years, Scrum became increasingly no matter how well they seem to work popular as a method to replace tradi- in other situations, are seldom the best XP tional project management with devel- solution to the problem at hand. There- The first agile process to become popu- opment iterations of one month (and fore lean thinking would recommend lar was XP, defined in the 2000 book later two weeks). Thus it has been an that organizations starting with prac- Extreme Programming Explained excellent way of introducing the lean tices such as XP or Scrum (or a com- (Addison-Wesley) by Kent Beck. XP fo- concept of flow into software develop- bination) should think of them as a cuses on several technical practices, the ment. Because of Scrum’s widespread starting point that will be adapted and most notable being test-driven develop- popularity, it’s often equated with ag- improved over time by the people and ment. TDD results in a unit test har- ile. However, it doesn’t claim to be a teams doing the work. ness, which is run frequently, making it complete methodology—it lacks tech- possible to detect defects almost imme- nical practices such as those found Agile Software diately after they are injected into the in XP and is generally limited to the Development code base. This approach has proven software development portion of the To put the concept of lean software to be an effective first step in mistake- value stream. Nevertheless, these early development in context, it’s useful to proofing a code base. Over time, meth- agile processes proved that incremen- point out similarities and differences ods have developed, keeping the test tal delivery of software was a viable with agile software development. “Ag- harness runtime to a minimum while approach for many domains, and that ile,” a common term in everyday lan- factoring the tests so they don’t grow quality could be significantly improved guage today, was used in the 1990s to beyond maintainable proportions. through a disciplined approach to test- refer to flexible production systems.6,7 Eventually, the concept behind TDD first development. It was then applied to software de- was expanded to include automated velopment in February 2001 when a product specifications, which set the Roles group of like-minded software experts stage for automated regression testing. Both Scrum and XP are sets of practices produced the Manifesto for Agile Soft- New techniques such as Framework for aimed at optimizing the software devel- ware Development.8 The manifesto Integrated Testing (FIT)9 and its part- opment process as a separate activity stated that when developing software, ner Fitness, acceptance test-driven de- in the value stream. They each define it’s preferable to value individuals and velopment (ATDD), behavior-driven a role—the “customer” in XP and the “product owner” in Scrum—to own the responsibility for deciding what Early agile processes proved that needs to be done. Development team members aren’t generally expected to incremental delivery of software was a assume the responsibility for the over- viable approach for many domains. all success of their work; that responsibility is delegated to the customer or product owner roles. In practice, the implementation of these roles has fre- interactions over processes and tools; development (BDD), and specification quently led organizations to violate the working software over comprehensive by example (SBE)10 emerged to tackle lean principle of optimizing the whole. documentation; customer collabora- this highly domain-dependent prob- Lean software development, as we tion over contract negotiation; and re- lem. Automated testing frameworks are view the term, places software devel- sponding to change over following a currently the most common technique opment as a step in a product value plan. These precepts were a reaction used to implement the lean principle of stream, regards developers as members against common software development building quality in. of a larger product team, and expects practices at the time, and they’re cer- all product team members to become tainly compatible with lean principles. Scrum engaged in the overall success of the However, we see lean principles as pro- The next agile process to gain traction product. There’s no product owner or viding more comprehensive guidance was Scrum, defined in the book Agile customer roles in lean development. for choosing development practices Software Development with SCRUM Lean development teams are led by

30 IEEE Software | www.computer.org/software someone in a role such as chief engi- Kanban systems provide a good A senior manager at this company re- neer (Toyota), program manager (Mi- framework for organizations getting cently noted that, in his opinion, speed crosoft), or product champion (3M), started with lean principles. Because drives all other lean principles: “If you whose roles are similar to an entrepre- there are very few rules or roles, Kan- deliver daily, waste is exposed almost neur leading a startup business. Al- ban systems require thoughtful consid- immediately; you have no choice but to though these roles might seem similar eration and adaptation. For example, a build quality in; you learn quickly what to the customer or product owner roles Kanban board might start out in a soft- customers value; everyone at every level in agile software development, they’re ware development environment, but is focused on making customers happy; quite different in practice. A chief en- can easily expand to include more steps problems are exposed quickly and so gineer has overall responsibility for a complete product, including its success in the marketplace, and engages every- Kanban systems provide a good one on a multidiscipline product team in delivering that success. There’s no framework for organizations getting intermediate role prioritizing work for started with lean principles. a separate software development team.

Kanban In his 2009 book, Scrumban: Essays in the value stream, such as marketing constant improvement is mandatory; on Kanban Systems for Lean Soft- and operations. This makes Kanban a and finally, optimizing just a part of ware Development (Modus Coope- good tool for value stream teams. Kan- the system simply is not an option with randi Press), Corey Ladas described ban systems specifically focus on the daily deployment.” how to use a Kanban (card) system to flow of value; in fact, flow and bottle- In 2011, Jez Humble and David both track and limit work in progress necks are the main topic of daily meet- Farley’s book Continuous Delivery in a software development environ- ings. Finally, Kanban board layouts (Addison-Wesley Professional, 2010) ment. This was followed in 2010 by and policies are expected to be evalu- described the technical practices that David Anderson’s book Kanban (Blue ated and improved on a regular basis. must be in place to enable a continual Hole Press, 2010), which describes For an excellent case study of the use flow of new software released to a pro- how to use Kanban as the basis of an of a Kanban system for a large govern- duction environment in a safe, defect- effective, flow-based software devel- ment contract, see Henrik Kniberg’s free manner. This book, written for opment system. book Lean from the Trenches (Prag- enterprise IT departments, lays out the In a Kanban system, the value matic Bookshelf, 2011). tools, technologies, and organizational stream is mapped on a chart (prefer- cooperation necessary to achieve the ably a physical chart on a wall) with Trends lean ideal of deploying software safely columns for each step in the value Throughout the last decade, increas- to production virtually immediately stream. Kanban cards (tokens for a ingly sophisticated tools have evolved, after it’s written. Forward-looking IT piece of work) are placed in the col- making the development process more departments around the world have umn, representing the current state of mistake-proof while safely allowing followed this playbook and found that the work. When work meets specified the delivery of a continuous flow of it takes about a year of hard work to policies for being complete, the Kan- small features into production. These reach a goal of weekly, daily, or even ban token moves to the next column, tools were initially developed for Web- continuous delivery. and over time, the card moves from left based platforms delivering software as to right across the board—you visually a service. Lean Startup see work flow through the system. The Agile methods usually provide for a key to a Kanban system is that work in Continuous Delivery customer or customer proxy to di- any column (representing a step in the As early as 2004, a large Web-based rect the work of the software develop- value stream) is limited. This means enterprise (which will remain anony- ment team; however, they provide few that within any value-adding activity, mous for the purposes of this article) mechanisms to ensure that what the there’s a limited amount of work; more- released to production all of the soft- customer proxy asks for will effectively over, the entire system contains a lim- ware that had been worked on during address the customer problem, nor do ited amount of work. the day at the end of every single day. they typically involve the development

september/october 2012 | IEEE Software 31 FOCUS: Lean software development

Increasingly, agile development practices are being thought of as good ways Mary Poppendieck is retired from 3M and is currently president of to organize software development, but Poppendieck.LLC. Her industry experience includes software develop- insufficient ways to address design. ment, supply-chain management, manufacturing operations, and new Because design is fundamentally itera- product development. Poppendieck has an MS in mathematics from the

uthors University of Maryland. Contact her at [email protected]. tive and development is fundamentally iterative, the two disciplines suffer if they are not carefully integrated with each other. Because lean development lays out a set of principles that demand Michael A. Cusumano is the Sloan Management Review Distin- a whole-product, complete life-cycle, guished Professor at the Massachusetts Institute of Technology’s Sloan cross-functional approach, it’s the more School of Management, with a joint appointment in MIT’s Engineering likely candidate to guide the combina- Systems Division. His research interests include technology management and strategy, especially in the software business. Cusumano has tion of design, development, deploy- a PhD in Japanese studies from Harvard University. Contact him at ment, and validation into a single feed- bout the A the A bout [email protected]. back loop focused on the discovery and delivery of value.

References 1. M. Holweg, “The Genealogy of Lean Produc- tion,” J. Operations Management, vol. 25, no. 2, 2007, pp. 420–437. team in assuring that the product is a as providing platforms for two-sided 2. J. Krafcik, “Triumph of the Lean Production market success. markets and creating engaging experi- System,” MIT Sloan Management Rev., vol. 30, no. 1, 1988, pp. 41-52. In 2011, Eric Reis addressed this ences. The most rapidly growing soft- 3. M. Cusumano, “Extreme Programming problem in the book Lean Startup ware companies provide ecosystems Compared with Microsoft-Style Iterative (Crown Business, 2011) where he advo- that attract traffic through their abil- Development,” Comm. ACM, vol. 50, no. 10, 2007, pp. 15–18. cates that companies start with a busi- ity to understand and address an im- 4. M. Cusumano et al., “Software Development ness plan and test the impact of features portant customer need that is not be- Worldwide: The State of the Practice,” IEEE on the key assumptions that form the ing adequately served. Increasingly, Software, vol. 20, no. 6, 2003, pp. 28–34. 5. R.N. Charette, “Challenging the Fundamental basis of that plan. In this approach, the the design of the customer experience Notions of Software Development,” white software development team becomes in- is a foundational element of such an paper, IT Metrics and Productivity Inst., 2003; www.itmpi.org/assets/base/images/ volved in setting up the feedback loops, ecosystem. At the same time, these itmpi/privaterooms/robertcharette/ such as split tests, necessary to deter- systems tend to use a continuous de- ChallengingtheFundamentalNotions.pdf. mine the impact of features. Thus, soft- livery approach rather than a project 6. P.T. Kidd, Agile Manufacturing: Forging New ware engineers are typically involved in approach, so the system architecture Frontiers, Addison-Wesley, 2004. 7. R. DeVor, R. Graves, and J. Mills, “Agile the process of validating the value of must be designed from the beginning Manufacturing Research: Accomplishments new features to customers while mak- to support dynamic updating and con- and Opportunities,” Inst. Industrial Engi- neers Trans., vol. 29, 1997, pp. 813–823. ing adjustments based on this feedback. tinuous change. 8. K. Beck et al., Manifesto for Agile Software Obtaining feature-by-feature feedback Development, 2001; http://agilemanifesto.org. is an effective way for companies tar- 9. R. Mugridge and W. Cunningham, FIT for geting a large customer base to validate gile development methods Developing Software, Addison-Wesley, 2005. 10. G. Adzic, Specification by Example, Manning the value of various approaches. have generally expected sys- Publications, 2011. A tem architecture and interac- Design Thinking tion design to occur outside the devel- Over the past decade, the epicenter of opment team, or to occur in very small software value creation has changed. It increments within the team. Because of used to be that orchestrating transac- this, agile practices often prove to be tions and controlling equipment were insufficient in addressing issues of so- Selected CS articles and columns the primary purposes of software. To- lution design, user interaction design, are also available for free at day, new purposes have emerged, such and high-level system architecture. http://ComputingNow.computer.org.

32 IEEE Software | www.computer.org/software