The Software Quality Challenge

Software Quality

Watts S. Humphrey The Software Engineering Institute

Many aspects of our lives are governed by large, complex systems with increasingly complex software, and the safety, security, and reliability of these systems has become a major concern. As the software in today’s systems grows larger, it has more defects, and these defects adversely affect the safety, security, and reliability of the systems. This article explains why the com- mon test-and-fix software quality strategy is no longer adequate, and characterizes the properties of the quality strategy we must pursue to solve the software quality problem in the future. oday, many of the systems on which defects per page while even poor-quality different ways they can be used, and the our lives and livelihoods depend are software has much less than one defect more ways users can use them, the harder Trun by software. Whether we fly in air- per listing page. This means that the qual- it is to test all of these conditions in planes, file taxes, or wear pacemakers, our ity level of even poor-quality software is advance. This was the logic behind the safety and well being depend on software. higher than that obtained for other kinds beta-testing strategy started at IBM with With each system enhancement, the size of human written text. Programming is an the OS/360 system more than 40 years and complexity of these systems increase, exacting business, and these professionals ago. Early copies of the new system releas- as does the likelihood of serious prob- are doing extraordinarily high quality es were sent to a small number of trusted lems. Defects in video games, reservations work. The only problem is that based on users and IBM then fixed the problems systems, or accounting programs may be historical trends, future systems will be they found before releasing the public ver- inconvenient, but software defects in air- much larger and more complex than sion. This strategy was so successful that it craft, automobiles, air traffic control sys- today, meaning that just to maintain has become widely used by almost all ven- tems, nuclear power plants, and weapons today’s defect levels, we must do much dors of commercial software. systems can be dangerous. higher quality work in the future. Unfortunately, however, the beta-test- Everyone depends on transportation To appreciate the challenge of achieving strategy is not suitable for life-critical networks, hospitals, medical devices, pub- ing 10 or fewer defects per million lines of systems. The V-22 Osprey helicopter, for lic utilities, and the international financial code, consider what the source listing for example, uses a tilting wing and rotor sys- infrastructure. These systems are all run such a program would look like. The list- tem in order to fly like an airplane and by increasingly complex and potentially ing for a 1,000-line program would fill 40 land like a helicopter. In one test flight, the defective software systems. Regardless of text pages; a million-line program would hydraulic system failed just as the pilot was whether these large life-critical systems are take 40,000 pages. Clearly, finding all but tilting the wing to land. While the aircraft newly developed or composed from mod- 10 defects in 40,000 pages of material is had a built-in back-up system to handle ified legacy systems, to be safe or secure, humanly impossible. However, we now such failures, the aircraft had not been they must have quality levels of very few have complex life-critical systems of this tested under those precise conditions, and defects per million parts. scale and will have much larger ones in the the defect in the back-up system’s soft- Modern, large-scale systems typically relatively near future. So we must do ware had not been found. The defect have enormous requirements documents, something, but what? That is the question caused the V-22 to become unstable and large and complex designs, and millions of addressed in this article. crash, killing all aboard. lines of software code. Uncorrected errors The problem is that as systems in any aspect of the design and develop- Why Defective Systems Work become more complex, the number of ment process generally result in defects in To understand the software quality prob- possible ways to use these systems grows the operational systems. The defect levels lem, the first question we must answer is If exponentially. The testing problem is fur- of such operational systems are typically today’s software is so defective, why aren’t there ther complicated by the fact that the way measured in defects per thousand lines of more software quality disasters? The answer is such systems are configured and the envi- code. A one million line-of-code system that software is an amazing technology. ronments in which they are used also with the typical quality level of one defect Once you test it and fix all of the prob- affect the way the software is executed. per 1,000 lines would have 1,000 undis- lems found, that software will always work Table 1 lists some of the variations that covered defects, while any reasonably safe under the conditions for which it was test- must be considered in testing complex system of this scale must have only a very ed. It will not wear out, rust, rot, or get systems. An examination of the number few defects, certainly less than 10. tired. The reason there are not more soft- of possibilities for even relatively simple systems shows why it is impractical to test The Need for Quality ware disasters is that testers have been able to exercise these systems in just about all possibilities for any complex system. So Software all of the ways they are typically used. So, why is complex software so defective? Before condemning programmers for to solve the software quality problem, all doing sloppy work, it is appropriate to we must do is keep testing these systems Some Facts consider the quality levels of other types in all of the ways they will be used. So Software is and must remain a human- of printed media. A quick scan of most what is the problem? produced product. While tools and tech- books, magazines, and newspapers will The problem is complexity. The more niques have been devised to automate the reveal at least one and generally more complex these systems become, the more production of code once the requirements

4 CROSSTALK The Journal of Defense Software Engineering June 2008 A The Software Quality Challenge and design are known, the requirements 1. Data rates Switche s Paths and design must be produced by people. 2. Data value s 1 2 Further, as systems become increasingly 3. Data err ors 4 6 complex, their requirements and design 4. Con figu ration variation s 9 20 grow increasingly complex. This complex- 5. Numbe r, type , and timing of 16 70 ity then leads to errors, and these errors simultaneou s process es 36 92 4 result in defects in the requirements, 6. Hardware f ail ures 49 3,43 2 design, and the operational code itself. 7. Network f ail ures 64 12 ,87 0 Thus, even if the code could be automati- 8. Ope rator err ors 81 48 ,62 0 cally generated from the defective require- 9. Version change s 10 0 184 ,75 6 ments and design, that code would reflect 10 . Power variation s 40 0 1.38 E+1 1 these requirements and design defects and, thus, still be defective. Table 1: Some of the Possible Testing Table 2: Possible Paths Through a Network When people design things, they make Variations defects. For example, a complex design mistakes. The larger and more complex likely to be exercised when such systems defect that produced a confusing operator their designs, the more mistakes they are are30 0 subjected to the stresses of high trans- message could pose no danger whileDes iagn/Code likely to make. From course data on thou- e action volume, accidents, failures, or mili- trivial typographical mistake that changed d

sands of experienced engineers learning o SM SM tary250 combat. a no to a yes could be very dangerous. c the Personal Software Process (PSP ), it/

f Since there is no way to tell in advance s t has been found that developers typically o The Defect Removal Problem which defects would be damaging, we c

s 200 inject about 100 defects into every 1,000e A defect is an incorrect or faulty con- must try to find them all. Then, after find- f e

lines of the code they write [1]. The distri-e n struction in a product. For software, ing them, we must fix at least all of the i d bution for the total defects injected by 810 l 150 Design Review Time l defects generally result from mistakes that ones that would be damaging. d experienced developers at the beginning ofa t

n the designers or developers make as they o PSP training is shown by the total bars in a produce their products. Examples are The Testing Problem T 100 Figure 1. While there is considerable varia- s u oversights, misunderstandings, and typos. Since defects could be anywhereTotal in a large tion and some engineers do higher-quality o Furthermore, since defects result from software system, the onlyTest way testing h 50 work, just about everybody injects defects. t mistakes, they are not logical. As a conse- could find them all would be to complete- Developers use various kinds of tools quence, there is no logical or deductive ly test every segment of code in the entire to generate program code from their 0 process that1. couldData possiblyrates find all of the program. To understand this issue, consid- designs, and they typically find and fix defects 1inst aQ usystem.artile They2nd Q coulduartile be 3any-rd Quaerrti lthee program4th Qua rstructuretile in Figure 2. This about half of their defects during this 2. Data value s where, and the only way to find all of the code fragment has one branch instruction process. This means that about 50 defects 3. Data err ors defects with testing is to exhaustively test at point B; threeUni tsegments:Test D/KL AO toC B, B to C, per 1,000 lines of code remain at the start 4. Con figu ration variation s every path, function, or system condition. and B to D; and two possible paths or of initial testing. Again, the distribution of 5. Numbe r, type , and timing of This leads to the next question which routes through the fragment: A-B-C and the defects found in initial testing is also simultaneou s process es concerns the testing objective: “Must we A-B-D. So, for a program fragment like shown by the test bars in Figure 1. 6. Hardware f ail ures Developers generally test their pro- find all of7. theNetwork defects, f orail urescouldn’t we just this, there could be defects on any of the grams until they run without obvious fail- find and fix8. thoseOpe ratorfew thaterr orswould be dan- code segments as well as in the branch ures. Then they submit these programs to gerous?”9. Obviously,Version wech aonlynge sneed to fix instruction itself. systems integration and testing where they the defects10 .thatPower would variati causeon strouble, but For a large program, the numbersC of are combined with other similar programs there is no way to determine which defects possible paths or routes through a pro- into larger and larger sub-systems and sys- these are without examining all of the gram can vary by program type, but pro- tems for progressively larger-scale testing. Figure 1: Total and Test Defect Rates of 810 Experienced Engineers The defect content of programs entering systems testing typically ranges between 10 and 20 defects per 1,000 lines. A 300 B The most disquieting fact is that testing can only find a fraction of the defects 250 in a program. That is, the more defects a program contains at test entry, the more it is likely to have at test completion. The 200 reason for this is the point previously D made about extensive testing. Clearly, if 150 t defects are randomly sprinkled through- h o

out a large and complex software system, u s

T 100 a some of them will be in the most rarely o n

t Total a used parts of the system and others will d l Test l d be in those parts that are only exercised i 50 n e e under failure conditions. Unfortunately, f e s c

these rarely used parts are the ones most o t s

f 0

SM / Personal Software Process and PSP are service marks of c

o 1st Quartile 2nd Quartile 3rd Quartile 4th Quartile Carnegie Mellon University. d e

June 2008 www.stsc.hill.af.mil 5 Software Quality

in unusual ways, their software is most C likely to encounter undiscovered defects. Fifth, under these stressful conditions, these systems are least likely to operate correctly or reliably. Therefore, with the current commonly used test-based software quality strategy, A B large-scale life-critical systems will be least reliable in emergencies – and that is when reliability is most important. Successful Quality Strategies Organizations have reached quality levels D of a few defects per million parts, but these have been with manufacturing and not design or development processes. In the Figure 2: A Three-Segment Code Fragment manufacturing context, the repetitive work grams generally have about one branch practical purposes, be considered infinite. is performed by machines, and the quality instruction for every 10 or so lines of code. Furthermore, even if comprehensive path challenge is to consistently and properly This means that a million-line program testing were possible, more than path test- follow all of the following steps: would typically have about 100,000 branch ing would be required to uncover the • Establish quality policies, goals, and instructions. To determine the magnitude defects that involved timing, synchroniza- plans. of the testing problem for such a system, tion, or unusual operational conditions. • Properly set up the machines. we must determine the number of test •Keep the machines supplied with high- paths through a network of 100,000 Conclusions on Testing quality parts and materials. switches. Here, from Figure 3, we can cal- At this point, several conclusions can be • Maintain the entire process under con- culate that, for a simple network of 16 drawn. First, today’s large-scale systems tinuous statistical control. switches, there are 70 possible paths from typically have many defects. Second, these • Evaluate the machine outputs. A to B. As shown in Table 2, the number of defects generally do not cause problems • Properly handle all deviations and possible paths through larger networks as long as the systems are used in ways problems. grows rapidly with 100 switches having that have been tested. Third, because of • Suitably package, distribute, or other- 184,756 possible paths and 400 switches the growing complexity of modern sys- wise handle the machine outputs. having 1.38E+11 possible paths. Clearly, tems, it is impossible to test all of the • Consistently strive to improve all the number of possible paths through a ways in which such systems could be aspects of the production and evalua- system with 100,000 switches could, for used. Fourth, when systems are stressed tion processes. Figure 3: Possible Paths Through a 16-Switch Network While these eight steps suggest some approaches to consider for software development, they are not directly applicable for human-intensive work such as design and B development. However,2 by considering an analogous approach with people instead of Possible machines, we begin to see how to proceed. Path The Eight Elements of Software Quality Management The eight steps required to consistently produce quality software are based on the five basic principles of software quality shown in the Software Quality Principles sidebar.With these principles in mind, we can now define the eight steps required for an effective software quality initiative. 1. Establish quality policies, goals, and plans. 2. Properly train, coach, and support the developers and their teams. 3. Establish and maintain a requirements quality-management process. 4. Establish and maintain statistical control of the software engineering process. 5. Review, inspect, and evaluate all product artifacts. A 6. Evaluate all defects for correction and

6 CROSSTALK The Journal of Defense Software Engineering June 2008 Switche s Paths The Software Quality Challenge

to identify, fix, and prevent other similar problems. Software Quality Principles* 7. Establish and maintain a configuration 1. Properly managed quality programs reduce total program cost, increase business management and change control sys- value and quality of delivered products, and shorten development times. tem. 1.1. If cost or development times increase, the quality program is not being proper- 8. Continually improve the development ly implemented. process. 1.2. The size of a product, including periodic reevaluation of size as changes occur, The following sections discuss each of must be estimated and tracked. these eight steps and relate them to the 1.3. Schedules, budgets, and quality commitments must be mutually consistent and software quality principles as shown in the based on sound historical data and estimating methods. 1.4. The development approach must be consistent with the rate of change in sidebar. requirements. Step 1: Quality Policies, Goals, and 2. To get quality work, the customer must demand it. 2.1. Attributes that define quality for a software product must be stated in measur- Plans able terms and formally agreed to between developers and customers as part Policies, goals, and plans go together and of the contract. Any instance of deviation from a specified attribute is a defect. form the essential foundation for all effec- 2.2. The contract shall specify the agreed upon quality level, stated in terms of the tive quality programs. The fundamental acceptable quantity or ratio of deviations (defects) in the delivered product. 3. The developers must feel personally responsible for the quality of the products they policy that forms the foundation for the produce. quality program is that quality is and must 3.1. The development teams must plan their own work and negotiate their commit- be the first priority. Many software devel- ments with management and the customer. opers, managers, and customers would 3.2. Software managers must provide appropriate training for developers. argue that product function is critical and 3.3. A developer is anyone who produces a part of the product, be it a designer, doc- that project schedule and program cost are umenter, coder, or systems designer. every bit as important as quality. In fact, 4. For the proper management of software development projects, the development teams themselves must plan, measure, and control the work. they will argue that cost, schedule, and 4.1. Project teams must have knowledge and experience in the relevant technolo- quality must be traded off. gies and applications domains commensurate with project size and other risk The reason this is a policy issue is given factors. in the first principle of software quality 4.2. Removal yield at each step and in total pre-delivery must be measured. stated in the sidebar: Properly managed quali- 4.3. Effort associated with each activity must be recorded. ty programs reduce total program cost, increase 4.4. Defects discovered by each appraisal method must be recorded. business value and quality of delivered products, 4.5. Measurements must be recorded by those performing the activity and be analyzed by both developers and managers. and shorten development times. Customers 5. Software management must recognize and reward quality work. must demand quality work from their sup- 5.1. Projects must utilize a combination of appraisal methods sufficient to verify the pliers and management must believe that if agreed defect levels. the quality program increases program 5.2. Managers must use measures to ensure high quality and improve processes. costs or schedules, that quality program is 5.3. Managers must use measurements with due respect for individuals. not properly managed. There is, in fact, no * These principles were defined by a group of 13 software quality experts convened by Taz Daughtrey. The cost/schedule/quality trade-off: manage experts are: Carol Dekkers, Gary Gack, Tom Gilb, Watts Humphrey, Joe Jarzombek, Capers Jones, Stephen quality properly, and cost and schedule Kan, Herb Krasner, Gary McGraw, Patricia McQuaid, Mark Paulk, Colin Tully, and Jerry Weinberg. improvements will follow. Everyone in the organization must understand and accept the developers, their teams, management, the skills required to measure and manage this point: it is always faster and cheaper to and the customer. When defective work is the quality of their work, requires training. do the job right the first time than it is to found, it must be promptly fixed. The While it would be most desirable for them waste time fixing defective products after principle is that defects cost money. The to get this skill and the required knowledge they have been developed. longer they are left in the product, the before they graduate from college, practic- Once the basic quality policy is in more work will be built on this defective ing software developers must generally place, customers, managers, and develop- foundation, and the more it will cost to learn them from using methods such as ers must then establish and agree on the find and fix them [2]. the PSP. quality goals for each project. The princi- With properly trained developers, the pal goal must be to find and remove all Step 2:Train and Coach Developers development teams then need proper defects in the program at the earliest pos- and Teams management, leadership, and coaching. SM sible time, with the overall objective of Quality work is not done by accident; it Again, the Team Software Process SM removing all defects before the start of takes dedicated effort and properly skilled (TSP ) can provide this guidance and sup- integration and system test. With the goals and motivated professionals. The third port [3, 4, 5]. established, the development teams must principle of software quality is absolutely Step 3: Manage Requirements make measurable quality plans that can be essential: The developers must feel personally tracked and assessed to ensure that the responsible for the quality of the products they pro- Quality project is producing quality work. This in duce. If they do not, they will not strive to One fundamental truth of all quality pro- turn requires that the quality of the work produce quality results, and later trying to grams is that you must start with a quality be measured at every step, and that the find and fix their defects will be costly, foundation to have any hope of producing quality data be reviewed and assessed by time consuming, and ineffective. a quality result. In software, requirements SM Team Software Process and TSP are service marks of Convincing developers that quality is their are the foundation for everything we do, Carnegie Mellon University. personal responsibility and teaching them so the quality of requirements is para-

June 2008 www.stsc.hill.af.mil 7 Software Quality mount. However, the requirements quality and agreement reached on how to incor- and manage the quality of the process used problem is complicated by two facts. porate this new understanding into the to produce the program’s parts. If, for First, the quality measures must not be development work. This means that the example, we could devise a process that abstract characteristics of a requirements requirements must be recognized as evolv- would consistently produce 1,000-line document; they should be precise and mea- ing through a sequence of versionsB while modules that2 each had less than a one per- surable items such as defect counts from the development estimates, plans, and cent chance of having a single defect, a sys- requirements inspections or counts of commitments are progressing through a tem of 1,000 of these modules would like- requirementsPossib ldefectse found in system test similar but delayed sequence of versions. ly have less than 10 defects per million or customerPath use. However, to be most help- And finally, the product itself will ulti- lines. One obvious problem with this strat- ful, these quality measures must also address mately be produced in a further delayed egy concerns our ability to devise and the precise understanding the developers sequence of versions. The quality manage- properly use such a process. themselves have of the requirements ment problem concerns managing the There has been considerable progress regardless of what the requirements origi- quality and maintaining the synchroniza- in producing and using such a process. nators believe or how good a requirements tion of this sequence of parallel require- This is accomplished by measuring each document has been produced. The develop- ments, plan, design, and product versions. developer’s process and producing a ers will build what they believe the require- Process Quality Index (PQI). The TSP ments say and not what the requirements Step 4: Statistical Process Control quality profile, which forms the basis for developers intended to say. This means that While statistical process control is a large the PQI measure, is shown in Figure 5 [6]. the quality-management problem the subject, we need only discuss two aspects: Then, the developers and their teams use requirements process must address is the process management and continuous standard statistical process management transfer of understanding from the require- process improvement. The first aspect, techniques to manage the quality of all ments experts to the software developers. process management, is discussed here, dimensions of the development work [7]. The second key requirements fact is and process improvement is addressed in Data on early TSP teams show that by fol- that the requirements are dynamic. As peo- Step 8. lowing this practice, quality is substantially ple learn more about what the users need The first step in statistical process improved [8]. and what the developers can build, their management is to redefine the quality Step 5: Quality Evaluation views of what is needed will change. This management strategy. To achieve high lev- fact enormouslyA complicates the require- els of software quality, it is necessary to Quality evaluation has two elements: eval- ments-management problem. The reason switch from looking for defects to manag- uating the quality of the process used to is that people’s understanding of their ing the process. As noted earlier, to achieve produce each product element, and evalu- needs Switevolvesche sgradually and Pathsoften without a quality level of 10 defects per million ating the quality of the products produced any conscious appreciation of how much lines with current software quality manage- by that process. The reason to measure 1 2 and evaluate process quality, of course,is their views4 have changed. There6 is also a ment methods, the developers would have to guide the process-improvement activi- time lag: Even9 when the users20 know that to find and fix all but 10 of the 10,000 to ties discussed in Step 8. The Capability their needs16 have changed, it takes70 time for 20,000 defects in a program with a 40,000 ® ® them to tr36uly understand their92 4new ideas page listing. Unless someone devises a Maturity Model Integration (CMMI ) and to comm49 unicate them to3, the43 2 develop- magic machine that could flawlessly identi- model and appraisal methods were devel- ers. Even 64after the developers12 ,8understand7 0 fy every software defect, it would be clear- oped to guide process-quality assessments, the changes,81 they cannot just48 ,drop62 0 every- ly impossible to improve human search and the TSP process was developed to thing and1 switch0 0 to the new1 version.84 ,75 6 and analysis skills to this degree. guide organizations in defining, using, and To implement40 0 a change,1. the38 E design+1 1 and Therefore, achieving these quality levels improving high-quality processes as well implementation implications of every through better testing, reviews, or inspec- as in measuring, managing, and evaluating requirements change must be appraised; tions is not feasible. product quality. plans, costs, and commitments adjusted; A more practical strategy is to measure To evaluate process quality, the developers and their teams must gather data on Figure 5: TSP Software Quality Profile [6] their work, and then evaluate these data Design/Code Time against the goals they established in their quality plan. If any process or process step falls below the team-defined quality threshold, the resulting products must be evaluated for repair, redevelopment, or replacement, and the process must be Design Review Time Compile D/KLOC brought into conformance. These actions must be taken for every process step and especially before releasing any products from development into testing. In product evaluation, the system integration and testing activities are also measured and evaluated to determine if the final product has reached a suitable quality level or if some remedial action is required.

® Capability Maturity Model and CMMI are registered in the Unit Test D/KLOC Code Review Time U.S. Patent and Trademark Office by Carnegie Mellon University.

8 CROSSTALK The Journal of Defense Software Engineering June 2008 The Software Quality Challenge

Step 6: Defect Analysis Regardless of the quality management References Perhaps the most important single step in methods used (i.e., International Organ- 1. Humphrey, W.S. PSP: A Self-Im- any quality management and improvement ization for Standardization, correctness-by- provement Process for Software Engi- system concerns defect data. Every defect construction, or AS9100) continuous neers. Reading, MA: Addison-Wesley, found after development, whether by final improvement strategies such as those 2005. testing, the users, or any other means must defined by CMMI and TSP should be 2. Jones, C. Software Quality: Analysis be carefully evaluated and the evaluation applied to the improvement process itself. and Guidelines for Success. New York: results used to improve both the process This means that the process quality mea- International Thompson Computer and the product. The reason that these sures, the evaluation methods, and the deci- Press, 1997. data are so important is that they concern sion thresholds must also be considered as 3. Humphrey, W.S. Winning With Soft- the process failings. Every defect found important aspects of continuous process ware: An Executive Strategy. Reading, after development represents a failure of improvement. Furthermore, since every MA: Addison-Wesley, 2002. the development process, and each such developer, team, project, and organization 4. Humphrey, W.S. TSP: Leading a De- failure must be analyzed and the results is different, it means that this continuous velopment Team. Reading, MA: used to make two kinds of improvements. improvement process must involve every Addison-Wesley, 2006. The first improvement – and the one person on every development team and on 5. Humphrey, W.S. TSP: Coaching Devel- that requires the most rapid turnaround every project in the organization. opment Teams Reading, MA: Addi- time – is determining where in the product Conclusion son-Wesley, 2006. similar defects could have been made and 6. Humphrey, W.S. “Three Dimensions of taking immediate action to find and fix all While we face a major challenge in improv- Process Improvement, Part III: The of those defects. The second improvement ing software quality, we also have substan- Team Process” CrossTalk Apr. 1998. activity is to analyze these defects to deter- tial and growing quality needs. It should 7. Florac, S., and A.D. Carleton. Measur- mine how to prevent similar defects from now be clear to just about everyone in the ing the Software Process: Statistical being injected in the future, and to devise a software business that the current testing- Process Control for Software Process based quality strategy has reached a dead means to more promptly find and fix all Improvement. Reading, MA: Addison end. Software development groups have such defects before final testing or release Wesley, 1999. struggled for years to get quality improve- to the user. 8. Davis, N., and J. Mullaney. “Team ments of 10 to 20 percent by trying differ- Software Process in Practice.” SEI Step 7: Configuration Management ent testing strategies and methods, by Technical Report CMU/SEI-2003-TR experimenting with improved testing tools, For any large-scale development effort, -014, Sept. 2003. and by working harder. configuration management (CM) is critical. The quality improvements required are About the Author This CM process must cover the product vast, and such improvements cannot be artifacts, the requirements, the design, and achieved by merely bulling ahead with the Watts S. Humphrey the development process. It is also essen- test-based methods of the past. While the joined the Software En- tial to measure and manage the quality of methods described in this article have not the CM process itself. Since CM processes yet been fully proven for software, we now gineering Institute (SEI) are relatively standard, however, they need have a growing body of evidence that they after his retirement from not be discussed further. will work – at least better than what we IBM. He established the SEI’s Process Program Step 8: Process Improvement have been doing. What is more, this quality strategy uses the kinds of data-based and led development of the CMM for The fundamental change required by this methods that can guide long-term contin- Software, the PSP, and the TSP. He man- software quality-management strategy is to uous improvement. In addition to improv- aged IBM’s commercial software devel- use the well-proven methods of statistical ing quality, this strategy has also been opment and was vice president of tech- process control to guide continuous shown to save time and money. process improvement [7]. Here, however, nical development. He is an SEI Fellow, Finally, and most importantly, software an Association of Computing Machin- we are not talking about improving the tol- quality is an issue that should concern every- ery member, an Institute of Electrical erances of machines or the purity of mate- one. Poor quality software now costs each of rials; we are talking about managing the us time and money. In the immediate future, and Electronics Engineers Fellow, and a quality levels of what people do, as well as it is also likely to threaten our lives and liveli- past member of the Malcolm Baldrige the quality levels of their work products. hoods. Every one of us, whether a develop- National Quality Award Board of While people will always make mistakes, er, a manager, or a user, must insist on qual- Examiners. In a recent White House cer- they tend to make the same mistakes over ity work; it is the only way we will get the emony, the President awarded him the N and over. As a consequence, when devel- kind of software we all need. National Medal of Technology. He opers have data on the defects they per- holds graduate degrees in physics and Acknowledgements sonally inject during their work and know business administration. how to use these data, they and their team- My thanks to Bob Cannon, David mates can learn how to find just about all Carrington, Tim Chick, Taz Daughtrey, SEI of the mistakes that they make. Then, in Harry Levinson, Julia Mullaney, Bill 4500 Fifth AVE defining and improving the quality-man- Nichols, Bill Peterson, Alan Willett, and Pittsburgh, PA 15213-2612 agement process, every developer must use Carol Woody for reviewing this article and Phone: (412) 268-6379 these data to optimally utilize the full range offering their helpful suggestions. I also Fax: (412) 268-5758 of available defect detection and preven- much appreciate the constructive sugges- E-mail: [email protected] tion methods. tions of the CrossTalk editorial board.

June 2008 www.stsc.hill.af.mil 9