Software Defect Reduction Top 10 List
Total Page：16
File Type：pdf, Size：1020Kb
SOFTWARE MANANGEMENT TWO Current software projects spend about Software 40 to 50 percent of their effort on avoid- able rework. Such rework consists of effort spent fixing software difficulties that could Defect Reduction have been discovered earlier and fixed less expensively or avoided altogether. By implication, then, some effort must con- sist of “unavoidable rework,” an obser- Top 10 List vation that has gained increasing credibility with the growing realization Barry Boehm, University of Southern California that better user-interactive systems result from emergent processes. In such Victor R. Basili, University of Maryland processes, the requirements emerge from prototyping and other multistakeholder- shared learning activities, a departure from traditional reductionist processes that stipulate requirements in advance, ecently, a National Science insight has been a major driver in focus- then reduce them to practice via design Foundation grant enabled us to ing industrial software practice on thor- and coding. Emergent processes indicate establish the Center for Empiri- ough requirements analysis and design, that changes to a system’s definition that R cally Based Software Engineer- on early verification and validation, and make it more cost-effective should not be ing. CeBASE seeks to transform on up-front prototyping and simulation discouraged by classifying them as avoid- software engineering as much as pos- to avoid costly downstream fixes.” able defects. sible from a fad-based practice to an engineering-based practice through deri- vation, organization, and dissemination Software’s complexity and of empirical data on software develop- accelerated development ment and evolution phenomenology. The phrase “as much as possible” reflects the schedules make avoiding defects fact that software development must remain a people-intensive and continu- difficult. These 10 techniques can ally changing field. We have found, how- help reduce the flaws in your code. ever, that researchers have established objective and quantitative data, relation- ships, and predictive models that help For this updated list, we have added Reducing avoidable rework can pro- software developers avoid predictable pit- the word “often” to reflect additional vide significant improvements in software falls and improve their ability to predict insights about this observation. One productivity. In our behavioral analysis and control efficient software projects. insight shows the cost-escalation factor of how software cost drivers affected Here we describe developments in this for small, noncritical software systems to effort for the Cocomo II model (B. Boehm area that have taken place since the publi- be more like 5:1 than 100:1. This ratio et al., Software Cost Estimation with cation of “Industrial Metrics Top 10 List” reveals that we can develop such systems Cocomo II, Prentice Hall, 2000), we in 1987 (B. Boehm, IEEE Software, Sept. more efficiently in a less formal, continu- found that most of the effort savings gen- 1987, pp. 84-85). Given that CeBASE ous prototype mode that still emphasizes erated by improving software process places a high priority on software defect re- getting things right early rather than late. maturity, software architectures, and soft- duction, we think it is fitting to update that Another insight reveals that good ware risk management came from reduc- earlier article by providing the following architectural practices can significantly tions in avoidable rework. Software Defect Reduction Top 10 List. reduce the cost-escalation factor even for large critical systems. Such practices THREE ONE reduce the cost of most fixes by confining About 80 percent of avoidable rework Finding and fixing a software problem them to small, well-encapsulated mod- comes from 20 percent of the defects. after delivery is often 100 times more ules. A good example is the million-line That 80 percent value may be lower expensive than finding and fixing it dur- CCPDS-R system described in Walker for smaller systems and higher for very ing the requirements and design phase. Royce’s book, Software Project Manage- large ones. Two major sources of avoid- As Boehm observed in 1987, “This ment (Addison-Wesley, 1998). able rework involve hastily specified January 2001 135 Software Management requirements and nominal-case design them later, we seek techniques that find tions. Improvements in fault detection and development, in which late accom- defects as early as possible. Numerous rates vary from 15 to 50 percent. Further, modation of off-nominal requirements studies confirm that peer review provides focused reading techniques facilitate train- causes major architecture, design, and an effective technique that catches from ing of inexperienced personnel, improve code breakage. A tracking system for 31 to 93 percent of the defects, with a communication about the process, and software-problem reports that records median of around 60 percent. Thus, the foster continuous improvement. the effort to fix each defect lets you ana- 60 percent value cited in the 1987 col- lyze the data fairly easily to determine umn remains a reasonable estimate. EIGHT and address additional major sources of Disciplined personal practices can rework. reduce defect introduction rates by up to 75 percent. FOUR Peer reviews, Several disciplined personal processes About 80 percent of the defects come analysis tools, have been introduced into practice. from 20 percent of the modules, and and testing catch These include Harlan Mills’s Cleanroom about half the modules are defect free. different classes of software development process and Watts Studies from different environments defects at different Humphrey’s Personal Software Process over many years have shown, with amaz- points in the (PSP). ing consistency, that between 60 and 90 development cycle. Data from the use of Cleanroom at percent of the defects arise from 20 per- NASA have shown 25 to 75 percent reduc- cent of the modules, with a median of tions in failure rates during testing. Use of about 80 percent. With equal consis- Cleanroom also showed a reduction in tency, nearly all defects cluster in about Factors affecting the percentage of rework effort so that only 5 percent of the half the modules produced. defects caught include the number and fixes took more than an hour, whereas the Obviously, then, identifying the char- type of peer reviews performed, the size standard process caused more than 60 per- acteristics of error-prone modules in a and complexity of the system, and the cent of the fixes to take that long. particular environment can prove worth- frequency of defects better caught by exe- PSP’s strong focus on root-cause analy- while. A variety of context-dependent cution, such as concurrency and algo- sis of an individual’s software defects and factors contribute to error-proneness. rithm defects. Our studies have provided overruns, and on developing personal Some factors usually contribute to error- evidence that peer reviews, analysis tools, checklists and practices to avoid future proneness regardless of context, however, and testing catch different classes of recurrence, has significantly reduced per- including the level of data coupling and defects at different points in the devel- sonal defect rates. Developers frequently cohesion, size, complexity, and the opment cycle. We need further empirical enjoy defect reductions of 10:1 between amount of change to reused code. research to help choose the best mixed exercises 1 and 10 in the PSP training strategy for defect-reduction investments. course. FIVE Effects at the project level are more scat- About 90 percent of the downtime SEVEN tered. They depend on factors such as the comes from, at most, 10 percent of the Perspective-based reviews catch 35 organization’s existing software maturity defects. percent more defects than nondirected level and the staff’s and organization’s Some defects disproportionately affect reviews. willingness to operate within a highly a system’s downtime and reliability. For A scenario-based reading technique structured software culture. When you example, an analysis of the software fail- (V.R. Basili, “Evolving and Packaging couple PSP with the strongly compatible ure history of nine large IBM software Reading Technologies,” J. Systems and Team Software Process (TSP), defect products revealed that about 0.3 percent Software, vol. 38, no. 1, 1997, pp. 3-12) reduction rates can soar to factors of 10 or of the defects accounted for about 90 offers a set of formal procedures for higher for an organization that operates percent of the downtime. Thus, risk- defect detection based on varying per- at a modest maturity level. Results tend to based testing—including understanding spectives. The union of several perspec- be less spectacular if the organization a system’s operational profiles and tives into a single inspection offers broad already employs highly mature processes. emphasizing testing of high-risk scenar- yet focused coverage of the document The June 2000 special issue of ios—is clearly cost-effective. being reviewed. This approach seeks to CrossTalk, “Keeping Time with PSP and generate focused techniques aimed at TSP,” offers a good set of relevant dis- SIX specific defect-detection goals by taking cussions, including experience showing Peer reviews catch 60 percent of the advantage of an organization’s existing that adding PSP and TSP to a CMM defects. defect history. Level 5 organization reduced acceptance Given that finding and fixing most Scenario-based reading techniques have test defects by about 50 percent overall, defects earlier in the project development been applied in requirements, object-ori- and reduced high-priority defects by cycle is more cost-effective than finding ented design, and user interface inspec- about 75 percent. 136 Computer NINE A 1987 study in this area (P.S. Brown software engineering research challenge All other things being equal, it costs 50 and J.D. Gould, “An Experimental Study is one of several identified by a National percent more per source instruction to of People Creating Spreadsheets,” ACM Science Foundation study, “Gaining develop high-dependability software Trans.