Engineering Software Under Statistical Quality-Control
Total Page:16
File Type:pdf, Size:1020Kb
University of Tennessee, Knoxville TRACE: Tennessee Research and Creative Exchange The Harlan D. Mills Collection Science Alliance 11-1990 Engineering Software Under Statistical Quality-Control R. H. Cobb Harlan D. Mills Follow this and additional works at: https://trace.tennessee.edu/utk_harlan Part of the Software Engineering Commons Recommended Citation Cobb, R. H. and Mills, Harlan D., "Engineering Software Under Statistical Quality-Control" (1990). The Harlan D. Mills Collection. https://trace.tennessee.edu/utk_harlan/14 This Article is brought to you for free and open access by the Science Alliance at TRACE: Tennessee Research and Creative Exchange. It has been accepted for inclusion in The Harlan D. Mills Collection by an authorized administrator of TRACE: Tennessee Research and Creative Exchange. For more information, please contact [email protected]. Engineering Software under Statistical Quality Control Richard H. Cobb and Harlan D. Mills, Software Engineering Technology Thecosbof ociety has been developing soft- culty producing reliable software there is continuing to develop ware for less than one human gen- a demand for even more complex, larger Seration. We have accomplished a software systems. failure-lden 8offws1ye great deal in this first generation when These problems are symptoms of a pre with its associated compared to the accomplishments of cess that is not yet under intellectual con- other disciplines:During the first genera- trol. An activity is under intellectual con- low prductivity are tion of civil engineering, the right triangle trol when the people performing it use a unaamptable. hadn't been invented; accountants did theoretically sound process that gives C1-r- not discover doubleentry concepts in the each of them a high probability of obtain- early generations of their field. ing a commonlyaccepted correct answer. englneeri~promises Yet despite such significant progress, When most endeavors begin, they are lower costs and softwaredevelopment practices need im- out of intellectual control. Intellectual improved qualit~c provement. We must solve such problems control is achieved when theories are de- as veloped, implementation practices are re- execution failures, which exist to the fined, and people are taught the process. extent that software failures are accepted A good example is long division. For as normal by most people, many generations, performing division projects that are late and/or over bud- with Roman numerals was error-prone. get, and Today, children who learn how to do long the labor-intensive nature of software division with Arabic numerals obtain the development - productivity increases correct answer most of the time. The long have been modest since the introduction division algorithm is: of Cobol. 1. If the division is not complete, invent And at the same time we are having diffi- (estimate) the next digit. 44 074@7459/90/11oO/O044/$01.oO 0 1990 IEEE IEEE Software 2. Verify the invention (estimate) made ects using some or all of these new soft- is because software developers rely on an in step 1. waredevelopment practices. incomplete theory, so their engineering 3. If the verification is correct and the practices don’twork. division is not complete, repeat step 1 for Software myths Software engineers should be required the next digit; if the verification is not cor- Myth: Softwarefailures are unavoidable. to use engineering practices that produce rect, repeat step 1 for the same digit by This myth holds that software always software that does not contain faults that adjusting the invention. contains latent execution failures that will cause latent execution failures. Users This is a powerful algorithm for estab be found by users. Therefore, we must want the same high assurance that soft- lishing intellectual control. A difficult learn to live with and manage around soft- ware will work according to its specifica- problem, which on the surface seems to ware failures. tion that they have for products designed require a large invention, has been di- Fact: Like other engineering activities, by other engineers. vided into a series of smaller problems, engineering software is a human activity The software profession is young, so we each requiring a smaller invention. Most subject to human fallibilities.Yet other en- might want to start with modest goals, important, each inventive step is followed gineering disciplines have learned how to such as: Design and implement a 100,000 immediately by a verification step to ap design large and complex products with a line system so, more often than not, no praise the invention’s correctness, so sub low probability that the designs contain execution failure will be detected during sequent inventions don’t build on incor- the software’s entire field life. rect results. Even this modest goal is beyond our ex- This algorithm also applies to software Ekperience to date pectations using the development prac- design and development. As software supports our belief that tices we now rely on. We believe such a goal is well within our capabilitiesifwe use technologists strive to find better ways to as software developers develop software,we believe that they are the ideas summarized in this article. For hindered by some widely accepted beliefs move from today’s example, the software for the IBM Wheel- about how to develop software. We believe heuristic proglamming writer typewriter, developed using some that if we adopt new perspectives about to rigwous somvare of these ideas, has been in use for more these development myths, we will open englneering, quality than six years with millions of users and the way to development practices that will will increase and costs no software failure has ever been re- permit the construction of software that ported.’ contains few, if any, latent failures. will decrease. We have used new perspectivesto derive Myth: Qualily costs mmq Cleanroom engineering practices. Clean- Many people believe that software de- room engineering develops software faults that will cause latent execution fail- signed to execute with no or few failures under statistical quality control by ures. When structural engineers design a costs more per line of code to produce. specifymg statistical usage, bridge there is a high expectation that the Fact: Failures and cost are positively cor- defining an incremental pipeline for bridge, when built, will not fall down. related. It is more expensive to remove la- software construction that permits statisti- In other engineering disciplines, design tent execution failures designed into the cal testing, and failures are neither anticipated nor ac- software than to rigorously design the separating development and testing cepted as normal. When a failure hap software to prevent execution failures. (onlytesters compile and execute the soft- pens, major investigationsare undertaken For example, touch-typing is both more ware being developed). to determine why it occurred. Other engi- reliable and productive than hunt-and- These practices have been demon- neering professions have minimized peck typing. strated to provide higher quality software error by developing a sound, theoretical We believe - and experience to date - software with fewer latent execution base on which to build design practices. supports our belief- that as software de- failures. These same engineering prac- But because software developersexpect velopers move from today’s heuristic pre tices also have been observed to improve and accept design failures, software users gramming to rigorous software engineer- productivity. Table 1 summarizes some cannot have the same high expectations ing, quality will increase and design and quality and productivity metrics for proj- as users of other products. We believe this development costs will decrease. November 1990 45 Table 1. Selected sample of Cleanroom projects. (All other projects known to authors report substantial improvements in quality and productivity.) Applied Year technologies Implementation Results 1980 Stepwise refinement Census,25 KLOC (Pascal) No failure ever found Functional verification Programmer received gold medal from Baldridge 1983 Functional verification Wheelwriter, 63 KLOC, Millions of users Inspections three processors No failure ever found 1980s Functional verification Space shuttle, 500 KLOC Low defect over entire function Inspections No defect in any flight Work received NASA’s Quality Award 1987 Cleanroom engineering Flight control, 33 KLOC (Jovial) ,, Completed ahead of schedule three increments 2.5 errors/KLOC before any execution Error-fix effort reduced by a factor of five 1988 Cleanroom engineering Commercial product, 80 KLOC (PL/I) Certification testing failure rate of 3.4 failures/KLOC Deployment failures of O.l/KLOC Productivityof 740 lines/man-month 1989 Partial Cleanroom Satellite control, 30 KLOC (Fortran) Certification testing error rate engineering of 3.3 failures/KLOC 50-percent improvement in quality Productivityof 780 lines/man-month 80-percentimprovement in productivity 1990 Cleanroom engineering Research project, 12 KLOC Certified to 0.9978 with 989 test cases; 36 with reuse and new Ada (Adaand ADL) failures found during certification (20 design language logic errors, or 1.7 errors/KLOC Myth: Unit vmjicatim by debugging works These failures are then either found dur- ects that used unit verification by logical on system ofany siz. ing integration testing or left in the prod- argument. All our experience with this Unitverification- debugging -is best