University of Tennessee, Knoxville TRACE: Tennessee Research and Creative Exchange The Harlan D. Mills Collection Science Alliance 10-5-1988 A Case Study in Cleanroom Software Engineering: The IBM Cobol Structuring Facility Richard C. Linger Harlan D. Mills Follow this and additional works at: https://trace.tennessee.edu/utk_harlan Part of the Software Engineering Commons Recommended Citation Linger, Richard C. and Mills, Harlan D., "A Case Study in Cleanroom Software Engineering: The IBM Cobol Structuring Facility" (1988). The Harlan D. Mills Collection. https://trace.tennessee.edu/utk_harlan/34 This Conference Proceeding is brought to you for free and open access by the Science Alliance at TRACE: Tennessee Research and Creative Exchange. It has been accepted for inclusion in The Harlan D. Mills Collection by an authorized administrator of TRACE: Tennessee Research and Creative Exchange. For more information, please contact [email protected]. after the Norman conquest was recorded in analysis. and structure chart generation for Roman numerals, but never added up in the output program procedure hierarchy. spite of the obvious interest in such a result. Some 52,000 lines of PL/I source code. new Now the experts in arithmetic of the day and changed, were written to produce would never have believed that with place Version 2. with 28,000 lines reused from notation and long division. school children Version l. of later centuries would be capable of arithmetic performance these experts Version 2 was developed by a Cleanroom deemed impossible. And so it will be that software team composed of a technical may current experts in heuristic. intuitive engineering manager. six software methods of software development will find engineers. and a certification engineer.l the use of mathematical verification in Three summer supplemental college place of trial and error unit debugging students also participated. Team members impossible to consider as a rational held BS or MS degrees in computer science methodology. or mathematics and had recently joined IBM. With the exception of the team The COBOL Structuring Facility manager and certification engineer, COBOL/SF was their first software The COBOL Structuring Facility (COBOL/SF development project. l988a. l988b) is comparable in function and complexity to a modem high-level The Version 2 development proceeded language compiler. It embodies proprietary through formal specification, design. graph- and function-theoretic technology functional verification, implementation, to automatically transform unstructured and Cleanroom testing in five increments, COBOL programs into hierarchies of beginning on April 15 and completing structured procedures. COBOL/SF helps December 15. 1987. Seventy person-months solve difficult software maintenance (eight full-time people for eight months. problems by reducing complexity and plus three supplementals for two months increasing understandability of program each) of effort were expended during this logic. development period, for an overall productivity of 740 lines of code per Table l summarizes the development person/month, including all specification, history of COBOL/SF. This paper reports on design. implementation, testing, and development of Version 2; results for other management activities. The system entered versions. also developed with Cleanroom field test at customer sites on January 6, Software Engineering, were similar. 1988. Lines of Code (KLOC) Version 2 development was a real-world Reused Changed New Total project in every respect, with shifting Prototype 0 0 20 20 requirements and an extremely short Version l 18 2 15 35 development schedule. All schedules and Version lA 30 5 11 46 budgets were met. and all committed Version 2 28 18 34 80 functions were delivered. All versions of COBOL/SF consist of four Table 1. major components as show in Figure 1. The COBOL/SF Development Summary System Control Program manages system software and user interfaces, and certain The COBOL/SF prototype and versions l common services. The Source Language and lA provided structuring capability for Parsing Subsystem parses the input VS COBOL II programs into VS COBOL II program and creates a knowledge base of only. Version 2 incorporated the following program structure. The input program is major additional functions: structuring of prepared for structuring by the Control OS/VS COBOL programs into either OS/VS Flow Analysis Subsystem, which deals COBOL or VS COBOL II, automation of optional manual steps to enhance the structuring process. complexity metrics ITeam members: K. Cannaday, M. Deck, P. analysis, program modularization Hausler. R Linger, C. Loving, L. Pedowitz, S. Rosen. A Spangler 2 A Case Study in Cleanroom Software Engineering: The IBM COBOL Structuring Facility Richard C. Linger Harlan D. Mills IBM Corporation University of Florida Bethesda, Maryland Gainesville, Florida and Software Engineering Technology, Inc. Vero Beach, Florida Abstract programs with only 10 errors detected. All errors were trivial, none requiring more than a few hours to find and fix, and most The IBM COBOL Structuring Facility just a few minutes. In all testing, only one Program Product was developed by a small error resulted in a COBOL program failing programming team using Cleanroon to execute functionally equivalent before Software Engineering technology in a and after structuring. As confidence in pipeline of increments with very high quality grew, field test participants engaged quality and productivity. In the Cleanroom in wholesale structuring of entire systems approach, programs are developed under of COBOL programs, in effect. treating the statistical quality control. and field test version of COBOL/SF as a final mathematical verification is used in place product. of unit debugging. The formal methods of specification, design, functional Since the common wisdom in software verification, and testing are d,escribed, engineering is that mathematical together with development and verification of sizable software products is management practices required for impractical and that unit debugging by maintaining intellectual control over the programmers is necessary, these results process. may appear incredible. As far as we know, the axiomatic verification (Hoare 1969. A Cleanroom Software Case Study Gries 1981) of software as widely taught in university computer science courses today The IBM COBOL Structuring Facility is indeed impractical for products of this (COBOL/SF) Version 2 Program Product size. However, functional verification automatically transforms unstructured (Linger 1979) was used for COBOL/SF. And COBOL programs into structured form. It even functional verification for products of was developed by a small programming this size is impractical. except for teams team using Cleanroom Software whose members are well educated in formal Engineering technology [Mills 1987). methods of specification, design and COBOL/SF Version 2 consists of 80,000 functional verification. Team members lines (52,000 new and changed over Version must be provided further intemships in 1) of high function source code that was team operations, for scaling up such formal developed under statistical quality control, methods into work products and processes being specified, then designed, that permit day-to-day work to accumulate mathematically verified, and coded with no into mathematical verifications of unit debugging in a pipeline of increments software products of any size. at very high productivity. Each increment was placed under engineering change It is understandable that the common control before any execution and subjected wisdom of such a new subject as software to system test under a sound statistical engineering can underestimate the design. potentials of human achievement in various ways. Centuries ago, the common As a result, COBOL/SF passed its field test wisdom in arithmetic with Roman of structuring a half-million .. lines of numerals was that large scale arithmetic COBOL code in over 300 application was impractical, so that the great inventory 1 with structural problems caused by Cleanroom Software Engineering complex perlormed procedure logic, ALTER statements. etc. The Structured Program Traditional software development proceeds Generation Subsystem transforms the through steps of specification. design. and input program into structured form and code, then unit, component, and system generates code. Finally. an off-line Parser testing. Selective tests are invented with Generator compiles COBOL grammars into knowledge of programmed intemals. often parse tables for use by the system. by the developers themselves, typically to exercise primary functions. then secondary COBOL/SF Version 2 was developed top functions, error cases. etc. On completion down in five increments as depicted in of testing, the software is known to work as Table 2. With no unit debugging permitted, tested. but can still fail in circumstances the error rates shown are measured from not tested. As a result, the reliability first execution through the completion of evidence of selective testing is entirely Cleanroom testing. They range from 1.4 to anecdotal; it is known only that the 5. 7 errors I KLOC of source code, with an software passed certain tests. with no average of 3.4 errors I KLOC. Table 2 inference possible of future failure rates. suggests a possible correlation between Worse. selective testing provides no increment size and error rate, however. no rational basis for managing development.