A Survey of Commonly Applied Methods for Software Process Improvement R. Austin D. Paulish February 1993 TECHNICAL REPORT CMU/SEI-93-TR-027 Software Process Measurement Project Unlimited distribution subject to the copyright. This technical report was prepared for the SEI Joint Program Office ESO/ENS Hanscom AFB, MA 01731-2116 The Ideas and findings In this report should not be construed as an official DoD position. ft Is published In the interest of scientific and technical Information exchange. Review and Approval This report has been reviewed and Is approved for publication. FOR THE COMMANDER Thomas R. Miller, Lt Col, USAF SEI Joint Program Office The Software Engineering Institute Issponsored by the U.S. Department of Defense. This report was funded by the U.S. Department of Defense. Copyright 0 1994 by Carnegie Mellon University. Copies of the documen are available from Research Access. Iinc., 800 Vinlal Street, 2fteburgh. PA 15212. Telephone: (412) 321-2902 or 148004"5410. Fax: (412) 321-2994. This documentf is waullble through the Defense Technical information Cente. IYTIC. provides acces to and transfer of scientifi and technical Informaton for DoD personnel, DoD) c wontacoraid potential corjsaclors, and othesr U.S. Government agnypersonnel and thief contractors. To obtain a cOpy. please Contact DTIC drectly: Defense Technial Information Ce tnte.A:FDRA. Carmeron Station. Alexandria. VA 2230446145. copies of this documnent are also avalale through the National Technwica nfornmaton service. For iniormation on ordering, pleas contact NTIS diectly: National Technical hIforniaton Service, U.S. Deparment of Commerce .Sprinfild VA 22161. Use of any trademmast Inthis report Isnot intended Inany way to kintnge on the rights of the trademark holder. Table of Contents Preface vii Introduction 1 1.1 Motivation for Software Process Improvement 1 1.2 Purpose and Scope 3 1.3 Related Efforts 4 2 What Is a Process Improvement Method? 7 2.1 Definition 7 2.2 Report Content 10 3 Estimation 13 3.1 Overview 13 3.2 History/Background 13 3.3 Estimation Models 13 3.3.1 Statistical Models 14 3.3.2 Staff-Month Estimates 14 3.3.3 Function Point Analysis 15 3.3.4 Independent Variables 15 3.3.5 Expert Judgment Based Estimation 16 3.4 Difficulties with Estimation Models 16 3.5 Experience with Estimation 17 3.6 Suggestions for Introduction and Use 19 3.7 How Estimation Is Related to the Capability Maturity Model 20 3.8 Summary Comments on Estimation 20 3.9 References and Further Readings - Estimation 20 4 ISO 9000 Certification 25 4.1 Overview 25 4.2 History/Background 25 4.3 Description of ISO 9000 25 4.4 Experience with ISO 9000 27 4.4.1 Reported Benefits 27 4.4.2 Potential Problems 28 4.5 Suggestions for Introduction and Use 29 4.6 How ISO 9000 Is Related to the Capability Maturity Model 31 4.7 Summary Comments on ISO 9000 32 4.8 References and Further Readings - ISO 9000 32 5 Software Process Assessment (SPA) 35 5.1 Overview 35 CMU/SEI-93-TR-27 5.2 History/Background 35 5.3 Description of Software Process Assessment 35 5.4 Assessment Results 36 5.5 Other Assessment Methods 37 5.6 Experience with Software Process Assessment 37 5.7 Potential Problems 38 5.8 Suggestions for Introduction and Use 38 5.9 How Software Process Assessment Is Related to the Capability Maturity Model 40 5.10 Summary Comments on Software Process Assessment 40 5.11 References and Further Readings - SPA 40 6 Process Definition 43 6.1 Overview 43 6.2 History/Background 43 6.3 Description of Process Definition 44 6.4 Experience with Process Definition 45 6.5 Related Research 46 6.6 Suggestions for Introduction and Use 47 6.7 How Process Definition Is Related to the Capability Maturity Model 48 6.8 Summary Comments on Process Definition 48 6.9 References and Further Readings - Process Definition 49 7 Formal Inspection 51 7.1 Overview 51 7.2 History/Background 51 7.3 Description of the Formal Inspection Method 51 7.4 Experience with Formal Inspection 52 7.5 Suggestions for Introduction and Use 53 7.6 How Formal Inspection Is Related to the Capability Maturity Model 55 7.7 Summary Comments on Formal Inspection 55 7.8 References and Further Readings - Formal Inspection 56 8 Software Measurement and Metrics 59 8.1 Overview 59 8.2 History/Background 59 8.3 Description of Software Measurement 60 8.3.1 Example Measures 60 8.4 Experience with Software Measurement 61 8.5 Suggestions for Introduction and Use 62 8.6 How Software Measurement Is Related to the Capability Maturity Model 63 8.7 Summary Comments on Measurement 65 CMU/SEI-93-TR-27 8.8 References and Further Readings - Measurement 65 9 Computer Aided Software Engineering (CASE) 67 9.1 Overview 67 9.2 History/Background 67 9.3 Description of CASE 68 9.3.1 Examples of CASE Tools 68 9.3.2 Upper and Lower CASE 69 9.4 Experience with CASE 70 9.5 Suggestions for Introduction and Use 71 9.6 How CASE Is Related to the Capability Maturity Model 72 9.7 Summary Comments on CASE 73 9.8 References and Further Readings - CASE 73 10 Interdisciplinary Group Methods (IGMs) 75 10.1 Overview 75 10.2 History/Background 75 10.3 Description of Interdisciplinary Group Methods 75 10.3.1 Group Dynamics Facilitation 76 10.3.2 Groupware Methods 77 10.3.3 Team Organization 78 10.4 Experience with Interdisciplinary Group Methods 78 10.4.1 Group Dynamics Facilitation 79 10.4.2 Groupware Methods 79 10.4.3 Team Organization 80 10.5 Suggestions for Introduction and Use 81 10.5.1 Group Dynamics Facilitation 81 10.5.2 Groupware Methods 82 10.5.3 Team Organization 82 10.6 How Interdisciplinary Group Methods Are Related to the Capability Maturity Model 83 10.7 Summary Comments on Interdisciplinary Group Methods 83 10.8 References and Further Readings - IGMs 84 11 Software Reliability Engineering (SRE) 87 11.1 Overview 87 11.2 History/Background 87 11.3 Description of Software Reliability Engineering 88 11.4 Experience with SRE 89 11.5 Suggestions for Introduction and Use 91 11.6 How SRE Is Related to the Capability Maturity Model 91 11.7 Summary Comments on SRE 91 11.8 References and Further Readings - SRE 92 CMU/SEI-93-TR-27 12 Quality Function Deployment (QFD) 95 12.1 Overview 95 12.2 History/Background 95 12.3 Description of QFD 95 12.4 Experience with QFD 99 12.5 Suggestions for Introduction and Use 100 12.6 How QFD Is Related to the Capability Maturity Model 101 12.7 Summary Comments on QOD 101 12.8 References and Further Readings - QFD 101 13 Total Quality Management (TQM) 103 13.1 Overview 103 13.2 History/Background 103 13.3 Description of TQM 103 13.4 TOM Tools 105 13.5 Experience with TQM 106 13.6 Suggestions for Introduction and Use 107 13.7 How TOM Is Related to the Capability Maturity Model 108 13.8 Summary Comments on TQM 108 13.9 References and Further Readings - TOM 109 14 Defect Prevention Process (DPP) 111 14.1 Overview 111 14.2 History/Background 111 14.3 Description of the Defect Prevention Process 111 14.4 Experience with Defect Prevention 112 14.5 Suggestions for Introduction and Use 113 14.6 How Defect Prevention Is Related to the Capability Maturity Model 113 14.7 References and Further Readings - DPP 114 15 Cleanroom Software Development 115 15.1 Overview 115 15.2 History/Background 115 15.3 Description of Cleanroom Software Development 115 15.4 Experience with Cleanroom Software Development 117 15.5 Suggestions for Introduction and Use 119 15.6 How Cleanroom Software Development Is Related to the Capability Maturity Model 119 15.7 Summary Comments on Cleanroom Software Development 120 15.8 References and Further Readings - Cleanroom 120 16 Conclusions 123 iv CMU/SEI-93-TR-27 List of Figures Figure 1-1 Which Software Process Improvement Methods? 2 Figure 1-2 Process Improvement Approach 3 Figure 2-1 Example Software Process Improvement Methods 10 Figure 8-1 Measurement Activities for Organizations at Various CMM Levels 64 Figure 12-1 *House of Quality" Diagram 98 Figure 16-1 Characteristics of a Software Development Organization 124 CMU/SEI-S3-TR-27 List of Tables Table 1: Software Process Improvement Methods 8 Table 2: Vertical CASE Tools 68 Table 3: Horizontal CASE Tools 69 Table 4: Implementation Issues Summary 125 vi CMU/SEI-93-TR-27 Preface This report is an output of a joint Software Engineering Institute (SEI)/Siemens project in which Siemens software development organizations are being used as case study sites to measure and observe the impact of methods used to improve the software de- velopment process. The project will result in the identification of specific interventions that can be tailored to the current maturity level of an organization that wishes to im- prove. The project is intended to assist software development managers in selecting from the commonly applied software process improvement methods. This report describes methods that are commonly used within industrial organizations for improving the software development process. The report can be used by software engineering managers who are considering alternative actions for improving their de- velopment process. It could also prove useful to change agents of software process improvement, such as found within a software engineering process group (SEPG), and practitioners such as software engineers and quality specialists. The reader of this report should learn about a number of specific process improvement methods in- cluding key references for obtaining further information about the application and ex- periences of other organizations with the described method.