Handbook of Software Quality Assurance, 4Th
Total Page:16
File Type:pdf, Size:1020Kb
Handbook of Software Quality Assurance Fourth Edition For a listing of recent related Artech House titles, turn to the back of this book. Handbook of Software Quality Assurance Fourth Edition G. Gordon Schulmeyer Editor a r techhouse. com Library of Congress Cataloging-in-Publication Data A catalog record for this book is available from the U.S. Library of Congress. British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library. ISBN-13: 978-1-59693-186-2 Cover design by Igor Valdman © 2008 ARTECH HOUSE, INC. 685 Canton Street Norwood, MA 02062 All rights reserved. Printed and bound in the United States of America. No part of this book may be reproduced or utilized in any form or by any means, electronic or mechanical, includ- ing photocopying, recording, or by any information storage and retrieval system, without permission in writing from the publisher. All terms mentioned in this book that are known to be trademarks or service marks have been appropriately capitalized. Artech House cannot attest to the accuracy of this informa- tion. Use of a term in this book should not be regarded as affecting the validity of any trade- mark or service mark. 10 9 8 7 6 5 4 3 2 1 For my grandchildren, Jimmy, Gabrielle, Chandler, Mallory, Neve, and Julian In memory of James H. Heil, prior contributor to former editions of this handbook The following copyrights, trademarks, and service marks appear in the book and are the property of their owners: Capability Maturity Model®, Capability Maturity Modeling®, CMM®, CMMI® are registered in the U.S. Patent and Trademark Office by Carnegie Mellon University. CMM® Integration, TSP, PSP, IDEAL, SCAMPI, SCAMPI Lead Assessor, and SCAMPI Lead Appraiser are service marks of Carnegie Mellon University. CobiT is registered in the U.S. Patent and Trademark Office by Information Systems Audit and Control Association. Excel, MS, Word for Windows are trademarks of Microsoft Corporation. Gold Practice is a Trademark of the Data and Analysis Center for Software. IBM is a registered trademark of IBM Corporation. IEEE Std 730TM-2002 is a trademark of the IEEE Computer Society. IEEE Standard for Software Reviews, IEEE Std 1028-1997 reprinted with permission from IEEE Std. 1028-1997, IEEE Standard for Software Reviews, Copyright © 1997, by IEEE. Standard for Software Safety Plans, IEEE Std. 1228-1994 reprinted with permission from IEEE Std. 1228-1994 for Software Safety Plans, Copyright © 1994, by IEEE. The IEEE disclaims any responsibility or liability resulting from the placement and use in the described manner. ISACA is registered in the U.S. Patent and Trademark Office by Information Systems Audit and Control Association. ITIL is a Registered Trade Mark, and a Registered Community Trade Mark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office. IT Infrastructure Library is a Registered Trade Mark of the Office of Government Commerce. Microsoft, MS-WORD, and Windows are registered trademarks of Microsoft Corporation. Trusted Pipe is registered with the U.S. Patent and Trademark Office by Don O’Neill. The excerpts from: 1. “Comments on Software Quality” by W. S. Humphrey; 2. “The Team Software ProcessSM (TSPSM)” by W. Humphrey; 3. “Mapping TSPSM to CMMI®” by J. McHale and D. S. Wall; 4. “SCAMPISM Upgrade Team, Standard CMMI® Appraisal Method for Process Improvement (SCAMPISM) A, Version 1.2: Method Definition Document,” Handbook CMU/SEI-2006-HB-002; 5. “Applications of the Indicator Template for Measurement and Analysis,” Technical Note CMU/SEI- 2004-TN-024; 6. “CMMI® for Development (CMMI-DEV), Version 1.2,” Technical Report CMU/SEI-2006-TR- 008, Copyright 2006 Carnegie Mellon University; 7. “The Measurement and Analysis Process Area in CMMI®,” Copyright 2001 Carnegie Mellon University; 8. “Relationships Between CMMI® and Six Sigma,” Technical Note CMU/SEI-2005-TN-005, Copyright 2005 Carnegie Mellon University; 9. “Engineering Safety-related Requirements for Software-Intensive Systems,” Carnegie Mellon University; 10. “Safety-Critical Software: Status Report and Annotated Bibliography,” Technical Report CMU/SEI- 93-TR-5, Copyright 1993 Carnegie Mellon University; 11. “Software Inspections Tutorial” by D. O’Neill and A. L. Ingram as contained in the Software Engineer- ing Institute Technical Review 1988; from Carnegie Mellon University and Software Engineering Institute are furnished on an “as is” basis. Carnegie Mellon University makes no warranties of any kind, either expressed or implied, as to any matter including, but not limited to, warranty of fitness for purpose or merchantability, exclusivity, or results obtained from use of the material. Carnegie Mellon University does not make any war- ranty of any kind with respect to freedom from patent, trademark, or copyright infringement. The SEI and CMU do not directly or indirectly endorse the Handbook of Software Quality Assurance, Fourth Edition. Contents Preface xvii CHAPTER 1 Organizing for Quality Management 1 1.1 The Quality Management Framework 1 1.1.1 Object (Entity) 2 1.1.2 Product 3 1.1.3 Process 3 1.1.4 Requirement 3 1.1.5 User 4 1.1.6 Evaluation 5 1.1.7 Measure and Measurement 5 1.1.8 Quality 6 1.2 Quality Program Concepts 8 1.2.1 Elements of a Quality Program 8 1.2.2 Considerations 15 1.3 Organizational Aspects of the Quality Program 17 1.4 Quality Program Organizational Relationships 17 1.4.1 Establish Requirements and Control Changes 18 1.4.2 Establish and Implement Methods 20 1.4.3 Evaluate Process and Product Quality 21 1.5 Mapping Quality Program Functions to Project Organizational Entities 22 1.5.1 Planning 23 1.5.2 Establish Requirements and Control Changes 24 1.5.3 Establish and Implement Methods 25 1.5.4 Evaluate Process and Product Quality 27 1.6 Example Organizational Implementations of a Quality Program 27 1.6.1 Project Engineering Process Group 28 1.6.2 Quality Program Structures in Large Projects 28 1.6.3 Quality Program Structures for Small Projects in Large Organizations 31 1.6.4 Quality Program Structures in Small Organizations with Small Projects 31 1.7 Summary 33 References 33 vii viii Contents CHAPTER 2 Software Quality Lessons Learned from the Quality Experts 35 2.1 Introduction 35 2.2 Kaoru Ishikawa 37 2.3 Joseph M. Juran 39 2.4 Yoji Akao 43 2.5 W. Edwards Deming 44 2.6 Genichi Taguchi 49 2.7 Shigeo Shingo 51 2.8 Philip Crosby 52 2.9 Watts S. Humphrey 56 2.10 Conclusion 60 References 60 CHAPTER 3 Commercial and Governmental Standards for Use in Software Quality Assurance 63 3.1 SQA in ISO Standards 63 3.1.1 ISO 9000:2005 and ISO 9001:2000 64 3.1.2 ISO/IEC 90003 64 3.1.3 ISO/IEC 2500n—ISO/IEC 2504n (SQuaRE) 65 3.1.4 ISO/IEC 14598 and ISO/IEC 15504 66 3.1.5 ISO/IEC 9126 67 3.1.6 The Special Role of ISO/IEC 12207 68 3.2 SQA in IEEE Standards 69 3.2.1 IEEE Std 730-2002 69 3.2.2 IEEE Std 829-1998 70 3.2.3 IEEE Std 1028-1997 70 3.2.4 The Special Role of IEEE/EIA 12207 71 3.3 SQA in COBIT® 72 3.4 SQA in ITIL® 74 3.4.1 ISO/IEC 20000 76 3.5 SQA and Other Standards 77 3.5.1 ANSI/EIA-748-A-1998 77 3.5.2 RTCA/DO-178B 79 3.6 Whatever Happened to U.S. Department of Defense Standards? 80 3.6.1 Influential Past Standards 80 3.6.2 SQA in Active DoD Standards 82 3.7 Reminders About Conformance and Certification 83 3.7.1 Conformance 83 3.7.2 Conformance to an Inactive Standard 83 3.7.3 Certification 83 3.8 Future Trends 84 3.8.1 Demand for Personal Credentials Will Increase 84 3.8.2 Systems Engineering and Software Engineering Standards Will Converge 85 References 85 Contents ix CHAPTER 4 Personnel Requirements to Make Software Quality Assurance Work 89 4.1 Introduction 89 4.2 Facing the Challenge 90 4.3 Organization Structure 92 4.4 Identifying Software Quality Assurance Personnel Needs 94 4.5 Characteristics of a Good SQA Engineer 97 4.6 Training the Hardware QA Engineer 99 4.7 Training the Software Engineer 99 4.8 Rotating Software Engineers 101 4.9 New College Graduates 102 4.10 SQA Employment Requisitions 103 4.11 What to Expect from Your SQA Engineering Staff 104 4.12 Developing Career Paths 106 4.13 Recommendations 106 References 107 Selected Bibliography 107 Appendix 4A Typical Software Quality–Related Job Descriptions 107 Software Quality Assurance Manager 107 Engineer Software Quality Assurance 108 Software Reliability Engineer 108 Software Configuration Management Specialist 108 Software Safety Engineer 109 Software Librarian Aide 109 Senior Software Librarian 109 Software Quality Assurance Engineering Assistant 110 Software Quality Engineering Assistant 110 Software Quality Assurance Aide 110 CHAPTER 5 Training for Quality Management 111 5.1 Introduction 111 5.2 Context for a Quality Evaluation Training Program 111 5.2.1 Quality Evaluation to Quality Assurance 111 5.2.2 Audience for Quality Evaluation Training 112 5.2.3 Organizational Training Program 112 5.2.4 Needed Skills and Knowledge 113 5.3 Two Examples 116 5.3.1 Evaluation of Adherence to Process (PPQA) 116 5.3.2 Evaluation of Product Quality 118 5.4 Summary 119 Reference 119 CHAPTER 6 The Pareto Principle Applied to Software Quality Assurance 121 6.1 Introduction 121 6.2 WWMCCS—Classic Example 1 123 x Contents 6.2.1 Manpower 123 6.2.2 Cost of Contracts 123 6.2.3 By Release 125 6.2.4 By Function 125 6.3 Federal Reserve Bank—Classic Example 2 127 6.4 Defect Identification 132 6.4.1 Rubey’s Defect Data 133 6.4.2 TRW Defect Data 135 6.4.3 Xerox Defect Data 138 6.5 Inspection 140 6.6 Pareto Charts Comparison 143 6.7 Conclusions 145 References 146 CHAPTER 7 Inspection as an Up-Front