Testing and Quality Assurance for Component-Based Software for a Listing of Recent Titles in the Artech House Computing Library, Turn to the Back of This Book
Total Page:16
File Type:pdf, Size:1020Kb
Testing and Quality Assurance for Component-Based Software For a listing of recent titles in the Artech House Computing Library, turn to the back of this book. Testing and Quality Assurance for Component-Based Software Jerry Zeyu Gao H.-S. Jacob Tsao Ye Wu Artech House Boston • London www.artechhouse.com Library of Congress Cataloging-in-Publication Data Gao, Jerry. Testing and quality assurance for component-based software / Jerry Zeyu Gao, H. -S. Jacob Tsao, Ye Wu. p. cm. — (Artech House computing library) Includes bibliographical references and index. ISBN 1-58053-480-5 (alk. paper) 1. Computer software—Quality control. 2. Computer software—Testing. I. Tsao, H.-S. J. II. Wu, Ye. III. Title. IV. Artech House computer library. QA76.76.Q35G38 2033 005.1’4—dc22 2003057725 British Library Cataloguing in Publication Data Gao, Jerry. Testing and quality assurance for component-based software. — (Artech House computing library) 1. Component software—Testing 2. Computer software industry—Quality control I. Title II. Tsao, H.-S. J. (H.-S. Jacob) III. Wu, Ye 005.3’0287 ISBN 1-58053-480-5 Cover design by Igor Veldman © 2003 ARTECH HOUSE, INC. 685 Canton Street Norwood, MA 02062 The following are registered in the U.S. Patent and Trademark Office by Carnegie Mellon University: Capability Maturity Model and CMM. Testing Maturity ModelSM and TMMSM are service marks of the Illinois Institute of Technology. All rights reserved. Printed and bound in the United States of America. No part of this book may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, or by any infor- mation storage and retrieval system, without permission in writing from the publisher. All terms mentioned in this book that are known to be trademarks or service marks have been appropriately capi- talized. Artech House cannot attest to the accuracy of this information. Use of a term in this book should not be regarded as affecting the validity of any trademark or service mark. International Standard Book Number: 1-58053-480-5 Library of Congress Catalog Card Number: 2003057725 10987654321 To my wife Tracey and my lovely son Kevin To my parents Ming Gao and YeFang Qin —Jerry Zeyu Gao To my mother Shu-Wen Pao, my brother Hsiao-Tzu Tsao, my wife Hueylian, and my children Allison and Jason —H.-S. Jacob Tsao To my wife Wanjie and to my lovely daughter Ya Ya —Ye Wu . Contents Preface ..............................xvii What is this book about? xvii Why has reuse of third-party software components become popular recently? xviii Why is testing software components and component-based software important? xviii Book organization xix Is this book for you? xx Acknowledgments .......................xxi I Introduction ............................1 1 Introduction to software components ..............3 1.1 The evolution of software components 3 1.2 Why is software component reuse important? 5 1.3 What is a software component? 6 1.4 Properties of software components in CBSE 8 1.5 Basic elements of software components 11 1.6 Software modules versus software components in CBSE 13 1.7 An engineering process for software components 17 vii viii Contents 1.8 Questions, concerns, and needs in component validation and quality control 19 1.9 Summary 21 References 22 2 Software component testing ..................25 2.1 Component testing background 26 2.2 Component testing in CBSE 28 2.2.1 Vendor-oriented component testing 29 2.2.2 User-oriented component testing 30 2.3 Issues and challenges of component testing in CBSE 34 2.3.1 Issues and challenges in user-oriented component validation 35 2.3.2 Issues and challenges in vendor-oriented component testing 38 2.4 Component testing myths and other needs 41 2.5 Summary 44 References 44 3 Introduction to component-based software ..........47 3.1 Introduction 47 3.2 Component-based software versus traditional programs 49 3.2.1 Properties of component-based software 49 3.2.2 Component-based software versus traditional programs 53 3.3 Component-based software infrastructure: component model 55 3.4 Engineering process for component-based software 58 3.4.1 Process models for traditional software 58 3.4.2 A process model for component-based software 60 References 65 4 Testing component-based software ..............67 4.1 Introduction 68 4.2 Issues and challenges of testing and maintaining component-based software 69 4.2.1 Why is adequate testing for component-based software necessary? 69 Contents ix 4.2.2 Difficulties in adequate testing and maintenance for component-based software 71 4.3 Testing model for component-based software 75 4.3.1 Interaction model 75 4.3.2 Behavior model 77 4.4 Testing and maintenance methodologies 79 4.5 Enterprise-based test process for component-based software 80 4.6 Summary 81 References 82 5 Testability of software components ..............85 5.1 Basic concepts of software testability 86 5.2 Understanding component testability 90 5.2.1 Component understandability 91 5.2.2 Component observability 91 5.2.3 Component traceability 92 5.2.4 Component controllability 95 5.2.5 Test support capability 96 5.3 Design for testability of software components 98 5.3.1 Understanding of testable software components 98 5.3.2 Methods to increase component testability 100 5.3.3 Current research efforts on building testable components 102 5.4 Verification and evaluation of component testability 105 5.4.1 Static verification approach 107 5.4.2 Statistic verification approach 107 5.5 Software testability measurement 109 5.5.1 What is software testability measurement? 109 5.5.2 How to measure software testability 110 5.6 Summary 113 References 114 II Validation methods for software components ........117 6 Black-box testing methods for software components ....119 6.1 Introduction 119 6.2 Black-box testing foundations 122 x Contents 6.2.1 Component specification 122 6.2.2 Component verification history 123 6.2.3 Component customization 125 6.3 Black box–based testing techniques 126 6.3.1 Random testing 126 6.3.2 Partition testing 127 6.3.3 Boundary value testing 130 6.3.4 Decision tables–based testing 133 6.3.5 Mutation testing 134 6.4 Summary 137 References 138 7 White-box testing methods for software components ...141 7.1 Flow graph notation 142 7.2 Path testing 142 7.2.1 Statement coverage 143 7.2.2 Branch coverage 144 7.2.3 Multiple-condition coverage 144 7.2.4 Path coverage 145 7.2.5 Loop coverage 146 7.3 Data-flow testing 147 7.4 Object-oriented testing 149 7.4.1 Inheritance 149 7.4.2 Polymorphism 150 7.4.3 GUI 151 7.4.4 Binding coverage 151 7.4.5 State-based testing 152 7.5 Issues in testing software components 153 7.6 Conclusion 154 References 154 8 Test automation and tools for software components ....157 8.1 Software test automation 158 8.1.1 Basics of software test automation 160 8.1.2 Software test automation process 164 8.1.3 Different types of test automation tools 166 8.2 Component-oriented software test tools 170 Contents xi 8.2.1 ADLscope: a specification-based test tool for components 170 8.2.2 The Java Compatibility Test Tools and technology compatibility kit 171 8.3 Component test automation in CBSE 174 8.3.1 Systematic management of component testing information 174 8.3.2 Automatic test execution for software components 176 8.3.3 Automatic test generation for co1mponents 179 8.3.4 Systematic performance testing for components 181 8.3.5 Systematic regression testing for software components 183 8.4 Summary 185 References 186 III Validation methods for component-based software ....189 9 Integration testing for component-based software .....191 9.1 Introduction 191 9.1.1 Type I: Intercomponent faults 192 9.1.2 Type II: Interoperability faults 192 9.1.3 Type III: Traditional faults and other faults 193 9.2 Traditional integration-testing methodologies 193 9.2.1 Function decomposition–based integration testing 193 9.2.2 Call graph–based integration testing 195 9.3 A test model for integration testing of component-based software 196 9.3.1 Test elements 197 9.3.2 Component interaction graph 198 9.3.3 Test-adequacy criteria 199 9.4 Black box–based integration-testing approaches 200 9.4.1 Interface nodes 201 9.4.2 Event nodes 201 9.4.3 Context-dependence and content-dependence relationships 202 9.5 UML-based integration-testing approaches 202 9.5.1 Context-dependence relationships 203 9.5.2 Content-dependence relationships 207 9.6 Summary 208 References 209 xii Contents 10 Regression testing for component-based software .....211 10.1 Introduction 212 10.2 Regression testing for corrective-maintenance activities 215 10.2.1 Representing modifications with UML diagrams 216 10.2.2 Regression testing for corrective maintenance 218 10.3 Regression testing for perfective and adaptive maintenance 221 10.3.1 Control similarity evaluation 221 10.3.2 Data dependence similarity evaluation 224 10.4 Summary 225 References 225 11 Performance testing and measurement ...........229 11.1 Basics of software performance testing and measurement 230 11.1.1 Major focuses in software performance testing and evaluation 231 11.1.2 Performance test process 234 11.1.3 Performance testing and evaluation challenges and needs in CBSE 236 11.2 Performance evaluation metrics 238 11.2.1 Utilization metrics 238 11.2.2 Speed-related metrics 239 11.2.3 Availability metrics 240 11.2.4 Reliability metrics 243 11.2.5 Throughput metrics 245 11.2.6 Scalability metrics 246 11.3 Performance evaluation approaches 248 11.3.1 Classification of performance evalution approaches 248 11.3.2 Component-based performance evaluation