5-(948-~gblb SOWARE ENGINEERING LABORATORY SERIES SEL-95-004 PROCEEDINGS OF THE TWENTIETH ANNUAL SOFTWARE ENGINEERING WORKSHOP DECEMBER 1995 National Aeronautics and Space Administration Goddard Space Flight Center Greenbelt, Maryland 20771 Proceedings of the Twentieth Annual Software Engineering Workshop November 29-30,1995 GODDARD SPACE FLIGHT CENTER Greenbelt, Maryland The Software Engineering Laboratory (SEL) is an organization sponsored by the National Aeronautics and Space AdministratiodGoddard Space Flight Center (NASAIGSFC) and created to investigate the effectiveness of software engineering technologies when applied to the development of applications software. The SEL was created in 1976 and has three primary organizational members: NASAIGSFC, Software Engineering Branch The University of Maryland, Department of Computer Science .I Computer Sciences Corporation, Software Engineering Operation The goals of the SEL are (1) to understand the software development process in the GSFC environment; (2) to measure the effects of various methodologies, tools, and models on this process; and (3) to identifjr and then to apply successful development practices. The activities, findings, and recommendations of the SEL are recorded in the Software Engineering Laboratory Series, a continuing series of reports that includes this document. Documents from the Software Engineering Laboratory Series can be obtained via the SEL homepage at: or by writing to: Software Engineering Branch Code 552 Goddard Space Flight Center Greenbelt, Maryland 2077 1 SEW Proceedings iii The views and findings expressed herein are those of, the authors and presenters and do not necessarily represent the views, estimates, or policies of the SEL. All material herein is reprinted as submitted by authors and presenters, who are solely responsible for compliance with any relevant copyright, patent, or other proprietary restrictions. SEW Proceedings CONTENTS Materials for each session include the viewgraphs presented at the workshop and a supporting paper submitted for inclusion in these Proceedings. Page 1 Session 1: The Software Engineering Laboratory 1- 3 What's Happening in the SoJtware Engineering Laboratory? R. Pajerski, S. Green, and D. Smith, NASNGoddard - 221 The Empirical Investigation of Perspective-Based Reading V.Basili, University of Maryland, S. Green, NASNGoddard, 0. Laitenberger, University of Kaiserslautern, F. Shull, University of Maryland, S. Sorurngaard, University of Trondheim, and M. Zelkowitz, University of Maryland x4 7 1 Porting Experience Factory Concepts to ~ewEnvironments 4 F. McGarry, Computer Sciences Corporation n..-.93 Session 2: Reliability - Discussant: J. Liu, Computer Sciences Corporation ;A 95 Empirical Study of SofTware Testing and Reliability in an Industrial Setting J. Slonim, J. Ye, and M. Bauer, IBM Canada ' 129 Software-Reliability-Engineered Testing r'. J. Musa, AT&T Bell Laboratories 6 145 Reusing Software Reliability Engineering AnalysisJFomLegacy to Emerging Client/Sewer Systems J. Cusick, AT&T Bell Laboratories $+ 161 Special Presentation: Software Engineering Survey - Presenter: J. Valett, NASAIGoddard c'p , 167 Session 3: Product Evaluation - Discussant: J. Valett, NASNGoddard ' i 169 A Family of User Interface Consistency Checking Tools: Design and Development b -- of SHERLOCK B. Shneiderman and R. Mahajan, University of Maryland - 189 A COTS Selection Method and Experiences of Its Use J. Kontio, R. Tesoriero, G. Caldiera, University of Maryland, S-F. Chen, K. Limperos, and M. Deutsch, Hughes Information Technology Corporation p" 0 - 21 5 Process Enactment within an Environment M. Zelkowitz and R. Tesoriero, University of Maryland SEW Proceedings v CONTENTS (casaslt9d) Page Session 4: Models - Discussant: D. Smith, NAShYGoddard Reliability and Risk Analysis of the NASA Space Shuttle Flight Somare N. Schneidewind, Naval Postgraduate School Modeling and Simulation of Somare Projects A. Drappa, M. Deininger, J. Ludewig, and R. Melchisedech, University of Stuttgart Evaluating Empirical Models for the Detection of High-Risk Components: Some Lessons Learned F. Lanubile and 6. Visaggio, University of Bari Session 5: Method &aluation - Discussant: S. Green, WASNGoddard Object-Oriented SoJtware Metrics for Predicting Reusability and Estimating Size D. Sanderson, Southwest Missouri State University, T-L. Tran, J. Sherif, and S. Lee, Jet Propulsion Laboratory Improving the Software Testing Process in NASA S SoJtware Engineering Laboratory S. Waligora and R. Coon, Computer Sciences Corporation How Do Formal Methods Aflect Code Quality? S. Lawrence Pfleeger, SystemsISoftware, Inc., and L. Hatton, Programming Research ktd. Panel Discussion: Has the Investment in Process Demonstrated an Impact on Software? -Moderator: V. Basili, University of Maryland T. DeMarco, The Atlantic Systems Guild, Inc. J. Herbsleb, Software Engineering Institute D. Rombach, University of Kaiserslautern T. Wasserman, IDE, Inc. Appendix A-Worbhop Arttendees Appendix B-Standard Bibliography of $EL Literature SEW Proceedings -2 - i I. ,y'/- c;, < !, Session 1: The Software Engineering Laboratory What's Happening in the Software Engineering Laboratory? Rose Pajerski, NASAGoddard The Empirical Investigation of Perspective-Based Reading Vic Basili, University of Maryland Porting Experience Factory Concepts to New Environments Frank McGarry, Computer Sciences Corporation SEW Proceedings SEW Proceedings What's Happening in the Software Engineering Laboratory? JrfJ,." . % '?' Rose Pajerski, Scott Green, and Donald Smith /&,P Code 552, Software Engineering Branch NASA/Goddard Space Flight Center ti6\ 5 cr '@; '\ {<< :: Greenbelt, Maryland 2077 1 Background Since 1976, the Software Engineering Laboratory (SFL) has been dedicated to understanding and improving the way in which one NASA organization, the Flight Dynamics Division PDD) at Goddard Space Flight Center, develops, maintains, and manages complex flight dynamics systems. The SFL consists of three member organizations: NASNGoddard, the University of Maryland, and Computer Sciences Corporation. Throughout the SEL's history, its overall goal has remained the same: to improve the Division's software products and processes in a quantifiable manner. Achieving this goal requires that each development and maintenance effort be viewed, in part, as a SEL experiment that examines a specific technology or builds a model of interest for use on subsequent efforts. The SEL has undertaken many technology studies while developing operational support systems for numerous NASA spacecraft missions. Data from over 120 software development projects in the organization have been collected and archived. From these data, the SEL has derived models of the development process and product and has conducted studies on the impact of new technologies. This paper presents an overview of recent activities and studies in the SEL, using as a framework the SEL's organizational goals and experience-based software improvement approach. It focuses on two SEL experience areas: the evolution of the measurement program and an analysis of three generations of Cleanroom experiments. Software Improvement Approach The SEL's basic approach toward software process improvement is to first understand and characterize the process and product as they *existto establish a local baseline (Figure 1). Only then can new technologies be introduced and assessed (phase two) with regard to both process changes and product impacts. Typically, several studies and assessments are in progress at any one time, each with a duration of 1 to 3 years. The third phase of the SEL approach (pac-ng) synthesizes the results of the first two phases and feeds them back into the cycle for use by software engineers on subsequent projects. Experience packages SEW Proceedings include prmess-~lohingguidelines, eiaiing courses, toolis, and Mdebmks. The S1Elt's process improvement approach has proven very effective in the Hi&t Qnmics DiGsion. The Division's sohaeproduct has shorn shsmtial improvements in error rates, cost, and development time. In 1994, the §EL rmivd the Computer Society Awmd for Sohwe Process Achievemet and a Fedad Techolm Ladership Awzard for its application of these concepts in beFDD environment. rt of Your Business Detemajne Effective Inqprovemnt. Wit Joint Team Approaches Help? e WiM FomlMethods Improve Reliability? * What Reuse Approach Will Best Cut Cost? Know Your Software Business s What Are PAy Software Characteristics? 0 What Processes Do I Use? What Measures Are keQd? Fi~rcz1. SEL Sgsfware Improvement Approach SEL Goals From its inception, the SEL has focused on both increasing reliability and reduGing life cycle costs. Over ths: past 8 yeas, he SEL has acGeved measured gains in both areas: reliability of delivered systems has increased threefold, and current mission support costs axe half that of older systems. However, ~thincreasing pressure to reduce "time to deploy," SEX, gods now emphasize redudng devdopment time as well as cost. Enabling process techn01ogies have been selected, and analyses are underway to measure and assess their impact on both cost and schedde. In addition, a viable process improvement infrasmcme must be miainlained to realize these organiza~ondgoals. The four improvement gds(based on the 1994 SEL baseline) and the study areaslprocess technologies being investigated with each are as follows: e By 1998, ddiver systems 30% faster4ommercid-off-be-shdf (COTS) and reuse processes, testing approaches s By 1998, redwe devdopment cost