CHAPTER 4 Fundamental Issues in Software Measurement James M. Bieman, Norman Fenton, David A. Gustafson, Austin Melton, and Linda M. Ott 4.1 Intro duction This chapter builds on the previous one. In the previous chapter the emphasis is on measurement theory, and software measurementis discussed as a typ e of measurement. In this chapter the emphasis is on software measurement, and ideas from measurement theory are used to guide and direct our development of software measurement ideas and metho ds. 4.1.1 Terminology Interestingly, the term metric has more than one meaning in soft- ware measurement. It is used in at least the three ways depicted b elow: 1. Anumb er derived from a pro duct, pro cess or resource. For ex- ample, one hears ab out the metric number of function points or the metric lines of code LOC per programmer month . 2. A scale of measurement. For example, one hears ab out a pro- p osed nominal scale, i.e., classi cation, of software failures as a metric. 3. An identi able attribute. For example, one hears ab out the met- ric portability of programs or the metric coupling in designs, even though no numb er or function is necessarily attached. This confusion is unnecessary since all of the ab ove ideas may b e accommo dated within the framework of scienti c measurement, using the well de ned terms and notions which have long b een 2 FUNDAMENTAL ISSUES IN SOFTWARE MEASUREMENT established there. For example in 1 ab ove, metric is really referring to direct or indirect measurements . In order to be precise, it is imp ortant that the terms be pre- cisely de ned. Following classical texts in the science of measure- ment Krantz et al.[71], Kyburg[84], Rob erts[79], the following de nitions for software measurement are used in this chapter. De nition 4.1.1 An attribute is a feature or prop ertyofanentity. De nition 4.1.2 Measurement is the pro cess of empirically and ob jectively assigning numb ers or symb ols to the attributes of entities including ob jects and events in the real world in sucha way as to describ e them. De nition 4.1.3 A measure is an empirical ob jective assignment of numb ers or symb ols to entities to characterize a sp eci c at- tribute. De nition 4.1.4 Direct measurement of an attribute is measure- ment which do es not dep end on the measurement of any other attribute. De nition 4.1.5 Indirect measurement of an attribute is mea- surement which involves the measurement of one or more other attributes. The following de nitions come from Fenton[91]. De nition 4.1.6 A model is 1 an abstraction of a class of ob jects that retains certain attributes of those ob jects and ignores other attributes or 2 a mathematical formula that relates two or more measures. For example, control owgraphs are an abstraction of programs. Control owgraphs retain the control attributes but throw away the data ow attributes. De nition 4.1.7 A product is an artifact, i.e., a deliverable or a do cument, that is pro duced during software development. De nition 4.1.8 A process is a software related activity. De nition 4.1.9 A resource is an input to a pro cess. De nition 4.1.10 An internal attribute of an entity is an at- tribute that can b e measured purely in terms of the entity itself. De nition 4.1.11 An external attribute of an entity is an at- tribute that can only b e measured with resp ect to how the entity relates to its environment. INTRODUCTION 3 4.1.2 The Need for a Foundation This chapter is motivated by critically lo oking at the current state of software measurement. Software metrics and their measurements should b e used to determine the state of software and to regulate and guide the development of software. However, most who design or use software metrics readily admit that our current metrics are not, in general, reliable, where reliability is assumed to mean accu- rately re ecting the real world. What is amazing is that given this acknowledged state of a airs, very few researchers are suggesting substantial changes in what we do; most suggestions for improve- ments represent only minor, cosmetic adjustments. But software measurement will not reach an acceptable state of reliability in the ab ove sense and acceptance via minor adjustments. Wehave to o far to go; wemust make substantial changes. In this chapter fundamental questions ab out what we are doing, howwe are doing it, and what we really need to b e doing are asked and answered. In spite of our past failures we can build a foundation up on which reliable and useful metrics can b e de ned. It is imp ortant to note the di erence b etween the everyday use of reliable and valid and the use of these terms in disciplines based on measurement theory, and it is imp ortant to understand the di er- ence b etween these two terms Stanley-Hopkings[72]. A measure is reliable if the metho d used to obtain the measurement provides con- sistent results. Thus, two di erent p eople using the same metho d to obtain the measurements will get the same value, or one p erson using the same metho d at di erent times will get the same value. A measure is valid if it accurately re ects the real world. Thus, for example, if measurements on program A and program B indicate that there is a higher degree of coupling in the co de for program A than in the co de for program B, then wewant it to b e the case that the coupling in the co de for program A really is at a higher degree than the coupling in the co de of program B. This is the rep- resentation condition of measurement as presented in the previous chapter. Although one do es not commonly encounter discussions of the formal notion of the reliability of a software measure, this concept was intro duced into the software metrics literature at least as early as 1979 Chapin[79]. 4 FUNDAMENTAL ISSUES IN SOFTWARE MEASUREMENT 4.1.3 The Bene ts of a Foundation Building a foundation for software measurement is imp ortant not only for software measurement but also for software engineering b e- cause with a formal foundation for software measurementwe could dramatically increase our understanding of software and software engineering. Software measurement theory may b e at the heart of software engineering. Such a claim may seem strange when rst heard { esp ecially given the current state of software measure- ment. However, when we realize that to truly measure something, one must understand what is b eing measured, then the claim is very reasonable. One very serious misunderstanding which we in software measurementhave for sometime had is the b elief that by assigning numb ers to software do cuments, we are measuring the do cuments, and thus, we must be understanding the do cuments and their attributes. To assign numb ers to a collection of entities do es not pro duce or increase understanding. In fact, just the opp o- site is often the case. Assigning numb ers without understanding, only leads to confusion or even worse { to an unfounded b elief that one has understanding. This de ning of metrics without under- standing what is b eing de ned, has led us to a situation analogous to the emp eror's new clothes. We have convinced ourselves that wehave develop ed profound and wonderful metrics; however, the simple observer sees that wehave only exp osed our lack of under- standing. For another argument showing that real measurement will help us b etter understand software engineering, consider the following. Wewant to measure many external factors of our software do cu- ments, e.g., reusability and maintainability, and we are instructed in our software engineering courses and by many leading software exp erts that byhaving certain internal structure suchaswell con- structed mo dules and low coupling we can improve imp ortant ex- ternal characteristics. Thus, we should b e trying to indirectly mea- sure or predict external characteristics based on direct measure- ments of internal ones. However, we do not understand how the internal characteristics in uence the external ones, and thus, we are unable to determine the needed connections to develop indi- rect measurement and prediction to ols. VALIDATING SOFTWARE METRICS 5 4.2 Validating Software Metrics Anumb er of reasons have b een cited for the p o or industrial accep- tance of software measurement. A common one is that managers simply do not know what to do with the measurements. Probably a more fundamental reason, and one which is also resp onsible for the con icting claims of researchers in this area, is the lack of seri- ous validation of prop osed metrics; and thus, a lack of con dence in the measurements. Validation when applied to software metrics has often b een mis- used. Managers and researchers often confuse worth of or ease of use with validity. While not b elittling these characteristics of met- rics, there are di erences b etween these characteristics and validity. Many p eople would consider that barometric measures of atmo- spheric pressure are not easy to obtain but that do es not mean that they are not valid. Let us consider demonstrating the validity of metrics in the formal sense; this is a necessary requirementof all measurement activity.