A Metrics-Based Software Maintenance Effort Model Jane Huffman Hayes Sandip C. Patel Liming Zhao Computer Science Department Computer Science Department Computer Science Department Lab for Advanced Networking University of Louisville University of Kentucky University of Kentucky [email protected] [email protected] [email protected] (corresponding author) Abstract planning a new software development project. Albrecht introduced the notion of function points (FP) to estimate We derive a model for estimating adaptive software effort [1]. Mukhopadhyay [19] proposes early software maintenance effort in person hours, the Adaptive cost estimation based on requirements alone. Software Maintenance Effort Model (AMEffMo). A number of Life Cycle Management (SLIM) [23] is based on the metrics such as lines of code changed and number of Norden/Rayleigh function and is suitable for large operators changed were found to be strongly correlated projects. Shepperd et al. [27] argued that algorithmic cost to maintenance effort. The regression models performed models such as COCOMO and those based on function well in predicting adaptive maintenance effort as well as points suggested an approach based on using analogous provide useful information for managers and maintainers. projects to estimate the effort for a new project. In addition to the traditional off-the-self models such as 1. Introduction COCOMO, machine-learning methods have surfaced recently. In [17], Mair et al. compared machine-learning Software maintenance typically accounts for at least 50 methods in building software effort prediction systems. percent of the total lifetime cost of a software system [16]. There has also been some work toward applying fuzzy Schach et al. found that 0 – 13.8% of changes made fall logic to building software metrics models for estimating under the category of adaptive maintenance [25]. It is software development effort [10]. Recent attempts have wise to design software that is maintainable since 18.2% also been made to evaluate the potential of genetic of project time is devoted to adaptive maintenance [25]. programming in software effort estimation [6]. Putnam et Software maintainability is defined as “the ease with al. [24] argue that the relationship between the metrics which a software system or component can be modified to size, effort, and time is nonlinear. Mendes et al. [18] correct faults, improve performance, or other attributes, or suggest that stepwise regression provides better adapt to a changed environment” [15]. Unfortunately, predictions than linear regression. developers and managers underestimate the time and effort required to change software. A lack of validated, widely accepted, and adopted tools for planning, 2.2 Maintenance Effort Estimation Models estimating, and performing maintenance also contributes to this problem. Basili et al. [2] attempted to develop a model to estimate This paper addresses the aforementioned problem of the cost of software releases that includes large planning and estimating when change is required. A enhancements. Gefen et al. [9] present a case study to metrics-based method is introduced that uses a model for estimate total lifecycle cost of a software system. The estimating adaptive maintenance effort. A study was estimation techniques for large information systems use performed to determine metrics that correlated closely lines of code or function points to calculate project effort with maintenance effort as measured in time (hours). in person-months [3, 4, 23]. These metrics were then used to build a model for The Albrecht Function Point model assumes that effort estimating adaptive maintenance effort. Two simple is primarily related to the size of a change [20]. In [20], regression models were built. The result, the Adaptive Niessink et al state that the size of the component to be Maintenance Effort Model (AMEffMo), appears changed (as opposed to the size of changes) is crucial. In promising from initial results. [21], they state that the size of the change and the size of the component to be changed are equally important. 2. Related Work Niessink et al. [20, 21] also emphasize the consistency of 2.1 Effort Estimation Models the process in maintenance effort data collection. Henry et al. found that the number of requirements changes that The Constructive Cost Model (COCOMO) [3,5,7,28] occur during maintenance can be used to improve effort supports the estimation of cost, effort, and schedule when estimates [14]. 3. Metrics Identification Table 2. Metric description. Metrics Description We hypothesize that the maintenance effort for a 1 Percent difference in total software application depends on measurable metrics that %Operators number of operators in the can be derived from the software development process. Changed application after maintenance We first identified the metrics that could affect the effort 2 LOC Difference Lines of code edited, added or required for maintaining an application to help managers (DLOC) deleted during maintenance 3 % Mod % Code modules changed during use the right metrics. Next, we worked on establishing change/add maintenance correlation between maintenance effort and the identified 4 metrics. Noprtr Total number of operators 5 Our first set of metrics was primarily taken from the CF Coupling factor results of two previous studies: [12] and [13]. The data 6 CR Comment ratio was taken from four sources: CS 499 and CS 616 courses 7 Hdiff Halstead’s difficulty (taught at the University of Kentucky), the research 8 LCOM Lack of cohesion in methods performed in [13] using Together® [29] and the industry 9 AC Attribute complexity research data from [12]. 10 CC Cyclomatic complexity Table 1 displays the results of correlation analysis: the 11 TCR True comment ratio higher the value of the coefficient, the stronger the 12 PM Perceived maintainability relationship between effort and the metric. The results 13 suggest that percentage of operators changed and the MP Maintainability product number of lines of codes changed edited, added or 14 Classes Changed Number of classes modified deleted (DLOC) were the most effective for predicting 15 Welker’s[30] Maintainability adaptive maintenance effort. Note that these will have to MI Index 16 be estimated. Table 2 describes the metrics in Table 1. HPVol Halstead program volume 17 Classes Added Number of classes added Table 1. Correlation between metrics and effort. 18 Heff Halstead’s effort Significance 19 LOC Total lines of code Coefficient of level Determination from ANOVA Metrics (R²) Regression 4. Development and Validation of AMEffMo 1 %Operators Changed 0.978 0.0002 We built an analytical model called Adaptive 2 LOC Delta -DLOC 0.779 0.00003 Maintenance Effort Model (AMEffMo) to predict 3 % Mod adaptive maintenance effort in terms of person-hours. We change/add 0.192 0.117 took the top two metrics from Table 1 to build AMEffMo. 4 Noprtr 0.152 0.444 “A typical estimation model is derived using regression 5 CF 0.108 0.525 analysis on data collected from past software projects” 6 CR 0.071 0.358 [22]. We used 70% of our data to build the model and the 7 Hdiff 0.046 0.683 remaining 30% to validate it. Table 3 lists the data 8 LCOM 0.034 0.725 availability and the average size of the applications 9 AC 0.033 0.518 constituting each dataset. 10 CC 0.032 0.581 11 TCR 0.011 0.741 4.1 Our Approach 12 PM 0.008 0.77 13 MP 0.006 0.79 In addition to a variation of the “leave one out” approach 14 Classes Changed 0.006 0.8 to evaluate the model, we performed standard checks such 15 MI 0.003 0.874 as examining residues, coefficient of determination, 16 HPVol 0.0008 0.929 significance level, performing a bias reduction technique 17 Classes Added 0.0006 0.946 [8], etc. 18 Heff 0.0004 0.969 19 LOC 0.00001 0.991 . Table 3. Number of data points per data source. where E is maintenance effort in person-hours. This Data Total Data points Data Avg size of model shows a significant linear relationship between Source data with DLOC points apps (lines DLOC and effort spent on changes (Figure 1.a and 1.b) in points information with of code), which the R square value and significance are 0.78 and DNoprtr min, max 0.000029 respectively. info Then, by using the course data to build the model and 1 CS 499 9 81 02 2688.2, 101, 6201 the rest of the data to validate, we obtained the following 2 CS 616 10 10 83 1934, model: 1192, 3425 E = 78 + .01DLOC (2) 3 Previous In- 8 8 02 58.6, 35, house 79 Research 1500 Data [12] Effort 4 Industry 6 6 6 2725.4, 238, 1000 Changing Research 18952 Data [11] 500 Predicted Effort 4.2 Multiple Regression Changing 0 First, we treated the percentage of operators changed and 0 50 100 150 the change in the total lines of code (DLOC) as multiple -500 variables for the data from CS 616. The multiple Predicting by DLOC(E=-40+6.56DLOC) regression output showed that none of the independent variables related to effort. But the simple regression (a) performed showed that the independent variable was 300 strongly related to effort. Next, we performed simple 200 regression using the LOC difference (DLOC). 100 4.3 Simple Regression 0 0 50 100 150 First, we used the data from the sources numbered 3 -100 and 4 in Table 3. Table 4 indicates that our models were -200 built based on small- to medium-sized maintenance tasks. -300 Table 4. Descriptive statistics data of DLOC and -400 Effort Spent on Changes (in person hours). Residues VS DLOC (of E=-40+6.56DLOC) DLOC Effort Spent on Changes (b) Mean 502.19 Mean 112.75 0 Standard 194.87 Standard 34.34 -1000 0 2000 4000 6000 8000 Deviation Deviation -2000 Minimum 0 Minimum 10 -3000 Maximum 6026 Maximum 1121 -4000 Sum 16070 Sum 3608 -5000 Using the least square method, we produced the -6000 following model: E = -40 + 6.56 DLOC (1) -7000 Residues vs DLOC- validating using course data 1 Did not have enough information from one student team.