Estimating Maintenance Costs for Royal Canadian Navy Ships a Parametric Cost Model

Home , HMCS Algonquin (DDG 283), HMCS Algonquin (R17), HMCS Calgary (FFH 335), HMCS Ottawa (FFH 341), HMCS Regina (FFH 334), HMCS Saskatoon (MM 709)

Estimating Maintenance Costs for Royal Canadian Navy Ships A Parametric Cost Model

Zakia Bouayed Christopher E. Penney Abderrahmane Sokri Tania Yazbeck DRDC – Centre for Operational Research and Analysis

Defence Research and Development Canada Scientific Report DRDC-RDDC-2017-R147 October 2017

IMPORTANT INFORMATIVE STATEMENTS

Disclaimer: Her Majesty the Queen in Right of Canada (Department of National Defence) makes no representations or warranties, express or implied, of any kind whatsoever, and assumes no liability for the accuracy, reliability, completeness, currency or usefulness of any information, product, process or material included in this document. Nothing in this document should be interpreted as an endorsement for the specific use of any tool, technique or process examined in it. Any reliance on, or use of, any information, product, process or material included in this document is at the sole risk of the person so using it or relying on it. Canada does not assume any liability in respect of any damages or losses arising out of or in connection with the use of, or reliance on, any information, product, process or material included in this document.

This document was reviewed for Controlled Goods by Defence Research and Development Canada (DRDC) using the Schedule to the Defence Production Act.

Endorsement statement: This publication has been peer-reviewed and published by the Editorial Office of Defence Research and Development Canada, an agency of the Department of National Defence of Canada. Inquiries can be sent to: [email protected].

Template in use: (2010) SR Advanced Template_EN (051115).dotm © Her Majesty the Queen in Right of Canada (Department of National Defence), 2017 © Sa Majesté la Reine en droit du Canada (Ministère de la Défence nationale), 2017

Abstract

This paper proposes a parametric costing model for Defence planners to conduct a first order estimate of the maintenance component of O&S costs for ships being considered for procurement. The model is built based on a parametric approach which incorporates relevant cost drivers as input parameters. The coefficients of these cost factors are estimated by fitting the model with a historical dataset of Royal Canadian Navy ships through regression analysis. Once the coefficient values are obtained, the cost model is used to produce out-of-sample fitted values to estimate maintenance costs for both currently active vessels and those being considered for acquisition.

The regression analysis returns highly significant explanatory variables, with an R-squared statistic of 0.875. The single most important factor in explaining yearly maintenance costs is the ship’s class. Additionally, the effect of an additional sea day per year is to increase yearly maintenance costs by 0.38%.

An Excel-based costing tool is also provided in this paper for potential use by personnel working within the Centre for Costing in Defence to produce rough order of magnitude maintenance cost estimates.

Significance to Defence and Security

This Scientific Report presents a regression-based parametric costing model to forecast the maintenance component of O&S costs for defence projects being considered for procurement. The outcome of this report supports the fourth Defence Priority – Ensuring Defence Resource Stewardship & Affordability. Estimating O&S costs while still in the early stages of a program is important for long-term budgeting and decision making and ensures that systems can be sustained over full program lifetimes, in addition to contributing to sound financial management of the Defence budget and stewardship of public resources.

DRDC-RDDC-2017-R147 i

Résumé

Le présent document propose un modèle paramétrique d’estimation du coût à l’intention des planificateurs de la Défense afin d’obtenir une évaluation de premier ordre des coûts d’O & M dans le cadre du volet sur l’entretien des navires que l’on songe à acquérir. Le modèle a été élaboré selon une méthode paramétrique qui utilise des facteurs de coût pertinents comme paramètres d’entrée. On détermine les coefficients de ces facteurs de coût en ajustant le modèle en fonction d’un ensemble de données historiques sur les navires de la Marine royale canadienne au moyen d’une analyse de régression. Lorsqu’on connaît la valeur du coefficient, on utilise le modèle de coûts pour obtenir des valeurs ajustées hors échantillon afin de calculer les dépenses d’entretien, tant pour les navires en service que pour ceux que l’on songe à acquérir.

L’analyse de régression donne des variables explicatives très importantes, notamment la valeur statistique R au carré de 0,875. Le plus important facteur permettant d’expliquer les frais annuels d’entretien est la classe du navire. En outre, chaque jour de mer supplémentaire par année fera augmenter les frais annuels d’entretien de 0,38%.

Un outil d’estimation du coût en format Excel accompagne également le présent document afin que le personnel qui travaille au Centre d’établissement des coûts de la Défense puisse l’utiliser pour obtenir le montant approximatif des dépenses d’entretien.

Importance pour la défense et la sécurité

Le présent rapport scientifique propose un modèle paramétrique d’estimation du coût fondé sur une régression pour prévoir les dépenses d’O & M dans le cadre du volet sur l’entretien pour les projets de défense que l’on songe à acquérir. Le rapport vient appuyer la quatrième priorité de la Défense, soit assurer l’intendance des ressources et la viabilité financière de la Défense. L’estimation des dépenses d’O & M au tout début d’un programme est importante afin d’établir un budget et de prendre des décisions à long terme pour garantir que les systèmes pourront être maintenus tout au long de la durée de vie du programme et contribuer à une saine gestion financière du budget de la Défense ainsi qu’à l’intendance des ressources publiques.

ii DRDC-RDDC-2017-R147

Table of Contents

Abstract ...... i Significance to Defence and Security ...... i Résumé ...... ii Importance pour la défense et la sécurité ...... ii Table of Contents ...... iii List of Figures ...... v List of Tables ...... v Acknowledgements ...... vi 1 Introduction ...... 1 1.1 Background and Context ...... 1 1.2 Literature Review ...... 2 1.3 Aim ...... 3 1.4 Scope ...... 3 1.5 Report Structure ...... 3 2 Model ...... 4 2.1 Cost Elements ...... 4 2.1.1 Operating Costs ...... 4 2.1.2 Maintenance Costs ...... 4 2.1.3 Sustaining Support Costs ...... 4 2.2 Parametric Approach ...... 5 2.3 Random Effects Model ...... 6 2.4 Dependent Variable ...... 6 2.5 Identifying Cost Drivers ...... 6 2.6 Developing Cost Estimating Relationships ...... 7 2.7 Model Limitations ...... 8 3 Regression Results ...... 9 3.1 Dataset ...... 9 3.2 Parameter Estimates ...... 9 3.3 Model Diagnostics ...... 10 3.4 Discussion ...... 11 4 Navy Maintenance Costing Tool ...... 12 4.1 Obtaining an Estimate ...... 12 4.2 Example ...... 13 5 Conclusion ...... 14 5.1 Recommendations ...... 14 6 Attachment ...... 16 References ...... 17

DRDC-RDDC-2017-R147 iii

Annex A Data ...... 18 Annex B Interpreting the Regression Results ...... 22 Annex C Potential Applications ...... 23 C.1 Aircraft and Land Vehicles ...... 23 C.2 Facilities ...... 23 C.3 Classification ...... 23 List of Symbols/Abbreviations/Acronyms/Initialisms ...... 25

iv DRDC-RDDC-2017-R147

List of Figures

Figure 1: Profile of annual program expenditures by major cost category across life cycles...... 1 Figure 2: RCN Ship Maintenance Costing Tool, Halifax Class Ship with 100 Sea Days. 13 Figure 3: RCN Ship Maintenance Costing Tool, Kingston Class Ship with 220 Sea Days...... 13 Figure C.1: Classification of defence equipment...... 24

List of Tables

Table 1: Description of O&S cost categories...... 5 Table 2: Regression results...... 9 Table 3: Selected regression diagnostics...... 10 Table A.1: RCN Dataset, 2011–2015 Fiscal Years...... 18

DRDC-RDDC-2017-R147 v

Acknowledgements

The authors would like to acknowledge the contribution of LCdr Guy Cadrin for making this study possible. His assistance in obtaining data and providing an understanding of Navy maintenance procedures contributed significantly to the successful execution of this study.

vi DRDC-RDDC-2017-R147

1 Introduction

1.1 Background and Context

The Centre for Costing in Defence (CCD) makes use of the concept of the Life Cycle Cost (LCC) to bring total cost visibility to a purchasing program. LCC can be thought of as a “cradle to grave” approach to managing a program throughout its useful life. This entails identifying all cost elements that pertain to the program from the initial concept all the way through to operations, support and disposal. LCC consists of four cost categories [1]:  Research & Development: all activities necessary to achieve expenditure approval.  Investment: all activities necessary to bring the system into operational service.  Operations and Sustainment: all activities necessary to operate, maintain, supply, and otherwise support a deployed system throughout its life cycle.  Disposal: the planning and managing of the demilitarization and removal of the system from service.

In defence acquisition projects, Operations and Sustainment (O&S) costs constitute the largest portion of the overall LCC, as shown in Figure 1 [2]. To ensure the sound financial management of the Defence budget and stewardship of public resources, Director Cost Analytics has requested that the Defence Economics Team (DET) provide a method for conducting a first approximation of O&S costs for projects being considered for procurement. To respond to this request, the DET proposes a parametric regression model approach that estimates Cost Estimating Relationships (CERs) between selected explanatory factors and the maintenance component of O&S costs.

Figure 1: Profile of annual program expenditures by major cost category across life cycles.

DRDC-RDDC-2017-R147 1

Director Cost Analytics is responsible for estimating the total life cycle costs of all new systems from the initial concept all the way through to operations, support and disposal. The investment cost of the system’s hardware and software can be estimated through market surveys. If a project requires research & development efforts, the cost variation risk can be extended to the contractor or estimated based on the scale and complexity of the project.

While O&S costs constitute the largest portion of LCC, there exist very few models for estimating them. Within the acquisition community, a 70:30 ratio1 relating O&S costs to the total investment cost of a given weapon system is often used [3]. Such a rule-of-thumb approach does not explain the cost drivers and ignores the contextual environment that affects actual expenditures. With a better understanding of factors affecting O&S costs, it is possible to provide a clearer and more accurate total cost estimate of the project under assessment.

O&S costs are typically comprised of: (1) Operating, (2) Maintenance, and (3) Sustaining support costs. Section 2 describes each of these cost categories in greater detail. Once the set of requirements for the new system has been determined, some existing cost models can be used to estimate operating costs. Maintenance and sustainment costs are more difficult to estimate and require an analysis of the cost drivers and historical data.

1.2 Literature Review

A few studies investigated the relationship between O&S costs and cost drivers specific to the context of the Navy.

In 1993, Ting [4] investigated the operating and support costs for the U.S. Navy’s main combatant vessels over the late 1970s and 1980s time span. Using regression analysis, Ting developed several models to describe manpower, material, maintenance and overhaul costs. He identified personnel costs as having the strongest effect on O&S costs.

Brandt [5] examined cost relationships among different independent variables using a 13 year data set ranging from 1984–1996. Using standard regression analysis, he constructed three univariate cost estimating equations to determine the total annual operating and support costs for given classes of ships in given time frames, excluding submarines and aircraft carriers. The three variables identified in the analysis as being significant in explaining O&S costs were ship light displacement (weight), ship overall length, and manpower.

In 2003, Hascall et al. [6] tested the accuracy of a now-dated version of the U.S. Navy’s cost estimating model, which did not encompass total operating and support costs, instead just focusing on operations and maintenance. A ship’s operational costs such as fuel, utilities, temporary additional duty, and operating target (includes repair parts and consumable purchases) were used to derive cost estimates. From the study’s findings, a more reliable forecasting model was recommended using a mixture of three-year averages or simple linear regression equations depending on the type of ship and its location in either the Atlantic or Pacific Fleet.

1 For instance, a 1M acquisition cost would be associated with 2.33M in operations and sustainment costs over the life cycle of the system.

2 DRDC-RDDC-2017-R147

More recently, in 2011, Antonucci [7] assessed the implications of the U.S. Navy’s long-term move towards a fleet size of 324 ships on overall operating and support costs. The paper identified manpower, fuel and maintenance as cost drivers in regression analysis, and supplements these findings with analogous modeling and expert testimony to estimate costs in cases where historical data does not exist. The findings showed that the real cost growth in operating and support costs associated with the increased fleet size would be a minimum of 17 percent.

1.3 Aim

The aim of this study is to develop a parametric regression model for maintenance costs, informed by an analysis of cost drivers and historical data. The model is to be capable of producing more accurate estimates of O&S costs for new defence projects.

1.4 Scope

The maintenance cost model can be applied to all categories of defence projects, but the scope of this Scientific Report is limited to commissioned Royal Canadian Navy (RCN) ships with the exception of submarines.

1.5 Report Structure

This paper is divided into seven sections. Following the introduction, Section 2 describes the components and workings of the parametric costing model. Section 3 presents the regression results. Section 4 presents the functionality of the costing tool. Section 5 concludes and provides recommendations.

For the interested reader, Annex A lists the data used in this study, Annex B provides additional information on the regression results, and Annex C describes some potential applications of the model.

DRDC-RDDC-2017-R147 3

2 Model

2.1 Cost Elements

The O&S cost elements refer to a set of resources that ensures the effective and economical support of a given military system. It includes all internal and contractor costs of (1) operating, (2) maintaining, and (3) supporting a given system throughout its life cycle as introduced in Reference [8].

2.1.1 Operating Costs

Operating costs are divided into three major categories:  The personnel directly related to the system operation (i.e., military, civilian, and contractor manpower);  The operating consumables (e.g., fuel, electricity, training munitions); and  All activities that support the system (e.g., administration, engineering).

2.1.2 Maintenance Costs

The maintenance costs are composed of all planned and unplanned activities intended to keep the system in (or to return it to) a given state or to restore it to provide additional operational capability. This includes maintenance at all lines/levels and continuing system improvements. Maintenance activities include detection, inspection, trouble shooting, prevention, testing and calibration, overhaul, and replacement of parts, components or assemblies performed by crew, by specialist repair personnel, by a depot or agency and industry (interim or continuous; this might be part of a logistic support package). A distinction is made between planned and unplanned maintenance.  Planned—This covers all activities carried out at regular time and performance intervals. Therefore, the terms “scheduled” or “preventive maintenance” are also used.  Unplanned—This covers all work that is not categorised as planned maintenance, such as the repair of defective equipment/subsystems/replacement parts. Therefore, the terms “unscheduled” or “corrective maintenance” are also used.

Continuing system improvements of a system is usually carried out at mid-life. This could be considered as new procurement occurring in the in-service phase of the system if the modifications undertaken to provide additional operational capability were not called for in the original design or performance specifications.

2.1.3 Sustaining Support Costs

Sustaining support costs are composed of all support activities provided by other organizations external to the organization that owns the systems (e.g., System specific training, security, legal).

4 DRDC-RDDC-2017-R147

Table 1 summarizes the generic cost elements in the O&S phase for defence projects.

Table 1: Description of O&S cost categories. Category Cost Element Description Personnel Military, civilian, and contractor manpower Operating Operating consumables Fuel, electricity, training munitions costs Operation support and Administration, engineering services Planned maintenance Preventive maintenance Maintenance Unplanned maintenance Corrective maintenance costs System improvements Upgrade and additional capabilities Sustaining support System specific training Specialty training activities costs Replenishment and Packaging, Handling, Storage and Transportation delivery Indirect support Security, legal

This study deals with preventive and corrective maintenance costs only: the “Maintenance Costs” component of O&S less the “system improvements” cost element. Activities related to mid-life refits were excluded from this analysis as they are generally treated as new procurements occurring in the O&S phase of the system’s life cycle.

2.2 Parametric Approach

This paper proposes a parametric regression model to investigate the relationship between selected cost drivers and maintenance costs in a Canadian context. A parametric model is a mathematical representation of cost relationships that provide a logical and predictable correlation between the physical or functional characteristics of a system and its resultant cost [9]. Fitting a linear regression allows for the estimation of the CERs between the independent variables (such as weight, number of days spent at sea, etc.) and the dependent variable (cost). The independent variables are cost drivers, and typically may be physical, performance, or operational characteristics associated with the system to be estimated.

Cost ratio estimates are simple examples of parametric estimates; however, sophisticated parametric models typically involve several independent variables or cost drivers. Similar to other conceptual estimating methods, parametric estimating is reliant on the collection and analysis of previous project cost data in order to develop the CERs. An underlying assumption of parametric estimation is that the historical framework upon which the parametric model is based continues to hold (i.e., the underlying operating environment, technology).

Regression models offer several advantages over competing techniques used in costing; in particular, they [10]:  Provide a framework for the statistical testing of model validity.  Allow for the identification of the effects of individual factors on the dependent variable.

DRDC-RDDC-2017-R147 5

 Can be tested against alternate specifications to determine which model has a superior fit.  Can produce out-of-sample predictions based on user-inputted values of the independent variables.

2.3 Random Effects Model

As the dataset is in panel format, having multiple observations for each ship over time (2011 to 2015 fiscal years), it is necessary to select a class of regression model that is capable of exploiting this structure. To this end, we opt for the random effects model. This class of model statistically controls for unobserved individual heterogeneity, i.e., the unobserved differences between each ship within the RCN fleet that can’t otherwise be explicitly included in the model.

This model type has several advantages over competing econometric specifications, such as pooled Ordinary Least Squares (OLS), fixed effects models, and time series models: pooled OLS models do not account for unobserved individual heterogeneity; fixed effects models are higher-variance;2 and time series models do not exploit cross-sectional information. For further information on random effects models, we suggest consulting Wooldridge [10].

2.4 Dependent Variable

The dependent variable in the model is Total Maintenance Costs. This is the sum of both preventive and corrective maintenance amounts. Returns (i.e., stripped equipment being added back into inventory) are removed from the totals.

2.5 Identifying Cost Drivers

The model requires the identification of relevant cost drivers in order to develop proper cost estimating relationships. These cost drivers as indicated in the literature review in Section 1.2 are generally physical, performance, or operational characteristics associated with the system to be estimated. Physical characteristics such as weight and length give an indication of the type and design of the system being estimated. Performance characteristics such as maximum speed can account for the technological aspects of the system. Operational characteristics represent both the rate at which the system is being used (demand) and the state of the system (quality), as both have an effect on cost. Equation (1) shows how maintenance costs can be expressed as a parametric function of these cost drivers.

Maintenance Costs = g(Physical Factors, Performance Factors, Demand Factors, Quality Factors) (1)

2 Fixed effects models are less efficient than random effects models, but this comes with the benefit of having more relaxed assumptions. The critical assumption that the random effects model relies upon is that the unobserved individual-specific effects are not correlated with the other covariates; this assumption is statistically tested in Section 3.3.

6 DRDC-RDDC-2017-R147

The dataset used in this study contains several potential explanatory variables to account for the right-hand side of Equation (1), including Ship Class, Age, and Sea Days. While physical factors such as Displacement and Length and performance factors such as Speed are available, these factors do not vary within classes of ship, i.e., each ship within a given class would have identical values. The effect of these variables could therefore not be identified by regression analysis. Instead, controlling for the class of ship allows for the identification of these factors collectively.

To arrive at a suitable selection of covariates, we proceed with a stepwise approach to variable selection. This process begins with a simple model that regresses yearly maintenance costs on an intercept term; we then compare the statistical properties of this model to those of alternate specifications with a single variable added (i.e., one of Ship Class, Age, or Sea Days). Once a variable is selected, the process continues in a similar manner, comparing the selected model to alternates containing an additional variable. To avoid overfitting, we evaluate model fit based on the adjusted R-squared statistic, which penalizes the measure of goodness-of-fit according to the number of included variables. A final model is reached when no further variables are selected.

2.6 Developing Cost Estimating Relationships

Equation (2) describes the relationship between total yearly maintenance costs and the identified cost drivers:

푙푛(푀푎𝑖푛푡푒푛푎푛푐푒 퐶표푠푡푠)푖푡 = 훼 + 훽푘 ∙ 푆ℎ𝑖푝 퐶푙푎푠푠푖푡 + 훾1 ∙ 푆푒푎 퐷푎푦푠푖푡 + 푢푖 + 휀푖푡 (2) where,  ln(Maintenance Costs) is the natural logarithm of the annual maintenance costs of the ship;  Ship Class is the class of ship;  Sea Days is the number of sea days recorded in the current year;

 훼 is the intercept term;

 훽푘 is a vector of ship class coefficients (with a value for each class, excepting the Halifax class, which is the comparison group) which describes the differences in base annual maintenance costs between classes;

 훾1 is a coefficient representing the effect of Sea Days on maintenance;  푢 is the unobserved individual ship-specific effect;  휀 is the stochastic error term; and,  subscripts 𝑖, 푡 refer to the individual ship and year, respectively.

Relating Equation (2) to Equation (1), physical factors, and performance factors are all taken into account through the inclusion of the Ship Class variable, while demand factors is accounted for via the inclusion of Sea Days.

Typically, we would expect that the quality factors component of the cost estimating relationship would be captured through the Age covariate; however, this variable is excluded within the

DRDC-RDDC-2017-R147 7

stepwise selection process. This exclusion is caused by a high level of collinearity with Ship Class: since (a) each ships within a given class are all produced back-to-back and (b) data covering the entire lifecycle of each ship is not available, the Age variable does not offer any additional explanatory power to the model that isn’t already account for by Ship Class.

2.7 Model Limitations

The model presented in this study is subject to several limitations:  As in any econometric analysis, the external validity of this approach is based on the assumption that historical cost estimating relationships would continue to hold over time.  The cost model controls only for a small amount of variables. Other variables that are not controlled for in the model are assumed to either not affect costs or affect them randomly.  The cost model does not take into account the economies of scale that can be generated from operating a fleet composed of multiple ships, as opposed to one ship only.  The relationships between the explanatory variables and maintenance costs are dependent upon the current operating environment of the RCN. Shifts in technology, fleet size, and geographical deployment, to name only a few factors, could alter the established costing relationships.  The dataset used in this study was not large enough to conduct a k-fold cross-validation exercise. Due to this, it is difficult to assess the quality of any out-of-sample estimates produced using the regression results.  The model strictly deals with the subset of RCN ships indicated. To produce estimates for other types of ships, relevant data would need to be procured, or a ratio-based forecasting approach as in Desmier, 2016 [11] could be used.  This model only attempts to estimate maintenance costs, not the whole of O&S costs. A more in-depth study would need to be undertaken to address this issue.

8 DRDC-RDDC-2017-R147

3 Regression Results

3.1 Dataset

The dataset used in this study consists of repeated yearly observations of commissioned Royal Canadian Navy ships: all 29 ships in active service across the Halifax, Iroquois, Kingston, and Protecteur classes from the 2011 to 2015 fiscal years. Variables include maintenance costs, sea days, age, and class. All maintenance data was originally pulled from the Defence Resource Management Information System for the purposes of an audit and provided to the DET by Director Maritime Management Support. Sea day counts were provided by Maritime Forces Atlantic and Maritime Forces Pacific. Age and class information is of public record.

All dollar figures are inflated to the end of March, 2017.

The data is presented in Table A.1 within Annex A.

3.2 Parameter Estimates

The regression results are presented in Table 2. The elements in the “Coefficient Estimate” column represent the natural logarithm of the relative effect each explanatory variable has on total O&S cost. The “Marginal Effect” column is calculated by exponentiating the coefficient estimates and deducting 1; multiplying by 100 then represents the percentage effect an incremental increase in an explanatory variable will have on the total yearly ISS cost. The Standard Error column provides the statistical uncertainty surrounding the coefficient estimate, and the p-Value column provides the likelihood that the estimate is actually zero and the stated result is obtained by chance.

Table 2: Regression results.

Marginal Coefficient Std. Variable p-Value Effect Estimate Error

Intercept (baseline) 15.2841 0.1297 ~0.0000

Iroquois class - 11.6% -0.1234 0.1426 0.3881

Kingston class - 98.9% -4.5202 0.1542 ~0.0000

Protecteur class - 64.9% -1.0477 0.1138 ~0.0000

Sea days per year + 0.38% 0.0038 0.0010 0.0001

Regression Variance: σ2 : (Sum of squared residuals / degress of freedom) = 0.6542. Standard errors and p-values are reported based on a heteroskedasticity-consistent variance-covariance estimator.

DRDC-RDDC-2017-R147 9

The “class” variables are dummy variables, with the Halifax class being the control group. Thus the coefficients for the Iroquois, Kingston, and Protecteur classes show the relative difference between the base costs of these classes and that of the Halifax class.

All explanatory variables are both jointly and individually significant, with the exception of the Iroquois class variable, which indicates that the baseline maintenance costs of this class are not statistically different from those of the Halifax class.

The most important factor in explaining yearly maintenance costs is the ship’s class. Interpreting the results, the base cost of the Kingston class is 98.9% cheaper; the Protecteur class is 64.9% cheaper, and the Iroquois class is not statistically different.

The effect of increasing the number of sea days per year by one is an increase of 0.38% on total yearly maintenance cost. Thus, a usage of 100 days per year would raise base costs by approximately 38%. Annex B provides an explanation of the interpretation of the model’s coefficient values and how to express the results of Table 2 in dollar terms.

3.3 Model Diagnostics

Table 3 presents selected diagnostics for the regression model. With a high adjusted R-squared3 of 0.87508, the model does very well in explaining the in-sample variance.

Serial correlation in the error term can adversely affect regression results, in particular from the perspective of consistency in the estimates and generating spurious or otherwise incorrect model results. The Breusch-Godfrey-Wooldridge test for serial correlation returns a p-value of 0.6355, indicating that there is no evidence of this problem in the data.

The Random Effects model is based on a crucial assumption: that the individual effects are not correlated with the explanatory variables in the regression. If this problem exists, then a Random Effects regression is not appropriate and cannot be estimated consistently; the Fixed Effects model would then need to be used as it does not rely on the same assumption. The Hausman test compares the efficiency of two models and tests whether both models are estimated consistently; if this is not rejected, the Random Effects model is preferable to Fixed Effects. The test returns a p-value of 0.1392, indicating that the null hypothesis that both models are estimated consistently is not close to statistical rejection.

Table 3: Selected regression diagnostics. Adjusted R-squared: 0.87508 Breush-Godfrey-Wooldridge test for Serial Correlation, p-value: 0.6355 Hausman Test for Random Effects vs. Fixed Effects Models, p-value: 0.1392

3 The adjusted R-squared statistic penalizes regression specifications for the inclusion of irrelevant variables; as such it is a more accurate measure of goodness-of-fit for regression models.

10 DRDC-RDDC-2017-R147

3.4 Discussion

These regression results largely agree with prior expectations. The Kingston class of vessel is the RCN’s smallest warship, with a displacement of approximate 1/5th that of the Halifax class; the result that maintenance costs are drastically lower is therefore to be expected. In the data, the Kingston class returned an average yearly maintenance cost of $84,560, compared to the Halifax class’s $7,149,000.

The Iroquois class, while slightly larger than the Halifax, had only three ships remaining in the last years of active service before the entire class was decommissioned over the 2015–2017 time period. Lower costs over this period may be attributed to a lack of desire for corrective and preventive maintenance in advance of a cessation of use, especially in the last year of service.

In the case of the Protecteur class, several reasons for lower maintenance costs than the Halifax stand out: while larger, the Protecteur is an Auxiliary Oiler Replenishment (AOR) ship that is not designed for a combat role; in addition, the Protecteur class was also phased out in the 2015 and 2016 fiscal years, being temporarily replaced by leased oil supply vessels.

Finally, the positive relationship between sea days and maintenance costs is in line with expectations, and the estimated effect of a 0.38 percent increase in annual maintenance costs per sea day is economically sensible when compared with observed usage rates and maintenance costs.

DRDC-RDDC-2017-R147 11

4 Navy Maintenance Costing Tool

In this section, we use the results of the parametric cost model in Section 3 to develop a simple tool that allows users to derive a first estimate of operations and sustainment costs of ships under consideration for procurement by the RCN. Essentially, this tool uses the results of the model in order to derive “out-of-sample” predictions based on inputted values of the independent variables.

The costing tool uses the model’s independent variables as inputs. Thus, to develop a fitted value for Cost, values for each variable must be provided by the user: Class and Sea Days.

We state two important caveats with regards to the use of this costing tool:

1. The validity of any out-of-sample prediction is contingent upon the similarity of the sample data to that which is being predicted. The model estimates used by the tool are based on the four classes of RCN ships. For instance, if a user is interested in estimating the maintenance costs of a new destroyer, the maintenance costs of the Iroquois-class destroyer would be the basis of comparison in this model. If this is not suitable, the tool should not be used in isolation; a ratio-based augmentation approach, as in Desmier [11], could be applied to the regression model’s outputs.

2. This tool is only meant to provide ROM estimates; thus, it must only be used in cases where no other, more relevant, data is available upon which an estimate can be based.

4.1 Obtaining an Estimate

The user must first enter an appropriate class for the ship being considered. The available options are the four classes included in the study: Kingston, Iroquois, Halifax, and Protecteur. Halifax is the base class, so a user need only enter a “1” next to one of the alternate classes (Iroquois, Kingston, or Protecteur) or leave them all at “0” to derive maintenance costs for a Halifax-class ship.

Following this, the user enters the assumed average number of sea days per year.

Once this is done, the tool produces an estimate of yearly maintenance costs.

The tool automatically rounds the estimated cost to the nearest thousand dollars CAD. This is done to decrease the false precision of the estimates; due to the uncertainty surrounding the data (as human error may occur during data entry) and the statistical uncertainty in the model, it is prudent to use no more than three or four significant digits in a model estimate.

Annex B offers greater detail on how the out-of-sample predictions are produced within the tool.

12 DRDC-RDDC-2017-R147

4.2 Example

Figure 2 and Figure 3 present sample estimates produced by the navy maintenance costing tool. In Figure 2, a Halifax class ship with a usage of 100 sea days is costed at $8,395,000 per year. Figure 3, meanwhile, derives a total of $144,000 for a Kingston class ship with 220 sea days.

Figure 2: RCN Ship Maintenance Costing Tool, Halifax Class Ship with 100 Sea Days.

Figure 3: RCN Ship Maintenance Costing Tool, Kingston Class Ship with 220 Sea Days.

DRDC-RDDC-2017-R147 13

5 Conclusion

In response to a Director Cost Analytics request, this Scientific Report presents a parametric costing model to forecast the maintenance component of O&S costs for RCN ships. The costing model is built based on a linear parametric approach which considers the relevant cost drivers as input parameters. The coefficients of these cost factors are estimated by fitting the model with a historical dataset of RCN ships through regression analysis. Once the coefficient values are obtained, the cost model is used to produce out-of-sample fitted values for operations and sustainment costs by specifying values for each input parameter.

The regression analysis returns highly significant explanatory variables, with an R-squared statistic of 0.875. The single most important factor in explaining yearly maintenance costs is the ship’s class. The effect of an additional sea day per year is to increase yearly maintenance costs by 0.38%; thus, a usage of 100 days per year would raise base costs by approximately 38%.

A simple Excel tool was developed based on the established cost estimating relationships. This tool allows CCD staff to produce ROM estimates of maintenance costs for any of the four classes included in the study.

The parametric costing model can be adapted and extended to estimate the O&S cost of other types of equipment and facility projects. This requires reliable historical observations for each class of defence system. Estimation of O&S costs is essential so as to better inform procurement decisions, particularly in the current context of budgetary austerity and the ongoing replacement of aging equipment systems.

5.1 Recommendations

Our primary recommendations concern increasing the scope of the study, collecting more data, and considering new methods to examine the data.

Firstly, maintenance costs are only a portion of total O&S costs; while certain components such as mid-life refits and fuel costs are often costed separately, others such as capital replacement could in principle be included in the current model.

Secondly, the dataset used in this study was not large enough to conduct a proper k-fold cross-validation test in order to determine the quality of out-of-sample predictions; adding additional years of data as they become available would enable this exercise.

Thirdly, traditional panel data models rely on standard econometric regression assumptions such as normality of the regression errors. This class of model is by no means the only possible way to estimate cost estimating relationships in the current context, nor is it necessarily the most preferable. Other classes of model, such as generalized linear models, could relax these assumptions. Replicating the analysis using alternate models could provide a sober second look at the results of this study.

14 DRDC-RDDC-2017-R147

Fourthly, if CCD wishes to use parametric costing models such as the one proposed in this paper in order to estimate maintenance and other O&S costs for future acquisitions, we recommend combining this methodology with a ratio-based forecasting approach such as the one proposed in Desmier, 2016 [11].

Finally, we recommend applying the parametric costing approach to other defence contexts, such as Royal Canadian Air Force aircraft, army vehicles and facilities.

DRDC-RDDC-2017-R147 15

6 Attachment

There is a costing tool that accompanies this document. To request access to this file, please email [email protected], citing the DRDC document number DRDC-RDDC-2017-R147 and file-specifics:

1. This tool is titled “Navy Maintenance Cost Tool.”

2. It is an Excel spreadsheet.

16 DRDC-RDDC-2017-R147

References

[1] Sokri, A., Yazbeck, T., A Predictive Method for In-Service Cost, Scientific Letter DRDC-RDDC-2015-L273, Defence Research and Development Canada, 2015.

[2] Office of the Secretary of Defence Cost Assessment and Program Evaluation, Operating and Support Cost-Estimating Guide, Department of Defence – United States of America, March 2014.

[3] Capt Jones, G. et al., Investigation into the Ratio of Operating and Support Costs to Life-Cycle Costs for DoD Weapon Systems, Defence ARJ, Vol.21 No.1:442–464, January 2014.

[4] Ting, C. W., Estimating operating and support cost models for U.S. naval ships. Master’s Thesis, Naval Postgraduate School, 1993.

[5] Brandt, J.M., A Parametric Cost Model for Estimating Operating and Support Costs of U.S. Navy (Non-Nuclear) Surface Ships, Thesis – Naval Postgraduate School, 1999.

[6] Hascall, A.M., Mathews, A.M., Gyarmati, M., Gantt, W.K., Hajdu, Z., Analysis of the Ship Ops Model’s Accuracy in Predicting U.S. Naval Ship Operating Cost, Master’s Thesis, Naval Postgraduate School, 2003.

[7] Antonnuci, K.C., Operating and Support Costs and Affordability of a 324 Ship Naval Battle Force, Master’s Thesis, Naval Postgraduate School, 2011.

[8] Sokri, A., Ghergari, V., Wang, L., Development of Cost Breakdown Structures for Defence Materiel Projects, Scientific Report DRDC-RDDC-2016-R086, Defence Research and Development Canada, 2016.

[9] International Society of Parametric Analysts, Parametric Estimating Handbook, Fourth Edition, April 2008. http://www.galorath.com/images/uploads/ISPA_PEH_4th_ed_Final.pdf. [Accessed: 14-April-2017.]

[10] Wooldridge J.M., Introductory Econometrics – A modern approach, 4th Edition, South-Western Cengage Learning, 2009.

[11] Desmier, P.E., Forecasting National Procurement Costs for the Light Armoured Vehicle (LAV) 6.0 Fleet, Scientific Report, DRDC-RDDC-2016-R178, Defence Research and Development Canada, 2016.

[12] Pugh, P.G., Source Book of Defence Equipment Costs, P. G. Pugh, 2007.

DRDC-RDDC-2017-R147 17

Annex A Data

Table A.1 displays the data used in the current study. The observations therein represent commissioned RCN vessels active within the 2011 to 2015 fiscal year time frame. The data contains yearly observations on age, sea days, and maintenance costs for each vessel.

All dollar figures are presented in Canadian dollars (CAD) inflated to the end of March, 2017.

Table A.1: RCN Dataset, 2011–2015 Fiscal Years. Ship Name Class Year Age Sea Days Maintenance ($) ln(Maintenance) HMCS Calgary Halifax 2011 16 0 $5,904,083.61 15.59115481 HMCS Calgary Halifax 2012 17 23 $2,922,474.24 14.88794116 HMCS Calgary Halifax 2013 18 102 $14,227,033.60 16.47065449 HMCS Calgary Halifax 2014 19 130 $7,210,273.56 15.79101745 HMCS Calgary Halifax 2015 20 96 $6,923,095.21 15.75037351 HMCS Charlottetown Halifax 2011 16 210 $6,244,127.13 15.64715192 HMCS Charlottetown Halifax 2012 17 128 $12,298,087.11 16.32495429 HMCS Charlottetown Halifax 2013 18 0 $4,485,261.61 15.31630738 HMCS Charlottetown Halifax 2014 19 0 $4,730,290.84 15.36949725 HMCS Charlottetown Halifax 2015 20 116 $11,158,476.01 16.22770995 HMCS Fredericton Halifax 2011 17 0 $2,970,531.05 14.9042513 HMCS Fredericton Halifax 2012 18 0 $1,562,423.69 14.26174882 HMCS Fredericton Halifax 2013 19 72 $1,224,748.43 14.01824602 HMCS Fredericton Halifax 2014 20 87 $9,140,622.76 16.02823908 HMCS Fredericton Halifax 2015 21 206 $16,060,567.80 16.59187762 HMCS Halifax Halifax 2011 19 0 $3,516,491.48 15.07297431 HMCS Halifax Halifax 2012 20 9 $1,026,791.19 13.84194915 HMCS Halifax Halifax 2013 21 101 $9,929,039.95 16.11097435 HMCS Halifax Halifax 2014 22 106 $5,339,750.94 15.49068957 HMCS Halifax Halifax 2015 23 200 $7,665,168.00 15.85219699 HMCS Montreal Halifax 2011 17 46 $2,020,863.66 14.51903553 HMCS Montreal Halifax 2012 18 0 $3,754,855.55 15.13856037 HMCS Montreal Halifax 2013 19 0 $1,407,852.37 14.15757596 HMCS Montreal Halifax 2014 20 17 $8,458,382.86 15.95066856 HMCS Montreal Halifax 2015 21 132 $7,605,642.36 15.84440095 HMCS Ottawa Halifax 2011 15 172 $8,008,925.82 15.8960672 HMCS Ottawa Halifax 2012 16 168 $7,352,178.09 15.81050717 HMCS Ottawa Halifax 2013 17 114 $9,438,239.53 16.06028003 HMCS Ottawa Halifax 2014 18 0 $8,454,865.38 15.95025262 HMCS Ottawa Halifax 2015 19 38 $6,425,664.96 15.67581068

18 DRDC-RDDC-2017-R147

Ship Name Class Year Age Sea Days Maintenance ($) ln(Maintenance) HMCS Regina Halifax 2011 17 66 $11,481,717.81 16.25626657 HMCS Regina Halifax 2012 18 215 $10,772,150.50 16.1924747 HMCS Regina Halifax 2013 19 123 $12,607,143.05 16.34977412 HMCS Regina Halifax 2014 20 127 $11,052,449.20 16.21816261 HMCS Regina Halifax 2015 21 0 $5,636,217.49 15.54472374 HMCS St. John's Halifax 2011 15 141 $6,030,263.48 15.61230126 HMCS St. John's Halifax 2012 16 114 $6,366,789.14 15.66660584 HMCS St. John's Halifax 2013 17 52 $6,480,628.47 15.68432805 HMCS St. John's Halifax 2014 18 0 $4,155,630.55 15.23997473 HMCS St. John's Halifax 2015 19 54 $7,324,093.02 15.80667989 HMCS Toronto Halifax 2011 18 7 $4,577,422.27 15.33664657 HMCS Toronto Halifax 2012 19 114 $8,328,554.15 15.93520043 HMCS Toronto Halifax 2013 20 231 $16,792,528.13 16.63644459 HMCS Toronto Halifax 2014 21 64 $18,374,525.84 16.7264758 HMCS Toronto Halifax 2015 22 0 $15,852,030.07 16.57880813 HMCS Vancouver Halifax 2011 18 228 $2,878,230.41 14.87268622 HMCS Vancouver Halifax 2012 19 94 $3,775,428.93 15.14402456 HMCS Vancouver Halifax 2013 20 0 $4,018,124.33 15.20632577 HMCS Vancouver Halifax 2014 21 56 $3,658,998.18 15.11269995 HMCS Vancouver Halifax 2015 22 123 $6,812,606.00 15.73428528 HMCS Ville-de-Quebec Halifax 2011 17 73 $4,340,045.78 15.28339546 HMCS Ville-de-Quebec Halifax 2012 18 117 $4,171,749.01 15.24384593 HMCS Ville-de-Quebec Halifax 2013 19 184 $2,730,844.60 14.8201215 HMCS Ville-de-Quebec Halifax 2014 20 0 $3,306,255.16 15.01132674 HMCS Ville-de-Quebec Halifax 2015 21 0 $6,902,721.56 15.74742632 HMCS Winnipeg Halifax 2011 16 50 $8,511,834.74 15.95696808 HMCS Winnipeg Halifax 2012 17 0 $10,432,613.83 16.1604474 HMCS Winnipeg Halifax 2013 18 17 $1,496,329.11 14.21852541 HMCS Winnipeg Halifax 2014 19 111 $10,905,406.00 16.20476919 HMCS Winnipeg Halifax 2015 20 184 $11,723,438.63 16.2771007 HMCS Algonquin Iroquois 2011 38 158 $3,252,265.65 14.99486244 HMCS Algonquin Iroquois 2012 39 130 $8,031,161.48 15.89883972 HMCS Algonquin Iroquois 2013 40 32 $7,878,146.02 15.87960316 HMCS Algonquin Iroquois 2014 41 0 $6,649,138.62 15.70999787 HMCS Algonquin Iroquois 2015 42 0 $4,768,857.92 15.37761741 HMCS Athabaskan Iroquois 2011 39 101 $3,121,832.58 14.95393075 HMCS Athabaskan Iroquois 2012 40 17 $2,330,806.43 14.66172487 HMCS Athabaskan Iroquois 2013 41 35 $3,540,372.21 15.07974242 HMCS Athabaskan Iroquois 2014 42 102 $6,164,600.89 15.63433395

DRDC-RDDC-2017-R147 19

Ship Name Class Year Age Sea Days Maintenance ($) ln(Maintenance) HMCS Athabaskan Iroquois 2015 43 150 $7,791,703.02 15.86857001 HMCS Iroquois Iroquois 2011 39 88 $3,324,373.66 15.01679184 HMCS Iroquois Iroquois 2012 40 113 $3,645,043.83 15.10887895 HMCS Iroquois Iroquois 2013 41 122 $8,238,368.78 15.92431292 HMCS Iroquois Iroquois 2014 42 0 $9,968,224.25 16.11491302 HMCS Iroquois Iroquois 2015 43 NA $5,157,499.28 15.45596238 HMCS Brandon Kingston 2011 12 120 $86,793.94 11.37129204 HMCS Brandon Kingston 2012 13 106 $153,009.71 11.93825667 HMCS Brandon Kingston 2013 14 0 $58,524.27 10.97719687 HMCS Brandon Kingston 2014 15 118 $131,361.33 11.78570708 HMCS Brandon Kingston 2015 16 65 $97,062.53 11.48311074 HMCS Edmonton Kingston 2011 14 53 $58,927.25 10.98405896 HMCS Edmonton Kingston 2012 15 112 $166,517.63 12.02285644 HMCS Edmonton Kingston 2013 16 100 $109,493.54 11.60362088 HMCS Edmonton Kingston 2014 17 44 $122,865.28 11.71884375 HMCS Edmonton Kingston 2015 18 43 $74,729.38 11.22162863 HMCS Glace Bay Kingston 2011 15 49 $26,377.57 10.18026915 HMCS Glace Bay Kingston 2012 16 34 $41,725.60 10.63887009 HMCS Glace Bay Kingston 2013 17 136 $20,849.39 9.945080158 HMCS Glace Bay Kingston 2014 18 80 $154,199.70 11.94600379 HMCS Glace Bay Kingston 2015 19 73 $41,402.64 10.63109986 HMCS Goose Bay Kingston 2011 13 107 $17,730.65 9.783050269 HMCS Goose Bay Kingston 2012 14 67 $113,583.10 11.64028999 HMCS Goose Bay Kingston 2013 15 0 $19,041.92 9.854398337 HMCS Goose Bay Kingston 2014 16 0 $231,351.36 12.35169287 HMCS Goose Bay Kingston 2015 17 95 $93,407.96 11.44473188 HMCS Kingston Kingston 2011 15 49 $1,399.79 7.244075171 HMCS Kingston Kingston 2012 16 122 $122,146.43 11.71297587 HMCS Kingston Kingston 2013 17 95 $121,058.43 11.70402859 HMCS Kingston Kingston 2014 18 67 $38,093.80 10.54780681 HMCS Kingston Kingston 2015 19 13 $28,028.67 10.24098305 HMCS Moncton Kingston 2011 13 88 $6,022.07 8.703186767 HMCS Moncton Kingston 2012 14 90 $71,751.94 11.18097011 HMCS Moncton Kingston 2013 15 0 $22,797.32 10.0343984 HMCS Moncton Kingston 2014 16 0 $112,651.72 11.63205618 HMCS Moncton Kingston 2015 17 146 $179,003.88 12.09516274 HMCS Nanaimo Kingston 2011 14 122 $187,101.19 12.13940489 HMCS Nanaimo Kingston 2012 15 105 $116,002.29 11.66136521 HMCS Nanaimo Kingston 2013 16 135 $107,554.96 11.58575724

20 DRDC-RDDC-2017-R147

Ship Name Class Year Age Sea Days Maintenance ($) ln(Maintenance) HMCS Nanaimo Kingston 2014 17 119 $76,205.99 11.24119536 HMCS Nanaimo Kingston 2015 18 63 $110,853.51 11.6159649 HMCS Saskatoon Kingston 2011 13 168 $46,985.05 10.75758469 HMCS Saskatoon Kingston 2012 14 101 $151,787.41 11.93023623 HMCS Saskatoon Kingston 2013 15 64 $56,031.20 10.93366391 HMCS Saskatoon Kingston 2014 16 21 $56,053.94 10.93406976 HMCS Saskatoon Kingston 2015 17 139 $83,964.93 11.33815452 HMCS Shawinigan Kingston 2011 15 62 $7,322.50 8.89870678 HMCS Shawinigan Kingston 2012 16 20 $71,645.77 11.17948945 HMCS Shawinigan Kingston 2013 17 105 $114,192.71 11.64564275 HMCS Shawinigan Kingston 2014 18 134 $70,295.35 11.16046098 HMCS Shawinigan Kingston 2015 19 99 $49,092.90 10.80146975 HMCS Summerside Kingston 2011 12 133 $29,758.50 10.30087014 HMCS Summerside Kingston 2012 13 119 $41,556.96 10.63482037 HMCS Summerside Kingston 2013 14 79 $69,199.17 11.14474411 HMCS Summerside Kingston 2014 15 16 $41,935.26 10.64388225 HMCS Summerside Kingston 2015 16 126 $61,908.10 11.03340627 HMCS Whitehorse Kingston 2011 13 88 $125,838.82 11.74275715 HMCS Whitehorse Kingston 2012 14 4 $163,592.72 12.0051352 HMCS Whitehorse Kingston 2013 15 113 $77,235.92 11.2546199 HMCS Whitehorse Kingston 2014 16 120 $141,525.94 11.86023827 HMCS Whitehorse Kingston 2015 17 122 $103,077.86 11.54323992 HMCS Yellowknife Kingston 2011 13 20 $19,327.12 9.869264476 HMCS Yellowknife Kingston 2012 14 125 $106,039.91 11.57157082 HMCS Yellowknife Kingston 2013 15 103 $67,599.10 11.12135001 HMCS Yellowknife Kingston 2014 16 92 $115,753.99 11.65922248 HMCS Yellowknife Kingston 2015 17 35 $82,185.85 11.3167384 HMCS Preserver Protecteur 2011 41 20 $69,165.39 11.14425581 HMCS Preserver Protecteur 2012 42 122 $4,416,953.60 15.30096079 HMCS Preserver Protecteur 2013 43 107 $5,890,447.74 15.58884257 HMCS Preserver Protecteur 2014 44 0 $5,591,607.10 15.5367773 HMCS Preserver Protecteur 2015 45 NA $10,717,989.08 16.18743411 HMCS Protecteur Protecteur 2011 42 96 $3,164,394.46 14.96747227 HMCS Protecteur Protecteur 2012 43 0 $2,830,548.08 14.85598092 HMCS Protecteur Protecteur 2013 44 119 $1,957,685.87 14.48727366 HMCS Protecteur Protecteur 2014 45 0 $4,861,977.27 15.39695576 HMCS Protecteur Protecteur 2015 46 0 $300,412.76 12.61291266

DRDC-RDDC-2017-R147 21

Annex B Interpreting the Regression Results

To obtain the dollar effect for a change in each factor we must first calculate the collective total after each factor is taken into account. For example, suppose we would like to work through the example provided in Section 4, where we want to obtain the yearly maintenance cost for a Kingston-class Coastal Defence Vessel with 200 sea days per year. The model projects the cost as follows:

휎2 푀푎𝑖푛푡푒푛푎푛푐푒 퐶표푠푡 = exp(15.284 + (−4.520) + 200 ∗ (0.0038)) ∙ exp ( ) (B.1) 2

휎2 exp ( ) where 2 is a correction term that is needed in order to obtain the mean of a log-normal distribution; the parameter σ2 is the regression variance, 0.6542.

0.6542 푀푎𝑖푛푡푒푛푎푛푐푒 퐶표푠푡 = exp(11.524) ∙ exp ( ) 2 (B.2) = $140,238.52

Note: to avoid displaying false precision, the dollar values should be rounded off to no more than 3 or 4 significant digits. In this case, the result would be $140,000.00. For the purposes of understanding marginal effects within this example, however, we continue with the total in Equation (B.2).

As discussed in Section 3, the relative marginal effect of an increase in a covariate (Sea Days, in particular) is calculated as exp (푐표푒푓푓𝑖푐𝑖푒푛푡 .− 1) For the Sea Days variable, this effect is 0.38% per day.

Suppose we increment Sea Days by 1, for a total of 201. We now have:

휎2 푀푎𝑖푛푡푒푛푎푛푐푒 퐶표푠푡 = exp(15.284 + (−4.520) + 201 ∗ (0.0038)) ∙ exp ( ) 2 0.6542 푀푎𝑖푛푡푒푛푎푛푐푒 퐶표푠푡 = exp(11.5278) ∙ 푒푥푝 ( ) (B.3) 2

= $140,772.45

So that an additional Sea Day costs $140,772.45 – $140,238.52 = $533.93 which is a 0.38% increase. Naturally, the marginal increase due to a change in Sea Days, being proportional, is dependent on the base amount of maintenance costs; an increase of one Sea Day for a Halifax Class ship with a base of 200 Sea Days costs an additional $48,926.

22 DRDC-RDDC-2017-R147

Annex C Potential Applications

This section describes potential applications of the methods employed in this analysis.

C.1 Aircraft and Land Vehicles

The methodology proposed in this paper is applicable to all major classes of Navy warships and can be adapted to other types of equipment. For example, the “number of sea days” variable can be replaced by a “yearly flying rate” (YFR) or “number of kilometers” (kms) variable to describe O&S costs in the context of aircraft or army vehicles. Equations (C.1) and (C.2) show how these models can be expressed.

푙푛 (푂&푆 퐶표푠푡) = 훼 + 훽1(퐴𝑖푟푐푟푎푓푡 _푇푦푝푒) + 훾1(푌퐹푅) + 휇1(푀푎𝑖푛푡) + 휀 (C.1)

푙푛(푂&푆 퐶표푠푡) = 훼 + 훽1(푉푒ℎ𝑖푐푙푒푇푦푝푒) + 훾1(푘푚푠) + 휇1(푀푎𝑖푛푡) + 휀 (C.2)

C.2 Facilities

The O&S costs for facilities may also be estimated using the present methodology. Assuming adequate data is available, a CER for facility projects can be expressed, for example, as a function of the facility type, the volume (size) of the facility, and maintenance:

ln(O&S Cost) = α + β1(FacilityType) + γ1(Volume) + μ1(Maint) + ε (C.3)

C.3 Classification

Figure C.1 shows all categories of defence systems for which O&S costs would need to be estimated. Due to the large number of defence systems, it is necessary to categorize projects. The classification we propose is based on Pugh [12]. The classes are made based on similarities of function, technology and degree of complexity.

DRDC-RDDC-2017-R147 23

Project Project Type System Category System Class

Sea Muti-Role Patrol Frigate (FFH)

Area Air Defence Destroyer (DDG) Aircraft & UAV Auxiliary Oil Replenishment (AOR)

Surface Vehicle Long-Range Patrol Submarine (SSK)

Coastal Defence Vessel (MCDVs) Missile

Manned Fixed-Wing Aircraft Equipment Ordnance Unmanned Aerial Vehicles

Launch Vehicle Rotary-Wing Aircraft

Electronic Tanks and Infantry Vehicles

Artillery Vehicles Space Engineer and Logistic Vehicles Automated Information

New Defence Gear Acquisition Project Administration

Operations

Hangars

Building Storage

Training

Single Quarters Facilities NPF

NPF Retail Outlets

Miscellaneous Land )LJXUH&&ODVVLILFDWLRQRIGHIHQFHHTXLSPHQW

'5'&5''&5

List of Symbols/Abbreviations/Acronyms/Initialisms

CCD Centre for Costing in Defence CER Cost Estimating Relationship DET Defence Economics Team DRDC Defence Research and Development Canada LCC Life-Cycle Cost O&S Operations and Sustainment OLS Ordinary Least Squares (regression model class) RCN Royal Canadian Navy

DRDC-RDDC-2017-R147 25

DOCUMENT CONTROL DATA (Security markings for the title, abstract and indexing annotation must be entered when the document is Classified or Designated) 1. ORIGINATOR (The name and address of the organization preparing the document. 2a. SECURITY MARKING Organizations for whom the document was prepared, e.g., Centre sponsoring a (Overall security marking of the document including contractor's report, or tasking agency, are entered in Section 8.) special supplemental markings if applicable.)

DRDC – Centre for Operational Research and Analysis CAN UNCLASSIFIED Defence Research and Development Canada 101 Colonel By Drive Ottawa, Ontario K1A 0K2 2b. CONTROLLED GOODS Canada (NON-CONTROLLED GOODS) DMC A REVIEW: GCEC DECEMBER 2013

3. TITLE (The complete document title as indicated on the title page. Its classification should be indicated by the appropriate abbreviation (S, C or U) in parentheses after the title.)

Estimating Maintenance Costs for Royal Canadian Navy Ships : A Parametric Cost Model

4. AUTHORS (last name, followed by initials – ranks, titles, etc., not to be used)

Bouayed, Z.; Penney, C.E.; Sokri, A.; Yazbeck, T.

5. DATE OF PUBLICATION 6a. NO. OF PAGES 6b. NO. OF REFS (Month and year of publication of document.) (Total containing information, (Total cited in document.) including Annexes, Appendices, etc.) October 2017 31 12

7. DESCRIPTIVE NOTES (The category of the document, e.g., technical report, technical note or memorandum. If appropriate, enter the type of report, e.g., interim, progress, summary, annual or final. Give the inclusive dates when a specific reporting period is covered.)

Scientific Report

8. SPONSORING ACTIVITY (The name of the department project office or laboratory sponsoring the research and development – include address.)

DRDC – Centre for Operational Research and Analysis Defence Research and Development Canada 101 Colonel By Drive Ottawa, Ontario K1A 0K2 Canada

9a. PROJECT OR GRANT NO. (If appropriate, the applicable research 9b. CONTRACT NO. (If appropriate, the applicable number under and development project or grant number under which the document which the document was written.) was written. Please specify whether project or grant.)

10a. ORIGINATOR’S DOCUMENT NUMBER (The official document 10b. OTHER DOCUMENT NO(s). (Any other numbers which may be number by which the document is identified by the originating assigned this document either by the originator or by the sponsor.) activity. This number must be unique to this document.)

DRDC-RDDC-2017-R147

11. DOCUMENT AVAILABILITY (Any limitations on further dissemination of the document, other than those imposed by security classification.)

Unlimited

12. DOCUMENT ANNOUNCEMENT (Any limitation to the bibliographic announcement of this document. This will normally correspond to the Document Availability (11). However, where further distribution (beyond the audience specified in (11) is possible, a wider announcement audience may be selected.))

Unlimited

13. ABSTRACT (A brief and factual summary of the document. It may also appear elsewhere in the body of the document itself. It is highly desirable that the abstract of classified documents be unclassified. Each paragraph of the abstract shall begin with an indication of the security classification of the information in the paragraph (unless the document itself is unclassified) represented as (S), (C), (R), or (U). It is not necessary to include here abstracts in both official languages unless the text is bilingual.)

14. KEYWORDS, DESCRIPTORS or IDENTIFIERS (Technically meaningful terms or short phrases that characterize a document and could be helpful in cataloguing the document. They should be selected so that no security classification is required. Identifiers, such as equipment model designation, trade name, military project code name, geographic location may also be included. If possible keywords should be selected from a published thesaurus, e.g., Thesaurus of Engineering and Scientific Terms (TEST) and that thesaurus identified. If it is not possible to select indexing terms which are Unclassified, the classification of each should be indicated as with the title.)

Parametric Costing Model; In-Service Support costs; Regression Analysis