Range Estimating Is Not Risk Analysis

CSC Project Management Services A partnership

Range Estimating is Not Risk Analysis

CSC CSC Excellence in Risk Management C

200, 321 – 19 Street N.W. Calgary, Alberta Canada T2N 2J2 (403) 233-7994 (phone) Range Estimating is Not Risk Analysis

(403) 263 7470 (fax) Email: [email protected] In today’s environment of intense competition for the resources to bring projects to a successful conclusion, sophisticated stakeholders are realizing that the criteria by which project results are judged - capital cost, schedule, operating costs, production levels, return on investment, environmental impacts - cannot be adequately expressed as the single point values that have been commonly utilized in the past. No estimate, however rigorously developed, is ever correct. The estimated cost, or duration, or production rate, only represents a single point within a range of possible values, with an indeterminate probability of being achieved, or exceeded, or under run.

Stakeholders need to know what the confidence level, or probability, is of achieving defined hurdle levels for each project criteria. They also need to understand the factors that may lead to the actual results being greater or less than the predicted values. Different processes, Risk Analysis and Range Estimating, have been proposed which purport to answer these questions. The two processes are not equivalent.

Risk Analysis is a comprehensive and exhaustive process that, if applied correctly, provides a distribution of potential values for any chosen project measurement criteria. One of the tools utilized is a Monte Carlo random sampling of probabilistic data. Distributions of results are developed from which the probability of success for any criteria can be established. The Analysis also identifies and quantifies the underlying reasons for the potential variances from base estimates.

Range Estimating is a simpler application of the Monte Carlo technique that is currently being utilized by some project management practitioners to develop a distribution of possible values for the capital cost estimate component of a project. This is then being used to predict a probability of achieving a specific capital cost. In both procedures, the difference between the current base capital cost estimate and the value at any specified confidence level can be established as a capital cost “contingency”.

Since Range Estimating solicits some similar data and provides an outcome which is similar in appearance to one of the outcomes of a Risk Analysis, it is considered by some to be a quick and easy (read cheaper) substitute for the more comprehensive and rigorous Risk Analysis. While the use of Range Estimating can be a useful tool in the determination of overall risk/opportunity, there are several reasons why the reliance upon Range Estimating as a stand-alone substitute for Risk Analysis can be a potentially misleading practice.

 Risk Analysis, properly applied, considers the total project in an integrated planning environment that anticipates the possibility of fundamental changes in definition, scope, timing, performance and market conditions. The most meaningful results come when project parameters over the total life of the project are subjected to the Analysis, so that relationships between capital cost, schedule, operating cost, production and revenue and total project value can be fully developed.

 Range Estimating is typically applied only to the capital cost estimate, with occasional application to the construction schedule. It is applied only to cost items identified in the capital cost estimate for the current definition of scope for the project. The ranges

B. A. Ostermann, P. Eng, M.Sc. 2 06/08/18 Range Estimating is Not Risk Analysis

solicited are based upon the assumption that the defined scope will be built, the project will operate within the design parameters, and that it will be constructed in the time frame of the project schedule. The estimator is simply being asked to provide the estimating variance that could be expected under fixed and constant conditions.

 The Risk Analysis process and modeling technique allows any number of strategic options, different project configurations and execution plans to be tested and compared within one model, utilizing common planning and assessment data. This can be done early in the project life with preliminary information. In this way project both expected value and degree of uncertainty can easily be used to rank options, so that the best project can be selected with some degree of confidence.

 While Range Estimating can be used to compare projects, the process generally requires relatively detailed cost estimates for each and the relative uncertainty between options is difficult to ascertain. Much more planning and estimating effort is required before options can be compared with any confidence.

 Risk Analysis puts the project in an uncertain execution, implementation and operating environment. It allows quantification of the impact of potential changes in the political, social, environmental, competitive and market conditions under which the project will be constructed and operated. These outside, or “soft issues,” which can constitute a risk or which may provide an opportunity for the project are used to condition the impact data solicited so that their influence on project outcomes can be identified and quantified.

 Range Estimating does not consider aspects of the overall project other than the capital cost for the current definition of scope. Changes to execution planning, schedules, product pricing, market size, resource availability, operability and production rates, which might cause fundamental changes to the scope and timing of the project, and hence the capital costs, are not considered. “Soft issues” which may cause changes to the environment in which the project may be constructed are also not considered in the ranges assessed, since they are outside the purview of the original capital cost estimate.

 Risk Analysis is applied to the building blocks of the line items within the cost estimate, such as labour, equipment, materials, indirect costs, construction management, etc. The variances in these components, rather than simply the line item unit cost, are quantified and applied across all the line items in the estimate. For example, a range in labour rates is applied to all line items that have a labour cost component.

 Range Estimating typically solicits a range for the unit or total cost of each line item in the capital cost estimate. It does not identify and document the underlying reasons for the ranges that are assigned to each cost. What would cause the high side of a unit cost range to be experienced - higher labour rates, material cost escalation, low labour productivity, etc.?

 The Risk Analysis model logic usually develops relationships between schedule and cost so that the value of delays and accelerations can be determined. Because the total project

is considered in a Risk Analysis the relationships of mechanical completion, start up, and ramp up milestones to production profiles, revenue and operating costs can also be used to calculate time based overall project values. This allows the trade off between different execution strategy costs and implementation milestones to be tested to determine the best options.

 Range Estimating treats schedule independently from cost, and generally only considers the project time line up to mechanical completion date, or end of the capital cost expenditure profile. There is no mechanism for testing cost impacts of schedule variances, and there is no relationship between time, capital cost, and overall project value.

 Risk Analysis provides a comprehensive suite of tools and outcomes that identify and quantify the reasons for the expected variances in outcomes. Influence diagrams establish relationships between issues and outcomes, and identify conditioning variables that define the different environments that may influence project results. A project specific probabilistic model is developed to manipulate base and impact data, simulating project performance under changing conditions. Distribution curves of results, “tornado diagrams”, expenditure and production profiles, and “step diagrams” can be produced for any project outcome. This allows ranking of the impacts from threat/opportunity issues. Rational development of mitigation planning for the most significant issues with quantified results, testing of different strategies, and evaluation of “what if” scenarios can then follow.

 Range Estimating utilizes a simple probabilistic model, which randomly applies the assessed high/expected/low cost ranges to the estimate line items to calculate a revised total cost on each model iteration. Results are generally presented in the form of a distribution curve for capital cost, from which the probability of achieving any specified cost can be estimated. There is no other data available that would allow the reason for the shape of the distribution to be determined, or to identify the drivers for the overall range in results.

 Correlation of risk issues through conditioning variables in the Risk Analysis allows the impact of any underlying causes of potential change to be applied in a logical and consistent manner. For example, a heated competitive environment would be expected to cause upward pressure on both labour and materials prices, and may also have a higher expectation of a corresponding reduction in labour productivity.

 There is no specific correlation between cost ranges in Range Estimating. Each unit cost is treated as an independent variable, leading to combinations of high/low values during the random sampling process. This allows calculations of offsetting results that logically would not exist. For example, in any probabilistic iteration a high value for one unit cost can be offset by a low value for a similar unit, even if the underlying reason for the high value is an overall increase in labour rates. Further, if the underlying cause of the high value is an expectation of higher labour rates because of a heated construction environment, is it logical to assume that material costs could be lower in the same

environment? The distribution of Range Estimate results can therefore be skewed and the overall range in results minimized.

 The capture of data to quantify probabilities and impacts for the identified risk issues is completely flexible in Risk Analysis, with no prestructured format or predetermined values. The experts are allowed to determine the definition of the risk variable and the type of distribution that best captures the impact of the variable. For example, a five or seven point discrete distribution may capture the expert’s expectations better than a triangular distribution.

 Range estimating forces the use of a partially predetermined distribution that has no flexibility. The probabilities of experiencing the high or low values for each unit cost are considered to be equal. There is no indication from the experts as to which is more or less likely to occur, so both values are assigned the same probability of occurrence in a triangular distribution, which is not necessarily true. Further, the absolute maximum and minimum values solicited are sometimes difficult to rationalize, leading the experts to inaccurate estimates.

 Risk Analysis, properly facilitated, solicits the views of a broad spectrum of project stakeholders on all aspects of the project. This eliminates potential biases and produces a more independent view of the project. The objective of the analysis is not just the production of a safe “contingency”, but the development of a “better” outcome for the project which takes into account all of the risks and opportunities inherent in the project environment.

 The proponents of Range Estimating generally take the view of the “Constructor” of the project, not the “Owner” of the project. The experts are usually drawn from the ranks of those that prepared or approved the estimates. This may introduce biases that may not be in the best interests of the project Owner.

A comprehensive Risk Analysis considers the project as a whole and captures the impact of all internal and external influences, even if capital cost was the only unit of measure being used to determine project success. It correlates the ranges assessed to eliminate the illogical combinations that may be included in the Range Estimate results.

The example below compares the results of a Range Estimate to a Risk Analysis on the capital cost distribution for a typical project, and shows how the Range Estimate generally understates the magnitude of the uncertainty on major projects.

Risk Analysis Expected Value for Capital is 4% higher than the Range Estimate p50. The Risk distribution has a wider range (-16/+15% vs -14/+15%) than Range Estimate results. The Risk Analysis 80% confidence level would be 100 $M higher than the Range Estimate prediction.

Project Capital Risk Analysis vs Range Estimating Distributions 100

90 Base Estimate 80 Without Contingency

70 1,972 $M y t i l i b a

b 60 o r

P Risk Analysis

v Range Estimate i

a EV = 2,392 $M l

u P50 = 2,290 $M m

u 40 P50 = 2,366 $M C

20 Range Estimating Risk Analysis 10

0 $1,500 $1,700 $1,900 $2,100 $2,300 $2,500 $2,700 $2,900 $3,100 $3,300 $3,500 $M 98 CSC Sept 1, 1998

A key aspect that distinguishes Risk Analysis from Range Estimating is the correlation of risk issues using conditioning variables. The relationships used are demonstrated graphically in the Influence Diagram, which defines the logic utilized in the Risk Model.

Legend Site Manhours Risk Analysis Conditioning Variable Variance Capital Cost Influence Diagram Impact Variable Execution Fix-up Capital % Organization Decision Performance Operating Result Plant Interference

Module Fabrication Plant Re-work Manhours Module # 1 Factor Variance Fabrication % Escalation Size of Scope Rates Plant Variance Site Plant - Plants Labour Rate # 2 Variance

Quality of Fabrication Project Labour Rate Plant Total Level of Definition Variance Winter # 3 Capital Competing Work Costs Projects Major Factors Equipment Cost Variance Bulk Mgmt/Eng. Materials Rate Pre- Miscellaneous Labour Rate Variance Investment Plants Variance Capital

Support Mgmt/Eng. Cost Manhour Variance Variance Construction Engineering Camp Construction Support/ & Project Cost Management Indirect Management Variance

CSC Project Management Services

The conditioning of risk variables can significantly influence the expected project outcomes, by eliminating illogical combinations of variables in the random sampling process.

Lack of correlation by conditioning risk variables can provide misleading results on a typical project. In this example, the unconditioned expected value of NPV is overstated by 112 $M (2%), while the risk range remains the same at +/- 13%.

Project NPV 100

80 y t i

l 70 i b a

b 60 o

r Conditioned Unconditioned P 50 e NPV = 6,428 $M NPV = 6,540 $M v i t

a 40 l u

m 30 u C 20

0 5000 6000 7000 8000 9000

NPV $M Note: NPV considers capital costs, schedule, operating costs, production and revenue factors.

In addition to the typical distribution curves of results shown above, the Risk Analysis provides some other tools that can be used to provide more insight into the issues that may lead to differences in project outcomes. The Tornado Diagram identifies the sources of most uncertainty in project results, thereby allowing mitigation planning to capture potential upsides and avoid downsides.

The Tornado Diagram ranks the Issues by their contribution to the uncertainty in overall Project outcomes (the slope of the distribution curve). Pipeline Capital Cost Delta to EV - $k -15,000 -10,000 -5,000 0 5,000 10,000 15,000 20,000 25,000

Level of Competing Projects for Pipeline Resources Low Heated EV = 0.951 Initial Pipe Supply Cost Variance - Multiplier 0.877 1.022 Base = 1.000 Project team Organizational Performance Excellent Poor EV = 4.69% Extra Work Factor- % of Total Field Cost 1.75% 8.00% Base = 0.00% EV = 3.88% Owners P/L Cost Factor- % of Total Installed Cost 2.00% 5.50% Base = 3.00% EV = 3.25% Pipeline EPCM Cost Factor - % of Total Field Cost 2.00% 5.00% Base = 3.00% EV = 2.32% Mainline Alignment Length Variance - % 0.00% 4.88% Base = 0.00% EV = 3.00% Owner Preapproval Cost - % of Total Installed Cost 2.00% 4.00% Base = 3.00% EV = 7.69 mos Pipe Delivery Duration - Months 6 13.5 Base = 5.00 mos EV = 0.37 mos Labour Disruption Delay - Months 0.00 0.75 Base = 0.00 mos EV = 0.955 Spread Labour Productivity Variance - Multiplier 1.154 0.747 Base = 1.000

EV = $403,000 k Notes: 1. The length of each bar is calculated by progressively fixing 3. All Variables are checked, but only the major contributors are each Variable at it’s extreme value, running the probabilistic shown graphically as the ones which require priority attention. model with all other Variables unconstrained, and measuring 4. The left side of each bar is an opportunity to be pursued; the the change in value from the unrestricted EV. right side indicates where mitigation efforts should be planned to 2. The zero point for each bar represents the expected value, or avoid negative impacts. weighted average, of the assessment data for that Variable. 5. The impact on Project Expected Value for the change of each The EV for a Variable is not necessarily the same as the base Variable from base to it’s EV is shown on the Step Diagram. value.

CSC E xcellence I n R isk Management November 10, 1999

The Step Diagram shows which areas of the project the experts expect to be different than the base assumptions inherent to the single point estimate, and quantifies the expected difference. This allows specific areas of the base plan and estimate to be revisited and modified if desired.

The Step Diagram shows where the change from Estimate Base to risk adjusted Expected Value occurs. It is developed by progressively changing groups of Variables from base values to assessed probabilistic ranges and calculating a new EV. The step is the change in Project EV from the previous setting, caused by the

405,000 new values. +5,315 +290

EV= $403,000 400,000 Pipeline +6,000 395,000 Capital Cost +3,000 +2,750 390,000 +3,750 k

$ 385,000 +3,500 +9,136 Notes: 380,000 +4,152 1. The steps represent the experts’ opinion of Base w/o Contingency -872 areas which will vary from the base estimate - e.g., the expectation is that Mainline Scope will 375,000 $368,000 +1,345 +1,940 be 4,152 $k greater than base. -6,360 +779 2. The steps may be used to allocate contingency 370,000 +665 allowances, or may indicate areas of the estimate which should be revisited. 365,000

n e t k t l t e n e y s r g g s e c s m o e a c n r o c e i e o n i c n i e o n o p l v e t n i o t i t t r i t t a e n a o u e a o I r s C c c C i e e i s s i m W a P a r h n c c s l l l e d i r a

l r u i i i i e e s n p o e u l u r r a v S u a C p o l r y a u r c c t r i

t o r p a e d h d i q a a v u g P a i t s e e V e n c i r

c t e e p R C c c l e A t x r n c L V t F I n i a e o a P e h i h S i s o u e i e

o n A c R l P E F r c r c t t t p n

a m i n n a C a t i o P S P S d e i

a e r a U C n r s c l l n i h a e u a M o r c a u U S c n a e i D C M V d L S l e R V e h p c i CSC P S E xcellen ce I n R isk M an agement November 10, 1999

The results that can be developed in a properly structured and facilitated Risk Analysis provide the project stakeholders with a comprehensive view of their project, allowing informed and rational decision making to develop a better project. Tools are available to test the impact of decisions and outside influences. The fundamental differences between Risk Analysis and Range Estimating can be summarized in the following table.

Risk Analysis Range Estimating

Objective Optimize project configuration and maximize To calculate project contingency. value returned to owner. Outputs Cost, schedule and value probability Cost probability distribution, contingency amount. distributions, contingency amount, key uncertainty drivers, mitigation plans, optimized configuration. Project Strategic options identified to test uncertain Project scope fixed for each estimate. Alternatives project scope and execution plans. Assessments Broad groups of experts, inclusion of all Project estimators provide data based on elements stakeholders, risks inside and outside of of estimate. project considered. Distributions Experts provide probability of occurrence and Fixed data distribution (triangular), with end points impacts for defined variables in flexible of distribution specified probabilities. formats. Correlation Risk issues correlated using conditioning Minor correlation, depending upon tool utilized. variables. Variables essentially treated as independent. Schedule Cost and schedule logically integrated to Treats schedule independently up to mechanical Integration calculate impacts of delays/accelerations. completion only. No logic to link cost or value Includes start up and ramp up schedules and with schedule. costs. Comprehensive Deals with underlying risk issues such as Focused on cost elements of the current estimate, Competing Projects, Organizational not risk issues. Performance. Benefits Owners and investors. Constructors, contractors and EPCM alliances.

B. A. Ostermann, P. Eng, M.Sc. 11 06/08/18