Range Estimating Is Not Risk Analysis

Range Estimating is Not Risk Analysis

CSC

Project Management Services

A partnership

Range Estimating is Not Risk Analysis

200, 321 – 19 Street N.W.

Calgary, Alberta

Canada

T2N 2J2

(403) 233-7994 (phone)

(403) 263 7470 (fax)

Email:

In today’s environment of intense competition for the resources to bring projects to a successful conclusion, sophisticated stakeholders are realizing that the criteria by which project results are judged - capital cost, schedule, operating costs, production levels, return on investment, environmental impacts - cannot be adequately expressed as the single point values that have been commonly utilized in the past. No estimate, however rigorously developed, is ever correct. The estimated cost, or duration, or production rate, only represents a single point within a range of possible values, with an indeterminate probability of being achieved, or exceeded, or under run.

Stakeholders need to know what the confidence level, or probability, is of achieving defined hurdle levels for each project criteria. They also need to understand the factors that may lead to the actual results being greater or less than the predicted values. Different processes, Risk Analysis and Range Estimating, have been proposed which purport to answer these questions. The two processes are not equivalent.

Risk Analysis is a comprehensive and exhaustive process that, if applied correctly, provides a distribution of potential values for any chosen project measurement criteria. One of the tools utilized is a Monte Carlo random sampling of probabilistic data. Distributions of results are developed from which the probability of success for any criteria can be established. The Analysis also identifies and quantifies the underlying reasons for the potential variances from base estimates.

Range Estimating is a simpler application of the Monte Carlo technique that is currently being utilized by some project management practitioners to develop a distribution of possible values for the capital cost estimate component of a project. This is then being used to predict a probability of achieving a specific capital cost. In both procedures, the difference between the current base capital cost estimate and the value at any specified confidence level can be established as a capital cost “contingency”.

Since Range Estimating solicits some similar data and provides an outcome which is similar in appearance to one of the outcomes of a Risk Analysis, it is considered by some to be a quick and easy (read cheaper) substitute for the more comprehensive and rigorous Risk Analysis. While the use of Range Estimating can be a useful tool in the determination of overall risk/opportunity, there are several reasons why the reliance upon Range Estimating as a stand-alone substitute for Risk Analysis can be a potentially misleading practice.

Risk Analysis, properly applied, considers the total project in an integrated planning environment that anticipates the possibility of fundamental changes in definition, scope, timing, performance and market conditions. The most meaningful results come when project parameters over the total life of the project are subjected to the Analysis, so that relationships between capital cost, schedule, operating cost, production and revenue and total project value can be fully developed.

Range Estimating is typically applied only to the capital cost estimate, with occasional application to the construction schedule. It is applied only to cost items identified in the capital cost estimate for the current definition of scope for the project. The ranges solicited are based upon the assumption that the defined scope will be built, the project will operate within the design parameters, and that it will be constructed in the time frame of the project schedule. The estimator is simply being asked to provide the estimating variance that could be expected under fixed and constant conditions.

The Risk Analysis process and modeling technique allows any number of strategic options, different project configurations and execution plans to be tested and compared within one model, utilizing common planning and assessment data. This can be done early in the project life with preliminary information. In this way project both expected value and degree of uncertainty can easily be used to rank options, so that the best project can be selected with some degree of confidence.

While Range Estimating can be used to compare projects, the process generally requires relatively detailed cost estimates for each and the relative uncertainty between options is difficult to ascertain. Much more planning and estimating effort is required before options can be compared with any confidence.

Risk Analysis puts the project in an uncertain execution, implementation and operating environment. It allows quantification of the impact of potential changes in the political, social, environmental, competitive and market conditions under which the project will be constructed and operated. These outside, or “soft issues,” which can constitute a risk or which may provide an opportunity for the project are used to condition the impact data solicited so that their influence on project outcomes can be identified and quantified.

Range Estimating does not consider aspects of the overall project other than the capital cost for the current definition of scope. Changes to execution planning, schedules, product pricing, market size, resource availability, operability and production rates, which might cause fundamental changes to the scope and timing of the project, and hence the capital costs, are not considered. “Soft issues” which may cause changes to the environment in which the project may be constructed are also not considered in the ranges assessed, since they are outside the purview of the original capital cost estimate.

Risk Analysis is applied to the building blocks of the line items within the cost estimate, such as labour, equipment, materials, indirect costs, construction management, etc. The variances in these components, rather than simply the line item unit cost, are quantified and applied across all the line items in the estimate. For example, a range in labour rates is applied to all line items that have a labour cost component.

Range Estimating typically solicits a range for the unit or total cost of each line item in the capital cost estimate. It does not identify and document the underlying reasons for the ranges that are assigned to each cost. What would cause the high side of a unit cost range to be experienced - higher labour rates, material cost escalation, low labour productivity, etc.?

The Risk Analysis model logic usually develops relationships between schedule and cost so that the value of delays and accelerations can be determined. Because the total project is considered in a Risk Analysis the relationships of mechanical completion, start up, and ramp up milestones to production profiles, revenue and operating costs can also be used to calculate time based overall project values. This allows the trade off between different execution strategy costs and implementation milestones to be tested to determine the best options.

Range Estimating treats schedule independently from cost, and generally only considers the project time line up to mechanical completion date, or end of the capital cost expenditure profile. There is no mechanism for testing cost impacts of schedule variances, and there is no relationship between time, capital cost, and overall project value.

Risk Analysis provides a comprehensive suite of tools and outcomes that identify and quantify the reasons for the expected variances in outcomes. Influence diagrams establish relationships between issues and outcomes, and identify conditioning variables that define the different environments that may influence project results. A project specific probabilistic model is developed to manipulate base and impact data, simulating project performance under changing conditions. Distribution curves of results, “tornado diagrams”, expenditure and production profiles, and “step diagrams” can be produced for any project outcome. This allows ranking of the impacts from threat/opportunity issues. Rational development of mitigation planning for the most significant issues with quantified results, testing of different strategies, and evaluation of “what if” scenarios can then follow.

Range Estimating utilizes a simple probabilistic model, which randomly applies the assessed high/expected/low cost ranges to the estimate line items to calculate a revised total cost on each model iteration. Results are generally presented in the form of a distribution curve for capital cost, from which the probability of achieving any specified cost can be estimated. There is no other data available that would allow the reason for the shape of the distribution to be determined, or to identify the drivers for the overall range in results.

Correlation of risk issues through conditioning variables in the Risk Analysis allows the impact of any underlying causes of potential change to be applied in a logical and consistent manner. For example, a heated competitive environment would be expected to cause upward pressure on both labour and materials prices, and may also have a higher expectation of a corresponding reduction in labour productivity.

There is no specific correlation between cost ranges in Range Estimating. Each unit cost is treated as an independent variable, leading to combinations of high/low values during the random sampling process. This allows calculations of offsetting results that logically would not exist. For example, in any probabilistic iteration a high value for one unit cost can be offset by a low value for a similar unit, even if the underlying reason for the high value is an overall increase in labour rates. Further, if the underlying cause of the high value is an expectation of higher labour rates because of a heated construction environment, is it logical to assume that material costs could be lower in the same environment? The distribution of Range Estimate results can therefore be skewed and the overall range in results minimized.

The capture of data to quantify probabilities and impacts for the identified risk issues is completely flexible in Risk Analysis, with no prestructured format or predetermined values. The experts are allowed to determine the definition of the risk variable and the type of distribution that best captures the impact of the variable. For example, a five or seven point discrete distribution may capture the expert’s expectations better than a triangular distribution.

Range estimating forces the use of a partially predetermined distribution that has no flexibility. The probabilities of experiencing the high or low values for each unit cost are considered to be equal. There is no indication from the experts as to which is more or less likely to occur, so both values are assigned the same probability of occurrence in a triangular distribution, which is not necessarily true. Further, the absolute maximum and minimum values solicited are sometimes difficult to rationalize, leading the experts to inaccurate estimates.

Risk Analysis, properly facilitated, solicits the views of a broad spectrum of project stakeholders on all aspects of the project. This eliminates potential biases and produces a more independent view of the project. The objective of the analysis is not just the production of a safe “contingency”, but the development of a “better” outcome for the project which takes into account all of the risks and opportunities inherent in the project environment.

The proponents of Range Estimating generally take the view of the “Constructor” of the project, not the “Owner” of the project. The experts are usually drawn from the ranks of those that prepared or approved the estimates. This may introduce biases that may not be in the best interests of the project Owner.

A comprehensive Risk Analysis considers the project as a whole and captures the impact of all internal and external influences, even if capital cost was the only unit of measure being used to determine project success. It correlates the ranges assessed to eliminate the illogical combinations that may be included in the Range Estimate results.

The example below compares the results of a Range Estimate to a Risk Analysis on the capital cost distribution for a typical project, and shows how the Range Estimate generally understates the magnitude of the uncertainty on major projects.

A key aspect that distinguishes Risk Analysis from Range Estimating is the correlation of risk issues using conditioning variables. The relationships used are demonstrated graphically in the Influence Diagram, which defines the logic utilized in the Risk Model.

The conditioning of risk variables can significantly influence the expected project outcomes, by eliminating illogical combinations of variables in the random sampling process.

In addition to the typical distribution curves of results shown above, the Risk Analysis provides some other tools that can be used to provide more insight into the issues that may lead to differences in project outcomes. The Tornado Diagram identifies the sources of most uncertainty in project results, thereby allowing mitigation planning to capture potential upsides and avoid downsides.

The Step Diagram shows which areas of the project the experts expect to be different than the base assumptions inherent to the single point estimate, and quantifies the expected difference. This allows specific areas of the base plan and estimate to be revisited and modified if desired.

The results that can be developed in a properly structured and facilitated Risk Analysis provide the project stakeholders with a comprehensive view of their project, allowing informed and rational decision making to develop a better project. Tools are available to test the impact of decisions and outside influences. The fundamental differences between Risk Analysis and Range Estimating can be summarized in the following table.

Risk Analysis Range Estimating

Objective

/ Optimize project configuration and maximize value returned to owner. / To calculate project contingency.
Outputs / Cost, schedule and value probability distributions, contingency amount, key uncertainty drivers, mitigation plans, optimized configuration. / Cost probability distribution, contingency amount.
Project Alternatives / Strategic options identified to test uncertain project scope and execution plans. / Project scope fixed for each estimate.
Assessments / Broad groups of experts, inclusion of all stakeholders, risks inside and outside of project considered. / Project estimators provide data based on elements of estimate.
Distributions / Experts provide probability of occurrence and impacts for defined variables in flexible formats. / Fixed data distribution (triangular), with end points of distribution specified probabilities.
Correlation / Risk issues correlated using conditioning variables. / Minor correlation, depending upon tool utilized. Variables essentially treated as independent.
Schedule Integration / Cost and schedule logically integrated to calculate impacts of delays/accelerations. Includes start up and ramp up schedules and costs. / Treats schedule independently up to mechanical completion only. No logic to link cost or value with schedule.
Comprehensive / Deals with underlying risk issues such as Competing Projects, Organizational Performance. / Focused on cost elements of the current estimate, not risk issues.
Benefits / Owners and investors. / Constructors, contractors and EPCM alliances.

B. A. Ostermann, P. Eng, M.Sc.109/14/18