This is a repository copy of Five things you should know about cost overrun. White Rose Research Online URL for this paper: http://eprints.whiterose.ac.uk/139423/ Version: Accepted Version Article: Flyvbjerg, B., Ansar, A., Budzier, A. et al. (11 more authors) (2018) Five things you should know about cost overrun. Transportation Research Part A: Policy and Practice, 118. pp. 174-190. ISSN 0965-8564 https://doi.org/10.1016/j.tra.2018.07.013 Article available under the terms of the CC-BY-NC-ND licence (https://creativecommons.org/licenses/by-nc-nd/4.0/). Reuse This article is distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs (CC BY-NC-ND) licence. This licence only allows you to download this work and share it with others as long as you credit the authors, but you can’t change the article in any way or use it commercially. More information and the full terms of the licence here: https://creativecommons.org/licenses/ Takedown If you consider content in White Rose Research Online to be in breach of UK law, please notify us by emailing [email protected] including the URL of the record and the reason for the withdrawal request. [email protected] https://eprints.whiterose.ac.uk/ Five Things You Should Know about Cost Overrun Draft 3.3, submitted to Transportation Research Part A: Policy and Practice Authors:1 Bent Flyvbjerg,2 Atif Ansar,3 Alexander Budzier,4 Søren Buhl,5 Chantal Cantarelli,6 Massimo Garbuio,7 Carsten Glenting,8 Mette Skamris Holm,9 Dan Lovallo,10 Daniel Lunn,11 Eric Molin,12 Arne Rønnest,13 Allison Stewart,14 Bert van Wee15 Abstract: This paper gives an overview of good and bad practice for understanding and curbing cost overrun in large capital investment projects, with a critique of Love and Ahiaga-Dagbui (2018) as point of departure. Good practice entails: (a) Consistent definition and measurement of overrun; in contrast to mixing inconsistent baselines, price levels, etc. (b) Data collection that includes all valid and reliable data; as opposed to including idiosyncratically sampled data, data with removed outliers, non-valid data from consultancies, etc. (c) Recognition that cost overrun is systemically fat-tailed; in contrast to understanding overrun in terms of error and randomness. (d) Acknowledgment that the root cause of cost overrun is behavioral bias; in contrast to explanations in terms of scope changes, complexity, etc. (e) De-biasing cost estimates with reference class forecasting or similar methods based in behavioral science; as opposed to conventional methods of estimation, with their century-long track record of inaccuracy and systemic bias. Bad practice is characterized by violating at least one of these five points. Love and Ahiaga-Dagbui violate all five. In so doing, they produce an exceptionally useful and comprehensive catalog of the many pitfalls that exist, and must be avoided, for properly understanding and curbing cost overrun. Five Key Questions about Cost Overrun Cost overrun in large capital investment projects can be hugely damaging, incurring outsize losses on investors and tax payers, compromising chief executives and their organizations, and even leading to bankruptcy (Flyvbjerg et al. 2009, Flyvbjerg and Budzier 2011). Accordingly, cost overrun receives substantial attention in both the professional literature and popular media. Yet it is not always clear how cost overrun is defined, why it happens, and how to best 1 All authors have co-authored or authored publications based on the data, theories, and methods commented on by Love and Ahiaga-Dagbui (2018). 2 BT Professor and Chair of Major Programme Management at University of Oxford's Saïd Business School, corresponding author, [email protected]. 3 Director of the MSc in Major Programme Management at University of Oxford's Saïd Business School. 4 Fellow in Management Practice at University of Oxford's Saïd Business School. 5 Associate Professor of Mathematical Statistics Emeritus at Aalborg University's Department of Mathematical Sciences. 6 Lecturer in Operations Management at the Sheffield University Management School. 7 Senior Lecturer in Entrepreneurship at the University of Sydney Business School. 8 Head of Strategy, Economics, and Finance at Viegand Maagøe A/S, Copenhagen. 9 Head of Section at Aalborg Municipality, Denmark. 10 Professor of Business Strategy at the University of Sydney Business School. 11 Professor of Statistics at University of Oxford's Department of Statistics. 12 Associate Professor in Transport Policy and Travel Behavior at Delft University of Technology's Faculty for Technology, Policy, and Management. 13 CEO at Esrum Kloster and Møllegård, Græsted, Denmark. 14 Project Director at Infrastructure Victoria, Melbourne, Australia; former Associate Fellow at University of Oxford's Saïd Business School. 15 Professor in Transport Policy at Delft University of Technology's Faculty for Technology, Policy, and Management. 2 avoid it, which has led to misperceptions about the concept with policy makers, planners, investors, academics, and the public. To help remedy this situation, below we address five fundamental questions about cost overrun in large capital investment projects: 1. What is cost overrun, and how is it measured? 2. Which data are used to establish cost overrun? 3. What is the size and frequency of cost overrun? 4. What are the root causes of cost overrun? 5. How is cost overrun best avoided? If your job is to research, plan, finance, or deliver large capital projects, you need to have good answers to these questions. Here, we answer the questions in a response to Love and Ahiaga- Dagbui (2018), invited by the editors.16 We appreciate this opportunity to clarify what good and bad practice is in understanding and curbing cost overrun, and the many pitfalls that exist for good practice, eminently exemplified by Love and Ahiaga-Dagbui. We are delighted that Love and Ahiaga-Dagbui acknowledge that our work on cost underestimation, "Undeniably ... has made an impact ... [and] brought to attention issues that ey say (p. YY).17 To understand and deal with the "elephant in the room" deliberate and non- deliberatewere possibly cost being underestimation overlooked in The large elephant capital in investment the room projects has been has recognized been a core as purposeth of our work. For Love and Ahiaga-Dagbui to recognize that we have succeeded is gratifying, and we thank them for their acknowledgement. Love and Ahiaga-Dagbui, however, are critical of our work. We welcome their objections, as criticism is the main mechanism for securing high levels of validity and reliability in scholarship. But we are surprised by the language used by Love and Ahiaga-Dagbui in communicating their commentary. For instance, they describe our research findings as "fake news", "myths" (no less than 15 times), "canards", "factoids", "flagrant", "rhetoric", "misinformation", and more (p. YY). We are further accused of having "fooled many people" by having "been just as crafty as Machiavelli" as we "have feigned and dissembled information" through our research (p. YY). As a factual observation, in our entire careers we have never come across language in an academic journal like that used by Love and Ahiaga-Dagbui. We suggest such language has no place in academic discourse. In what follows, we address Love and Ahiaga-Dagbui's critique by relating it to each of the five key questions about cost overrun listed above. 1. What Is Cost Overrun, and How Is It Measured? Cost overrun is the amount by which actual cost exceeds estimated cost, with cost measured in the local currency, constant prices, and against a consistent baseline. Overrun is typically measured in percent of estimated cost, with a positive value indicating cost overrun and a negative value underrun. Size, frequency, and distribution of cost overrun should all be measured as part of measuring cost overrun for a certain investment type. Cost overrun is the difference between actual and estimated capital costs for an investment. The difference may be measured in absolute or relative terms. In absolute terms cost overrun is 16 ///Note to the editor and copy editor: The version of Love and Ahiaga-Dagbui (2018) we have read contains many errors and oddities of language. We have marked these with "[sic]." To the extent the errors and oddities are eliminated before publication the "sic"s should be eliminated, too./// 17 ///Note to the editor and copy editor: Kindly insert correct page numbers instead of "YY" for all quotes from Love and Ahiaga-Dagbui's paper, once the final (published) page numbers are known./// 3 measured as actual minus estimated cost.18 In relative terms overrun is measured as either (a) actual cost in percent of estimated cost, or (b) the ratio of actual divided by estimated cost. In our studies, we measure cost overrun in relative terms, because this makes for accurate comparison across investments, geographies, and time periods. It also makes for accuracy in forecasts of cost risk. Estimated cost may be, and typically is, established at different time points or baselines in the investment and delivery cycle, e.g., at the outline business case, final business case, and contracting. The cost estimate will normally be different at different time points, and it typically becomes more accurate the closer to final delivery the investment is, although there may be large variations in this, for instance where bad news about cost overruns are hidden as long as possible and cost estimates therefore suddenly explode when the project is well into delivery, when the overruns can no longer be kept secret, which is not an uncommon occurrence for large capital investment projects. The baseline one chooses for measuring cost overrun depends on what one wants to understand and measure. We want to understand whether decision makers make well-informed decisions.