Report 3: Baseline and Counterfactual Meta-Evaluation of the Impacts and Legacy of the London 2012 Olympic Games and Paralympic Games FINAL REPORT – January 2012 For: Department for Culture, Media and Sport Prepared by: Grant Thornton Ecorys Loughborough University Oxford Economics January 2012 2012 Games Meta-evaluation: Report 3 (Final Report) Contents 1 Introduction 1 2 Methodological overview 3 3 Harnessing the UK's passion for sport 8 4 Exploiting opportunities for economic growth 74 5 Promoting community engagement & participation 111 6 Driving the regeneration of East London 140 7 Summary 194 Appendices A Macroeconomic modelling 207 B Elite sport policy context 214 C Additional East London indicators 220 D East London regeneration 225 2012 Games Meta-evaluation: Report 3 (Final Report) 1 Introduction 1.1 Background The London 2012 Olympic Games and Paralympic Games ('the Games' or '2012 Games') will be one of the largest events ever hosted in the UK. A key element of London's bid for the 2012 Games was the commitment that they would result in a lasting legacy for the whole of the UK. The Department for Culture, Media and Sport (DCMS) has commissioned a consortium led by Grant Thornton, including Ecorys, Loughborough University and Oxford Economics, to undertake a comprehensive and robust 'meta-evaluation' of the additionality, outputs, results, impacts and associated benefits of the investment in the 2012 Games. The Economic and Social Research Council (ESRC) has part funded this project to recognise its contribution to advancing meta-evaluation methods. The meta-evaluation consists of four phases: • Phase 1: Inception (March 2010 – April 2011); • Phase 2: Baseline and pre-Games interim evaluation (February 2011 – Summer 2012); • Phase 3: Post-Games initial evaluation (June 2012 – Spring 2013); • Phase 4: Longer-term evaluation of the impacts and legacy of the Games. It is planned that Phase 4 of the work, looking at the longer-term impacts and legacy of the Games, will be commissioned separately at a later date and cover the impacts up to 2020. This report (Report 3) is the third in a series of the following five reports which cover phases 1 to 3 of the meta-evaluation: • Report 1: Scope, research questions and data strategy; • Report 2: Methods; • Report 3: Baseline and counterfactual; • Report 4: Interim evaluation; • Report 5: Post-Games initial evaluation. 1.2 Report 3 Report 3 of the meta-evaluation sets out the baseline and counterfactual scenarios for each of the four legacy themes. The baseline and counterfactuals is of critical importance to the meta- evaluation as it sets out the overall context in which the impact of legacy initiatives will be measured and evaluated. The report is structured around the following legacy themes as set out in the Government's legacy plans:1 • Sport: harnessing the UK's passion for sport to increase grass roots participation and competitive sport and to encourage physical activity; • Economy: exploiting the opportunities for economic growth offered by hosting the Games; 1 DCMS (December 2010). Plans for the Legacy from the 2012 Olympic and Paralympic Games. 1 2012 Games Meta-evaluation: Report 3 (Final Report) • Community Engagement: promoting community engagement and achieving participation across all groups in society through the Games; • East London regeneration: ensuring that the Olympic Park can be developed after the Games as one of the principal drivers of regeneration in East London. The legacy plans cover existing and planned activity across the six host boroughs, London and the nations and regions of the UK. Within each of the four legacy areas, the cross-cutting themes of 'Sustainability' and 'Disability' are also covered. The sustainability ambition is to use the construction and staging of the Games to inspire sustainable living, and the disability legacy aims to use the Games to change attitudes towards disabled people in society, particularly their economic contribution to society, to support opportunities for disabled people to participate in sport and physical activity and to promote greater participation in the community.2 The approach taken to developing the baseline and counterfactual is set out under each of the legacy themes. This is followed by an analysis of the baseline indicators, set out under the sub- themes defined in Reports 1 and 2. This section also sets out potential drivers of change which will enhance understanding of the no-Games scenario. The counterfactual scenarios build on high-level counterfactuals set out in Report 2, and are also set out under sub-themes. Each indicator is described in terms of the policy counterfactual and the outcome counterfactual, which are defined in the following chapter. 1.3 Report structure The rest of the report is structured as follows: • Chapter 2 provides an overview of the baseline and counterfactual methodology; • Chapters 3 to 6 set out the baseline and counterfactual scenarios for each of the four legacy themes; • Chapter 7 summarises the key baseline and counterfactual findings. The Appendices provide further analysis at the disaggregated level, for example over time, across population and socio-economic groups, nations and regions, etc. 2 Office for Disability Issues (April 2011). London 2012: A legacy for disabled people. 2 2012 Games Meta-evaluation: Report 3 (Final Report) 2 Methodological overview 2.1 Introduction This chapter summarises the approach taken to develop the baseline and define the counterfactual scenarios. The baseline sets the context in which the impact of legacy initiatives will be measured and evaluated. The counterfactual is a description of the results, outcomes or impacts which would have been expected to have occurred in the absence of the 2012 Games and/or legacy intervention. Alongside evidence from project-level evaluations and primary research, both the baseline data and counterfactual scenario will be used to assess and evaluate the additional impacts across the four legacy themes. The baseline provides a statistical and quantitative framework in which to measure the impact of the 2012 Games, and the quantitative and qualitative evidence to help inform the development and description of the counterfactual scenario. Whilst the baseline is a factual description of underlying trends, the counterfactual scenario is an 'hypothesised' assessment or prediction of what would otherwise have happened to the same indicators if the 2012 Games did not take place. The additionality or net impact of legacy interventions can be calculated by taking the actual impacts under the 2012 Games and subtracting the impacts set out in the counterfactual. 2.2 Baseline The baseline comprises of a range of economic, social, cultural and environmental indicators, developed under the research framework set out in Reports 1 and 2. The baseline also includes an assessment of the factors likely to have driven any changes, and identifies any issues or uncertainties with the data. (i) Selection of baseline indicators The baseline indicators have been selected taking into account: • The outputs, outcomes and results hypothesised in the logic models developed for each legacy theme; • The need to provide evidence to help answer the research questions; • The availability of data for each indicator, the source of data and the continuity of this going forward; • The granularity of data at the relevant spatial and group levels, particularly with regard to the need to drill down to the individual borough level across the six host boroughs; • Whether the indicator will provide a measurement of direct or indirect relevance to the legacy themes. A baseline is usually defined as a specific year and has the purpose of ensuring "that the situation before the policy can be captured".3 The baseline in the meta-evaluation is defined as 2003, which is the year the Government first committed support to London's bid. The scope of the meta- evaluation is to consider the impacts of the Games from the time when the Government first committed support to London's bid, which bidding, construction, staging and legacy impacts.4 These impacts of the 2012 Games are therefore expected to occur in any subsequent years since 2003, initially through the economic impact of the construction of the venues and 3 HM Treasury (2011). The Magenta Book, Guidance for Evaluation. 4 See Report 1 for a more detailed description of the scope of the meta-evaluation. 3 2012 Games Meta-evaluation: Report 3 (Final Report) infrastructure followed by other pre-Games programmes and activities (eg Cultural Olympiad), event related activity in 2012 and longer-term legacy and regeneration effects post-2012. However, it is not simply the case that any increase (or decrease) in the baseline indicators since 2003 is as a result of the 2012 Games. There will be a multitude of 2012 Games and other factors which will impact on the baseline indicators since then. In addition, many legacy activities have only just started in the past few years and in other areas the impacts will only be felt in 2012, or post-2012 in the case of regeneration effects. A baseline of 2003 is therefore less relevant to how the trend changes over the 2003 to 2010 period and whether the changes which occur can be attributed to the 2012 Games or to other non-Olympic factors (eg the economy, policy changes, organisational changes). The trend before 2003 may also be relevant, particularly in the case of economic data and so it may be sensible to examine a longer-time series covering the period before 2003, as this allows analysis of the change in trends pre- and post-2003 to be examined, providing more evidence to identify attribution of changes to the 2012 Games. (ii) Methodological issues A number of issues were encountered in the collection of baseline data across the legacy themes. These include: • The availability of the data across the full 2003 to 2010 period, most notably the lack of available data for the first few years' of this sample (ie 2003 to 2006).