First External Review of the OBR
Total Page:16
File Type:pdf, Size:1020Kb
External review of the Office for Budget Responsibility Kevin Page September 2014 External review of the Office for Budget Responsibility Kevin Page Presented to Parliament pursuant to Schedule 1 Section 16 of the Budget Responsibility and National Audit Act 2011 September 2014 © Crown copyright 2014 You may re-use this information (excluding logos) free of charge in any format or medium, under the terms of the Open Government Licence v.2. To view this licence visit www.nationalarchives.gov.uk/doc/open- government-licence/version/2/ or email [email protected]. uk Where third party material has been identified, permission from the respective copyright holder must be sought. This publication is available at www.gov.uk/government/publications and http://budgetresponsibility.org.uk Any enquiries regarding this publication should be sent to us at [email protected]. Print ISBN 9781474110594 Web ISBN 9781474110600 PU1690 Printed in the UK by the Williams Lea Group on behalf of the Controller of Her Majesty’s Stationery Office ID 28081411 09/14 Printed on paper containing 75% recycled fibre content minimum Contents Page Foreword 3 Executive summary 5 Chapter 1 Introduction 17 Chapter 2 Methodology 19 Chapter 3 Context 21 Chapter 4 Inputs 31 Chapter 5 Outputs: technical assessment 41 Chapter 6 Outputs: transparency assessment 53 Chapter 7 Outcomes 63 Chapter 8 Conclusions and recommendations 75 Annex A Review team biographies 79 Annex B External review terms of reference 81 Annex C Selection of IFI evaluation framework paper for OECD 83 Annex D Letter from OBR Chairman confirming access to data 95 Annex E Stakeholder survey and methodology 97 Annex F Survey of Parliamentarians and guidelines for reissuing 103 Annex G Media tracking methodology 109 Annex H Acknowledgements 111 Annex I Abbreviations and glossary 113 Annex J Bibliography 115 1 Foreword The Budget Responsibility and National Audit Act 2011 requires the Non-executive members of the United Kingdom’s (UK) Office for Budget Responsibility (OBR) to commission an external review every five years, to examine the content and quality of the reports published by the OBR. On 4 April 2014, I was officially appointed by the two Non-executive members of the OBR, Lord Burns and Dame Kate Barker, to undertake the organisation’s first independent review. Prior to the appointment by the Non-executives, Mr Robert Chote, the OBR Chairman, had approached me in the summer of 2013 to inquire as to whether I would be available (and interested) to undertake such a project. At the time, I had just completed my five-year mandate as Canada’s first Parliamentary Budget Officer and had begun a new assignment at the University of Ottawa (Faculty of Social Sciences), as the Jean-Luc Pepin Research Chair (JLP). I was principally interested in the project because, from afar, I had been aware of both the quality and perceived impact of the work of the OBR. Independent fiscal institutions (IFI) like the OBR or Canada’s Parliamentary Budget Office (PBO) tend to have relatively small budgets and, contrastingly, significant mandates. So, as practitioners, we tend to look abroad to see what our peers might be doing. We look for references, for instance in the provision of fiscal forecasts or sustainability analysis and the resulting impact that these reports might have on political discourse, transparency and accountability. At about the same time as I received the appointment for the OBR review, I had also been tasked by the Organisation for Economic Co-operation and Development (OECD) to develop a framework for the evaluation of IFIs, whose numbers had grown rapidly in the wake of the 2008 global financial crisis. Within this new community of practice, some of the IFIs, like that in Canada, had experienced trials and tribulations towards political acceptance even though the gains in fiscal transparency for legislatures and other stakeholders had been significant. The remit of the OBR is clear and includes as its main duty ‘to examine and report on the sustainability of the public finances’ and the responsibility to carry out that work ‘objectively, transparently and impartially’. The terms of reference for the external review are equally clear and include the examination of the quality of the OBR’s main publications, the effectiveness of its methodologies and governance, and the capability of the OBR team. It is my view that the UK Parliament should be commended for including the provision of a periodic external review in the OBR’s founding legislation. Periodic external reviews should be considered a best practice to be emulated by other IFIs. The approach taken in the evaluation framework, which includes four distinct elements (context, inputs, outputs and outcomes), follows a performance framework approach used by governments around the world. It starts with context – in this case, the institutional structure and objectives of the organisation. The OBR has a very distinct structure given its dual accountabilities to both the executive branch and legislative branch (i.e. Parliament). Capability, methodology, governance (i.e. inputs) and products (i.e. outputs) are assessed against benchmarks in other OECD countries and, where possible, against observable counterfactuals. Two separate detailed surveys were conducted for Parliamentarians and stakeholders to assess the impact and quality of OBR work from an outcome perspective. This work was supplemented by expert interviews. The interview list and the survey questions are provided in the annexes to this report. 3 I also wish to attest that the evaluation effort was undertaken both with complete independence and with the full cooperation and support of the entire OBR team and the Non-executives. The interactions included in-person interviews, the exchange of documents as well as other data and information. An embargoed draft copy of the evaluation was shared with the two Non-executive members on 20 August 2014, the Budget Responsibility Committee on 26 August 2014 and senior civil servants in HM Treasury on 27 August 2014. The purpose of sharing the embargoed document was to enable these parties to prepare their respective official responses to the published report, if they deemed it necessary. The evaluation team included my two colleagues from the University of Ottawa, Sahir Khan and Helaina Gaspard, as well as a former colleague from Canada’s Parliamentary Budget Office, Randall Bartlett. Mr Khan is a former Assistant Parliamentary Budget Officer in Canada and Dr Gaspard is a senior research associate with JLP Chair. The team also included two seconded (but independent) civil servants from Her Majesty’s Treasury, Victoria Jones and Naomi Adams. Their work was invaluable and the result of their tireless effort and dedication to the project. Kevin Page Jean-Luc Pepin Research Chair University of Ottawa Canada 4 Executive summary The Office for Budget Responsibility (OBR) was established in 2010 ‘to address past weaknesses in the credibility of economic and fiscal forecasting and, consequently, fiscal policy’.1 Its legislative foundations lay in the Budget Responsibility and National Audit Act (2011) (the Act) and are operationalised through the Charter for Budget Responsibility (2011) (the Charter). Accountable to the Chancellor of the Exchequer (Chancellor) and to Parliament (namely through the Treasury Select Committee), the OBR has five principle responsibilities arising from the Act and the Charter: 1 the production of economic and fiscal forecasts; 2 an assessment of the extent to which the government’s fiscal targets have been, or are likely to be, achieved; 3 an assessment of the accuracy of its previous fiscal and economic forecasts; 4 an assessment of the long-term sustainability of the UK public finances; and 5 scrutiny and certification of the government’s policy costings (the OBR itself does not produce costings). To fulfil its mandate, the OBR produces reports and related documents, supports the Chancellor and his/her department in their management of the public finances and responds to stakeholder questions, including those of Parliamentarians. The OBR is required to lay its legally mandated reports before Parliament and to provide testimony and evidence to Parliamentary committees related to budgetary and fiscal matters. The terms of reference for this review require that the quality of the OBR’s publications, the effectiveness of its methodology and governance, and the capability of its staff to fulfil its mandate be evaluated, pursuant to its legislation. For this first evaluation, the OBR was assessed using a framework generally recognised among governments, via a methodology being developed by the University of Ottawa’s Jean-Luc Pepin Research Chair, with the supporting sponsorship of the Organisation for Economic Co-operation and Development (OECD). This IFI evaluation framework is comprised of four categories – context, inputs, outputs and outcomes – defined below. 1 Context: The unique political and institutional environment housing the independent fiscal institution (IFI) as well as its legally defined mandate. For IFIs there are both domestic and international contexts for their operations. 2 Inputs: Quantification and quality assessment of the IFI’s resources (both financial and human). It is necessary to consider organisational inputs as a factor for comparison and possible influencers of institutional capacity, productivity and relevance. 3 Outputs: The quality of products developed and released by the IFI based on the functions defined in its mandate (i.e. economic and fiscal forecasting, fiscal sustainability, costing, budgetary advisory). 1 HM Treasury (2014) Charter for Budget Responsibility, ch.2.2. 5 4 Outcomes: Stakeholders’ perceived confidence in the IFI’s work and their use of its products. This evaluation is based on: data collected from the OBR, government departments and agencies, and international organisations; expert stakeholder interviews; a stakeholder survey; media tracking analysis; and an evaluation of the OBR’s products. Context The OBR’s domestic context is composed of its legislative mandate and its political and institutional environments.