Evaluation of the Social Impact Bond Trailblazers in Health and Social Care Final report Alec Fraser, Stefanie Tan, Kristy Kruithof, Megan Sim, Emma Disley, Chris Giacomantonio, Mylene Lagarde and Nicholas Mays PIRU Publication 2018-23 For further details, please contact: Nicholas Mays Professor of Health Policy and Director Policy Innovation Research Unit Department of Health Services Research & Policy London School of Hygiene and Tropical Medicine 15–17 Tavistock Place London WC1H 9SH Email: [email protected] www.piru.ac.uk © Copyright 2018. Not to be reproduced without permission. Evaluation of the Social Impact Bond Trailblazers in Health and Social Care Final report Alec Fraser, Stefanie Tan, Kristy Kruithof1, Megan Sim1, Emma Disley1, Chris Giacomantonio2, Mylene Lagarde3 and Nicholas Mays Policy Innovation Research Unit (PIRU), Department of Health Services Research and Policy, London School of Hygiene and Tropical Medicine, RAND Europe, Cambridge1, formerly RAND July 2018 Europe (now Halifax Regional Police, Halifax, Nova Scotia, Canada)2, LSE Health3 Funding This report is based on independent research commissioned and funded by the NIHR Policy Research Programme via its core support for the Policy Innovation Research Unit. The views expressed in the publication are those of the authors and are not necessarily those of the NHS, the NIHR, the Department of Health and Social Care, ‘arms’ length bodies or other government departments. Acknowledgements We would like to thank all the research participants who kindly agreed to give up their time to be interviewed throughout this evaluation and also answered questions informally at times over the past three years. Thanks to the organisations that shared documents with us that greatly aided our analysis. Thanks to the PPI panel for sharing their time and reflections on our research. Thanks to Noemi Kreif for undertaking the preliminary analysis of quantitative data from two SIBs. Thanks are also due to our advisory board, Pauline Allen, Michael Spackman, and Mildred Warner, for their comments on our proposals and reports. The interpretation of all the data remains that of the research team. Evaluation of the Social Impact Bond Trailblazers in Health and Social Care – Final report Contents Abstract 1 Summary 3 1. Introduction 21 1.1 Background 21 1.2 SIB Trailblazers in Health and Social Care 23 1.3 Aims and objectives of the evaluation 25 2. Methods 27 2.1 Overview 27 2.2 Qualitative research design 27 2.3 Semi-structured interviews 30 2.4 Analysis of data from semi-structured interviews 31 2.5 SIB contract analyses 32 2.6 Quantitative comparison of outcomes between SIB-funded and other similar services provided without a SIB 34 2.7 Patient and public involvement 35 2.8 Ethical approval 37 3. Framework for analysis 38 3.1 Introduction 38 3.2 The literature on SIBs: three themes and three narratives 39 3.3 Using the literature to develop an analytical approach 43 3.4 Summary and implications 50 4. Overview of the SIB Trailblazers 51 4.1 Evolution of the commissioned Trailblazer SIBs 53 4.1.1 Manchester TFCO-A 53 4.1.2 London Thames Reach 56 4.1.3 Newcastle Ways to Wellness 58 4.1.4 Worcester Reconnections 60 4.1.5 Shared Lives 63 4.2 Analysis of SIB contracts 66 4.2.1 Introduction 66 4.2.2 Model 1: Direct Provider SIB model 66 4.2.3 Model 2: SIB with SPV model 71 4.2.4 Model 3: Social Investment Partnership (SIP) model 77 4.3 Non-commissioned Trailblazer SIBs 80 4.3.1 Cornwall 80 4.3.2 East Lancashire 80 4.3.3 Leeds 81 4.3.4 Sandwell and Birmingham 81 5. Trailblazers that were not commissioned 82 5.1 Site overview 82 5.2 Findings 83 5.3 Summary and implications 87 Evaluation of the Social Impact Bond Trailblazers in Health and Social Care – Final report 6. SIB model structure: risk allocation 90 6.1 Types of risk 90 6.2 Summary and implications 95 7. Incentives: data collection and outcome measurement, and their impacts on service delivery 96 7.1 Collection of data for outcome and output measurement 96 7.2 Approaches to measuring and attributing outcomes and cashable savings to SIBs 100 7.3 Impact on service delivery 104 7.4 Summary and implications 110 8. Management style: performance management and the role of SIB specialist organisations 111 8.1 Introduction 111 8.2 Management by the SIB specialist organisation 112 8.3 Approach to performance improvement 118 8.4 Differences between the SIB models 122 8.5 Summary and implications 122 9. Discussion 124 9.1 Aims and objectives of the evaluation of the Trailblazers 124 9.2 Summary of the research 125 9.3 The contribution of the current study to the evidence on SIBs 132 9.4 The limitations of this study 135 9.5 Implications for policy and practice 137 9.6 Implications for further research 140 9.7 Conclusion 141 References 144 Note: Appendices are presented in a separate volume. Evaluation of the Social Impact Bond Trailblazers in Health and Social Care – Final report Abstract Objectives Social Impact Bonds (SIBs) are a relatively new type of payment for performance (P4P) contract focused on outcomes (i.e. ‘payment by results’ (PbR)) in which public service commissioners partner with private for-profit or philanthropic social investors to finance interventions to tackle social problems. These services are often delivered by third sector provider organisations. SIB specialist organisations may play important coordinating roles. The investors provide the up-front finance to mount the intervention and are repaid, including an agreed premium, if specified client outcomes are achieved. Sometimes, the intention is that the premium to investors should be paid wholly or in part from cashable savings generated for the commissioner. If the outcomes are not met, the investors stand to lose all their initial investment. The overall aim of this evaluation was to assess the potential costs and benefits, for different actors involved, of the SIB Trailblazer programme in health and social care over three years from the planning stage to their early years of service provision, June 2014 – May 2017. Methods Literature review to develop a conceptual framework to guide subsequent data collection and analysis; analysis of Trailblazer plans and contracts; semi-structured interviews with national policy makers, local participants in Trailblazer SIBs (commissioners, investors, SIB specialist organisations and providers) and local participants in comparable non-SIB services. Planned quantitative comparison of SIB and non-SIB sites providing similar services to similar clients via the same providers proved impossible due to problems of finding suitable comparators, data access and data quality. Results Of the nine sites in the programme, four eventually decided not to proceed to a SIB for a variety of reasons. The five SIBs that went ahead funded a wide range of different interventions for different clients: older people who are socially isolated; people with multiple chronic health conditions; entrenched rough sleepers; adolescents in care; and people with disabilities requiring long-term supported living. Typically, the planning of the SIB services and subsequent oversight were better resourced and the services more flexibly provided than similar non-SIB services. Investment came from philanthropies and socially minded investors rather than commercial sources. Three models of SIB were identified: Direct Provider SIB; SIB with Special Purpose Vehicle (SPV); and Social Investment Partnership (an evolution of the SIB concept without payments being tied to outcomes). Each allocated financial risks differently, with providers bearing more of the financial risk in the Direct Provider model than in the others. Front-line staff were more aware of the financial incentives associated with meeting client outcomes in the Direct Provider model than in the SPV model. Likewise, providers in the Trailblazers were more outcome-focused than providers of comparable non-SIB services. Up-front financing of providers by investors tended to be provided in instalments contingent on hitting volume and/ or throughput targets. During the three-year evaluation which covered the early period of the Trailblazers, the bulk of the payments to investors came from central government and sources such as the Big Lottery rather than local commissioners in most cases. Only one of the Trailblazers reported having made any cashable savings during the evaluation period as a result of the SIB-financed interventions. Only one of the Trailblazers had set up a counterfactual outcome evaluation to use as the 1 Evaluation of the Social Impact Bond Trailblazers in Health and Social Care – Final report basis for making outcome payments to investors in the period of the study. The two SIB specialist organisations involved adopted different roles (intermediary and adviser versus investment manager) and different management styles, accordingly: the one a more ‘informal’ approach stressing cooperation between commissioners, providers and investors; and the other a more ‘formal’ style, emphasising contractual obligations and outcome delivery to meet the expectations of investors. Conclusions The SIB Trailblazers in health and social care appeared to encourage a stronger emphasis on demonstrating results than comparable non-SIB services but it is not possible to ascertain whether this was translated into better client outcomes. It was difficult to reach a clear verdict on the costs and benefits of SIBs in this field over the three years of the evaluation. 2 Evaluation of the Social Impact Bond Trailblazers in Health and Social Care – Final report Summary Social impact bonds Social Impact Bonds (SIBs) are a relatively new type of payment for performance (P4P) contract focused on outcomes (i.e. ‘payment by results’ (PbR)) for the delivery of public services, frequently through third sector organisations over a period typically between three and five years but sometimes longer.