The Qualitative Impact Protocol (Quip)
Total Page:16
File Type:pdf, Size:1020Kb
ATTRIBUTING DEVELOPMENT IMPACT DEVELOPMENT IMPACT ATTRIBUTING ATTRIBUTING DEVELOPMENT IMPACT Substantiating cause and effect is one of the great conundrums for those aiming to have a social impact, be they an NGO, social impact investment fund, or multinational corporation. All face the same quandary: how do you know whether, or how, you contributed to an observed social change? A wide range of impact ATTRIBUTING evaluation methodologies exist to address this need, ranging from informal feedback loops to highly elaborate surveys. But generating useful and credible information in a timely and cost-effective way remains an elusive goal, particularly for organizations working in complex, rapidly evolving and diverse contexts. Attributing Development Impact brings together responses to this challenge using DEVELOPMENT an innovative impact evaluation approach called the Qualitative Impact Protocol (QuIP). This is a transparent, flexible and relatively simple set of guidelines for collecting, analysing and sharing feedback from intended beneficiaries about significant drivers of change in their lives. Innovative features include the use of ‘blindfolded’ interviewing to mitigate pro-project bias, and the application of a flexible coding system to make analysis and reporting faster and more transparent. IMPACT THE THE QUALITATIVE IMPACT PROTOCOL CASE BOOK PROTOCOL IMPACT THE QUALITATIVE QUALITATIVE The QuIP has now been used in many countries, and this book uses case studies from seven countries (Ethiopia, India, Malawi, Mexico, Tanzania, Uganda and UK) IMPACT assessing a range of activities, including food security, rural livelihoods, factory PROTOCOL working conditions, medical training, community empowerment and microcredit CASE BOOK for house improvement. It includes comprehensive ‘how to’ QuIP guidelines and practical insights based on these case studies into how to address the numerous methodological challenges thrown up by impact evaluation. Essential reading for evaluation specialists within NGOs, governments and donor agencies; social impact investors; community development practitioners; and researchers and students interested in evaluation methodologies. James Copestake is Professor of International Development at the University of Bath, and has thirty years’ experience at the interface between development research, policy and practice. Marlies Morsink worked on the book whilst a Research Officer at the University of Bath, and has since joined Bath Social & Development Research as a QuIP Project Manager. Fiona Remnant helped to develop the QuIP and has spearheaded the creation of Bath Social & Development Research. ‘ QuIP offers a simple, transparent method to deliver timely, cost-effective and credible causal attributions.’ Nancy Cartwright, University of California San Diego and Durham University, UK James Copestake, Marlies Morsink & Fiona Remnant Copyright practicalactionpublishing.com Attributing Development Impact Copyright practicalactionpublishing.com Praise for this book ‘QuIP is well geared to do what it promises: it offers a simple, transparent method to deliver timely, cost-effective and credible causal attributions. And it is well grounded. The theory, history and case studies in this book show why we can trust that it can do what it says. QuIP is a really welcome contribution to methodology for causal inference.’ Nancy Cartwright, University of California San Diego and Durham University, UK ‘The assessment of complex interventions is defined by the need to make difficult trade-offs: time, money, talent and support always seem inadequate. But such pressures only intensify the need for good theory, breadth of experience, depth of commitment to professional standards, and giving stakeholders a reasoned basis on which to act. The strategies and cases outlined in this insightful book demonstrate how this can be realized in practice. The Qualitative Impact Protocol enables applied social science to do its job: to faithfully uphold accountability norms while generating sound and usable conclusions.’ Michael Woolcock, World Bank and Harvard University ‘An enormously important addition to impact evaluation approaches, with detailed examples and explanation. This book offers practical and theoretically informed guidance on how to bridge the increasing mismatch between the complexity of interventions (and the contexts in which they operate) and the counterfactual impact evaluation methods that are often advocated.’ Patricia Rogers, Director, Better Evaluation Copyright practicalactionpublishing.com Attributing Development Impact The Qualitative Impact Protocol (QuIP) Case Book James Copestake, Marlies Morsink and Fiona Remnant Copyright practicalactionpublishing.com Practical Action Publishing Ltd 27a Albert Street, Rugby, Warwickshire, CV21 2SG, UK www.practicalactionpublishing.org © James Copestake, Marlies Morsink and Fiona Remnant and the contributors, 2019 The moral right of the editors to be identified as editors of the work and the contributors to be identified as contributors of this work have been asserted under sections 77 and 78 of the Copyright Designs and Patents Act 1988. The PDF version of this book is distributed under a Creative Commons Attribution Non-commercial No-derivatives CC BY-NC-ND license. This allows the reader to copy and redistribute the material; but appropriate credit must be given, the material must not be used for commercial purposes, and if the material is transformed or built upon the modified material may not be distributed. For further information see https://creativecommons.org/licenses/by-nc-nd/4.0/legalcode Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. A catalogue record for this book is available from the British Library.A catalogue record for this book has been requested from the Library of Congress. ISBN 978-1-78853-024-8 Paperback ISBN 978-1-78853-023-1 Hardback ISBN 978-1-78044-747-6 eBook ISBN 978-1-78044-746-9 Library Pdf Citation: Copestake, J., Morsink, M., Remnant, F. (ed.) (2019) Attributing Development Impact: the Qualitative Impact Protocol case book, Rugby, UK, Practical Action Publishing, <http://dx.doi.org/10.3362/9781780447469> Since 1974, Practical Action Publishing has published and disseminated books and information in support of international development work throughout the world. Practical Action Publishing is a trading name of Practical Action Publishing Ltd (Company Reg. No. 1159018), the wholly owned publishing company of Practical Action. Practical Action Publishing trades only in support of its parent charity objectives and any profits are covenanted back to Practical Action (Charity Reg. No. 247257, Group VAT Registration No. 880 9924 76). The views and opinions in this publication are those of the author and do not represent those of Practical Action Publishing Ltd or its parent charity Practical Action. Reasonable efforts have been made to publish reliable data and information, but the authors and publisher cannot assume responsibility for the validity of all materials or for the consequences of their use. Cover photo shows a market scene in Tigray, Ethiopia. Credit: James Copestake Cover design: RCO.design Printed in the United Kingdom Typeset by vPrompt eServices Copyright practicalactionpublishing.com Contents List of figures, tables, and boxes ix Acknowledgements xii Foreword xiii 1. Introducing the causal attribution challenge and the QuIP 1 Introduction 1 How the book is organized and how to use it 5 An overview of the QuIP 6 The backstory of the QuIP and this book 18 References 24 2. Comparing the QuIP with other approaches to development impact evaluation 29 Introduction 29 Defining the field of impact evaluation 30 Comparing the QuIP with other approaches to impact evaluation 33 Choosing between approaches to impact evaluation 42 Conclusions 46 Appendix: comparing the QuIP with 30 other approaches to impact evaluation 47 References 54 3. A deep dive into Diageo’s malt barley supply chain in Ethiopia 59 Introduction 59 The study 61 Findings 62 Sample selection 68 Political economy and public policy context 70 Conclusions 71 References 72 4. Improving working conditions in the Mexican garment industry 75 Introduction: commissioner and project background 75 The 2016 external evaluation 78 Selected findings 81 Copyright practicalactionpublishing.com http://dx.doi.org/10.3362/9781780447469.000 vi ATTRIBUTING DEVELOPMENT IMpact Interpreting the findings 82 Conclusions 89 References 91 5. Exploring the social impact of housing microfinance in South India 95 Introduction 95 The India context and a profile of the selected MFIs 97 The QuIP evaluation study 102 Illustrative findings 104 Discussion 111 References 114 6. Faith-based rural poverty reduction in Uganda 117 Introduction 117 The theory and practice of Church and Community Mobilisation (CCM) 119 The QuIP study in Uganda 121 Illustrative findings 127 Discussion and conclusions 135 References 138 7. Harnessing agriculture for better nutritional outcomes in southern Tanzania 141 Introduction 141 The QuIP study 147 Findings 150 Methodological reflections and conclusions 157 References 164 8. Placing volunteer educators: the Global Health Service Partnership in Uganda, Tanzania, and Malawi 167 Introduction 167 The Global Health Service Partnership 168 The QuIP study 169 Illustrative findings 173 Reflections 181 Appendix: questionnaire outline for student interviews and focus group discussion