Evidence and Lessons Learned from Impact Evaluations on Social Safety Nets
Total Page:16
File Type:pdf, Size:1020Kb
Evidence and Lessons Learned from Impact Evaluations on Social Safety Nets Evidence and Lessons Learned from Impact Evaluations on Social Safety Nets ©2011 The International Bank for Reconstruction and Development / The World Bank 1818 H Street NW Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org E-mail: [email protected] All rights reserved 1 2 3 4 5 14 13 12 11 This volume, except for the Management Comments, is a product of the staff of the Independent Evaluation Group of the World Bank Group. The findings, interpretations, and conclusions expressed in this volume do not necessarily reflect the views of the Executive Directors of the World Bank or the governments they represent. This volume does not support any general inferences beyond the scope of the evaluation, including any inferences about the World Bank Group’s past, current, or prospective overall performance. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of the World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The International Bank for Reconstruction and Development / the World Bank encourages dissemination of its work and will normally grant permission to reproduce portions of the work promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center Inc., 222 Rosewood Drive, Danvers, MA 01923, USA; telephone: 978-750-8400; fax: 978-750-4470; Internet: www.copyright.com. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: [email protected]. How to cite this report: IEG (Independent Evaluation Group). 2011. Evidence and Lessons Learned from Impact Evaluations on Social Safety Nets. Washington, DC: World Bank. Cover: Ethiopian beneficiaries of a work-opportunity program supported by the World Bank and a consortium of other donors. Photo courtesy of Jennie I. Litvack. ISBN-13: ISBN-10: World Bank InfoShop E-mail: [email protected] Telephone: 202-458-5454 Facsimile: 202-522-1500 Printed on Recycled Paper Independent Evaluation Group Strategy, Communication, and Learning (IEGCS) E-mail: [email protected] Telephone: 202-458-4497 Facsimile: 202-522-312 Evaluation Managers Vinod Thomas Director-General, Evaluation Cheryl Gray Director, Independent Evaluation Group, Public Contents Sector Mark Sundberg Manager, Independent ABBREVIATIONS .......................................................................................iii Evaluation Group, Private ACKNOWLEDGEMENTS ............................................................................v Sector EXCUTIVE SUMMARY ................................................................................vii Javier Baez CHAPTER 1: INTRODUCTION ...................................................................1 Task Manager, Independent The Role of Social Safety Nets in the Current Context ................................................. 1 Evaluation Group, Public Recent Reviews of Impact Evaluations of Social Safety Nets ...................................... 3 Sector Motivation and Objectives of this Review ...................................................................... 4 Definition of SSNs Adopted in this Review ................................................................... ............................. 5 CHAPTER 2: OVERVIEW OF IMPACT EVALUATIONS IN SOCIAL SAFETY NETS ..................................................................................................................................... 11 Methodology ................................................................................................................. ............................. 11 Description of the Evaluations Reviewed ...................................................................... ............................. 14 CHAPTER 3: IMPACT OF SOCIAL SAFETY NETS ON DEVELOPMENT OUTCOMES .......................................................................................................................... 19 Impacts on Short-Term and Intermediate Outcomes .................................................... ............................. 21 Impacts on Final Outcomes and Sustainability of Effects ............................................. ............................. 33 Indirect Effects on Other Outcomes of Interest ............................................................. ............................. 39 CHAPTER 4: LESSONS LEARNED AND KNOWLEDGE GAPS ON OTHER ASPECTS OF IMPACTS ....................................................................................................... 45 Distribution of Impacts across Beneficiaries ................................................................. ............................. 45 Factors that Influence Impacts ...................................................................................... ............................. 49 Insights into the Possible External Validity of some Program Impacts.......................... ............................. 52 Efficiency Analyses of Social Safety Nets ..................................................................... ............................. 58 Gaps in Knowledge and Impact Evaluations in Progress ............................................. ............................. 61 CHAPTER 5: WORLD BANK CONTRIBUTION TO IMPACT EVALUATION IN SOCIAL SAFETY NETS ................................................................................................... 65 World Bank Efforts to Promote Impact Evaluations ...................................................... ............................. 65 Characteristics of Bank’s Impact Evaluations in Social Safety Nets ............................. ............................. 67 CHAPTER 6: CONCLUSIONS .............................................................................................. 71 APPENDICES ........................................................................................................................ 75 REFERENCES ....................................................................................................................... 111 ENDNOTES ........................................................................................................................... 123 i CONTENTS Boxes Box 1. Definitions of Social Safety Nets ........................................................................5 Box 2. Impact Evaluation Methods ................................................................................13 Box 3. Rethinking Program Design for Improved Learning ...........................................35 Box 4. Impact of SSNs on Private Transfers: The Case of Oportunidades ...................40 Box 5. Possible Incentives of Program Design on Fertility Decisions ............................41 Box 6. Impact of SSNs on Sexual Behavior, Fertility, and Marriage: The Case of Zomba Cash Transfer Program .................................................................................................42 Tables Table 1. SSN Interventions and Their Objectives ..........................................................7 Table 2. Examples of Measured Intermediate and Final Outcome Indicators ...............17 Table 3. Summary of Impacts on Income, Consumption, and Poverty Outcomes ........23 Table 4. Summary of Intermediate Educational Inputs and Outcomes .........................26 Table 5. Summary of Impacts on Health and Nutrition Outcomes .................................29 Table 6. Summary of Impacts on Labor Supply and Economic Activities ......................31 Table 7. Summary of Impacts on Protection against Shocks ........................................33 Table 8. Summary of Impacts on the Stock of Human Capital ......................................36 Table 9. Summary of Impacts on the Stock of Physical Capital ....................................37 Table 10. Summary of Impacts on Remittances and Other Private Transfers ..............40 Table 11. Summary of Impacts on Sexual Behavior, Fertility, and Marital Status .........42 Table 12. Summary of Impacts on Other Intra-household Behavioral Responses ........43 Table 13. Summary of Spillover and General Equilibrium Effects .................................44 Table 14. Evidence on the Heterogeneity of Impacts across Interventions ...................46 Table 15. Evidence on the Heterogeneity of Impacts by Baseline Characteristics of Beneficiaries ..................................................................................................................47 Table 16. Comparison of Program Impacts between Comparable CCTs of Different Scales in Nicaragua .......................................................................................................57 Table 17. Comparison of Similar CCTs of Different Sizes across Countries .................57 Table 18. Comparison across Similar CCTs Evaluated with Different Methods ............58 Table 19. Summary of Efficiency Analyses and Results ...............................................60