A REVIEW OF THE ATTRIBUTES OF TRANSFORMATIONAL CHANGE IN THE ENERGY AND PUBLIC HEALTH SECTORS APPROACH PAPER July 2020 GREEN CLIMATE FUND INDEPENDENT EVALUATION UNIT A review of the attributes of transformational change in the energy and public health sectors APPROACH PAPER 07/2020 ©IEU | i CONTENTS LIST OF AUTHORS ................................................................................................................... IV ACKNOWLEDGEMENTS ........................................................................................................... IV ABBREVIATIONS .................................................................................................................... V A. Background ................................................................................................................................... 1 1. Description of the problem ...................................................................................................................1 a. The need for transformational change in climate change mitigation and adaptation ..................1 b. Searching for evidence on drivers of transformational change in the energy and public health sectors ..........................................................................................................................................2 c. Drivers of climate change ............................................................................................................2 2. Categorization of interventions and outcomes .....................................................................................3 a. Public Health ...............................................................................................................................4 b. Energy ..........................................................................................................................................6 3. Why is it important to do this review?..................................................................................................7 B. Objective of the review ................................................................................................................. 8 C. Methods ......................................................................................................................................... 9 1. Criteria for inclusion and exclusion of studies in the systematic review .............................................9 a. Population ....................................................................................................................................9 b. Interventions ..............................................................................................................................10 c. Comparison ................................................................................................................................10 d. Outcomes ...................................................................................................................................10 e. Study design ..............................................................................................................................10 f. Exclusion criteria .......................................................................................................................11 2. Search method for identification of studies ........................................................................................11 a. Search steps ...............................................................................................................................11 b. Databases, repositories and individual journals ........................................................................12 c. Search terms ..............................................................................................................................13 d. Combinations of search terms ...................................................................................................13 3. Data collection and Analysis for the Systematic Review ...................................................................14 a. Screening of studies ...................................................................................................................14 b. Data extraction and management ..............................................................................................15 c. Analysis .....................................................................................................................................15 d. Evidence and Gap Maps (EGMs) ..............................................................................................15 e. Methods for handling dependent effect sizes ............................................................................16 f. Data synthesis ............................................................................................................................16 g. Assessment of heterogeneity and contributors to transformational change ..............................17 h. Discussion of cross-sector learnings..........................................................................................17 D. Distribution of main roles ........................................................................................................... 19 References ........................................................................................................................................... 20 APPENDICES ................................................................................................................ 23 Appendix 1. Theory of Change (ToC) for each sector ....................................................................... 24 Appendix 2. EGM frameworks ........................................................................................................... 26 ii | ©IEU Appendix 3. Inclusion and Exclusion criteria ..................................................................................... 28 Appendix 4. Illustrative data extraction form draft ............................................................................ 37 Appendix 5. Search terms for each sector .......................................................................................... 43 Appendix 6. Draft search strategy example (sector public health, Web of Science).......................... 54 Appendix 7. Benchmark studies in the energy sector ......................................................................... 59 Appendix 8. Benchmark studies in the public health sector ............................................................... 61 Appendix 9. Findings from the Pilot Screening in the Energy Sector ................................................ 63 ©IEU | iii LIST OF AUTHORS The authors of the review of the attributes of transformational change in the energy and public health sectors are (in alphabetical order by last name): FULL NAME AFFILIATION Zharas Aitmambet Center for Evaluation and Development, Mannheim, Germany Joseph Dickman Climate Investment Funds, Washington DC, USA Markus Frölich Center for Evaluation and Development, Mannheim, Germany Atika Pasha Center for Evaluation and Development, Mannheim, Germany Martin Prowse Independent Evaluation Unit, Green Climate Fund, Songdo, South Korea Jyotsna Puri Independent Evaluation Unit, Green Climate Fund, Songdo, South Korea Neha Sharma Center for Evaluation and Development, Mannheim, Germany Arne R. Weiss Center for Evaluation and Development, Mannheim, Germany ACKNOWLEDGEMENTS The authors, who are listed alphabetically, are grateful for the comments received on this approach paper from Howard White, Ashrita Saran, Eirini Pitta, Emma De Roy, Frank Renkewitz, John Eyers, Sarah Khan and Ekwaro A. Obuku. Any errors or inconsistencies are entirely our responsibility. iv | ©IEU ABBREVIATIONS 3IE International Initiative for Impact Evaluation ATT Average treatment effect on treated C4ED Center for Evaluation and Development CAIT Climate Access Indicators Tool CIF Climate Investment Fund CO2 Carbon dioxide COP21 Conference of the Parties DID Difference-in-differences approach EGM Evidence gap map EPPI Centre Evidence for Policy and Practice Information and Co-ordination Center F-gases Fluorinated gases GCF Green Climate Fund GHG Greenhouse gas GRADE Grading of Recommendation, Assessment, Development and Evaluation HIV Human immunodeficiency viruses ICTs Internet and information and communications technologies IPCC Intergovernmental Panel on Climate Change IV Instrumental variable LATE Local average treatment effect LMICs Low- and middle-income countries NAPA National Adaptation Program of Action ORS Oral rehydration solution PICOS model Population, intervention, comparison, outcome and study design model PRISMA Preferred Reporting Items for Systematic Reviews and Meta-Analyses PSM Propensity score matching RCT Randomized controlled trial RDD Regression discontinuity design SURE ranking Supporting the Use of Research Evidence ToC Theory of Change TWh Terawatt hour UN United Nations UNFCCC United Nations Framework Convention on Climate Change WASH Water, sanitation and hygiene WHO World Health Organization WRI World Recourses Institute ©IEU | v A REVIEW OF THE ATTRIBUTES OF TRANSFORMATIONAL CHANGE IN THE ENERGY AND PUBLIC HEALTH SECTORS APPROACH PAPER A. BACKGROUND 1. DESCRIPTION OF THE PROBLEM a. The need for transformational change in climate change mitigation and adaptation The Intergovernmental Panel on Climate Change (IPCC) estimates that if greenhouse