Terminal Evaluation of the Adaptation Fund / UNEP Project “Promoting
Total Page:16
File Type:pdf, Size:1020Kb
Terminal Evaluation of the Adaptation Fund / UNEP Project “Promoting Climate Resilience in the Rice Sector through Pilot Investments in Alaotra-Mangoro Region” (AFB-5060-1111-2G49) Evaluation Office of the United Nations Environment Programme Distributed: September 2020 Terminal Evaluation of the Project : Promoting Climate Resilience in the Rice Sector through Pilot Investments in Alaotra-Mangoro Region Photos Credits: Front cover: Pierre Bégat Annex XI: Pierre Bégat (all pictures) ©UNEP/ @UNEP/ (Pierre Bégat), United Nations Environment Programme, Evaluation Mission (2020) This report has been prepared by independent consultant evaluators and is a product of the Evaluation Office of UNEP. The findings and conclusions expressed herein do not necessarily reflect the views of Member States or the UN Environment Programme Senior Management. For further information on this report, please contact: Evaluation Office of UNEP P. O. Box 30552-00100 GPO Nairobi Kenya Tel: (254-20) 762 3389 Email: [email protected] Website: https://www.unenvironment.org/about-un-environment/evaluation Promoting Climate Resilience in the Rice Sector through Pilot Investments in Alaotra- Mangoro Region Project AFB-5060-1111-2G49 September 2020 All rights reserved. © (2020) UN Environment Programme Page 2 Terminal Evaluation of the Project : Promoting Climate Resilience in the Rice Sector through Pilot Investments in Alaotra-Mangoro Region ACKNOWLEDGEMENTS This Terminal Evaluation was prepared for UNEP by Pierre Bégat, as an independent consultant. The evaluator would like to express their gratitude to all persons met and who contributed to this evaluation, as listed in Annex IV. The evaluation team would like to thank the project team and in particular Ms. Jane Razanamiharisoa and Ms. Anna Kontorov for their contribution and collaboration throughout the evaluation process. Sincere appreciation is also expressed to the Project Steering Committee who took time to provide comments to the draft report. The Evaluation Office of UNEP would also like to thank the Ministry of Environment and Sustainable Development of Madagascar. Pierre Bégat would like to express his gratitude to the project team, especially Jane Razanamiharisoa, Todisoa Manankasina and Eric for facilitating the in-country mission; Lucille Palazy and Anna Kontorov for sharing their insights; and Pauline Marima and Mercy Mwangi from the UNEP Evaluation Office for steering the evaluation process. The evaluation consultant hopes that the findings, conclusions and recommendations will contribute to the successful finalisation of the current project, formulation of a next phase and to the continuous improvement of similar projects in other countries and regions. BRIEF CONSULTANT BIOGRAPHY Pierre Bégat International consultant in climate change adaptation and climate finance. He holds a MSc. in Environmental and Energy Economics, MSc. in Economic Analysis and Policy (APE), and a BSc. in Economics & Econometrics. Evaluation team Pierre Bégat – Principal Evaluator Evaluation Office of UNEP Pauline Marima– Evaluation Manager Mercy Mwangi – Evaluation Programme Assistant Page 3 Terminal Evaluation of the Project : Promoting Climate Resilience in the Rice Sector through Pilot Investments in Alaotra-Mangoro Region ABOUT THE EVALUATION Joint Evaluation: No Report Language: English (Executive Summary in English and French) Evaluation Type: Terminal Evaluation Brief Description: a UNEP-Adaptation Fund project implemented between 2012 and 2019.The project's overall development goal was to demonstrate pathways towards the transformation of the rice sub-sector to make it more resilient to current climate variability as well as expected climate change and associated hazards, through implementation of pilot investments in the Alaotra-Mangoro region that have the potential of being upscaled at national level. The evaluation sought to assess project performance (in terms of relevance, effectiveness and efficiency), and determine outcomes and impacts (actual and potential) stemming from the project, including their sustainability. The evaluation has two primary purposes: (i) to provide evidence of results to meet accountability requirements, and (ii) to promote learning, feedback, and knowledge sharing through results and lessons learned among UNEP, the Adaptation Fund and their executing partner Ministry of Environment and Sustainable Development and the relevant agencies in Madagascar. Key words: Ecosystem-based Adaptation; Agriculture; Resilience; Rice; Water Management; Reforestation; Storage; Project Evaluation; Climate Change; Climate Change Adaptation; Ecosystem Management; TE; Terminal Evaluation; Adaptation Fund; Adaptation Fund Project; Madagascar; Alaotra-Mangoro Primary data collection period: January 2020 – April 2020 Field mission dates: February 2020 Page 4 Terminal Evaluation of the Project : Promoting Climate Resilience in the Rice Sector through Pilot Investments in Alaotra-Mangoro Region Table of Contents ACKNOWLEDGEMENTS ........................................................................................................................................ 3 BRIEF CONSULTANT BIOGRAPHY ............................................................................................................................... 3 ABOUT THE EVALUATION .................................................................................................................................... 4 1 INTRODUCTION ................................................................................................................................................ 21 2 EVALUATION METHODS ................................................................................................................................... 23 2.1 INCEPTION REPORT .......................................................................................................................................... 23 2.2 DOCUMENT REVIEW ........................................................................................................................................ 23 2.3 INTERVIEWS AND FIELD MISSION ......................................................................................................................... 23 2.4 ANALYSIS AND REPORTING ................................................................................................................................ 24 2.5 LIMITATIONS TO THE EVALUATION ...................................................................................................................... 24 2.6 ETHICS AND HUMAN RIGHTS .............................................................................................................................. 25 3 THE PROJECT .................................................................................................................................................... 26 3.1 CONTEXT ....................................................................................................................................................... 26 3.2 OBJECTIVES AND COMPONENTS.......................................................................................................................... 27 3.3 STAKEHOLDERS ............................................................................................................................................... 28 3.4 PROJECT IMPLEMENTATION STRUCTURE AND PARTNERS .......................................................................................... 30 3.5 CHANGES IN DESIGN DURING IMPLEMENTATION .................................................................................................... 32 3.6 PROJECT FINANCING ........................................................................................................................................ 43 4 THEORY OF CHANGE AT EVALUATION .............................................................................................................. 44 4.1 CAUSAL PATHWAYS FROM OUTPUTS TO OUTCOMES ............................................................................................... 44 4.2 CAUSAL PATHWAYS FROM OUTCOMES TO INTERMEDIATE STATES .............................................................................. 45 4.3 CAUSAL PATHWAYS FROM INTERMEDIATE STATES TO IMPACTS.................................................................................. 47 5 EVALUATION FINDINGS .................................................................................................................................... 55 5.1 STRATEGIC RELEVANCE ..................................................................................................................................... 55 5.2 QUALITY OF PROJECT DESIGN ............................................................................................................................. 57 5.3 NATURE OF THE EXTERNAL CONTEXT ................................................................................................................... 58 5.4 EFFECTIVENESS ............................................................................................................................................... 59 5.4.1 Delivery of outputs ................................................................................................................................. 59 5.4.2 Achievement of direct outcomes ........................................................................................................... 81 5.4.3 Likelihood of impact