United Nations Environment Program Terminal Evaluation of the Project “Communities for Conservation: Safeguarding the World’s Most Threatened Species (Andes Region)” Charles B. Kenny Jordan Evaluation Office November 2014 Contents Abbreviations and Acronyms ............................................................................................................... iii Executive summary ................................................................................................................................. 1 I. Introduction ..................................................................................................................................... 6 II. The Evaluation ................................................................................................................................ 6 III. The Project ...................................................................................................................................... 7 A. Context ........................................................................................................................................ 7 B. Project Objective and Components ............................................................................................. 8 C. Target areas/groups ..................................................................................................................... 9 D. Milestones/key dates in the project design and implementation ............................................... 10 E. Implementation arrangements ................................................................................................... 11 F. Project financing ....................................................................................................................... 11 G. Project partners ......................................................................................................................... 14 H. Changes in design during implementation ................................................................................ 14 I. Reconstructed Theory of Change of the project (ToC) ............................................................. 15 IV. Evaluation findings ....................................................................................................................... 18 A. Strategic relevance .................................................................................................................... 18 B. Achievement of outputs according to the project objective ...................................................... 19 C. Effectiveness: Attainment of project objectives and results ..................................................... 31 D. Sustainability ............................................................................................................................. 34 E. Catalytic roles and replication................................................................................................... 36 F. Efficiency .................................................................................................................................. 36 G. Factors affecting performance .................................................................................................. 37 H. Complementarity with UNEP strategies and programs ............................................................ 40 V. Conclusion, lessons learned and recommendations ..................................................................... 40 A. Conclusions ............................................................................................................................... 40 B. Lessons learned ......................................................................................................................... 46 C. Recommendations ..................................................................................................................... 47 i Annex 1: TOR of the evaluation without annexes ................................................................................ 50 Annex 2: List of project documents consulted ...................................................................................... 67 Annex 3: Contact List ........................................................................................................................... 68 Annex 4: Intervention Site Evaluation Report ...................................................................................... 71 Annex 5: The Evaluation Framework ................................................................................................... 87 Annex 6: Costed monitoring and evaluation plan ................................................................................. 90 Annex 7: GEF-RARE sources of co-financing .................................................................................. 100 Annex 8:Tracking Tool for Biodiversity Projects in GEF-3, GEF-4, and GEF-5 .............................. 101 Annex 9. Discussions Notes on Project Outputs with RARE at Headquarters ................................... 116 Annex 10. Brief summary of consultant ............................................................................................. 119 ii Abbreviations and Acronyms AC: Advisory Committee APC: “Adminstracion Publica Cooperativa Manatiales de Chucuri” APECO: Asociación Peruana de Conservación de la Naturaleza ARA: Reciprocal Agreements for Watershed Services Arcoiris: Name of partner NGO working out of Loja, Ecuador AZE: Alliance for Zero Extinction CORPOGUAVIO: Corporación Autónoma Regional del Guavio Colombia EO: Evaluation Officer ESP: Ecosystem Management Programme ETAPA: Empresa de Telecomunicaciones, Agua Potable y Alcantarillado de Cuenca FNB: Fundación Natura de Bolivia FNC Fundación Natura de Colombia FPR: Fundación Proaves de Roncesvalles, Colombia GEB: Global Environmental Benefits GEF: Global Environment Facility HMP: Habitat Management Plans IBC: Instituto del Bien Común del Perú IMTR: Internal Mid-Term Review IR: Inception Report M&E: Monitoring and Evaluation MTE: Mid-Term Evaluation NCI: Naturaleza y Cultura Internacional, Ecuador PES: Payment for Environmental Services PP: Percentage Points Prodoc: The Project Document Pride campaigns: Marketing campaigns that inspires change to benefit the community as well as the environment. RARE: Name of the Project Executing Agency ROtI: Review of Outcomes to Impacts Manual SDC: Swiss Development Corporation of Peru STAP: Scientific and advisory Committee of GEF SWOT: Strength, Weaknesses, Opportunities and Threats TOR: Terms of Reference of the Evaluation ToC: Theory of Change UNEP: United Nations Environment Programme iii Executive summary 1. This GEF/UNEP project was executed by RARE1. The project sought to protect a variety of AZE2 species at intervention sites in the Tropical Andes of Bolivia, Colombia, Ecuador and Peru. This goal was to be accomplished by placing under protection important water regulating ecosystems3 through the implementation of PRIDE4/Reciprocal Agreements for Watershed Services (ARAs) Campaigns. Farmers that agreed to participate in ARAs were asked to set aside and conserve possible AZE habitats in exchange for in-kind incentives and technical assistance aimed at boosting farm productivity. In the outcome to impact analysis, the project received a BB rating; with the greater parts of the outcomes having been achieved. Furthermore, the evaluation showed “implicit forward linkages” to intermediary stages and impacts and that the measures designed to move towards intermediate states have started and have produced results, but they give no indication that they can progress towards the intended long term Global Benefit Goals (GBGs). 2. Twelve partner institutions were selected to implement 12 project intervention sites. Project achievements varied depending on the partner institution. One partner institution withdrew before the project ended. Using UNEP’s scale, the evaluation developed a sustainability rating for each of the remaining partner institutions. Due to internal institutional problems, the Asociación Peruana de Conservación de la Naturaleza (APECO) (Peru) and Arcoiris (Ecuador) are unlikely to implement Pride/ARA Campaigns in the future, while the Empresa de Telecomunicaciones, Agua Potable y Alcantarillado de Cuenca (ETAPA) (Ecuador), Fundacion Natura (Bolivia), Fundacion Proaves (Colombia) and Parques Nationales Farallones (Colombia) are likely to continue to implement Pride/ARA Campaigns at the original project intervention sites beyond the lifespan of the project Caritas (Peru), Fundacion Natura Colombia (FNC) and Naturaleza y Cultura Internacional (NCI) (Ecuador) are highly likely to continue to implement Pride/ARA Campaigns at the original project implementation sites and beyond after project termination. Strong commitment and well-organized and effective extension programs are two reasons why, for example, Caritas, FNC and NCI were successful in meeting project objectives. 3. Pre and post surveys conducted by the project indicated that Pride/ARA Campaigns produce increases in knowledge, attitudes, interpersonal communication and behavioral change for both up and down stream populations. The average percentage point increases 1 RARE is not acronym. It is the name of the international non-governmental organization that executed the project on behalf of UNEP. RARE uses it name to draw attention to its principal objective: saving endangered species. 2 Formed in 2000 and launched globally in 2005,