Terminal Evaluation of the UNEP/GEF Project “Renewable Energy-Based Electricity Generation for Isolated Mini-Grids in Zambia”
Total Page:16
File Type:pdf, Size:1020Kb
Terminal Evaluation of the UNEP/GEF Project “Renewable Energy-based Electricity Generation for Isolated Mini-grids in Zambia” Helene Rask Grøn and Nixon Chisonga Evaluation Office May 2015 Contents I. EXECUTIVE SUMMARY 4 II. INTRODUCTION 10 III. THE EVALUATION 11 IV. THE PROJECT 11 A. Project background 11 B. Project objective(s) and components 12 C. Implementation arrangements 14 D. Project Finance 14 E. Project Partners 14 F. Design changes during project implementation 15 G. Reconstructed Theory of Change (ToC) 15 V. EVALUATION FINDINGS 17 A. Strategic relevance 17 B. Achievement of Outputs 18 C. Effectiveness 26 D. Sustainability and replication 30 E. Efficiency 35 F. Factors affecting performance 35 VI. CONCLUSIONS AND RECOMMENDATIONS 42 ANNEX 1. TERMS OF REFERENCE 50 ANNEX 2. EVALUATION MATRIX 72 ANNEX 3. EVALUATION PROGRAMME 77 ANNEX 4. PROJECT EXPENDITURE AND SUMMARY OF CO-FINANCE 78 ANNEX 5. LIST OF DOCUMENTS AND STAKEHOLDERS INTERVIEWED 84 ANNEX 6. ASSESSMENT OF THE QUALITY OF THE PROJECT DESIGN 88 ANNEX 7. UNEP EVALUATION REPORT QUALITY ASSESSMENT 92 ANNEX 8. BRIEF CV OF THE EVALUATION TEAM 95 Abbreviations AEG Advisory Expert Group CEC Copperbelt Energy Corporation DBZ Development Bank of Zambia DoE the Department of Energy (of Zambia) EIA Environmental Impact Assessment ERB Energy Regulation Board (Zambia) GEF Global Environmental Facility GHG Green House Gas GRZ Government of the Republic of Zambia HIV Human Immunodeficiency Virus LFA Logical Framework Approach KW Kilo Watt(s) KWh Kilo watt hour KWp kilowatt peak MW Mega Watt(s) PDF Project Preparation and Development Facility plf Plant load factor PMU Project Management Unit POPs Persistent Organic Pollutants REA Rural Electrification Authority (of Zambia) RE Renewable Energy SAPP Southern African Power Pool ToC Theory of Change ToR Terms of Reference UNEP United Nations Environment Programme UNIDO United Nations Industrial Development Organisation UNZA University of Zambia ZEMA Zambia Environmental Management Authority ZESCO Zambia Electricity Supply Corporation The Terminal Evaluation was conducted by a team of two external evaluators, Ms Helene Rask Grøn as the Team Leader and supported by the Team Member Mr Nixon Chisonga. The evaluation team expresses their sincere gratitude to the Evaluation Office of UNEP for efficiently and effectively supporting the evaluation process leading to the production of this report. The evaluation team greatly appreciates all project partners and stakeholders in Zambia for open and engaged discussions on the project and lessons learned. This report presents a reflection of the evaluation team’s assessment of the GEF funded UNEP project based on systematic data collection in the field, interactions with project stakeholders, and project documents review. The evaluation team are confident that this Terminal Evaluation Report is a fair and balanced assessment that adds value to future project formulations and implementation. Pretoria, November 2014 I. Executive Summary 1. This report is the Terminal Evaluation of the GEF – funded UNEP project “Renewable Energy Based Electricity Generation for Isolated Mini-grids in Zambia”. The project was formulated in response to, and in support of, the Zambian National Energy Plan’s goal to increase rural electricity access from (the then) 2% rural population to 15% by 2010. The project’s global objective was “Energy related CO2 emissions are reduced through promotion of environmentally sound renewable energy technology based mini-grids for rural electrification in Zambia”. The project document identified several formulations of the project’s main objective, but in general terms, the objective was to address key barriers to the deployment of renewable energy based mini-grids for rural electrification in Zambia. This was to be achieved through the following project components: Designing and establishing legal, institutional and policy framework; Building national and local capacities to promote renewable energy based mini-grids; Planning and setting up innovative project financing mechanisms and structure; Implementing pilot renewable energy based mini-grids to demonstrate business models; Establishing project coordination and management structures and dissemination of information and lessons. 2. The project was implemented by the United Nations Environment Programme, Division of Technology, Industry and Economics (UNEP DTIE) with the United Nations Industrial Development Organization (UNIDO) acting as the executing agency. Key partners included the Government of Zambia, the Project Management Unit (PMU) comprising the Department of Energy (DoE) at the Ministry of Energy and Water Development, Zambia, Development Bank of Zambia (DBZ), Rural Electrification Authority of Zambia (REA), Zambia Electricity Supply Corporation (ZESCO), Copperbelt Energy Company (CEC), the Forestry Department of Zambia and Zambia Environmental Management. 3. The original budget was for 7.506 million US Dollars. This was made up of a Global Environment Facility (GEF) grant of US$ 2.950,000, contribution by Government of Zambia of 1.256 million USD, UNEP/UNIDO contribution of 0.55 million USD and anticipated private sector investment of 2.75 million USD. The budget was revised twice to accommodate expenditure reallocations from delays and towards the extension of the project. The anticipated private sector investment did not materialise, however the co-funding from the Government of Zambia was higher than anticipated. 4. The project was initiated in 2006 and was designed to run for 5 years. Due to substantial implementation delays it was agreed by all partners to extend the project (budget neutrally) by one extra year. The project closed in December 2012. 5. This terminal evaluation was structured with a set of general questions related to achievement of project objectives followed by specific questions structured in line with GEF and UNEP criteria for ex-post evaluations, namely contribution to and achievement of: A: Strategic relevance; B: Achievement of outputs; C: Effectiveness: Attainment of project objectives and results; D: Sustainability & replication; E: Efficiency and F: factors affecting project performance. The evaluation approach used encouraged open-ended sharing of the most successful aspects of the project as well as aspects of the project which could have been improved. 6. The project was successful in engaging the Government of Zambia in the development of pilot mini grids. The establishment of these grids has received much attention and is felt to have contributed to the increase of awareness of the benefits, and practical knowledge of the mechanisms for developing Renewable Energy (RE) mini-grids. The Government of Zambia is now planning further development of both hydro and solar mini- grids as part of its national electrification programme. If successfully finalised, the revolving fund set up by the project will enable ongoing investment in RE projects in years to come. 7. All this was achieved despite considerable unanticipated problems such as the reluctance of the private sector to invest in RE, and a change of project manager midway through the project. 8. While the positive outcomes of the project and the flexibility and adaptive management skills of the project team are notable, the evaluation also notes that; several key project outputs were not achieved, some key adaptive management decisions were not documented, nor were the implications of these decisions 4 adequately considered. This led to some shortcomings in terms of stakeholder engagement (particularly of the private sector and community members), policy support and dissemination of information. For this reason, despite the considerable successes of the project, the overall rating is Moderately Satisfactory. 9. A summary of the project’s performance in the key evaluation areas is discussed below. More detailed description and discussion is found in the main body of the report. A Strategic Relevance 10. The evaluation finds that the project design was highly relevant to the climate change mitigation mandates of UNEP and GEF, to the aims of the Bali Strategic Plan and to Zambia’s national priorities. B Achievement of Outputs 11. With regard to the planned outputs, the project was successful in achieving the following: Building national and local capacities to promote renewable energy based mini-grids: Workshops, study tours and ‘on the job’ learning were used to build capacity of the main project partners. Planning and setting up innovative project financing mechanisms and structures: The project contributed funds to start up a revolving loan scheme for RE. This fund was used to provide loans to the three pilot plants. Some training was given to DBZ staff and the project team provided DBZ with assistance in the development of guidelines for implementation of the revolving loan scheme. However the full rotating loan mechanism was not yet finalised at project closure. Implementing pilot renewable energy based mini-grids to demonstrate business models: The project facilitated the construction of two pilot mini-grids and identified an investor for the third pilot. This was considerable achievement as the original plan to work with private sector was found to be unrealistic and the implementing of the pilot mini-grids was considerably delayed. The two plants which were constructed during the project life time were built by government agencies. However, no activities were implemented towards supporting productive end use of