The European Union’s EDF Programme for Angola “FINAL EVALUATION MISSION FOR THE NON-STATE ACTORS SUPPORT PROGRAMME – (PAANE) PROJECT 9.ACP.ANG.19” Letter of Contract No. 2011/262444 FINAL REPORT GRAZIANO TONELLOTTO (TL) MARION BAUMGART DOS SANTOS September 2011 The project is financed by the European Union The project is implemented by IBF International Consulting In collaboration with BAA (Spain) and Euroconsultants (Greece) “This report was prepared with the financial aid of the European Commission. The opinions expressed herein are those of the consultants and in no way reflect the official opinions of the Commission.” 2 CONTENTS 1. INTRODUCTION ............................................................................................................................ 4 2. WORK ORGANISATION METHOD ............................................................................................... 4 3. SUMMARY OF THE PAANE FA .................................................................................................... 5 4. EXECUTIVE SUMMARY ................................................................................................................ 7 4.1. PERTINENCE .................................................................................................................................. 7 4.2. EFFECTIVENESS ............................................................................................................................. 8 4.3. EFFICIENCY .................................................................................................................................... 9 4.4. IMPACT ........................................................................................................................................ 10 4.5. SUSTAINABILITY ............................................................................................................................ 10 4.6. GENERAL CONCLUSION ................................................................................................................. 11 5. ANSWERS TO THE EVALUATION QUESTIONS ...................................................................... 13 5.1. PERTINENCE ................................................................................................................................ 13 5.1.1. Elements of context analysis .............................................................................................. 13 5.1.2. Preparation and Conception of the Project ......................................................................... 14 5.1.3. Incorporating past and present EC/EU actions and those of other partners ...................... 15 5.1.4. State-Civil Society relations and the PAANE response ...................................................... 16 5.1.5. Coherence between areas of work and results and NSA needs ........................................ 18 5.1.6. Transverse themes ............................................................................................................. 19 5.2. EFFECTIVENESS ........................................................................................................................... 20 5.2.1. Analysis of Objectives and Results .................................................................................... 20 5.2.2. Analysis of objectively verifiable indicators (OVI) ............................................................... 22 5.2.3. Analysis of the range of results without using OVI ............................................................. 23 5.3. EFFICIENCY .................................................................................................................................. 25 5.3.1. Organisation and structure of the PMU .............................................................................. 25 5.3.2. Monitoring and evaluation ................................................................................................... 26 5.3.3. Administrative aspects and management methods ............................................................ 27 5.4. IMPACT ........................................................................................................................................ 29 5.4.1. Impact result 1 .................................................................................................................... 29 5.4.2. Impact result 2 .................................................................................................................... 30 5.4.3. Impact result 3 .................................................................................................................... 31 5.5. SUSTAINABILITY ............................................................................................................................ 33 5.5.1. Strengths ............................................................................................................................. 33 5.5.2. Threats ................................................................................................................................ 34 6. VERIFICATIONS AND RECOMMENDATIONS .......................................................................... 35 6.1. PERTINENCE ................................................................................................................................ 35 6.2. EFFECTIVENESS ........................................................................................................................... 37 6.3. EFFICIENCY .................................................................................................................................. 38 6.4. IMPACT ........................................................................................................................................ 39 6.5. SUSTAINABILITY ............................................................................................................................ 41 6.6. CONCLUSION ................................................................................................................................ 42 7. GLOSSARY AND ABBREVIATIONS .......................................................................................... 43 3 1. INTRODUCTION The present document is the Draft Final Report, as requested in the Terms of Reference (TOR) of the final Evaluation Mission (EM) of the PAANE (Non-State Actors Support Programme) in Angola, after the field phase and debriefing. The mission, composed of the consultants Marion Baumgart Dos Santos and Graziano Tonellotto (TL), began its field phase in Angola (22 days) with a briefing meeting with the EUD – EU Delegation in Angola (Ms Filipa Corte-Real) on May 23, and with Mr. José Fortes, Assistant Director of the NAO, on May 24 2011. From that date, the EUD supported the mission by supplying a list of contacts of the key actors and providing the mission with a room in which it could consult documents and store additional forms. On May 24th, it was also possible to begin organising the Evaluation Work Plan by arranging dates/times for interviews/consultations and visits in Luanda and the provinces of Huila and Malanje. The mission began its meetings and interviews in Luanda on May 25 2011 (see list of people and chronogram of visits and interviews in Appendix 4). In accordance with the TOR, an Aide Memoire was presented on 31 May 2011, in which the evaluators indicated the methodology adopted by the mission and the initial results/preliminary conclusions. The debriefing was held on 10 June 2011 at the NAO head office. Ms Filipa Corte-Real of the EUD and Mr. Fortes, Assistant Director of the NAO (National Authorising Office) took part in the meeting at which a finalised Aide Memoire was presented with the provisional results, in compliance with the TOR. For further information on the objectives and expected results of the evaluation, see the TOR (Appendix 2). We are grateful to all those who contributed in one way or another to this evaluation, and who were willing to share their experiences, impressions and criticisms with us. We would particularly like to thank Ms Filipa Corte-Real of the European Commission, Mr. José António Fortes, of the Technical and Administrative Unit for ACP/EE Cooperation and Ketty Tirzi, ex-coordinator of the former PAANE PMU. There was excellent long-distance collaboration (via telephone and e-mail) with the latter as well as during the mission’s last day-and-a-half in Luanda. We would also like to thank Mr. Guilherme Santos, chairman of the ADRA for his wonderful collaboration with our work and our stay in Lubango and Huila. 2. WORK ORGANISATION METHOD The Mission is divided into three parts: Initial phase: preparation and analysis of documents; perfecting and developing the outlined chronogram for the mission in Angola approved by the EUD when the Mission was hired, taking into consideration the possible logistical constraints in Luanda and the anticipated visits to 2 provinces; first long-distance contacts (e-mails) for the aspects of logistical organisation and mobilisation of the mission in Luanda and the 2 provinces to be visited; organisation of the entry visas for Angola (in Italy and South Africa); preliminary long-distance gathering of opinions and information by the ex-Head of the Programme’s PMU (with a meeting also held in Rome); constant coordination with the consulting companies of the IBF consortium (BAa Consultors and Euroconsultant) and the Delegation of the European Commission in Angola; detailed organisation of the