Draft Report of Final Evaluation of FARM Africa WCBP

EUROPEAN UNION

WOREDA CAPACITY BUILDING PROGRAMME in Five Special Woredas of the SNNPR

FINAL EVALUATION REPORT

Project Ref: ONG-PVD/2003/064-438

Co-Financed By: EU, CordAid and FARM Africa

Prepared By: Solomon Gebre-Tsadik (Team Leader) Berhanu Worku Rural and Pastoral Consultant November 2009 Addis Ababa, Ethiopia TABLE OF CONTENTS

TABLE OF CONTENTS ...... i LIST OF TABLES ...... iii LIST OF FIGURES ...... iii LIST OF ACRONYMS ...... iv EXECUTIVE SUMMARY ...... v 1. INTRODUCTION ...... 1 1.1 OVERVIEW OF THE PROJECT ...... 1 1.2 PURPOSE OF THE FINAL EVALUATION ...... 3 1.3 EVALUATION METHODOLOGY ...... 4 1.4 COMPOSITION OF THE EVALUATION TEAM ...... 5 2. Project Design ...... 6 3. Relevance of the Project ...... 7 4. Project Efficiency ...... 9 4.1 BENEFICIARY TARGETING PROCEDURE AND APPLIED CRITERIA ...... 9 4.2 RESOURCE ALLOCATION AND SPATIAL ASPECTS ...... 11 4.3 PROJECT MANAGEMENT ...... 13 4.3.1 Project Inputs ...... 13 4.3.2 Internal Project Cycle Management and Working Relation with Other Actors ...... 15 4.3.3 Support/Inputs to Project Management from Country Office and External Bodies ...... 17 4.4 TIMELINESS OF THE PROJECT INTERVENTION ...... 19 4.5 COST-EFFECTIVENESS OF THE PROJECT ...... 20 4.6 CONSIDERATION ON GENDER ASPECTS ...... 21 4.7 OVERALL EVALUATION OF EFFICIENCY OF THE PROJECT ...... 22 5. Effectiveness of the Project Outputs/Results ...... 22 5.1 SUMMARY OF ACHIEVEMENTS OF THE PROJECT ACTIVITIES ...... 22 5.1.1 Output 1: Five Strong, Sustainable and Locally Accountable WDAs Built that are able to Implement a Process of Locally Initiated Participatory Development ...... 23 5.1.2 Output I1: The Capacity of Relevant Woreda Government Offices, WDAs, KDCs and VDCs Built to Develop, Implement, Monitor and Evaluate a Participatory Planning System ...... 25 5.1.3 Output II1: Increase the Livelihood Base of the Local Community and Most Vulnerable Households in each of the Five Woredas ...... 26 5.1.4 Output IV: Write up Lessons Learnt from the WCBP and Disseminate to Donors, Government, NGOs and Regional Networks ...... 27 5.2 ASSESSMENT OF QUANTITY OF PROJECT OUTPUTS/RESULTS ...... 27 5.3 ASSESSMENT OF QUALITY OF PROJECT OUTPUTS/RESULTS ...... 29 5.4 USE OF THE OUTPUTS/RESULTS BY THE PROJECT BENEFICIARIES ...... 30 5.5 EXTENT TO WHICH THE PROJECT ACHIEVED ITS OUTPUTS AND REINFORCING ITS OBJECTIVE ....31 5.6 OVERALL ASSESSMENT OF EFFECTIVENESS OF THE PROJECT ...... 34 6. Assessment of Outcomes and/or Impact-Trends of the Project ...... 35 6.1 OUTCOMES AND/OR IMPACT-TRENDS OF THE PROJECT AT HOUSEHOLD LEVEL...... 35 6.2 OUTCOMES AND/OR IMPACT-TRENDS OF THE PROJECT AT COMMUNITY LEVEL ...... 37 6.3 OUTCOMES AND/OR IMPACT-TRENDS OF THE PROJECT AT INSTITUTIONAL LEVEL ...... 38 6.4 OUTCOMES AND/OR IMPACT-TRENDS OF THE PROJECT ON CROSSCUTTING ISSUES ...... 40 6.5 OVERALL ASSESSMENT OF OUTCOMES AND IMPACT-TRENDS OF THE PROJECT ...... 41

______i Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR 7. Sustainability of the Project ...... 41 7.1 FINANCIAL/ECONOMIC SUSTAINABILITY ...... 41 7.2 TECHNICAL AND SOCIAL SUSTAINABILITY ...... 43 7.3 INSTITUTIONAL AND MANAGERIAL SUSTAINABILITY ...... 45 7.4 OVERALL ASSESSMENT OF SUSTAINABILITY OF THE PROJECT ...... 46 8. Summary of Strengths, Weaknesses, Opportunities and Threats of WCBP ...... 46 8.1 MAJOR STRENGTHS OF THE PROJECT ...... 46 8.2 MAJOR WEAKNESSES OF THE PROJECT ...... 47 8.3 OPPORTUNITIES FOR SUSTAINABILITY OF PROJECT OUTPUTS ...... 49 8.4 THREATS TO SUSTAINABILITY OF PROJECT OUTPUTS ...... 50 9. Traceability and EC Visibility ...... 50 10. Best Practices and Lessons Learnt ...... 51 10.1 BEST PRACTICES ...... 51 10.2 LESSONS DRAWN ...... 51 11. Major Challenges/Constraints/Problems ...... 52 12. Conclusions, Recommendations and the Way Forward ...... 52 12.1 CONCLUSIONS ...... 52 12.2 RECOMMENDATIONS ...... 54 12.3 THE WAY FORWARD ...... 54 Annexes ...... 56 LIST OF CONTACTS ...... 56 ITINERARY OF EVALUATION TEAM...... 57 CHECKLIST FOR EVALUATION ...... 62 SUMMARY OF RATINGS BASED ON MAJOR EVALUATION INDICATORS (BY EVALUATION TEAM) ...... 67 SUMMARY OF PROJECT BUDGET AND UTILIZATION ...... 76 UPDATED LOCAL STAFF OF WCBP ...... 83 MINUTES OF WCBP FINAL EVALUATION WORKSHOP ...... 84 TERMS OF REFERENCE FOR THE FINAL EVALUATION ...... 94

______ii Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR LIST OF TABLES

Table 1: Resource Allocation for Saving and Credit Associations in Burji Special Woreda ...... 12 Table 2: Assessment of Project Inputs in the Intervention Woredas ...... 13 Table 3: Assessment of Quantity of Results Achieved so far ...... 28 Table 4: Overall Assessment of Quality of Outputs of the Project ...... 30 Table 5: Overall Assessment of Use of the Outputs by the Project Beneficiaries ...... 31 Table 6: Assessment of the Extent of Achievement of the Project Outputs & Reinforcing its Objective ...... 31 Table 7: Case Study: Summary on the Impact of Goat Provision at Household Level ...... 35

LIST OF FIGURES

Figure 1: Summary of Overall Assessment of Design of the Project ...... 7 Figure 2: Overall Assessment of Relevance of the Project ...... 8 Figure 3: Overall Assessment of Beneficiary Targeting and Applied Criteria ...... 10 Figure 4: Assessment of Resource Allocation in the Intervention Woredas ...... 12 Figure 5: Overall Assessment of Project Inputs ...... 15 Figure 6: Assessment of Internal Management & Working Relationship with Other Actors ...... 16 Figure 7: Assessment of External Support/Input to Management...... 18 Figure 8: Percentage of Male and Female Members in Three Saving and Credit Associations (Burji Special Woreda) ...... 21 Figure 9: Assessment of Overall Efficiency of the Project ...... 22 Figure 10: Summary of Overall Assessment of Effectiveness of the Project ...... 35 Figure 11: Berek Spring Development in Burji Woreda ...... 38 Figure 12: Sorobo Roof Water Catchments in Konso Woreda ...... 38 Figure 13: The Effects of the Project at Institutional Level ...... 39 Figure 14: Area Enclosure in Derba Kebele (Amaro Woreda) ...... 40 Figure 15: Koto Nursery in Loltu Kebele (Konso Woreda)...... 40 Figure 16: Assessment of Overall Outcome and/or Impact-Trend of the project ...... 41 Figure 17: Assessment of Financial Viability of the Project ...... 42 Figure 18: Technical and Social Sustainability of the Project ...... 44 Figure 19: Assessment of Institutional and Managerial Viability of the Project ...... 45 Figure 20: Summary of Overall Assessment of Sustainability of the Project ...... 46 Figure 21: An Example of EC Field Visibility in Arfaide Kebele of Konso Special Woreda ...... 50 Figure 22: Summary of Overall Evaluation of the Project ...... 53

______iii Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR LIST OF ACRONYMS

BoARD Bureau of Agriculture and Rural Development BoFED Bureau of Finance and Economic Development CAHW Community Animal Health Worker CAP Community Action Plan CBI Community Based Institution CBO Community Based Organization CDF Community Development Fund CRDA Christian Relief and Development Association CSO Civil Society Organization DA Development Agent EC European Commission EGS Employment Generation Scheme EU European Union EWS Early Warning System FARM-Africa Food and Agriculture Research Management in Africa HIV/ AIDS Human Immunodeficiency Virus/ Acquired Immunity Deficiency Syndrome KDC Kebele Development Committee M&E Monitoring and Evaluation MOT Mobile Outreach Team MoU Memorandum of Understanding MTR Mid-Term Review NDA National Development Association NGO Non Governmental Organization NRM Natural Resource Management PM Programme Manager PRUP Participatory Resource Use Planning PSNP Productive Safety Net Program PW Public Work RDFS Rural Development and Food Security RUPA Rural and Pastoral Consultant SCA Saving and Credit Association SEPDA Southern Ethiopia Peoples Development Association SNNPR Southern Nations, Nationalities and Peoples Region SWOT Strength, Weakness, Opportunity and Threat ToR Terms of Reference VDC Village Development Committee WCBP Woreda Capacity Building Project WDA Woreda Development Association WGA Women Goat Association WGG Women Goat Group WoARD Woreda office of Agriculture and Rural Development WoFED Woreda office of Finance and Economic Development WPC Woreda Planning Committee

______iv Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR EXECUTIVE SUMMARY

INTRODUCTION

In the last five years, FARM Africa has been implementing a project entitled "Woreda Capacity Building Project (WCBP) in Five Special Woredas of SNNPR". The project was funded by the European Union (75%), CordAid (15%) and FARM-Africa (10%) with the aim of reducing poverty in the most vulnerable households by strengthening key institutional structures and building the capacity of local government offices and NGOs, particularly Woreda Development Associations (WDAs), which could promote sustainable development at Woreda level. The project also focused on developing a ‘bottom-up’ participatory planning system, and in developing skills and expertise in project implementation through co-implementing a range of micro-projects and then establishing a Community Development Fund (CDF). The WCBP commenced its operation in 2004 after signing project agreement with the European Union (EU) in July 2004 and implementation agreement with the Regional government in September 2004. The project actually started its implementation during the first and second quarters of the year 2005. The Mid Term Review (MTR) of the project was took place in August 2006. This final evaluation for the project was conducted in July 2009. Although separate reports are prepared by the Independent Consultants and Regional Counterparts, the evaluation was conducted jointly by a team of experts deployed from Regional BoFED and External Consultants. The evaluation exercise was mainly focused on assessing the design, relevance, efficiency, effectiveness, impact and sustainability of the project actions.

PROJECT DESIGN

The evaluation revealed that the participation of various stakeholders during design and planning stage was generally poor. It seems that the active participation of beneficiaries and other stakeholders was overlooked at initial stage which would be useful to minimize and/or avoid the inherent over ambitious nature of the project. Specifically, the involvement of the Regional counterparts at initial stage was mainly limited to appraisal and signing of the project agreement. In line with this, the appropriateness and clarity of the project preparation and submission process was generally inadequate, and thus, lost the opportunity to design feasible project proposal at initial stage. The project design was also overambitious in terms of realization of five outputs in the original project document and four outputs in the amended proposal with wider area coverage and a range of activities. In general, the project design was poor in terms of active participation and consultation of relevant stakeholders; specification of outputs and outputs targets; validity of means-ends relationships; specification of implementation arrangement and managerial structure; clarify and realism of work plan; and realism of pre-requisite and risks for project success. Therefore, though the project design was more or less found to be satisfactory in terms of clarity of objective, specification of target groups, specification of source of project input, and identification of WDAs as implementing partner, the limitation in the quality of the project design mentioned above together with the management problems observed in the course of the project period were underlined as the major constraining factors which undermined the efficiency and effectiveness of the project.

RELEVANCE OF THE PROJECT

FARM Africa’s Woreda Capacity Building Project (WCBP) was generally relevant in terms of addressing the genuine development constraints of the intervention Woredas. Most of the project interventions in most of the Woredas found to be relevant and appropriate to overcome pressing development constrains/problems in terms of promotion of social and economic services and facilities as well as livelihood security of target beneficiaries. In most of the cases, the planned project outputs were also relevant and appropriate in terms of their integration and coherence with the national/ regional development programs. Accordingly, the approaches and strategies employed by the project was generally relevant and in line with government’s strategy and program of decentralization, capacity building, livelihood and food security initiatives. However, based on the impression after community discussions, there were some specific activities (like the construction of slaughtering house, public toilet outside social institutions and the grain store in Derashe Special Woreda) for which the evaluation team doubted to accept them as community priority/felt need activities. In general, although the project was satisfactorily relevant in addressing the development problems of the intervention areas and reasonably coherent with the government development strategies, however, it was poor in providing a cost-effective response to the identified and prioritised development problem in terms of quantity, quality and use of project outputs. Indeed, it is rational to underline the fact that the relevance of most of the interventions in true sense depend on the full completion of remaining activities at hand and the proper use and management of the project outputs in the future. This in turn requires the institutionalisation of the community based management of

______v Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR common resources created by the WCBP like the grain mill, slaughterhouse, grain bank, public toilets, etc through a consolidation action.

PROJECT EFFICIENCY

The evaluation team has assessed project efficiency in areas of Beneficiary Targeting Procedure and Applied Criteria, Spatial Resource Allocation, Project Management, Timeliness, Cost-Effectiveness and consideration on Gender Aspects. The assessments of project efficiency on specific indicators are summarized below.

Beneficiary Targeting Procedure and Applied Criteria

The project was strong in terms of targeting the remote Woreda in SNNPR (especially Basketo Special Woreda) and inaccessible Kebeles in Burji and Amaro Special Woredas. This was considered to be one of the strengths of FARM Africa in terms of area targeting. Accordingly, the evaluation team would like to acknowledge the effort made by FARM Africa in targeting of Basketo Special Woreda and its remote Kebeles as one of the breakthrough actions in terms of area targeting. In addition, as learnt from the discussion made with different project stakeholders, the applied criteria for community targeting for most of the CDF activities within the intervention Woredas was reasonably fair and satisfactory. In this respect, it was indicated that there was participatory targeting process in selecting of the project localities for the CDF activities through involving the local partners including WDAs, project staff and staff of concerned government offices.

In relation to targeting of individual beneficiary households, which was mainly the responsibility of WDAs and grassroots level institutions (like Kebele Administrations and Kebele/Village Committees), the situation was different from area targeting. In this regard, it was learnt that the targeting process in Konso, Amaro and Burji Special Woredas was relatively better in terms of participation and community representation compared to that of beneficiary targeting in Derashe and Basketo Special Woredas. Despite what is stated above, in almost all of the intervention Woredas, well defined household targeting procedure and applied criteria were lacking for CDF activities for which guidelines had to be prepared at initial stage of project implementation. As a result, although wealth ranking was conducted at initial stage of the project, except for the provision of goats, there was no evidence that whether some interventions have targeted the most vulnerable sections of the local communities. In line with this, the project holder (FARM Africa) has the opinion that although the overall objective of WCBP was aimed at reducing poverty in vulnerable households, however, the fact in some project activities has required individual interest, commitment, experience and skill to run the interventions. On the other hand, it was revealed that the process of beneficiary targeting at grassroots level for some activities was not clear for the evaluation team. For instance, there was no guideline on how and who should directly be targeted for saving and credit schemes, slaughterhouse, grain mill and grain banks. This in turn may put the proper use and management of these project outputs in the future at a stake unless appropriate consolidation activities are done to institutionalise the ownership and future management of these outputs.

Resource Allocation and Spatial Aspects

At initial stage, there was poor capacity assessment of the WDAs in different Woredas where there were more or less similar resource allocation for different Woredas despite heterogeneity of the WDAs in terms of capacity and the need for external support. There was also inadequate resource allocation for livelihood security in terms of provision of two goats per household which was found to be very minimal compared to other similar intervention in the Region. In line with this, FARM Africa indicated that the provision of two goats per household was deemed appropriate for introducing goats to these households and for them to increase their herd gradually whilst learning about appropriate fodder provision and accessing CAHW support. In addition, the NGO stated that learning from evaluations of other FARM Africa projects suggested that the provision of two goats was sufficient to significantly increase household assets and enabled WCBP to reach a large number of beneficiaries within the project timeframe. According to the project holder, what matters is not disbursing huge amount of money which is not necessarily used for investing in assets leading to long term improvements in household food security. The intention was to make most of the resources available in being able to provide assets to as many poor households as possible. On the other hand, the evaluation team has the opinion that given lack of other complementary livelihood intervention under the project and/or other projects and the need for relatively more resources to enhance the household livelihood status in tangible manner, the provision of two goats per household was found to be very minimal.

______vi Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Furthermore, the evaluation team identified certain constraints in resource allocation to CDF micro-projects. For instance, the amount of seed capital (revolving fund) allocated to each Saving and Credit Association (SCA) in Burji Special Woreda was more likely depended on the amount of saving capital rather than on the number of members of SCA. In general, it was observed that the amount of fund allocated for most of the CDF activities was minimum compared to the requirement to complete the activities. The recent price escalation has also aggravated budget constraint which in turn affected quality and quantity of some CDF activities and even resulted in problems of completion of some CDF activities before the end of the project period particularly in Derashe Special Woreda. Despite the above constraints in allocation of resources, however, in most of the cases, the spatial aspect of resource allocation was found to be more or less satisfactory in most of the intervention Woredas.

Project Management and Implementation

Project Inputs: The findings revealed that the donor disbursed the planned budget of the project as per its commitment though there was a significant delay in releasing the second payment of the project which was released following the MTR. Mainly for this and other internal management reasons, the timeliness of the financial disbursement to WDAs for CDF activities was found to be less than satisfactory and it was one of the reasons for poor working relationship between the project holder (i.e. FARM Africa) and WDAs. In addition, the project was constrained by poor staffing and high staff turnover. However, the quality and quantity of most of the equipments and supplies provided through the project were adequate but constrained with timely provision of the inputs. On the other hand, despite its positive impact in terms of increasing household income and asset holding, the quantity of inputs for household livelihood promotion particularly the provision of only two goats per household to poor women to improve their livelihoods were found to be very low so as to bring about significant change in the livelihood of the beneficiaries (see also the section on Resource Allocation and Spatial Aspects). With respect to the input contribution of other project stakeholders, except for Derashe Special Woreda, the contribution of the government partners was somehow satisfactory, especially in Konso and Amaro Special Woredas. Similarly, community contribution in terms of cash and in kind contribution was encouraging.

Internal Management and Working Relation with Other Actors: Although there have been some improvements over the last one year, the project was poor in terms of systematic planning, monitoring and follow-up and consolidation of the project initiatives. There was ambitious planning at initial stage of the project. It has also lacked logical sequence in planning and implementation of some of the project activities. The project was weak in terms of coordination and management of the project intervention which mainly due to poor commitment, weak project management capacity, frequent staff turnover and unhealthy relationship among the project staff. However, despite the constraints in relation to internal management, the project has more or less good working relationship with its partners at Woreda levels. On the other hand, it was noted that the project stakeholders were weak in timely solving the problem prevailed between the newly established Community Based Institution (CBI) in Amaro Woreda known as Dicho-Tulto which has been supported by Agri-Service and Kore Development Association which was assisted through FARM Africa’s WCBP. The evaluation team noted that there had been competition in terms of membership and disagreements in the use of stamp and receipts which had affected the relationship between the two CBOs which latter on called for signing of partnership agreement in January 2009 between FARM Africa/WCBP and Agri-Service Ethiopia to maintain good relations. Generally speaking, the key factors which affected the timely and proper implementation of the project actions were mainly related to some underlining limitations which are related to the poor design of the project and the weak management and coordination of the project.

External Support/Inputs to Project Management and Implementation: The support provided by the EC Delegation in terms of monitoring and technical assistance was found to be satisfactory where by the donor agency has been conducted regular monitoring exercise and provided feedback for necessary corrective actions, particularly in the last 11 months. On the other hand, the external support provided for the project through external consultants, such as MTR and review of the project, was found to be useful in the provision of valuable recommendations which were helpful for managerial decisions. On the other hand, the technical and administrative support provided by the FARM Africa Country Office was found to be poor. For instance, the termination of the contract of all project staff by the Country Office at a time was devastating in terms of losing institutional memory of the project. The Country Office was reluctant in terms of replacement for Program Manager who resigned in September 2008 which has negatively affected the effectiveness of the project and consolidation of the project actions.

______vii Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Timeliness of the Project Intervention

There was significant delay in initiating the project actions at the beginning of the project period. Similarly, significant delays were occurred after the commencement of project implementation. The project was also constrained by poor logical sequence of implementation where some of the capacity building activities were implemented at the end of the project period though such capacity building initiative should have been planned and implemented at initial stage of the project period to allow them the required follow-up so as to ensure their effectiveness and sustainability. Among others, the major factors which mainly affected the timely initiation and implementation of the projects include: (i) unrealistic organizational structure initially setup by FARM Africa (i.e. the establishment of two mobile outreach teams) for the implementation of the project; (ii) the instability of internal management, poor staff commitment, poor project management capacity and unhealthy and poor team spirit among the project staff; (iii) logistical constraint mainly transportation; (iv) lack of CDF implementation guidelines; and (v) delay in CDF fund transfer and stringent financial settlement procedure.

Cost-Effectiveness of the Project

The findings revealed that the project was generally poor in terms of effectively converting the project inputs (human, material, time and financial resources) into the envisaged outputs. Accordingly, the evaluation team identified various shortcomings in relation to cost-effectiveness of the project. Among others, the major shortcomings include: (i) the initial strategy of the project which was designed to implement the project activities through two mobile outreach teams was expensive in terms of resource utilization related to logistics, human and financial resources; (ii) the project has been expending significant and more amounts of overhead and/or administrative costs as compared to the outputs realized during the last five years; (iii) the pace of project implementation was in general very low, and thus, most of the project activities were implemented with significant delay and even some of the planned activities were either not implemented at all or not completed at the right time which has negatively affected cost effectiveness of the project; (iv) the project has provided matching funds to cover at least 80% of salaries of four staff of WDAs, however, these staff members have not delivered the expected result of ensuring strong Development Association; and (v) the project has lost the opportunity to enhance cost-effectiveness through linking some of its interventions with pertinent government sector offices. For example, the nursery management in Derashe Special Woreda, Koto Spring Water Development (Loltu Kebele) in Konso Special Woreda, the area enclosure in Amaro Special Woreda, provision of goats and some of the CDF activities in the intervention Woredas (such slaughterhouse, seed bank and grain mills) would be more cost-effective if they were properly linked to pertinent Woreda line offices and/or integrated with other development interventions like the PSNP. Linkage and integration with appropriate government sector offices might also create an enabling ground for effective and sustainable management and use of the project outputs beyond the life span of the project.

Consideration on Gender Aspects

The project has more or less addressed gender aspects in most of its undertakings. Accordingly, most of the project activities have targeted male and female members of the beneficiary communities in reasonable proportion. Specifically, the project attempted to address and empower vulnerable women groups through the provision of goats. The various micro-projects funded through CDF (such as water development, school rehabilitation/construction, flourmill, saving and credit schemes, etc) were also found to be gender sensitive as they have addressed the strategic and practical needs of male and female members of the community including children and school boys and girls. On the other hand, it seems that the project was mainly biased towards male in terms of project staffing. Although male and female members of the community were represented in various committees and groups established through the project intervention, however, the percentage of women was mostly found to be low.

EFFECTIVENESS OF THE PROJECT OUTPUTS/RESULTS

Assessment of Quantity of Results Achieved so far

Despite the delay in the timely implementation of the activities, it was noted that the project has implemented most of the plan of actions under output one. For instance, the project results in terms of capacity building through the provision of technical skills and trainings (such as financial management, organizational management, computer training, proposal and report writing, etc) were more or less satisfactory in terms of quantity of outputs produced. On the other hand, the project support provided in terms of assisting WDAs to be financially sustainable as well as support ______viii Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR provided to WDAs to be independent CSO and assistance on fund raising were found be below expectation. The project has more or less implemented most of its envisaged activities in terms of output three to enhance the capacity of relevant government offices, WDAs, KDCs and VDCs to plan, implement, monitor and evaluate community based projects. The same is true in relation to output four i.e. write up and dissemination of lessons learnt in the course of the project implementation process. With respect to output three, despite the initial plan of implementing about 315 CDF micro-projects, following the mid-term review of the project, FARM Africa jointly with its local partners revised CDF project activities in terms of quantity and budget allocation and managed to realize 60 CDF projects (out of the revised 61 CDF projects) during the last five years. However, out of these 60 CDF projects, some of them, particularly those in Basketo Special Woreda, were not fully completed during the evaluation period. In general, despite the observed variation in the level of performance among the intervention Woredas and the failure to fully complete some of CDF activities, the attempt made by the project to produce the planned outputs in terms of quantity was rated as satisfactory.

Quality of Outputs/Results

In terms of quality, the project was relatively rated as satisfactory in its output 2 and output 4 where the project provided exemplary capacity building initiatives in the promotion of bottom-up planning process as well as documentation and dissemination of its experience in the implementation of WCBP in the five Woredas, respectively. On the other hand, despite the attempt to undertake almost all of the planned activities under output one, the quality of outputs of the project in terms of building strong, sustainable and locally accountable WDAs was relatively found to be below expectation where the outputs produced were either poor in realizing the intended purpose (such as weak WDAs). It seems that the failure in the attainment of the anticipated quality for output one might be somewhat attributed to lack of systematic approach of the capacity building interventions by the project and their timeliness. In addition to this and even more importantly the limitations from the side of the WDAs themselves in terms of lack of technical capacity, commitment and strategic approach by the WDAs management and Boards were also the major factors for the failure of the project in realizing the anticipated quality of achievement with respect to output one. Other constraining factors include high staff turnover in the WDAs, lack of recording of institutional knowledge to pass to new WDA managers and lack of building the skills of the new WDA Managers who have been left the Associations for various reasons.

With respect to output three, although the project intervention was basically meant to mitigate the effects of rampant poverty through planning and implementation of community preferred livelihood interventions, however, the project was mainly failed to plan and implement different integrated household level livelihood interventions, allocated limited financial resource per individual household. As a result, the attainment of the project in terms of quality was highly undermined. Besides this, due to budget limitation and weak technical support by some Woreda sector offices, some of the infrastructures constructed under CDF interventions were constrained by some quality problems. However, despite the limitations mentioned above, the quality of output two, three and four was found to be more or less satisfactory and output one was below satisfactory. In general, the overall quality of the project outputs/results was rated as satisfactory and anticipated to be further reinforced through the consolidation phase of the project as proposed by the evaluation team and endorsed by project partners during the final evaluation workshop.

Use of the Outputs/Results by the Project Beneficiaries

The findings in general have shown that the use of project outputs/results by target beneficiaries was found to be less than satisfactory (except for Konso Special Woreda) in terms of output one (i.e. realizing five strong and sustainable WDAs). This is mainly for the reasons mentioned under SWOT Analysis Section (major weaknesses). The evaluation has also revealed that the use of output three by the project beneficiaries in Derashe, Amaro and Basketo Special Woredas was relatively lower as compared to Konso and Burji Special Woredas. However, it is worth mentioning that there will be a possibility for proper and sustainable use of most of the CDF project outputs if the CDF activities are fully completed and the necessary arrangement for sustainable future use and management is installed by FARM Africa and its partners through initiating the necessary consolidation phase. With respect to output two, i.e. introduction of a bottom up planning approach, the expected use of outputs by users/beneficiaries was found to be satisfactory. This rating for the bottom-up planning was made mainly underlining the interest of the local government to adopt the planning approach in the future. However, the exercise was confined within the intervention Kebeles of the WCBP and even discontinued after the end of the

______ix Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR CDF planning process. Concerning the fourth output, FARM Africa, the Donor, WDAs and other stakeholders (those who got access to the materials) are expected to adopt and/or utilize the materials in their future project planning and implementation efforts. In general, the overall rating for the use of outputs/results of the project was somehow satisfactory.

Extent to which the Project Achieved its Outputs and Reinforcing its Objective

The extent to which the project achieved its expected outputs/results was assessed by output types for each respective Woreda. Accordingly, with respect to output one (i.e. realization of five strong WDAs at the end of the program), the outcomes vary among the intervention Woredas. For instance, the status of Konso WDA was somehow promising and rated as good by the evaluation team. Despite the implementation of different capacity building support activities by the WCBP for the Development Associations at Burji, Derashe, Amaro and Basketo Special Woredas, the institutional capacity of these associations was not strengthened as anticipated due to various reasons. Among others, the major constraining factors include lack of systematic approach and timeliness of the capacity building effort made from the project side as well as poor commitment of WDAs to mobilize local communities, weak support from respective government bodies, frequent turnover of WDA staffs and weak governance by WDA boards to capitalise on the capacity building supports made by the WCBP were the major ones. But, it is worth to mention that the current situation of these WDAs is by far better than compared to their situation before five years. Although most of the physical inputs mentioned above were delivered by the project as planned, however, failure to implement some other important aspects of technical supports (like failure to link the associations with potential donors, failure to enhance the fund raising capacity of the associations from internal and external sources, etc) have constrained the associations.

With respect to the second output, the extent of the work done in all the five Woredas was rated as satisfactory. Here, it is also worth to mention that if not for lack of replication of the bottom-up planning approach in other Kebeles of the intervention Woredas as well as lack of institutionalization of the approach within the existing local government planning system, the work done in the pilot Kebeles through KDCs and VDCs was satisfactory. However, the approached was mainly used in the planning process of CDF projects of WCBP and discontinued following the completion of the CDF planning exercise. The project holder indicated that although the institutionalization of the approach was planned at initial stage while it was cancelled during the second amendment of the project. On the other hand, the evaluation team noted that piloting of such best practice (participatory planning approach) would be more beneficial if it were followed by the initiative to scale-up and replicate to wider communities and geographical areas rather than cancelling the intended intervention through project amendment after MTR.

The third output anticipated to be achieved by the project was related to the enhancement of the livelihood base of the local community and most vulnerable households. In this regard, the CDF project activities planned for the enhancement of livelihoods at community and household levels were potentially capable to positively change the livelihood of their target group. However, for the constraints related with their completion status, their quality standard, level of integration and complementarily, volume of livelihood assets transferred per target household and lack of design of the appropriate management system for their proper future use and management, the evaluation team rated the achievement of the project with respect to this criteria as less than satisfactory in Derashe, Amaro and Basketo Special Woredas. The findings revealed that the achievements in Burji and Konso Special Woredas were relatively found to be good and satisfactory as compared to the rest of the intervention Woredas. However, if appropriate exit strategies are designed and implemented in collaboration of concerned stakeholder, it seems that most of the livelihood enhancement activities implemented by WCBP will have potential impact to improving the human and livestock health, access to education, addressing the problem of natural resources management and food security, addressing work load of women, improving the household income of target beneficiaries and provision of safe and clean water which reduces prevalence of water borne diseases. As a result, despite some constraints, the project was generally rated as satisfactory in terms of achievement of output three.

______x Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR ASSESSMENT OF OUTCOMES AND/OR IMPACT-TRENDS OF THE PROJECT

The evaluation team attempted to assess the outcomes and/or impact trends of the project at household, community and institutional levels. The impacts and/or impact trends of the project on crosscutting issues were assessed including gender, environment and prevention and control of HIV/AIDS.

Outcomes and/or Impact-Trends of the Project at Household Level

As it is clearly illustrated in the case studies presented in the body of this report, the project has shown positive impact at household level in terms of increasing household income and asset through the provision of goats and saving and credit scheme financed by CDF. In line with this, the majority of goat beneficiaries contacted during the evaluation exercise indicated that they have benefited from goat provision in terms of cash income from the sale of animals as well as increment of household asset. Similarly, most of the beneficiaries of saving and credit associations stressed that the scheme has positively contributed to increase their household income and asset through using the revolving credit fund for various income generation activities. Besides, though most of the CDF activities were completed during the last quarter of the project period, and most of them didn’t start to deliver services till the evaluation period. However, the social service infrastructures constructed through CDF activities (like the construction/rehabilitation of school blocks/classrooms, animal and human health posts, water facilities and grinding mills) are expected to improve the access of the households in the intervention localities for these basic social and economic services.

Outcomes and/or Impact-Trends of the Project at Community Level

Some of the infrastructural facilities completed and started their operation like water supply schemes and school constructions/rehabilitations have improved the quality of social services and their service coverage. Therefore, if the ongoing and uncompleted CDF funded activities are finalized and properly handed over to pertinent government sector offices and user communities, there will be possibilities that the communities may benefit from the outputs of the CDF projects from: (a) better coverage of primary education for the school age children; (b) better access to clean and safe drinking water and reduced prevalence of water borne and water related diseases; (c) reduced labour and time requirement for fetching water, and hence, increased time for productive, reproductive and other household chores; (d) improved hygiene and environmental sanitation, and hence, reduced prevalence of communicable diseases; (e) improved access to primary healthcare and preventive health service; and (f) improved access to livestock health service and improved livestock productivity.

Outcomes and/or Impact-Trends of the Project at Institutional Level

At institutional level, the capacity building supports made to WDAs, the establishment of mini database (resource centre) at Woreda level and establishment of VDCs and KDCs at intervention Kebeles and their involvement in the development of Community Action Plan (CAP) were among the major relevant and appropriate interventions of the project. However, despite the relevance of the interventions, the institutional impact of the support actions was very minimal. The support provided for the WDAs didn’t bring the expected outcome in most of the intervention Woredas. Therefore, there is no ground to expect an institutional impact of these actions in Amaro, Burji, Basketo and Derashe Special Woredas. As a result, the rating for institutional impact of the support for WDAs was rated as poor which implies that the project was found to be less than satisfactory in terms of institutional impact of the action.

Outcomes and/or Impact-Trends of the Project on Crosscutting Issues

Some of the project interventions found to be positively affected the gender aspects in terms of livelihood enhancement through addressing the felt need of men and women. In addition, women involvement in the management of group resources (such as in water point management) was one positive outcome which somehow enhanced the involvement of women in communal decision making process. Besides, most of the CDF projects if completed and being operational are expected to serve both male and female members of the community. The other cross cutting issue addressed by the project outcome was the rehabilitation of the microenvironment such as through increased level of vegetation coverage by new plantation and area enclosure.

______xi Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR HIV/AIDS was planned and implemented in Holte Kebele of Derashe Special Woreda through funding from CDF. Mainstreaming HIV/AIDS is an approach that is relevant in both development and humanitarian work. By mainstreaming HIV/AIDS, we mean a process that enables development actors to address the causes and effects of HIV/AIDS in an effective and sustained manner, both through their usual work and within their workplace. Mainstreaming of HIV/AIDS is not an intervention per se. Due to its adverse impact on health and development initiatives, any development interventions should be integrated into planned and ongoing development processes by linking them to the core business..

SUSTAINABILITY OF THE PROJECT

Financial/Economic Sustainability: In relation to financial sustainability of some CDF project activities, most of the Woreda sector offices indicated that they were ready to assign the required resources (including human, materials, supplies and other resources) for proper functioning and operation of the infrastructures and/or facilities after the end of the project period as long as the social service facilities are properly completed and officially handed over to pertinent government sector offices. However, the evaluation exercise identified some constraints regarding financial viability which include, but not limited to, the following: (a) the financial viability of some interventions, such as the grinding mill, grain store and slaughterhouse, may be at stake unless effective and proper community management system and technical back-up is in place; (b) the current financial capacity as well as the capacity to generate funds by the four WDAs (Amaro, Basketo, Burji and Derashe) was still found to be very limited so that the financial viability of the Associations may negatively be affected; and (c) it is unlikely that the insurance scheme designed for goat provision scheme would continue to operate to replace for animals lost through death and related factors.

Technical and Social Sustainability: There has been satisfactory level of sense of ownership among the target communities for most of the CDF projects like schools, health posts, veterinary posts and water schemes. This promising sense of ownership by the community for CDF project output was mainly the outcome of the active involvement of the target communities in the need identification, planning and implementation of CDF projects. The participation of the community through labour and locally available material contributions during the implementation of the activities was the reflection of communities’ sense of ownership. However, it was noted that the ownership of some CDF project outputs was not clearly determined such as slaughterhouse, grain store, public toilets and grinding mill. The other major concern for technical viability of the WDAs is the high turnover of trained staffs of WDAs and this will adversely affect the opportunity of the WDAs in using the added value obtained from the various capacity building initiatives such technical trainings provided by the project. In line with this, the project holder has also indicated that although the project has equipped the WDA’s staff with a number of important experience sharing tours, new and refresher trainings, etc, however, most of them have left the organizations (WDAs) taking training materials and manuals with them leaving little institutional memory which has contributed to create negative effect on the technical sustainability of the WDAs.

Institutional and Managerial Sustainability: The analysis on how far the project embedded in the existing local institutional structures in order to survive beyond the end of the project indicated that most of the project outputs are expected to be officially handed over to pertinent Woreda government offices, target CBOs and beneficiary communities. Despite capacity limitation of local project stakeholders at different levels, the handed over project outputs might have the possibility to be sustained institutionally. The project also organized various training sessions for WDAs, community members, CBO leaders and staff of government offices in relation to the management of the project outputs after the end of the external support. However, some individuals who were provided with managerial training (especially WDA staff) might not serve the project outputs due to staff turnover, transfer, replacement, demotion, etc. The other critical issue in terms of institutional viability is related to the fragile institutional and managerial capacity of the four WDAs at Derashe, Burji, Amaro and Basketo Special Woredas. In line with this, the WDAs are not in a position to operate at office level let alone the implementation of development projects at community level unless they are supported in the near future.

Summary of Strengths, Weaknesses, Opportunities and Threats of WCBP

Major Strengths of the Project . FARM Africa was said to be strong in working in inaccessible and remote areas; . The project was strong in establishing KDC and VDC as well as in need identification, prioritization, project planning, preparation of CAP and implementation of micro-projects funded through CDF; ______xii Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR . The project was strong in organizing useful training programs especially project planning for WDAs; . Organising experience sharing events for WDAs was useful for WDAs to draw lessons and experience; . The various capacity building initiatives provided to WDAs were very important to enhance their capacity in terms of planning, implementation, monitoring and evaluation of development projects; . The project was strong in enhancing community participation; . The project attempted to integrate some of the project interventions with PSNP Public Work Activities and regular government intervention which were useful in enhancing effectiveness and sustainability of the interventions; . The project has created positive externality through being a lesson; . The project was strong in organizing quarterly review meetings with its local partners (especially since 2007); . FARM Africa was strong in terms of its approach/strategy to use WDAs as entry point (phase in). . The project was relatively effective and strong in enhancing saving and credit activities which has positively contributed to improve the livelihood of the beneficiaries; . The attempt made by the project to establish Woreda level resource centre (mini-data) and training on database management was useful in addressing information needs of various actors.

Major Weaknesses of the Project . The initial project proposal was over ambitious in terms of area/Woreda coverage and volume of activities; . Delay in the preparation of guidelines for the implementation of CDF; . Poor internal management system i.e., failure to install effective internal management system; . Delay in timely disbursement/transfer of project budget especially for micro-projects funded through CDF; . Delay in timely initiation and implementation of most of the project activities; . Delay in operation (provision of services) of completed projects; . Poor phase out and handing over of most of the completed projects to pertinent stakeholders; . Absence of permanent WDA Managers working on full-time basis and frequent turnover of WDA managers; . Weak capacity, poor commitment and dedication of WDAs’ board, managers and staff; . In some cases, there was overlap of roles and duplication of efforts in CDF implementation between WDAs and the project holder. For instance, the evaluation team noted that FARM Africa handled some of the roles and responsibility of Basketo WDA in the implementation of the CDF activities; . FARM Africa has failed to properly consider some of the recommendations forwarded during the MTR; . Lack of logical sequence in the implementation of various project activities; . Lack of analysis of clear targets and beneficiaries for some activities such as flourmill and slaughterhouse; . Inadequate resource allocation for livelihood security such as provision of two goats per beneficiary household; . The Regional project signatories failed to properly appraise the project proposal at initial stage; . Poor follow up and support from the Regional government partners; . Unable to sustain the functioning of WDAs beyond the life of the project; and . Poor capacity assessment of the WDAs in different Woredas where at initial stage there were more or less similar resource allocation for different Woredas despite heterogeneity of the WDAs in terms of capacity.

Opportunities for Sustainability of Project Outputs . Almost all Woreda Administrations have shown positive attitude (willingness) and ready to support WDAs; . The presence SEPDA was found to be an opportunity for WDAs to build their capacity; . The availability of other projects which are implemented by some of the WDAs might be an opportunity to continue the existence of the WDAs in the near future. . Existence of promising community commitment and willingness to support WDAs through contribution and membership fee found to be an opportunity for sustainability of WDAs. . The plan to replicate and scale-up of bottom-up planning approach by the Regional government could be an opportunity to sustain the impact of the project after the termination of project. . Availability of potential donors to support community development projects could be taken as a good opportunity for the continuation of WDAs’ operations in the intervention Woredas.

Threats to Sustainability of Project Outputs . Some of the WDAs complained for budget constraints for completion of on-going activities; . Shortage of financial resources by most of the WDAs might highly affect and even threaten their existence; . Part-time managers who are paid externally by respective Woreda Administration might continue to affect the effectiveness of the WDAs in the future too; and

______xiii Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR . Conflict and insecurity situation (especially in Burji Special Woreda) might continue to affect the sustainability of the project.

TRACEABILITY AND EC VISIBILITY

BEST PRACTICES AND LESSONS LEARNT

Best Practices: Among others, the evaluation identified the following best practices from the implementation of FARM Africa’s WCBP in five Special Woredas of SNNPR: (i) although not institutionalized at Woreda level, the attempt made by the project to introduce bottom-up planning process was taken as one of the best practices which should be scaled-up in other areas and institutionalized at Woreda level; (ii) establishment of mini- database/documentation centre at Woreda level through the provision of various capacity building initiatives was the best practice to be introduced and replicated in other Woredas of the Region and/or Country; (iii) the partnership approach installed to work through local level development partners (WDAs) was found to be best practice so as to promote sustainable development and empowerment of target communities through facilitation of their direct involvement in their own local development concerns; and (iv) documentation of challenges, best practices and dissemination to partners were best practices to be instituted by development actors in their future endeavours to be used as reference and lessons.

Lessons Drawn: The major lessons drawn from the implementation of WCBP include: (i) bottom-up planning approach followed by the project was effective in identification and prioritization of needs and preparation of community development projects which should be scaled-up and replicated in other areas; (ii) local communities could easily be the major actor in their own development endeavours through the provision of pertinent training on participatory resource use planning, implementation, monitoring and evaluation; (iii) the strategy/approach in the implementation of CDF projects should be piloted in modest scale before replicating and scaling-up in larger geographical areas. There is also the need to have clear guidelines and understanding on CDF implementation; (iv) the project has managed to implement its planned activities under various internal and external challenges including poor commitment and support from some of its partners, ambitious plans in terms of volume of activities and geographical coverage and others. This is found to be a good lesson for similar initiative in the future where there is the need for critical assessment during project design/planning and implementation in terms of various aspects of project management including technical, social, institutional, organizational, managerial, financial and economic aspects at all stages of project cycle management; and (v) from the implementation of WCBP, one could learn the need for well defined partnership arrangement and the importance of clearly defined working procedures and principles for different activities to be incorporated under any project. In addition, joint and timely review of project progress by all stakeholders could have positive implication for the success of a project.

MAJOR CHALLENGES/CONSTRAINTS/PROBLEMS

The major problems encountered by the project include the following: (a) conflict and/or insecurity in some of the Woredas has hampered the efficiency and effectiveness of the project intervention (especially in Burji Special Woreda); (b) price escalation of project inputs, especially of industrial products; (c) frequent staff turnover in both the project and the WDAs; (d) wider geographical project intervention areas and inaccessibility of some project Kebeles, particularly during the rainy season; (e) logistical constraints which were mainly due to delay in purchase of project vehicle and motor bikes which delayed the timely implementation of planned activities. In addition, a project vehicle was taken by flood which hampered the movement of staff. Similarly, flooding of Segen River affected the movement of staff to project sites; and (f) recurrent drought has negatively affected communities’ participation.

______xiv Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR CONCLUSIONS, RECOMMENDATIONS AND THE WAY FORWARD

Conclusions

Although the project was relevant and appropriate in addressing the genuine development problems of the intervention areas, however, the project design was too ambitious where a very wide range of activities were planned to be implemented over vast areas. The final evaluation of the project revealed that the project design and efficiency of the project were relatively given lower ratings as compared to the rest of evaluation indicators which might be related to poor project planning, delay in timely implementation of some project activities, poor project management and others. It was untimely to assess the impact of the project as the implementation of most of the project actions were either completed more recently or not completed until the evaluation period. Similarly, the sustainability of the project was constrained by various internal and external factors. Despite the great effort exerted by the current project team in the implementation of most of the project activities during the last one-year, however, FARM Africa’s WCBP was generally weak in realizing its envisaged overall and specific objectives due to the internal and external challenges.

Recommendations a) FARM Africa should assist its partners (especially WDAs) through availing the required resources to finalize uncompleted and on-going projects in intervention Woredas; b) FARM Africa should assist the WDAs through provision of technical assistance in areas of fund-raising, human resource development, and preparation of on-shelf projects and project design templates to be used in the future; c) WDA Boards in collaboration with its pertinent partners should critically examine the internal and external challenges/constraints hampering efficiency and effectiveness of WDAs and look for possible remedies; d) All concerned actors at Regional and Woreda levels should assist and promote the scaling-up/replication of the Bottom-Up Planning Approach based on the lessons drawn from the implementation of FARM Africa’s WCBP; e) There is the need to exert greater effort to regularly and timely update the Woreda Database System and proper use of information centres which were established through the project action; and f) All concerned government actors at different levels should strictly follow-up, monitor and support the sustainable use of the project outputs including economic and social services and facilities;

The Way Forward

The following suggestions are forwarded for urgent actions/consideration by FARM Africa, Regional partners (especially BoFED), WDAs and respective Woreda Administration.

 The Project Holder (FARM Africa) should: (i) seek ways for the proper completion of the remaining activities including the consolidation and handing over of the project initiatives; and (ii) urgently initiate, jointly with BoFED, the development of action plans for implementation of the remaining activities.  Regional BoFED should: (i) closely follow the timely development and approval of the action plan jointly with FARM Africa for implementation of the remaining project activities; (ii) provide the required technical and administrative support such as the approval of the project extension and the use of project facilities.  The WDAs in the Five Special Woredas should: (i) urgently call meetings of the board members and the general assembly to report their current situations and discuss on critical issues which need urgent actions; (ii) urgently develop mechanisms for fund raising actions; and (iii) organize urgent meeting with SEPDA to discuss on their future partnership arrangements and funding opportunities.  The Five Woreda Government Administrations should (i) provide the required support through its concerned sector offices, and if necessary, the temporary deployment of the required management staff and experts to the WDAs; and (ii) consider funding of WDAs projects from government capital budget sources if there is any possibility.

______xv Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR 1. INTRODUCTION

1.1 Overview of the Project

FARM-Africa is an international Non-Governmental Organisation (NGO) which aims to reduce poverty in Eastern and South Africa through working in partnership with marginal farmers and herders, helping them to manage their natural resources more effectively and build sustainable livelihoods on their land. It has a vision of a prosperous rural Africa which aims to make this a reality by being innovative and practical, being effective by working with local people and their institutions rather than building parallel structures, and bridging the gap between researchers and rural communities, ensuring that its research addresses practical problems facing communities.

In the last five years, FARM Africa has been implementing a project entitled "Woreda Capacity Building Project (WCBP) in Five Special Woredas of SNNPR". The project was funded by the European Union (75%), CordAid (15%) and FARM-Africa (10%) with the aim of reducing poverty in the most vulnerable households by strengthening key institutional structures and building the capacity of local government offices and NGOs, particularly Woreda Development Associations (WDAs), which could promote sustainable development at Woreda level. The project aimed to work with and develop the capacities of a range of Woreda-level organisations, including Woreda government offices and WDAs, and also to establish and develop grassroots community organisations at Kebele and sub-Kebele level, namely Kebele Development Associations (KDCs) and Village Development Associations (VDCs). Once strengthened, it was anticipated that these organisations would be able to work together to plan, implement, monitor and evaluate a range of development activities in the Woreda. An assumption of the project was that funds from the World Bank Food Security Project and from other NGOs would soon be made available to WDAs to carryout development activities in the Woredas, and thus, building the capacity of the WDAs beforehand would increase their effectiveness in these activities.

The project also focused on developing a ‘bottom-up’ participatory planning system which would include everybody’s voice in community development, and in developing skills and expertise in project implementation through co-implementing a range of micro-projects and then establishing a Community Development Fund (CDF) which would fund further projects to be planned and implemented by the community. In this way, it was envisaged that all stakeholders in local communities would be empowered to carryout their own sustainable development, and thus, alleviate poverty and increase the livelihood base in the local community. The project initially envisaged that once developed, these local organisations would also be able to develop an early warning system that would pick up crises before they happen and facilitate community’s preparedness for the recurrent droughts that plague most of these Woredas. Accordingly, the project was initially designed to realize five objectives, referred to as ‘outputs’ in the original project document. These were: a) Five strong locally accountable WDAs built that are able to lead a process of locally initiated participatory development, one in each of the five Woredas. b) The capacity of WDAs, VDCs and local government staff built to develop, implement and monitor and evaluate a participatory planning system enhanced. c) Increase the livelihood base of the local community and most vulnerable households in each of the five Woredas. d) A model of community based early warning for emergency and identification of community infrastructure projects for Employment Generation Scheme (EGS0. e) A model of Woreda level capacity building developed and disseminated to donors, government, NGOs and regional networks.

The WCBP commenced its operation in 2004 after signing project agreement with the European Union (EU) in July 2004 and implementation agreement with the Regional government in September 2004. The project

______1 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR actually started its implementation during the first and second quarters of the year 2005. The Mid Term Review (MTR) of the project was took place in August 2006. The review identified numerous challenges from the very start due to design problems in the initial proposal, assumptions that did not materialise, management problems within the project team and external challenges. As the result, these challenges and constraints led to significant delays in achieving project outputs. External challenges faced by the project include inaccessibility to numerous target Kebeles due to flooding of the Segen River on the road to Burji Woreda and inadequate road infrastructure in Amaro and Burji Woredas. Some challenges were also faced in working with Regional and Woreda authorities in terms of high staff turnover and lack of responsiveness, especially government officials were too busy with other duties. On the other hand, internal management issues also played a part in constraining project progress including overambitious project design, weak staff capacity and management problems within the project team. Furthermore, lack of clarity on implementation modalities in the original project document led to confusion among project staff. These factors, combined with delays in recruiting well qualified and experienced staff, recurrent turnover of project staff and lack of sufficient vehicles also contributed to the delays.

Accordingly, the MTR provided recommendations to streamline and simplify the project so that it was a good chance for achieving its goal. Several management measures were undertaken by FARM-Africa and the EU in order to critically review progress and streamline the project so that it could achieve its intended outputs. In addition, a review of the project was carried out by an external consultant in February 2008 including recommendations for the way forward and developing a realistic action plan for the remaining timeframe until July 2009.

The findings and recommendations of the review exercise were presented to FARM-Africa and EU management teams and formed the basis for the second amendment of the project. In line with this, a project amendment workshop was held during March/April with participants representing the EC Delegation to Ethiopia, Woreda Administrations, WDAs’ Board and Management, Regional Bureau of Finance and Economic Development (BoFED), Bureau of Agriculture and Rural Development (BoARD), the Southern Ethiopia Peoples Development Association (SEPDA) and FARM-Africa staff. As the result, the amendment proposal was prepared based on the consensus reached among key stakeholders regarding what could be achieved in the remaining timeframe given the prevailing external and internal contexts. The main objective of the amendment was to focus down on project deliverables (outputs, activities and locations) to concentrate on the areas where the project has a good chance of achieving an impact in the time remaining. The reason for the amendment were mainly to ensure the proper accomplishment of those activities initiated with some delay as well as for the proper consolidation and the smooth handing over of the project imitative including to ensuring their continuity and sustainability. The amendment involved revision of some of the initial envisaged activities and inclusion of some complementary activities, and reallocation of some budget items which were found necessary towards achieving the intended results of the project.

Finally, the outputs of the project were revised in the amendment document as follows:

Output 1: Five strong, sustainable and locally accountable WDAs built that are able to implement a process of locally initiated participatory development Output-2: The capacity of relevant Woreda government offices, WDAs, KDCs and VDCs built to develop, implement, monitor and evaluate a participatory planning system Output-3: Increase the livelihood base of the local community and most vulnerable households in each of the five Woredas. Output-4: Write up lessons learnt from the WCBP and disseminate to donors, government, NGOs and regional networks

This final evaluation for the project was conducted in July 2009. Although separate reports are prepared by the Independent Consultants and Regional Counterparts, the evaluation was conducted jointly by a team of experts deployed from Regional BoFED and External Consultants from Rural and Pastoral Consultant (RUPA) consulting firm. The evaluation exercise was mainly focused on assessing the design, relevance,

______2 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR efficiency, effectiveness, impact and sustainability of the project actions. The evaluation has also identified major Strengths, Weaknesses, Opportunities and Threats (SWOT) of the project. Furthermore, the final evaluation identified best practices to be replicated/scaled-up as well as identified lessons from the implementation of the project. Finally, based on the findings of the evaluation and inputs from the final evaluation workshop, the consultants forwarded some recommendations for consideration by concerned bodies including the project holder (FARM Africa), WDAs and pertinent government partners. This report presents the major findings and recommendations of the final evaluation of the project.

1.2 Purpose of the Final Evaluation

As per the Terms of Reference (ToR) provided by FARM Africa, the final evaluation was mainly focused on assessing how successful the project’s strategy for WDA capacity building and empowerment has been, in order to identify what worked well, what did not work well and how FARM-Africa should approach such work in the future. Therefore, the overall objective of this final evaluation was to assess the attainment of the project objectives against the project log-frame, impact trends, and sustainability. The evaluation framework involved five core areas:

 Relevance: assessment of the objectives of an intervention, particularly regarding their justification in the light of problems and needs.  Efficiency: a measure of how well the resources are used to produce achievements and results.  Effectiveness: the degree to which the objectives of the intervention are fulfilled.  Impact: effects attributable to an intervention.  Viability: the degree to which the desired effects of an intervention last beyond its end.

The Specific Objectives of the evaluation were to: a) Evaluate the achievements of the project against its purpose, outputs and targets and its contribution to the overall objective as outlined in the project proposal; b) Assess the overall project efficiency and effectiveness, and the relevance and coherence of the project components/approaches in addressing the issue of WDA capacity building and empowerment; c) Evaluate the impact of the project on household livelihoods in the 5 special Woredas, and assess the sustainability of project impact; d) Draw best practices and key lessons learnt during project implementation which can be taken forward by FARM Africa or can be taken forward in the next phase project if recommended; e) Evaluate the effectiveness and efficiency of Farm Africa’s planning, management and M&E systems in the delivery of the programme; f) Provide recommendations for future actions by FARM-Africa’s and other actors.

In addition to above, the final evaluation considered each of the key strategies/approaches devised by the WCBP to achieve its purpose: i). The effectiveness and appropriateness of bottom up planning approaches. ii). Review the impact of Community Development Fund against community livelihood/ social and economic benefit of the target community iii). Capacity Building of partners iv). Overall recording/documentation of project interventions v). Partnership approach (project /community partnership VDC, KDC) vi). Overall sustainability of project interventions vii). Consider the specific challenges of the project at five Special Woredas. viii). Visibility actions as per the contractual agreement with the EC

______3 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR From the above key areas, the following Specific Issues for evaluation were identified:

Key Area 1: Bottom up planning approach: Assess the appropriateness of the Bottom up planning approach (preparation of action plans, M&E, etc.)

Key Area 2: Review the impact of Community Development Fund against community livelihood: Assess the strategies and approaches of the CDF Projects, and their potential impacts. Review CDF management and utilization, Assess the impact of CDF in improving the livelihood of target beneficiaries.

Key Area 3: Capacity Building of partners: Assess the capacity building strategies and their impacts with particular focus on WDAs skills and attitude to support projects identified earlier.

Key Area 4: Overall recording and documentation of project interventions: Review File handling/organization of documents soft/hard copies. Review the level of project activity/achievements documentation (monthly, quarterly, annual reports, workshop proceedings, others like leaflets). Review if the overall project best practices, approaches, challenges and lessons in terms of capacity building, bottom up planning and Community Development Fund has been recorded.

Key Area 5: Partnership approach (with Community/GO/NGO partnership): Assess whether the project partnership strategy with Community and government has been appropriate and effective.

Key Area 6: Overall sustainability of project interventions: Financial sustainability - after funding has finished, the project activities are the responsibilities of WDAs and community. Assess if such trends are on line. Assess the overall commitment of WDAs to continue developing and monitoring the projects through its regular budget. Institutional sustainability - review the role of government with respect to community and WDAs as a regulator, advisor and supplier of service. Review the role of FARM Africa as a technical supporter, bringing new ideas, systems and knowledge through training, capacity building and local experimentation for both the community and the WDAs.

Key Area 7: Consider the specific challenges of the project at five sites: Draw out the key progress points of the projects in terms of development context issues (e.g. participation, changes in attitude; changes in practice; sustainability). Draw out the key challenges / blockages to the projects in terms of development issues (access, area /beneficiary targeting, conflict, infrastructure, etc.).

1.3 Evaluation Methodology

During the evaluation exercise, the consultants reviewed relevant project documents (project proposal, mid- term review document, EC mission reports, external review report, project progress reports, etc), conducted field visits, consulted beneficiaries, and undertook discussion with key informants/focus, WDAs (board and staff members), relevant Woreda Offices including Woreda Administration and sector offices, FARM Africa Project Staff in Arba-Minch as well as FARM Africa Woreda Coordinators in Basketo, Burji and Derashe Special Woredas. The consultants also conducted community meetings in selected sites and undertook participatory evaluation of the project. Furthermore, the evaluation team met and discussed with two NGOs including Agri-Service Ethiopia which has been working in a similar EU-funded project in Amaro Special Woreda as well as SEPDA at Hawassa. Based on the data collected, the consultants in collaboration with representatives from Regional BoFED evaluated the achievements of the project against its purpose,

______4 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR outputs and targets and its contribution to the overall objective as outlined in the project proposal, and analyzed the information largely in quantitative and qualitative terms and produced this report1. Specifically, the following approaches were employed in undertaking the final evaluation of FARM Africa’s project in Five Special Woredas of SNNPR: a) Reviewed relevant documents including program proposals, progress reports, study documents, review documents including mid-term review, EC monitoring mission report, and other; b) Preparation of checklist for discussion with various project partners and other actors at village, Kebele, Woreda and Regional levels; c) Met and discussed with various partners and actors at different levels including: . Target beneficiaries (such as beneficiaries of goat and saving and credit scheme), . Focus groups (KDCs, VDCs and other beneficiary groups such as grain marketing, water users, etc), . Woreda government offices including Woreda sector and Administrative offices, . Board and staff member of WDAs in five Special Woredas, . Project staff at Arba-Minch and intervention Woredas, . Agri-Service – Ethiopia (an NGO implementing EU-funded food security project in Amaro Special Woreda), and . Regional umbrella (but independent) Development Association, SEPDA, in Hawassa town; d) Visited sample project implementation sites and activities in Amaro, Basketo, Burji, Derashe and Konso Special Woredas; e) At community level, collected data/information through case studies, direct observation, discussion with community members, interviewing beneficiary households and key informants (Kebele Officials, schoolteachers, Development Agents (DAs), Health Extension Workers, etc); f) Discussed with concerned staff of EC Delegation to Ethiopia (Rural Development and Food Security Section); g) Presentation of findings of the evaluation to the participants of evaluation workshop at Hawassa town to consolidate the field assessment, debrief the stakeholders and validate the recommendations of the evaluation; h) Preparation and submission of draft report based on document review, field assessment and feedback from stakeholders' workshop; and i) Preparation and submission of final report after incorporating comments from the client, FARM Africa Country Office.

1.4 Composition of the Evaluation Team

Despite the preparation of separate reports by the Regional Evaluation Team and External Consultants, the evaluation2 was jointly conducted by a team of two experts from the Regional BoFED and two Independent Consultants who were recruited by FARM Africa from Rural and Pastoral Consultant (RUPA) Consulting Firm. The evaluation team members include:

a) Mr. Solomon Gebre-Tsadik, External Consultant, from Rural and Pastoral Consultant (Team Leader); b) Mr. Behanu Worku, External Consultant, from Rural and Pastoral Consultant (Team Member);

1 Note that although the Consultants (from Rural and Pastoral Consultant Firm) and Regional Partner (BoFED) jointly conducted the final evaluation, however, separate reports are prepared by the External Consultants and Regional Counterpart (BoFED). This report is prepared by the Consultants. 2 After the completion of document review and fieldwork in the intervention Woredas including discussion with project office in Arba-Minch, the evaluation team attempted to analyse and summarize the results of the evaluation and also rated the project in relation to identified evaluation indicators related to design, relevance, efficiency, effectiveness, impact-trend and sustainability using 1-5 value scale where 1 = Very Poor, 2 = Less than Satisfactory/Poor, 3 = Average/Satisfactory, 4 = Above Average/Very Good, 5 = Excellent.

______5 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR c) Mr. Aregawi Tesfaye, SNNPR, BoFED, NGOs' Projects Monitoring, Evaluation & Support Service Provision Officer; and d) Mr. Wondu G/Mariam, SNNPR, BoFED, Expert.

At Woreda level, the evaluation was also conducted in collaboration with pertinent stakeholders in each of the five intervention Woredas including representatives from WDAs (board and staff members), Woreda Administrations and concerned Woreda Sector Offices such as Water, Education, Health, Finance and Economic Development, and Agriculture and Rural Development. In line with this, the list of individuals who took part and contacted during the evaluation process is annexed in the report.

2. Project Design

While assessing the appropriateness and adequacy of design of the project, the evaluation team attempted to review various documents including the initial project proposal and subsequent amendment documents along with the logical framework of the project and other relevant documents. The MTR report, on which the project design had critically reviewed, was also consulted to clearly understand the nature of the project in more detail. Besides, the necessary discussion with pertinent bodies like Woreda stakeholders and community groups was also made to understand the level of participation of these stakeholders in the project design and the subsequent amendment processes. Accordingly, the findings of the document review exercise and the discussions made with concerned bodies are summarized as follows.

It was learnt that the project was mainly designed based on the experience of FARM Africa in the implementation of similar intervention in Konso Special Woreda. The evaluation has attempted to assess the extent to which beneficiaries (and other interest groups/stakeholders) involved in planning and design processes and the findings revealed that the participation of various stakeholders (including Woreda government offices) during design and planning stage was generally found to be unsatisfactory. It seems that the active participation of beneficiaries and other stakeholders was overlooked at initial stage which would be useful to minimize and/or avoid the inherent extreme ambition of the project. In addition, it was noted that the involvement of the Regional counterparts during the project design was mainly limited to appraisal and signing of the project agreement. As a result, the input provided by the Regional project counterparts to improve the quality of the project was observed to be poor and they failed to critically review and appraisal of the proposal.

Therefore, lack of critical appraisal of the project in terms of technical, financial, institutional and operational viability criteria by the donor and the Regional counterparts was also identified as one major underlining constraining factors responsible for the poor quality of the project design and hence the subsequent challenges faced in the course of the project implementation process. In line with this, the evaluation team concluded that the appropriateness and clarity of the project preparation and submission process was generally inadequate, and thus, lost the opportunity to design a feasible project proposal at the initial stage through conducting extensive consultations with all relevant stakeholders.

In line with what is stated above, the evaluation team revealed that the project design was found to be poor in terms of active participation and consultation of relevant stakeholders; lack of internal coherence of the log frame in specification of input-outputs relationship (i.e. validity of means-ends relationships); specification of objectively verifiable indicators; specification of implementation arrangement and managerial structure; clarity and realism of work plan; and realism of risk and uncertainty assessment which are underlining factors for the success of any development project.

On the other hand, the project design was more or less found to be satisfactory in terms of clarity of immediate objectives, specification of target groups and identification of input sources (including contribution of donor agency, implementing NGO and target groups/communities). It was noted the project proposal more or less clearly stated its overall objective and specific objective along with area of intervention. However, it

______6 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR seems that it would be more convenient and practical for operational purpose if the proposal had been clearly disaggregated its specific objectives attached to specific outputs. The initial and amendment proposal documents have also somehow clearly specified the project targets groups.

Participation during Need Identification 2 5 Overall Assessment of the Project Design Project Preparation & Submission Process 2 4 2

Adequacy of Partnerships 3 Adequacy of Consultation with Partners 3 2 2

1 Realism of Prerequisites & Risks for Project Success Clarity of Immediate Objectives & Targets 2 0 3

Clarity and Realism Work Plan Specification of Beneficiaries 2 3

Implementation Arrangements & Managerial Structure Specification of Outputs & Output Targets 2 2 Specification of Inputs: Donor, NGO, Target Groups & Validity of Means - Ends Relationship Community 3 2

Figure 1: Summary of Overall Assessment of Design of the Project

In general, the project design was poor in terms of active participation of stakeholders; validity of means-ends relationships; specification of implementation arrangement and installation of effective managerial structure; clarify and realism of work plan; and realism of pre-requisite and risks for project success. Therefore, though the project design was more or less found to be satisfactory in terms of clarity of objective, specification of target groups, specification of source of inputs, and identification of WDAs as implementing partner, the limitation in the quality of the project design mentioned above together with the management problems observed in the course of the project period were underlined as the major constraining factors which undermined the timeliness, quality and effectiveness of the project implementation process.

3. Relevance of the Project

The evaluation team attempted to assess the extent to which the project addressed the genuine development problems of its target groups including communities and other beneficiaries of the project. The findings revealed that the WCBP was generally found to be relevant in terms of addressing the genuine development constraints of the intervention Woredas. In this regard, the evaluation team ascertained that most of the project interventions were found to be relevant and appropriate to address important development constraints/problems in terms of promotion of social and economic services and facilities as well as livelihood security of the target beneficiaries. Accordingly, the project was rated as very good in all Woredas with the exception of Derashe Woreda where some of the most important needs of the local communities were missed including water development but concentrated in least priority needs such as construction of toilet, slaughtering houses, and store. Here, it is worth mentioning that the selection of the CDF project activities in all Woredas including Derashe were mainly determined by the KDC/VDCs, and these community level organs were the major responsible bodies for the selection of CDF project activities though WDAs and WCBP were expected to appraise CDF projects before endorsement.

On the other hand, despite a few cases of constraints, the evaluation team found that the project was in general relevant and appropriate in addressing the important development needs of the local communities in

______7 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR most of its components and activities, and thus, rated as good and satisfactory where most of the implemented project activities were found to address important problem of the target communities. In most of the cases, the project was also found to be relevant and appropriate in terms of coherence with existing national development programmes in most of the visited intervention areas. In line with this, the approaches and strategies employed by the project were relevant and in line with government’s strategy/programme of decentralization, capacity building, livelihood and food security initiatives.

Assessment of Relevance of the Project

Addressing Genuine Development Problem 4 5 4 How well the project provide a Cost- Overall Assessment of Project 3 Effective response to the Genuine Relevance (Appropriateness) 2 Development Problem 3 1 2 0

Realistic Expectation of Project Outputs to Continue to be Used once the Project Coherent with National Program was Completed 4 3

Reasonable Expectations that Adequate Resources be committed during or after Project Phase Out 3

Figure 2: Overall Assessment of Relevance of the Project

On the other hand, the relevance of the project was evaluated in terms of how well the project provided a cost-effective response to the development problems in intervention Woredas. Accordingly, the evaluation team noted that the project was poor in responding to the development constraints in a cost-effective way which might be mainly due to poor management and delay in execution of the development intervention. As per the understanding of the evaluation team members, the project would be rather cost effective if it had been implemented the CDF project activities through direct provision of the funds to KDCs/VDCs, and if it had initiated most of the activities in timely manner. As learnt from the assessment made in the course of the evaluation process, the major factor which significantly inflated the cost of CDF activities was the high price escalation of construction materials during the lifespan of the project. Moreover, direct provision of funds to KDCs/KDCs would significantly reduce the overhead cost of the project which were spent for procurement and delivery of materials and other management costs. On the other hand, an alternative approach of linking some activities like area enclosure, nursery management and some other CDF activities directly to pertinent sector offices and integration with existing development programs like Productive Safety Net Program (PSNP) would enhance the cost-effectiveness of the project and quality of project outputs. Moreover, the administrative and overhead costs incurred during the five years period of the project was somehow considered as significantly high compared to the actual quantity and quality of the output obtained under the four project outputs.

For the reasons stated above, although the project was highly relevant in addressing the development problems of the intervention areas, however, it was considered as poor in providing a cost-effective response to the identified and prioritised development problem. In line with this, most of the informants at Woreda and community level stressed that the project was appropriate and relevant but could not met their expectations in terms of quantity, quality and use of project outputs.

______8 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR

The relevance of the project was also evaluated in terms of expectations that adequate resources could be committed to the project for meaningful follow-up after the termination of the project. The results revealed that the necessary resource for the future management and use of most of the interventions was expected to be available even after the project phase out. This is true particularly for project outputs which were directly linked with concerned government sectors offices including projects related to education, health, water and the like. On the other hand, there are some level of uncertainty concerning sustainable management and use of some outputs mainly for project outputs for which there is poor linkage with Woreda government offices such as grain mill, slaughterhouse, grain bank, public toilet and others.

In general, despite certain shortcomings in cost-effectiveness and resource availability for future management of some project outputs, the evaluation team found that the project was more or less satisfactory and appropriate in terms of its relevance in addressing local development needs and coherence with national development program. Therefore, as it is illustrated in Figure 2 above, the overall relevance and appropriateness of the project strategy was rated as satisfactory.

4. Project Efficiency

It was attempted to assess the efficiency of WCBP in terms of how economically resources/inputs (funds, expertise, time, materials and other resources) were converted to outputs/results. In line with this, the evaluation team has assessed project efficiency in areas of Beneficiary Targeting Procedure and Applied Criteria, Spatial Resource Allocation, Project Management, Timeliness, Cost-Effectiveness and consideration on Gender Aspects.

4.1 Beneficiary Targeting Procedure and Applied Criteria

The evaluation team attempted to identify the major target group of the project and to assess the targeting procedure and applied criteria used for the selection of the project beneficiaries. In this respect, it was identified that the major targets of the project include various groups/institutions (such as Woreda sector offices, WDAs, KDCs, VDCs, users of grain bank, slaughterhouse, grain mill, etc) and local communities who are expected to be benefited from the construction of schools, health posts, veterinary posts, water development schemes, grain mills, etc as well as poor and vulnerable women households who were provided with goats and beneficiaries of saving and credit schemes in some of the intervention Kebeles of Burji Special Woreda.

Overall, beneficiary targeting and applied criteria were found to be satisfactory and fair in most of the project interventions. In other words, for most of the project activities, there was participatory targeting process through involving the local communities and in the presence of project staff and staff of concerned government offices. Accordingly, it was learnt that beneficiary targeting process in Konso, Amaro and Burji Special Woredas was relatively better in terms of participation and community representation compared to

______9 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR that of beneficiary targeting in Derashe and Basketo Special Woredas (see Figure 3). Specifically, FARM Africa was strong in targeting of poor women in the provision of goats.

Assessment of Beneficiary Targeting & Applied Criteria

Amaro 3 5

3 3 Average Basketo2 2

4 Konso Burji 3

Derashe 2

Figure 3: Overall Assessment of Beneficiary Targeting and Applied Criteria

On the other hand, it was revealed that the process of beneficiary targeting at grassroots level for some activities was not clear for the evaluation team. For instance, there was no guideline on how and who should directly be targeted for saving and credit schemes, slaughterhouse, grain mill and grain banks. This in turn may put the proper use and management of these project outputs in the future at a stake unless appropriate consolidation activities are done to institutionalise the ownership and future management of these output. To be specific, the evaluation team identified the following weaknesses and/or constraints in relation to beneficiary targeting and applied criteria:

. It was learnt from discussions with project staff and Woreda counterparts in Burji Special Woreda that beneficiaries of saving and credit scheme were the most vulnerable members of the community. However, there was no criteria and/or guideline on how and who should be targeted on the scheme. There were cases where some better-off households were targeted based on their capacity in terms business skills in Tinishua Qeyate Kebele of Burji Special Woreda.

. Although the construction of slaughterhouse were found to be beneficial for the local communities in Gatto Kebele of Derashe Special Woreda to have clean and safe meat, it was, however, found that it was not clear how and who should directly be targeted to utilize and manage the facility.

______10 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR . Despite the construction of grain banks (storages) in Gatto Kebele of Derashe Special Woreda as well as in Medita and Obcha Kebeles of Basketo Special Woreda, however, there was no evidence that whether the target beneficiaries were the poor segment of the community or better-off households.

. The evaluation team noted that there was no clear target group to directly use and manage the grain mill constructed in Shela-Kanabola Kebele of Basketo Special Woreda. Many informants in the Kebele indicated that the grain mill planned to be owned and managed by the Kebele Administration. Such issue was raised during debriefing and feedback section with project stakeholders at Woreda level so that the Woreda partners were ready to design appropriate strategy to have ownership and sustain the grain mill service.

. During implementation, the project has lost the opportunity to link its Natural Resource Development intervention in Amaro Special Woreda with existing ongoing government development interventions like PSNP and this shortcoming in integration prevented the potential complementarity of the various interventions and the reinforcement among themselves. For instance, the project has lost the opportunity to link its natural resource development activities with other similar intervention through complementing and targeting PSNP Public Work (PW) beneficiaries such as Dereba (1,086 PW beneficiaries) Kebele in Amaro, Lemu Kebele (1,300 PW beneficiaries) and Kilicho Kebele (830 PW beneficiaries) in Burji, and Busa-Baso Kebele (541 PW beneficiaries) and Gatto Kebele (1,667 PW beneficiaries) in Derashe Special Woredas so as to ensure the sustainability of the interventions. Such a constraint would be avoided or minimized if there were adequate coordination and consultation among the parties at Woreda level whereby the intervention would be complemented to and integrated with PSNP to create greater synergy.

In general, despite the above targeting constraints, the overall beneficiary targeting process and applied criteria was found to be fair and satisfactory, and thus, it was rated as average (a value of 3) in a value of one to five value scale ratings where: 1 = Very Poor; 2 = Less than Satisfactory; 3 = Average/Satisfactory; 4 = Above Average/Good; and 5 = Excellent.

4.2 Resource Allocation and Spatial Aspects

In relation to resource allocation and/or spatial aspects of resource allocation, the evaluation team noted that there was poor capacity assessment of the WDAs in different Woredas where there were more or less similar resource allocation for different Woredas despite heterogeneity of the WDAs in terms of capacity and the need for external support. At initial stage, the determination of the number of operational Kebeles in the five Special Woredas (i.e. equally five Kebeles per Woreda) and the allocation of more or less equal number of CDF projects per Kebele seem arbitrary without considering the variation among the target Woredas with regard to the abovementioned factors including total number of Kebeles available and the size of their population. For instance, there might be the need to provide more emphasis in resource allocation during initial planning for Basketo Special Woreda due to its poor social and economic development, poor access to external resources, relatively poor capacity of development actors, and its remoteness and/or inaccessibility. Although the project reported the implementation of more CDF projects in Basketo Special Woreda, however, many of them were not completed during the evaluation period mainly due to shortage of funds and price escalation of construction materials.

On the other hand, it was noticed that there was inadequate resource allocation for livelihood security in terms of provision of two goats per household which was found to be very minimal compared to other similar intervention in the Region such as the provision of up to 4,000 Birr per household by the Regional government to enhance food/livelihood security of poor households. In line with this, FARM Africa indicated that the provision of two goats per household was deemed appropriate for introducing goats to these households and for them to increase their herd gradually whilst learning about appropriate fodder provision and accessing CAHW support. In addition, the NGO stated that learning from evaluations of other FARM

______11 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Africa projects suggested that the provision of two goats was sufficient to significantly increase household assets and enabled WCBP to reach a large number of beneficiaries within the project timeframe. According to the project holder, what matters is not disbursing huge amount of money which is not necessarily used for investing in assets leading to long term improvements in household food security. The intention was to make most of the resources available in being able to provide assets to as many poor households as possible. On the other hand, the evaluation team has the opinion that given lack of other complementary livelihood intervention under the project and/or other projects and the need for relatively more resources to enhance the household livelihood status in tangible manner, the provision of two goats per household is found to be very minimal.

The other major shortcoming observed concerning the resource allocation of the project was the failure to allocate the necessary financial budget during project amendment for the completion of CDF project activities which were initiated before the MTR of the project. On the contrary, the project holder made the de-commitment of Euro 130,453 from the initial total budget of the project during the amendment. The price inflation of construction materials in recent years has also negative impact on project funding. As a result, some CDF projects initiated before the mid-term review were left incomplete and were not in a position to render effective service to the target communities.

Resource Allocation/Spatial Aspects

Amaro 4 5 4 3 Average Basketo 3 2 2 1 0

Konso Burji 3 4

Derashe 2 Figure 4: Assessment of Resource Allocation in the Intervention Woredas

On the other hand, the evaluation team identified some constraints in resource allocation to CDF micro- projects. For instance, the WDA in Derashe Special Woreda was found to be weak in allocation and disbursement of about Birr 1,000 for the purchase of teff straw for wall plastering of a school block in Gatto Kebele as the result of which the rehabilitation of the school block was delayed for more than two years which even was not completed until the evaluation period. In addition, the criteria for determination of the level of seed capital (revolving fund) allocated to each saving and credit association in Burji Special Woreda was not clear. As it is illustrated in Table 1 below, the allocation of seed capital seems to be determined based on the amount of saving capital rather than on the number of members of SCA.

Table 1: Resource Allocation for Saving and Credit Associations in Burji Special Woreda Name of Saving and Members Total Capital and Loan released from CDF No. Credit Association Male Female Total Saving in bank in Birr in Birr 1 Berek 90 66 156 14,870 80,000 2 Gamiyo 77 5 82 6,750 45,000 3 Tinishua Kayate 66 18 84 5,420 40,000 Total 233 89 322 27,060 165,000

______12 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR For instance, the amount of seed capital given to Gamiyo SCA is greater by 5,000 Birr than that of the other SCA in Tinishua Kayate in the same Woreda despite the fact that the number of members for Tinishua Keyate SCA is slightly higher than the Gamiyo SCA. This difference in seed capital allocation seems the outcome of the difference in the saving capacity of the SCA since Gamiyo SCA has slightly higher saving amount compared to the saving amount of Tinishua Kayate SCA. Similarly, Obcha and Medita Grain Bank beneficiary groups in Basketo Special Woreda have received Birr 20,000 and Birr 50,000 Birr, respectively, although both groups were engaged in similar intervention but found in different Kebeles. Such discrepancies might be related to lack of guidelines and working manuals in the implementation and management of CDF micro-projects. This might also contradict with poverty focus of the project where the poor segment of the community members might save less which in turn affected the amount of loan received from the revolving fund (see Table 1 above).

In general, it was observed that the amount of fund allocated for most of the CDF activities was minimum compared to the requirement to complete the activities. The recent price escalation has also aggravated budget constraint which in turn affected quality and quantity of some CDF activities and even resulted in problems of completion of some CDF activities before the end of the project period particularly in Derashe Special Woreda. Despite the above constraints in allocation of resources, however, in most of the cases, the spatial aspect of resource allocation was found to be more or less satisfactory in most of the intervention Woredas.

4.3 Project Management

4.3.1 Project Inputs

FARM Africa signed the project contract with European Commission with a total budget of €1,462,129. EC contribution was 75% of the total budget (€1,096,597) while CordAid and FARM Africa agreed to contribute 15% (€219,319) and 10% (€146,213), respectively, of the total budget. In line with the recommendations of the EC mission in March 2006, the MTR in August 2007 and External Review in Feb/March 2008, the project went through two amendments to streamline and simplify the project interventions with cancelling of some of the initial envisaged activities found to be too-ambitious and not feasible. The project initial 48 months period extended with 12 months no-cost extension granted i.e. up to July 2009 and reallocation of some budget items made with de-commitment of Euro 130,453 from the initial total budget of the project as per the request of FARM Africa. As a result, the total budget of the project was reduced to €1,331,676. It was revealed that the donor disbursed the planned budget of the project as per its commitment with significant delays especially after the MTR which called for revision and amendment of some of the project activities and budget.

Table 2: Assessment of Project Inputs in the Intervention Woredas Project Inputs Amaro Basketo Burji Derashe Konso Average 1. Budgetary Disbursements (Donor/ WCBP) 2 2 2 2 2 2 i.e. amount and timelines 2. Project Personnel (FARM Africa) including 2 2 2 2 2 2 consultants, i.e. quality and quantity 3. Equipment & Supplies (FARM Africa) i.e. 2 2 2 2 2 2 quality, quantity and timeliness 4. Personnel/Experts (Government) i.e. quality and quantity 3 2 3 2 4 3 5. Community Contribution (in cash and/or in kind) 5 4 4 2 3 4 6. Other Government's Input 4 3 3 2 4 3 Overall Assessment of Project Inputs 3 3 3 2 3 3

______13 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR During the evaluation exercise, it was noted that there were only two technical staff (including Project Coordinator and Training Officer) to run the project actions at project office in Arba-Minch. It was also learnt that the country office has transferred its Woreda Coordinator in Amaro Special Woreda to its other project in the Region so that the position has been vacant in the last few months. In general, the project has been constrained with poor staffing.

In the last five years, the project employed a total of 30 technical and support staff including replacements for those terminated by and resigned from the project. Out of the total staff members, about 50% (15 staff) left the project through termination (three staff members), resignation (11 staff members) and transfer to other project (one staff member). There was frequent turnover of project coordinator post where four coordinators were employed by the project during the last five years. The detail of project staffing in the last five years is summarized and annexed in the report. It is interesting that only one technical staff (Training Officer) who has been working since the start of the project period. On the other hand, it was also learned that the strategy of using two mobile outreach teams at initial stage was found to be ineffective so that the project holder revised such arrangement so that only one project office was decided to be stationed at Arba- Minch town. Despite such constraints in terms of project personnel, the evaluation team noted that the project staff has been effective in the implementation of so many project activities since the deployment of the last project coordinator in September 2008.

As learnt from the discussions made with the project team and the WDAs in the five project intervention Woredas, the timeliness of the resource disbursement from the donor to the project as well as from the project to respective WDAs was constrained with some limitations and mentioned as one of the underlining factors for the delay of project progress. The delay for the budget release from the donor’s side was specifically related with the second payment of the project which was delayed by more than seven months after the MTR exercise. In addition, the delay in disbursement of the financial resource for CDF activities to WDAs was mainly resulted due to the stringent financial procedure followed by the project holder for the settlement of CDF budget utilized by WDAs and the internal management problem of the project office of FARM Africa at Arba-Minch and the Country Office at Addis Ababa.

It was revealed that the project has provided various equipments, materials and logistical resources as per its plan of actions. The quality and quantity of most of the equipments and supplies provided through the project (such as computers, office equipment and other materials) were found to be adequate. However, the evaluation team ascertained that the quantity of goats provided to poor women to improve their livelihoods were found to be very low in terms of bringing significant change on the livelihood of the beneficiaries as compared to other similar intervention implemented by the local government including support for PSNP beneficiaries through other food security project/programmes. In line with this, the project holder (FARM Africa) did not agree with this finding of the evaluation team where it is inappropriate to compare the introduction of a livelihood asset scheme intended as a demonstration project to the WDAs, with the government PSNP programme which is operational on a totally different financial and human resource scale and with different objectives. On the other hand, the evaluation team indicated that despite its positive impact in terms of increasing household income and asset holding, the quantity of inputs for household livelihood promotion particularly the provision of only two goats per household to poor women to improve their livelihoods were found to be very low to bring about significant change at household level (see also the section 4.2 Resource Allocation and Spatial Aspects).

On the other hand, the quantity of equipments supplied through CDF micro-projects was also insufficient to effectively realize the local needs which partly due to shortage of project budget and price escalation in the country. For instance, there were cases where some of the trained water caretakers and Community Animal Heath Workers (CAHWs) were not provided with working toolkits. Although the project holder transferred financial resources to WDAs, however, some of the WDAs did not buy and provide the toolkits as some WDAs haven’t purchased toolkits due to poor planning, shortage of financial resource and price inflation. For the same reasons, most of the construction infrastructures in Basketo Special Woreda were also incomplete partly due to material/supply constraints.

______14 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR

Assessment of Project Inputs

Budgetary Disbursements (Donor / WCBP) 2 5 4 Overall Assessment of Project 3 Project Personnel (FARM Africa) Inputs 3 2 2 1 0 Equipment & Supplies (FARM Other Government's Input Africa) 2 3

Community Contribution Personnel (Government) 3 4

Figure 5: Overall Assessment of Project Inputs

Despite the need to work in close collaboration with concerned government sector offices, the project has been implemented most of its activities without properly linking them with the sector office (especially in Basketo and Derashe Special Woredas). As a ,ARM Africa project office together with respective WDAs were the direct responsible bodies for the overall implementation of most of the activities implemented through CDF such as water, heath, education and other activities. For instance, the nursery sites in Derashe and Konso Woredas were not linked with respective Woredas Agriculture and Rural Development Offices. The same is true for goats provided to women in almost all Woredas. In addition, the project was poor in appropriately transferring the responsibility of future management of project outputs through an official handing over arrangement of project outputs to pertinent government offices after their completion such as provision of goats, water schemes, slaughterhouse, grain mill, grain banks, nursery sites, etc.

On the other hand, there were cases where the government staffs at different levels and WDAs have participated in various trainings and exchange visits. Similarly, except for Derashe Woreda, the contribution of the government partners in most of the cases was found to be satisfactory especially in Konso and Amaro Special Woredas. For instance, Amaro Water Resource Development Office has provided a water tanker with a capacity of 10,000 litres for the implementation of spring water development in Gumure Kebele. Konso Special Woreda has also provided the Konso Development Association with sand collection sites to support the income generation initiatives of the WDA. Similarly the WoARD of Basketo Special Woreda has supplied its tractor to assist CDF construction activities.

In most of the cases, community participation in terms of cash and in kind contribution was found to be encouraging. In relation to this, the local communities in Amaro, Basketo and Burji Special Woredas were specifically very strong in the provision adequate support for the project through the delivery of locally available construction material and unskilled labour. Such contribution was relatively found to be poor in Derashe Special Woreda.

4.3.2 Internal Project Cycle Management and Working Relation with Other Actors

The evaluation attempted to assess the internal management and working relation with its partners and other actors in terms of work planning, monitoring and reporting; coordination and relation with other actors/partners; management flexibility in response to problems and changed circumstances; and capacity of

______15 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR project team to run the project. Accordingly, the project was generally found to be poor in project management and working relation with actors and its partners. Although there have been some improvements over the last one year, the project had been weak in terms of work planning and monitoring of the project activities in its initial four years period. It was noted that there was ambitious planning at initial stage of the project. The project was also poor in terms of monitoring of its outputs related to provision of goats and most of the micro-projects implemented through CDF. Most of the outputs of micro-projects, which were implemented before a year, were not monitored to know their status and assist the continuation of the intended impacts of the interventions on the target beneficiaries.

Overall, the project was poor in terms of systematic monitoring and follow-up and consolidation of the project initiatives. It has also lacked logical sequence in planning and implementation of some of the project activities. For instance, it would have been more beneficial if most of the trainings provided in the last one year for the Woredas offices and WDAs were organized at initial stage of the project period which would help to monitor their effectiveness and assist in the realization of the intended impacts. For instance, although the implementation of CDF activities has been underway since 2008, however, CDF guideline was finalized at the end of the project period (June 2009). On the other hand, many activities related to building the capacity of WDAs were implemented in the last one year which has lost the opportunity to allow the required follow-up and monitoring of the outputs of the initiatives.

On the other hand, the findings indicated that the involvement of local stakeholders (such as the WDAs, government sector offices and target communities) in the decision-making process of the project particularly in financial issues as well as revision of activities and budget was minimal which mainly handled by the project holder. In line with this, most of the WDAs complained that even after some major decisions made by the project holder, other project partners including WDAs were rarely communicated about the changes and revisions on budget and activities which in some cases resulted in misunderstanding between the project holder and its partners including WDAs.

Assessment of Internal Management & Working Relationship with Other Actors

Project Work Planning, Monitoring and Reporting 2 5 4 Overall Assessment of Internal 3 Coordination & Relation with Other Actors Management & Working Relationship 2 3 2 1 0

Management Flexibility in Response to Effectiveness of Woreda Coordinators Problems & Changed Circumstances 2 2

Capacity of Project Coordinator(s) and Technical Staff to run Project Management 2

Figure 6: Assessment of Internal Management & Working Relationship with Other Actors

The evaluation team ascertained that the project was weak in terms of coordination and management of the project intervention which mainly due to poor commitment and weak project management capacity as well as the frequent staff turnover including the replacement of four project coordinators in the last five years of project life. There had also been unhealthy relationship among the project staff which has negatively affected

______16 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR the efficiency and effectiveness of the project. Similarly, the evaluation has shown that Woreda Coordinators and other technical staff of the project were not effective in the realization of the project outputs.

Despite the constraints in relation to internal management, the project has more or less good working relationship with its partners at Woreda levels including WDAs and Woreda sector government offices. Most of the capacity building initiative of the project have participated the local actors during planning and implementation. On the other hand, it was noted that the project stakeholders were weak in timely solving the problem prevailed between the newly established Community Based Institution (CBI) in Amaro Woreda known as Dicho-Tulto which has been supported by Agri-Service and Kore Development Association which was assisted through FARM Africa’s WCBP. The evaluation team noted that there had been competition in terms of membership and disagreements in the use of stamp and receipts which had affected the relationship between the two CBOs which latter on called for signing of partnership agreement in January 2009 between FARM Africa/WCBP and Agri-Service Ethiopia to maintain good relations. It was also noted that there was poor coordination in terms of facilitating the linkage of the different project intervention with pertinent government sector offices. Accordingly, the project would have to facilitate the linkages with Woreda offices during planning, implementation and monitoring of the project intervention so as to ensure effective implementation, smooth handing over and phasing out for enhancement of project sustainability.

The key problems which affected the timely and proper implementation of the project actions were mainly related to the weak management and coordination both at the project field team and the Farm Africa country office levels including the high staff turnover and also there had been unhealthy staff relationship and poor project staff commitment. Furthermore, after five years of the project support, the five WDAs may still not able to sustain their functioning after the phasing out of the project.

4.3.3 Support/Inputs to Project Management from Country Office and External Bodies

It was attempted to assess the level of technical and administrative support/inputs delivered for the project management at field level from different actors including the technical and administrative support provided by FARM Africa Country Office, the donor agency (EC Delegation to Ethiopia), government partners and external consultancy/advisory bodies. Accordingly, the findings revealed that the support provided by the EC Delegation in terms of monitoring and technical assistance was found to be satisfactory whereby the donor agency has been conducted regular monitoring exercise and provided feedback for necessary corrective actions. The donor agency was especially strong in closely following-up of the progress of the project after the MTR which has continuously been communicating the project coordination office on bi-monthly basis through receiving updates on the project progress and provision of the necessary feedback and constructive comments.

On the other hand, the external supports provided for the project by external consultants through conducting MTR and the documentation of the project’s experience and lesson was found to be useful in the provision of valuable recommendations which were helpful for managerial decisions and for the dissemination of the project experience to project stakeholders and other interested bodies. Accordingly, the MTR identified various critical issues which assisted the project management to amend its budget and activities. The input provided by an external reviewer in February/March 2008 was also crucial and contributed for the progress achieved so far in the last one year.

______17 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Assessment of External Support/Input to Management

Technical Support by FARM Africa Country Office 2 5 4 Overall Assessment of External 3 Administrative Support by Support 3 2 FARM Africa Country Office 2 1 0

Monitoring & Technical Support Evaluation & Review Processes by Donor Agency 4 3

Management Support by Government 2

Figure 7: Assessment of External Support/Input to Management

On the other hand, the technical and administrative support provided by FARM Africa Country Office to the project field office was found to be poor. For instance, the termination of the contract of most of the technical project staff by the Country Office at a time was devastating in terms of losing institutional memory of the project. The Country Office was also poor in terms of replacement for Program Manager (PM) who resigned in September 2008 (no PM since September 2008) which has negatively affected the effectiveness of the project in terms of timely completion and consolidation of the project actions. This has also adversely affected the realisation of the planned coordination of the project efforts with project partners including Regional government Bureaus and SEPDA. Accordingly, due to this limitations and associated constraints, the project could not benefit from the intended coordination and other technical support to be obtained from its Regional and Woreda government counterparts.

Although almost all of the Woreda Administrations in the five special Woredas have good willingness to support and strengthen the WDAs (e.g. through paying salaries of WDA Managers), however, there were cases where WDA managers were also engaged in other government assignments (as a secondment) since they were paid by the government. In other cases (such as Basketo Special Woreda), the WDA managers were frequently transferred to government offices leaving their respective WDAs which in some cases have affected the institutional autonomy of WDAs. It was also noted that most of the WDAs were constrained by poor commitment of members WDA board and WDA staff. Similarly, despite the efforts made by the project holder to link WDAs with SEPDA, however, there was no major support to strengthen the capacity of WDAs by SEPDA.

In line with what is stated above, the level of technical and administrative support delivered by the FARM Africa Country Office to the project and the project management support obtained from other project stakeholders (such as Woreda government counterparts, WDAs and SEPDA) was found to be below satisfactory while the technical support the project benefited from the review and follow up exercise of the donor and the technical support obtained from external consultancy bodies were rated as satisfactory and above average, respectively. In general, despite the constraints stated above, the overall efficiency of the project in benefiting from the project stakeholders outside the project field office in terms of technical and administrative support was more or less found to be satisfactory (see the Figure 7 above).

______18 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR 4.4 Timeliness of the Project Intervention

It was attempted to assess timeliness of the project in terms of timely implementations of activities including how well the different activities of the project implemented at the right time and according to the project plan of implementation. In this regard, almost all the project stakeholders contacted during the evaluation exercise including WDAs, Woreda government sector offices and community groups stressed that significant delays occurred in initiating the project actions at the beginning of the project period as well as after commencement of project implementation. In general, the project was found to be very poor in terms of timely initiating and implementation of most its plan of actions. The project was also constrained by poor logical sequence of implementation where some of the capacity building activities were implemented at the end of the project period though such capacity building initiative should have been planned and implemented at initial stage of the project period to allow them the required follow-up so as to ensure sustainability of outputs and associated impacts.

Among others, the major factors which mainly affected the timely initiation and implementation of the project include, but not limited to, the following:

 The unrealistic organizational structure initially setup by FARM Africa (i.e. the establishment of two mobile outreach teams stationed at Arba-Minch and Hawasa) for the implementation of the project. This arrangement coupled with overstretching the project area over five Woredas has constrained the timely implementation of the project actions.  The instability of internal management, poor staff commitment, poor project management capacity and unhealthy and poor team spirit among the project staff were initially the inherent attributes of FARM Africa’s WCBP which negatively affected the timeliness of the project intervention.  The logistical constraints, particularly field vehicle, where one of the field vehicles had been non- functional for long periods due to car accident.  Lack of clear implementation procedure (such as for the micro-projects implemented through CDF) has negatively affected the timeliness of the project.  Delay in CDF fund transfer and stringent financial settlement procedure have also contributed to delay in the implementation of some of the project activities

Following the mid-term review, FARM Africa in collaboration with its partners managed to amend the project so as to make it realistic in its activities, organizational setup and institutional arrangement. Moreover, a one- year non-cost extension was made in the expectation of successful completion of the project activities. Despite the efforts made to implement many activities in the last one year, however, some of the ongoing activities were not completed until the final evaluation period. Among others, there were cases where some of the planned activities were not completed until the evaluation period in Derashe Special Woreda. The project has not given sufficient time to monitor the outputs and outcome/impact-trend of most of the activities implemented during the last five years. For instance, the evaluation team noted that most of the CDF activities in Basketo Special Woreda were not completed till the time of the evaluation, and thus, most of the project outputs were not handed over to concerned bodies and not utilized by project beneficiaries. Similarly, the project was poor in terms of timely supporting WDAs to solicit fund such as through organizing fund raising events and/or linking with potential donors available at different levels.

In general, the overall assessment on the timeliness of the project was found to be very poor in terms of timely initiating and implementing its planned activities, and thus, the evaluation team rated the timeliness of the project as one using the 1-5 value scale rating where one is very poor and five is excellent.

______19 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR 4.5 Cost-Effectiveness of the Project

The evaluation team attempted to assess the cost-effectiveness of the project through assessing how efficient the inputs were converted to outputs i.e. the extent to which the project was implemented at the minimal cost to produce the expected results. Here, it is worth mentioning that the cost effectiveness of the WCBP was assessed by considering the total cost of the project (i.e. Direct Project Cost + Overhead/Indirect Project Cost) vis-à-vis the outputs produced from the project course of action. In line with this, assessment of cost-effectiveness of the project was conducted using indirect and/or proxy indicators through reviewing documents and discussion with project partners at different levels.

The findings revealed that although some of the project interventions (such the provision of goats, construction of water supply schemes, health posts and education facilities) have resulted in cost-effective transformation of input to project outputs (physical structures as outputs). However, if we consider the cost concept strictly and consider the total cost of the project in the assessment of cost-effectiveness, the project was generally found to be poor in terms of effectively converting the project inputs (human, material, time and financial resources) into the envisaged outputs. In this regard, the evaluation team identified various factors which undermined the cost-effectiveness of the project. The major ones include, but not limited to, the following:

. The initial strategy of the project which was designed to implement the project activities through two project’s mobile outreach teams based at Hawassa and Arba-Minch was found to be expensive in terms of resource utilization related to logistics, human and financial resources.

. Although the project holder (FARM Africa) stressed that the overall administrative budget utilized by the project remained within acceptable limits, however, it seems that the project has been expending significant and more amounts of overhead and/or administrative costs as compared to the outputs realized during the last five years. In line with this, the team noted that the frequent staff turnover has negative implication on cost-effectiveness of the project.

. The pace of project implementation was in general very low, and thus, most of the project activities were implemented with significant delay. As a result, the cost of most of the activities was significantly escalated due to the inflationary situation observed in the last few years in the country and this has negatively affected cost effectiveness of the project.

. The project has provided matching funds to cover at least 80% of salaries of four staff of WDAs. For instance, the project covered a monthly salaries of four staff amounted to Birr 1,800, 1,200, 700 and 656 for Burji Development Association Field Coordinator, Field Officer, Administration and Finance Officer and Cashier, respectively. Although most of the WDAs have managed the implementation of many CDF projects, however, these staff members have not delivered the expected result of ensuring strong, accountable and capable WDA to lead local development (Output One) in terms of mobilizing resources and sustaining the development initiatives of the Association despite the provision of salaries and other capacity building endeavours by the project.

. Failure to systematize the capacity building support to WDAs which was meant for the attainment of Output One (Strong, accountable and capable to lead local development) of the project made the significant spending of the project somehow unproductive.

It seems that the project has lost the opportunity to enhance cost-effectiveness through linking some of its interventions with pertinent government sector offices. For example, the nursery management support interventions in Derashe Special Woreda, Koto Spring Water Development (Loltu Kebele) in Konso Special Woreda, the area enclosure in Amaro Woreda, provision of goats and some of the CDF activities in the

______20 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR intervention Woredas (such slaughterhouse, seed bank and grain mills) would be more cost-effective if they were properly linked to pertinent Woreda line offices and/or integrated with other development interventions like the PSNP. Linkage and integration with appropriate government sector offices might also create an enabling ground for effective and sustainable management and use of the project outputs beyond the life span of the project. In line with this, the evaluation team ascertained that Koto nursery site (Loltu Kebele) in Konso Special Woreda has established close partnership with Konso WoARD and it has been working with PSNP, and Development Agents (DAs) have been closely working with the project through the provision of technical support to the nursery site.

4.6 Consideration on Gender Aspects

On the other hand, it seems that the project was mainly biased towards male in terms of project staffing. In relation to this, it was noted that almost all of the technical staffs of the project at project coordination office and Woreda Coordinators were males. The same is true for WDAs staffs who were paid through the project support. The findings revealed that although male and female members of the community were represented in various committees and groups established through the project intervention (such as water committee, saving and credit associations, etc), however, the percentage of women was mostly found to be low. For instance, out of the 322 members of the three Saving and Credit Associations (SCA) in three Kebeles of Burji Special Woreda, only 28% (89 members) were females although women were effective in various business skills (see Figure 8).

Percentage of Male and Female Members of Saving & Credit Members in Selected Kebeles of Burji Special Woreda

100 94 90 79 80 72 70 60 58 50 42 40 30 28 Percentage (%) Percentage 21 20 10 6 0 Male Female Male Female Male Female Male Female

Berek Gamiyo Tinishua Kayate Total

Figure 8: Percentage of Male and Female Members in Three Saving and Credit Associations (Burji Woreda)

______21 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR 4.7 Overall Evaluation of Efficiency of the Project

The evaluation of efficiency of the project revealed that FARM Africa’s WCBP was found to be poor in terms of project management, timeliness and cost-effectiveness whereas the project was fair/satisfactory with respect to the other three criteria of efficiency i.e. beneficiary targeting, resource allocation/spatial aspects and gender consideration of the project. In general, the overall efficiency of the project with respect to the six efficiency criteria was found to be below satisfactory which is summarized in Figure 9 below.

Assessment of Overall Efficiency of the Project

Project Management and Implementation 2 5

4 Assessment of Overall Project Beneficiary Targeting Procedure & Efficiency 3 Applied Criteria 2 2 3 1 0 Resource Allocation/Spatial Cost Effectiveness Aspects 2 3

Timeliness Consideration of Gender Aspects 1 3

Figure 9: Assessment of Overall Efficiency of the Project

5. Effectiveness of the Project Outputs/Results

It was attempted to evaluate the effectiveness of the project through assessing the quantity, quality of outputs/results and utilization of the outputs/results by the project beneficiaries as well as through assessing the extent to which the project has achieved its expected outputs/results, and the extent to which the results have reinforced the project objectives.

5.1 Summary of Achievements of the Project Activities

Initially, the project had planned to implement various activities to achieve outputs/results categorized under five major headings including: (i) Output 1 : Five strong locally accountable WDAs built that are able to lead a process of locally initiated participatory development, one in each of the five Woredas; (ii) Output 2: The capacity of WDAs, Village Development Committees (VDCs) and local government staff built to develop, implement and monitor & evaluate a participatory development planning system enhanced; (iii) Output 3 : Increase the livelihood base of the local community and most vulnerable households in each of the five Woredas; (iv) Output 4 : A model of community based early warning for emergency and identification of community infrastructure projects for EGS; and (v) Output 5 : A model of Woreda level capacity building developed and disseminated to donors, government, NGOs and regional networks.

______22 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Based of on the recommendation of MTR, independent review and consultation with project partners, the outputs of the project were revised, especially by cancelling the fourth output (a model of community based early warning for emergency and identification of community infrastructure projects for EGS), and thus, the project was expected to achieve its goal and purpose through the following four outputs:

Output 1: Five strong, sustainable and locally accountable WDAs built that are able to implement a process of locally initiated participatory development;

Output-2: The capacity of relevant Woreda government offices, WDAs, KDCs and VDCs built to develop, implement, monitor and evaluate a participatory planning system; Output-3: Increase the livelihood base of the local community and most vulnerable households in each of the five Woredas; and

Output-4: Write up lessons learnt from the WCBP and disseminate to donors, government, NGOs and regional networks

It was attempted to summarize the major achievements of the projects in last five years of its life under each of the outputs while the details of budget and expenditures of the project along with percentage of utilization against its budget is annexed in the report.

5.1.1 Output 1: Five Strong, Sustainable and Locally Accountable WDAs Built that are able to Implement a Process of Locally Initiated Participatory Development

The major activities which were planned and implemented with the expectation of building strong, sustainable and locally accountable WDAs in five Woredas are summarized as follows; a) Working with Local Stakeholders to Define the Role to be played by WDAs in future Local Development

The project reported that it organised community forum in each of the intervention Kebele in order to define and establish the role of WDA in the five Woredas. Despite the plan to conduct four surveys in four Special Woredas, the project had produced baseline information about organizational status of three WDAs at the beginning of the project period. b) Assisting the Formal Registration of WDAs with Relevant Government Departments

The project assisted three out of five WDAs to renew their licenses with the Ministry of Justice. The planned assistance for the other two WDAs was postponed due to the introduction of new proclamation on NGOs and Associations by the Federal government. c) Each WDA Produces Clear and Achievable Three Year Strategic Plan

The project organised training session on concepts, tools and procedures of strategic planning for about 110 WDA members found in the five Special Woredas. Each of the WDA has produced strategic plan. The strategic plans were also approved by the respective General Assemblies of the WDAs through organising Strategic Planning Workshop in the five Woredas.

______23 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR d) Technical Skills of WDAs Staff Developed in Financial Management, Organizational Management, Planning and Reporting as well as Computer and Internet Use

In order to enhance the technical skills of WDA staff in the five Special Woredas, the project had planned and implemented various activities. In line with this, the following major activities were implemented in the last five years: (i) development of financial and personnel procedures for the five WDAs through organizing a workshop; (ii) conduct an assessment and review on how to ensure proper implementation and amendment of financial and personnel procedures for the WDAs through hiring external consultant; (iii) organising training as well as refresher sessions for WDA members in personnel and leadership management; (iv) provision of financial management training for financial and non-financial WDA managers; (v) organizing a workshop to develop financial policies and procedures for WDA and government staff in each Woreda; (vi) organising on-the-job training for the Finance and Administrative Officers of each WDA regarding the implementation of financial and personnel management systems as well as double entry accounting system; (vii) organising quarterly review and evaluation of WDAs’ budget and utilization; (viii) assist WDAs in identifying and commissioning suitable external auditors to conduct independent financial audit; (ix) assist WDAs to produce annual and quarterly plans for the WCBP and for their Associations; and (x) organising basic computer training for WDAs on Micro-Soft Word, Micro-Soft Excel, Internet and E-Mail application. e) Training on Proposal and Report Writing

The project organized a training session for WDA staff found in the five Special Woredas on project proposal writing as well report writing. f) Assist on Fund Raising, Promotion and Publicity of WDAs

The project attempted to assist WDAs through organizing awareness raising events which aimed at assisting WDAs in fundraising. It also assisted the WDAs in the areas of publicity and promotional activities including setting up notice boards and posting quarterly financial summaries for public consumption. The project has also assisted on the promotional activities of the WDAs through the preparation of leaflets and business cards. g) Contribute to Construct WDA Offices

The project has contributed and covered part of costs for the construction of WDA offices in four Special Woredas including Derashe, Burji, Amaro and Basketo Special Woredas. h) Assist WDAs to be Independent Civil Society Organization

In order to assist the five WDAs so as be strong and independent Civil Society Organization (CSO), the project conducted various activities in the last five years through: (i) reviewing the bylaws and sharing experience of two Development Associations to draw lessons and good practices; (ii) building the capacity of KDCs in assessing and accessing the budget, plan, reports and accountability of WDAs; (iii) organizing awareness raising events for local community on planned process of revising WDA bylaws and role and participation of community members in WDAs’ General assembly; (iv) assisting WDAs to conduct stakeholders workshop to produce draft and/or revise bylaws such as composition of WDA boards, general assembly, election procedures, how to assure autonomy of WDAs, etc; (v) organising WDAs’ general assembly meetings to discuss and approve revised bylaws; (vi) assisting WDAs to implement the revised bylaws; (vii) assisting two of the five WDAs to be registered as associate member

______24 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR of Christian Relief and Development Association (CRDA); and (viii) assist WDAs and SEPDA to draft and sign Memorandum of Understanding (MoU) on partnership agreement. i) Assist WDAs to be Financially Sustainable

With the aim of ensuring the financial sustainability of the five WDAs, the project attempted to implement the following major activities in the last five years: (i) assisting WDAs to streamline their staff based on their financial capacities; (ii) organising training sessions on fundraising aspects; (iii) orientation of the then Mobile Outreach Team (MOT) members on FARM Africa’s proposal preparation guidelines and review of internal documents related to project proposals; (iv) assisting MOTs and WDAs by FARM Africa Country Office (through its Senior Program Staff) on how to prepare project proposal; and (v) assisting WDAs in mapping non-locally resident potential supporters of WDAs, and also assisting in the preparation of plan for fundraising from non-resident potential supporters of WDAs. j) Development of Institutional Learning for WDAs

The project assisted the WDAs so as to establish a procedure for ensuring proper documentation and filing of important WDA documents (such as working manuals). It also organized experience sharing visits into two Development Associations in the Region including Guragie and Wollayta Development Associations in the SNNPR. k) Assist the Establishment of Resource and Documentation Centres at Woreda Level

The project assisted in the establishment of five mini-database centres in the five Woredas through the provision of the required materials and equipments such as furniture, computers and other related materials. In addition, the project assisted the compilation of Woreda profiles and baseline information of the five Woredas through hiring a consulting firm.

5.1.2 Output I1: The Capacity of Relevant Woreda Government Offices, WDAs, KDCs and VDCs Built to Develop, Implement, Monitor and Evaluate a Participatory Planning System

With the expectation of building the capacity of relevant Woreda government offices, WDAs, KDCs and VDCs in order to develop, implement, monitor and evaluate a participatory planning system, the project planned and implemented various activities in the last five years. Accordingly, the major project intervention under this component is summarized below. a) Development and Embedment of Participatory Planning Process in Government Planning System at Woreda Level and Clarification of Role of Stakeholders

The project attempted to introduce participatory planning through the implementation of various activities. The major ones include: (i) identification and establishment of Woreda Planning Committee (WPC) to lead Woreda planning process in the five intervention Woredas; (ii) organizing workshop in each of the intervention Woredas to legitimize and clarify the role and functioning of WPC, relevant government sector offices and WDAs; and (iii) assisting the five Woredas to produce a diagram showing the roles and responsibilities of each office in the planning process.

______25 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR b) Assisting Government Staff and WDAs to Facilitate the Implementation of Participatory Planning Process by KDCs and VDCs

The project conducted action oriented training to village communities on community based planning tools for each of VDC established in the intervention Kebeles where a total of 215 VDC members participated in the Participatory Resource Use Planning (PRUP) training at grassroots levels. It also organized action planning on PRUP workshop for the same number of participants at Kebele levels for members of KDC. It was also reported that the project facilitated refresher training on PRUP for relevant government offices. c) Other Assistances

The project assisted in the organization of field visits to all VDCs in order to follow-up the implementation of community action plans. In addition, WDAs were engaged in the assistance of KDCs to compile Village Action Plans into Kebele Action Plans through organizing field visits. The project also assisted to produce procedures and manuals for community based Monitoring and Evaluation (M&E) system.

5.1.3 Output II1: Increase the Livelihood Base of the Local Community and Most Vulnerable Households in each of the Five Woredas

The major areas of intervention under this component include the establishment of baseline information about the target Kebeles in each Woreda; assisting WDAs so as to support watershed committees in managing and monitoring Natural Resource Management (NRM); establishment of CDF and provision of grants to implement micro-projects; establishment of Woreda-level project appraisal system; and disbursement of grants to selected projects. In line with this, the achievements of the project are summarized below. a) Establishment of Baseline Information of Target Kebeles in each Woreda

In its life span, the project has produced three out of four reports on the status of watershed interventions and handed over to the WoARD. It has also conducted wealth ranking and need assessments of poorest households in collaboration with VDCs. In relation to the provision of goats to poor women households, the project conducted inventory of animals provided in three out of five Woredas, produced reports and handed over to respective WoARD. The project provided training and refresher to about 23 Community Animal Health Workers (CAHWs) in the last five years. b) Assist in Management and Monitoring of Natural Resource Management

The project organized training workshop on soil and water conservation skills as well as assisted WDAs to purchase working tools and planting materials for the implementation of watershed management activities. c) Establishment of Community Development Fund (CDF)

The project has assisted in the preparation of CDF implementation guidelines which mainly concerned with micro-project development, appraisal, approval, funding, implementation and M&E. Assessments were also conducted to identify feasible local community institutions for the implementation of micro- projects. Training session was organised for KDC members on project formulation and management of

______26 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR PRUP in the five Woredas. Government experts also assisted the community to develop project proposal. d) Establishment of Woreda-Level Project Appraisal System

Five project appraisal committees were established in the five intervention Woredas. The project assisted in the development of appropriate selection criteria and procedure through the provision of training on same. Project staff and concerned government staff participated and assisted project appraisal committee in their undertakings. e) Disbursement of CDF Grants to Selected Micro-Projects

In relation to this, the project has conducted various activities including the establishment of procurement procedure for micro-projects; provision of budget for about 60 micro-projects in 25 Kebeles of the five intervention Woredas; provision of technical assistance to micro-projects; reviewing of credit services and supporting Saving and Credit Associations to develop and implement loan repayment procedures; consolidation and handover of Saving and Credit intervention, assisting in financial management, record keeping and documentation.

5.1.4 Output IV: Write up Lessons Learnt from the WCBP and Disseminate to Donors, Government, NGOs and Regional Networks a) Documentation and Dissemination of Project Findings

The project attempted to document the project activities, methodologies, best practices, challenges and lessons learned from the implementation WCBP, especially those related to micro-projects (such as CDF Grants, Grain Bank, Saving and Credit Associations, Watershed Management, Women’s Goat Schemes and training and deployment of CAHWs). The project prepared a summary of document regarding WCBP and lessons drawn from the implementation of the project and disseminated to Woreda and Regional Government bodies, partner NGOs/CSOs and donors. b) Organising Workshops and Other Experience Sharing Events with Others

The project organised a dialogue and advocacy forum at Regional level jointly with other CSOs/NGOs working in the Region. The project also organized experience sharing tours to Wolayita and Gurage Development Associations for WDA board and staff. A final experience sharing workshop was organized for its stakeholders based on lessons learnt from the implementation of WCBP in five Special Woredas of SNNPR.

5.2 Assessment of Quantity of Project Outputs/Results

The evaluation attempted to assess the project outputs in terms of quantity and quality of results achieved so far in the intervention Woredas. In line with this, the findings revealed that the achievements of the project in terms of producing the required quantities of outputs from the implementation of its activities in the last five years were found to be more or less satisfactory except for the Output III (i.e. increase the livelihood base of the local community and most vulnerable households in each of the five Woredas). Despite the initial plan of about 315 CDF projects by WCBP, it has actually achieved 60 projects out of the amended 61 projects. The evaluation team felt that the project would implement more CDF project if there were efficient and effective in the overall project management and implementation. The de-commitment of more than Euro 130,000, price increment for construction

______27 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR materials, and poor commitment and support from local actors have also contributed for poor performance of CDF projects. Accordingly, the results produced to increase livelihood base at community and household level (output three) was found to be low in terms of coverage and/or quantity. The findings of the team on the assessment of the quantity of results achieved so far are summarized in the Table 3 below.

Table 3: Assessment of Quantity of Results Achieved so far Assessment of Quantity of Outputs Achieved Amaro Basketo Burji Derashe Konso Average Output 1: Five strong, sustainable and locally accountable WDAs built that are able to implement a process of locally initiated participatory development. 3 3 3 3 3 3 Output-2: The capacity of relevant Woreda government offices, WDAs, KDCs and VDCs built to develop, implement, monitor and evaluate a participatory planning system. 3 3 3 3 3 3 Output-3: Increase the livelihood base of the local community and most vulnerable households in each of the five Woredas. 2 2 3 2 2 2 Output-4: Write up lessons learnt from the WCBP and disseminate to donors, government, NGOs and Regional networks. 3 3 3 3 3 3 Overall Assessment of Quantity of Outputs/Results Achieved 3 3 3 3 3 3

Output 1: Five strong, sustainable and locally accountable WDAs built that are able to implement a process of locally initiated participatory development

Output-2: The capacity of relevant Woreda government offices, WDAs, KDCs and VDCs built to develop, implement, monitor and evaluate a participatory planning system.

The project has more or less implemented most of its envisaged activities in terms of quantity to enhance the capacity of relevant government offices, WDAs, KDCs and VDCs to plan, implement, monitor and evaluate projects. However, the project cancelled the planned training on PRUP and M&E for the members of WPC. Similarly, the project did not institutionalize the bottom-up planning process

______28 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR at Woreda level where the planned assistance for planning experts so as to integrate the participatory plans into their regular office plans.

Output-3: Increase the livelihood base of the local community and most vulnerable households in each of the five Woredas

At community level, despite the initial plan of about 315 micro-projects, however, the project actually managed to realize only 60 projects (out of the total 61 amended CDF projects after MTR) in the last five years through the provision of CDF for WDAs. Even, some of these micro-projects were not completed during the evaluation period. The evaluation also noted the envisaged assistance in management and monitoring of natural resource management was poor in terms of quantity and coverage. On the other hand, increment of livelihood of vulnerable households was limited to the provision of two goats per household in the five Woredas and income generation scheme in three Kebeles of Burji Special Woreda. Thus, the quantity of outputs produced in improving household livelihood was low to bring about the envisaged objective at household level in most of the intervention Woreda except for Burji Special Woreda where the output of saving and credit was relatively found to be satisfactory in terms of quantity as compared to other Woredas.

Output-4: Write up lessons learnt from the WCBP and disseminate to donors, government, NGOs and regional networks

The project produced a document and disseminated its project activities, methodologies, best practices, challenges and lessons learned from the implementation WCBP. It also organised a dialogue and advocacy forum at Regional level jointly with other CSOs/NGOs working in the Region. However, with existing time constraint, especially at the last quarter of the project life, it is unlikely that the project has disseminated its lessons with the required quantity and coverage at donors, governments and NGOs levels. In general, the attempt made by the project to produce the output in terms of quantity was rated as satisfactory.

5.3 Assessment of Quality of Project Outputs/Results

In terms of quality, the project was found relatively rated as more or less satisfactory except for Output One (Five strong, sustainable and locally accountable WDAs built that are able to implement a process of locally initiated participatory development). Specifically, the project provided exemplary capacity building initiatives in the promotion of bottom-up planning process (Output Two) as well as documentation and dissemination of its experience in the implementation of WCBP in the five Woredas (Output Four), On the other hand, the quality of outputs of the project in terms of building strong, sustainable and locally accountable WDAs (output 1) was relatively found to be below expectation where the outputs produced were poor in realizing the intended purpose (such as weak WDAs). In one instance, the team observed a major defect in which the spring development in (Loltu Kebele) Konso Special Woreda (i.e. Koto Spring) was found to be technically defective, and thus, the local communities have still used unsafe river water for drinking.

It seems that the failure in the attainment of the anticipated quality for output one might be somewhat attributed to lack of systematic approach of the capacity building interventions by the project and their timeliness. In addition to this and even more importantly the limitations from the side of the WDAs themselves in terms of lack of technical capacity, commitment and strategic approach by the WDAs management and Boards were also the major factors for the failure of the project in realizing the anticipated quality of achievement with respect to output one. Other constraining factors include high staff turnover in

______29 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR the WDAs, lack of recording of institutional knowledge to pass to new WDA managers and lack of building the skills of the new WDA Managers who have been left the Associations for various reasons.

The Table below summarizes the overall assessment of quality of output of the project in each of the intervention Woredas.

Table 4: Overall Assessment of Quality of Outputs of the Project Assessment of Quality of Outputs Amaro Basketo Burji Derashe Konso Average Output 1: Five strong, sustainable and locally accountable WDAs built that are able to implement a process of locally initiated participatory development. 1 1 2 2 4 2 Output-2: The capacity of relevant Woreda government offices, WDAs, KDCs and VDCs built to develop, implement, monitor and evaluate a participatory planning system. 3 3 3 3 3 3 Output-3: Increase the livelihood base of the local community and most vulnerable households in each of the five Woredas. 2 2 4 2 3 3 Output-4: Write up lessons learnt from the WCBP and disseminate to donors, government, NGOs and regional networks. 3 3 3 3 4 3 Overall Assessment of Quality of Outputs/Results 2 2 3 3 4 3

5.4 Use of the Outputs/Results by the Project Beneficiaries

______30 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR With respect to output two, i.e. introduction of a bottom up planning approach, the expected use of outputs by users/beneficiaries was found to be satisfactory. This rating for the bottom-up planning was made mainly underlining the interest of the local government to adopt the planning approach in the future. However, the exercise was confined within the intervention Kebeles of the WCBP and even discontinued after the end of the CDF planning process. Concerning the fourth output, FARM Africa, the Donor, WDAs and other stakeholders (those who got access to the materials) are expected to adopt and/or utilize the materials in their future project planning and implementation efforts. In general, the overall rating for the use of outputs/results for all the four outputs was somehow satisfactory (see Table 5 below).

Table 5: Overall Assessment of Use of the Outputs by the Project Beneficiaries Assessment of Use of Outputs by Target Beneficiaries Amaro Basketo Burji Derashe Konso Average Output 1: Five strong, sustainable and locally accountable WDAs built that are able to implement a process of locally initiated participatory development. 1 1 2 2 4 2 Output-2: The capacity of relevant Woreda government offices, WDAs, KDCs and VDCs built to develop, implement, monitor and evaluate a participatory planning system. 3 3 3 3 3 3 Output-3: Increase the livelihood base of the local community and most vulnerable households in each of the five Woredas. 2 2 4 2 3 3 Output-4: Write up lessons learnt from the WCBP and disseminate to donors, government, NGOs and regional networks. 3 3 3 3 4 3 Overall Assessment of Use of Outputs/Results by Project Beneficiaries 2 2 3 3 4 3

5.5 Extent to which the Project Achieved its Outputs and Reinforcing its Objective

It was attempted to assess the extent to which the project produced the anticipated results/outputs as well as the extent to which the results achieved have reinforced the project objective in each of the intervention Woredas including Amaro, Basketo, Burji, Derashe and Konso Special Woredas. In line with this, the extent of achieving the expected results and reinforcing the project objective under each of the project outputs were assessed and summarized as follows:

Table 6: Assessment of the Extent of Achievement of the Project Outputs & Reinforcing its Objective Assessment of the Extent of Achievement of the Project Outputs and Reinforcing Project Objective Amaro Basketo Burji Derashe Konso Average Output 1: Five strong, sustainable and locally accountable WDAs built that are able to implement a process of locally initiated participatory development. 1 1 2 2 4 2 Output-2: The capacity of relevant Woreda government offices, WDAs, KDCs and VDCs built 3 3 3 3 3 3

______31 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Assessment of the Extent of Achievement of the Project Outputs and Reinforcing Project Objective Amaro Basketo Burji Derashe Konso Average to develop, implement, monitor and evaluate a participatory planning system. Output-3: Increase the livelihood base of the local community and most vulnerable households in each of the five Woredas. 2 2 4 2 3 3 Output-4: Write up lessons learnt from the WCBP and disseminate to donors, government, NGOs and regional networks. 3 3 3 3 4 3 Overall Assessment of the Extent of Achievement of the Project Outputs and Reinforcing Project Objective 2 2 3 3 4 3

Output 1: Five strong, sustainable and locally accountable WDAs built that are able to implement a process of locally initiated participatory development

The extent to which the project achieved its expected outputs/results was assessed by output type for each respective Woreda. Accordingly, with respect to output one (i.e. realization of five strong WDAs at the end of the program), the outcomes vary among the intervention Woredas. Accordingly, the status of Konso WDA was somehow promising and rated as good by the evaluation team. It was noted that Konso Development Association (KDA) has a well established office & office facilities, organizational structure, logistical capacity and somehow reliable source of fund from internal and external sources. It was noted during the evaluation period that KDA was implementing two micro projects financed by CRDA and Mercy Corps with a total budget amounted to Birr 320,000. This fund raising capacity of the association and the capacity of the association to collect membership fee and the quarry site donation made by the local government to the association are expected to increase the likelihood of sustainable functioning of the association even following the termination of the project support. Of course, all these achievements of KDA were not only attributed to the current WCBP of FARM Africa but it was also the outcome of over 10 years technical assistance and support made by the NGO to the association including under the current WCBP.

The findings for the other four special Woredas were somehow different. Despite the implementation of different capacity building support activities by the WCBP for the Development Associations at Burji, Derashe, Amaro and Basketo Special Woredas, however, the institutional capacity of these associations was not strengthened as anticipated due to various reasons. Among others, the major constraining factors include lack of systematic approach and timeliness of the capacity building effort made from the project side as well as poor commitment of WDAs to mobilize local communities, weak support from respective government bodies, frequent turnover WDA staffs and weak governance by WDA boards to capitalise on the capacity building supports made by the WCBP were the major ones. Although most of the physical inputs mentioned above were delivered by the project as planned, however, failure to implement some other important aspects of technical supports (like linking the associations with potential donors, failure to enhance the fund raising capacity of the associations from internal and external sources, etc) have constrained the associations.

But, it is worth to mention that the current situation of these Development Associations is by far better than compared to their situation before five years. At present, all these associations have their own office structures which were constructed by a budget contribution of the WCBP (in some cases, the project contributed up to 50 % of construction costs), better office facilities like furniture and equipments, better transportation facilities (including through the provision of motorbikes), etc. Moreover, all the five development associations have well documented financial and human resource administration manuals. All

______32 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR these are believed to be a better ground paved by the WCBP for their future operation if they can sustain as organizations.

As learnt from the reflection of Woreda Administrative bodies during the validation workshop of the final evaluation and as observed from the follow-up actions made after the evaluation exercise, there are some promising initiatives taken by FARM Africa, Regional and Local Governments and the WDAs themselves. In this regards, practical initiatives observed like allocation of inputs by FARM Africa for consolidation phase as well as allocation of budget to maintain the key staffs of the WDAs and initiation for the organization of a fund raising event, particularly in Amaro and Basketo Special Woredas, respectively, are tangible actions taken to reinforce the capacity of the WDAs in terms of financial resource capacity and managing local level development initiatives. For instance, the allocation of budget (130,000 Birr) by Amaro Special Woreda Administration for Kore Development Association for the year 2009/10 is a good initiative towards financial sustainability of the WDA.

In line with this, the purpose of most of the capacity building initiatives targeted towards the WDAs was to realize a strong, self reliable, sustainable and accountable Woreda Development Associations at the end of the project period. Although most of the physical inputs mentioned above were delivered by the project as planned, however, failure to implement some other important aspects of technical supports (like linking the associations with potential donors, failure to enhance the fund raising capacity of the associations from internal and external sources, etc) have constrained the associations and even they can be fragile if the external support by the project is terminated. Out of the four Woredas, it was only from Burji and Derashe Special Woredas that the evaluation team got conformation for the continuing operation of the WDAs for the next three months with one or two staffs. For Amaro and Basketo Special Woredas, the Woreda administrators stressed that until the local governments take some measures in the medium and long terms, for the time being there is no means to sustain the operation of the WDAs even with a single personnel. This implies the failure of the project to meet one of its major outputs, and thus, the overall rating for this specific output was rated as less than satisfactory.

Output-2: The capacity of relevant Woreda government offices, WDAs, KDCs and VDCs built to develop, implement, monitor and evaluate a participatory planning system.

With respect to the second output (i.e. strengthening capacity of relevant Woreda government offices and WDAs in participatory planning approach), the extent of the work done in all the five Woredas was rated as satisfactory. Here, it is also worth to mention that if not for lack of replication of the bottom-up planning approach in other Kebeles of the intervention Woredas as well as lack of institutionalization of the approach within the existing local government planning system, the work done in the pilot Kebeles through KDCs and VDCs was satisfactory. However, the approached was mainly used in the planning process of CDF projects of WCBP and discontinued following the completion of the CDF planning exercise. The project holder indicated that although the institutionalization of the approach was planned at initial stage while it was cancelled during the second amendment of the project. On the other hand, the evaluation team noted that piloting of such best practice (participatory planning approach) would be more beneficial if it were followed by the initiative to scale-up and replicate to wider communities and geographical areas rather than cancelling the intended intervention through project amendment after MTR. Inline with this, the Regional BoFED has appreciated the approach followed by WCBP, and thus, have shown willingness and commitment to take over and scaling up the good lesson obtained from this exercise as its planning approach in the future

Output-3: Increase the livelihood base of the local community and most vulnerable households in each of the five Woredas.

The third output anticipated to be achieved by the project was related to the enhancement of the livelihood base of the local community and most vulnerable households. In this regard, the extent of achievement of most of the CDF activities which were meant for enhancing community livelihood was generally more or less satisfactory. The findings revealed that the achievements in Burji and Konso Special Woredas were relatively

______33 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR found to be good and satisfactory as compared to the rest of the intervention Woredas. In some of the intervention Woredas (such as Basketo and Derashe), most of the planned CDF activities were not even completed until the evaluation period. Even, many of them are unlikely to be completed unless additional resources will be allocated for the completion of ongoing micro-projects.

With respect to activities which were aimed to enhance the livelihood of vulnerable households like provision of goat and revolving fund were not able to bring the anticipated outcomes in terms of the aim of reducing poverty of most vulnerable households (i.e. in reference to overall objective of reducing poverty in the most vulnerable households) mainly due to lack of follow-up, shortage of financial resources and weak technical support.. As explained earlier, some of the social and economic infrastructure were not completed and/or no started operation (such as in Basketo and Derashe Special Woredas) while increment of the livelihood of most vulnerable households was mostly limited to goat provision (in all the intervention Woredas) and saving and credit intervention in selected Kebeles of Burji Special Woreda. On the other hand, if appropriate exit strategies are designed and implemented in collaboration of concerned stakeholder, it seems that most of the livelihood enhancement activities implemented by WCBP will have potential impact to improving the human and livestock health, access to education, addressing the problem of natural resources management and food security, addressing work load of women, improving the household income of target beneficiaries and provision of safe and clean water which reduces prevalence of water borne diseases. Despite some constraints, the project was generally rated as satisfactory in terms of achievement of output three (i.e. increasing the livelihood base of the local community and most vulnerable households in each of the five Woredas).

Output-4: Write up lessons learnt from the WCBP and disseminate to donors, government, NGOs and regional networks

The fourth output of the project was related to the write up and dissemination of lessons learnt to the wider public including Donors, Government, NGOs, etc. In this respect, the extent of work achieved by the project was good in Konso Special Woreda while it was rated as satisfactory in the rest of the intervention Woredas. The project office managed to document the approach, challenges and lessons learnt through commissioning an external consultant, and the document was distributed among stakeholders. Moreover, the outcome of the project reviews made during the life span of the project were documented and disseminated for different stakeholders of the project. However, the degree of circulation of these materials was limited to project stakeholders at Woreda and Regional levels, and thus, the dissemination of the documents at community level was somehow constrained by language barrier since they were prepared in English language.

5.6 Overall Assessment of Effectiveness of the Project

Based on the above findings, the evaluation team noted that the overall effectiveness of the project (in terms of the extent to which the project achieved the expected outputs, the extent to which the results achieved reinforcing the project object, quantity, quality and use of the outputs by the target groups) was found to be either poor or below satisfactory as it is summarized and illustrated in the Figure below.

______34 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Overall Assessment of EFFECTIVENESS of the Project

The Extent to which the Project has Achieved the Expected Results/Outputs 3 5

Overall Assessment of the Effectiveness of the 3 Extent to which the Outputs/Results achieved are

Project 2 Reinforcing the Programme Objective 3 3 1

Use of Outputs/Results by the Project Quantity of Outputs/Results Achieved so far Beneficiaries 3 3

Quality of Outputs/Results Achieved so far 3

Figure 10: Summary of Overall Assessment of Effectiveness of the Project

6. Assessment of Outcomes and/or Impact-Trends of the Project

The evaluation team attempted to assess the outcomes and/or impact trends of the project at household, community and institutional levels. The impacts and/or impact trends of the project on crosscutting issues were also assessed including gender, environment and prevention and control of HIV/AIDS.

6.1 Outcomes and/or Impact-Trends of the Project at Household Level

It was revealed that some of the project activities like goat provision and micro-credit schemes have shown promising impact at target household level in terms of increasing household income and asset. In line with this, the majority of goat beneficiaries contacted during the evaluation exercise indicated that they have benefited from goat provision in terms of cash income from the sale of animals as well as increment of household asset. In line with this, Table 7 summarizes case studies on the impact of goat provision at household level while the details are presented in the annex.

Table 7: Case Study: Summary on the Impact of Goat Provision at Household Level Value Value of Value Net Value of Goats Sold No. of of Added (in No. of Goats or Goats Goats Amount Cash and Name of Woreda Goats Provided Slaughtered at at Hand Repaid in Kind) Remark Beneficiary (Kebele) Provided (Birr) (Birr) Hand (Birr) (Birr) (Birr) She is secondary beneficiary who received 2 kids from primary Burji beneficiary and did (Tinishua not transfer to the Abebech Gelgelo Keyate) 2 100 5 1,050 950 tertiary beneficiary Basketo One goat was (Shella slaughtered for the Kana household Dirbete Engida Bola) 2 180 100 2 250 180 350 consumption

______35 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Value Value of Value Net Value of Goats Sold No. of of Added (in No. of Goats or Goats Goats Amount Cash and Name of Woreda Goats Provided Slaughtered at at Hand Repaid in Kind) Remark Beneficiary (Kebele) Provided (Birr) (Birr) Hand (Birr) (Birr) (Birr) The income from Basketo the sale of goats (Shella was used to cover Kana various household Meselech Kojinko Bola) 2 250 170 1 170 250 340 expenses The income from Basketo the sale of goats (Shella was used to cover Kana various household Shela'ane Amanu Bola) 2 210 400 2 350 200 740 expenses Sale of a goat to cover school Amaro related expense Tayech Endale (Bunity) 2 215 190 9 2,250 180 2,405 (clothing) Loan repaid through purchase Konso from local market Tinno Oltech (Afraide) 2 160 4 380 160 220 from own source Average 2 186 143 4 742 162 834 Net Income per Unit (One Birr) of Investment 4.5

The result of the case studies revealed that an average sample household contained in the case study has obtained an average of Birr 834 of net income (in the form of cash and/or in kind) after settling the loan (paying to the secondary beneficiary) which ranges between Birr 220 and Birr 2,405. The analysis of the case study findings revealed that an investment of one Birr (in the form of provision of goats through WCBP) has created an additional net income of Birr 4.5 during the last four years i.e. an investment of one Birr has generated an additional net income of Birr 1.12 per annum. Interpretation of this finding should be made with caution taking into account the inflation during the specified period of time.

Mrs. Tayech Endale, a resident of Bunity Kebele in Amaro Special Woreda, received two goats valued at Birr 215 from WCBP before four years. She indicated that she transferred two goat kids to the secondary beneficiary which was estimated at Birr 180 at the time of transfer. On the other hand, two goats died which would be sold at Birr 240 Birr at the time of death. Mrs. Tayech stressed that goat provision through WCBP has benefited (Birr 2,405) her in terms of increasing her household asset holding and income. She sold one of the goats at Birr 190 to cover school supplies for her children including school uniform. At the time of the evaluation, she owned a total of nine goats (including the goat provided and the kids). She estimated that the nine goats would be valued at a total of Birr 2,250 at the current price. In general, after settling her loan, Mr. Tayech obtained a net income of Birr 2,405 in terms of cash income (Birr 155) and asset (Birr 2,250) in the form goats.

Similarly, most of the beneficiaries of saving and credit associations in Burji Special Woreda stressed that the scheme has positively contributed to increase their household income and asset through using the revolving credit fund in various income generation activities including petty trading, local beverage, grain marketing, etc. A case in point was noted in Tinishua Keyate Kebele of Burji Special Woreda were saving and credit scheme has positively affected the income and asset of beneficiary households. For instance, Mr. Dembela Gelo who resides in the Kebele was provided with Birr 1,000 from revolving fund disbursed by WCBP. He used the money in grain marketing through buying and selling of maize, teff and other crops. Mr. Dembela stated that he obtained about Birr 5,000 after paying his credit including the principal (Birr 1,000) and a 5% interest (Birr 50). Similarly, Mrs. Ayime Kene, the resident of the same Kebele, took credit amounted to Birr 1,000. She was engaged in the local beverage making. From the profit which she earned from her business, she bought a cow at Birr 900 before two years. Currently, the cow estimated to be sold at about Birr 2,000. Mrs. Ayime has also paid her loan including the interest (Birr 1,050). In addition, she has saved a total of Birr 500 and has

______36 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR been used the money to run her business. Accordingly, Mr. Ayime has obtained a total net income of Birr 2,500 in the form of asset (Birr 2,000) and cash (Birr 500) after settling all of her loans (principal and interest)

As a result, the impact of the project at household level was rated as somewhat satisfactory. However, due to the relatively low level of household livelihood asset or cash transfer through the two interventions, the level of livelihood impact achieved through the project was somewhat undermined. For activities like grain storage, slaughterhouse and grain store which were anticipated to serve a group of households, the evaluation team couldn’t assess their impact at beneficiary household level since the facilities didn’t start to provide the necessary service.

6.2 Outcomes and/or Impact-Trends of the Project at Community Level

Concerning the impact of most of the CDF activities which are meant to serve a wider community, since almost all of the project outputs, except some of the public toilets, school blocks and two water supply facilities didn’t start to deliver their service for the wider community the assessment of impact was constrained somehow. However, the evaluation team identified various indicators to assess the possible impacts of the project outputs to be enjoyed by larger numbers of people and communities in the intervention areas in the future. In this regard, the team observed that some of the infrastructural facilities completed and started their operation like water supply schemes, public toilets and school rehabilitations have improved the quality of social services and their service coverage. It was unable to obtain disaggregate data on service coverage of different social and economic facilities due to lack of baseline data and lack of compiled data on each of the CDF project activities. It was also untimely to assess the impact of uncompleted project activities at time of evaluation. However, the evaluation noted that if the ongoing and uncompleted CDF funded activities are finalized and properly handed over to pertinent government sector offices and user communities, there will be possibilities that the communities may benefit from the outputs of the CDF projects from:

 Better coverage of primary education for the school age children  Better access to clean and safe drinking water and reduced prevalence of water related communicable diseases,  Reduced labour and time requirement for fetching water, and hence, increased time for productive, reproductive and other household chores,  Improved hygiene and environmental sanitation, and hence, reduced prevalence of communicable diseases,  Improved access for primary healthcare and preventive health service, and  Improved access for livestock health service and improved livestock productivity.

For instance, the evaluation team noted that a total of about 500 households have got access to clean and safe water from the Kombolcha Spring Development constructed through CDF funding in Lemo Kebele of Burji Special Woreda. Similarly, many water schemes have been constructed and utilized by the local communities Derashe, Konso, Amaro, Burji and Basketo Special Woredas.

______37 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR

Figure 11: Berek Spring Development in Burji Woreda Figure 12: Sorobo Roof Water Catchments in Konso Woreda

Therefore, based on the presentation above, the impact trend of the project at community level was rated as somewhat satisfactory. However, the realization of all these impacts is directly depended on/subjected to the action to be made by the project holder and its partners so as to complete all the CDF projects and install appropriate linkage of the social facilities after completion with pertinent stakeholders for their future management and use.

Moreover, the situation of Konso Development Association, the initiatives aimed towards capacitating the WDAs are expected to bring a communal benefit and impact in the form of better capacity of local actors in initiating and managing community based development projects. However, for the other WDAs this can be realized if the already established organizational set-up and capacity is reinforced by the development associations by themselves and other stakeholders like the Woreda governments, SEPDA, and the people at large. In this regard, though the capacity of the WDAs at present is not promising, except in Konso, the attempt made by Derashe and Basketo Development Associations to solicit fund for micro projects is also one promising indicator for potential impact in this regard.

With respect to the bottom-up planning approach introduced by the project, the WCBP couldn’t manage to replicate the approach to other Kebeles outside its operational areas till the end of the project. However, though the approach was not institutionalized so far, the enthusiasm from the side of the Woreda government offices and the initiative of the Regional BoFED to adopt the approach (especially using VDCs and KDCS) in the regular government planning process is one indicator for possible positive impact if the system is properly adopted. This in turn is expected to enhance the empowerment of the community through better say in its own development concerns.

6.3 Outcomes and/or Impact-Trends of the Project at Institutional Level

Among others, the evaluation identified the following major indicators of impacts of the project at institutional level: (i) establishment of accountable, independent and self-relayed Development associations; (ii) enhancing the planning capacity of local government partners; and (iii) institutionalization of the bottom- up planning approach in the government planning process. In this regard, different capacity building activities were implemented through targeting respective Woreda Development Associations, Woreda sector offices and the community group in the project intervention Kebeles.

______38 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Accountable, Financially Independent WDAs to Effectively Iidentify Needs, to 2 Develop, Implement, M&E Projects. 5 4 3 2 1 Overall Effects of the Project at VDCs and KDCs to Develop CAP, Institutional Level in terms of 0 Implement, M&E Projects Effectively. Institutional Factor 3 2

Bottom-up Planning Institutionalized in Woreda level Planning Processes 1

Figure 13: The Effects of the Project at Institutional Level

In line with this, the capacity building supports made to WDAs, the establishment of mini database (resource centre) at Woreda level and establishment of VDCs and KDCs at intervention Kebeles and their involvement in the development of Community Action Plan (CAP) were among the major relevant and appropriate interventions of the project. However, despite the relevance of the interventions, the institutional impact of the support actions was very minimal. The support provided for the WDAs didn’t bring the expected outcome in all but one of the intervention Woredas. Therefore, there is no ground to expect an institutional impact of these actions in Amaro, Burji, Basketo and Derashe Special Woredas. As a result, the rating for institutional impact of the support for WDAs was rated as poor which implies the project was found to be less than satisfactory in terms of institutional impact of the action.

With respect to enhancement of the planning capacity of Woreda government sector offices, the establishment of database system and Woreda Information Centres are expected to improve the planning capacity of the local government by making the process better informed with realities at ground. Since the database centres are planned to be linked with WoFED, the contribution to improve the planning process is expected to be satisfactory if the data base system is updated regularly and managed properly.

The other institutional outcome that was expected to be attained was the institutionalization of the bottom- up planning approach introduced by WCBP. Accordingly, the evaluation team noted that though the idea was very relevant and innovative, however, its application was limited to the preparation of community action plan during the planning phase of the CDF projects. It was not replicated in other Kebeles found outside the project intervention areas. It was also noted that the approach was also not institutionalized so far within the regular planning process of the local government at Woreda and Regional levels. In line with this, the project holder (FARM Africa) stressed that the institutionalization of bottom-up planning approach in Woreda level planning was cancelled during the amendment so that it should not be considered in the evaluation process. However, it seems that the major reasons for lower institutional impacts of the project interventions was mainly related to weakness of the project to timely initiate these activities and poor attention for the sustainability and institutionalization of project benefits in the existing institutional framework of the local government structures.

______39 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR 6.4 Outcomes and/or Impact-Trends of the Project on Crosscutting Issues

The evaluation attempted to assess the impacts of the project on crosscutting issues including gender, environment and HIV/AIDS related issues. Accordingly, the results ascertained that some of the project interventions found to be positively affected the gender aspects in terms of livelihood enhancement through addressing the felt need of men and women. For instance, the output of some CDF projects like water supply schemes and grinding mills are expected to reduce the workload of women, girls and children as well as time spent to fetch water and grinding of grains. Moreover, the provision of goats which specifically targeted women group of the society and provision of micro loan for both gender groups are expected to enhance asset ownership of women, and thus, is expected to improve the access to and/or control over resources at household level. On the other hand, women involvement in the management of group resource (such as in water point management) is one positive outcome which somehow is expected to enhance the involvement of women in communal decision making process. Besides, most of the CDF projects if completed and being operational are expected to serve both male and female members of the community.

The other cross cutting issue to be addressed by the project outcome if the benefit of the already established nurseries is going to sustain is the rehabilitation of the microenvironment which anticipated to be achieved with increased level of vegetation coverage by new plantation. The nurseries established through the support of the project are expected to be handed over to the respective Woreda Agriculture and Rural Development Offices and to sustain their operation of development of locally sound tree seedling and their distribution in the future. It was also noted that the area enclosure in Amaro Special Woreda was found to be effective in rehabilitation and conservation of natural resources in the area.

Figure 14: Area Enclosure in Derba Kebele (Amaro Woreda) Figure 15: Koto Nursery in Loltu Kebele (Konso Woreda)

Despite the positive impact of the project intervention in terms of gender and environment, it was noted that the project design as well as implementation was generally failed to mainstream HIV/AIDS control and prevention initiatives under the overall project framework. On the other hand, the evaluation team ascertained that intervention related to HIV/AIDS was planned and implemented in Holte Kebele of Derashe Special Woreda through funding from CDF. Similarly, it was leant that beside the local government intervention through the deployment of health extension workers in each Kebele, WDAs in Konso, Derashe and Basketo Special Woredas have also secured financial resources from Christian Relief and Development association (CRDA) to be engaged in the control and prevention of HIV/AIDS and reproductive health intervention.

Mainstreaming HIV/AIDS is an approach that is relevant in both development and humanitarian work. By mainstreaming HIV/AIDS, we mean a process that enables development actors to address the causes and effects of HIV/AIDS in an effective and sustained manner, both through their usual work and within their

______40 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR workplace. It means «wearing AIDS glasses» while working in all sectors and at all levels. Mainstreaming of HIV/AIDS is not an intervention per se. It constitutes a range of practical strategies for scaling up responses and addressing the developmental impacts of HIV and AIDS. Due to its adverse impact on health and development initiatives, any development interventions should be integrated into planned and ongoing development processes by linking them to the core business.

6.5 Overall Assessment of Outcomes and Impact-Trends of the Project

Based on the findings presented above, the evaluation team attempted to assess the overall outcome and/or impact-trend of the project based on the extent to which the overall objects were achieved successfully and how the project impact its target groups at household, communities and institutional level and based on the project impact on other cross-cutting issues like gender, environment, poverty. Besides to this, the extent of sustainability of impact of the project after the withdrawal of the project ensured and capacity to countermeasure external negative effects on project impacts were assessed and the findings are summarized. Accordingly, the overall impact of the project is found to be somewhat satisfactory as illustrated in the figure below.

The extent to which the overall objectives were achieved and the contribution of the project to their achievement 3 5 The OUTCOME (IMPACT-TREND) of the project on Overall Assessment of Project Impact 4 its Direct Beneficiaries in terms of Technical and 3 3 Economic Factor. 3

2 The effects of the project on larger numbers of Impacts of the Project on Poverty Issues 1 people (communities) in terms of Technical, 3 0 Economic, Social and Institutional Factor 3

Impacts of the project on Environmental-related The Effects of the Project at Institutional level in Issues 3 terms of Institutional Factor 2 Extent of Sustainability of Impact of the Project after the withdrawal FARM Africa and Capacity to Impacts of the Project on Gender-related Issues countermeasure external negative effects on 3 Project Impact 3

Figure 16: Assessment of Overall Outcome and/or Impact-Trend of the project

7. Sustainability of the Project

It was attempted to evaluate the viability of the project action through assessing the financial/economic, technical, social, institutional and managerial sustainability.

7.1 Financial/Economic Sustainability

The financial/economic sustainability of the project was conducted to ascertain whether funds likely to be made available to sustain the benefits (outputs/services/results) produced by the project. In this regard, it was attempted to assess whether the responsible bodies/ institutions are ready and capable to assume their responsibilities to finance the operation and maintenance cost of the project outputs and for replacement of

______41 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR materials/ equipments introduced by the project after the termination of the project support in the future. The major findings related to financial/economic viability of the project benefits are summarized as follows:

 In relation to financial sustainability of water supply schemes, it was revealed that the Rural Water Resource Development Offices at Woreda level would take the responsibilities for future operation and maintenance. Besides, since the operational and maintenance costs of most the water schemes (especially spring water development) constructed by the project were relatively low due to the nature of the technology used so that the beneficiary communities may cover the maintenance costs required for continuing functioning of the water schemes. However, the remoteness of some the schemes from the Woreda centre may constrain the timely and close follow-up and supervision by Woreda Water Resource Development Offices.

 Concerning the nurseries established through the project support, it seems that the nursery sites may likely operate after the end of project period without much difficulty by the WoARD using its own staffs at each Kebele. It is also expected that the resources required to run the day-to-day operation of the nurseries may be mobilized from the government’s regular program and/or the ongoing government’s PSNP capital and administrative budget.

 During the evaluation exercise, it was noted that some Woreda representatives confirmed the allocation of budget for future operation of most of the micro-projects which were funded by CDF including schools, human health and veterinary posts. Accordingly, most of the Woreda sector offices indicated that they were ready to assign the required resources (including human, materials, supplies and other resources) for proper functioning and operation of the infrastructures and/or facilities after the end of the project period.

The Extent to which the services (results) are to be supported 3 Institutionally i.e., the Likelihood of Availability of Funds 5 4 3 2 1 Affordability of Services for the Final Overall Assessment of 0 Beneficiaries at the Completion of Financial/Economic Viability the Project 3 4

Assuming of Financial/Economic Responsibilities by Concerned 3 Persons/Institutions including Maintenance and Replacement

Figure 17: Assessment of Financial Viability of the Project

On the other hand, the evaluation team identified some constraints which are related to the financial viability of some project interventions. These include, but not limited to, the following:

 It seems that the financial viability of some interventions, such as the grinding mill, grain store and slaughterhouse, may be at stake unless effective and proper community management systems for these institutions are installed.

______42 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR  The current financial capacity as well as the capacity to generate funds by the four Woreda Development Associations (Amaro, Basketo, Burji and Derashe) was still found to be very limited so that the financial viability of the Associations may negatively be affected.

 It is unlikely that the insurance scheme designed for goat provision scheme would continue to operate to replace for animals lost through death and related factors. Such poor financial viability might also be related to weak follow-up and poor linkage with mainstream government office such as WoARD.

 The financial sustainability of the micro loan funds (saving and credit schemes) established by the project is also subject for the quality and intensity of future follow-up and support from the side of pertinent government sector offices, mainly the cooperative promotion office of the respective Woreda. Therefore, FARM Africa and concerned Woreda offices shall work together to consolidate the financial and loan management of these community groups and to link them with pertinent government institutions for future follow-up and support.

The evaluation team has also assessed the affordability of the project benefits for final users. In this regard, it was noted that most of the target communities would likely afford for the services rendered by the project outputs including service charges for water and health (human and livestock). For education related services, school operational costs such as salaries for schoolteachers and other supplies would be covered by the local government but the costs of sending schoolchildren to school is limited to school supplies which have low costs for the parents. On the other hand, the affordability of the service of grinding mill for final users might be low because of the ever increasing trend of the price for fuel and spare parts as well as the possibility of higher maintenance and replacement costs. Such constraints might be overcome by installing appropriate cost recovery mechanism and financial controlling system.

7.2 Technical and Social Sustainability

Evaluation of social and technical viability of the project was conducted through assessing the level of ownership of the project by beneficiaries and technical capacity of local actors, respectively, to ensure continuity of project services/outputs after the end of the project support. In line with this, the findings of the evaluation in terms of sense of ownership of communities for the project outputs (including equipment, infrastructure, etc) revealed that there has been satisfactory level of sense of ownership among the target communities for most of the CDF projects like schools, health posts, veterinary posts and water schemes. This promising sense of ownership by the community for CDF project output is mainly the outcome of the active involvement of the target communities in the need identification, planning and implementation of CDF projects. The participation of the community through labour and locally available material contributions during the implementation of the activities was the reflection of communities’ sense of ownership.

However, it was noted that the ownership of some CDF project outputs was not clearly determined, and thus, the project holder in collaboration with its partners should work to clarify the ownership of and devise mechanism to ensure proper and sustainable use and management of these project outputs. These include the slaughterhouse and grain store at Gatto Kebele in Derashe Special Woreda, the public toilets constructed in Holte Kebele of Derashe Special Woreda, and the grinding mill installed at Shala Kanabola and stores constructed for grain marketing in other Kebeles of Basketo Special Woreda. Accordingly, the withdrawal strategy should consider such situations and facilitate the smooth handing over of the outputs through clearly identifying user groups. Similarly, the nursery sites established through the project action by the WDAs should be transferred to the WoARD to ensure the continuity of the operation of the nurseries including in Konso & Derashe Woredas.

______43 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Sense of Ownership of Communities for the Project Outputs including Equipment, 3 Infrastructure, etc. 5

4 The Extent to which the Project Overall Assessment of Technical and 3 embedded in the Local (Community) Social Sustainability 2 Structures 3 3 1

The Extent to which Beneficiaries & Other Proper Training of Project Partners to Stakeholders Involved in Planning, accept Handover of the Project - Design Processes, Decision Making & Technically Project Implementation 2 3

The Likelihood that Target Groups/Beneficiaries continue to Use 3 relevant Services after external support has ended

Figure 18: Technical and Social Sustainability of the Project

The other issue at hand is the poor social capital available in the project area to effectively operate and manage the community resources produced by the project. The lower level of literacy and numeracy at grass root level and low experience in management of group resources, which is common in most rural set-ups of the country, may constrain the social sustainability of the project outputs those which must be managed by user communities themselves.

Assessments were also conducted whether project partners were being properly trained on technical matters to takeover project outputs. In line with this, the findings revealed that for some of the project outputs like water supply schemes the necessary training on water point management and sanitation was given for community members. However, the necessary provisions for Village Level Operation and Maintenance (VLOM) and cost recovery system were not made in most cases. As a result, there were cases where water fee was collected without the necessary receipt and this community resource might be open for individual embezzlement and misuse. In addition, due to lack of the necessary tools at grassroots level, communities may be obliged to use other unsafe water sources even for minor defects on the water supply schemes which can be easily maintainable otherwise. In line with this, the provision of technical training for community operators of the water supply schemes and the grinding mills as well as provision of the necessary toolkits for operation and maintenance found to be indispensable if these schemes are meant to deliver sustainable service to user communities.

The other major concern for technical viability of the WDAs is the high turnover of trained staffs of WDAs and this will adversely affect the opportunity of the WDAs in using the added value obtained from the technical trainings provided by the project. In line with this, the project holder has also indicated that although the project has equipped the WDA’s staff with a number of important experience sharing tours, new and refresher trainings etc, however, most of them have left the organization (WDAs) taking training materials and manuals with them leaving little institutional memory which has contributed to create negative effect on the sustainability of the WDAs.

______44 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR 7.3 Institutional and Managerial Sustainability

During the evaluation exercise, institutional and managerial sustainability of the project was assessed to examine how well the project has contributed to institutional and managerial capacity of its partners to ensure sustainable management and use of project outputs/results. Accordingly, the analysis on how far the project embedded in the institutional structures in order to survive beyond the end of the project indicated that most of the project outputs are expected to be officially handed over to pertinent Woreda government offices, target CBOs and beneficiary communities. Despite capacity limitation of local project stakeholders at different levels, the handed over project outputs might have the possibility to be sustained institutionally.

Assessment was also conducted on how well the project partners were being properly trained on managerial aspects to ensure project sustainability. The findings revealed that similar to the discussion under technical sustainability topic, the project organized various training sessions for WDAs, community members, CBO leaders and staff of government offices in relation to the management of the project outputs after the end of the external support. However, some individuals who were provided with managerial training (especially WDA staff) might not serve the project outputs due to staff turnover, transfer, replacement, demotion, etc

The Likelihood that the Project embedded in the Institutional Structures to Survive beyond the end of the Project 3 5

4 Properly Training of Project Partners to Overall Assessment of Institutional 3 accept Handover of the Project - and Managerial Sustainability 2 Managerially 2 1 2

Relations with New or Existing Availability of defined Phase-out Institutions and Capability of Strategy Plans for its Implementation continuing the flow of Project Benefits 1 3

Capacity Building Effort to Support Project Sustainability 3

Figure 19: Assessment of Institutional and Managerial Viability of the Project It was also attempted to assess the relations of the project with new or existing institutions and the capacity of the institutions to continue the flow of project benefits. Accordingly, it was noted that some of the CDF project outputs like schools, health facilities, nursery and water schemes fall under the mandate of local government sector offices and these sector offices are expected to handle the responsibility of future management of the schemes. However, capacity limitation from the side of Woreda sector offices in terms of logistics, finance and human resource may hinder the proper follow-up and technical backstopping to be given by pertinent sector offices to project users. On the other hand, other project outputs like the initiative of the project to introduce a bottom-up planning approach, Woreda level Database System and Information Centres may need further institutionalisation and reinforcement in order to synchronise them with the existing institutional structure at Woreda and community level.

The other critical issue in terms of institutional viability is related to the fragile institutional and managerial capacity of the four WDAs at Derashe, Burji, Amaro and Basketo Special Woredas. In this regard, despite

______45 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR the provision of the necessary capacity building support to some extent the institutional capacity of the WDAs is remaining weak. This is mainly due to the fact that individuals who were provided with managerial & technical training (from the WDAs) might not serve the project outputs due to staff turnover, transfer, replacement, demotion, etc. Besides to this, the assignment of the WDAs staffs, particularly the management staffs, to other government duties and lack of capacity of the governing bodies of the associations (i.e. board members) are other underlining factors for the weak institutional capacity of the WDAs. As learnt in the course of the evaluation exercise, the continuity of the operation of these Development Associations even for some three to six months without the project support seems questionable. Technical and management staffs of some WDAs have already started to resign (such as in Basketo Special Woreda). In line with this, the local partners in the target Woredas stressed that the WDAs are not in a position to operate at office level let alone the implementation of development projects at community level unless they are supported in the near future.

7.4 Overall Assessment of Sustainability of the Project

Based on the findings of the assessment on financial/economic, technical, social, institutional and managerial sustainability of the project presented above, the overall assessment of the sustainability of the project benefit is summarized in the Figure 16 below. The findings in general revealed that the likelihood of overall sustainability of the project outputs was found to be reasonable/ satisfactory.

Financial/Economic Sustainability 3 5

1 Overall Assessment of Technical and Social 0 Sustainability of the Project Sustainability 3 3

Institutional and Managerial 2 Sustainability Figure 20: Summary of Overall Assessment of Sustainability of the Project

8. Summary of Strengths, Weaknesses, Opportunities and Threats of WCBP

During the evaluation exercise, the evaluation team attempted to assess the major Strengths, Weaknesses, Opportunities and Threats (SWOT) of the project based on the understanding obtained through the evaluation exercise. Among others, the major ones are summarized as follows.

8.1 Major Strengths of the Project

Among others, the evaluation has identified the following major strengths of the project:

______46 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR . FARM Africa was said to be strong in working in inaccessible and remote areas including remote areas of Amaro Woreda and Basketo Woreda where there is no other NGO;

. The project was strong in terms of establishing KDC and VDC and attempting to use the local institutions in need identification, prioritization, project planning, preparation of Community Action Plan (CAP) and implementation (including bid and procurement process, etc) of micro-projects funded through CDF;

. Despite poor/lack of fund raising efforts made so far, the project was strong in terms of organizing useful training programs especially project planning for WDAs which in turn found to be useful in designing project proposals so as to solicit funds from different sources;

. Organising experience sharing events for WDAs to better-off Development Associations (such as Wolaita and Guragie Zones) was found to be useful for WDAs to draw lessons and experience;

. Despite significant implementation delay, the various capacity building initiatives provided to WDAs by the project were very important to enhance their capacity in terms of planning, implementation, monitoring and evaluation of community based development projects;

. The project was strong in enhancing community participation in terms of provision of local materials and labour as well as participation in need identification and prioritisation (especially in Bunity Kebele of Amaro Special Woreda);

. In some cases, the project attempted to integrate the project intervention with PSNP Public Work Activities (such as roof catchments and nursery development in Konso Special Woreda) and regular government intervention (such as water development in Amaro Woreda) which were found to be useful in enhancing effectiveness and sustainability of the interventions;

. The project has created positive externality through being a lesson so that local community in Bunity Kebele has constructed a school block in Amaro Special Woredas through mobilizing its resources;

. Despite poor follow-up actions, the project was strong in organizing quarterly review meetings with its local partners (especially since 2007);

. FARM Africa was strong in terms of its approach/strategy to use WDAs as entry point (phase in).

. The project was found to be relatively effective and strong in enhancing saving and credit activities which has positively contributed to improve the livelihood of the beneficiaries (especially in Burji Special Woreda);

. The attempt made by the project to establish Woreda level resource centre (mini-data) and training on database management was useful in addressing information needs of various actors including government, NGOs, researchers and private sector (such as investors).

8.2 Major Weaknesses of the Project

During the evaluation exercise, a number of weaknesses and/or constraints were identified. Among others, the major ones include, but not limited, to the following:

. The initial project proposal was over ambitious in terms of area/Woreda coverage and volume of activities. In line with this, the project was overstretched in five Special Woredas where some of the

______47 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR intervention Kebeles were inaccessible (such as in Amaro) and remote (such as Basketo Woreda which is about 320km from the project office in Arba-Minch);

. Delay in the preparation of guidelines for the implementation of CDF;

. Poor internal management system (i.e., failure to install effective internal management system) which was fragile to effectively manage the project. As a result, the project failed to maintain its core project staff, delay in decision-making process, etc. For instance, frequent project staff (especially project coordinator) has negatively affected the progress of the project where the project employed five project coordinators in five years of the project period. In addition, it was noted that there had been unhealthy staff relationship and poor staff commitment;

. Delay in timely disbursement/transfer of project budget especially for micro-projects funded through CDF. There was also lack of clarity and transparency in the amount of budget disbursement for same;

. Delay in timely initiation and implementation of most of the project activities including micro-projects funded by CDF and capacity building/training components (e.g., an elementary school in Derashe Special Woreda);

. Delay in operation (provision of services) of completed projects related to water, health post, veterinary post, schools, grinding mills, grain bank, slaughterhouse and others;

. Poor follow-up and monitoring of some project activities such as provision of goats;

. Poor phase out and handing over of most of the completed projects to pertinent stakeholders including grain mill in Basketo Special Woreda, abattoir (a slaughterhouse) in Derashe Special Woreda, nurseries in Konso and Derashe Special Woredas;

. Absence of permanent WDA Manager working on full-time basis. In line with this, there has been frequent turnover of WDA managers. As a result, lack of permanent managers employed by WDAs has negatively affected the efficiency and effectiveness of the overall WDAs;

. Weak capacity, poor commitment and dedication of WDAs’ board, managers and staff;

. In some cases, there was overlap of roles and duplication of efforts in CDF implementation between WDAs and the project holder. For instance, the evaluation team noted that FARM Africa handled some of the roles and responsibility of Basketo WDA in the implementation of the CDF activities where in some of the cases, there was power imbalance where one acts as provider and the other recipient.

. Fail to effectively achieve most of the envisaged project outputs.

. FARM Africa has failed to properly consider some of the recommendations forwarded during the MTR including application of clear CDF guidelines, consolidation of the initiatives which had been implemented before the MTR, commissioning technical consultants to carryout feasibility study on potential business opportunities, and others.

. Lack of transparency from the NGO side in financial budget and use where financial resources were disbursed on lump sum basis without breakdown. There was also stringent fund transfer procedure.

. Lack of logical sequence in the implementation of various project activities. For instance, the various capacity building activities which were implement in final year of project life should have been implemented in the first of the project period including training on proposal preparation.

______48 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR . Lack of analysis of clear targets and beneficiaries for flourmill in Basketo Special Woreda and slaughterhouse in Derashe Special Woreda.

. Inadequate resource allocation for livelihood security in terms of provision of two goats per household which was found to be very minimal compared to other similar intervention in the Region such as the provision of about 4,000 Birr per household by the Regional government to enhance food/livelihood security of poor households.

. The project signatories (especially Regional partners) failed to properly appraise the project proposal at its initial design and amendment period.

. It was noted that there was poor follow up and support from the Regional government partners;

. Unable to sustain the functioning of WDAs beyond the life of the project. There was also poor performance of WDAs in terms of the utilization of the potential local resources through organising local fund raising events, etc.

. Poor capacity assessment of the WDAs in different Woredas where there were more or less similar resource allocation for different Woredas despite heterogeneity of the WDAs in terms of capacity.

8.3 Opportunities for Sustainability of Project Outputs

During the evaluation exercise, the following major opportunities were identified which could have positive impact on and/or favourable conditions to contribute to ensure sustainability of the project:

. The evaluation team noted that almost all Woreda Administrations have shown positive attitude (willingness) towards the WDAs. Accordingly, the Woreda Administrations are ready to support the WDAs through organizing fund raising events and deployment of staff until the WDAs stand by themselves, etc.

. The presence of Southern Ethiopia Peoples Development Association (SEPDA) was found to be an opportunity for WDAs where the WDAs might get the chance to be supported from SEPDA’s various capacity building initiatives.

. The availability of other projects which are run/implemented by some of the WDAs (such as in Konso, Derashe and Basketo Special Woredas) might be an opportunity to at least continue the existence of the WDAs in the near future.

. Existence of promising community commitment and willingness to support WDAs through contribution and membership fee found to be an opportunity for sustainability of WDAs.

. The plan to replicate and scale-up of bottom-up planning approach by the Regional government could be an opportunity to sustain the impact of the project after the FARM Africa’s project.

. Availability of potential donors to support community development projects could be taken as a good opportunity for the continuation of WDAs’ operations in the intervention Woredas.

______49 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR 8.4 Threats to Sustainability of Project Outputs

The evaluation has identified various factors which could have adverse effect on the sustainability of the project actions. Among others, the major threats to ensure sustainability of the project outputs are summarized below.

. It was noted that some of the WDAs (such as in Derashe and Basketo Special Woredas) complained for budget constraints for completion of on-going activities which might be non-functional unless some technical and financial resources are available. It was also noted that the WDAs might encounter problem in terms of paying the unsettled balance for constructors (especially in Basketo Woreda) which might also have legal repercussion, especially from the side of contractors. In addition, local communities and other stakeholders might loss confidence on WDAs and FARM Africa.

. Shortage/lack of financial resources by most of the WDAs might highly affect and even threaten their existence. In line with this, almost all staffs of Basketo WDA have already resigned. Similarly, staff of the WDA in Burji could only pay salary of two of its staff members for the next two-three months.

. Part-time managers who are paid externally by respective Woreda Administration might continue to affect the effectiveness of the WDAs in the future too.

. Conflict and insecurity situation (especially in Burji Special Woreda) might continue to affect the sustainable use of project outputs in the future.

9. Traceability and EC Visibility

It was virtually possible to identify EC’s contribution in all visited intervention Woredas including at project office in Arba-Minch. This was the case for almost all intervention Kebeles and Woredas where the various project activities were implemented by WCBP. Accordingly, the project holder (FARM Africa) put signboards, stating the name of the project along with the donor (EU) and implementing partners, in almost all project sites visited by the evaluation team. In most of the cases, project partners at different levels such as concerned Woreda sector offices, Woreda Administration and WDAs as well as Regional partners (such as BoFED and SEPDA) have full information on the role and contribution of EC in relation to FARM Africa’s WCBP in Amaro, Basketo, Burji, Derashe and Konso Special Woredas.

Figure 21: An Example of EC Field Visibility in Arfaide Kebele of Konso Special Woreda

______50 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR 10. Best Practices and Lessons Learnt

10.1 Best Practices

Among others, the evaluation identified the following two best practices from the implementation of FARM Africa’s WCBP in five Special Woredas of SNNPR:

. Although not institutionalized at Woreda level, the attempt made by the project to introduce bottom-up planning process was taken as one of the best practices which should be scaled-up in other areas and institutionalized at Woreda level.

. Establishment of mini-database/documentation centre at Woreda level through the provision of various capacity building initiatives (such as provision of training, materials/equipment, assist in the collection and compilation of Woreda profile and/or baseline data, etc) was the best practice to be introduced and replicated in other Woredas of the Region and/or Country.

. The partnership approach installed to work through local level development partners (WDAs) was found to be best practice so as to promote sustainable development and empowerment of target communities through facilitation of their direct involvement in their own local development concerns; and

. Documentation of challenges, best practices and dissemination to partners were best practices to be instituted by development actors in their future endeavours to be used as reference and lessons.

10.2 Lessons Drawn

Among others, the major lessons drawn from the implementation of WCBP include, but not limited to, the following:

. Bottom-up planning approach followed by the project was found to be effective in identification and prioritization of needs and preparation of community development projects which should be scaled-up and replicated in other parts of the project Woredas and other Woredas in the Region;

. It was noted that local communities could easily be the major actor in their own development endeavours through the provision of pertinent training on participatory resource use planning, implementation, monitoring and evaluation;

. One of the lessons drawn from the implementation of FARM Africa project was that the strategy/approach in the implementation of CDF should be piloted in modest scale before replicating and scaling-up in larger geographical areas. There is also the need to have clear guidelines and understanding on CDF implementation;

. The project has managed to implement its planned activities under various internal and external challenges including poor commitment and support from some of its partners, ambitious plans in terms of volume of activities and geographical coverage and others. This is found to be a good lesson for similar initiative in the future where there is the need for critical assessment during project design/planning and implementation in terms of various aspects of project management including technical, social, institutional, organizational, managerial, financial and economic aspects at all stages of project management; and

. From the implementation of WCBP, one could learn the need for well defined partnership arrangement and the importance of clearly defined working procedures and principles for different activities to be

______51 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR incorporated under any project. In addition, joint and timely review of project progress by all stakeholders could have positive implication for the success of a project.

11. Major Challenges/Constraints/Problems

Among others, the major problems encountered by the project include the following:

. Conflict and/or insecurity in some of the intervention Woredas has hampered the efficiency and effectiveness of the project intervention (especially in Burji Special Woreda);

. Price escalation project inputs, especially of industrial products, including construction materials such as cement, metal sheets and others;

. Frequent staff turnover in both the project and the WDAs;

. Wide geographical project intervention areas and inaccessibility of some of the project intervention Kebeles, particularly during the rainy season; and

. Logistical constraints which were mainly due to delay in purchase of project vehicles and motor bikes which delayed the timely implementation of planned activities. In addition, a project vehicle was taken by flood which hampered the movement of staff. Similarly, flooding of Segen River affected the movement of staff to project sites.

. Recurrent drought has affected the communities’ participation.

12. Conclusions, Recommendations and the Way Forward

12.1 Conclusions

The final evaluation of FARM Africa’s WCBP revealed that the project has encountered various constraints since its inception. The findings have shown that although the project was relevant and appropriate in addressing the genuine development problems of the intervention areas, however, the project design was too ambitious where a very wide range of activities were planned to be implemented over vast areas found in 25 Kebeles of five Woredas. As a result, efforts were made to refine the initial poor design of the project interventions through MTR, external/internal review and with the two Amendments/revisions made in 2007/2008.

______52 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Project Design 2 5 4 Summary of Overall Project Relevance Evaluation of the Project 3 3 3 2 1 0

Sustainability of the Project Project Efficiency 3 2

Outcomes and/or Impact- Effectiveness of Project Trends of the Project Outputs 3 3

Figure 22: Summary of Overall Evaluation of the Project

Following the MTR, FARM Africa had a chance to critically review the feasibility of the envisaged activities and develop a realistic revised action plans that would ensure the proper and timely implementation of those feasible activities during the initial 48 months of project lifespan and the 12 months no-cost extension period. However, the final evaluation has ascertained that the project period has elapsed before fully realizing its intended results. The reflections during the final evaluation workshop also revealed that the effectiveness of the investment on the project initiatives has been highly affected by the unaccomplished activities which all are critical towards achieving the project key purposes. It was also noted most of project activities were implemented during the last 12 months period, and thus, not allowing to have enough time to undertake the follow-up and consolidation of activities which are very essential for ensuring continuity and sustainability of the project initiatives.

As it is illustrated in Figure 18, the project design and efficiency of the project was relatively given lower ratings as compared to the rest of evaluation indicators which might be related to poor project planning, delay in timely implementation of some project activities, poor project management and others. On the other hand, the findings ascertained that the project was also poor in terms of achieving most of its expected outputs in the last five years. It was untimely to assess the impact of the project as the implementation of most of the project actions were either completed more recently or not completed until the evaluation period. However, the trends have shown that the project outputs may have a possibility to bring about positive impact on its targets and beneficiaries in future if further actions are designed and implemented as per the recommendations forwarded by the consultants. Similarly, the sustainability of the project was constrained by various internal and external factors which also need critical attention to ensure viability of the project in terms of financial, technical, social, institutional and managerial sustainability. In general, the evaluation team concluded that despite the great effort exerted by the current project team in the implementation of most of the project activities during the last one-year, however, FARM Africa’s WCBP was generally found to be weak in realizing its envisaged overall and specific objective due to the internal and external challenges faced by the project, which have negatively affected its design, efficiency, effectiveness, impact and sustainability.

______53 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR 12.2 Recommendations a) The project holder (FARM Africa) should assist its partners (especially WDAs) through availing the required resources to finalize uncompleted on-going projects in intervention Woredas. In line with this, there is the need to allocate financial resources and assign focal person at its Country Office in order to facilitate the completion of ongoing projects, handing-over of project outputs to pertinent stakeholders and phase out; b) It is commendable that FARM Africa should further assist the WDAs to make them strong and locally accountable development actor through provision of technical assistance in areas of fund-raising (including linking the WDAs with potential donors), human resource development, and preparation of on-shelf projects and project design templates to be used in the future; c) WDA Boards in collaboration with local governments and other pertinent partners (such as SEPDA) should critically examine the internal and external challenges/constraints hampering efficiency and effectiveness of WDAs and look for possible remedies. In addition, strategic leadership of the WDAs should be strengthened through the assignment of competent and committed board members. Similarly, there is also the need for the assignment of full-time managers for the WDAs on permanent basis from own source; d) It is advisable that all concerned actors at Regional and Woreda levels should assist and promote the scaling-up/replication of the Bottom-Up Planning Approach based on the lessons drawn from the implementation of FARM Africa’s WCBP; e) The evaluation would like recommend the need to exert greater effort to regularly and timely update the Woreda Database System and proper use of information centres which were established through the project action; and f) It is advisable that all concerned government actors at different levels should strictly follow-up, monitor and support the sustainable use of the project outputs including economic and social services and facilities related to water development, health (human and livestock), education (schools), nursery sites, slaughterhouse, grain stores, grinding mill, saving and credit schemes, etc;

12.3 The Way Forward

Based on the threats and recommendations presented by the evaluation team and discussed on the final evaluation workshop, the following suggestions are forwarded for urgent actions/consideration by concerned bodies including the project holder and its Regional and local partners such as Regional Partner (especially BoFED), WDAs and respective Woreda Administration.

A. The Project Holder - FARM Africa

i. FARM Africa has to seek ways for the proper accomplishment of the remaining activities including the consolidation and handing over of the project initiatives using its own source of funding to cover the required costs for keeping the existing project core field staff and providing of additional backstopping by its Country Office. ii. FARM Africa should urgently initiate, jointly with the appointed staff of the Regional BoFED, the development of action plans for implementation of the remaining activities with required human and financial resources and forward to both FARM Africa and BoFED management as soon as possible.

______54 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR B. Regional Bureau of Finance and Economic Development (SNNPR)

iii. The Regional BoFED should closely follow the timely development and approval of the action plan jointly with FARM Africa for implementation of the remaining project activities.

iv. The Regional BoFED should provide the required technical and administrative support including the approval of the project extension and also use of the project facilities such as vehicles, office equipment/materials, etc.

C. Woreda Development Associations (WDAs) in the Five Special Woredas (SNNPR)

v. The WDAs in Amaro, Basketo, Burji, Derasho and Konso Special Woredas should urgently call meetings of the board members and the general assembly to report their current situations and discuss on critical issues which need urgent actions such as means of keeping the existing key WDA staff and the required budget to cover other operational costs.

vi. The WDAs should urgently develop mechanisms for fund raising actions including the timely collection of membership fee and organization of the planned fund raising bazaar. vii. The WDAs should organize urgent meeting with SEPDA to discuss on their future partnership arrangements and funding opportunities through joint implementation.

D. The Five Woreda Government Administrations viii. The Administration in the project intervention Woredas should provide the required support through its concerned sector offices and if necessary the temporary deployment of the required management staff and experts to the WDAs.

ix. If there is any possibility, the Woreda Administration should consider funding of WDAs projects from its government capital budget sources.

______55 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Annexes

List of Contacts No. Name of Contacts Institution Position/Title 1 Tamiru Getaneh Derashe Development Association Manager 2 Aschalew Eshetu Derashe Development Association Field Officer 3 Desalegn Kingawa Derashe Finance and Economic Development Office Deputy Head 4 Sileshi Bogale Derashe Education Office Education Development Coordinator 5 Negussie Acha Derashe Health Office Acting Head 6 Degu Datiko Derashe Administration Office Administrator 7 Shita Shi’o Derashe Water Resource Development Office 8 Aman Enyew FARM Africa Derashe WCBP Capacity Building Officer 9 Netsanet Hailu Derashe Development Association Board Member 10 Tarkaso Aji’a Derashe Agriculture and Rural Development Office 11 Baleko Kesiya Konso Agriculture and Rural Development Office Acting Head 12 Geremew Geyeto Konso Education Office Acting Head 13 Tesfaye Kalala Konso Finance and Economic Development Office Development Planning Officer 14 Karale Kasawo Konso Development Association Program Manager 15 Boda Ado Burji Finance and Economic Development Office Planning Officer 16 Shibru Gale Burji Health Office Health Service Promotion Team Leader 17 Wondimu Kassa Burji Cooperative Development Office Coordinator 18 Birhane Dejene Burji Water Resource Development Office Team Leader , Community Participation 19 Ote Kamile Burji Development Association Board, Vice Person 20 Addisu Tuke Burji Development Association Manager 21 Kana Zarko Burji Development Association Field Coordinator 22 Kusya Bare FARM Africa Burji WCBP Capacity Building Officer 23 Dejene Kassaye Burji Administration Office Deputy Administrator 24 Mekonnen Emale Burji Education Office Expert 25 Deguye Sheno Burji Development Association Development Worker 26 Belay Bekele Tinishua Keyate Kebele Development Committee Chair Person 27 Megos Moroma Amaro Administration Office Administrator 28 Lemma Beyene Amaro Education Office Coordinator of Education Promotion 29 Israel Tadesse Amaro Water Resource Development Office Planning and Programming Expert 30 Mesele Agafari Kore Development Association Acting Manager 31 Asmamaw Mekuria Kore Development Association Board, Deputy Chair Person

______56 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR No. Name of Contacts Institution Position/Title 32 Alemayehu Ture Kore Development Association Field Officer 33 Tesema Tamiru Amaro Health Office Acting Head 34 Kifle Girma Amaro Finance and Economic Development Office Project Monitoring & Evaluation Officer 35 Tariku Teklu Kore Development Association Administration & Finance Service Head 36 Temesgen Kassa Agri Service – Ethiopia (Amaro Project) Program Director 37 Hailu Olango South Ethiopia Peoples Development Association (SEPDA) Acting General Manager 38 Muluneh Belachew Basketo Administration Office Administrator 39 Tariku Bekele Basketo Administration Office Deputy Administrator 40 Berhanu Chulido Basketo Development Association Board Member 41 Kena’a Koto Basketo Development Association (Basketo Police Office) Board Member 42 Zelalem Mekuria Basketo Finance and Economic Development Office Project Planning, Monitoring & Evaluation Coordinator 43 Tariku Mala Basketo Health Office Health Extension Programme Coordinator 44 Atnafu Wondimu Basketo Development Association Manager 45 Behailu Babaro Basketo Agriculture and Rural Development Office Head 46 Delelegn Habtamu Basketo Cooperative Office Head 47 Kefyalew Belete Basketo Water Resource Development Office Head 48 Mandefrot Worku Basketo Development Association Board, Deputy Chair Person 49 Mesfin Mersha FARM Africa Basketo WCBP Capacity Building Officer 50 Wondu Gebre-Mariiam SNNPR Bureau of Finance and Economic Development Expert 51 Aregawi Tesfaye SNNPR Bureau of Finance and Economic Development Team Leader 52 Johnathan Napier FARM Africa Country Office Country Director 53 Wondimu Ababu FARM Africa WCBP Office WCBP Administration & Officer 54 Tekle-Aregay Jirane FARM Africa WCBP Office WCBP Coordinator 55 Berhanu Taye Delegation of European Commission to European (Rural Task Manager Development and Food Security Section, NGO Program)

Itinerary of Evaluation Team Date Place Major Task July 10-11, 2009 Addis Ababa  Meeting and discussion with concerned staff of EC Delegation to Ethiopia.  Document review.  Preparation of methodology and detailed work plan including checklist. July 12, 2009 Addis Ababa to Arba-Minch  Travel from Addis Ababa to Arba-Minch

______57 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Date Place Major Task July 13-14, 2009 Arba-Minch  Briefing with FARM Africa Project Office Staff in Arba-Minch.  Setting-up participatory project evaluation team.  Orientation of and discussion with evaluation team on evaluation methodology.  Discussion and agree on evaluation checklist with evaluation team members. July 15, 2009 Derashe Special Woreda  Meeting and discussion with Derashe Special Woreda partners including WDA, Woreda Administration and concerned Woreda Sector Offices.  Visit project activities in Gatto Kebele including grain bank, slaughter house, nursery, school block rehabilitation and school furniture (desk).  Meeting and discussion with members of KDC, VDC, Kebele officials, Development Agents and beneficiary community members in Gatto Kebele.  Visit project activities in Holte Kebele including public toilet and school furniture (desk).  Meeting and discussion with members of KDC, VDC, Kebele officials, Health Extension Workers and beneficiary community members in Holte Kebele. July 16, 2009  Visit project activities in Busa-Baso Kebele including Tekolcho Spring Development, school block floor rehabilitation and school furniture (desk).  Meeting and discussion with Busa-Baso Kebele Manager, Development Agent, Tekolcho Spring water users and members of community in Busa-Baso Kebele.  Discussion and debriefing with Derashe Special Woreda partners including FARM Africa WCBP Capacity Building Officer, Staff and Board of WDA, Woreda Administration and concerned Woreda Sector Offices. July 17, 2009 Konso Special Woreda  Visit project activities in Sorobo Kebele including water development activity (roof catchments) and Toilet in Sorobo Elementary School.  Meeting and discussion with members of KDC, VDC, Kebele officials, school teachers and community members in Sorobo Kebele.  Visit water development activity (pipe line extension) in Afrayide Kebele.  Meeting and discussion with members of KDC, VDC, Kebele officials, Water Committee, Water Users, Goat Beneficiaries, CAHWs and community members in Afrayide Kebele. July 18, 2009  Visit project activities in Loltu Kebele including water development activity (Koto spring development), education (school block construction and school furniture in Loltu primary school) and Koto Nursery site.  Meeting and discussion with members of KDC, VDC, Kebele officials, school teachers, Water Users and beneficiaries of Nursery development in Loltu Kebele.  Discussion and debriefing with Konso Special Woreda partners including Staff of WDA and concerned Woreda Sector Offices (Offices of Education, Agriculture & Rural

______58 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Date Place Major Task Development and Finance & Economic Development). July 19, 2009 Burji Special Woreda  Visit project activities in Tinishua-Keyate Kebele including education (school furniture (Morning) and school block expansion) and human health post.  Meeting and discussion with members of KDC, VDC, Kebele officials, beneficiaries of goats and saving & credit groups in Tinishua Keyate Kebele. July 19, 2009 Amaro Special Woreda  Visit project activities in Bunity Kebele including education (school block expansion) and (Afternoon) human health post.  Meeting and discussion with members of KDC, VDC, Kebele officials, health workers, elder, religious leader, schoolteachers, beneficiaries of goats and community members in Bunity Kebele. July 19, 2009 Burji Special Woreda  Visit water development activity (Berek Spring Development) in Berek Kebele (Afternoon) July 20, 2009 Amaro Special Woreda  Visit water development activity (Gumure Water Supply) in Gumure Kebele.  Meeting and discussion with members of community including water users and water committee members in Gumure Kebele.  Visit project activities including Godoba Access School and area enclosure in Derba Kebele. July 21, 2009  Discussion with Amaro Special Woreda partners including board & staff of WDA and (Morning) concerned Woreda Sector Offices (Offices of Education, Health, Water Resource Development and Finance & Economic Development). July 21, 2009 Burji Special Woreda  Visit project activities including veterinary post in Kilicho Kebele and spring water (Afternoon) development in Lemo Kebele.  Discussion and debriefing with Burji Special Woreda partners including FARM Africa WCBP staff members, staff and board of WDA, Woreda Administration and concerned Woreda Sector Offices (Offices of Education, Health, Water Resource Development, Finance & Economic Development and Cooperative). July 22, 2009 Amaro Special Woreda  Final discussion and debriefing with Amaro Special Woreda partners including board & (Morning) staff of WDA, Woreda Administration and concerned Woreda Sector Offices (Offices of Education, Health, Water Resource Development and Finance & Economic Development).  Meet and discuss with Agri-Service Ethiopia (Amaro Project) Program Director in Amaro Woreda. July 22, 2009 Hawassa  Meet and discuss with South Ethiopia Peoples Development Association (SEPDA) (Afternoon) Acting General Manager in Hawassa. July 23, 2009 Hawassa to Baketo  Travel from Hawassa to Baketo Special Woreda

______59 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Date Place Major Task July 24, 2009 Basketo Special Woreda  Meeting and discussion with Basketo Special Woreda partners including FARM Africa WCBP Capacity Building Officer, board & staff of WDA, Woreda Administration and concerned Woreda Sector Offices (Offices of Health, Water Resource Development, Cooperative and Finance & Economic Development).  Visit project activities in Mendita Kebele including grain bank, health post and spring water development.  Discussion with beneficiaries of grain bank, contractors (water and health post contractors) Kebele officials and members of local communities in Mendita Kebele.  Visit project activities in Obcha Kebele including grain bank, school block construction and veterinary post.  Discussion with beneficiaries of Kebele officials and members of local communities in Obcha Kebele. July 25, 2009  Visit project activities in Shela-Kanabola Kebele including school block construction, flourmill and spring water development.  Discussion with Kebele officials and members of local communities including goat beneficiaries in Shela-Kanabola Kebele.  Final discussion and debriefing with Basketo Special Woreda partners including FARM Africa WCBP Capacity Building Officer, board & staff of WDA, Woreda Administration and concerned Woreda Sector Offices (Offices of Health, Water Resource Development, Cooperative and Finance & Economic Development). July 26, 2009 Basketo to Arba-Minch  Travel from Basketo Special Woreda to Arba-Minch. July 27, 2009 Arba-Minch  Wrap-up discussion and debriefing with FARM Africa WCBP Staff in Arba-Minch. July 28, 2009 Arba-Minch to Hawassa  Travel from Arba-Minch to Hawassa. July 29, 2009 Hawassa  Compilation of findings and preparation for final evaluation workshop. July 30, 2009  Presentation of findings and recommendations at final evaluation workshop. July 31, 2009 Hawassa to Addis Ababa  Travel from Hawassa to Addis Ababa. August 31, 2009 Addis Ababa  Submission of the draft report to FARM Africa. October 28, 2009  Provision of comments by FARM Africa. November 3, 2009  Discuss with FARM Africa Country Office on the comments of the draft report. November 11, 2009  Submission of Final Report to FARM Africa.

______60 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Table: Case Study on the Impact of Goat Provision at Household Level Source of Status of Repayment Value of Repayment (goat off Net Value No. of Value of No. of No. of Goats Goats Sold No. of Value of (Fully, springs, Added (in Woreda Goats Goats Goats Sold or or Goats Goats Partially, other own Amount Cash and Name of Beneficiary (Kebele) Provided Provided Died Slaughtered Slaughtered at Hand at Hand None) sources) Repaid in Kind) Remark Repaid through purchase from Other local market Konso Fully own from own 1. Tinno Oltech (Afraide) 2 160 1 4 380 Repaid Source 160 380 source Sale of a goat to cover school Amaro Fully Goat Off related expense 2. Tayech Endale (Bunity) 2 215 2 1 190 9 2,250 Repaid springs 180 2,405 (clothing) The income from the sale of goats was used Basketo to cover (Shella various Kana Fully Goat Off household 3. Shela'ane Amanu Bola) 2 210 2 3 400 2 350 Repaid springs 200 740 expenses The income from the sale of goats was used Basketo to cover (Shella various Kana Fully Goat Off household 4. Meselech Kojinko Bola) 2 250 2 170 1 170 Repaid springs 250 340 expenses Basketo One goat was (Shella slaughtered for Kana Fully Goat Off the household 5. Dirbete Engida Bola) 2 180 1 100 2 250 Repaid springs 180 350 consumption

Average 2 203 1 1 172 4 680 - - 194 843

______61 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Checklist for Evaluation

General Introduction

This checklist is meant to solicit information so as to evaluate FARM Africa’s project entitled “Woreda Capacity Building Programme” in Five Special Woredas (Konso, Derashe, Basketo, Burji and Amaro) of the SNNPR. The Checklist will serve to guide interviews and group discussions with partner organizations and target groups of the projects. While the questions below touch up on important issues that are common to most of the informants, the set of questions used for particular project component may vary according to the context of a particular project activities and types of partners involved. It is envisaged that the evaluation team will conduct discussion with key informants (beneficiaries), community group at village level, different project groups which were established by the project, concerned government offices at Woreda levels. Meeting and discussion will also be conducted with the project staff and donor agency. In addition to the discussions and meetings with different stakeholders, the team will also make field visits to observe the project activities.

1. Overall Program Design

1.1. What is the extent of participation of beneficiaries and other stakeholders in need identification? 1.2. How clear have the project preparation and submission process been? 1.3. Did partners get adequate consultation in the preparation and submission of projects at different levels? 1.4. Were different stakeholders ready for working in partnership? 1.5. Existence of well defined performance indicators for the project objectives and activities. 1.6. Degree of objective-oriented planning; existence of baseline data, clarity of expected outputs, assumptions and risks, monitoring and evaluation plan. 1.7. Use of Logical Framework Analysis (LFA) with well defined performance indicators for each objectives and activities.

2. Relevance of the Project (positive attitude towards the given intervention)

2.1. What were the most important problems, needs and priorities of the communities of the project area before the start of the project? 2.2. What were the development priorities of local institutions, associations and government before the start of project activities in the area? 2.3. How relevant was the project approach in relation to addressing the project objectives? 2.4. How is your judgement/perception/observation on the contribution/ability of the project in reducing poverty? 2.5. To what extent is the project design consistent with the local situation and the coherence of activities? i.e., were the project activities consistent with circumstances, traditions and perceptions of the community? 2.6. How appropriate were the project objectives? Did the project objectives accurately address stakeholders need? i.e., did the project addresses priority areas and pressing problems of the communities and other stakeholders? 2.7. Were higher-level community and government priorities & policies addressed by the project? 2.8. The extent to which the project reflects beneficiaries’ needs and address problems identified during project preparation, 2.9. Were the different project activities complement each other and lead to the overall objective of the project? How?

______62 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR 2.10. The consistency of the design with the local situation and the coherent complementarily of the proposed activities within its integrated approach. 2.11. How responsive and flexible is the project document to problems identified during proposal development and the Mid Term Review? i.e., was the project design flexible and posses the room for amendments and incorporation of encountered important problems of the communities after its design?

3. Efficiency of the Project

3.1. How does the projects’ performance and outputs compare against its resource utilization? 3.1.1. What disbursement procedures were followed and who were involved in decision making on financial expenditures? 3.1.2. Have there been unanticipated expenditures? 3.1.3. Have there been costs not quantified at the start of the project? If any, how have these been accounted for? 3.2. How efficiency was assessed during the project implementation? 3.3. How efficient the inputs are converted to outputs? i.e., did the project achieve the expected outputs according to the resources allocated and the time frame given to each activity? 3.4. How did the project costs and benefits compare? Are the project benefits exceed project costs for those activities that are achieved during the given time and resources? 3.5. The extent to which activities were implemented and managed, i.e., how the quality of project management at different levels of the project and in managing of the project budget? 3.6. The extent to which activities were carried out in the most appropriate manner given available resources and time, i.e. according to your opinion, was the project efficient in achieving its planned objectives with the allocated resources and the given time? 3.7. The extent to which the above was done at the minimal cost to produce the expected results. 3.8. Whether similar results could have been achieved in a better way, by other means, at lower costs within the same time-frame, or at the same cost but in less time, i.e., was there any other alternative that could help to achieve similar project benefits with lower cost or the same cost but lesser time? 3.9. Availability of cases of duplication and overlapping of activities with other actor(s) in the same locality. 3.10. Was the project launched at the right time in the area? 3.11. Timely implementations of activities, i.e., were different activities of the project implemented at the right time and according to the project plan of implementation? 3.12. How was the work relationship of the project management and staff with different stakeholders, beneficiaries and other local institutions and authorities, joint monitoring, agreement with stakeholders, working relationship, etc? 3.12.1. How efficient the partnership among different stakeholders? 3.12.2. What measures have been taken to strengthen partnership (e.g. sharing of responsibilities, regular interaction and exchanges, other aspects, etc) 3.13. The quality of internal monitoring/review procedures including the use of efficiency indicators. 3.14. Availability of implementation procedure between project management and concerned stakeholders/institutions such as memorandum of understanding. 3.15. Level of community participation in decision making as well as contribution for the project during implementation of various project activities. 4. Effectiveness of the Project (achievement of the project objectives, success attribution, objective realization, expectation realization, etc)

______63 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR 3.1. What is the progress of the project to date towards these outputs? i.e., the extent to which the project is achieving expected results. Did the outcomes of the project meet the project objectives? 3.1.1. Output 1: Five strong, sustainable and locally accountable WDAs built that are able to implement a process of locally initiated participatory development. 3.1.2. Output-2: The capacity of relevant Woreda government offices, WDAs, KDCs and VDCs built to develop, implement, monitor and evaluate a participatory planning system. 3.1.3. Output-3: Increase the livelihood base of the local community and most vulnerable households in each of the five Woredas. 3.1.4. Output-4: Write up lessons learnt from the WCBP and disseminate to donors, government, NGOs and regional networks. For instance, how has the project gone about documenting and disseminating project experience and impact? And how effective has it been? 3.2. Extent to which the results achieved are reinforcing the programme’s Objective. 3.3. Whether the planned purposes/outcomes have been achieved and whether the planned benefits have been reaped by the intended beneficiaries. 3.4. In relation to the programme as a whole 3.4.1. The factors influencing the achievement of the programme objective including unforeseen external factors. 3.4.2. The management capacity to ensure that the results help to the attainment of the stated purpose. 3.4.3. Did the project actually achieve stakeholder satisfaction? 3.4.4. The reaction of beneficiaries and the use of project results and benefits. 3.4.5. The unplanned effects of obtained results on cross-cutting issues such as gender, the environment and poverty reduction. 3.4.6. How do you see the strategies used to implement the project? Were they appropriate and effective? 3.4.7. What monitoring arrangements have been made for a joint follow-up progress in implementation? (e.g. joint supervision, reporting, etc)

5. Impact of the Project

5.1. What are the positive and negative changes that the project brought for different stakeholders? i.e., the effects of the project on its direct beneficiaries as well as to its wider, overall effect on larger numbers of people, within the sector or in a geographic area in terms of technical, economic, socio-cultural and institutional factors. 5.2. What evidence is there that the project has had an impact and in which output(s)? 5.2.1. Output 1: Five strong, sustainable and locally accountable WDAs built that are able to implement a process of locally initiated participatory development. 5.2.2. Output-2: The capacity of relevant Woreda government offices, WDAs, KDCs and VDCs built to develop, implement, monitor and evaluate a participatory planning system. 5.2.3. Output-3: Increase the livelihood base of the local community and most vulnerable households in each of the five Woredas. 5.2.4. Output-4: Write up lessons learnt from the WCBP and disseminate to donors, government, NGOs and regional networks. 5.3. The extent to which the overall objectives were achieved and the contribution of the project to their achievement.

______64 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR 5.4. In your opinion, what will be the impact of the project after the withdrawal of the implementing agency? Including impact trends of the project. 5.5. The external factors which influenced the overall impact as well as the capacity of the project to respond to these factors i.e. influence of the changing context on the project. 5.5.1. How much has the context of WCBP changed since the proposal was written and how has this affected implementation? 5.5.2. What are the key issues in the new context of Ethiopia that need to be taken into account for the future of the project? 5.5.3. How can opportunities in the changing context be taken into account to ensure successful project impact? 5.6. Possible unplanned impacts of the project and the effects of this on the overall impact. 5.7. What are the impacts of the project on gender-related, environment and poverty issues? 5.8. Are project objectives still realistic and achievable?

6. Sustainability of the Project

6.1. What arrangements have been made to ensure sustainability of the project? 6.2. Sustainability of activities already implemented. 6.3. Assess their potential to continue to help attain the programme objective once the programme has been phased out, for instance, the capacity of the beneficiaries/communities to maintain and continue the activities and/or outputs of the projects, the support required by different stakeholders to sustain the different project activities, unforeseen obstacles on sustainability, and so on. 6.4. Was there any capacity building effort to support the project implementation and its sustainability? 6.5. Effect of external factors on sustainability. 6.6. How is the sense of ownership of communities for the project outputs including equipment, infrastructure, etc? 6.7. How did the self-reliance of the communities and their determination in taking over and guaranteeing the overall project continuity?

7. Other Important Issues

7.1. What do you think were the major strengths of the project? 7.2. What were its weaknesses? 7.3. What were the major opportunities/favourable conditions for the realization of project objectives? Including opportunities for sustaining the WDAs. 7.4. What can be said about threats of the project? 7.5. What problems or (capacity and others) constraints have been faced in the implementation of the project? For instance, what are the constraints affecting WDAs? 7.6. What do you think of the major critical issues that have hampered for effective realization of the project objectives? For instance, what are the key issues at project/Woreda/region level influencing the progression of WDAs to accountable and independent civil society organisations? 7.7. What could be done to improve the problems and constraints? 7.8. What are the key lessons drawn from this project that will be used for similar intervention in the future? 7.9. Financial status of the project. 7.10. Monitoring and supervision mechanism including reporting. 7.11. Capability of target communities to resist shocks, and so on.

______65 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR 7.12. What are the best partnership practices which could be replicated in the area and considered for future development activities? 7.13. What are your practical recommendations in relation to implementation strategy and modalities as well as to improve the overall programme including participation, phasing out, sustainability, and the way forward for the extension of the programme by the agency in the future, etc? 7.14. Any other issues you would like to raise for the evaluation team.

______66 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Summary of Ratings based on Major Evaluation Indicators (By Evaluation Team)

1. Project Design Assess the key elements of the project using the following (0-5) value scale where 0 = Not mentioned in the project document, 1 = Poor, 2 = Weak/Less than Satisfactory, 3 = Amaro Basketo Burji Derashe Konso Average Average/Satisfactory/Adequate, 4 = Good/More than Satisfactory, 5 = Excellent 1) Level of participation of beneficiary & other stakeholders in need identification 2 2 2 2 2 2 2) Clarity of the project preparation and submission process 2 2 2 2 2 2 3) Adequacy of consultation with partners in the preparation and submission of projects at different 2 2 2 2 2 levels 2 4) Clarity of immediate objectives, including specification of targets 3 3 3 3 3 3 5) Specification of beneficiaries 3 3 3 3 3 3 6) Specification of outputs and output targets 2 2 2 2 2 2 7) Specification of inputs: donor, NGO, Community 3 3 3 3 3 3 8) Validity of means - ends relationship between inputs, outputs and objectives 2 2 2 2 2 2 9) Clarity and Appropriateness of Implementation arrangements and managerial structure 2 2 2 2 2 2 10) Clarity and Realism Work Plan including timing of inputs, activities and outputs 2 2 2 2 2 2 11) Realism of identified prerequisites and risks for project success 2 2 2 2 2 2 12) Adequacy of partnerships with other related institutions and organizations 3 3 3 3 3 3 Overall Assessment of the Project Design (Score 1-5) 2 2 2 2 2 2

2. Project Relevance Use a value scale of 1 to 5 where 1=very poor 2= rather unsatisfactory 3=satisfactory 4=good Amaro Basketo Burji Derashe Konso Average 5=excellent/highly relevant a) Did the project address a genuine development problem? (1=not at all 2=hardly 3=yes, 4 4 4 3 4 somewhat 4=yes to an important problem 5= yes, to a major problem) 4 b) If the project addresses a genuine development problem, how well did the project 2 2 2 2 2 provide a cost-effective response to that development problem? (1=not at all 2=barely appropriate 3=satisfactory 4=highly appropriate 5=the most appropriate possible) 2

______67 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Use a value scale of 1 to 5 where 1=very poor 2= rather unsatisfactory 3=satisfactory 4=good Amaro Basketo Burji Derashe Konso Average 5=excellent/highly relevant c) Did the project form part of a coherent national program? (1=not at all 2=only slightly 4 4 4 3 3 3=linked 4=well integrated 5=totally integrated) 4 d) Were there reasonable expectations that adequate resources could be committed to the 3 2 3 2 3 project activities during or after project phase out? (1=not at all 2=only slight 3=reasonable 4=very little doubt 5=absolutely certain) 3 e) Was it realistic to expect project outputs to continue to be used once the project was 3 2 3 2 3 completed and adequate resources to be committed for meaningful follow-up? (1=no or very little expectation 2= slight expectation 3=some expectations 4=very reasonable expectations 5=very strong expectations) 3 Overall assessment of project relevance (appropriateness) 3 3 3 2 3 3

3. Project Efficiency

Assess Project Efficiency on the following (1-5) value scale where: 1 = Very Poor, 2 = Less than Satisfactory, 3 = Average/Satisfactory, 4 = Above Average/Good, 5 = Excellent Amaro Basketo Burji Derashe Konso Average I. Project Management & Implementation

a) Inputs

1. Donor and NGO Inputs:

(i) Budgetary disbursements, i.e. amount and timelines 2 2 2 2 2 2 (ii) Project personnel including consultants, i.e. quality and quantity 2 2 2 2 2 2 (iii) Equipment and supplies, i.e. quality, quantity & timeliness 2 2 2 2 2 2 Overall Assessment of Donor and NGO Inputs 2 2 2 2 2 2

2. Local Government and community inputs (if any)

(i) Personnel/Experts, i.e. quality and quantity 3 2 3 2 4 3 (ii) Community contribution (in cash and/or in kind) 5 4 4 2 3 4 (iii) Other local government's input, if any 4 3 3 2 4 3

______68 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Assess Project Efficiency on the following (1-5) value scale where: 1 = Very Poor, 2 = Less than Satisfactory, 3 = Average/Satisfactory, 4 = Above Average/Good, 5 = Excellent Amaro Basketo Burji Derashe Konso Average Overall Assessment of Local Government and Community Input 4 3 3 2 4 3

Overall Assessment of Inputs 3 3 3 2 3 3

b) Internal Management

3. Project work planning, monitoring and reporting, i.e. quality, quantity & timelines 2 2 2 2 2 2 4. Coordination and relation with other organizations/departments 3 3 3 3 3 3 5. Flexible management response to problems and/or changed circumstances 2 2 2 2 2 2 6. Area Project Coordinator - Was the Area Project Coordinator with sufficient technical staff to 2 2 2 2 2 run project management 2 7. Assessment of the effectiveness of Woreda Coordinator 2 2 2 2 Overall Assessment of Internal Project Management 2 2 2 2 2 2 c) External Support/Inputs to Management and Implementation

8. Technical support by the NGO head office 2 2 2 2 2 2 9. Administrative support by the NGO head office 2 2 2 2 2 2 10. Monitoring and Technical support by the Donor Agency 3 3 3 3 3 3 11. Management support/Decision-making by local government 2 2 2 2 2 2 12. Assessment of evaluation and review processes 4 4 4 4 4 4 Overall Assessment of External Support 3 3 3 3 3 3

d) Implementation of Project Activities

13. The extent to which activities were implemented in most appropriate manner given available 3 1 2 2 3 resources & time 2 14. How efficient the inputs are converted to outputs? i.e., did the project achieve the expected 2 2 2 2 2 outputs according to the resources allocated and the time frame? 2

______69 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Assess Project Efficiency on the following (1-5) value scale where: 1 = Very Poor, 2 = Less than Satisfactory, 3 = Average/Satisfactory, 4 = Above Average/Good, 5 = Excellent Amaro Basketo Burji Derashe Konso Average Overall Assessment of Implementation of Project Activities 3 2 2 2 3 2

OVERALL ASSESSMENT OF PROJECT IMPLEMENTATION & MANAGEMENT (Score 2.59 2.20 2.37 2.20 2.55 2.38 1-5) II. Beneficiary Targeting Procedure & Applied Criteria 3 2 3 2 4 3 III. Resource Allocation/Spatial Aspects 4 2 4 2 3 3 IV. Consideration of Gender Aspects 3 3 3 3 3 3 V. Timeliness i.e., timely implementations of activities, i.e., different activities of the project 1 1 1 1 1 implemented at the right time and according to the project plan of implementation 1 VI. Cost Effectiveness i.e., how efficient the inputs are converted to outputs? i.e., the extent to 2 2 2 2 2 which the project was implemented at the minimal cost to produce the expected results 2 OVERALL PROJECT EFFICIENCY ASSESSMENT (Score 1-5) 3 2 3 2 3 2

4. Effectiveness Project Outputs

The assessment of the outputs produced should be made with respect to the planned targets. Assess the outputs of the project using the following (1-5) value scale where: 1= Very Poor, 2= Less than Satisfactory, 3= Average/Satisfactory, 4= Good, 5= Excellent Amaro Basketo Burji Derashe Konso Average 1. Has the project produced the anticipated results/outputs? i.e. extent to which the project is achieving expected results/outputs Output 1: Five strong, sustainable and locally accountable WDAs built that are able to implement a 1 1 2 2 4 process of locally initiated participatory development. 2 Output-2: The capacity of relevant Woreda government offices, WDAs, KDCs and VDCs built to 3 3 3 3 3 develop, implement, monitor and evaluate a participatory planning system. 3 Output-3: Increase the livelihood base of the local community and most vulnerable households in 2 2 4 2 3 each of the five Woredas. 3 Output-4: Write up lessons learnt from the WCBP and disseminate to donors, government, NGOs 3 3 3 3 4 and regional networks. 3

______70 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR The assessment of the outputs produced should be made with respect to the planned targets. Assess the outputs of the project using the following (1-5) value scale where: 1= Very Poor, 2= Less than Satisfactory, 3= Average/Satisfactory, 4= Good, 5= Excellent Amaro Basketo Burji Derashe Konso Average OVERALL ASSESSMENT OF OUTPUTS ACHIEVED 2 2 3 3 4 3

2. Extent to which the results achieved are reinforcing the programme’s Objective i.e. have the Objectively Verifiable Indicators (targets according to the logical frame) been achieved as planned to date? 0 Output 1: Five strong, sustainable and locally accountable WDAs built that are able to implement a 1 1 2 2 4 process of locally initiated participatory development. 2 Output-2: The capacity of relevant Woreda government offices, WDAs, KDCs and VDCs built to 3 3 3 3 3 develop, implement, monitor and evaluate a participatory planning system. 3 Output-3: Increase the livelihood base of the local community and most vulnerable households in 2 2 4 2 3 each of the five Woredas. 3 Output-4: Write up lessons learnt from the WCBP and disseminate to donors, government, NGOs 3 3 3 3 4 and regional networks. 3 OVERALL ASSESSMENT OF OUTPUTS ACHIEVED ARE REINFORCED 2 2 3 3 4 3

3. Quantity of Outputs/Results achieved so far

Output 1: Five strong, sustainable and locally accountable WDAs built that are able to implement a 3 3 3 3 3 process of locally initiated participatory development. 3 Output-2: The capacity of relevant Woreda government offices, WDAs, KDCs and VDCs built to 3 3 3 3 3 develop, implement, monitor and evaluate a participatory planning system. 3 Output-3: Increase the livelihood base of the local community and most vulnerable households in 2 2 3 2 2 each of the five Woredas. 2 Output-4: Write up lessons learnt from the WCBP and disseminate to donors, government, NGOs 3 3 3 3 3 and regional networks. 3 OVERALL ASSESSMENT OF OUTPUT/RESULT ACHIEVED 3 3 3 3 3 3

4. Quality of Outputs/Results achieved so far

Output 1: Five strong, sustainable and locally accountable WDAs built that are able to implement a 1 1 2 2 4 process of locally initiated participatory development. 2

______71 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR The assessment of the outputs produced should be made with respect to the planned targets. Assess the outputs of the project using the following (1-5) value scale where: 1= Very Poor, 2= Less than Satisfactory, 3= Average/Satisfactory, 4= Good, 5= Excellent Amaro Basketo Burji Derashe Konso Average Output-2: The capacity of relevant Woreda government offices, WDAs, KDCs and VDCs built to 3 3 3 3 3 develop, implement, monitor and evaluate a participatory planning system. 3 Output-3: Increase the livelihood base of the local community and most vulnerable households in 2 2 4 2 3 each of the five Woredas. 3 Output-4: Write up lessons learnt from the WCBP and disseminate to donors, government, NGOs 3 3 3 3 4 and regional networks. 3 OVERALL ASSESSMENT OF QUALITY OF OUTPUT/ RESULT ACHIEVED 2 2 3 3 4 3

5. Use of Outputs/Results by the Project Beneficiaries

Output 1: Five strong, sustainable and locally accountable WDAs built that are able to implement a 1 1 2 2 4 process of locally initiated participatory development. 2 Output-2: The capacity of relevant Woreda government offices, WDAs, KDCs and VDCs built to 3 3 3 3 3 develop, implement, monitor and evaluate a participatory planning system. 3 Output-3: Increase the livelihood base of the local community and most vulnerable households in 2 2 4 2 3 each of the five Woredas. 3 Output-4: Write up lessons learnt from the WCBP and disseminate to donors, government, NGOs 3 3 3 3 4 and regional networks. 3 OVERALL ASSESSMENT OF USE OF OUTPUT/ RESULTS BY BENEFICIARIES 2 2 3 3 4 3

OVERALL ASSESSMENT OF EFFECTIVENESS OF PROJECT OUTPUTS 2 2 3 3 3 3

5. Impact of the Project

Assess the IMPACT of the project using the following (1-5) value scale where: 1= Very Poor, Amaro Basketo Burji Derashe Konso Average 2= Less than Satisfactory, 3= Average/Satisfactory, 4= Good, 5= Excellent 1. The extent to which the overall objectives were achieved and the contribution of the project 3 2 3 2 3 to their achievement 3 2. The effects of the project on its direct beneficiaries in terms of technical and economic 3 2 3 2 3 factor i.e. project impact on beneficiaries 3

______72 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Assess the IMPACT of the project using the following (1-5) value scale where: 1= Very Poor, Amaro Basketo Burji Derashe Konso Average 2= Less than Satisfactory, 3= Average/Satisfactory, 4= Good, 5= Excellent 3. The effects of the project on larger numbers of people (communities) in terms of technical, 3 2 3 2 3 economic, social and institutional factor 3 4. The effects of the project at institutional level and/or within the different sectors in terms of institutional factor a. Accountable, financially independent WDAs that are responsive to locally identified needs 1 1 2 2 4 and able to develop, implement and monitor and evaluate micro projects effectively. 2 b. Village Development Committees (VDCs) and Kebele Development Committees (KDCs) 3 3 3 3 3 are able to develop community action plans and develop, implement and monitor and evaluate micro projects effectively. 3 c. Bottom-up planning institutionalized in Woreda level planning processes 1 1 1 1 1 1 Average of 4 (4a-4c) 2 2 2 2 3 2 5. Extent of sustainability of impact of the project after the withdrawal FARM Africa and 3 3 3 2 3 capacity to countermeasure external negative effects on project impact 3 6. Impacts of the project on gender-related issues 3 3 3 3 3 3 7. Impacts of the project on environmental-related issues 4 3 3 3 3 3 8. Impacts of the project on poverty issues 3 3 3 3 3 3 Overall Assessment of Project Impact (1-8) 3 2 3 2 3 3

6. Sustainability of the Project Assess the potential SUSTAINABILITY of the project using the following (1-5) value scale: 1= Amaro Basketo Burji Derashe Konso Average Very Poor, 2= Less than Satisfactory, 3= Average/Satisfactory, 4= Good, 5= Excellent 1. Financial/Economic Viability

3 2 3 2 3 a) If the services (results) are to be supported institutionally, are funds likely to be made available? 3 4 4 4 4 4 b) Are the services affordable for the final beneficiaries at the completion of the project? 4 3 3 3 3 3 c) Are the responsible persons/institutions assuming their financial/economic responsibilities including maintenance and replacement of the technologies introduced and/or used by the project? 3 Overall Assessment of Financial/Economic Viability 3 3 3 3 3 3

______73 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Assess the potential SUSTAINABILITY of the project using the following (1-5) value scale: 1= Amaro Basketo Burji Derashe Konso Average Very Poor, 2= Less than Satisfactory, 3= Average/Satisfactory, 4= Good, 5= Excellent

2. Technical and Social Viability i.e. the level of ownership of the project by beneficiaries and how it will likely be after the end of external support? 4 3 3 3 3 a) How is the sense of ownership of communities for the project outputs including equipment, infrastructure, etc? 3 3 3 3 3 3 b) The extent to which the project embedded in the local (community) structures 3 4 3 3 2 4 c) The extent to which beneficiaries (and other interest groups/stakeholders) involved in planning, design processes, decision making, project implementation 3 4 3 3 3 3 d) What is the likelihood that target groups/beneficiaries will continue to use relevant services after external support has ended? i.e. the capacity/self-reliance of the beneficiaries/communities and their determination to maintain and continue the activities and/or outputs of the project. 3 2 2 2 2 2 e) Are project partners being properly trained to accept handover of the project - technically? 2 Overall Assessment of Technical and Social Viability 3 3 3 3 3 3

3. Institutional and Managerial Viability i.e., how well is the project contributing to institutional and management capacity? 3 3 3 2 3 a) How far is the project embedded in the institutional structures that are likely to survive beyond the end of the project? 3 2 2 2 2 2 b) Are project partners being properly trained to accept handover of the project - managerially? 2 3 2 3 2 3 c) Are there good relations with new or existing institutions and are they capable of continuing the flow of project benefits? 3 3 2 3 2 3 d) Was there any capacity building effort to support project sustainability? 3 1 1 1 1 1 e) Is there a phase-out strategy defined and are there plans for its implementation? 1 Overall Assessment of Institutional and Managerial Viability 2 2 2 2 2 2 Overall Assessment of Sustainability of the Project 3 3 3 2 3 3

______74 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Overall Summary of Ratings of Evaluation Indicators by the Evaluation Team

Value Scale of 1 to- 5 where: 1 = Poor, 2 = Weak/Less than Satisfactory, 3 = Average/Satisfactory/Adequate, 4 = Good/More than Satisfactory, 5 = Excellent Key Evaluation Indicators of the Project Amaro Basketo Burji Derashe Konso Average

i. Project Design 2 2 2 2 2 2 ii. Project Relevance 3 3 3 2 3 3 iii. Project Efficiency 3 2 3 2 3 2 iv. Effectiveness Project Outputs 2 2 3 3 3 3 v. Impact of the Project 3 2 3 2 3 3

vi. Sustainability of the Project 3 3 3 2 3 3 Overall Assessment of the Project Design 3 2 3 2 3 3

______75 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Summary of Project Budget and Utilization Woreda Capacity Building Project TOTAL YEAR Expenditure Budget Vs Expenditure Follow Up Currency: Euro Total A/C Project Comp. Code Code Code Description Budget Expenditure Expenditure %age To July '09 To July '09 Utilization EURO Birr EURO 1. Human resources 1.1 Salaries (gross amounts, local staff) 1.1.1 Technical 1.1.1.1 FARM Project coordinator/Organisation Capacity Building Officer 5005 ET24 PM0 (Arba Minch) 47,949.50 770,400 61,643 129% 1.1.1.2 FARM Natural Resource Management Training Officer X 2 (Arba 5015 ET24 PM0 Minch) 55,302.00 402,514 32,915 60% 5015 ET24 PM1 1.1.1.3 FARM Enterprise & Finance Training Officer X 2 (Arba Minch) 36,868.00 655,067 49,331 134% 5015 ET24 PM2 1.1.1.4 FARM Regional Government Liaison Officer (Awassa) 21,840.00 231,503 17,266 79% 5045 ET24 PM1 1.1.1.5 FARM Admin/Finance officer (Arba Minch) 26,000.00 287,415 24,011 92% 5010 ET24 PM0 1.1.1.6 WDA Field Coordinator X 5 35,200.00 419,644 33,467 95% 5025 ET24 PM0 1.1.1.7 WDA Field Officer x 5 30,380.00 570,761 43,793 144% 1.1.2 Administrative/ support staff 5040 ET24 PM0 1.1.2.1 WDA Typist/casher X 5 29,680.00 348,576 27,783 94% 5041 ET24 PM0 1.1.2.2 FARM Secretary /Cashier (Arba Minch) 14,398.00 190,079 15,066 105% 5042 ET24 PM0 1.1.2.3 FARM Driver X 2 (Arba Minch) 23,800.00 311,207 23,390 98% 5043 ET24 PM0 1.1.2.4 FARM Guards X 3 (Arba Minch) 20,460.00 237,539 18,135 89% 5050 ET24 PM0 1.1.2.5 FARM Office Attendant (Awassa) 7,320.00 71,907 5,795 79% Casual Labour 1.2 Salaries (gross amounts, expatriate/international staff) 1.3 Per diems for missions/travel 1.3.1 Abroad (project staff) 5560 ET24 PM0 1.3.1.1 Two visits to Ethiopia per year by Capacity Building Adviser 3,750.00 31,276 2,523 67% 1.3.1.2 Three annual reviews/final evaluation by international and regional 7010 ET24 PM2 consultants 1,875.00 2,774 165 9% 5550 ET24 PM0 1.3.2 On the spot (project staff) 30,744.00 439,953 34,956 114% 6680 ET24 PM0 1.3.3 Seminar/conference participants - 462 6692 ET24 PM0 1.3.3.1 Two project staff to Capacity Building conference outside Ethiopia 236.84 5,459 459 194%

______76 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Total A/C Project Comp. Code Code Code Description Budget Expenditure Expenditure 6700 ET24 PM7 1.3.4 Technical support from the CO PM (25%) 5,530.00 - - 0%

Total Subtotal Human Resources 391,333.34 4,976,075.81 391,159.97 100% 2.Travel 2.1 International travel 5520 ET24 PM0 2.1.1 Two flights to Ethiopia per year by Capacity Building Adviser 2,000.00 6,813 507 25% 2.1.2 Two flights for project staff to Capacity Building conference outside 5520 ET24 PM1 Ethiopia - - 2.1.3 Four flights for two consultants (international & regional) for annual 7010 ET24 PM1 reviews and final evaluation 3,352.00 43,063 3,557 106% 7010 ET24 PM0 2.2. Local transportation 1,980.45 2,146 3,227 163%

Subtotal Travel 7,332.45 52,021.84 7,290.59 99% 3. Equipment, materials and supplies*** 3.1 Purchase of machinery and tools 4570 ET24 PM0 3.1.1 Tool kits for soil & water conservation (boards, rope, shovels) 1,709.00 19,490 1,709 100% 3.1.2 Planting material for soil & water conservation (grasses, fruit seeds, 6260 ET24 PM0 cuttings) 720.00 8,472 720 100% 3.2 Purchase or rent of vehicles 4010 ET24 PM0 3.2.1 Purchase of Four Wheel Drive Vehicle X 2 65,491.00 748,261 65,491 100% 4020 ET24 PM0 3.2.2 Purchase of Motorbikes X 10 18,705.00 226,830 17,992 96% 4030 ET24 PM0 3.2.3 Vehicle rent 6,000.00 51,405 3,622 60% 3.3 Purchase of Furniture, computer equipment 4130 ET24 PM0 3.3.1 FARM Office furniture 1,341.00 14,467 1,341 100% 4100 ET24 PM0 3.3.2 FARM One Desktop computer/printer and three laptop computers 11,250.00 228,919 18,307 163% 4110 ET24 PM0 3.3.3 FARM Photocopier 1,394.00 15,916 1,394 100% 4500 ET24 PM0 3.3.4 FARM Telephone/fax 658.00 7,073 658 100% 4140 ET24 PM0 3.3.5 FARM Generator - 32 4130 ET24 PM1 3.3.6 WDA Office furniture 4,760.00 58,953 5,056 106% 4100 ET24 PM1 3.3.7 WDA Desktop computer/printer 9,500.00 139,998 10,850 114% 4110 ET24 PM1 3.3.8 WDA Photocopier 3,618.00 41,334 3,618 100% 3.3.9 WDA Generator 4140 ET24 PM1 8,616.00 96,322 8,707 101% 3.4 Spare parts/supplies for machinery and tools 4560 ET24 PM1 3.5 Other (e.g. medicines) - 3,020 4100 ET24 PM2 3.6 Purchase of set of furniture/ materials and 2x5 computers with 10,000.00 103,036 7,152 72%

______77 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Total A/C Project Comp. Code Code Code Description Budget Expenditure Expenditure accessories for establishment of mini documentation center and with a mini- database at each of the five Woredas in the WoFED office 3. Subtotal Equipment, Materials and Supplies 143,762.00 1,760,475.38 149,668.85 104% 4. Local office/project costs 4.1 Cost of vehicle(s) 6010 ET24 RE0 4.1.1 Fuel and maintenance of FARM vehicles 65,250.00 931,890 74,248 114% 6020 ET24 RE0 4.1.2 Fuel & maintenance of WDA motorbikes X 5 X 2 16,077.60 220,571 16,983 106% 4.2 Office rent 6120 ET24 RE0 4.2.1 FARM Office rent 23,983.00 286,594 23,882 100% 6120 ET24 RE1 4.2.2 WDA Office rent X 5 - - 4.3 Consumables - Office supplies 6150 ET24 RE0 4.3.1 FARM Stationary and office consumables 14,163.60 194,732 16,313 115% 6150 ET24 RE1 4.3.2 WDA Stationary and office consumables X 5 26,100.00 318,561 25,707 98% 4.4 Other services (tel/fax, electricity/heating, maintenance) 6130 ET24 RE0 4.4.1 FARM Other services (tel/fax, electricity/heating, maintenance) 15,667.54 212,467 17,199 110% 6130 ET24 RE1 4.4.2 WDA Other services (tel/fax, electricity/heating, maintenance) X 5 16,200.00 197,850 15,127 93% 4. Subtotal local office/project costs 177,441.74 2,362,665.85 189,458.79 107% 5. Investments 5.1 Purchase of land 5.2 Building construction 4430 ET24 LS1 5.2.1 Design and rehabilitation of 4 WDA offices 20,571.00 236,375 21,129 103% 6900 ET24 LS0 5.3 Credit funds 4,025 254 5.4 Other 6820 ET24 LS0 5.4.1 Women’s' Goat Credit scheme in each VDC 14,260.00 151,755 14,398 101% 6810 ET24 LS0 5.4.2 Community Development Fund for NRM micro projects 121,800.00 1,811,811 130,197 107% 6900 ET24 LS1 5.4.3 Community Development Fund for Micro Enterprise projects 72,000.00 580,422 42,953 60% 6310 ET24 LS0 5.4.4 Community Development Fund for Community Health projects 76,500.00 1,265,009 91,617 120% 5. Subtotal investments 305,131.00 4,049,397.61 300,547.81 98% 6. Other costs, services 6.1 Evaluation costs ** 7010 ET24 PM3 6.1.1 External consultants (two) for annual reviews and final evaluation 36,850.00 377,706 26,520 72% 7050 ET24 PM0 6.2 Audit costs ***** 9,000.00 245,403 17,228 191% 5230 ET24 PM0 6.3 Financial services (bank guarantee costs, etc) - -

______78 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Total A/C Project Comp. Code Code Code Description Budget Expenditure Expenditure 8010 ET24 PM0 6.4 Publications** - - 5055 ET24 PM1 6.4.1 External consultant produced CB guidelines - - 18 8002 ET24 CB2 6.4.2 Profile raising material for WDAs (leaflet and business card) 1,250.00 6,414 475 38% 8002 ET24 CB0 6.4.3 Produce procedures and guidelines/manual of M&E 2,759.00 17,153 1,094 40% 6.5 Studies, research** 6.5.1 Conduct community lead participatory baseline survey with WDA and relevant GO staff using particicipatory survey approaches and techniques (56 6710 ET24 PM0 persons x 3 days for each survey) 3,635.00 38,737 3,578 98% 6.5.2 Conduct diagnostic survey in each new Woreda (35 persons x 3 days 6710 ET24 PM1 per survey) 2,549.00 28,315 2,549 100% 6.5.3 Undertake study on local information system and traditional 6700 ET24 PM0 communication methods in all Woredas (35 persons x 3 day study) 1,988 417 6.5.4 Conduct assessment to identify feasible local community institutions for 6720 ET24 PM0 micro projects (12 persons x 2 days per Woreda) 1,546.00 18,079 1,564 101% 6750 ET24 PM0 6.5.5 Develop photo-mosaics maps for each PA 484.00 5,123 484 100% 6.5.6 Conduct need assessment of poorest households in each VDC with 6701 ET24 PM0 WDA and GO (15 GO persons x 5 days & 2 WDA persons x 20 days) 821.70 9,253 822 100% 6.5.7 Develop and appraise all VDC proposals (7 GO persons x 8 days & 1 6720 ET24 PM1 WDA person x 20 days) 1,634.80 12,248 927 57% 6.5.8 VDCs/WDAs identified donor, secure funding and micro project 6900 ET24 CB2 reported upon (276 persons x 3 days) 63.00 710 63 100% 6.5.9 Produce procedures and guidelines/manual of M&E (12 person x 2 days 8002 ET24 CB4 per Woreda) - - 6.5.10 Develop action plans for M&E in each PA (46 persons x 5 days per 7020 ET24 CB0 Woreda) 4,500.00 45,572 3,111 69% 6.5.11 Conduct assessment to identify gaps in technical skills for micro 6740 ET24 CB0 projects (18 persons x 3 days per Woreda) - - 6.5.12 Study to develop a Community Based EWS in each of the new 6700 ET24 PM1 Woredas (37 persons x 4 day per Woreda) - - 6.5.13 Identify possible membership system for WDAs (support from 6700 ET24 PM2 external specialists) - - 6700 ET24 PM3 6.5.14 Map non-locally resident WDA supporters 243.00 4,043 296 122% 6700 ET24 PM4 6.5.15 Commission a feasibility study on WDA supporters 893.00 1,083 92 10% 6.5.16 organise study tours to other NDAs (both within the project and 6700 ET24 PM5 outside) 2,529.00 29,988 2,104 83% 6700 ET24 PM6 6.5.17 Activities support and monitoring 5,250.00 116,743 9,051 172% 6.5.18 Establish MoUs with SEPD or official partnership arrangements with the five WDAs and the project and also for supporting establishment of platform (two forums) for regional level WDAs net working and 6700 ET24 PM8 coordination. 5,000.00 37,835 2,753 55% 6.5.19 Collection and compiling of all relevant baseline information related to the Woreda profile including assessment of institutional capacities of line 6700 ET24 PM9 departments/Woreda administration sector offices, available potential 5,000.00 126,588 8,143 163%

______79 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Total A/C Project Comp. Code Code Code Description Budget Expenditure Expenditure resources, data related to on 6.6 Translations, interpreters ** 8001 ET24 CB1 6.6.1 Translation costs of manuals 800.00 - - 0% 6.7 Costs of conferences/seminars** 6.7.1 Community forum debates held in each Kebele to establish role of 8003 ET24 CB0 WDA (250 persons x 1 day per Woreda) 954.00 11,128 943 99% 6.7.2 Conduct workshop to revise bylaws in each Woreda (23 persons x 6 8004 ET24 CB0 days per Woreda) 979.00 11,751 923 94% 5210 ET24 CB0 6.7.3 Register WDAs with government and SEPDA 488.00 6,731 556 114% 6.7.4 Conduct training on concepts, tools and procedures of strategic 6600 ET24 CB0 planning workshops (100 persons x 3 days) 4,250.00 36,352 2,609 61% 6.7.5 Produce and disseminate strategy document (7 persons x 3 days per 8001 ET24 CB0 WDA) 6,000.00 96,774 6,581 110% 6.7.6 Conduct workshop to approve strategic document (35 persons x 2 6675 ET24 CB0 days per Woreda) 1,272.96 17,964 1,247 98% 6.7.7 External course in financial management to finance and non-finance 6625 ET24 CB0 managers 3,140.00 42,920 3,470 110% 6.7.8 Conduct workshop to develop financial policies and procedures with 6676 ET24 CB2 WDA and government staff in each Woreda (25 persons x 6 days) 3,784.00 49,644 4,102 108% 6.7.9 Conduct training in community based participatory Monitoring & 6630 ET24 CB0 Evaluation (16 persons x 10 days) 3,763.88 39,510 3,688 98% 6.7.10 Conduct workshop to develop project Monitoring & Evaluation procedures/system with WDAs and government staff in each Woreda (31 6676 ET24 CB1 persons x 8 days) 1,444.00 20,202 1,622 112% 6.7.11 Conduct training in personnel and leadership management for WDAs 6615 ET24 CB1 (37 persons x 4 days) 5,850.90 90,275 7,113 122% 6.7.12 Conduct workshop to develop personnel procedures for all 6675 ET24 CB1 WDAs/government (6 persons x 10 days) 5,594.00 63,308 5,586 100% 6.7.13 Conduct training in proposal writing, fundraising, report writing and mobilising resources from within the community, and in networking for 6665 ET24 CB0 WDA/VDAs (30 persons x 10 days) 2,600.00 45,557 3,168 122% 6.7.14 Produce information & dissemination guidelines on WDA communication strategy with VDCs/government in each Woreda ( 46 8031 ET24 CB0 person x 3 days per Woreda) - 1,067 6.7.15 Pilot and adapt information system/guidelines with VDCs (14 persons 8031 ET24 CB1 x 5 days per Woreda) - 6 6.7.16 Workshop to plan and develop ways of working with local institutions 8030 ET24 CB0 for all stakeholders ( 1 WDA person x 2 days x 5 workshops per Woreda 651.00 7,575 651 100% 6.7.17 Conduct action oriented training to village communities on 6620 ET24 CB0 community based planning tools with each VDC (224 persons x 3 days) 5,953.00 64,233 5,953 100% 6.7.18 Conduct action planning workshop at local level with VDC (224 6620 ET24 CB1 persons x 3 days) 4,616.64 50,530 4,714 102% 6.7.19 Conduct workshop to produce manuals, guidelines and procedures for project formulation and appraisals in each Woreda (23 persons x 3 days 6675 ET24 CB2 per Woreda)-TA 3,000.00 63,052 3,991 133%

______80 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Total A/C Project Comp. Code Code Code Description Budget Expenditure Expenditure 6.7.20 Train VDC in project formulation and management (224 persons x 3 6615 ET24 CB2 days) 2,688.00 20,996 1,548 58% 6615 ET24 CB3 6.7.21 Train VDCs in participatory M&E technique (224 persons x 3 days) 2,688.00 10,998 756 28% 6.7.22 Provision of technical assistance by Woreda experts to CDF project identification, formulation and Implementation process E.g. 6650 ET24 CB0 Watsan/Construction,etc-TA 3,840.00 48,916 3,663 95% 6.7.23 Review of loan activities and support to existing SACCOs schemes to review/develop by lows and implement loan repayment procedures in Burji- 6650 ET24 CB1 TA 3,000.00 56,732 3,927 131% 6.7.24 Run Woreda level planning forum and share KDCs & VDCs planning 6645 ET24 CB0 experience between Woredas (30 person days/Woreda/year 844.00 4,691 406 48% 6645 ET24 CB1 6.7.25 Integrate participatory plans into Woreda plans 172.00 - - 0% 6.7.26 Run Woreda level bottom up planning traing for GO staff (113 6645 ET24 CB2 persons x 3 days) 376.95 4,898 343 91% 6.7.27. Support the understanding and establishment of CBPM&E methods 6675 ET24 CB3 for GO staff ( 135 persons x 4 days) 417.00 5,105 404 97% 6.7.28 Prepare action plan for CBPM&E 6950 ET24 CB0 - - - 6.7.29 WDA visits to all VDCs to implement action plans (2 persons x 5 days per Woreda x 9 visits) 6690 ET24 CB0 8,541.98 193,289 14,399 169% 6.7.30 Improve Soil & Water conservation skills through training and mentoring ( 46 persons x 3 days per Woreda) 6630 ET24 CB1 1,441.00 16,385 1,425 99% 6.7.31 Training communities and WDA in management of EWS in the 6620 ET24 CB3 community ( 40 persons x 1 day per Woreda) - - 6.7.32 Conduct mid term and final experience sharing workshop based on 6675 ET24 CB4 lesson learnt (50 persons x 2 days per Woreda) 12,000.00 32,249 2,759 23% 6.7 33. Produce material for manual /guideline of Woreda level capacity 6676 ET24 CB0 building-TA 5,000.00 10,618 809 16% 6.7.34 Organise basic computer, data management and internet training for WDAs and selected staff of the five Woredas sector offices and 6676 ET24 CB11 establishment of mini database by hiring IT specialist 7,000.00 68,726 4,517 65% 6.7.35 organise awareness raising (and possibly fundraising) event(s) for the WDA in the Woredas and sponsoring regional radio air time about the 6676 ET24 CB12 activities of the 5 Woredas and the project activities 8,000.00 93,480 6,133 77% 6676 ET24 CB3 6.7.36 Set up notice boards outside each WDA for relevant local information 250.00 4,618 381 152% 6.7.37 Raise awareness of community about WDAs, the planned process of revising byelaws, and the importance of community representation in 6676 ET24 CB10 General Assembly, in all Kebeles of the Woreda (not just the five Kebeles) 431.00 79,271 5,546 1287% 6.7.38 Facilitate relevant experts from the government offices to carry out 6676 ET24 CB5 refresher PRUP training for KDCs and VDCs 495.00 9,980 743 150% 6.7.39 Government experts to assist individuals, group and/or the 6676 ET24 CB6 community to develop project proposals 1,531.00 24,317 1,817 119% 6.7.40 Run Woreda level forum and share experience between Woredas (64 6676 ET24 CB7 persons per Woreda forum) - -

______81 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Total A/C Project Comp. Code Code Code Description Budget Expenditure Expenditure 6676 ET24 CB8 6.7.41 Conduct training for WDAs in fund raising (16 persons @ 4 days) - - 6.7.42 Train relevant Woreda level GO staff in the project formulation and 6676 ET24 CB9 management (PRUP Phase III (33 @3 days)) 792.00 10,059 723 91% 6.7.43 Carry out quarterly review workshops with WDAs 6675 ET24 CB5 (11events*4days*30 persons) 7,080.00 140,255 9,850 139% 6.7.44 Documentation of all activities and methodologies, best practices, challenges and lessons learned, pictures, etc-TA A. Micro projects (CDF grants, Grain Bank, SACCO, Watershed, Women's goat scheme / CAHWs) 6675 ET24 CB6 B. Organisational capacity building 3,000.00 53,871 3,476 116% 6.7.45 Organise one dialogues and advisory forum at regional level jointly 6675 ET24 CB7 with other CSOs/NGOs -TA 4,000.00 26,108 1,667 42% 6.7.46 Organising monthly review and planning forums at the Woreda level with all stakeholders including representatives from VDC/KDC 6675 ET24 CB8 4,320.00 32,334 2,418 56% 6.7.47 Organise advocacy and dialogue forum one at each of the Woredas level (five forums in the five Woredas) for all stakeholders including 6675 ET24 CB9 representatives from VDC/KDC and other actors. 2,500.00 13,441 897 36% 6. Subtotal Other Costs, Services 219,556.81 2,801,820.24 207,850.93 95% 7. Other Sub total Other 8. Subtotal Direct Eligible Project Costs (1.-7.) 1,244,557.34 16,002,456.73 1,245,976.93 100% 7210 ET24 LS0 Admin Cost 7% 87,119.01 1,001,455 79,477 91% Admin to be charged GRAND TOTAL 1,331,676.35 17,003,912.15 1,325,454.11 100%

______82 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Updated Local Staff of WCBP

FARM AFRICA - WCBP - Staff List ( 2004 - 2009 ) Date of Expiry Total employment Name Position Employment Date duration Remark 1. Aman Enyew CBO 1-Jul-08 21-Jul-09 1 years End of Project 2. Ashenafi Bogale Guard & 17-Aug-04 14-Jul-08 3 years& 11 months Resignation Gardner 3. Balcha Bekele Driver 6-Apr-05 21-Jul-09 3 years& 3 months End of Project 4. Dereje Asamenwe NAM 7-May-07 29-Feb-08 10 Months Termination 5. Ejigu Jonfa Project 7-May-07 29-Feb-08 10 Months Termination Coordinator 6. Gudisso Ejigu CBO 3-Sep-07 8-Feb-08 6 months Resignation 7. Haile Abdisa Guard 1-Jan-09 21-Jul-09 7 Months End of Project 8. Jemal Abdulahi Guard & 22-Dec-04 7-Nov-07 2 years & 11 months Resignation Gardner 9. Jorge Jole Guard 1-Jan-09 21-Jul-09 7 Months End of Project 10. Kassahun Bishaw Team Leader 31-Jul-07 31-Jul-08 1 years Resignation 11. Kussia Bare CBO 1-Jul-08 21-Jul-09 1 years End of Project 12. Mekonnen Mamo Guard & 17-Aug-04 21-Jul-09 4 years & 11 Months End of Project Gardner 13. Mesefin Mersha CBO 1-Jul-08 21-Jul-09 1 years End of Project 14. Michaeal Assefa Project 1-Oct-03 18-Jun-06 2 years & 8 Months Resignation Coordinator 15. Midekesa Bogale CBO 1-Jul-08 30-Apr-09 10 Months Transfer to Other Project 16. Muleye Aschisew Secretary 1-Sep-04 1-Oct-04 1 Months Resignation Cashier 17. Muluken Tefera Guard 1-Jan-09 21-Jul-09 7 Months End of Project 18. Mulusew Demu Driver 17-Aug-04 21-Jul-09 4 years & 11 Months End of Project 19. Seble Worku Filed Officer 24-Nov-04 31-Dec-05 1 years & 1 Months Resignation 20. Selamawit Belachew Office attendant 26-Dec-04 21-Jul-09 4 years & 7 Months End of Project 21. Sintayehu Tsegaye EFTO 24-Nov-04 21-Jul-09 4 years & 7 Months End of Project 22. Tariku Wodago Accountant 31-Jan-07 31-Mar-08 1 years & 2 Months Resignation /Admin.Offier 23. Tekelaregay Jirane Program 1-Sep-08 21-Jul-09 10 Months End of Project Manager 24. Tesfayesus Yimenu NRM TO 28-Dec-04 25-Feb-07 2 years & 2 Months Resignation 25. Theodros Mikre NRM TO 29-Dec-04 8-Jan-07 2 years & 1 Months Resignation 26. Tilahun Demissie Fin. & Adm. 30-Dec-04 9-Feb-07 2 years & 2 Months Termination 27. Tsigereda Mekonnen Secretary 7-Nov-04 21-Jul-09 4 years & 7 Months End of Project Cashier 28. Wondimu Ababu Finance Officer 1-Jul-08 21-Jul-09 1 Years End of Project 29. Yeruksew Belay Office 16-Aug-04 1-Oct-05 1 Years & 2 Months Resignation Attendant 30. Yiregalem Abaynehe Driver 1-Jun-07 21-Jul-09 2 years & 1 months End of Project

______83 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Minutes of WCBP Final Evaluation Workshop

Place: Hawassa (G/kirstos Hotel) Date: 30/07/2009 Time: 09:30am Participants’:-

1. Berhanu Taye Task Manager -RDFS-NGO Program Delegation of the European Commission to Ethiopia 2. Aregawi Tesfaye Team leader, SNNPR BOFED representative 3. Wondu G/Mariam member SNNPR BOFED representative 4. Alemayehu Koree Development Association project coordinator 5. Atnafu Wondimu Basket Development Association ex- Manager 6. Berhanu Chulido Development Association Manager 7. Samuel Karafo Kono Development Association Manager 8. Tamiru Manaluh Derashe Development Association Manager 9. Adisu Tuke Buji Development Association Manager 10. SNNPR BOARD,BOWRD,BOH,BOE, representatives /not present / 11. SEPDA representative /not present / 12. board chair persons of 5woreda 13. 5 woreda administration and finance and economic development offices’ representatives 14. Temesgen Kassa (Programme Director) Agri-service -Amaro project representative 15. Zenebu Huluf, FARM Africa- country office representatives 16. Wondosen Balcha FARM Africa- country office 17. Teklearegay Jirane FARM Africa- WCBP Coordinator 18. Wondimu Ababu FARM Africa- WCBP Adimin/finance Officer 19. Aman Enyew FARM Africa- WCBP Capacity Building Officer 20. Kusia Bare FARM Africa- WCBP Capacity Building Officer 21. Mesfin Mersha FARM Africa- WCBP Capacity Building Officer 22. Solonon G/Sadik consultants RUPA 23. Berhanu Worku consultants RUPA

Agendas of the day 1. WCBP overall implementation report 2. Documentation centre (mini data base) establishment report 3. Findings and recommendation of final evaluation report 4. Discussions on reports, conclusions and ways forward

The day’s program was started by Ato T/aregay Jirane, WCBP project manager, well come speech and his explanation of the day’s agenda and the meeting program followed by the participants’ introduction to each other. Ato T/aregay presented the 5 years overall woreda capacity building project implementation strategies achievements, challenges, and lessons. WCBP is one of the FA’s largest project in SNNPR which evolved from experiences of the former Konso capacity building project .It was started on July 22, 2004 with the objective of poverty reduction and building the capacity of government and civil society organization to better assess needs, plan, implement, Mand E their own development in 5 special Woredas (Amaro, Burji, Konso, Derashe, and

______84 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Basketo). The project was funded by EU, FA, and Coradid, and has total beneficiaries of 158, 627 people. The project implementation approaches were to demonstrate the project that can reduce poverty, build human and institutional capacity of local partners, establishing key grass root level structures and exercising the bottom up planning process.

Ato T/aregay explained the physical achievements and also the financial utilization report of the project during the last 60 months of project operation by categorizing under four out puts; The internal and external challenges and problems that the project faced over the last 5 years were also presented as follows; 1. project design 2. delay in project implementation 3. high staff turn over both in WCBP and WDA 4. inaccessible/overstretched area 5. ethnic conflict 6. logistic problems(vehicle) 7. WDA financial sustainability 8. reluctance and low commitment from WDA staff were of the main

Ato T/aregay finally pointed out the following as lessons learnt during the course of the project implementation processes:- 1. The project boasts the moral of the community 2. Participatory planning implementation exercised and community negotiation power enhanced 3. Working closely with GOs should be given priority 4. Community learn cost effectiveness, participatory action planning, participation 5. Realization of such scale and type of project requires more than 5 years (at different phases) of work 6. Some capacity building trainings should be under taken early 7. WADs need government close support and assistances.

The next report was about establishment of documentation centres in Woredas, and its data base development processes which was presented by the private consultant who has been participated on this process. The consultant started his briefings with the objectives of the establishment of the documentation centre as to how one has to develop and use real time and structured data to support sustainable development and in order to establish systems which can provide woreda information for various development projects. He also explained different socio-economic information’s that should be included in the documentation centre. Further more he has explained some of the efforts made to establish the centre in training people on data base management from the 5 Woredas, data base software development and loading program on computers and other technical supports. Finally the consultant winded up its report by explaining problems faced while touring in different project Woredas as 1. lack of awareness on the benefit of the resource centre and its implementation

The last report was presented by Ato Solomon G/tsadiq, the private consultant; Ato Solomon started his report by appreciating the cooperation and willingness of BOFED evaluation team in the FE process. Further more he has elaborated the on the final evaluation processes, methodology and tools used to evaluate the project. He said that the evaluation was conducted taking the 4 out puts planned to be achieved in the project relating to the following parameters.

1. project design 2. relevance

______85 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR 3. efficiency 4. effectiveness 5. out come and or impact trends of the project 6. sustainability

Accordingly he briefed each parameter with its respective rate and presented also the SWOT analysis; lessons drown, major challenges, conclusions and recommendations as follows:

STRENGTH  Willingness to work in inaccessible /remote area

 Its approach/strategy to use WADs as entry point (phase in)

 Organization of capacity building training programs in need identification and project design for WADs staff

 Organization of experience sharing visits for WADs to development associations with better track records

 Woreda resource centres establishment

 Efforts made to enhance the institutional capacity of WDA

 Enhancing community participation for CDF project implementation

WEAKNESS

1. Over ambition nature of the initial project design (area coverage and volume of activities)

2. Poor internal management systems

3. Delay in timely initiation and implementation of most CDF project and capacity building training components

4. Delay in transfer and settlement of CDF project budget

5. Absence of permanent WDA manager working on full time basis

6. Weak capacity, poor commitment and depiction of WDAs board ,managers and staff

7. Failure to properly considers some of the recommendations forwarded during the mid term review

8. Failure of project signatories to properly appraise the project at its initial design and amendment stages

9. Unable to sustain the proper functioning of WDAs beyond the life of the project

10. Poor performance in terms of raising and utilization of the potential local resources

OPPORTUNITIES

______86 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR  positive attitude(willingness) of local administrations to wards WDAs and their readiness to support them

 The presence of SEPDA its potential and willingness to provide capacity building supports to WDAs

 Availability of other projects which are run by some of the WDAs and opportunity to sustain the operation of WDAs in near future

 Existence of promising community commitment and willingness to support WDAs through contribution and membership fee

 The initiative taken by the regional government to replicate and scale up of bottom up planning approach in all Woredas

 Availability of potential donors to support community development projects.

THREATS

 lack of financial resource by WDAs for completion of on going activities

 shortage of financial resources by most of the WDAs to finance their operation in the future

 failure of the WDAs to assign permanent management by their own

 lack of capacity of the WDAs boards to assure their leadership role as required

 conflict and insecurity situation

LESSONS DRAWN

 Bottom up planning approach was founded to be effective in empowering community groups and enhance their participation PCM of development projects and should be scaled- up and replicated in other areas in the region

Major challenges/problems

 conflict and/ or insecurity

 High inflation in price of construction materials and labour cost

 logistic constraints

Recommendations

1. The project holder (FARM Africa) should assist its partners (especially WDAs) through availing the required resources to finalize uncompleted on-going project intervention Woredas.

______87 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR 2. FARM Africa should assign a focal person at its country office in order to provide the required backstopping to pertinent stakeholders and phase out;

3. it is commendable to further assist the WDAs to make them strong and locally accountable development actor through provision of technical assistance in areas of;

- Fund-raising.

- Human resource development.

- Preparation of on-shelf projects and project templates to be used in the future;

4. To plan fund raising activities; and FA to lead and negotiate with the donor to access funds for extension period.

5. FA to retain core field staff BoFED to allow project extension, technical support M+E and woreda government to support by community mobilization, fill gaps in required financial support, ensure regular follow up, updates and support, WDAs to retain current staff, plan for financial resources required collect members fee and call general assembly meeting.

6. To sit together with BoFED to agree and put together the consolidation phase activities and thereby draft MoU for signature by involved organizations.

7. FARM Africa should compile and disseminate the experience, challenge, and lessons learnt from the implementation of its woreda capacity

8. All concerned actors should assist and promote the scaling –up/replication of the Bottom-up Planning Approach.

9. Strategic leadership of the WDAs shall be strengthening through the assignment of competent and committed staff, mangers and board members.

10. WDA Board in collaboration with local governments should critically examine the internal and external challenges/constraints hampering efficiency and effectiveness of WDAs and seek effective remedies.

11. The need for the assignment of full-time managers for the WDAs on permanent basis from own sources.

12. Effort to update the woreda database system and proper use of information centres established shall be insured by all concerned.

13. Need for financial self-sufficiency of WDAs through mobilizing local resource to sustain their operation in short and long-run is identified (mobilization of members, collection of members fee and efficient management of resources.)

14. To call a general assembly meeting to discuss the current situation and jointly act for effective actions

15. SEPDA has to support WDAs as un umbrella regional organization

______88 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR 16. Gov’t to take leading role in maintaining peace and security in project Woredas and NGOs to work closely to complement such efforts

17. Government to play important role in allocating capital budget to projects that can be implemented by WDAs

18. All concerned government actors at different levels should strictly follow-up, monitor and provide the required support, ensure the service delivery, sustainability and proper handing over projects.

In the after noon session additional comments and suggestion were presented on the reports by the facilitation of Ato Berhanu Taye, EC delegation to Ethiopia representative as follows

Comments by BOFED representatives

 the processes of project implementation has been costly and expensive in relation to time and inputs used

 the project made WDAs dependent rather than building their capacity

 the project has been busy with internal problems which consumed much of the project time

 the bottom up planning approach practiced was good but the community has not given enough time to exercise it

 poor exit strategy

Next to the additional comments general discussions questions and comments were made on the reports by the participants as follows

Questions in relation to the Documentation centre/data base

Q1. Establishing the documentation centre is good however how could it be managed? Who manage it?

A1- All documentation centres are located in woreda centre at finance and economic development office and a staff is assigned to managing the doc centre and provides the required information to public members who are in need of any information that are available for development use. The woreda staffs are trained in data base management.

Q2. What are the source of information /data of the centre?

A2. The sources of the data are woreda sector offices and the central stat ices office.

Q3. Information/data used as an input for the centres has been collected in short time. Therefore how could it be possible to get reliable information?

All the information gathered are official figures that are known by either the regional or federal government

WCBP

______89 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR  Q1. WCBP report stated that the preparation of CDF guideline is on process where as the evaluation team report stated that there was no CDF guideline.

 A1. The assignment was given to WDAs (Participatory M+E guideline for CDF was complete and distributed for WDAs training was conducted on the same)

 Q2. WCBP has started its interventions on the already existed WADs. Therefore what would have been expected from the project rather than building on the former existed situation?

 A2. Even though WCBP has started its interventions on the already existed WDAs it has no good phase out strategy (page 8). Its strong side however is its relation with the woreda government body has resulted in poor performance in the woreda

- Comments: (Amaro woreda administrator). It is obvious that there were conflicts in some operational Woredas however; there has not been as such serious conflict in the target Kebeles which affected the interventions of the project in Amaro. Therefore since this problem is reported exaggerated it does not represent our situation (amaro representative)

- Comment Basketo DA manager FA is the first NGOs in our woreda and also has showed good things in the target 5 Kebeles. However there are some uncompleted activates due to different reasons. (Basketo)

- Q3. How will unsettled budget balances treated? (amaro, Burji)

Thanks to FA since it is the only NGO in our woreda. However the conflict situation in our woreda was not affected the intervention as stated on the report.

- Provision of goat to poor women is good however the number of goats provided was too small to bring impact on the life of the women.

Project final Evaluation

 The overall evaluation report is good. However; how you did related joint planning implementation and monitoring? (Agri service representative)

 How did you rate the evaluation? What does it mean when you say 1, 2, 3- ( Agri service)

 The evaluation report is detail and good. However it did not consider the contribution of the presence of poor and uncommitted WDA management on the performance of the intervention. (Amaro representatives)

 The rating system of the evaluation is not clear? The rating doesn’t mach with what has or reflects un been described in words what has been described as very good was given the rating of 3.8 (relevance of the project )this seems that the feedback lacks a balanced view of the situation at ground in terms of rating (WCBP staff)

Ato Birhanu Taye EC delegation to Ethiopia representative’s comment

This project has got fund from EU due to its good project idea but it has faced so many problems on it is implementation processes. The project’s first three years were not implemented effectively rather it has implemented with it is full capacity on the last year. In addition to the problems faced about 133,000

______90 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Euro to budget was deducted and returned to the donor. Generally the problems were not only with farm Africa but also lack of commitment and dedication from the WDAs side as well. We have been following up the project through regular monthly report and site visits.

Concerning the smallness of the number of goats provided to vulnerable women it is not a matter of the number of goats but it is a strategy by which the women are re exercising and developing the economic power processes also to create asset in HHS level.

The project evaluation team should already identify the role of SEPDA.

How did the evaluation team see the visibility of the project?

Comment by WCBP

Concerning the project final evaluation the evaluation methodology used by the team is correct and good. However; the rating for efficiency and effectiveness in relation to various parameters are not clear and fair to me. The bottom up planning processes has been appreciated by regional government bodies and also by this evaluation team but when we see the rating it is 1and 3.Similary the best practices (out put4) was described to be good but rated 3.2, was targeting of beneficiaries and area not correct? Generally I will not agree with the rating due to the fact that the actual fact of what is on the ground doesn’t mach with the scores offered as rating and its regrettable that the evaluation was mostly focused and try to emphasize on the problems by neglecting and undermining achievements that was obtained regardless of so many internal and external problems and challenges.

The final CDF implementation manual is on process, however this doesn’t mean that we don’t have procedures to follow rather there are strict procedures, guidelines and steps used to implement CDF project. The guidelines are well familiar to the VDC/KDC and the WDAs who are at frontline in implementing the CDF projects.

Regarding the CDF project implementation process cost effectiveness. Since these projects were community initiated and implemented by the active participation of the community so they were very cost effective.

For the comment side that the project made WDAs dependent I leave this case for the judgment of the audience but just give a brief overview of what has been done by the project was then to strengthening their human and institutional capacity of the WDAs, reconstruction of WDA’s office, provision of office materials, equipments and motorbikes, support WDAs in obtaining the legal entity, provide technical and financial support to the local communities to implement community based micro projects initiated by the community itself and contributed to improvement of livelihoods of poorest community members. Therefore it is not clear how that the project made the WDAs dependent.

Concerning the conflicts that have been happened in project areas. It was not a secret and were known by all who live in this part of the region it is obvious that there were conflicts in most parts of the project areas except Basketo and the sources of such reports was WDAs S them selves and there is no reason to provide wrong information on this. And most recently also from our experiences conflicts were taking place in Kebeles of Derashe, Konso and Burji indeed.

The information /data collected to be used for the documentation source was collected from woreda sector offices verified by seal of the respective woreda’s sector offices to ensure the reliability of the data.

______91 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Data base consultant comment

When we developed the data base for these established documentation centre all variables were tried to be considered. In fact I want to share the fear on the documentation centre management and that was why we have been tried to share some advices to woreda concerned bodies during our tour.

To ensure the centre’s sustainable services there must be responsible government body to manage the overall activities of the centre and stable trainees were tried to be included in the training of data base management to avoid the technical personnel turn over risk. The information/data that have been collected from Woredas was not entered to data base except using it as reference due to narrowness of the data. The developed data base for the centre has its own password to protect the data from being corrupted, deleted, or misused.

Final Evaluation Consultant (Ato Solomon G/tsadiq)

- As far as the rating of evaluation is concerned we rated separately each innervations in each woreda based on the selected parameters and then take the average to get the project’s overall result. The evaluation has been conducted not only for the last one implementation year of the project rather for all project life (from the beginning up to the end of project life cycle), therefore; although much efforts has been made and good performance has been seen in the last project implementation year, the poor implementation performance of the first three years has negatively affected the overall project performance (evaluation result).

- It is obvious that efforts have been made to strengthening WDAs institutional capacity. However the support given to them was not enough. In addition the efforts made to familiarize the bottom up planning approach were good on the target areas, however; it was not scaled up to other similar areas/Kebeles within the Woredas.

Ato Birhanu Taye- EC- Ethiopian delegate representative’s comment

 In our opinion CDF projects implementations main objective is beyond structure building which is rather it is a means by which the local community express its ideas and exercise bottom up planning process.

 The other issues raised in relation to the May 97 election and conflicts where not only an issue of this project’s operational area but it is also nation wide problems. In the mean time the comments and suggestion given on the project final evaluation are good. Every project has to be evaluated by the out puts planned to be achieved on its project document.

After answers and comments session, the participants divide in to three groups to discuses and present the relevance of the evaluators’ recommendation and action to be taken and role and responsibilities of the stake holders (FA, WDAs, Government, and SEPDA). Accordingly the groups discussed and presented there results and after presentation general comments and suggestion were made on it. And finally the following summarized points and actions have forwarded.

1. FARM Africa

- FA should have finished the ongoing and uncompleted project activities. However the issue requires through analysis of the required time and resources to handle the matter, and most importantly the ultimate decision will be made later after providing

______92 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR feedback information to the executive director. In fact it is difficult to set the exact time right away however, Farm Africa and BOFED will work together to sort out the key remaining activities with in the next one week and forward suggestion to decision makers of the both organizations.

- FA should retain field staffs needed for the completion of activities as much as possible.

- FA has to give clear back stopping support

- FA has to get the necessary operational permission from BOFED.

2. Regional and woreda Government bodies

-Has to give approval for completion of on going activities and sign Memorandum of understanding with FARM Africa

Has to give permission to use all necessary project logistics for the Completion of the activities.

- Has to give technical support.

-Has to fill gaps as much as possible.

-Should provide financial support or fill the gaps in implementing the remained activities that can be done by WDAs from its budget.

3.WDAs and SEPDA

- WDAs have to keep their staffs until these problems will be solved.

- Implement and follow up activities.

- Plan and implement fund raising events as soon as possible.

- Calling and notifying the general assembly about the situation as soon as possible (with in two weeks).

SEPDA

- SEPDA has signed partnership agreement with WDAs therefore it should help WDAs to get external support.

- Peace issue is not only one body’s concern; therefore all stakeholders should support the peace building processes.

- Finally the meeting was ended up by Amaro special woreda administrator’s closing remark and was adjourned @ 7:30 pm.

______93 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Terms of Reference for the Final Evaluation

Project Title: Woreda Capacity Building Programme, in 5 Special Woredas (Konso, Derashe, Basketo, Burji and Amaro) of the SNNPR.

Project Ref: ONG/PVD/2003/064-438 (Co-funded by the EC)

1. Introduction

FARM-Africa has a vision of a prosperous rural Africa which aims to make this a reality by being innovative and practical, being effective by working with local people and their institutions rather than building parallel structures, and bridging the gap between researchers and rural communities, ensuring that our research addresses practical problems facing communities

FARM-Africa is an international non- governmental organisation that aims to reduce poverty in eastern and South Africa. We work in partnership with marginal farmers and herders, helping them to manage their natural resources more effectively and build sustainable livelihoods on their land.

FARM-Africa’s Woreda Capacity Building Project (WCBP) is a European Union (EU) funded project aiming to reduce poverty in the most vulnerable households in five special Woredas of the Southern Nations, Nationalities and Peoples Region (SNNPR) of Ethiopia by strengthening key institutional structures and building the capacity of governmental and non-governmental organisations (particularly Woreda Development Associations – WDAs) that can promote sustainable development at Woreda level. The WCBP commenced its operation in 2004 after the signing of the project agreement with the European Union on 22nd of July 2004 and of the implementation agreement with the regional government on 9th of September 2004.

The project has so far made progress towards its intended outputs and achievements have been made in terms of enhancing the institutional capacity of WDAs through physical and programmatic capacity building activities such as trainings in organisational development and strengthening their technical skills in financial and organisational management. Progress has also been made towards instituting participatory planning processes through the establishment of structures such as Woreda planning committees and Village and Kebele Development Committees. In order that planning processes and project initiatives are effectively informed and a baseline established, diagnostic studies and identification of wealth groups in pilot Kebeles have been undertaken. The WDAs supported by the programme have since been able to facilitate the compilation of community action plans (CAPs), appraise community projects and oversee project initiatives such as women goat groups and other initiatives e.g. school building and spring development.

In light of the fact that the WCBP is nearing its completion after 5 years (July 2004 to July 2009) a final external evaluation of the impact and effectiveness of the project is required. The final evaluation will be undertaken by a team of two external contractors.

This final evaluation encompasses five WDAs in five project special Woredas funded by EU located in SNNPR of Ethiopia (Konso, Burji, Basketo, Derashe and Amaro).

2. Objectives of the project of the final evaluation:

______94 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR The overall objective of FARM-Africa WCBP is “To reduce poverty in vulnerable households in 5 Woredas through building the capacity of government and civil society organizations to better access needs, plan, implement, monitor and evaluate their own development.

The Specific Objectives of the project are: 1. Accountable, financially independent WDAs that are responsive to locally identified needs and able to develop, implement and monitor and evaluate micro projects effectively. 2. Village Development Committees (VDCs) and Kebele Development Committees (KDCs) are able to develop community action plans and develop, implement and monitor and evaluate micro projects effectively. 3. Bottom-up planning institutionalized in Woreda level planning processes

The above specific objectives will be realised through the following Outputs:

Output 1: Five strong, sustainable and locally accountable WDAs built that are able to implement a process of locally initiated participatory development.

Output-2: The capacity of relevant Woreda government offices, WDAs, KDCs and VDCs built to develop, implement, monitor and evaluate a participatory planning system.

Output-3: Increase the livelihood base of the local community and most vulnerable households in each of the five Woredas.

Output-4: Write up lessons learnt from the WCBP and disseminate to donors, government, NGOs and regional networks.

3. The Aim of the Final Evaluation

The aim of the evaluation is to assess how successful the project’s strategy for WDA capacity building and empowerment has been, in order to identify what worked well, what did not work well and how FARM-Africa should approach such work in the future.

Therefore, the overall objective of this final evaluation is to assess the attainment of the project objectives against the project log-frame, impact trends, and sustainability. The evaluation framework involves five core areas:

The Specific Objectives of the evaluation are:

a) To evaluate the achievements of the project against its purpose, outputs and targets and its contribution to the overall objective as outlined in the project proposal;

______95 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR b) Assess the overall project efficiency and effectiveness, and the relevance and coherence of the project components /approaches in addressing the issue of WDA capacity building and empowerment; c) Evaluate the impact of the project on household livelihoods in the 5 special Woredas, and assess the sustainability of project impact; d) Draw best practices and key lessons learnt during project implementation which can be taken forward by FARM Africa or can be taken forward in the next phase project if recommended; e) Evaluate the effectiveness and efficiency of Farm Africa’s planning, management and M&E systems in the delivery of the programme; f) Provide recommendations for future actions by FARM-Africa’s and other actors.

The Key Areas of the evaluation review are:

In addition to above the final evaluation will consider each of the key strategies/approaches devised by the WCBP to achieve its purpose:

1. The effectiveness and appropriateness of bottom up planning approaches 2. Review the impact of Community Development Fund against community livelihood/ social and economic benefit of the target community 3. Capacity Building of partners 4. Overall recording/documentation of project interventions 5. Partnership approach (project /community partnership VDC, KDC) 6. Overall Sustainability of project interventions 7. Consider the specific challenges of the project at five Special Woredas. 8. Visibility actions as per the contractual agreement with the EC

From the above key areas the following Specific Issues for evaluation have been identified:

Key Area 1. Bottom up planning approach

Assess the appropriateness of the Bottom up planning approach (preparation of action plans, M+E etc.)

Key Area 2. Review the impact of Community Development Fund against community livelihood Assess the strategies and approaches of the CDF Projects, and their potential impacts. Review CDF management and utilization, Assess the impact of CDF in improving the livelihood of target beneficiaries.

Key Area 3. Capacity Building of partners Assess the capacity building strategies and their impacts with particular focus on WDAs skills and attitude to support projects identified earlier.

Key Area 4. Overall recording and documentation of project interventions Review File handling/organization of documents soft/hard copies Review the level of project activity/achievements documentation (monthly, quarterly, annual reports, workshop proceedings, others like leaflets. Review if the overall project best practices, approaches, challenges and lessons in terms of capacity building, bottom up planning and Community Development Fund has been recorded.

______96 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Key Area 5. Partnership approach (with Community/GO/NGO partnership) Assess whether the project partnership strategy with Community and government has been appropriate and effective.

Key Area 6. Overall sustainability of project interventions Financial sustainability - after funding has finished, the project activities are the responsibilities of WDAs and community. Assess if such trends are on line. Assess the overall commitment of WDAs to continue developing and monitoring the projects through its regular budget. Institutional sustainability - review the role of government with respect to community and WDAs as a regulator, advisor and supplier of service. Review the role of FARM Africa as a technical supporter, bringing new ideas, systems and knowledge through training, capacity building and local experimentation for both the community and the WDAs.

Key Area 7. Consider the specific challenges of the project at five sites. Draw out the key progress points of the projects in terms of development context issues (e.g. participation, changes in attitude; changes in practice; sustainability). Draw out the key challenges / blockages to the projects in terms of development issues (access, area /beneficiary targeting, conflict, infrastructure, etc.).

Some additional specific guiding questions are provided in Annex 1.

4. Final Evaluation Methodology:

The exercise will essentially consist of:

 An in depth review of relevant documentations including the project proposal documents, revised action plans, amendment documents, quarterly, bi-annual and annual progress reports, and EC Delegation, Farm Africa and others monitoring mission reports, mid-term evaluation, and other assessment and study reports.  Field visits (direct observation) to project sites in Burji, Amaro, Basketo, Konso and Derashe in the SNNPR, including focus group discussions and participatory assessment with the targeted beneficiaries, project staff, woreda administration sector offices, and collection of additional information through questionnaires and semi-structured interviews including on other similar initiatives implemented during the year 2005/2009 by both the government and other actors.  Conducting stakeholder mapping and SWOT analysis in consultation with various stakeholders.  Conducting comparative analysis with the baseline data (in as far as it is available) and performance results of the project in terms of project progress and changes observed since then.  A one day workshop to be organized at Hawassa for all stakeholders to review the preliminary findings and recommendations of the final evaluation exercise, and agree on those critical findings and future actions by concerned partners. Participants will be representative of WDAs/beneficiary community representatives, government partners at both Woredas and regional levels, and other actors NGOs, etc implementing similar initiatives in the Woredas/Region).  Write up of the findings incorporating field assessment results/findings, relevant material from secondary data, comments and agreed points coming out of the workshop as well as recommendations of the team so that the resulting document will inform follow-up activities.

______97 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR 5. Evaluation Team Composition

The evaluation team will consists of:

 Two local consultant (2 – one is team leader);  Representative from the regional BoFED (1 persons);  Project team of experts (2 staff);  Representatives of the five Woredas WDAs and SEPDA (6 persons);  Representatives of partner woreda sector offices (5 persons).

The final product will be the result of teamwork using participatory methodologies. The consultant – team leader will do the planning, orientation, check list update, field level analysis and for presenting the findings at the workshop. He will also be responsible for putting together the final document and for delivering the output as per agreement entered with Farm Africa.

The local consultants: The team leader should have minimum MSC and extensive experience of institutional strengthening and capacity building, rural development and livelihoods diversification and improvement. The second consultant should have the required qualification and experience related to participatory planning and monitoring of community based rural development programs and will work for only 15 days.

6. Time Frame for the evaluation exercise

The exercise will take an overall timeframe of about 29 days out of which the net working days for the assignment consultant will be 25 starting from the day of commencement (tentatively 6th July 2009) as per the time plan shown here below.

Time Required Activities Responsible

Planning including travel Document review; briefing with Farm-Africa and with Consultant days fro AA to A.minch EC Delegation; fieldwork design; preparation of (days 1 to 2) checklists. Travel to the project site; briefing with project team; setting up work teams; giving orientation on the Evaluation methodologies team led by Agreeing on checklists and approach; data collection consultant Field Work through focus group discussions, key informant (days 3 to 15) interviews, field visit to activity sites, etc. Discussions with five woreda WDAs and partner sector offices Pulling ideas together; preparation of summary of findings for presentation at the stakeholders’ workshop Debriefing with the project management team at Arba Minch.

Days 16 to 17 Back to Addis and debriefing with Farm-Africa country Farm-Africa office and EC Delegation and Consultant Days 18 to 19 Preparation for the workshop; conducting the Farm-Africa workshop with all stakeholders (representatives of and

______98 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR WDAs, local government partners, project Consultant beneficiaries, project team, the Donor - EC). Draft Report Compilation, write up and submission of draft report Consultant (days 20 to 22) 4 Reading the draft report and giving feedback to the Farm-Africa days consultant by Farm-Africa and EC Delegation Final Report. Incorporation of comments on the draft report; Consultant (days 23 to 25) preparation of final report; printing, binding and submission of final report. A total maximum of 25 days of work for a consultant

7. Expected Outputs

The final output of the evaluation exercise shall be:

 Submission of the draft report (in English language) strictly following the out line provided for standard presentation with out the annexes in three bound hard copies and a soft copy.

 Submission of the final draft report incorporating the comments from Farm Africa and the EC Delegation with three bound hard copies and a soft copy as per the following; * Volume ONE: synthesizing the findings and recommendations of the interventions with respect to each component/main activity. This volume should not exceed 50 pages. * Volume TWO: details of observations and findings with respect to each component/main activities including presentation and analysis of hard data’s collected.

7.1 Out line of the Evaluation report

 The structure of the report including the findings and recommendations shall follow the specific objectives/Outputs/Results/Activities as presented in the project document and LFM

 It is important, when drafting the report, to acknowledge clearly where changes in the desired direction are known to be already taking place, and so avoid misleading readers.

The findings and recommendations of the evaluation exercise shall be presented in accordance with the following outline:

I. Executive Summary: a tight drafted, to the point and free-standing Executive Summary is an essential component. It should be short no more than five pages with structured synthesis of: Introduction, Relevance, Efficiency, effectiveness, Impact and Sustainability. It should focus only on the key issues including the validity of the relevance of the action, outline the main analytical points, and clearly indicate the main conclusions, Lessons learned and specific recommendations. Cross – references should be made to the corresponding page or paragraph numbers in the main text that follows.

II. Main Text: the main text should start with an introduction describing, first, the all relevant back ground information in relation to the project intervention, second, the evaluation objectives. The body or core of the report should follow the standard evaluation criteria’s describing the facts and interpreting or analyzing them in accordance with the key questions pertinent to each criterion’s. The overall presentation need to be structured as per the project intervention logic/LFM and hard data collected with adequate sample case studies.

______99 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR

III) Conclusions and Recommendations: these should be the subject of a separate final chapter. Wherever possible, for each key conclusion there should be a corresponding recommendation. The key points of the conclusion will vary in nature but will often cover aspects of the key evaluation criterion including performance ratings.

IV) Annexes: All relevant hard data’s collected, Methodologies applied for the study and ToR, LFM, Map of the project intervention woredas/kebeles, list of persons/organizations consulted, documentations consulted, other technical annexes, etc

8. Proposal for the Consultancy

The lead consultant must provide: 1. A technical proposal 2. A work plan that includes a timetable, 3. A properly disaggregated financial proposal

Local transport will be provided by Farm Africa.

The submission deadline for proposal bids to the FARM-Africa Country Office in Addis Ababa is 1700 hours, 26 June 2009. Please call 0114-40-10-04 or 0114-40-02-05 for directions.

All proposal bids must be delivered in a sealed envelop and clearly marked WCBP (ONG/PVD/2003/064-438) Final Evaluation Bid Proposal.

______100 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR Annex 1: Specific guidance questions for the evaluation team are as follows:

The Final Evaluation (FE) will investigate the following questions: The over all progress of WCBP towards its four outputs in the project logical framework  What is the progress of the project to date towards these outputs?  What evidence is there that the project has had an impact and in which output(s)? Influence of the changing context on the project  How much has the context of WCBP changed since the proposal was written and how has this affected implementation?  What are the key issues in the new context of Ethiopia that need to be taken into account for the future of the project?  How can opportunities in the changing context be taken into account to ensure successful project impact? Current situation and opportunities for WDAs  What are the key issues at project/Woreda/region level influencing the progression of WDAs to accountable and independent civil society organisations?  What are the opportunities and constraints affecting WDAs? Project Design  How appropriate were the project objectives?  To what extent is the project design consistent with the local situation and the coherence of activities?  How responsive and flexible is the project document to problems identified during proposal development and the Mid Term Review? Documentation  How has the project gone about documenting and disseminating project experience and impact? And how effective has it been?

______101 Report on Final Evaluation of EU Funded FARM Africa’s Woreda Capacity Building Project in Five Special Woredas of SNNPR