PROJECT EVALUATION

Federal Democratic Republic of Ethiopia

Southern Region Cooperatives Development and Credit Programme

Completion Evaluation

November 2008 Via Paolo di Dono 44 - 00142 Rome, Italy Tel: +39 06 54592048 - Fax: +39 06 54593048 E-mail: [email protected] Web: www.ifad.org/evaluation

Document of the International Fund for Agricultural Development

Federal Democratic Republic of Ethiopia

Southern Region Cooperatives Development and Credit Project (SOCODEP)

Completion Evaluation

November 2008 Report No. 1909-ET

Evaluation T eam

Overall Responsibility Mr Luciano Lavizzari, Director, Office of Evaluation

Lead Evaluator Mr Andrew Brubaker, Evaluation Officer

Associate Evaluator Mr Michael Carbon, Associate Evaluation Officer

Consultants:

Team Leader Mr Richard C Carter

Team Members Mr Michael Assefa, Cooperatives and Capacity Building Specialist

Mr Tsegaye Asafaw, Credit and Microfinance Specialist

Mr Ayele Gebremariam, Socio Economist

Photo on cover page: Ox ploughing in Wolayta Zone Source: Evaluation Mission 2007

Federal Democratic Republic of Ethiopia

Southern Region Cooperatives Development and Credit Project IFAD Loan Nos. 342-ET, SRS 37-ET, BSF Grant BG-29-ET

Completion Evaluation

Table of Contents

Abbreviations and Acronyms iii Exchange Rate iv Map of Project Area v Executive Summary vii Agreement at Completion Point xvii

I. INTRODUCTION 1

II. COUNTRY BACKGROUND 2 A. Project Background 2 B. Objectives and Methodology of the Evaluation 7

III. PROJECT PERFORMANCE 8 A. Design Features 8 B. Implementation and Outputs 10 C. Restructuring of Service Cooperatives 12 D. Provision of Loans to Cooperatives and their Members 13 E. Attaining Project Objectives 20 F. Assessment: Relevance, Effectiveness and Efficiency 27 G. Performance of IFAD and its Partners 29

IV. PROJECT IMPACTS 30 A. Rural Poverty Reduction Impacts 30 B. Sustainability and Ownership 32 C. Innovation, Replicability and Scaling-Up 33

V. CONCLUSIONS AND RECOMMENDATIONS 34 A. Overall Assessment 34 B. Conclusions 35 C. Recommendations 37 D. Questions for the Forthcoming Country Programme Evaluation (CPE) 37

APPENDICES

1. Summary of Project Objectives 39 2. Project Costs and Financing 41 3. SOCODEP Timeline 43 4. Reconstructed Logframe for SOCODEP 45 5. Trainings carried out by SOCODEP 63 6. Evaluation Mission Itinerary 67 7. Organisations and Individuals Interviewed 69 8. Bibliography 73

TABLES

1. Statistical Data 3 2. SOCODEP Objectives 4 3. SOCODEP Numerical Targets and Achievements 12 4. Loans Made Through SOCODEP for Individual Activities 14 5. Rural Roads Constructed by SOCODEP (in km) 17 6. Changes in Service Cooperative Performance between 1993 and 2006 22 7. SOCODEP: Achievement of Specific Objectives 28 8. Project Ratings Summary 35

FIGURES

1. Budget Costs by Component (million US$, total 22.72m) 5 2. Main Project Stakeholders 6 3. SOCODEP: Quantitative Achievements against Targets at Appraisal 20 4. Cooperatives: Findings from the Evaluation 21

BOXES

1. Two Examples from Individual Beneficiaries of OMFI Loans 14 2. Three Case Studies of Women Loan Beneficiaries 23 3. Four Examples of Ox-loan Beneficiaries 25

ANNEXES*

1. Development of Cooperatives and Institutional Capacity Building 2. Evaluation of Credit and Input Supply Component of SOCODEP 3. Socio-Economic Impact

* All annexes are available from IFAD’s Office of Evaluation ( [email protected] ).

ii

Abbreviations and Acronyms

ACE Agricultural Cooperatives in Ethiopia Project AfDB African Development Bank AIDB Agricultural and Industrial Development Bank (now DBE) AISCO Agricultural Inputs Supply Corporation BOARD Bureau of Agriculture and Rural Development BOFED Bureau of Finance and Economic Development BOH Bureau of Health BSF Belgian Survival Fund BWMED Bureau of Water, Mines and Energy Development BPWUD Bureau of Public Works and Urban Development CBE Commercial Bank of Ethiopia CBO Community Based Organizations CLP Core Learning Partnership CPD Cooperatives Promotion Department CPE Country Programme Evaluation CPM Country Programme Manager DBE Development Bank of Ethiopia EB Ethiopian Birr GoE Government of Ethiopia HIV/AIDS Human Immuno-Deficiency Virus/Acquired Immuno-Deficiency Syndrome HSC Health and Sanitation Committees IFAD International Fund for Agricultural Development IFI International Financing Institution (W)IGA (Women’s) Income Generating Activity ILO International Labour Organisation KDP Kafa Development Programme (formerly SUPAK-S) M&E Monitoring and Evaluation MOFED Ministry of Finance and Economic Development MTR Mid-term Review NBE National Bank of Ethiopia OE Office of Evaluation OMFI Omo Micro Finance Institution PC/PCU Programme Coordination Unit PCR Project Completion Report PSC Project Steering Committee RA Roads Authority RR30/RR50 Rural Roads 30/50 Vehicles per Day Design Standards RUFIP Rural Financial Intermediation Project RWAT Rural Women’s Affairs Team SACCO Savings and Credit Cooperative SC(s) Service Cooperative(s) SNNPRS Southern Nations, Nationalities and Peoples Regional State SO Special Objective SOCODEP Southern Region Cooperatives Development and Credit Project SRAB Southern Region Agricultural Bureau SSE Small Scale Enterprise SUPAK-S Sustainable Poverty Alleviation Kaficho-Shakicho (renamed KDP) TA Technical Assistance UNOPS United Nations Office for Project Services USAID United States Agency for International Development US$ United States Dollar VOCA Volunteers in Overseas Cooperative Assistance VRDF Veterinary Revolving Drugs Fund WIGA Women Income Generating Activities WSHBS Water Supply, Health and Basic Sanitation

iii

Exchange Rate (US$ to Ethiopian Birr)

9.50 9.00 8.50 8.00 7.50 7.00 6.50 6.00

Ethiopian Birr to one USD 5.50 5.00

0 1 2 -0 t-0 t-0 t-03 t-04 t-05 t-06 c c c c c Oct-93 Oct-94 Oct-95 Oct-96 Oct-97 Oct-98 Oct-99 Oct O O O O O Oc

iv

v

vi

Federal Democratic Republic of Ethiopia

Southern Region Cooperatives Development and Credit Project (SOCODEP)

Completion Evaluation

Executive Summary

I. INTRODUCTION

1. Introduction. The Southern Region Cooperatives Development and Credit Project (SOCODEP) was undertaken in the Southern Region of Ethiopia between 1994 and 2005. This document is the completion evaluation report for the project. The evaluation mission took place between 17 th September and 11 th October 2006. It was conducted by a four-man team and the conduct, analysis and reporting of the evaluation has been carried out according to the Office of Evaluation (OE) guidelines for project evaluations.

A. Country Background

2. Ethiopia. Ethiopia is a very large (1.1m km 2), very diverse (geographically and ethnically), and extremely poor country. It ranks 170 th out of 177 countries in the current (2006) Human Development Rankings. The predominantly rural population depends heavily on subsistence agricultural production, the vast majority of which exists under rainfed conditions. Cash incomes are very low except in coffee-producing areas, and access to rural financial services is very limited. Physical infrastructure (roads, power supplies, water and sanitation, and telecommunications) is very poorly developed. Population growth, increasing prevalence of the Human Immuno-Deficiency Virus/Acquired Immuno- Deficiency Syndrome (HIV/AIDS), environmental degradation, and increasing climate instability are among the trends which add to the challenges of poverty alleviation in Ethiopia. Politically, Ethiopia has undergone a virtually continuous process of change since the 1974 revolution in which Emperor Haile Selassie was overthrown. The Marxist rule of the Derg from 1974 to 1991 led to many reforms, but the authoritarian use of power over the peasantry did not change, or if anything, it became more restrictive. After the fall of the Derg in May 1991, a slow process of liberalisation of economy and political systems has taken place, but there is still a long way to go before the rural population will be fully able to participate in a free market and transparent democratic processes of government. There is no doubt that Ethiopia is a very difficult country in which to make progress in terms of development and poverty alleviation.

3. The Southern Region. At 232,000km 2, the Southern Nations, Nationalities and Peoples Regional State (SNNPRS) is one of the four largest Regions in Ethiopia (out of the 9 Regions and 2 City Authorities into which the country is presently divided). It is probably the most diverse Region in ethnic terms, and it contains some of the most remote and wettest parts of Ethiopia. At the time of project commencement in 1994, the Region consisted of 53 Woredas (districts), and had a total population of about 11m. Twelve years later, at the time of the completion evaluation the number of Woredas had grown to 133 and the total population to about 14.5m. Mean population density in the Region has consequently increased from about 47 to 63 per km 2, but these averages hide a great deal of variation; at Appraisal in 1994, population densities were thought to range from less than 10 to over 500 per km 2. Rainfall, temperatures, altitudes, cropping patterns, infrastructure, and cultures vary widely within the Region. B. The Project

4. The project. SOCODEP was one of the first significant internationally funded interventions in Ethiopia following the fall of the Derg in May 1991. The Project aimed to respond to the then new legislation (Proclamation 85/1994) concerning Cooperatives, which ostensibly set out a means of turning the former Government-imposed and politically-dominated Producer Cooperatives of the Derg

vii

into farmer-owned viable business entities serving their members’ interests. In particular, it aimed to make rural finance (specifically micro-credit) available to so-called Service Cooperatives and their members. The Principal Objective of the Project was to “increase agricultural productivity and raise income levels of the rural poor through support to Service Cooperatives’ development in order to facilitate efficient provision of sustainable services to members”. The Project had 7 specific objectives which are set out in Table ES1 and further elaborated in the reconstructed logframe in Appendix 4. Together these objectives were designed not only to turn Service Cooperatives (SCs) into effective vehicles for the service of their members, but also to deliver some specific services (road construction, veterinary drug supply, health facility up-grading, and provision of water supply and sanitation services) directly through the relevant Government organs. The Water Supply, Health and Basic Sanitation Component (WSHBS) was separately financed by the Belgian Survival Fund as a later addition to the Project.

Table ES1. SOCODEP Specific Objectives SO1 Provide a model for developing Ethiopian cooperatives under the new legislation, particularly with respect to improvement of financial intermediation services in rural areas, which could be replicated in other areas of the country. SO2 Increased capital and income among the rural poor in the project area through off-farm income-generating activities particularly for women and families in densely populated areas with limited land for farm expansion. SO3 Strengthen the SRAB to carry out its mandate with respect to cooperative development. SO4 Provide credit to meet financial requirements for agricultural inputs and draught oxen and facilitate the supply of inputs through support to local traders and cooperatives. SO5 Relief of livestock health constraints, particularly with respect to draught animals, through provision of veterinary drugs. SO6 Improve access of rural families to services and markets by rehabilitating and maintaining rural roads. SO7 (BSF Component) Reduce the burden of disease in 8 woredas of the SOCODEP area

C. Objectives and Methodology of the Evaluation

5. Overall objectives. The main objectives of the evaluation were to: (i) assess the performance and impact of the SOCODEP project; and (ii) generate a series of findings and recommendations that would serve the International Fund for Agricultural Development (IFAD), the Government of the Ethiopia, and other donors in designing and implementing similar projects and programmes in the future.. The special focus of the evaluation, as highlighted in the Approach Paper, was to be on Cooperatives Development, Rural Microfinance, Socio-economic impact, and Institutional Capacity- Building.

6. Evaluation methodology. The evaluation followed OE’s guidelines for project evaluations. 1 The evaluation team conducted a field visit to the project area and interviews with key stakeholders. The evaluation notes the limited range of reports and other documents available through the project and partners due to the frequent moving of the project office which made it challenging to find many documents.

1 This included assessing the project against internationally recognized evaluation criteria, namely: (i) project performance, including relevance, effectiveness and efficiency; (ii) impact on rural poverty; and (iii) performance of partners involved in the project, including IFAD, government institutions, and others.

viii

7. However, the core documents were available and provided a sufficient source of secondary data for the evaluation. As per standard OE practice, a Core Learning Partnership (CLP) 2 was constituted for the evaluation, which provided critical inputs and views at key stages of the evaluation process.

II. PROJECT PERFORMANCE

A. Implementation and Outputs

8. Cooperatives restructuring (US$0.95m, four per cent of total budget at Appraisal). Cooperatives under the former Government (the Derg) were an instrument of Marxist ideology and coercion. A great deal of work would have to be done to turn these into autonomous businesses owned and democratically controlled by their members. The number of cooperatives restructured through SOCODEP under the new legislation exceeded the Appraisal target, 200, by about 30 per cent. Credit delivery (US$7.68m, 34 per cent of total budget at Appraisal).

9. Credit was to be delivered to cooperatives for their own businesses (flour mills, shops, produce marketing) and through the cooperatives to the individual members (for farm inputs, income- generating activities, small-scale enterprises, and work oxen). The Commercial Bank of Ethiopia (CBE) was to be the channel for disbursement of loans to the restructured cooperatives. Overall, the disbursement of loans fell below target, at EB 18.76m by the time of closure of this project component. This is a little over US$2.0m or approximately 30 per cent of target. This low number of loans disbursed was attributable partly to project design shortcomings and partly to problems encountered during project implementation. Specific factors included: CBE’s lack of experience in rural credit implementation; delays in loan processing; lack of capacity in the restructured Service Cooperatives (SC) to borrow and on lend; and defaults in loan repayment.

Promotion of Off-farm Income Generating Activities (US$0.29m, 1 per cent)

10. Small enterprise promotion was undertaken through a pilot project involving training at the Rural Technology Centre, Sodo, and subsequent loans to individuals setting up in business. Training was provided to 226 beneficiaries, in bamboo work, tailoring, weaving, spinning, woodwork, beekeeping and pottery. Loans for subsequent business development amounted to EB 55,456 (approximately US$8,000), three percent of the target, in total over the entire project period. The Women Income Generating Activities (WIGAs) performed much better providing 7,600 loans (76 percent of the target) to women.

Institutional Capacity Building

11. Capacity-building, through training and provision of physical resources, was a major aspect of the project. Explicit capacity building inputs focused on Southern Region Agricultural Bureau (US$2.80m, 12 per cent of total budget) to enable the Bureau to deliver support to the SCs and CBE (US$0.39m, two per cent) to provide effective rural finance to the SCs. But there were significant allocations also to the other Regional Government organs involved (Water, Health, and Roads), as well as to the cooperatives themselves.

12. As far as physical resources are concerned, the project provided large numbers of vehicles (65 cars, three buses, 300 motorcycles), as well as office accommodation, office equipment and road- building equipment. Perhaps unsurprisingly, disbursement for these items proceeded quickly, with

2 Members of the partnership included: Ministry of Finance and Economic Development, Bureau of Finance and Economic Development, Bureau of Agriculture and rural Development, Southern Nations, Nationalities and Peoples Regional State (SNNPRS), Department of Cooperatives, SNNPRS, Rural Women’s Affairs Team, SNNPRS, Bureau of Health, SNNPRS, Bureau of Water Resources, SNNPRS, Planning and Programme Department, Rural Roads Authority, SNNPRS, Sodo Rural Technology Centre, Commercial Bank of Ethiopia, Omo Microfinance Institution, the Association of Ethiopian Microfinance Institutions, Former SOCODEP Project Director, field presence officer, and the IFAD Country Programme Manager.

ix

large overdrafts on the vehicles and equipment category being highlighted in the 1997 and 1998 United Nations Office for Project Services (UNOPS) Supervision Reports.

Road Construction (US$4.70m, 21 per cent)

13. The target at Appraisal was for the rehabilitation and heavy maintenance of a total of 700 km of rural roads, to a design standard that would permit labour-based maintenance. In the end, 122 km was constructed, to a higher standard (and consequently higher unit cost) than originally agreed. Thus reducing the amount of construction and rehabilitation possible and requiring machine-based maintenance in the future.

Veterinary Revolving Drugs Fund (US$1.89m, 8 per cent)

14. This was intended to be a complement to other project activities – specifically to help to protect the livestock which were involved in ox loans and credit for small ruminants. The first purchases of drugs under this component were delayed significantly, taking place in 2001, and resulted in only about 19 per cent of the budgeted funds being disbursed by the time of closure of the project loans.

Water Supply, Health and Basic Sanitation (US$4.02m, 18 per cent) – Belgium Survival Fund (BSF) Funded

15. Despite the assessment of a pre-Appraisal mission in 1993 that “…water supply is not a major problem in many parts of the project area … and in areas where needs are pronounced several NGOs are providing assistance”, a formulation mission addressing this and the wider issues of health and sanitation was sent in 1996 and the component commenced in 1999. This component has performed well meeting its targets related to health services (through construction, rehabilitation, equipping and training) and construction of water supply points in the 8 component Woredas.

B. Assessment Relevance, Effectiveness and Efficiency

16. Relevance. SOCODEP addressed some real needs of the rural poor in southern Ethiopia (need for credit, need for improved market access, need for better health services and environmental health) and it was consistent with the government’s regionalization programme. At the time of design the rhetoric of the new Government and the framing of the new legislation made working through service cooperatives attractive and promising. However the use of service cooperatives as the channel for services to the poor turned out to be a significant design weakness. Similarly, the project had limited choices for partners with rural finance experience and should have been addressed with greater seriousness.

17. Effectiveness. The Project achieved some, but not all of its objectives. The Specific Objectives (SO) 1, 3, and 5 were mostly not met. These include SO1, providing a model for developing Ethiopian cooperatives under the new legislation, SO3 strengthening the Southern Regional Agricultural Bureau (SRAB) to carry out its mandate with respect to cooperative development, and SO5 relief of livestock health constraints, particularly with respect to draught animals, through provision of veterinary drugs. Specific Objectives 2, 4, 6 and 7 were met to varying degrees and include SO2 increasing capital and income among the rural poor in the project area through off-farm income-generating activities, SO4 providing credit to meet financial requirements for agricultural inputs and draught oxen and facilitate the supply of inputs through support to local traders and cooperatives, SO6 improving access of rural families to services and markets by rehabilitating and maintaining rural roads, and SO7 (BSF Component) reducing the burden of disease in eight woredas of the SOCODEP area.

18. Efficiency. Given the difficult operating environment in Ethiopia the project could have been more efficient if it had been more realistic and less ambitious. The vast geographic area, poor infrastructure and communication meant that the projected resources were not optimally used. Also, the overall emphasis on the Project’s numerical outputs (i.e. exceeding the number of restructured cooperatives) rather than the quality of those outputs, has resulted in a significant amount of wasted resources. As a result, many of the project’s activities were diluted to the extent that they failed to

x

achieve their intended outcomes. Efficiency would have been greatly enhanced if the project design had concentrated on a smaller project area, involving a smaller number of SCs, and provided greater intensity of effort and resources.

C. Project Impact by Component

Cooperatives Restructuring

19. The number of cooperatives restructured through SOCODEP under the new legislation exceeded the Appraisal target by about 30 per cent. However, the quality of the restructuring and re- orientation undertaken has not translated into an effective cooperatives sector in the project areas. The cooperatives encountered in this evaluation, and many others for which data was obtained, are suffering significant problems of poor management, misappropriation of funds and de facto bankruptcy. The findings of the evaluation in this regard are not new. To quote the internal review of the project carried out in 1999: “…with such limited capital, unprofitable business, increased misappropriation, weak management, absence of member education, and poor participation of members, the conceptual framework of promoting economically viable cooperatives … could not be achieved.” 3 Three years later, at the time of closure of the main loans, the situation had not changed significantly: “…the state of some cooperatives restructured under the project appear to be weak and in the worst case on the verge of bankruptcy … most restructured cooperatives critically need the support of the project …” 4. In short, although SOCODEP re-registered about 267 cooperatives under the new law, it failed to re-orient the officers and members of those organisations, and leave behind a strong and sustainable cooperatives sector.

Credit Delivery

20. The disbursement of loans achieved approximately 30 per cent of its target. Although many cooperatives and individuals received and benefited from loans, many of the target populations did not. Loans for cooperatives’ own businesses were generally unsuccessful. Because the cooperatives found that they could not compete with the leaner and fitter private sector, the majority of these ventures failed, and the cooperatives failed to repay their loans. Loans to individuals were generally much more successful, with ox loans being particularly popular (although the required ox insurance was not popular, and it failed to pay out in the majority of cases of mortality). Loans to women for the multiplication and fattening of small ruminants were also especially valued, and repayment rates were high. In general, repayment of loans was better in the western part of the project area than in the east and south-east.

21. CBE learned in the course of the project that provision of rural microfinance is a very different activity to the provision of commercial, urban, loans. It found it was ill-equipped to manage this project component, and it withdrew in 2002. The transfer of this component to the Region’s only Omo Micro Finance Institution (OMFI) in the last few months before closure of project loans was the only solution to a crisis. Because of the difficulties experienced by CBE, and the poor performance of many of the cooperatives, the opportunities for cooperatives or individuals to access further loans has been curtailed. Real benefits enjoyed by individuals made a difference for a time, but they may not have brought about lasting change.

Promotion of Off-farm Income Generating Activities

22. Small enterprise promotion was undertaken through a pilot project involving training at the Rural Technology Centre, Sodo, and subsequent loans to individuals setting up in business. The impact of this activity on a few individuals was significant; however, the extent of that impact was negligible.

3 SNNPRS Cooperative Office (1999) Comprehensive Analaysis of Current Situation and Requirements of Cooperatives in SOCODEP Areas, April 1999, Awassa. 4 UNOPS Supervision Report, April 2002.

xi

Institutional Capacity Building

23. A great deal of training was carried out (estimates suggest that nearly 57,000 individuals received some form of training). Individuals have benefited to some extent, but the ability of the project stakeholder institutions to perform more effectively, during and subsequent to the project, is questionable. A lasting impact on the institutions involved has been severely undermined by the capacity buildings component’s ad hoc nature 5 and by the frequent Government restructuring and re- deployment of personnel.

24. SOCODEP’s capacity-building efforts largely ignored issues of attitude change, organizational reform, and policy dialogue. These were major omissions.

Road Construction

25. The roads built are of good quality and have benefited those people living near them. The Roads Authority has been left with a useful pool of heavy equipment for future road construction work. As far as SOCODEP outcomes are concerned however, the aim of this aspect of the project was to extend access to markets for cooperatives and their members. As with other project components, a few people have benefited from the component, and its main shortcoming has been its limited reach.

Veterinary Revolving Drugs fund

26. The component was able to provide temporary relief for the few participants who were able to access the fund. At the time of the evaluation, drug shortages were being felt, and animal health problems were reported to be increasing, especially in those areas where tyrpanomiasis is prevalent.

Water Supply, Health and Basic Sanitation (US$4.02m, 18 per cent) – BSF funded

27. In some respects the WSBHS component has delivered the most visible and potentially successful aspects of the entire project, with its most effective aspects being the strengthening of health services (through construction, rehabilitation, equipping and training) and construction of water supply points in the eight component Woredas. However, the functional sustainability of water points, and the utilisation of improved excreta disposal facilities at household level, are areas of concern. In particular, user fees for water point maintenance are too low, and access to spare parts for hand pumps is extremely problematic.

Integration Across Components

28. SOCODEP was a multi-component project, involving at least 7 Regional Government organs 6, three financial institutions 7, and numerous other stakeholders 8. There were important potential synergies between components and stakeholders. The cooperatives were fundamental. Their capacity to function as viable democratic business entities would determine the extent to which all the individual project beneficiaries would gain. In the event, their lack of effective re-orientation and capacity-building has limited the extent to which individual men and women have enjoyed the benefits of cooperative membership. Credit disbursement depended on the sound functioning of both the cooperatives and CBE, as the financial intermediary. Neither the cooperatives nor CBE were effective in channelling loans to individual members. Institutional strengthening of Government organs was greatly limited in its impact, because of the unsystematic and incomplete way in which it was done, and because of frequent Government restructuring. Supply of veterinary drugs, construction of roads, enhancement of health services, and construction of water supplies and sanitation facilities were

5 The evaluation team could find no evidence of any systematic training needs assessment. 6 Those responsible for Agriculture, Cooperatives, Veterinary Services, Water, Health, Finance & Planning, and Roads, organised in different ways at different times. 7 CBE, OMFI, and Ethiopian Insurance Corporation. 8 Cooperatives and their members, contractors, and consultants.

xii

carried out as separate activities, and in the case of the first two, very little was actually achieved. The potential synergies were not realised, and the project as a whole could not be described as integrated.

D. SOCODEP Assessment

29. Table 2 summarizes the ratings attributed to the project by the evaluation, with the average ratings given to project evaluated in 2005 for comparison. On many criteria SOCODEP scored one point under the 2005 average. Impact on physical, financial and human assets is comparable with 2005 averages, but the performance of the Southern Region Government is assessed as particularly below the 2005 average. Because of fundamental weaknesses in project implementation, caused partly by unrealistic design, partly by insufficiently decisive management, partly by insufficient political commitment, and partly by externalities including frequent Government restructuring and reshuffling, SOCODEP is assessed overall by this evaluation as ‘Moderately unsuccessful’ (rating of 3).

Table ES2. SOCODEP Project Performance: Rating Summary 2005 Project Evaluation Criteria SOCODEP Rating Evaluations Average Project performance Relevance 5 5 Effectiveness 3 4 Efficiency 3 4 Partner performance IFAD 3 4 Government 3 4 BSF 4 - UNOPS 4 4 Project impact Agriculture Productivity 3 - Physical and Financial Assets 3 4 Human Assets 4 4 Institutions and Services 2 - Social Capital and Empowerment 2 4 Food Security 3 4 Environment and Common Resource 4 4 Base Markets 2 - Overall impact 3 - Sustainability 2 4 Innovation, Replicability and Scaling-Up 3 4 Overall assessment 3 - Source: The Evaluation Mission 2007

V. CONCLUSIONS, RECOMMENDATIONS AND KEY ISSUES FOR THE FUTURE

A. Conclusions

30. As with most multi-component projects, the project performance and impacts of SOCODEP have been mixed. There seems little doubt that the Project has had a net benefit to the Region, to individuals in Government, and to some of the target beneficiaries. The key questions relate to understanding how the benefits could have been greater, and what lessons can be learned for present and future projects in Ethiopia and further a field.

31. Context. SOCODEP was one of the first significant internationally funded interventions in Ethiopia following the fall of the former Marxist-Leninist regime (the Derg) in May 1991. The Project aimed to respond to the then new legislation concerning Cooperatives, which ostensibly set out a means of turning the former Government-imposed and politically-dominated Producer Cooperatives of

xiii

the Derg into farmer-owned viable business entities serving their members’ interests. In particular, it aimed to make rural finance (specifically micro-credit) available to so-called Service Cooperatives and their members.

32. Design. The Project aim, as described in paragraph 31, was an imaginative attempt to respond to the then new legislation concerning Cooperatives. However, this was an ambitious and in hindsight an unrealistic goal. A number of factors contributed to the difficulties faced by the SOCODEP. First, the Region’s size, diversity, poor infrastructure and poverty posed numerous development challenges. Second, the project underestimated the obstacles to and rate of beneficial change and the capacity of the government for implementation in the post Derg period.

33. Quality of project delivery. SOCODEP concentrated on delivery of numerical outputs, such as cooperatives restructured, credit disbursed, trainings delivered, drugs purchased, ٛ ilometres of road constructed, water points built, and so on. Insufficient emphasis was placed on the quality of these outputs. For example, insufficient consideration was given to the intensity and duration of activities required to achieve the desired quality standards. In particular, the human factor of individual and group (community, cooperative, institution) attitudes were addressed minimally. For example, the design was too optimistic about the speed with which the former model of cooperatives, centrally controlled by the government, could be turned around into a member-owned and member-controlled viable business model. To turn around a failing, politically-established cooperative to become a viable business serving its members, or to bring about community ownership and management of a water point demanded a great deal of attention to quality of the investment, not just numbers.

34. However, some of the activities such as upgrading of health facilities and training of staff and community health workers performed better.

35. Relevance, effectiveness, efficiency. The project objectives and activities were relevant to needs of the rural poor in southern Ethiopia (e.g., in terms of the need for credit, improved market access, better health services and environmental health) and were consistent with the government’s regionalization programme. The Project was moderately ineffective achieving some, but not all of its objectives. Similarly, the project was moderately inefficient. Given the difficult operating environment in Ethiopia, the project could have been more efficient if it had been more realistic and had less ambitious objectives and coverage. The vast geographic area, poor infrastructure and communication meant that the projected resources were not optimally used.

36. Sustainability. SOCODEP’s benefits are unlikely to continue, partly due to the lack of a defined exit strategy. Moreover, institutional sustainability is limited and on-going access to credit, and water supply is not assured.

37. Innovation. At the time of design, SOCODEP’s focus on cooperatives and credit represented a response to the apparent liberalization of national politics and economics, and to the change in cooperatives legislation. Unfortunately, the country context changed rapidly and the design became less relevant given the new context. Despite the positive efforts at the Mid-term Review (MTR), the design adjustments were not adequate given the changing realities. On another issue, the BSF component introduced an effective monitoring and evaluation system, which however was not integrated into the other project components. As the project was overstretched and its components were not integrated, SOCODEP offered little opportunity for the learning being generated to be feed back into the project. Hence, the evaluation considered the project to be moderately unsuccessful in terms of innovations, replicability and upscaling.

38. Policy dialogue. SOCODEP was largely responsive to policy changes and government-led restructuring. There is little evidence however of the project contributing to IFAD’s effective engagement in policy dialogue in the country.

39. Participation. Probably the single greatest assurance of sustainability at the level of households and communities is through real commitment to beneficiary participation. However, the evaluation found that ensuring beneficiary participation in an area with a weak tradition of participation is

xiv

challenging and requires greater commitment in terms of time and resources. As such, approaches which build on existing social capital (i.e., using indigenous Community Based Organizations), rather than working through structures imposed from above and outside the beneficiary communities, are most likely to succeed in the short and long term.

40. Integration. This evaluation report has highlighted at a number of points the lack of integration between the numerous stakeholders and components within SOCODEP. Although integration is not easy, particularly given the restructuring of government organs and redeployment of personnel, it is the only way to create synergies which can maximize the impact of limited budgets.

41. Management. The management model used by SOCODEP limited its effectiveness and responsiveness to rapid contextual changes that occurred during project implementation. The Project Coordinator, attempted to harmonize and synchronize the work of several Government organs and other stakeholders over whom he has no real authority. And support from IFAD through supervision mission mounted annually by the designated cooperating institution the United Nations Office for Project Services (UNOPS) was insufficient. However, it should be recognized that during the time of SOCODEP, IFAD did not have modalities such as direct supervision or field presence to support project implementation. Although, IFAD was responsive in using the tools it had at the time, for example by undertaking an early and useful MTR and facilitating the inclusion during implementation of the important BSF component.

42. Weak linkages between partners during implementation. The linkages between the key stakeholders have mainly been achieved by the efforts of the Project Coordinators (four in total, at various stages of the Project), IFAD’s Country Programme Manager (CPM), and UNOPS. The Project Coordinator’s role was challenging because of his inherent lack of authority, and the CPM’s role is distant from the day-to-day project management issues to do more than provide general support and guidance. Also, annual visits by a cooperating institution are insufficient to rescue an under- performing project. Consequently, the effectiveness of the partnership was limited. Partnership with the private sector was not a viable option in the early years of the Project, but this option could have been pursued as implementation progressed. The extent to which partnerships outside of the project were developed by the implementing stakeholders is limited.

B. Recommendations

43. Design. It is recommended that consideration be given to interventions which are far more focused in terms of numbers of beneficiaries to be reached and geographic coverage, within the overall framework of IFAD’s targeting policy. This would ensure greater synergies across activities and ultimately deeper impact on rural poverty. Similarly, the project duration should be long enough to achieve the desired results and in particular take into account the time needed to implement attitude and cultural changes. Project management structures should be kept simple to ensure the integration and harmonization among different implementing agencies.

44. Quality of Project Delivery. It is recommended that greater attention be given in future project design and implementation to country context issues, and the identification of indicators of quality, and actions necessary to achieve real and lasting impact, alongside those relating to numerical outputs.

45. Policy Dialogue. It is recommended that more explicit attempts be made to engage in policy dialogue with Government and other development actors, where appropriate and required involving a wider range of national and international specialists, rather than just IFAD and cooperating institution staff.

46. Participation. It is recommended that future IFAD-funded projects and programmes in Ethiopia pay more attention to people’s participation, especially as it has not been a tradition of development practice in Ethiopia in the past.

47. Integration. In future multi-component projects, it is recommended that greater attention be paid to the linkages between the components and between those agencies responsible for delivering

xv

them. The evaluation team is in favour of projects which involve multiple components addressing the diverse needs of target populations – but the difficulties of integrating such efforts should be carefully considered.

48. Management. Management is more than coordination or supervision, and it should be addressed with greater rigour in future projects and programmes particularly in challenging contexts as found in SOCODEP. Projects need to be much more decisively managed. It is recommended that new approaches be explored, either through IFAD itself taking a more hands-on role during execution, facilitated by the Fund’s field presence officer, which allow closer monitoring and follow-up to implementation.

49. Role of the field presence officer. The field presence officer can, among other tasks, provide implementation support to IFAD-funded operations and has the potential to enhance partnerships and policy dialogue in Ethiopia. Hence, the country presence should be further strengthened, so that it can play a greater role in enhancing IFAD’s development effectiveness in Ethiopia.

C. Questions for the Forthcoming Country Programme Evaluation (CPE)

50. In view of the CPE for Ethiopia which is planned for 2007, a number of key questions arise from the present evaluation, which should be addressed in that context. These are:

• Have the present project identification, formulation and appraisal processes encouraged the setting of unrealistic targets and spreading project of activities too thinly? If so, how can these tendencies be avoided in future?;

• Are the existing project identification, formulation and appraisal processes sufficiently participative, in a country in which participation is not a strong tradition? If not, can they be made more so?

• To what extent have the detailed capacities of project stakeholder institutions been routinely assessed at the formulation stage, in order to design appropriate capacity-building programmes? What improvements can be made to this process?

• What has been the impact of the BSF contributions in Ethiopia? How can the partnership be enhanced in future activities?

• How can the present model of project management and supervision be modified to create a significantly greater degree of the Programme Coordination Unit (PCU) authority and effectiveness, without de-coupling projects from the implementing institutions? Should separate project management structures be set up, perhaps using consulting firms, or can the existing PCU framework be made more effective? How could the field presence officer be more effective? What model best fits the Ethiopian context? Should clearer guidelines be framed, setting out responsibilities for taking actions on supervision and MTR recommendations? Where does the buck stop in terms of project management?; and

• In view of the weak performance of Monitoring and Evaluation (M&E) in some Ethiopian projects, how can a monitoring culture be encouraged, and how can manageable frameworks be developed and implemented for monitoring of project performance, reflecting both quantitative and qualitative achievements?

xvi

Federal Democratic Republic of Ethiopia

Southern Region Cooperatives Development and Credit Project (SOCODEP)

Completion Evaluation

Agreement at Completion Point

I. BACKGROUND AND CORE LEARNING PARTNERSHIP

1. The Southern Region Cooperatives Development and Credit Project (SOCODEP) was one of the first significant internationally funded projects in Ethiopia following the fall of the former Marxist- Leninist regime (the Derg) in May 1991. The Project aimed to respond to the then new legislation (Proclamation 85/1994) concerning Cooperatives, which set out a means of turning the former Producer Cooperatives into farmer-owned viable business entities serving their members’ interests. In particular, it aimed to make rural finance (specifically micro-credit) available to so-called Service Cooperatives and their members.

2. The project was implemented in the Southern Nations, Nationalities and Peoples Regional State (SNNPRS), which is one of the four largest Regions in Ethiopia and is probably the most diverse Region in ethnic terms, and it contains some of the most remote and wettest parts of the country. SOCODEP was fully implemented from September 1994 until December 2005. More specifically, the components funded by the International Fund for Agricultural Development (IFAD) (excluding the water, sanitation and health component) were implemented from September 1994 to July 2002, a total of nearly 8 years with the Mid-term Review (MTR) at the end of 1996. Work on the Belgium Survival Fund (BSF) component (i.e., the one on water, sanitation and health) commenced in November 1998 and continued until the Project was finally closed in December 2005. The responsibility for implementing the project changed several times, but began with Southern Regional Agricultural Bureau (SRAB) and eventually migrated to the Bureau of Finance and Economic Development (BOFED). The evaluation of the project was conducted in late 2006/7 by IFAD’s Office of Evaluation (OE).

3. Following usual practice for OE evaluations, a Core Learning Partnership 1 (CLP) was established providing critical inputs at key stages in the evaluation, including comments on the main evaluation deliverables.

II. MAIN EVALUATION FINDINGS

4. Design. The Project aim, as described in paragraph one, was an imaginative attempt to respond to the then new legislation concerning Cooperatives. However, this was an ambitious and in hindsight an unrealistic goal. A number of factors contributed to the difficulties faced by the SOCODEP. First, the Region’s size, diversity, poor infrastructure and poverty posed numerous development challenges. Second, the project underestimated the obstacles to and rate of beneficial change and the capacity of the government for implementation in the post Derg period.

1 Members of the partnership included: Ministry of Finance and Economic Development, Bureau of Finance and Economic Development, Bureau of Agriculture and rural Development, SNNPRS, Department of Cooperatives, SNNPRS, Rural Womens Affairs Team, SNNPRS, Bureau of Health, SNNPRS, Bureau of Water Resources, SNNPRS, Planning and Programme Department, Rural Roads Authority, SNNPRS, Sodo Rural Technology Centre, Commercial Bank of Ethiopia, Omo Microfinance Institution, the Association of Ethiopian Microfinance Institutions, Former SOCODEP project Director, field presence officer , and the IFAD country programme manager.

xvii

5. Quality of project delivery. SOCODEP concentrated on delivery of numerical outputs, such as cooperatives restructured, credit disbursed, trainings delivered, drugs purchased, kilometres of road constructed, water points built, and so on. Insufficient emphasis was placed on the quality of these outputs. For example, insufficient consideration was given to the intensity and duration of activities required to achieve the desired quality standards. In particular, the human factor of individual and group (community, cooperative, institution) attitudes were addressed minimally. For example, the design was too optimistic about the speed with which the former model of cooperatives, centrally controlled by the government, could be turned around into a member-owned and member-controlled viable business model. To turn around a failing, politically-established cooperative to become a viable business serving its members, or to bring about community ownership and management of a water point demanded a great deal of attention to quality of the investment, not just numbers.

6. However, some of the activities such as upgrading of health facilities and training of staff and community health workers performed better.

7. Relevance, effectiveness, efficiency. The project objectives and activities were relevant to needs of the rural poor in southern Ethiopia (e.g., in terms of the need for credit, improved market access, better health services and environmental health) and were consistent with the government’s regionalization programme. The Project was moderately ineffective achieving some, but not all of its objectives. Similarly, the project was moderately inefficient. Given the difficult operating environment in Ethiopia, the project could have been more efficient if it had been more realistic and had less ambitious objectives and coverage. The vast geographic area, poor infrastructure and communication meant that the projected resources were not optimally used.

8. Sustainability of the SOCODEP’s benefits are unlikely to continue, partly due to the lack of a defined exit strategy. Moreover, institutional sustainability is limited and on-going access to credit, and water supply is not assured.

9. Innovation. At the time of design, SOCODEP’s focus on cooperatives and credit represented a response to the apparent liberalization of national politics and economics, and to the change in cooperatives legislation. Unfortunately, the country context changed rapidly and the design became less relevant given the new context. Despite the positive efforts at the MTR, the design adjustments were not adequate given the changing realities. On another issue, the BSF component introduced an effective monitoring and evaluation system, which however was not integrated into the other project components. As the project was overstretched and its components were not integrated, SOCODEP offered little opportunity for the learning being generated to be feed back into the project. Hence, the evaluation considered the project to be moderately unsuccessful in terms of innovations, replicability and upscaling.

10. Policy dialogue . SOCODEP was largely responsive to policy changes and government-led restructuring. There is little evidence however of the project contributing to IFAD’s effective engagement in policy dialogue in the country.

11. Participation. Probably the single greatest assurance of sustainability at the level of households and communities is through real commitment to beneficiary participation. However, the evaluation found that ensuring beneficiary participation in an area with a weak tradition of participation is challenging and requires greater commitment in terms of time and resources. As such, approaches which build on existing social capital (i.e., using indigenous Community Based Organizations), rather than working through structures imposed from above and outside the beneficiary communities, are most likely to succeed in the short and long term.

12. Integration . This evaluation report has highlighted at a number of points the lack of integration between the numerous stakeholders and components within SOCODEP. Although integration is not easy, particularly given the restructuring of government organs and redeployment of personnel, it is the only way to create synergies which can maximize the impact of limited budgets.

xviii

13. Management . The management model used by SOCODEP limited its effectiveness and responsiveness to rapid contextual changes that occurred during project implementation. The Project Coordinator, attempted to harmonize and synchronize the work of several Government organs and other stakeholders over whom he has no real authority. And support from IFAD through supervision mission mounted annually by the designated cooperating institution the United Nations Office for Procurement Services (UNOPS) was insufficient. However, it should be recognized that during the time of SOCODEP, IFAD did not have modalities such as direct supervision or field presence to support project implementation. Although, IFAD was responsive in using the tools it had at the time, for example by undertaking an early and useful MTR and facilitating the inclusion during implementation of the important BSF component.

14. Weak linkages between partners during implementation . The linkages between the key stakeholders have mainly been achieved by the efforts of the Project Coordinators (four in total, at various stages of the Project), IFAD’s Country Programme Manager (CPM), and UNOPS. The Project Coordinator’s role was challenging because of his inherent lack of authority, and the CPM’s role is distant from the day-to-day project management issues to do more than provide general support and guidance. Also, annual visits by a cooperating institution are insufficient to rescue an under- performing project. Consequently, the effectiveness of the partnership was limited. Partnership with the private sector was not a viable option in the early years of the Project, but this option could have been pursued as implementation progressed. The extent to which partnerships outside of the project were developed by the implementing stakeholders is limited.

III. KEY RECOMMENDATIONS AGREED BY PARTNERS

15. The following recommendations from the evaluation have been agreed upon by the GOE and IFAD.

16. Design. It is recommended that consideration be given to interventions which are far more focused in terms of numbers of beneficiaries to be reached and geographic coverage, within the overall framework of IFAD’s targeting policy. This would ensure greater synergies across activities and ultimately deeper impact on rural poverty. Similarly, the project duration should be long enough to achieve the desired results and in particular take into account the time needed to implement attitude and cultural changes. Project management structures should be kept simple to ensure the integration and harmonization among different implementing agencies.

17. Quality of project delivery. It is recommended that greater attention be given in future project design and implementation to country context issues, and the identification of indicators of quality, and actions necessary to achieve real and lasting impact, alongside those relating to numerical outputs.

18. Policy dialogue. It is recommended that more explicit attempts be made to engage in policy dialogue with Government and other development actors, where appropriate and required involving a wider range of national and international specialists, rather than just IFAD and cooperating institution staff.

19. Participation. It is recommended that future IFAD-funded projects and programmes in Ethiopia pay more attention to people’s participation, especially as it has not been a tradition of development practice in Ethiopia in the past.

20. Integration. In future multi-component projects, it is recommended that greater attention be paid to the linkages between the components and between those agencies responsible for delivering them. The evaluation team is in favour of projects which involve multiple components addressing the diverse needs of target populations – but the difficulties of integrating such efforts should be carefully considered.

21. Management. Management is more than coordination or supervision, and it should be addressed with greater rigour in future projects and programmes particularly in challenging contexts as found in SOCODEP. Projects need to be much more decisively managed. It is recommended that new

xix

approaches be explored, either through IFAD itself taking a more hands-on role during execution, facilitated by the Fund’s field presence officer, which allow closer monitoring and follow-up to implementation.

22. Role of the Field Presence Officer. The field presence officer can, among other tasks, provide implementation support to IFAD-funded operations and has the potential to enhance partnerships and policy dialogue in Ethiopia. Hence, the country presence should be further strengthened, so that it can play a greater role in enhancing IFAD’s development effectiveness in Ethiopia.

xx

Federal Democratic Republic of Ethiopia

Southern Region Cooperatives Development and Credit Project (SOCODEP)

Completion Evaluation

Main Report

I. INTRODUCTION

1. This document is the Evaluation Report prepared following the Completion Evaluation Mission to the Southern Region Cooperatives Development and Credit Project (SOCODEP). The mission was carried out between September 17 th and October 11 th 2006 by a team of four external evaluators 1 accompanied in the field by two officers of the Southern Region Government 2 and assisted in the Regional capital by the former Project Coordinator 3.

Interviewing members of the Argoba cooperative, Gimbo Woreda, Kafa Zone Source: Evaluation Mission 2007

1 Richard Carter, Team Leader; Ayele Gebre-Mariam, Socio-economist; Tsegaye Asfaw, Microfinance Specialist; and Michael Assefa, Cooperatives and Capacity-building Specialist. Andrew Brubaker was the Evaluation Officer from OE responsible for the evaluation. 2 Ato Berhanu Asfaw, Cooperatives Promotion Sector Head; Ato Fekadu Tadesse, Rural Women’s Affairs Team Expert. 3 Ato Shimekit Gebretsadik.

1

II. COUNTRY BACKGROUND

2. Ethiopia is a large (1.1m km 2), very diverse and extremely poor country and both its population density and growth rate are higher than the average for east Africa (Table 1). Ethiopia was rated 170 out of 177 in the Human Development rankings in 2006. There is little incentive for farmers to produce surpluses for distant markets where they will receive very low prices, even in areas where this is possible. Coffee is Ethiopia’s main export, securing the livelihood of 700-800,000 households. In the rural areas, over 32m people live on less than US$0.50 per day. Subsistence agriculture is the main source of income for 87 per cent of rural households. Agricultural productivity and income levels are low and fluctuate due to periodic climatic shocks. Livestock (72 per cent of households own livestock) is the primary asset to cushion weather shocks. Other than climatic variability, the main direct causes of rural poverty are an ineffective and inefficient agricultural marketing system, underdeveloped transport and communication networks, very basic manual production technologies, limited access of rural households to support services, and environmental degradation (soil erosion and deforestation).

3. The fundamental causes of poverty however are political, social and cultural. The political basis of poverty relates to the ways in which power has been exercised by successive Governments – particularly over land and resources, and in relation to individual freedoms, and the consequent failure to empower rural households and communities. The social and cultural aspects of poverty concern the disadvantages which some occupations and most rural women experience in terms of marginalisation and lack of ready access to health care, education and employment opportunities. Female-headed households are more likely to be food insecure than male-headed households. Furthermore, the political and social elements have combined to discourage participation by rural poor people in decisions that affect their livelihoods.

A. Project Background

4. SOCODEP’s origins. SOCODEP was one of the first significant internationally funded interventions in Ethiopia following the fall of the former Marxist-Leninist regime (the Derg) in May 1991. The Project aimed to respond to the then new legislation (Proclamation 85/1994) concerning Cooperatives, which ostensibly set out a means of turning the former Government-imposed and politically-dominated Producer Cooperatives of the Derg into farmer-owned viable business entities serving their members’ interests. In particular, it aimed to make rural finance (specifically micro- credit) available to so-called Service Cooperatives and their members.

5. The Principal Objective of the Project was to “increase agricultural productivity and raise income levels of the rural poor through support to Service Cooperatives’ development in order to facilitate efficient provision of sustainable services to members”. The Project had 7 specific objectives which are set out in Table 2 and further elaborated in the reconstructed logframe in Appendix 4.

6. Together these objectives were designed not only to turn Service Cooperatives (SCs) into effective vehicles for the service of their members, but also to deliver some specific services (road construction, veterinary drug supply, health facility up-grading, and provision of water supply and sanitation services) directly through the relevant Government organs. The Water Supply, Health and Basic Sanitation Component (WSHBS) was separately financed by the Belgian Survival Fund as a later addition to the Project.

2

Table 1. Statistical Data Area and Population Total area 1,127,127 km 2 CIA, 2005, World Fact Population Approx. 74 million Book Population density 65 inhabitants/km 2 Average population density for East Africa 43 inhabitants/km 2 Population growth 2.36 per cent per year Average population growth for East Africa 2.18 per cent per year Number of ethnic groups > 80 Number of dialects Approx. 200 Economy and Agriculture Per cent of population living in rural areas (2003) 84 FAO, 2005, Food and Contribution of agriculture to GDP 42 per cent Agriculture Indicators for Contribution of agriculture to exports 75 per cent Ethiopia Per cent total labour force employed in agriculture 81 per cent Average GDP growth 1993-2003 4.7 per cent The Economist GDP growth in 2005 9 per cent Intelligence Unit, Jan- 2006 – Ethiopia Country Report per cent total land area arable 45 per cent IFAD per cent total land area cultivated 9 per cent FAO, 2005. Food and per cent cultivated land irrigated <2 per cent Agriculture Indicators for Part of crops in agriculture GDP 60 per cent Ethiopia Main crops Teff, maize, sorghum, barley, wheat and pulses Animal husbandry products in agric. GDP 30 per cent Main animal husbandry products Beef and milk Human Development GNI per caput (World Bank Atlas method) US$160 World Bank online data Per cent of population living below the national 44.2per cent and statistics, 2006; poverty line 4 UNDP, 2006, Human Life expectancy at birth 47.8 years Development Report Enrolment in primary school 46 per cent HDI 0.371 HDI ranking 170 th of 177 Administration Semi-autonomous Regions 9, plus 2 city Afar, Amhara, Benishangul-Gumuz, Gambella, Harari, administrations Oromia, Somali, Tigray, Southern Nations, Nationalities and Peoples Regional State (SNNPRS), plus Addis Ababa, Dire Dawa. Zones Varies from 4-12 per Region Woredas (districts) Typically 10-20 Total of about 550 in the early 2000s, but increasing per Zone. rapidly. Kebeles Numerous Based on the former Peasant Associations, areas of approx 800ha.

4 The national poverty line used in Ethiopia is based for 60 per cent value on a food basket providing 2,200kcal per adult equivalent per day and for 40 per cent value on a non-food basket. At 1995/96 prices, the basket cost was estimated at US$ 0.50 per day. The same “basket” and poverty line is used in 1999/00 to maintain comparability between the two survey years. Eighty-two percent of the population has to live on less than one dollar per day.

3

Table 2. SOCODEP Objectives Principal objective: increase agricultural productivity and raise income levels of the rural poor through support to Service Cooperatives’ development in order to facilitate efficient provision of sustainable services to members. SO1 Provide a model for developing Ethiopian cooperatives under the new legislation, particularly with respect to improvement of financial intermediation services in rural areas, which could be replicated in other areas of the country. SO2 Increased capital and income among the rural poor in the project area through off-farm income-generating activities particularly for women and families in densely populated areas with limited land for farm expansion. SO3 Strengthen the SRAB to carry out its mandate with respect to cooperative development. SO4 Provide credit to meet financial requirements for agricultural inputs and draught oxen and facilitate the supply of inputs through support to local traders and cooperatives. SO5 Relief of livestock health constraints, particularly with respect to draught animals, through provision of veterinary drugs. SO6 Improve access of rural families to services and markets by rehabilitating and maintaining rural roads. SO7 (BSF Component) Reduce the burden of disease in 8 woredas of the SOCODEP area Source: Appraisal Report and Approach Paper

7. Project targeting. The special programming mission (1991) distinguished between the more highly populated Woredas on the east side of the project area (Wolayta), where the carrying capacity of the land was thought to have been exceeded, and “the rest of the project area”, which had been neglected previously, but where population pressure was less. Poverty was said to be “… widespread and fairly evenly distributed…”, and consequently it was “… not considered appropriate to further refine definition of the target group within the project area on the basis of poverty criteria”. Although the re-assessment of the nuances of poverty within the project area was not a specific objective of the evaluation, it would appear that this rather general assessment at Appraisal was somewhat simplistic.

8. Project timescale at appraisal. The project was designed to be implemented over a six year period. On this basis, as the loans became effective in 1994, the project completion would have been July 2000, with loan closure in July 2001.

9. The project budget at appraisal (Figure 1). At completion actual expenditure was split approximately 61 per cent from the International Fund for Agricultural Development (IFAD), 21 per cent from GoE, 15 per cent from the Belgian Survival Fund (BSF), two per cent from Commercial Bank of Ethiopia (CBE), and 1 per cent from Beneficiaries.

4

Figure 1. Budget Costs by Component (million US$, total 22.72m) WSHBS Mgt & Training of SCs, Coordination, 0.17 0.95 (4%) Water Supply & (1%) SSE Promotion, San, 2.28 (10%) 0.29 (1%)

Health & Support to SRAB, Nutrition, 1.57 2.80 (12%) (7%)

Roads, 4.70 (21%) Credit line thro Veterinary Drugs, Support to CBE, CBE, 7.68 (34%) 1.89 (8%) 0.39 (2%)

Source: Appraisal Report

10. A Note on terminology. The main form of cooperatives which were established and active under the previous Government (The Derg), between the 1974 revolution which overthrew Emperor Haile Selassie and the fall of the Derg in 1991, were known as Producer Cooperatives. These were instruments of socialism and Government control, and by popular approval they were comprehensively looted and destroyed by their members when the Derg fell. Service Cooperatives (SCs) co-existed with the Producer Cooperatives, but did not have the same political significance, and many continued to function through the political changes and up to the present day. Most SCs are now referred to as Multi-purpose Farmers’ Cooperatives. The legislation (Proclamations 85/1994 and 147/1998) makes no reference to different named types of cooperatives, while allowing a wide range of flexibility in terms of objectives.

11. Components. The achievement of the Project objectives necessitated the implementation of 7 key project components, namely: • restructuring of service cooperatives; • provision of loans to cooperatives and their members; • institutional strengthening/capacity-building of project stakeholders; • setting up a veterinary revolving drug fund; • rural roads construction; • rural water supply; and • improvements to rural health services and sanitation.

12. In section II, the achievements under each of these components, and the degree to which the specific objectives have been met, are addressed in detail. The wider impacts of the project are described and assessed in section III.

13. Project stakeholders. The list of Project stakeholders is long, and complicated by the numerous reorganizations within the Southern Region Government. Figure 2 shows the main stakeholders in a simplified form.

5

Figure 2. Main Project Stakeholders

Regional Regional Agricultural Bureau of Bureau: Health: Health Cooperatives and Sanitation Promotion, co mponents

Cooperatives and their Regional Sodo Rural Members Bureau of Technology Centre: Water Skills Training for Resources: Small scale Enterprise Development. Regional (Rural) Financial Roads Authority: Institutions: Road Construction Commercial Bank and Maintenance of Ethiopia, Omo

Source: Appraisal Report

14. Organizational structure. The planned organizational structure for the project was as follows: the Southern Regional Agricultural Bureau (SRAB) would have overall responsibility for the project. The Project Coordinator would be the Head of SRAB’s Cooperative Promotion Department. A Project Steering Committee would draw its membership from:

• SRAB Chief (Chair); • Project Coordinator (Secretary); • Representative, Ministry of External Economic Cooperation; • Head, SRAB Planning and Project Services; • Head, Rural Credit Department, CBE; • Chief, Regional Finance Bureau; • Head, Regional Bureau for Planning and Economic Development; • Chief, Regional Bureau for Public Works and Urban Development; • Deputy Chief, Field Operations, SRAB; • Heads of participating Zonal Agricultural offices; and • Head, Agricultural Inputs Supply Corporation (AISCO) Regional Coordination Office.

15. Numerous changes in organizational structure took place during implementation. These are outlined in section II.B.

16. Project timescale. With agreed extensions, the original components of SOCODEP (that is, excluding the water, sanitation and health component) ran from September 1994 until July 2002, a total of nearly eight years with the Mid-term Review (MTR) at the end of 1996. Work on the BSF component commenced in November 1998 and continued until Project closure in December 2005, extending the project to 11 years in total.

17. Progress at key steps. The timescale conceals a number of important points related to implementation and performance over the project period, however:

• by the time of the Mid-Term Review– carried out in November-December 1996, report dated March 1997 – a total of 77 Service Cooperatives had been restructured, only EB 4.975m (approx. US$790,000 or ten per cent of the US$7.68m originally allocated) had been disbursed in the form of loans, the veterinary revolving drugs fund had not commenced operation, and no roads had yet been completed. The water supply, health and basic sanitation (BSF) component was still three years away from commencement;

6

• the supervision mission which followed the MTR (October 1997) referred to a situation of crisis and unacceptably low pace and efficiency; a year later the 1998 supervision report, although presented in a more up-beat manner, clearly indicated poor progress in most components; and

• even at the time of the March 2002 supervision visit, although “about 225” service cooperatives had been restructured, still only EB 18.83m (approx US$2.28m) had been disbursed in loans. This represented 42per cent of the reduced credit line of US$5.42m recommended by the MTR. The Women’s Income Generating Activities (WIGA) sub- component had recorded strong performance, as had the Water Supply, Health and Basic Sanitation (WSHBS) BSF project. However, only 74.5km of road rehabilitation and heavy maintenance had been completed – 10 per cent of the appraisal target of 250km rehabilitation and 450km heavy maintenance. Four months from planned loan closure in July 2002, US$10.2m (45per cent of the original budget) remained unspent 5.

B. Objectives and Methodology of the Evaluation

18. Overall objectives. The main objectives of the evaluation were to: (i) assess the performance and impact of the SOCODEP project; and (ii) generate a series of findings and recommendations that would serve IFAD, the Government of the Ethiopia, and other donors in designing and implementing similar projects and programmes in the future. The special focus of the evaluation, as highlighted in the Approach Paper, was to be on Cooperatives Development, Rural Microfinance, Socio-economic impact, and Institutional Capacity-Building.

19. Evaluation methodology. The evaluation followed OE’s guidelines for project evaluations. 6 This included making an assessment of the SOCODEP project according to internationally recognized evaluation criteria, namely: (i) project performance, including relevance, effectiveness and efficiency; (ii) impact on rural poverty; and (iii) performance of partners involved in the project, including IFAD, government institutions, and others.

20. The approach used for the Evaluation Mission therefore was largely a critical review of secondary data, and triangulation through correlation of different data sources, a field visit to the project area, semi-structured interviews, focus groups discussions and comparisons with other donor funded projects namely the IFAD/AfDB-funded the Rural Financial Intermediation Project (RUFIP)7, Netherlands supported the Kafa Development Programme (KDP)8, and the United States Agency for International Development (USAID) supported the Agricultural Cooperatives in Ethiopia Project (ACE)9. Specifically, the evaluation included key informant interviews which were held with all relevant Government organs and other stakeholders at National, Regional, Zonal and Woreda levels. At National level, the Federal Cooperatives Agency, Ministry of Finance and Economic Development, and Commercial Bank of Ethiopia staff were interviewed, as well as personnel from the RUFIP and the Volunteers in Overseas Cooperative Assistance (VOCA) projects. At Regional level, officers from the Bureau of Finance and Economic Development, Bureau of Agriculture and Rural Development, Bureau of Health, Bureau of Water, Mines and Energy, and the Rural Roads Authority were interviewed. Government officials were interviewed in two of the five project Zones (Kafa and the former North Omo – now Gamo Gofa and Wolayta) and one of the two special Woredas (Derashe). A

5 2002 Supervision report, paragraph 27. 6 This included assessing the project against internationally recognized evaluation criteria, namely: (i) project performance, including relevance, effectiveness and efficiency; (ii) impact on rural poverty; and (iii) performance of partners involved in the project, including IFAD, government institutions, and others. 7 Rural Financial Intermediation Project. 8 Kafa Development Programme, formerly Sustainable Poverty Alleviation Kaficho-Shakicho (SUPAK-S) 9 Agricultural Cooperatives in Ethiopia project, implemented since 1999 by ACDI/VOCA (a merger between the NGOs Agricultural Cooperative Development International and Volunteers in Overseas Cooperative Assistance).

7

total of ten of the original 33 project Woredas were visited, and discussions held with Government officials. In each Woreda visited, project-assisted cooperatives were identified and interviews were held with officials and members. In this way a total of 14 cooperatives were visited, and data collected on many more. Whenever possible loan beneficiaries were interviewed, and their number in this evaluation exceeds 20. Four of the six project roads were traveled, and three of the Woredas visited were chosen specifically because of their inclusion of the BSF component.

21. Secondary sources. The evaluation made good use of available secondary sources of data 10 . The standard project documentation (Appraisal, Mid-Term Review, and Supervision Reports) was very valuable, but special note is made of the internal reports carried out in 1999 11 and 2001 12 , which contained some very valuable and critical insights, the significance of which appears to have been somewhat overlooked at the time. The recently completed and extremely thorough external Project Completion Report (June 2006) was of great value too, especially for its data on Project achievements. Our assessment ratings have however frequently diverged from those of the external Project Completion Report (PCR), concurring more closely with those of the MTR, Supervision Reports, and internal project reports. This matter is elaborated further in section II.D of this report.

22. Rating scales used. In section III of this report the assessment ratings of project and partner performance and impact follow IFAD’s 6-point rating scale, with scores of 6 representing the highest achievable, and 1 the lowest. The verbal descriptors of the rating scale scores are those set out in the December 2005 edition of the IFAD Office of Evaluation’s (OE) Evaluation Manual 13 .

III. PROJECT PERFORMANCE

A. Design Features

23. Country context. Again Ethiopia is an extremely poor country and ranks 170 out 177 on the Human development rankings. Ethiopia’s size, diversity, poor infrastructure and poverty pose numerous development challenges. The worsening trends in population pressure, environmental degradation, communicable disease and climate instability add to these challenges. Politically, Ethiopia has undergone a virtually continuous process of change since the 1974 revolution in which Emperor Haile Selassie was overthrown. The Marxist rule of the Derg from 1974 to 1991 led to many reforms, but the authoritarian use of power over the peasantry did not change, or if anything, it became more restrictive. After the fall of the Derg in May 1991, a slow process of liberalisation of economy and political systems has taken place, but there is still a long way to go before the rural population will be fully able to participate in anything approaching a free market and transparent democratic processes of government. Thus there is no doubt that Ethiopia is a very difficult country in which to make progress in terms of development and poverty alleviation.

24. Project design context. Most if not all of the development challenges posed by Ethiopia’s context were known in the early 1990s, and recognised in the Appraisal Report. However, in retrospect, there was an unrealistic optimism about the rate at which beneficial change (e.g. to the SCs) could take place, and an implicit assumption that the Government institutions involved were fully committed to that change process.

10 The evaluation notes the limited range of reports and other documents available through the project and partners due to the frequent moving of the project office which made it challenging to find many documents. However, the core documents were available and provided a sufficient source of secondary data for the evaluation. 11 Comprehensive Analysis of Current Situation and Requirements of Cooperatives in SOCODEP Areas, April 1999; Brief Summary of Comprehensive Study on the Small Scale Enterprise and Income Generating Activities, May 1999. 12 A Baseline Survey Report on Small Scale Enterprises in SNNPR, September 2001. 13 IFAD Draft Evaluation Manual. Office of Evaluation, December 2005, and subsequent enhancements.

8

25. The Project correctly identified some of the key difficulties of the Region’s farming households, namely their inability to access credit and some key services, and poor infrastructure. At the time of design the Project was imaginative in responding to the new Cooperatives legislation which promised a significant liberalization of the sector and it included six relevant but only loosely integrated components. However the project design was flawed in a number of important respects, which are explained in the following paragraphs.

26. The Project area was too large. With its centre in Awassa, the Regional capital, the most distant parts of the project area were at least two days’ drive away, and often inaccessible in the rainy season which stretches from April to September (the western parts of the project area are some of the wettest in Ethiopia) 14 . The sheer size of the project area diluted the impacts of nearly all the project activities, and helped to undermine sustainability. The Project had a scatter-gun effect, delivering significant benefits to a relatively small number of individuals (a few tens of thousands) within a much larger population (4-6m population of the project Woredas, out of the Region’s 14.5m 15 people).

27. The Project duration at design was too short. It was entirely predictable that a new Regional authority, with no experience of handling projects financed by external donors or IFIs, would take considerable time to get up to speed. The fact that relatively little was achieved between project commencement in 1994 and the MTR in 1996 is testimony to this. Similar experiences were observed in the Special Country Programme, evaluated in 200416 . Furthermore, there was no effective start-up phase during which personnel were effectively oriented to the goals and modus operandi of the Project.

28. The design was over-optimistic about attitude change. The design exhibited too optimistic a view of the speed with which the former model of cooperatives – instruments of coercion and control by the former Government – could be turned around into a member-owned and member-controlled viable business model. Such a change in mind-set, understanding, attitude and practice was never going to take place quickly. Even now, 12 years after the first reforming legislation, there is a long way to go.

29. The Project was never fully owned by Southern Region Government. It is understood that the negotiations which gave rise to SOCODEP took place between the Federal Government and IFAD, and that the Southern Region Government was not involved in the process. Furthermore, it was explained to the evaluation team by senior Government officials that external project financing has correspondingly reduced Federal and Regional allocations of funding which otherwise would have been spent at the discretion of the Regional bureaux. These two factors have contributed to a lack of ownership by the Region. These issues should have been understood and taken into account at the design stage.

30. The partnerships were too weak given the project complexity. At design, the project was to be managed from one Government Bureau (SRAB), with major inputs from two other stakeholder institutions (CBE and the Bureau of Public Works and Urban Development (BPWUD) – later to be the Roads Authority). The Rural Technology Centre at Sodo was to be involved in the pilot development of small-scale enterprises. The addition of the BSF component brought in three other Bureaux (Water, Health, Finance and Economic Development). When CBE withdrew, the Omo Micro Finance Institution (OMFI) took its place. Project complexity grew, but project ownership and coordination, and the ability to manage a multi-sectoral project did not.

31. The Project had no single logical framework and its many components and stakeholders were not well integrated. This was a multi-component, multi-stakeholder project, but its parts and its actors

14 Even in Kaffa Zone, a relatively accessible part of the western side of the project area, one day’s drive from Awassa, only 4 out of the 10 Woredas were accessible at the time of the evaluation (September-October 2006). Some parts of the Zone lie up to 3 days’ walk from the nearest road. 15 Population estimate for 2005. 16 http://www.ifad.org/evaluation/public_html/eksyst/doc/country/pf/ethiopia.pdf

9

were never fully harmonized and integrated. The logical framework used in this evaluation, and a corresponding version used in the external PCR, had to be reconstructed from the narrative descriptions in the Appraisal documents and MTR Report. The lack of integration 17 of project components was in large part (but not only) due to the constant restructuring of Government organs and the consequent redeployment of personnel, a feature of Ethiopia’s Government over at least the last two to three decades.

32. Implementation mechanisms set out at design were bound to weaken effectiveness and impact. The design was unrealistic about the mechanisms of implementation. Even without the numerous re- organisations of Government offices and the constant reshuffling of staff which have characterized the Derg and post-Derg period, the assumption that the Government staff at Regional, Zonal and Woreda level would effectively implement a new project according to a set of externally introduced ideas and expectations – including western free-market ideas of business and of cooperative development - was over-optimistic. The Appraisal Report foresaw this: “Cooperatives Promotion Department (CPD) staff lack practical business promotion, management and marketing know-how. Their training has been for cooperatives under a socialistic system, which is no longer relevant in Ethiopia, and major reorientation is required in the context of operations under a market economy”. In reality this major reorientation never took place.

33. In relation to specific project components, the selection of the CBE as the vehicle for transferring credit to service cooperatives was a mistake in hindsight. The Development Bank of Ethiopia (DBE) then known as the Agricultural and Industrial Development Bank (AIDB) was an alternative at the time, having greater experience of providing rural credit services than CBE. The design of the Veterinary Revolving Drug Fund (VRDF) was extremely cumbersome, as evidenced by its very delayed commencement (the first drugs being acquired in 2001, 7 years after the Project began). The inclusion of a rural roads component was a strong design feature, although the assumption that service cooperative members would maintain the new roads was flawed. The later decision to upgrade the road design standard (from RR 30 to RR 50), with corresponding implications for construction costs (a 5-fold increase per km) and maintenance (non-suitability for labour-based maintenance) produced good quality roads; however, it limited the reach of this project component. Water supply was deliberately omitted at the design stage, then a few years later introduced through the BSF component. The BSF intervention, while much more focused (in only 8 woredas rather than the original 33 of SOCODEP as a whole), in effect represented a separate project, having little relation to the rest of SOCODEP. Overall, SOCODEP was a multi-component, multi-stakeholder, project, which began with a focus on Cooperatives and Credit and the corresponding institutional Capacity- Building, but which, by adding Veterinary Drugs, Roads, and later Water, Health and Sanitation, still failed to become an integrated project. A notable omission from the project design was that of markets – for agricultural inputs and outputs, and the products of small-scale enterprises.

B. Implementation and Outputs

34. Project achievements. Table 3 summarizes the main project numerical targets and achievements, which are discussed in the following paragraphs.

35. Institutional structure during implementation. Reference has already been made to the changes in institutional structure, which took place during the project. The main changes are highlighted here, in order to place the project achievements in context:

• The project as described at Appraisal was to be managed by the Head of the Cooperatives Promotion Department of SRAB (as Project Coordinator), and guided by a Project Steering Committee (PSC);

• Over the course of the project, the post of Project Coordinator was held by four different individuals, each serving for 2-3 years, so interrupting continuity. Successive Supervision

17 Noted also in the 2002 Supervision Report.

10

missions reported on the weaknesses of the project coordination unit and ineffective guidance by the project steering committee;

• For the first two years of the project a liaison office in Addis Ababa served the main project coordination office in Awassa. This was closed in 1997, against the better judgment of the United Nations Office for Project Services (UNOPS)18 ;

• The organ of the Southern Region Government which was central to the functioning of the project, namely the Cooperatives Promotion Department started out as a department of SRAB, then became a full Bureau, and subsequently became a department of the Regional Bureau of Agriculture and Rural Development. These re-organizations inevitably affected the effectiveness of this organ of Government;

• After the MTR, and with the commencement of the BSF component (which involved two more Bureaux of Regional Government), the PCU migrated from the Bureau of Agriculture and Rural Development (BOARD) to the Bureau of Finance and Economic Development (which had been recently created from the two former separate Bureaux of Finance and Planning). This move was probably necessary because of the widening of SOCODEP’s activities, but it caused some understandable resentment in the Cooperatives Promotion Office; and

• CBE, as the channel for disbursement of credit to the project cooperatives, soon found itself in difficulties, recording this in an internal report dated January 2001. By July 2002 CBE terminated all loan disbursements to cooperatives.

36. The frequent changes to the institutional structure of the project, the weak capacity of some players, the limited real authority of the Project Coordinators, and the influence of those with greater power, provides the context within which the (limited) achievements of the project should be seen.

18 UNOPS Supervision Report December 1997, paragraph 12.

11

Table 3. SOCODEP Numerical Targets and Achievements Appraisal Revised Target Percentage Percentage Aspect Achieved Target (at MTR) of MTR of Appraisal Cooperatives Coops restructured (No.) 200 150 267 178 134 Credit Individual loans for inputs 60000 6,219 10 WIGA loans 10000 7,600 76 Ox loans 50000 14,579 29 Flour mills & maize shellers 170 23 14 Coops with loans for Marketing 75 98 131 Coops with loans for Stores 75 0 Coops with loans for Shops 200 66 33 Credit disbursed (US$m) 7.68 Reduce by US$1.4m 2.96 43 39 Small-scale Enterprise Development Vocational Training (trainees) 215 316 147 Jobs created 2,100 Negligible Loans for SSEs (EB) 2,200,000 55,456 3 Veterinary Revolving Drugs Fund Veterinary drugs (US$m) 1.89 Increase by US$0.58m 0.29 12 15 Roads Rehab to RR 30 standard (km) 250 Reduced to 125km 122 98 49 Heavy maintenance (km) 450 0 0 Health and Sanitation Health facilities upgraded 8 N/A 8 100 Human RDF locations 8 N/A 8 100 Household latrines 1,000 N/A 2,320 232 Household refuse pits 1,000 N/A 269 27 Water Supply Waterpoints - new and rehab 164 N/A 171 104 * Source: Appraisal Report, MTR, internal PCR, external PCR

C. Restructuring of Service Cooperatives

37. Restructuring of Service Cooperatives (now known as Multi-Purpose Farmers’ Cooperatives). At Appraisal it was recognised that the Cooperatives Promotion Division of SRAB had experience of supporting cooperatives under a socialist economy, but that there was no experience of supporting cooperatives established “…in accordance with international principles of cooperation”. It was further recognised that “…even before disturbances at the time of the change in Government, which resulted in looting and destruction of some cooperatives’ property, SCs were not operating efficiently”. SOCODEP therefore aimed to turn SCs into financially viable autonomous organisations which would provide sustainable services in the future. The Cooperative Development component of the Project had three sub-components, namely training, small-scale enterprise promotion, and institutional support. SOCODEP would train and equip Government Cooperatives staff, appoint professional managers to SCs, train Cooperatives officers and members.

38. The Project more than met its numerical targets in terms of re-organising SCs under the new legislation. The Appraisal target of 200 cooperatives restructured 19 was exceeded by about 30 per cent. A great deal of training of SC staff, executive committees and members was also carried out 20 . It is estimated 21 that almost 57,000 individuals in Government or cooperatives received some form of

19 This target was reduced by the MTR from 200 to 150. 20 Project Completion Report, June 2006, Annex II, Table 2 reproduced as Appendix 2 of this Report. 21 External PCR, June 2006, Annex II, Table 2.

12

training, varying in duration from one day to two years, and much of the shorter-term training was targeted at cooperative officers and members.

39. In terms of achievement, it is clear that although the project activities resulted in significant numerical outputs, the quality of those outputs (the extent of cooperative re-orientation and the enhancement of business competence) could have been much higher. Comparison of SOCODEP activities with those of the USAID-supported VOCA/ACE project suggest that an intensive programme of well-designed and structured training by very experienced individuals, with sound monitoring and follow-up, was needed to achieve the project goals.

D. Provision of Loans to Cooperatives and their Members

40. Credit. Loans which were initially channelled through CBE (until the withdrawal of this Bank in 2002), and later through Omo Microfinance Institution (OMFI), were used for two main purposes: (a) financing the activities of the Cooperatives themselves (consumer shops, flour mills, purchase and sale of agricultural produce), and (b) individual loans to cooperative members (for purchase of work oxen, purchase of sheep and goats for fattening or rearing, and investment in small-scale enterprises). Because of the weak business skills of many of the cooperatives, the first of these have been largely unsuccessful (except in the coffee producing areas, where marketing of coffee has been lucrative), while many of the individual loans supplied to cooperative members have made a real difference to the lives of the recipients (see sections II.C and III).

41. Loan disbursement and recovery for cooperative activities (flour mills, consumer shops and produce marketing) have generally fallen short of target. Initially, the rehabilitation and installation of 160 flour mills was planned, but only 13 new flour mills were installed and 10 were rehabilitated at a total cost of EB 1.1m. The flour mills performed poorly and the recovery rate of the loans was also very poor at 47 per cent. The operations of most flour mills were discontinued because of poor maintenance and stiff competition from private flour mill operators. A total of 66 loans were provided to SCs for operating consumer shops. The total disbursement was EB 2.1m, i.e. 53 per cent of the EB 4.0m initially budgeted. Although the repayment of the loans was reported to be 93 per cent, at present most consumer shops are not operational due to lack of maintenance and competition from the private sector. The fact that cooperatives cannot generally compete with the private sector in flour mill operation and consumer shops is evidence of their lack of business-orientation. In respect to produce marketing service, 98 SCs received a total loan of EB 2.45m, 130 per cent of the Appraisal target, and the loan repayment was 77 per cent.

42. Loan disbursement for individual activities of SC members generally did better than those for cooperative activities, although repayment rates were often low. In particular 7,600 loans to women for fattening and multiplication of small ruminants, petty trading and other activities were made, and in their case, repayment rates were high, especially after Omo Microfinance took over from CBE. Box 1 sets out two case studies of women beneficiaries, both of whom benefited from loans supplied through small group membership along the pattern established by OMFI. These illustrate in a positive way what is possible.

43. An observation made during the evaluation was that the loan repayment culture differs significantly between the west and east of the project area. In the west, individuals take great pride in repaying loans, and they take great care not to risk going to their graves in debt. In the east and south- east of the project area however this culture does not appear to be so strong, and defaulting is more prevalent 22 .

22 The reasons for this difference in repayment culture are not clear, but the evaluation team was not the first to observe it.

13

Box 1. Two Examples from Individual Beneficiaries of OMFI Loans Lateshe Lache, aged 35, a house wife with six children, with 9th grade education, was a member of Sake SC in Woyde Woreda in Wolayta Zone. She became a member of one of the groups organized in the area by the OMFI. Subsequently, she was provided with a loan of EB 500 by OMFI through the Sake SC to carry out butter trading. She said the loan of the first cycle was properly utilized for the intended purpose and she made a profit of EB 300. Upon settling her debts, she was again provided with a second cycle loan of EB 500 for the same type of petty trading activity and managed to make a profit of EB 360 and fully settled all her debts to OMFI. With a continuous success, she was granted a third cycle loan of EB 1,000 and reportedly made a profit of EB 700. Subsequently she repaid all her debts to OMFI. However, as her business continued to expand, she borrowed for a fourth cycle loan amount of EB 4,000 to carry out and expand her trade activities, mainly the butter and grain trading activities. Lateshe was extremely happy about her trade business activities and, as a result, she was highly optimistic for further success in her trade. She said she has a house worth EB 15,000 and other assets. Bezunesh Mena, aged 30, a house wife with two children, became a member of Wachiga SC in Sodo Zuria Woreda in Wolayta Zone. Subsequently, she joined a group organized by OMFI and got access to a loan of EB 250. She utilized the loan for grain and butter trading activities and made profits. After one year, she fully repaid her loan, and with the profits she established a small bakery., while continuing her butter and grain trading activities. Bezunesh has carried out her business activities for the last ten years, and now she own a house worth EB 10, 000 plus 3 cows and a pair of draught oxen. She also works as a family planning agent in her community and gets EB 75 per month, working two days per week. However, she spends most of her time attending her usual business activities. * Source: The Evaluation

44. The total disbursement of the credit component of the project was generally low, both for cooperative and individual member activities. Table 4 illustrates this for the individual member activities. Overall, EB 20.7m (approx US$2.96m) was disbursed to and through SCs, representing about 39 per cent of the target at Appraisal.

45. The project failed to provide a wider range of microfinance services other than credit and ox insurance. Although SOCODEP correctly identified lack of access to credit as a limiting factor for farmers and cooperatives, and although insurance was a condition of ox loans, other financial services – especially savings - do not seem to have been considered for inclusion, and the ox insurance programme did not serve its clientele well 23 .

Table 4. Loans Made Through SOCODEP for Individual Activities Purpose of loan Target at Appraisal Achievement Comments 60,000 individual Total loan utilization 22 per cent achievement in terms of credit beneficiaries of input Input supply EB 2.01m, reaching disbursed; 10 per cent in terms of credit at average of 6,219 SC members. beneficiaries. Recovery rate 68 per cent. EB 150 per farmer. Average loan EB 300 (approx US$45). 7,600 women Loans to 10,000 Total disbursed EB 2.2m (approx US$ WIGA recipients (76 per women. 320,000). Recovery rates 79 per cent cent of target). (CBE) and 99.8 per cent (OMF). 170 enterprises; Only EB 55,456 (approx US$8,000) SSE negligible 2,100 jobs created. disbursed during entire project period. Loan recovery rate 68 per cent; high 50,000 oxen 14,579 oxen (29 per Draught oxen mortality (14 per cent); low payout from purchased. cent of target). insurance scheme. * Source: PCR and Evaluation

23 The ox insurance scheme was not popular with loan beneficiaries, because it represented an additional financial burden, and it only paid out in a minority of cases of mortality. This was most likely because of poor administration of the claim execution process, in which the death of an ox had to be reported to the Cooperative, cause of death verified by a Veterinary Officer, and completed claim forms delivered to the offices of the Ethiopian Insurance Corporation.

14

Institutional Strengthening/Capacity-Building of Project Stakeholders

46. Institutional capacity-building. If institutional capacity-building is generally taken as including changes to the policy environment, organizational reform, putting in place management systems, technical and management training, provision of physical resources, and changing of attitudes, then SOCODEP only partially addressed the matter of capacity-building of Government and other stakeholder institutions. SOCODEP delivered a great deal of training (see Appendix 2) and physical resources, to Regional and lower levels of Government, as well as to CBE, OMFI, and SC personnel and members.

47. Training. In order to strengthen institutional capacity, 23 international and 39 domestic trainings were delivered to 56,912 trainees from partner bureaux at all levels and SCs executive committee, staff and members between 1998 and 2005. Eight of the distance learning students from the Bureau of Finance and Economic Development (BOFED) are still following their MSc study in Financial Management from UK in 2006, four years after the main project loans were closed.

48. At the time of Appraisal, the profound needs for training and re-orientation were clear: “… Cooperative Promotion Department (CPD) staff lack practical business promotion, management and marketing know-how. Their training has been for cooperatives under a socialistic system, which is no longer relevant in Ethiopia, and major reorientation is required in the context of operations under a market economy”. Two years later the mid-term review of the project said that: “…the basic restructuring and the breaking up of the CPD into different teams requires redesigning the training programme conceived at appraisal. A comprehensive cooperative promotion and training programme for the next four years is considered to be necessary and urgent. It will replace all training that was proposed for the remaining four years at appraisal and has been designed to ‘restart from scratch’”.

49. Physical resources. The stakeholder organisations received a good deal of physical equipment 24 including 65 cars (double cabin pick-ups and land cruiser station wagons) and 3 buses, 300 motorbikes, two newly built offices, computers, office furniture, 6-communication radios, and different types of audiovisual equipment. Road building equipment including: 9 dump trucks, 3 bulldozers, 3 graders, 2 loaders, 2 rollers, many spare parts and other supplies were provided by the project. But during this evaluation mission, and as has been the case during the Supervision missions, the destination of some of the project vehicles was impossible to trace. It was also confirmed by the Supervision missions 25 and during the evaluation mission that vehicles and other equipment were not always used for the intended purposes.

Setting Up a Veterinary Revolving Drug Fund

50. Veterinary Revolving Drugs Fund (VRDF). The Veterinary RDF was implemented late in the project, with the first drugs procurement in 2001. Discussions at Woreda level confirmed that the acquisition of SOCODEP drugs made a significant contribution to the ability of Government Veterinary personnel to provide treatments, but, as the PCR points out, the procurement process was initially very cumbersome. Now that the RDF is dwindling, supplies to Woredas are reducing, and the incidence of animal disease is increasing again.

51. Aim of the component. SOCODEP was not directly concerned with livestock production, but a considerable amount of credit was provided for work oxen, and to women fattening small ruminants and cattle. The objective of the VRDF therefore was “to provide limited relief of livestock health constraints, particularly with respect to draught animals, through provision of veterinary drugs”. This was intended to improve animal health in the project area, verified by reduced mortality rates and incidence of diseases. The sub-component was intended to back-up the credit activities associated with the supply of work oxen and women’s income generating activities involving cattle and sheep fattening. The project was to finance the importation of a limited supply of veterinary drugs and

24 Information obtained from PCU. 25 eg the 2003 Supervision Report, paragraph 30.

15

allowances for veterinary assistance in the project area. The supply of drugs was expected to operate on a revolving fund basis with the regional veterinarian advising on drugs to be purchased. At appraisal, it was proposed that the funds allocated for this purpose would be held by SRAB with the National Bank of Ethiopia. The sale and purchase of drugs was to be performed through the Zonal and Woreda veterinary clinics. All drugs were to be sold to stockowners on a cash basis with a 25 per cent mark-up. Funds collected from the sales were expected to be reconverted into foreign currency and deposited in the National Bank of Ethiopia (NBE) account and used for future drug purchase.

52. Achievements. T he operational modalities for the revolving fund were put in place in 1999. The Regional BOARD received 6 vehicles and motorcycles to support the WIGA livestock fattening activities and the administration of the fund. At Zonal and Woreda levels 140 veterinarians and assistant veterinarians, and 300 veterinary technicians received training on the operation of the revolving fund. Appropriate receipt and ordering forms for drugs were designed, printed and distributed at field level. After drug purchase and distribution, follow up field visits were conducted to check the sales and reporting system. According to the veterinary reports 313,140 animals have been treated for trypanosomiasis; more than 20,000 animals were treated for various infectious diseases; 450,000 sheep and goats have been treated for internal parasites; about 500,000 animals have been treated for external parasites; and more than 10,000 animals have been treated for other diseases. 26

53. Disbursement. A total of EB 10.7m (US$1.89m) was budgeted for the component. Actual utilization was only EB 2.1m (19 per cent of allocated fund).

Rural Roads Construction

54. Rural roads construction. A total of 6 roads were constructed through SOCODEP, to a higher standard than originally assumed (RR 50 rather than RR 30), and at a consequently much higher unit cost. This meant that the Appraisal targets had to be revised downwards at MTR. However, despite the relatively small amount of construction and its scattered nature, the quality of at least the four roads traveled in this evaluation is good 27 , and the impact on those now able to use them 28 is significant. Furthermore, the equipment provided to the Roads Authority has allowed it to continue construction elsewhere, and this is expected to continue for some time to come.

55. A total of 122km of rural road construction was carried out. Table 5 shows the achievements of this component. Unit costs varied widely, from approximately EB 36,000 (US$4,200) to EB 170,000 (US$20,000) per km.

56. Overall, the roads component, like many of the other components described so far, resulted in useful achievements, but which fell far short of those anticipated at Appraisal. The main reason for this was the raising of the design standards and hence unit costs of the roads constructed. Furthermore, while it is acknowledged that SOCODEP had little influence on roads policy, a greater effort should have been made in pursuing a labour-based maintenance strategy which would have been more appropriate for the benefiting communities.

26 SOCODEP, undated, Veterinary Component Project Activity report. 27 As good, that is, as other non-SOCODEP roads travelled. However, gullying and erosion problems are evident, and maintenance is needed. 28 Mostly on foot.

16

Table 5. Rural Roads Constructed by SOCODEP (in km) Total Year Year Cost Total Road name length started completed (EB) beneficiaries 29 (km) Gazer – Tolta 22.0 2001 2004 3,343,078 29,661 Arba Minch – 21.5 2001 2004 3,657,797 38,900 Zigiti Chiri - Udadish 22.5 2001 2004 1,916,711 8,920 Shewa - Bench- 12.0 2002 1998 1,499,238 46,000 Maji Gojeb - Argoba 10.0 1997 1997 357,518 5,373 Wajifo - Boreda 34.0 <1997 1997 2,732,000 99,457 Total 122.0 13,506,342 228,311 * Source: SRAB Planning Department

The Udadish-Chiri Road, Decha Woreda, Kafa Zone Source: Evaluation Mission 2007

Rural Water Supply, Improvements to Rural Health Services and Sanitation

57. Water supply. The addition of the BSF component in 1999 introduced water supply, health and basic sanitation to SOCODEP. However, since in reality water supply is dealt with by one Government office and health and sanitation by another, they are treated separately here. As with rural roads, the provision of water supply physical infrastructure has been carried out to a generally adequate standard (with some reservations). Access to safe water has improved for somewhere between 60,000 and 187,000 people 30 . However, the “soft infrastructure” (community participation, health, sanitation and hygiene promotion, support for operation and maintenance) has been very weak, leading to real doubts about the long-term functional sustainability of most of the water supply improvements (see section III).

58. New and rehabilitated water points. Nineteen hand dug wells and 46 shallow boreholes were constructed, and 82 shallow wells were rehabilitated. In addition 17 new springs were developed, and protected, with associated distribution points, and 7 springs were rehabilitated. SOCODEP also

29 These numbers are based on the total populations of the kebeles lying within 5km of the road. A more realistic estimate is based on the numbers of people living within 5-10km of a road. Available population density figures would suggest that total beneficiary numbers are probably about half those quoted in the table. 30 The estimate of 187,000 in the PCR is probably an over-estimate, and the functionality of many of the newly constructed and rehabilitated sources is far from guaranteed.

17

constructed two motorized water schemes from spring sources and one motorized scheme from a shallow borehole 31 . In total 171 new or rehabilitated water points were constructed (as against target at Appraisal of 164).

59. The quality of construction of the water points seen in the evaluation was generally adequate, but the water points were not well separated from livestock and drainage was poor. Upkeep and maintenance of water points betrayed weaknesses in community ownership and management, issues which are explored further in section III.

60. Training of local artisans and the community to construct and operate water facilities. A total of 311 members of Community Water Committees received training in the management of water points, financial management and environmental hygiene, sanitation and health. In year 2000 two water officers, 5 community organizers, 13 water committees and 15 water scheme operators were trained in water supply management. In the same year, 175 water technicians and Water Committee members received training. In 2003, 14 Woreda technicians were trained in the design and construction of water supply systems 32 . The existing technical manual for the operation and maintenance of water supply and sanitation facilities was simplified and made available to caretakers of water points. At least two women were included in each Community Water Committee and every participating village was represented at a community-training workshop.

61. Overall, the water supply component has the potential to deliver significant benefits to the target populations. Improved access to better quality domestic water can lead to significant time savings for women in particular, and it can contribute to improved health. However, it is widely recognised that significant health impacts are only achieved through an integrated approach to water supply, sanitation and hygiene promotion. Because of weaknesses in the integration of these three aspects, and because of the limited emphasis on hygiene promotion, the evaluation team had serious concerns about impact in this area (on health, in particular). A Public Tapstand in the Yakima Gravity Scheme, Furthermore, concerns about functional Ofa Woreda, Wolayta Zone sustainability of water supply infrastructure Source: Evaluation Mission 2007 undermine the limited impacts achieved.

62. Health and sanitation. The construction, rehabilitation and equipping of health facilities, and the training of staff and community members have no doubt contributed significantly to the improvement of the health condition of local populations. There has however been little integration between the activities focused on delivery of health services, and the supposedly health-targeted activities of sanitation and water supply. In the case of sanitation, the reported high levels of latrine construction 33 and ownership 34 may have more to do with the Government’s push through the Woreda and kebele councils to increase coverage quickly, than with the BSF component itself. Some Woreda officials admit privately that the top-down approach of Government may well be increasing numbers of latrines, but that usage is another question.

31 IFAD, 2006. SOCODEP Project Completion Report. 32 IFAD, 2006. SOCODEP Project Completion Report. 33 Using local materials rather than concrete sanplats. 34 75 per cent in the BSF woredas, according to the BSF Impact Assessment.

18

63. Health centres and equipment. The health centers at Sawla and Karat were rehabilitated in addition to a new construction of Health Station at Mure in Offa Woreda. Medical equipment ranging from basic weighing scales to operating tables was supplied to 3 Health Centres and 2 Health Stations in order to promote improved health care delivery. Non-medical equipment was also supplied to the health facilities and to three Zonal health offices. Revolving Drug Funds were set up in 8 health facilities.

64. Transport and logistical support. Two twin cab pick-ups and 22 motorcycles were supplied and added to the pool of transport, which distributes drugs and vaccines and is used for the supervision of health services.

65. Training. Under the health and nutrition sub-component was expected to cover (i) training of qualified physicians, nurses and anesthetists in emergency surgery; (ii) training of 24 trainers to provide in service training for health workers; (iii) training of 80 health workers in health management; (iv) training of 80 health workers and 80 traditional birth attendants; (v) training for 12 nutrition officers; (vi) training for a Monitoring and Evaluation Officer on participatory Monitoring and Evaluation (M&E) methodologies; and (vii) training for health and administrative staff on prescriptions and use of essential drugs. 35

66. Results achieved in training health department staff. Nine clinical staff were trained in anesthesiology; 18 health staff were trained as trainers and 77 were trained in health management. Nutrition training was provided to 26 nutrition officers and 18 general health staff. Refresher training was provided in aspects of health service delivery to 86 health workers and 50 sanitation officers received training on latrine installation and community health activities. A total of 92 environmental health officers attended refresher training in environmental sanitation, financial management, and the operation of the drug revolving fund. In addition, the focal person in the Regional Health Bureau attended a degree course in community health for one year in planning and community organization. 36

67. Results achieved in training the communities. A total of 43 village people were trained to become Primary Health workers and 21 were trained to become community health agents. An additional 190 existing Community Health Workers and 42 Traditional Birth Attendants received refresher training. More than 760 members of the Kebele Health and Sanitation Committees were trained in basic health and sanitation. The community based voluntary workers have become community educators, mobilizing the community to promote health and basic sanitation in the villages. 37

68. Basic sanitation sub-component. A total of 227 demonstration latrines were constructed in public places such as schools, health facilities, SCs and Kebeles. 2,320 household latrines were constructed. 68 demonstration public refuse pits and 269 household refuse pits were constructed. Members of the Health and Sanitation Committees (HSC) carried out campaigns in promoting sanitation. 38

69. Overall, the health and sanitation component has delivered significant improvements in health facilities, training and equipment in the 8 BSF project Woredas. However, long-term impact and sustainability of these interventions are a matter of concern, as all these services need on-going investment and support.

70. SOCODEP achievements. Figure 3 summarizes 20 numerical achievements of SOCODEP as compared to the targets set at Appraisal.

35 FAD, 2006. SOCODEP Project Completion Report. 36 IFAD, 2006. SOCODEP Project Completion Report 37 IFAD, 2006. SOCODEP Project Completion Report 38 Opp.Cit p.27

19

Figure 3. SOCODEP: Quantitative Achievements against Targets at Appraisal 250

200

150

100

Appraisal Target Appraisal 50 Achievement as % of of % Achievementas 0 ) (EB) Dm) ns (No.) rketing Shops dard (km) ld latrines ans for Ainputs loans (No.)aize shellersns forns Stores forrsed (USDmJobs created o Ox loa lities upgraded structuredl (No.) aintenancei (km) WIG m an RDF locations R30 stan Househosehold refuse pits ps re Loans for SSEs Water supply points Hum Hou Coo Individual Credit disbu Veterinary drugsHeavyHealth (US fac Flour mills &Coops m Coops with loa with loa ocational Training (individuals)Rehab to R Coops with loans for MaV * Source: Appraisal Report, PCRs, Supervision Reports, and the Evaluation

E. Attaining Project Objectives

71. Specific objective 1. Provide a model for developing Ethiopian cooperatives under the new legislation, particularly with respect to improvement of financial intermediation services in rural areas, which could be replicated in other areas of the country.

72. The need for intensive re-orientation over several years, and the very mixed performance of cooperatives in reality, puts in question the achievement of this objective. Because of weak management and design flaws, the project failed to provide a replicable model. Even with more intensive and concerted re-orientation, this evaluation team questions the appropriateness of the cooperative model, except in situations where cooperatives are demonstrably advantageous, or where there is no alternative. Cooperatives as businesses retain many of the features of Government-imposed institutions, namely salaries, per diems, and a bureaucratic mentality. Consequently they often fail to compete with the private sector.

73. At the end of the SOCODEP intervention, Cooperatives are weak. Working Paper 1 Table 1 presents a long list of weaknesses and causes of those weaknesses of 7 cooperatives assessed in the evaluation. These include inability to compete with the private sector; misappropriation of funds; low morale; lack of trust between members and cooperative officers; lack of management competence; lack of full-time managers; and lack of on-going support from the Cooperatives office. Figure 4 summarizes this situation visually. During the evaluation, the opportunity was taken to compare the approaches of SOCODEP and VOCA’s ACE programme in regard to the strengthening of cooperatives. VOCA’s approach is more intensive, more organized, well monitored, and consequently more effective in building the capacity of its target cooperatives. It should be noted however that VOCA has been more selective in its choice of cooperatives than SOCODEP, focusing on merit rather than need, and picking the ‘low-hanging fruit’ rather than taking the less discriminating approach of SOCODEP. VOCA did not provide loans to cooperatives, unlike SOCODEP. Discussions with the RUFIP programme revealed that, just as with SOCODEP, it is relatively easy to make quantitative progress in terms of establishing or re-registering cooperatives, but to deliver well-planned and effective training is much more time-consuming and demanding.

20

Figure 4. Cooperatives: Findings from the Evaluation Low confidence of membership, limited participation by members, poor loan recovery

Assets looted High maintenance following change costs of flour mills in Government Lack of access to wholesalers No full-time manager Internal External factors factors Strong competition from private sector Poor governance

Misappropriation Difficulty of of funds accessing credit Weak business and Lack of capital Other external management factors (climatic capacity and price shocks)

Ineffective restructuring and lack of follow-up support

* Source: The Evaluation

74. In light of the present-day performance of cooperatives, the quality of restructuring and re- orientation of cooperatives delivered by SOCODEP has to be questioned. In the eastern and southern parts of the project area (Wolayta, Gamo Gofa, and the special Woreda of Derashe visited in this evaluation), many of the SCs are de facto bankrupt, and failing to provide useful services to their members. In ventures where they compete with the private sector (consumer shops, flour mills) their overheads exceed the combined profits and overheads of their private sector competitors – hardly testimony to the business orientation of the cooperatives. In the west of the project area the cooperatives are in a better state (the team visited Kafa Zone, but it seems that conditions are also similar in Sheka and Bench Maji), but in areas where the private sector competes, the cooperatives still cannot do better.

75. At MTR, it was clear that training of Cooperatives members and officers had thus far been largely ineffective. The MTR Report recommended the urgent redesign and implementation of the cooperative promotion and training programme, in its own words, “starting from scratch” with the re- orientation and training of individuals from both Government and the cooperatives themselves.

76. Between the MTR and the closure of the project, with the exception of the BSF component, little appears to have changed. The anticipated institutional capacity enhancement which would result from training and other capacity-building inputs has failed to adequately contribute to policy changes, organizational reform, establishment of working systems (including a functioning M&E system), and changing of attitudes. Furthermore, the partner responsible for cooperative development (the Cooperative Promotion Department) has been less involved in the international and domestic trainings than other stakeholders 39 .

77. Training needs assessment. Many of those interviewed during the evaluation expressed the view that the majority of the trainings (particularly international trainings) were not organized in a systematic way based on the needs of the project, but rather selected to benefit individuals who in the end seem to have contributed little to the project. Furthermore, it is evident that the trainings provided for the SCs did not bring about effective re-orientation and enhanced managerial capacity in the SCs.

39 See Appendix 5.

21

The condition of many of the SCs (described above) is testimony to the limited impact of the training provided.

78. By 2002, a few months away from closure of the project loans, it was noted that “…the state of some cooperatives restructured under the project appear to be weak, and in the worst cases on the verge of bankruptcy. If this is allowed to continue, such cooperatives may not survive as viable entities. Moreover, most restructured cooperatives critically need the support of the project, including training, follow-ups and technical assistance …”. 40

79. In the present evaluation, these concerns about the quality of the cooperatives restructuring process, and the business viability of the cooperatives were magnified. Restructuring of the service cooperatives in SOCODEP has contributed little to their transformation from the mentality of a command economy to that of the free market. Out of seven SCs visited in the eastern part of the project area, none were functioning before the establishment of a cooperative union in Wolayta Zone in 2005. Most 41 of the cooperatives in Konso, Arba Minch Zuria and Chencha Woredas are not functioning. Almost all suffer from high levels of outstanding debt. In the western project Zones, cooperative performance is better, with lower levels of debt, and continued functioning of some services. Table 3 illustrates the situation for 5 SCs in Arba Minch Zuria, Chencha, Boreda, and Soddo Zuria Woredas (Gamo Gofa and Wolayta Zones).

Table 6. Changes in Service Cooperative Performance between 1993 and 2006 Service Outstanding loan Misappropriation Members Cash balance Birr Cooperative Birr Birr 1993 2006 1993 2006 1993 2006 1993 2006 Kolana Dega 600 614 11361 22391 2736 63705 3597 0 Shara Eazo 3561 1535 9725 461 0 51603 65113 70380 Gaga Gocho 1004 617 28000 17000 3155 20828 0 14444 Kokate 1552 613 1650 3587 25573 340893 0 40000 Wachiga 1760 1554 940 39235 329363 314161 16420 20000 * Source: The Evaluation

80. In relation to Cooperatives, overall two conclusions may be drawn: (a) SOCODEP, like many projects, placed too great an emphasis on numerical outputs, and too little on the quality of those outputs; and (b) the general preference in SOCODEP for working through cooperatives rather than considering other groups having greater social capital (such as the indigenous Community Based Organizations (CBO) which were well established at the time), that would have been a more viable alternative.

81. Specific objective 2. Increased capital and income among the rural poor in the project area through off-farm income-generating activities particularly for women and families in densely populated areas with limited land for farm expansion.

82. The project provided credit to individuals, including women, for a limited period. This has undoubtedly increased capital and incomes for some individuals through income-generating activities and small-scale enterprises, although some have failed for lack of markets for their products.

83. Although many individuals benefited from loans provided through SOCODEP (Box 2), the extent of achievement of this objective has been limited because of the under-performance of the cooperatives re-structuring and credit disbursement aspects of the project.

40 2002 Supervision Report, para 39. 41 Out of 17 in total, 11 are completely non-functional, 4 partly functioning, and 2 functioning satisfactorily, according to the Woreda Cooperatives Desks.

22

Box 2. Three Case Studies of Women Loan Beneficiaries Belaynesh Keyito received a loan of EB 240 from a cooperative around Derashe town, Derashe special woreda. She started selling vegetables and bought one sheep for EB 55 and sold the sheep for EB 175. With the money obtained, she managed to send her three children to school and feed the family. She owns farmland in her compound and grows potatoes, maize and onions. Keito Taya has 3 children (one son, two daughters) and her children do not go to school. Her husband died some years ago due to fight with a villager. Keito belongs to Mermere Cooperative, Derashe zone of SNNPR. With a loan of EB 150 she bought one female sheep. The female sheep had 10 offspring. Two of the offspring had 7 lambs. The eight sheep were exchanged for one ox. She fattened the ox and sold it for EB 350. With the cash she bought an immature bull for EB 190. The rest of the money was used for buying food for the household. She bought an ox for EB 400 but the ox died when being trekked to her home. Later she sold some sheep and bought another ox. Now she owns 8 female sheep and one donkey. The donkey is used as a beast of burden. Lomitu Abebe belongs to Baha Cooperative, Decha woreda of Keffa zone. She is 25 years of age and married and has three children (two sons, one daughter). Two of her children go to school. She received EB 400 as a loan and bought four sheep for fattening. After repaying the loan she bought two female sheep. One of the sheep had 6 offspring and the offspring were sold to buy a heifer. The other had three offspring and the dam was sold. Now she has a heifer and one sheep. Some of the money from the sheep sales were used for house construction, buying clothes for the family and running a tej bet. She also sells beer and soft drinks. The family owns a plot of 2000-meter square of land on which enset, taro, coffee, avocado and banana are grown Source: The Evaluation

84. OMFI and continued access to credit. The credit repaid to OMFI as a consequence of Project lending appears not to have been used as a revolving source of credit to the project cooperatives, but to have been absorbed more generally into OMFI’s funds, and lent onwards through OMFI’s branch offices. Although this has diluted the impact of SOCODEP on its target cooperatives and population, it has allowed OMFI to extend its operations more widely in the Region. OMFI tends to work, not with cooperatives, but with small informal groups established by its own branch officials – a better option than the SOCODEP model.

85. The slow rate and low total amount of disbursement of credit would be less of an issue were there the prospect of further rounds of credit being available to SCs and their members in the future. However, the financial status of many of the cooperatives rightly disqualifies them from further lending. This illustrates the fundamental importance of the quality of cooperative restructuring – this is the foundation on which the rest of the project depended, and it has demonstrably failed.

86. Overall, the credit component of SOCODEP has been significantly under-spent, and it has provided one-off loans with little or no continuity of access to credit. It has consequently been of limited effectiveness. In retrospect, the channeling of credit through cooperatives (especially those with the particular political history of those in Ethiopia) was probably not the best strategy. The experience of SOCODEP has demonstrated that individuals can make good use of small amounts of credit and (women especially) repay loans in a timely manner. However, it has also shown that commercial banks are not necessarily the most appropriate lending organisations for rural credit and cooperatives are not necessarily the most appropriate channels for lending. The effectiveness and efficiency with which loans can be disbursed to and recovered from individuals depends to a very high degree on the competence and professionalism of the intermediary organisations.

87. Specific objective 3. Strengthen the Southern Region Agricultural Bureau to carry out its mandate with respect to cooperative development.

88. Staff development programmes through well planned practical trainings were anticipated to be central to the whole SOCODEP implementation, with the aim of making the implementing staff capable of delivering effective support to the SCs. Targeted training of participating cooperatives was anticipated to lead to their emergence as independent viable business entities operating in a free market economy by the time the project phased out. This objective has been partly achieved, through

23

provision of physical resources, and through the (temporary) impact of training and project experience. However, most of the trained and experienced staff have now moved on, and the extremely high level of Government restructuring and reshuffling has undermined this objective to a great extent.

89. Training. A great deal of training has been delivered, but it has been of more benefit to individuals than to the organisations which had responsibility for implementing SOCODEP. The training delivered to cooperatives officers and their members appears to have had little or no impact on the business viability of these organisations.

90. Training largely benefited individuals. It was evident from the evaluation that the trainings provided may have significantly benefited individuals (in terms of knowledge or opportunities advancement) from Government bureaux and SCs. However the benefit to the project and particularly to the intended project purpose was minimal. The evaluation team was informed 42 that for example out of 24 staffs of BOFED who have attended 13 types of international trainings; only 9 of them are now retained by the Bureau.

91. Training in M&E was particularly ineffective. Although the appraisal document clearly stressed the importance of establishing a sound monitoring and evaluation system right from the beginning of the project, and most if not all of the subsequent Supervision reports reinforced this point, M&E remained a major problem area through out the project life 43 . This was despite sending two former project coordinators for MSc study in UK on Project Planning, M&E. It is still questionable to what extent this particular training has contributed to the design of an M&E system for the project.

92. Physical resources. A large number of vehicles and other items of physical equipment (road- building equipment, office equipment, and a few buildings) have been provided, and these have been absorbed into the wider activities of the SNNPRS Government and the other institutions involved. No doubt these have contributed to the enhanced effectiveness of those organizations, but, as several of the UNOPS Supervision Reports observed, there has also been a good deal of misuse of project vehicles over the course of the project.

93. Other aspects of capacity-building. The programme of institutional strengthening carried out through SOCODEP failed to address wider issues of Government policy and strategy, organizational reform, management systems, and fundamental attitudinal change. By restricting capacity-building to the delivery of training and provision of physical resources, the objective of building Government capacity to promote cooperative development was largely missed.

94. Capacity-building efforts did not positively influence institutional change. It appears that institutional capacity-building efforts were restricted to delivery of training and physical resources. There is no evidence that SOCODEP influenced Regional Government policy on cooperatives and credit, nor that it led to organizational or management system reforms. On the contrary, frequent and significant organizational changes took place in the organizations implementing the project, and these had a detrimental effect on the ability of the project to function effectively.

95. In particular, capacity-building activities neglected to address fundamental attitudes. Despite the recognition of the importance of re-orienting both Government personnel and cooperatives officers and members, there is no evidence that this took place to any significant degree. To move from a situation in which cooperatives were viewed as an arm of Government, and in which their management was similarly bureaucratic, to the perspective in which cooperatives are independent business entities is a very large step. There is still a long way to go in making this conceptual and practical change.

42 This was clearly revealed during Oct. 8, 2006 meeting with BOFED Officials. 43 See supervision mission reports 2001-2005.

24

96. Overall, institutional capacity-building efforts have resulted in limited outputs for three main reasons: (i) they focused on only two of the many relevant aspects of capacity-building, namely training and physical resources. The issues of attitude change, policy reform and management systems were not addressed; (ii) training was not planned and delivered systematically according to project needs, and it probably had more benefit for individuals than for the organizations involved, and for the project; and (iii) frequent restructuring and reshuffling of Government organs and their personnel severely limited the effectiveness of capacity-building efforts.

97. Specific objective 4. Provide credit to meet financial requirements for agricultural inputs and draught oxen and facilitate the supply of inputs through support to local traders and cooperatives.

98. Credit to cooperatives for their own businesses (consumer shops, flour mills, produce marketing) have mostly been unsuccessful, except in areas of coffee production. Credit through service cooperatives to individuals for oxen and small ruminants has been much more promising, with many individuals benefiting. However, the culture of loan repayment has been variable, generally with those in the western parts of the project area having a better repayment record.

99. Ox loans in particular made a big difference to the farmers fortunate enough to receive them (Box 3). The main shortcoming in relation to meeting this objective was in terms of: (i) relatively small numbers of loans disbursed, and (ii) lack of continuing access to loans.

100. Overall, the credit component was under-utilized; loans to cooperatives for their own business activities were largely ineffective; loans through cooperatives to individuals were better used, but repayment levels (with the exception of loans to women) were not impressive. Arguably the single most important objective of the project – the delivery of small loans to poor rural farmers – was let down by its limited reach, low prospects of sustainability and lack of continuity.

Box 3. Four Examples of Ox-loan Beneficiaries Dana Dedetto is aged 55 years and belongs to Gagagocho Cooperative of Boreda Woreda, Gamo Gofa zone. He is married and has 6 children (4 sons, 2 daughters). Due to the sudden death of his ox, he used to cultivate by begging an ox from the community. He took credit for oxen purchase in 1999. Now he cultivates 2ha of farmland and has set aside 1ha for grazing land. Currently he owns two oxen and cultivates maize and teff and sells crops after harvest in markets. He sends his four children to school and two are now employed as civil servants in the Woreda. As a result of the oxen purchase, he uses improved household utensils and furniture. Instead of the traditional beds made of cattle hides he uses a modern bed. The household members use plastic items for drinking water instead of pumpkins. He states that his children are properly fed as well. Darge Semane belongs to Gebele Beno Cooperative, Derashe Woreda and is 47 years of age. Three of his children do farm work and one goes to school. He received an ox loan of EB 650, added EB 100 of his own money, and bought an ox for EB 750. After fattening the ox, he sold it for EB 1400. With the cash obtained, Darge bought an ox for EB 600, paid EB 77 for insurance of the ox, built a house, bought clothing for his children and paid for medical treatment. He bitterly complains about the high insurance cost he paid. As the locality is ridden with diseases, he also bought trypanocidal drugs for treating his animals from trypanosomiasis. Currently he has one ox and cultivates 1.5ha of farmland. The main crop he grows is maize. A farmer named Temesgen Kebede lives in Bitta Cooperative, Bitta Woreda of Keffa zone. He received an ox loan of EB 650 and used the ox for cultivation in the last four years. He has paid the loan and interest. He has an arrangement with another farmer to cultivate his 2ha of land. He also owns an enset plot of 0.25ha. Currently he has two oxen and sells grain for buying clothing for the family and last year sold grain worth EB 1000. Mammo Bongo is a farmer who belongs to Boha Cooperative, Decha Woreda, in Kefa zone. He is married and has 6 children (4 sons, 2 daughters) and none of the children go to school. He bought an ox with a loan of EB 600. After repaying the loan, he sold the ox for EB 880.The cash was used for treating problems he had on his eyes. Now he has recovered from his illness and is intending to build a house of corrugated iron sheet. * Source: The Evaluation

25

101. Specific objective 5. Relief of livestock health constraints, particularly with respect to draught animals, through provision of veterinary drugs.

102. This component started late, and is now winding down. It provided temporary relief, but now there is evidence of increasing disease prevalence due to the inability of Government veterinarians to provide drugs. There are no significant alternative suppliers of veterinary medicines than Government. A better option than the one pursued in SOCODEP would have been the promotion of veterinary drugs through the private sector.

103. The component only took off toward the end of the project (2000/01). Drugs for relevant livestock diseases were bought by the government veterinary offices and made available to stockowners. The government drug supply for a Woreda lasts about six months and drug shortages are felt from July to September before the fiscal year budget is released. After the closure of the project in 2005, the incidence of diseases is reported to be increasing because the VRDF was dwindling. A better approach to dispensing drugs to farmers may be the use of Community Based Animal Health Workers selected from the community by the community and linked to drug vendors in Awassa and Addis Ababa.

104. Overall, the achievement of the VRDF was limited. It started very late, and by the time of closure of the project loans it had disbursed a very small proportion of its budgeted funds, despite the great need for the benefits which it promised.

105. Specific objective 6. Improve access of rural families to services and markets by rehabilitating and maintaining rural roads. 106. As far as it went, this component has improved access for many people 44 . Maintenance by cooperative members alone was never going to be a viable option 45 , and the adoption of the SOCODEP roads into the Regional road maintenance programme is a positive feature, as long as Government budgets permit maintenance to actually be carried out. The heavy equipment provided by SOCODEP will continue to be used and maintained for some time to come. This element of the Project does give rise to some environmental concerns however: erosion, destruction of natural vegetation, lack of restoration of quarries, and abandonment of equipment and materials are among the negative aspects observed.

107. Design standards and maintenance strategies. The main issues in the roads component of the project centre around design standards and maintenance strategies. The change to the project design agreed after the MTR, in which the design standard (and hence unit cost) was increased from RR 30 to RR 50 46 not only limited the amount of construction possible within a fixed budget, but had maintenance implications too. RR 30 roads were more suitable for labour-based maintenance, while RR 50 roads required heavy maintenance by a centralized authority. The expectation at Appraisal that maintenance would be carried out by cooperatives was far-fetched, as it is not only cooperative members who benefit from such access, but the wider community too. The communities in SNNPR have allocated one day a week for community endeavours such as access roads construction, school construction and maintenance, health facilities construction and maintenance, and soil and water conservation, which is in line with the construction of roads of RR 30 standard and below.

108. In terms of achieving the project objective, two aspects are relevant. First, the change in design standard significantly increased cost per km, and so greatly reduced the number of beneficiaries. Only 122km out of a target at Appraisal of 700km was achieved. Second, although the ability of some rural

44 Based on typical population densities, probably around 100 000 people live within 5km of the respective roads. 45 Especially after the agreed alteration of design specification from RR 30 to RR 50 standard. 46 RR30 and RR50 refer respectively to road designs for 30 and 50 vehicles per day. The designs specify different standards of drainage structures, gravel thickness and road width. The former RR standards have subsequently been replaced by a range of functional design standards referred to by DS numbers.

26

people and communities to move themselves and their goods has improved, no other measures were taken in SOCODEP to improve market access. Physical road access is only one necessary (but not sufficient) condition for achieving better linkages to markets.

109. Specific objective 7. (BSF Component) Reduce the burden of disease in 8 woredas of the SOCODEP area.

110. Probably the greatest effect on health will be through the upgrading of physical health facilities and staff and community skills. The sanitation component will only have a significant effect if latrines are actually used, and this is most likely if the Government strategy changes to a more persuasive and participative approach. The water supply interventions made by the BSF project will have a significant short-term effect on water quality and access for those living close by, but long term functional sustainability is in serious doubt.

111. Evaluating health impact is particularly difficult, but the measures taken in the BSF component at least provide some of the necessary pre-conditions for improved community health. However, without a significantly increased focus on sanitation utilization, hygiene promotion and water supply system operation and maintenance, these potential health benefits will fail to be realized.

F. Assessment: Relevance, Effectiveness and Efficiency

112. Relevance. SOCODEP addressed some real needs of the rural poor in southern Ethiopia (need for credit, need for improved market access including rural roads, need for better health services and environmental health) and it was consistent with the government’s regionalization programme. At the time of design the rhetoric of the new Government and the framing of the new legislation made working through service cooperatives attractive and promising. However the use of service cooperatives as the channel for services to the poor turned out to be a significant design weakness. At the time of project formulation, cooperatives were seen as an arm of Government, and public attitudes to cooperatives would inevitably take a long time to change. As far as the selection of financial intermediaries is concerned, at the time of project design there was limited choice. However, CBE’s lack of experience in implementing rural finance, including through cooperatives, should have been addressed with greater seriousness. Weaknesses in other project design features have been highlighted above.

113. At a strategic level, the project was mostly consistent with the 1999Ethiopia Country Strategic Opportunities Paper and the 2002 IFAD Regional Strategy particularly in its support in developing rural financial services. Although the strategy provides a larger emphasis on establishing market linkages. According to IFAD’s rating scale therefore we assess SOCODEP as ‘Relevant’ (rating of 5) 47 .

114. Effectiveness. The Project achieved some, but not all of its objectives. Table 4 summarizes the extent to which each of the specific objectives were met, both in quantitative and qualitative terms. As indicated in Table 7, Specific Objectives 1, 3, and 5 were mostly not met. Specific Objectives 2, 4, 6 and 7 were partially met (to varying degrees). According to IFAD’s rating scales therefore we assess SOCODEP as ‘Moderately Ineffective’ (score 3) 48 .

47 “The project objectives are relevant to the needs of the poor or to IFAD’s country strategy” (IFAD OE Project Evaluation Guidelines, December 2005) 48 “The project met only a few of its stated project objectives” (IFAD OE Project Evaluation Guidelines, December 2005).

27

Table 7. SOCODEP: Achievement of Specific Objectives (see also Table 3 and Figure 3) Specific Objective Achievement (quantitative) Achievement (qualitative) SO1. Provide a model for Numerical targets for restructuring Many, if not most, restructured developing Ethiopian and re-registering cooperatives were cooperatives have failed to compete in cooperatives under the new exceeded (130 per cent of Appraisal their business activities with the private legislation, particularly target; 178 per cent of MTR revised sector; many are in default of loan with respect to target). repayments and ineligible for further improvement of financial credit; many have suffered significant intermediation services in levels of misappropriation of funds; rural areas, which could be members lack confidence in the officers. replicated in other areas of the country. SO2. Increased capital and Reach of small-scale enterprise Some small businesses have been income among the rural development sub-component was assisted to take off, and the loans to poor in the project area very limited. The total amount of women for small ruminants have been through off-farm income- training, and loans disbursed was particularly effective. However, generating activities very small. Loans to women for continuity of access to credit is limited to particularly for women and small ruminants were much greater the reach of OMFI. families in densely in number. populated areas with limited land for farm expansion. SO3. Strengthen the SRAB Training and vehicles budgets were There is little direct evidence with which to carry out its mandate disbursed rapidly and fully. Many to assess the quality of training needs with respect to cooperative trainings were delivered. assessments and trainings delivered. development. However, the indirect evidence suggests that much of the capacity-building effort has been ineffective. SO4. Provide credit to meet Numerical targets were not met. Cooperative members have benefited financial requirements for Total loan disbursement was low, from individual loans, but not from agricultural inputs and and number of loan recipients fell (failed) cooperative businesses. Loan draught oxen and facilitate well below target. repayment rates (with some exceptions, the supply of inputs especially loans to women) have been through support to local poor under CBE. traders and cooperatives. SO5. Relief of livestock Disbursement from the Veterinary The Veterinary RDF was partially health constraints, Drugs Revolving Fund was very effective for a short period. Its demise is particularly with respect to late and very limited in total contributing to an increase in livestock draught animals, through amount. disease. provision of veterinary drugs. SO6. Improve access of The total length of roads Those living within a few km of the new rural families to services constructed fell well below the roads have undoubtedly benefited, but and markets by Appraisal target (which was revised the elevated cost of the higher design rehabilitating and downwards at MTR). standard adopted has limited the reach of maintaining rural roads. this component. SO7. (BSF Component) The majority of the quantitative The achievement of the component has Reduce the burden of targets for this component have established the potential for sustained disease in 8 woredas of the been met. health improvements. However, all SOCODEP area aspects require continuing support and funding for long-term sustainability. * Source: The Evaluation

115. Efficiency. Because of the nature of the project no attempt was made to assess the economic rate of return in the appraisal or completion reports. Thus, in the evaluation a number of qualitative indicators are used to assess the efficiency of the conversion of project funds into outputs and consequent impact. The four indicators outlined here are well established in project documentation, and were confirmed during the evaluation mission.

116. First, the project contained a number of unrealistically ambitious design aspects (most notably relating to the large geographical extent of the project area). The vast geographic area, poor

28

infrastructure and communication meant that the projected resources were not optimally used. Second, the overall emphasis on the Project’s numerical outputs (i.e. exceeding the number of restructured cooperatives) rather than the quality of those outputs, has resulted in a significant amount of wasted resources. As a result, many of the project’s activities were diluted to the extent that they failed to achieve their intended outcomes. For example, the generally poor quality of the restructuring of the cooperatives, and the highly compromised institutional strengthening component weakened the project significantly. Efficiency would have been greatly enhanced if the project design had concentrated on a smaller project area, involving a smaller number of SCs, and provided greater intensity of effort and resources.

117. Third, the regional context and many disruptions caused by changes in Government structures further limited the ability of personnel to deliver services in an efficient manner (see para 28 and 33). And fourth, the absorptive capacity of the Government institutions involved was limited to the extent that significant sums of money remained unspent at the end of the project. At project completion, in spite of several extensions to the project completion date, substantial amounts of IFAD’s loan funds and also the BSF grant funds had not been fully utilized (only 68 per cent of the Loan Funds and 93 per cent of the Grant Funds were disbursed).

118. Also, the construction of roads to a higher standard than necessary raised unit costs and limited the outputs achieved.

119. Some allowance has to be made for Ethiopia’s particularly difficult operating environment, so according to IFAD’s rating scales we assess SOCODEP as ‘Moderately Inefficient’ (score 3) 49 .

G. Performance of IFAD and its Partners

120. Assessment of partner performance. IFAD’s Evaluation Manual requires a qualitative assessment of each partner, and the partnership as a whole, against stated criteria selected from a list in the manual 50 . In the following paragraphs each partner is considered in turn and assessed on IFAD’s 6-point rating scale, and the partnership as a whole is assessed.

121. IFAD. As the initiator and main source of loan funding for the project, IFAD bore an important part of the shared responsibility for its successful execution. As successive supervision reports flagged similar critical issues (problematic management, political interference, ineffective M&E, questionable quality of cooperatives restructuring, poor performance of cooperatives, and lack of institutional ownership of the project), new mechanisms should have been sought to improve project management, rather than relying on the annual supervision missions by UNOPS. Although IFAD’s role was supportive and responsible – including its initiation of an early MTR and follow-up actions and the inclusion of the BSF component - it was insufficiently flexible and decisive in responding to the problems raised by successive Government reorganizations and the reported lack of Government ownership at the highest levels in the Region. However, it should be recognized that during the time of SOCODEP, IFAD did not have modalities such as direct supervision or field presence to support project implementation. Recognizing the many difficulties faced by the project (e.g. the credit component) and the limited chance of making dramatic improvements IFAD let the project close with about 35 percent of the IFAD loan unspent. Overall, IFAD delegated too much responsibility to institutions which were not well prepared to perform effectively. The evaluation judges IFAD’s performance moderately unsuccessful (rating of 3).

122. Government of Ethiopia (GoE). Although GoE observed the loan conditions and participated in the project implementation, two factors seriously weakened its performance. The first was the repeated reorganization of Government organs and the reshuffling of Government staff, to the point at which institutional capacity-building was seriously undermined. Although such reorganizations were

49 “Poor use of resources: unit costs are above those of comparators, and rates of return are lower than alternative investments and negative” (IFAD OE Project Evaluation Guidelines, December 2005). 50 “Participatory, Delegates, Coordination, Communication & Sharing of Information, Flexible, Supportive, Decisive, Responsible, Learning” (IFAD OE Project Evaluation Guidelines, December 2005).

29

the prerogative of Government, little was done to compensate for their negative effects on the Project. The second factor was the reported lack of political commitment at the level of the Regional Council, and political interference with the project, to the point that the Project Coordinator’s role was compromised in the early years of project implementation. At the beginning this was caused by a disjoint between Federal and Regional Government actions, but even after the Regional Government signed up to the project this did not significantly improve. CBE and OMFI. Commercial Bank of Ethiopia was insufficiently committed to the goals of the project, and its reach did not extend to the client groups which were the focus of SOCODEP. It had failed to understand the nature of the business it was entering. As a late arrival (2001) to the project, OMFI was better placed to fulfill its requirements. It has disbursed loans through service cooperatives, as well as through its own groups, but since the closure of the IFAD loans it has apparently not operated the IFAD credit line as a revolving fund to the SOCODEP cooperatives and their members, but rather used the repaid loans to finance its wider lending programme. Accordingly, the Government’s performance is found moderately unsatisfactory (rating of 3).

123. Belgian Survival Fund. The performance of BSF can only be assessed indirectly, through the manner in which the WSHBS component was conducted. The far greater geographical focus (8 Woredas, rather than 33) 51 was no doubt instrumental in BSF’s effectiveness, and the component was notable for the thoroughness of its capacity-building efforts and the continuity of its technical assistance. However, the limited degree of integration of the BSF sub-component with the remainder of SOCODEP, and the concerns over sustainability of the BSF interventions, led the evaluation team to reduce what would otherwise have been a high rating to a somewhat lower score. BSF performance is rated 4 (moderately successful).

124. Cooperating institution (UNOPS). The Supervising Institution is placed in a difficult situation when its inputs are limited effectively to a single annual visit. The Supervision Mission Reports seen by this evaluation team have been competently conducted, but the repeated flagging of critical issues year after year makes it clear that a much stronger hand was needed to manage the project effectively – not simply loan and project implementation supervision, but strong management and frequent monitoring of progress. Accordingly, UNOPS performance is considered moderately successful (rating of 4).

IV. PROJECT IMPACTS

A. Rural Poverty Reduction Impacts

125. Impact domains. IFAD considers the wider impact of projects under 9 impact domains. These are set out in the following paragraphs, with comments on the intended and actual impacts of the Project. Our evidence base here consists of the beneficiary interviews conducted, review of secondary data, the evaluation team’s extensive past experience in the Region, and its professional judgments as to the extent to which findings from a small sample can be extrapolated to the wider population. The key areas of intended direct impact for SOCODEP were those of (a) financial assets (through credit), (b) physical assets (roads, water and sanitation, health facilities), (c) human assets (skills and knowledge), (d) institutions and services (strengthened Government organisations and functioning service cooperatives), and (e) social capital and empowerment (through effective cooperatives). Through delivering impact in these areas, SOCODEP was to have an indirect impact on agricultural productivity, food security and on market access.

126. Agriculture productivity. This related to one of the two identified higher goals of the Project (“increase agricultural productivity …”). The greatest impacts in this domain relate to the increased access to work oxen and small ruminants for fattening and rearing (Boxes 1 and 2). This has been a significant impact at the individual farmer level. However, because overall project impact on agricultural productivity has been below expectations, the evaluation judges project impact moderately unsuccessful in this domain (rating of 3).

51 Although these were still widely spread geographically.

30

127. Financial assets. Increasing individual financial assets was the second of the two identified higher goals of the Project (“…and raise income levels of the rural poor…”. In many of the cases of individual beneficiaries interviewed, increases in personal financial assets were evident. The expectation was that service cooperatives would be effectively developed, “… to facilitate efficient provision of sustainable services to members.” In the majority of the cooperatives interviewed, or for which data were obtained (especially in Gamo Gofa, Wolayta and Derashe), the cooperatives are de facto bankrupt. Furthermore, the enhancement of individual financial assets has had limited reach and, for many of those who have benefited, the impact is likely to have been short-lived. Consequently, the evaluation considers project impact on financial assets moderately unsuccessful (rating of 3).

128. Physical assets. Through accessing credit, cooperatives and their members were able to increase their physical asset base. Numerous examples were encountered of individual loan beneficiaries who had, for a time, increased their physical assets (in terms of livestock, housing, and equipment for small-scale enterprises). See Boxes 1 and 2 for examples. Some communities have benefited from improved physical assets in the form of rural roads, water supplies and latrines. Their number however, compared to target beneficiary population, is relatively small. Furthermore, the questions over sustainability, raised elsewhere in this report, threaten long-term impact. Overall, considering their limited extent, the evaluation still finds project impacts on physical assets moderately unsuccessful (rating of 3).

129. Human assets. Enhancement of human skills, knowledge and health were explicit aspects of the Project. Training of Government staff, cooperative personnel and cooperative members had a small impact, as judged by the effectiveness of the institutions involved. The impact of training given has to be questioned. On the other hand, the health impact of parts of the BSF component – in particular the upgrading of health facilities and training of staff and community health workers – had the greatest potential impact. Project impact on human assets is therefore found moderately successful (rating of 4).

130. Institutions and services. Institutional strengthening was a specific objective of the Project. Changes to laws, policies, and “the rules of the game” were not. The impacts of the institutional strengthening components of the Project were largely lost due to the constant reorganizations and reshuffling of Government staff. The Project apparently had little or no impact at higher levels of policy dialogue. Consequently, project impact on institutions and services is found unsuccessful (rating of 2).

131. Social capital and empowerment. Collective capacity of cooperative members was to be built under the new law, and through the restructuring process built into the Project. Service cooperatives (now called multi-purpose farmers’ cooperatives) still represent rather artificial groupings of loosely- linked members. There is little evidence of the social capital of these groups being significantly enhanced by the Project. Some women (but mostly men) have been personally empowered through use of individual loans for small ruminants and for small scale enterprise development, but their numbers are rather small in the context of the population of the project area. In all, project impact on social capital and empowerment has been far below expectations and is thus judged unsuccessful (rating of 2).

132. Food security. By implication, SOCODEP’s goal was to improve food security among cooperative members. The Project’s impact on food security was indirect, and inherently not measurable in this evaluation. However, the possession of work oxen, and the increase in livestock and other physical and individual financial assets, may be assumed to have had some beneficial impact in this domain. Considering its limited extent, project impact on food security is considered moderately unsuccessful (rating of 3).

133. Environment and common resource base. SOCODEP had no specific objectives in this domain. The Project was generally environmentally benign, neither increasing nor decreasing people’s access to natural resources. The Roads component may have had a small unintended negative environmental impact, but this was outweighed by its much greater socio-economic impact. Environmental impact assessment should be given greater emphasis in this area in future. The

31

evaluation finds project impact on the environment and common resource base therefore moderately successful (rating of 4).

134. Markets. Attention to markets and market access was an omission from the Project design. The Project did little to increase access to markets, other than through financing a small amount of rural roads. Market linkages are crucial to raise small farmers above subsistence level, but the Project design omitted this key domain. Many of those trained in small scale enterprises subsequently failed to go further, for lack of markets for their products. It should be noted that SOCODEP as designed (covering much too large and disparate a geographical area) could not effectively address marketing constraints, within its overall budget. However, there could have been opportunities to more directly address marketing within Special Objective (SO)s 1, 2, 4 and 6 and with a design which was more focused geographically, and addressing producers’ needs in a more integrated way, should have considered markets for increased production as well as the outputs of small enterprises. As a result, project impact on markets is considered unsuccessful (rating of 2).

135. Overall assessment of impact. The main beneficial impacts of the Project were largely limited to those individuals who received loans through cooperatives, or who now benefit from physical infrastructure and services such as roads and health facilities. In the case of credit, these impacts have been relatively few in number, and short-lived. The expected impacts in relation to increased individual financial assets (and consequent improved access to physical assets, agricultural productivity and food security) were limited in extent and continuity. Some benefits of training impacted beneficially upon individuals, but, with the exception of the BSF component, training and capacity-building had limited impact on the institutions involved. Impact on social capital was very limited, because of the largely ineffective restructuring of cooperatives. Consequently, in relation to the scale presented in IFAD’s Evaluation Manual, overall SOCODEP impact is rated as moderately unsuccessful (rating of 3) 52 .

B. Sustainability and Ownership

136. The inherent unsustainability of projects. Projects are by definition time-limited inputs of resources to achieve stated goals. While the immediate project objectives may be met, it is unusual to achieve real and permanent step-changes to institutions, services and the lives of the poor as a consequence of short projects. Sustainability needs to be designed in from the beginning, and interventions need to be long enough in duration and with clear exit strategies, or the achievement of project objectives and impacts quickly fades into history. SOCODEP is no exception in having paid too little attention to these issues.

137. Components of sustainability. The main aspects of sustainability relevant to SOCODEP relate to (a) whether or not the enhanced capacity of the Project’s institutional stakeholders (Government, cooperatives and lending agencies) will enable them to continue functioning over the long term, (b) whether cooperatives and individuals will continue to have access to credit services, and (c) whether physical infrastructure (water supplies, roads, buildings, vehicles and equipment) will continue to function over time.

138. Institutional sustainability. This represents the least sustainable aspect of the Project. The reorganizations of Government organs and the reshuffling of staff has greatly diluted the enhancement of institutional capacity. Physical resources provided as part of institutional capacity-building activities have been of significant assistance, but they have a limited life.

139. Continued access to credit. While the impact of individual loans has in many cases been a very significant positive aspect of the Project, continued access to credit and other financial services is more questionable. In its discussions with individual beneficiaries, the evaluation team found little evidence that those who had benefited once from loans were able to continue to obtain further credit –

52 “The project generated rural poverty reduction impacts in fewer domains than expected (impact domains that were relevant to the project as per design at the beginning and/or after revision) and to a lesser extent than expected.” IFAD OE Project Evaluation Guidelines, December 2005.

32

despite their generally good repayment record. Many of the cooperatives visited, or for which data were obtained, are now de facto bankrupt, and therefore ineligible for further loans.

140. Functional sustainability of physical assets. Roads constructed under the Project were expected at Appraisal to be maintained by members of the restructured cooperatives. This was unrealistic for two reasons, (a) because those benefiting from the roads are a much greater population than simply the cooperative members, and (b) to expect community members to add an even greater labour burden to their already over-stretched daily lives was impracticable. The fact that the roads constructed under SOCODEP have been adopted under the Region’s road maintenance programme is actually better assurance of sustainability than relying on community-based maintenance. The heavy equipment supplied to the Region will allow other road construction activities to take place for some time to come. In regard to water supply, the prospects for on-going functional sustainability of especially handpumps and gravity water supply schemes are very poor. The financial contribution made by households for repairs and maintenance are based on affordability rather than need, and they are insufficient to cover the costs of repairs. Furthermore, spare parts are particularly difficult to obtain, and insufficient attention has been paid to operation and management aspects of this component.

141. Assessment of sustainability. In view of the very limited sustainability of institutional strengthening efforts, the limited on-going access to credit, and the unlikely sustainability of the water supply component, sustainability is assessed overall as unlikely (rating of 2) 53 .

C. Innovation, Replicability and Scaling-Up

142. Innovation. At the time of project design, SOCODEP’s focus on cooperatives and credit represented a response to the apparent liberalization of national politics and economics, and to the change in cooperatives legislation. The inclusion of the other components (veterinary revolving drug fund, rural road construction, small-scale enterprise development) was not particularly innovative, and their lack of integration represented a lost opportunity. The BSF component (water supply, health and basic sanitation) in effect represented a new project, itself split into three components, and again the opportunities for integration and synergy were largely lost.

143. Worth mentioning as being innovative at the time in Ethiopia, was the effective monitoring and evaluation system of the BSF component. When the BSF component began it introduced a comprehensive Baseline Line Survey and before project completion it undertook an Impact Assessment Study. This M&E system, which did not exist for the other components, formed the basis for developing trainings, capacity building, learning and measuring results that helped contribute to the performance of this component. The innovative methodology of the Impact Assessment Study made extensive use of local knowledge and participation, and used simple, inexpensive techniques for impact measurement in the field.

144. Knowledge management. The collation and sharing of knowledge among the project stakeholders was limited by the weak M&E which was repeatedly highlighted in Supervision reports. After the MTR a number of internal assessments were carried out (dated 1999 and 2000), which presented some critical and insightful views, but the extent to which these led to subsequent action was limited. The lack of physical availability of internal and external project documentation is a critical issue. Not surprisingly, the extent to which project experiences were used by project stakeholders in useful policy dialogue at Regional or higher level was also very limited.

145. Replicability. The same factors which have limited the impact of institutional strengthening and constrained the quality of many the project actions and outputs remain in place today. Were the Project to be implemented today using similar mechanisms (ie operating through a Project Coordinating Unit attempting to synchronize and harmonize the work of disparate Government organs), similar outcomes would result. The model is not replicable.

53 “Hardly any of the supporting factors are in place”. IFAD OE Project Evaluation Guidelines, December 2005.

33

146. Scaling-up. The Project was over-stretched in terms of geographical focus, and the quality of Project outputs suffered at the expense of quantity. It does not represent a successful pilot which can be usefully scaled-up.

147. Overall assessment. SOCODEP had no significant innovative design elements, and in implementation, little of the learning which was being generated was fed back into the execution of the Project. There is little or no potential for replication or up-scaling. Therefore, innovation, replicability and scaling-up are assessed overall as moderately unsuccessful (rating of 3) 54 .

V. CONCLUSIONS AND RECOMMENDATIONS

A. Overall Assessment

148. Outreach and targeting. SOCODEP was unsophisticated in its targeting. There was little detailed understanding of the nature and variations of rural poverty within the Project Woredas, and although some women benefited, their numbers were rather small. Even IFAD’s Country Programme Issues Sheet (May 2006) refers to the view that “…poverty is somewhat evenly spread across rural areas where 85 per cent of the population live…”, an assertion that is superficially true since most rural people are very poor, but which hides significant variation, sometimes over small distances. Furthermore there is also anecdotal evidence of elite capture as the individual beneficiaries of training and loans were not selected in a uniform and transparent manner.

149. Attaining IFAD’s strategic objectives and the MDGs. To the extent that individuals have benefited from loans, training in small-scale enterprise development, improved water, sanitation and health facilities, and new roads, SOCODEP has gone some way to improving health and education, and reducing poverty and hunger. This impact has however, generally been relatively short-lived.

150. Overall assessment. Table 8 summarizes the ratings attributed to the project by the evaluation, with the average ratings given to project evaluated in 2005 for comparison. On most criteria SOCODEP scored one point or more under the 2005 average. Relevance, Impact on physical, financial and human assets is comparable with 2005 averages, but the performance of the Government is assessed as below the 2005 average. Because of fundamental weaknesses in project implementation, caused partly by unrealistic design, partly by insufficiently decisive management, partly by insufficient political commitment, and partly by externalities including frequent Government restructuring and reshuffling, SOCODEP is assessed overall by this evaluation as ‘Moderately unsuccessful’ (rating of 3) 55 .

54 “The project had very few innovative elements in the project design and implementation. Lessons that should have been learnt from the experience were not.” IFAD OE Project Evaluation Guidelines, December 2005. 55 “The project was rated just below expectations (category 3) for at least three of the six ratings, but none was rated highly successful.” IFAD OE Project Evaluation Guidelines, December 2005.

34

Table 8. Project Ratings Summary Evaluation criterion SOCODEP rating 2005 project evaluations average Project performance Relevance 5 5 Effectiveness 3 4 Efficiency 3 4 Partner performance IFAD 3 4 Government 3 4 BSF 4 - UNOPS 4 4 Project impact Agriculture Productivity 3 - Physical and Financial Assets 3 4 Human Assets 4 4 Institutions and Services 2 - Social Capital and Empowerment 2 4 Food Security 3 4 Environment and Common Resource Base 4 4 Markets 2 - Overall impact 3 - Sustainability 2 4 Innovation, Replicability and Scaling-Up 3 4 Overall assessment 3 - * Source: The Evaluation B. Conclusions

151. As with most multi-component projects, the project performance and impacts of SOCODEP have been mixed. There seems little doubt that the Project has had a net benefit to the Region, to individuals in Government, and to some of the target beneficiaries. The key questions relate to understanding how the benefits could have been greater, and what lessons can be learned for present and future projects in Ethiopia and further a field.

152. Context. SOCODEP was one of the first significant internationally funded interventions in Ethiopia following the fall of the former Marxist-Leninist regime (the Derg) in May 1991. The Project aimed to respond to the then new legislation concerning Cooperatives, which ostensibly set out a means of turning the former Government-imposed and politically-dominated Producer Cooperatives of the Derg into farmer-owned viable business entities serving their members’ interests. In particular, it aimed to make rural finance (specifically micro-credit) available to so-called Service Cooperatives and their members.

153. Design. The Project aim, as described in paragraph 152, was an imaginative attempt to respond to the then new legislation concerning Cooperatives. However, this was an ambitious and in hindsight an unrealistic goal. A number of factors contributed to the difficulties faced by the SOCODEP. First, the Region’s size, diversity, poor infrastructure and poverty posed numerous development challenges. Second, the project underestimated the obstacles to and rate of beneficial change and the capacity of the government for implementation in the post Derg period.

154. Quality of project delivery. SOCODEP concentrated on delivery of numerical outputs, such as cooperatives restructured, credit disbursed, trainings delivered, drugs purchased, kilometers of road constructed, water points built, and so on. Insufficient emphasis was placed on the quality of these outputs. For example, insufficient consideration was given to the intensity and duration of activities required to achieve the desired quality standards. In particular, the human factor of individual and group (community, cooperative, institution) attitudes were addressed minimally. For example, the design was too optimistic about the speed with which the former model of cooperatives, centrally controlled by the government, could be turned around into a member-owned and member-controlled

35

viable business model. To turn around a failing, politically-established cooperative to become a viable business serving its members, or to bring about community ownership and management of a water point demanded a great deal of attention to quality of the investment, not just numbers.

155. However, some of the activities such as upgrading of health facilities and training of staff and community health workers performed better.

156. Relevance, effectiveness, efficiency. The project objectives and activities were relevant to needs of the rural poor in southern Ethiopia (e.g., in terms of the need for credit, improved market access, better health services and environmental health) and were consistent with the government’s regionalization programme. The Project was moderately ineffective achieving some, but not all of its objectives. Similarly, the project was moderately inefficient. Given the difficult operating environment in Ethiopia, the project could have been more efficient if it had been more realistic and had less ambitious objectives and coverage. The vast geographic area, poor infrastructure and communication meant that the projected resources were not optimally used.

157. Sustainability. SOCODEP’s benefits are unlikely to continue, partly due to the lack of a defined exit strategy. Moreover, institutional sustainability is limited and on-going access to credit, and water supply is not assured.

158. Innovation. At the time of design, SOCODEP’s focus on cooperatives and credit represented a response to the apparent liberalization of national politics and economics, and to the change in cooperatives legislation. Unfortunately, the country context changed rapidly and the design became less relevant given the new context. Despite the positive efforts at the MTR, the design adjustments were not adequate given the changing realities. On another issue, the BSF component introduced an effective monitoring and evaluation system, which however was not integrated into the other project components. As the project was overstretched and its components were not integrated, SOCODEP offered little opportunity for the learning being generated to be feed back into the project. Hence, the evaluation considered the project to be moderately unsuccessful in terms of innovations, replicability and upscaling.

159. Policy dialogue. SOCODEP was largely responsive to policy changes and government-led restructuring. There is little evidence however of the project contributing to IFAD’s effective engagement in policy dialogue in the country.

160. Participation. Probably the single greatest assurance of sustainability at the level of households and communities is through real commitment to beneficiary participation. However, the evaluation found that ensuring beneficiary participation in an area with a weak tradition of participation is challenging and requires greater commitment in terms of time and resources. As such, approaches which build on existing social capital (i.e., using indigenous Community Based Organizations), rather than working through structures imposed from above and outside the beneficiary communities, are most likely to succeed in the short and long term.

161. Integration. This evaluation report has highlighted at a number of points the lack of integration between the numerous stakeholders and components within SOCODEP. Although integration is not easy, particularly given the restructuring of government organs and redeployment of personnel, it is the only way to create synergies which can maximize the impact of limited budgets.

162. Management. The management model used by SOCODEP limited its effectiveness and responsiveness to rapid contextual changes that occurred during project implementation. The Project Coordinator, attempted to harmonize and synchronize the work of several Government organs and other stakeholders over whom he has no real authority. And support from IFAD through supervision mission mounted annually by the designated cooperating institution (UNOPS) was insufficient. However, it should be recognized that during the time of SOCODEP, IFAD did not have modalities such as direct supervision or field presence to support project implementation. Although, IFAD was responsive in using the tools it had at the time, for example by undertaking an early and useful MTR and facilitating the inclusion during implementation of the important BSF component.

36

163. Weak linkages between partners during implementation. The linkages between the key stakeholders have mainly been achieved by the efforts of the Project Coordinators (four in total, at various stages of the Project), IFAD’s Country Programme Manager (CPM), and UNOPS. The Project Coordinator’s role was challenging because of his inherent lack of authority, and the CPM’s role is distant from the day-to-day project management issues to do more than provide general support and guidance. Also, annual visits by a cooperating institution are insufficient to rescue an under- performing project. Consequently, the effectiveness of the partnership was limited. Partnership with the private sector was not a viable option in the early years of the Project, but this option could have been pursued as implementation progressed. The extent to which partnerships outside of the project were developed by the implementing stakeholders is limited.

C. Recommendations

164. Design. It is recommended that consideration be given to interventions which are far more focused in terms of numbers of beneficiaries to be reached and geographic coverage, within the overall framework of IFAD’s targeting policy. This would ensure greater synergies across activities and ultimately deeper impact on rural poverty. Similarly, the project duration should be long enough to achieve the desired results and in particular take into account the time needed to implement attitude and cultural changes. Project management structures should be kept simple to ensure the integration and harmonization among different implementing agencies.

165. Quality of project delivery. It is recommended that greater attention be given in future project design and implementation to country context issues, and the identification of indicators of quality, and actions necessary to achieve real and lasting impact, alongside those relating to numerical outputs.

166. Policy dialogue. It is recommended that more explicit attempts be made to engage in policy dialogue with Government and other development actors, where appropriate and required involving a wider range of national and international specialists, rather than just IFAD and cooperating institution staff.

167. Participation. It is recommended that future IFAD-funded projects and programmes in Ethiopia pay more attention to people’s participation, especially as it has not been a tradition of development practice in Ethiopia in the past.

168. Integration. In future multi-component projects, it is recommended that greater attention be paid to the linkages between the components and between those agencies responsible for delivering them. The evaluation team is in favour of projects which involve multiple components addressing the diverse needs of target populations – but the difficulties of integrating such efforts should be carefully considered.

169. Management. Management is more than coordination or supervision, and it should be addressed with greater rigour in future projects and programmes particularly in challenging contexts as found in SOCODEP. Projects need to be much more decisively managed. It is recommended that new approaches be explored, either through IFAD itself taking a more hands-on role during execution, facilitated by the Fund’s field presence officer, which allow closer monitoring and follow-up to implementation.

170. Role of the Field Presence Officer. The field presence officer can, among other tasks, provide implementation support to IFAD-funded operations and has the potential to enhance partnerships and policy dialogue in Ethiopia. Hence, the country presence should be further strengthened, so that it can play a greater role in enhancing IFAD’s development effectiveness in Ethiopia. .

D. Questions for the Forthcoming Country Programme Evaluation (CPE)

171. In view of the CPE for Ethiopia which is planned for 2007, a number of key questions arise from the present evaluation, which should be addressed in that context. These are:

37

• Have the present project identification, formulation and appraisal processes encouraged the setting of unrealistic targets and spreading project of activities too thinly? If so, how can these tendencies be avoided in future?

• Are the existing project identification, formulation and appraisal processes sufficiently participative, in a country in which participation is not a strong tradition? If not, can they be made more so?

• To what extent have the detailed capacities of project stakeholder institutions been routinely assessed at the formulation stage, in order to design appropriate capacity-building programmes? What improvements can be made to this process?

• What has been the impact of the BSF contributions in Ethiopia? How can the partnership be enhanced in future activities?

• How can the present model of project management and supervision be modified to create a significantly greater degree of the Programme Coordination Unit (PCU) authority and effectiveness, without de-coupling projects from the implementing institutions? Should separate project management structures be set up, perhaps using consulting firms, or can the existing PCU framework be made more effective? How could the field presence officer be more effective? What model best fits the Ethiopian context? Should clearer guidelines be framed, setting out responsibilities for taking actions on supervision and MTR recommendations? Where does the buck stop in terms of project management?; and

• In view of the weak performance of M&E in some Ethiopian projects, how can a monitoring culture be encouraged, and how can manageable frameworks be developed and implemented for monitoring of project performance, reflecting both quantitative and qualitative achievements?

38

APPENDIX 1

Summary of Project Objectives

Principal objective : increase agricultural productivity and raise income levels of the rural poor through support to Service Cooperatives’ development in order to facilitate efficient provision of sustainable services to members

SO1 Provide a model for developing Ethiopian cooperatives under the new legislation, particularly with respect to improvement of financial intermediation services in rural areas, which could be replicated in other areas of the country. O1.1 Service Cooperatives (SCs) restructured and registered under the new legislation O1.2 Short term technical assistance provided, to facilitate: - training of the staff - development of accounting systems in the cooperatives - development of auditing procedures to be followed by the Cooperatives Promotion Department (CPD) O1.3 Long term technical advisory support provided to SCs O1.4 The staff of SCs and Executive Committee members trained to enable them to effectively manage the SCs as business organizations O1.5 The members of SCs trained to enable them to actively participate in the decision making processes of the SCs O1.6 The CPD staff trained to reorient them in the operation of cooperatives in a free market economy and to increase their skills in supervision and audit of the SCs. SO2 Increased capital and income among the rural poor in the project area through off-farm income-generating activities particularly for women and families in densely populated areas with limited land for farm expansion O2.1 Growth of new Small Scale Enterprises (SSE) fostered by pilot projects to introduce new business ideas and providing vocational training O2.2 The amount and nature of credit services to SC members provided by SCs for SSE activities increased O2.3 New forms of rural women’s groups capable of promoting development activities and facilitating access to credit for their members promoted O2.4 Participating banks’ experience developed and cost effective methods for rural lending to reach small enterprises and rural women supported SO3 Strengthen the Southern Regional Agricultural Bureau (SRAB) to carry out its mandate with respect to cooperative development O3.1 SRAB’s structural capacity to promote SC development enhanced O3.2 SRAB’s capacity to monitor and evaluate SC development enhanced SO4 Provide credit to meet financial requirements for agricultural inputs and draught oxen and facilitate the supply of inputs through support to local traders and cooperatives. O4.1 A credit programme is implemented for increasing agricultural development, production and marketing and improving income levels of the rural poor O4.2 Technical Assistance (TA) to facilitate training of the CBE staff provided (Rural Banking expert for 3 months) O4.3 CBE staff trained in rural credit methodologies to improve their skills to effectively implement the credit programme O4.4 Transport facilities to CBE to facilitate effective implementation of the credit programme provided SO5 Relief of livestock health constraints, particularly with respect to draught animals, through provision of veterinary drugs. O5.1 Access to veterinary drugs improved SO6 Improve access of rural families to services and markets by rehabilitating and maintaining rural roads.

O6.1 Road construction equipment provided to Bureau of Public Works and Urban Development (BPWUD) O6.2 Improvement of rural road network and access roads to SCs is properly planned O6.3 250 km of the existing dry season roads rehabilitated to RR-30 standard O6.4 Heavy maintenance on 450 km dry season roads carried out O6.5 SCs regularly carrying out routine maintenance on the 700 kms of roads rehabilitated and heavily maintained SO7 (BSF Component) Reduce the burden of disease in 8 woredas of the SOCODEP area O7.1 Sustainable and effective water supply provided to 8 woredas of the SOCODEP area O7.2 Basic sanitation facilities provided and used by the 8 woredas of the SOCODEP area O7.3 Institutional capacity of the Regional Water, Mines and Energy Development Bureau (RWMEDB) and CBOs strengthened O7.4 Sustainable and effective health facilities provided to 8 woredas of the project area

39

40

APPENDIX 2

Project Costs and Financing 56

Budget Costs by Component (million US$, total 22.72m)

WSHBS Mgt & Training of SCs, Coordination, 0.95, 4% Water Supply & 0.17, 1% SSE Promotion, San, 2.28, 10% 0.29, 1%

Health & Nutrition, 1.57, 7% Support to SRAB, 2.80, 12% Roads, 4.70, 21% Credit line thro Veterinary Drugs, Support to CBE, CBE, 7.68, 34% 1.89, 8% 0.39, 2%

Actual Financing (million US$, total 19.65m)

Beneficiaries, 0.20, 1% GoE, 4.06, CBE, 0.46, 2% 21%

IFAD, 11.99, BSF, 2.94, 15% 61%

56 Source: Project Completion Report, June 2006

41

42

APPENDIX 3

SOCODEP Timeline

Event 91 92 93 94 95 96 97 98 99 00 01 02 03 04 05 06 IFAD Preparation Mission Pre-Appraisal Baseline Survey of SCs IFAD Appraisal Mission Report & Rec of the President to IFAD's EB Loans effective SLA for credit line between MOF and CBE Start-up Workshop Establishment of PCU at SRAB, CPD Formulation Mission for WSHBS (BSF) Component Mid-Term Review Move of PCU to Bureau of Cooperatives Start-up Workshop for BSF Component BSF Grant approved Move of PCU to BOFED Baseline Study for BSF Completed BSF Grant effective End of project, except Roads and BSF components Closure of SRS 037-ET and credit part of loan 342-ET Completion Report by GoE End of Roads and BSF components Closure of remaining loans BSF Impact Study BSF Grant closed IFAD/UNOPS Completion Report Mission Completion Evaluation

43

44

APPENDIX 4

Reconstructed Logframe for SOCODEP OVI Targets Data sources (star * means data still to be collected by evaluation team) Principal objective : increase agricultural For different population groups, the poor in particular, - Socio-economic survey (SES)* productivity and raise income levels of the rural members and non-members of SCs in the project area: - poor through support to Service Cooperatives’ Yields of main crops (maize, teff, sorghum, barley, development in order to facilitate efficient pulses and rape seed) provision of sustainable services to members - Product quality - Production cost - Labour productivity - Profitability of agriculture production - Income level SO1 Provide a model for developing Ethiopian - Number of financially sustainable restructured 200 - SRAB SC baseline survey; coop cooperatives under the new legislation, cooperatives staff survey (CSS)*; SC financial particularly with respect to improvement of reports 49 financial intermediation services in rural areas, - Cost of the cooperative development process - Financial reports which could be replicated in other areas of the - Volume of services provided to SC members (i.t.o. - CSS*; coop members survey country. (CMS)*; SC activity reports number of clients, value, units of inputs etc.) - CMS*; CSS* - Quality of services provided to SC members - Membership (number and nature) - CSS* - Nature and extend of replication of the SC - Cooperative promotion bureau development model (CPB) reports; national and regional - Government commitment to restructure SCs and gov key informants (KI)* carry trough policy changes - New laws and regulations in SNNPR; national and regional gov KI* O1.1 Service Cooperatives (SCs) restructured and - Nature and consequences of restructuring - CSS*; ‘94&’98 coop proclamations; registered under the new legislation - Consequences of registration CPB KI* A1.1.1 Restructuring of SCs - Restructuring process - CSS*; Project progress reports - Number of SCs restructured 200 (PPRs); CPB KI*; CPB reports

A1.1.2 Registration SCs - Registration process - CSS*; PPRs; CPB KI*; CPB - Number of SCs registered 200 reports; regional cooperative register A1.1.3 Appointment of paid SC - Job description for SC managers - CSS*; PPRs; CPB KI* managers in all restructured SCs - Selection process (cost partly reimbursed for 3 first - Number of SC managers appointed 200 years) O1.2 Short term technical assistance provided, to - Duration of presence in first year (months) 3 m - PPRs facilitate: - Duration of presence in second and fourth year 2 x 2 w - TA reports - training of the staff (weeks) - development of accounting systems in the - Application of improved accounting system within - CSS*; SC accountancy documents cooperatives SCs - development of auditing procedures to be - Application of improved audit procedures by CPD - CSS*; SC audit reports; Auditors* followed by the CPD A1.2.1 Recruitment of 2 training - Relevance of TOR - Updated TOR for TA specialists in cooperatives - Timely recruitment - PPRs management, supervision and

50 audit A1.2.2 Development of training - Adequacy of training needs assessment - Needs assessment report; CSS*; programmes - Training contents and design contributed by TA CMS* - Training documents A1.2.3 Preparation of curricula and - Technical soundness of curricula and materials - Training documents and materials; training materials prepared trainees* - Pedagogical quality of curricula and materials prepared A1.2.4 Training of trainers - Duration of TfT - TfT report - Quality of TfT - Trainees* - Number of trainers trained - TfT report - Competence of trainers trained - Training reports; Trainees* A1.2.5 Development of accounting - Technical soundness of procedures - Accounting manual procedures for cooperatives - Applicability of procedures - CSS*; SC accountancy documents; SC accountants*

A1.2.6 Development of audit procedures - Technical soundness of procedures - Audit manual for the cooperative auditors - Applicability of procedures - CSS*; SC audit reports; CPB auditors* A1.2.7 Follow up of SCs and CPD - Nature and frequency of follow up - TA reports operations - CPB KI* A1.2.8 Provision of guidelines for - Technical soundness of guidelines - Guidelines refresher courses - Applicability and pedagogical quality of guidelines - Trainees* O1.3 Long term technical advisory support - Duration of presence 2 years - TA report provided to SCs A1.3.1 Recruitment of a Cooperatives - Relevance of TOR - TOR Expert - Timely recruitment - TA report A1.3.2 Guidance and technical advice to - Relevance, quality and adequacy of guidance and - TA report; CSS*; CPB KI* SCs in all aspects of their advice given operation A1.3.3 Technical support to CPD - Relevance, quality and adequacy of technical support - TA report; CPB KI* given

51 A1.3.4 Half-year meetings between SC - Frequency and quality of the meetings 2 per - Meeting reports; PPRs; TA reports; presidents to discuss operational - Number of participants year SC activity reports issues - Relevance of issues discussed and solutions found - CSS*; CPB KI* A1.3.5 Develop a platform for - Platform/apex body development process - TA report; CPB KI* organization of an apex body - Nature of the platform - CPB KI*; CSS* - Existence and nature of the apex body - CPB KI*; Apex body documents O1.4 The staff of SCs and Executive Committee - Management capacity of SC staff and EC members - CSS*; CMS* members trained to enable them to effectively manage the SCs as business organizations O1.4.1 Organize 4 training courses for - Training contents and design - Training preparatory documents and 200 SC managers consisting of a 3 - Number and duration of induction courses organized 4 x 3m materials; trainees* months induction course - Number and nature of participants 200 - PPRs - Training reports

O1.4.2 Organize 5 day refresher courses - Quality of refresher training needs assessment - Needs assessment report; trainees* during subsequent years for SC - Training contents and design - Training preparatory documents and managers - Number and duration of refresher courses organized materials; trainees* - Number and nature of participants - PPRs - Training reports O1.4.3 Train other SC professional staff - Number, duration and nature of courses organized - Training preparatory documents and including storekeepers, - Number and nature of participants materials; trainees*; PPRs bookkeepers and shopkeepers - Training reports O1.4.4 Organize a study tour for 10 - Number of participants on the tour 10 - Tour report; PPRs officers from the CPD and SC - Relevance of visits - Tour preparatory documents; Tour managers to neighboring countries - Quality of the organization participants* - Tour participants* O1.4.5 Sensitize and train 200 SC - Sensitization/training contents and design - S/T preparatory documents and executive committee members - Nature, number and duration of s/t activities materials; S/T reports; PPR organized - Number and nature of participants 1000 - CSS* O1.5 The members of SCs trained to enable them - Number and nature of SC members present at - CSS*; CMS*; SC activity reports

52 to actively participate in the decision General Assemblies making processes of the SCs - Nature of their participation during the GAs - CMS*; CSS* - Nature and frequency of other forms of participation - CMS*; CSS* - SC members’ understanding of SC objectives, - CMS*; CSS* structure and functioning - SC members’ understanding of their role in SCs’ - CMS*; CSS* decision making processes

A1.5.1 Sensitization/orientation of all SC - Sensitization contents and design - Sensitization preparatory documents members - Nature, number and duration of activities organized and materials - Number and nature of participants - Sensitization reports; PPR O1.6 The CPD staff trained to reorient them in - Knowledge and skills of CPD staff for managing - CMS*; CPB KI* the operation of cooperatives in a free cooperatives in a free market economy market economy and to increase their skills - Supervision and audit skills of CPD staff - Idem in supervision and audit of the SCs.

A1.6.1 Organize initial orientation - Quality of the WS (organization, interest of subjects, - Workshop preparatory documents workshop for all senior staff effectiveness of activities…) and materials; WS report; WS - Number and nature of participants participants*; CPB KI* - WS report; PPRs A1.6.2 Organize orientation training - Orientation training contents and design - Training preparatory documents and course for 15 days for all staff in - Number and nature of participants 158 materials; trainees* the CPD - Duration of the training - Training reports, PPRs A1.6.3 Organize 2 month training courses - Audit training contents and design - Training preparatory documents and for 5 persons appointed as - Number and nature of participants 5 materials; trainees* cooperative auditors - Duration of the training 2m - Training reports, PPRs A1.6.4 Organize 10 days orientation - Audit orientation training contents and design - Training preparatory documents and course for 23 existing cooperative - Number and nature of participants 23 materials; trainees* auditors - Duration of the training 10d - Training reports, PPRs A1.6.5 Organize 15 days orientation - Orientation training contents and design - Training preparatory documents and course for 33 cooperative teams - Number and nature of participants materials; trainees* - Duration of the training 15d - Training reports, PPRs

53 A1.6.6 Organize orientation course in - SSE management training contents and design - Training preparatory documents and business management of small - Number and nature of participants materials; trainees* enterprises for zonal and woreda - Duration of the training - Training reports, PPRs cooperative promotion marketing officers A1.6.7 Organize 5 day refresher course - Quality of refresher training needs assessment - Training needs assessment report; during each year of project - Training contents and design trainees* implementation to all cooperative - Number and duration of refresher courses organized - Training preparatory documents and promotion staff and Rural - Number and nature of participants materials; trainees* Womens’ Affairs staff - Training reports, PPRs A1.6.8 Organize a study tour by 10 - Number of participants on the tour 10 - Tour preparatory documents officers from the CPD and SC - Relevance of visits - Tour report managers to neighbouring - Quality of the organization - Tour participants* countries

SO2 Increased capital and income among the rural poor - Change in assets and income through the - SES* in the project area through off-farm income- development of SSEs among different population generating activities particularly for women and groups families in densely populated areas with limited - Change in assets and income through the - Idem land for farm expansion development of off-farm IGAs among different population groups O2.1 Growth of new small scale enterprises - Change in scope, nature and profitability of SSEs - SES*; CSS*; PPRs (SSE) fostered by pilot projects to introduce among different population groups new business ideas and providing vocational - Number and nature of SSEs created or supported 170 - PPRs, SSEs* training - Number of SSE employees 2100 - SSEs* A2.1.1 Information to beneficiaries - Nature of information provided - PPRs - Number of recipients - CMS* A2.1.2 Study tour to India for 2 RTU - Number of participants on the tour 2 - Tour report; PPRs agents to investigate appropriate - Relevance of visits - Tour preparatory documents; tour commercial operations - Quality of the organization report; tour participants* - Lessons learned and used for project support - PPRs A2.1.3 Basic business training from ILO - Quality of training needs assessment - Training needs assessment report; 54 for selected RWAD and CPD staff - Training contents and design trainees* - Number and duration courses organized - Training preparatory documents and - Number and nature of participants materials; trainees* - Training reports, PPRs A2.1.4 National and international study - Number of participants on the tours 53 - Tour report; PPRs tours for entrepreneurs - Relevance of visits - Tour preparatory documents; tour - Quality of the organization report; tour participants* A2.1.5 3 Pilot SSE projects in 5 woredas - Quality of baseline survey - Baseline survey report - Number and nature of pilot SSE projects 3 - PPRs; SSEs* - Setting up process of the pilot SSEs - CPB and SC KI* - Quality of pilot evaluation and lessons learned from - Pilot evaluation report pilot SSEs - PPRs; CPB KI*; SSEs* - Dissemination process of pilot experiences A2.1.6 Investigation of potential to - Duration of TA 1,5m - TA report develop spinning and weaving - Quality of investigation report - Investigation report - Dissemination and use of investigation results - PPRs; CPB KI*

A2.1.7 Vocational training to - Quality of training needs assessment - Training needs assessment report; entrepreneurs/artisans (480 - Training contents and design trainees* months) - Number and duration of courses organized 215 - Training preparatory documents and - Number and nature of participants materials; trainees* - Training reports, PPRs O2.2 The amount and nature of credit services to - Amount and nature of credit services provided by 2100 - SCs financial reports; CBE reports; SC members provided by SCs for SSE SCs for SSE activities CBE KI*; MFI reports; MFI KI* activities increased - Number and nature of credit recipients - Idem; CMS*; SSEs* A2.2.1 Information to beneficiaries - Nature of information provided - CMS* - Number of recipients O2.3 New forms of rural women’s groups capable - Forms of women’s groups joining SCs promoted - CSS*; women’s groups* of promoting development activities and - Number and nature of women joining these women’s - Women’s groups* facilitating access to credit for their groups members promoted - Volume of credit extended to women’s groups’ - Women’s groups*; SCs financial members reports; CBE reports - Nature and volume of development activities - Women’s groups* undertaken by women’s groups - Women’s groups*; RWAD KI*

55 - Competence of RWAD staff to support women’s groups A2.3.1 Information to beneficiaries - Nature of information provided - Women’s groups*; SES* - Number of recipients A2.3.2 Promotion of formation of - Women’s groups formation process - Women’s groups*; RWAD KI* women’s groups - Women’s groups formation promotion process - Women’s groups*; RWAD KI* - Number and nature of groups formed - CSS*; RWAD KI* A2.3.3 Training of RAWD staff - Training contents and design - Training preparatory documents and - Number of courses organized materials; trainees* - Number and nature of participants - Training reports, PPRs - Duration of the courses O2.4 Participating banks’ experience developed - Changes in loan recovery - SC financial reports; CBE reports; and cost effective methods for rural lending - Changes in cost of rural credit CBE KI*; MFI reports; MFI KI* to reach small enterprises and rural women supported No activities specified at appraisal - -

SO3 Strengthen the SRAB to carry out its mandate - SRAB’s approach to promote cooperative - CPB KI*; CSS* with respect to cooperative development development - SRAB’s performance to promote cooperative development O3.1 SRAB’s structural capacity to promote SC - PCU performance - PPR*; CPB KI* development enhanced - SRAB’s structural cooperative promotion capacity A3.1.1 Establishment of PCU (1 PC, I - Composition of the PCU - PCU members*; CPB KI* FC, I PMEO, I RWA, 3 CPD staff, - Clarity of roles of the PCU and its members 2 auditors, 3 support staff) - Competence of the PCU members to fulfill their roles - Adequacy of resources for the PCU A3.1.2 Acquisition of transport, office - Number and nature of vehicles acquired - PCU members*; CPB KI* and communication equipment - Number and nature of office and communication equipment acquired - Adequacy of equipment acquired - State and maintenance of the equipment acquired O3.2 SRAB’s capacity to monitor and evaluate - Information collected, analyzed and disseminated for - M&E reports SC development enhanced management and steering 56 - PPRs; CPB KI* - Use of information collected for management and steering A3.2.1 Development of an M&E manual - M&E indicators developed - M&E indicator list - Availability and quality of M&E manual - M&E manual - Use of M&E manual - CPB KI*; M&E reports A3.2.2 Baseline survey - Availability and quality of baseline survey - Baseline survey report - Baseline survey report A3.2.3 Collection, analysis and - Regularity and adequacy of monitoring reports at - M&E reports dissemination of monitoring data different levels A3.2.4 Follow-up evaluation studies - Number and nature of studies - Evaluation study reports; PPRs

SO4 Provide credit to meet financial requirements for - Amount of credit provided - SC financial reports; CBE reports; agricultural inputs and draught oxen and - Destination of the credit provided CBE KI*; MFI reports; MFI KI*; facilitate the supply of inputs through support to - Reimbursement rate CSS*; SES*; local traders* local traders and cooperatives. O4.1 A credit programme is implemented for - Nature and number of households that received credit - SC financial reports; CBE reports; increasing agricultural development, - Nature and volume of credit extended CBE KI*; MFI reports; MFI KI*; production and marketing and improving - Nature and volume activities financed by credit CSS*; SES*; local traders* income levels of the rural poor A4.1.1 Provision of funds for credit line - Availability of funds to CBE and other MFI - CBE reports; CBE KI*; MFI reports; MFI KI* A4.1.2 Preparation of annual lending - Annual lending programmes - Annual lending programmes programmes A4.1.3 Appraisal of loan proposals - Number and nature of loan proposals - CBE reports; CBE KI*; MFI - Number and nature of accepted loans reports; MFI KI*; SC financial reports A4.1.4 Follow-up of loans - Regularity and nature of follow-up - CBE reports; CBE KI*; MFI reports; MFI KI* A4.1.5 Monitoring of credit programme - Regularity and nature of monitoring of the credit - CBE reports; CBE KI*; MFI 57 implementation programme implementation reports; MFI KI*; PPRs O4.2 TA to facilitate training of the CBE staff - Quality of training needs assessment - Training needs assessment report; provided (Rural Banking expert for 3 - Training contents and design contributed by TA trainees* months) - Quality of trainings given by TA - Training preparatory documents and - Number and nature of participants materials; trainees* - Training reports, PPRs No activities specified at appraisal - - O4.3 CBE staff trained in rural credit - Skills of CBE staff to implement credit programme - CSS*; CBE KI* methodologies to improve their skills to effectively implement the credit programme A4.3.1 Training of 15 CBE loan officers - Training contents and design - Training preparatory documents and - Number of courses organized materials; trainees* - Number and nature of participants - Training reports, PPRs - Duration of the courses O4.4 Transport facilities to CBE to facilitate - Number and nature of transport means provided - CBE KI*; PPRs effective implementation of the credit programme provided

SO5 Relief of livestock health constraints, particularly - Animal heath situation in the project area - SES* with respect to draught animals, trough provision of veterinary drugs. O5.1 Access to veterinary drugs improved - Availability of veterinary drugs in the project area - SES* - Access to veterinary drugs by animal breeders - Use of veterinary drugs A5.1.1 Preparation of annual demand - Survey design - Survey design documents estimates - Survey results (data collected and analyzed) - Survey report A5.1.2 Establishment of revolving fund - Volume of the fund established - PPRs; NBE KI*; CSS* account held by NBE - Functioning of the fund A5.1.3 Distribution of drugs. - Volume and nature of drugs distributed - PPRs; CSS* SO6 Improve access of rural families to services and - Number of SCs benefiting from rehabilitated roads 45 - CSS*; PPRs markets by rehabilitating and maintaining rural - Number of SCs benefiting from heavy maintenance 100 - Idem roads. on roads - Project financial reports; BPWUD - Cost of road improvement works financial reports - Sustainability of road improvements - Roads’ actual appearance*

58 O6.1 Road construction equipment provided to - Type and number of equipment provided - Equipment*; PPRs BPWUD A6.1.1 Preparation of specifications for - Tender and procurement documents - Tender and procurement documents procurement of road construction equipment A6.1.2 Procurement of equipment - Number of replies from sellers - Procurement files - Selection process - Procurement manual; BPWUD KI* - Timely delivery of equipment - BPWUD KI*; PPRs O6.2 Improvement of rural road network and - Correspondence of plans with project objectives - BPWUD KI* access roads to SCs is properly planned - Feasibility of plans A6.2.1 Preparation of the roads - Programming approach - Programme preparatory documents; rehabilitation and maintenance - Roads rehabilitation and maintenance programme BPWUD KI* programme - Programme O6.3 250 km of the existing dry season roads - Length of dry season roads rehabilitated 250 km - BPWUD reports; PPRs; Roads’ rehabilitated to RR-30 standard - Nature of rehabilitation work actual appearance* A6.3.1 Preparation of the rehabilitation - Plans drawn - Plans and budget documents works - Budget estimated

A6.3.2 Supervision of rehabilitation - Number of visits to the works - Rehab supervision reports; BPWUD works - Reporting and feed-back process reports; BPWUD KI* O6.4 Heavy maintenance on 450 km dry season - Length of roads on which heavy maintenance has 450 km - BPWUD reports; PPRs; Roads’ roads carried out been carried out actual appearance* - Nature of heavy maintenance works A6.4.1 Preparation of heavy and routine - Plans drawn - Plans and budget documents maintenance works - Budget estimated A6.4.2 Supervision of heavy and routine - Number of visits to the works - Rehab supervision reports; BPWUD maintenance works - Reporting and feed-back process reports; BPWUD KI* O6.5 SCs regularly carrying out routine - Actual state of the improved roads - BPWUD reports; PPRs; Roads’ maintenance on the 700 kms of roads - Regularity and nature of maintenance actual appearance* rehabilitated and heavily maintained - Cost of maintenance - Degree of involvement of SCs in maintenance - CSS* A6.5.1 Preparation of routine - Maintenance plans - Plans maintenance works

59 A6.5.2 Supervision of routine - Number of visits to the works - Rehab supervision reports; BPWUD maintenance works - Reporting and feed-back process reports; BPWUD KI*

Added components with BSF grant funding since 1999 (as of Grant Agreement): OVI Targets Data sources (star * means data still to be collected by evaluation team) Principal objective: enhance social and economic - Reduction of water borne disease - Socio-economic survey (SES)* standards through improved health and nutritional status - Improved nutritional status - Available government health of poor rural households in 8 woredas of the SOCODEP - Improved quality and quantity of assets figures area - Increased social/human capital through knowledge and technology transfer SO7 Reduce the burden of disease in 8 woredas of the - Increased use of safe water and health - PPRs SOCODEP area services - Project M&E - Changes in behaviour due to increased - SES* recognition of good health and hygiene - Regional Water, Mines and Energy - Changes in water usage Development Bureau (RWMEDB) - Improved institutional capacity to provide figures * water and health services

O7.1 Sustainable and effective water supply - Reliability of safe water source - PPRs provided to 8 woredas of the SOCODEP area - Accessibility of safe water source - Project M&E - Changes in time spent collecting water - Changes in quality and quantity of water - A7.1.1 Construct hand-dug wells - Number of wells constructed 90 - PPRs - Project M&E

A7.1.2 Protect additional springs - Number of springs protected 90 - PPRs - Project M&E

A7.1.3 Rehabilitate hand-dug or drilled - Number of rehabilitated or drilled wells 40 - PPRs wells - Project M&E

A7.1.4 Rehabilitate springs - Number of rehabilitated springs 35 - PPRs

60 - Project M&E

O7.2 Basic sanitation facilities provided and used by - Sanitation facilities utilized - Health centre records * the 8 woredas of the SOCODEP area - Changes in health practices - PPRs - Project M&E

A7.2.1 Construct refuse pits and VIP latrines - Number of pits and latrines constructed 1,400 (1,200 - PPRs of these for - Project M&E demonstration) A7.2.2 Conduct public awareness campaigns - Number of trainings and quality - Training documents and trainee to promote improved habitat practices - Number of beneficiaries trained surveys * - PPRs - Project M&E

O7.3 Institutional capacity of the Regional Water, - Water supply and sanitation knowledge and - SES* Mines and Energy Development Bureau skills of RWMEDB and CBOs enhanced - Available RWMEDB figures (RWMEDB) and CBOs strengthened - Changes in services provided - CBO records *

A7.3.1 Provide Training - Number of trainings and quality - Training documents and trainee - Number of people trained surveys * - Number of community water and health - PPRs centre committees established and still - Project M&E functioning A7.3.2 Assessment of water capacity of the - Conduct hydro-geological studies - Available RWMEDB figures area O7.4 Sustainable and effective health facilities - Accessibility of health facilities - Health centre records* provided to 8 woredas of the project area - Changes in use of health facilities - SES* (particularly women and children) A7.4.1 Renovate, upgrade and equip health - Number of centres upgraded 2 - PPRs centres - Project M&E - Health centre records*

Equip and supply w/ drugs additional 61 A7.4.2 - Number of centres supplied 6centres - PPRs health centres and health stations 2stations - Project M&E - Health centre records*

A7.4.3 Provide and train medical and - Number of trainings - Training documents and trainee administrative staff for these primary - Number of people trained surveys * health care facilities - PPRs - Project M&E

A7.4.4 Establish a revolving drug fund at - Number of funds established - PPRs each facility, including the supply of - Number of beneficiaries participating - Project M&E impregnated mosquito nets - Use of bed nets - Health centre records* - SES*

A7.4.5 Strengthen the institutional capacity - Number of trainings and quality - Training documents and trainee of the RHB - Number of people trained surveys * - PPRs - Project M&E

62

APPENDIX 5

Training Carried Out by SOCODEP (Source PCR, June 2006) No. of Persons Type of Training Function of Trainee Place Trained Year Duration Cost Trained International training Financial training Senior bank expert CBE UK 1 1998 5 weeks Project planning Experts in cooperative office Belgium 2 1999 3 months Project feasibility studies M&E Windhoek (Namibia) 1 May 2000 18 days Financial Controller and Project management Rome and Turin 2 June 2000 1 month Project Coordinator Financial controller and Loan administration Nairobi UNOPS 3 2000 1 week project coordinator Financial Controller and Loan administration Nairobi UNOPS 3 2001 1 week Project Coordinator Financial controller and Loan administration Nairobi UNOPS 2 2003 1 week project coordinator Cooperative training Cooperative experts Kenya, Nairobi 20 2002 1 month 63 Project management Credit officers of CBE UK Bradford University 2 Sept./01 3 months Project planning, monitoring and Project coordinator UK Bradford University 1 Sept./01 18 months evaluation Water engineering Focal person Netherlands 1 Sept./03 18 months Project planning, monitoring and Project coordinator UK Bradford University 1 Sept./03 18 months evaluation Bureau head and experts in 2004 (2) and 24 months Distance learning UK financial management 11 BOFED 2005 (9) and above Water Experts in Regions Water engineering India Hydrabad 9 Nov/04 15 days and zones Focal person from Health Environmental health UK Liverpool 1 Sept./03 18 months Dept PCU, FC and cooperative IFAD implementation workshops Zimbabwe 10 2000 1 week head of zone and region PCU, FC and experts from IFAD implementation workshops Madagascar 7 2001 1 week the regional bureau PCU, FC and experts from IFAD implementation workshops Zambia 6 2002 1 week the regional bureau

PCU, FC and BOFED IFAD implementation workshops Mozambique 4 2003 1 week Bureau heads PCU, FC and BOFED IFAD implementation workshops Lesotho 4 2004 1 week Bureau heads IFAD implementation workshops PCU and FC Rwanda 2 2005 1 week Bureau heads water and Exchange visit Tanzania 4 2001 10 days health Training in M&E and impact assessment PCU and FC Tanzania 2 2005 10 days

Domestic training Bureau Heads, PCU, IFAD implementation workshops financial controller and Addis Abeba 5 1999 5 days M&E Training of water staff Experts zones and woredas 200 2005 Refreshment training Cooperative staff zones and woredas 476 2002 Cooperative managers and Refreshment training zones and woredas 186 2002 64 shopkeepers Organisation and management, marketing, Cooperative staff zones and woredas 574 2002 credit and saving cooperative principles Experience sharing Committee members western zones 458 2002 1 week Orientation about cooperatives Cooperative members Service cooperatives 789 2002 1 day Management of animal health Veterinarians Zones 70 2002 Health and sanitation committee training Village committees Zones 102 2002 3 days 48,490 Hygiene education Community members Zones 27,341 2002 2 days Women farmers training Female farmers Woredas 1,544 2001 - 119,400 Women affairs expert training Women’s affairs experts 201 2001 3 days Women’s group managers training Women managers of IGA project zones 54 2001 19,300 Field visits IGA experts project zones 130 cooperatives 2001 23,650 Veterinary technical training Veterinary experts Project zones 300 2001 25,000 Development agent training Development agents Project zones 126 2001 8,200 New experts training Cooperative experts Awassa 8 2001 Committee members exchange visits Cooperative members Woredas 122 2001 Peasant Association Cooperative members training Cooperatives 10,179 2001 members SC shopkeeper training Shopkeepers Cooperatives 36 2001

SC new shopkeepers training Shopkeepers Cooperatives 11 2001 SC new managers training Managers Cooperatives 39 2001 SC old managers training Managers Cooperatives 34 2001 Revolving drug fund training Health professionals Zone 34 2001 1 week Environmental health training Health professionals Zone 23 2001 106,030 Community health agents training Community health agents Zone 5 2001 152,860 Traditional birth attendants training Traditional birth attendants Zone 25 2001 Cooperative staff training Cooperative staff 98 2000 Training committee members Peasant association members 613 2000 Shopkeepers 35 2000 Orientation for cooperative members Cooperative members 11,970 2000 Committee member training Cooperative members 74 2000 Community health agents training Community health agents 5 2000 Traditional birth attendants 25 2000 Public health workers 28 2000 Community health workers training Community health workers 18 2000 Computer training Focal person of health 1 2000 65 Water supply management training Water officers 2 2000 Community organisers 5 2000 Water committees 13 2000 Water operators 15 2000 Water technicians and water committee 2000 training Staff water committee Tello woreda 42 Staff water committee Konso S. woreda 7 2000 Staff water committee Chencha 49 2000 Staff water committee Offa 21 2000 Staff water committee Sodo 42 2000 Staff water committee Damot Galle 14 2000 Bank and credit disbursement training CBE experts Awassa 10 2000 Cooperative educational training Cooperative staff Zone 149 1998 7 days Executive members Woreda 368 1998 3 days Cooperative employed 1998 5 days workers Woredas 118 Small scale enterprise training cooperative members Sodo town 149 1 1998 FC: Financial Controller, PCU: Project Coordination Unit, BOFED: Bureau of Finance and Economic Development, SC service cooperatives 1 Of those: 81 Small scale enterprises, 8 bamboo works, 7 weaving, 5 beehives, 48 tailoring

66

APPENDIX 6

Evaluation Mission Itinerary, 18 th September-11 th October 2006

Date Activities Mon 18 th Sept Full team assembles in Addis Ababa. Introductions, logistics, making appointments. Tues 19 th Sept Meetings with MoFED, RUFIP, VOCA, Obtaining fieldwork advance from UNDP. Travel to Awassa. Weds 20 th Sept Planning: familiarity with documentation, explanation of OE Evaluation Manual, understanding of changes at MTR, findings of Supervision missions, content of BSF component. Arranging meetings. Thurs 21 st Sept Meetings with Regional Bureau officials (BoARD) and OMFI. Field work planning and transport logistics. Introduction to Report structure. Fri 22 nd Sept Meetings with Roads Authority, Bureau of Water Resources. Sat 23 rd Sept Preparation of invitations for wrap-up meetings. Final field work planning. Sun 24 th Sept Team splits: Socio-economist to Gamo Team leader, Coops Specialist and Credit Mon 25 th Sept Gofa Zone and Derashe Special Woreda, Specialist to Addis Ababa. Tues 26 th Sept with Expert from Rural Women’s Affairs Meetings with VOCA, RUFIP, CBE and Team. Meetings with Woreda officials Cooperatives Agency. and Service Coops in Boreda, Chencha, Arba Minch Zuria and Derashe. Visits to BSF activities in Chencha, and roads in Chencha (Arba Minch-Zigity) and Boreda Wajifo-Boreda). Travel to Sodo. Weds 27 th Sept Socio economist travels to Jimma Team leader to Jimma, Coops Specialist and Credit Specialist to Arba Minch with Expert from Cooperatives Promotion office. Thurs 28 th Sept Team leader and Socio-economist in Coops and Credit Specialists to Gamo Gofa Fri 29 th Sept Kafa Zone. Discussions with Zonal Zonal offices, and Arba Minch Zuria, Chencha Sat 30 th Sept officials. Visits to Gimbo, Bita and and Boreda Woreda offices. Meetings with Decha Woredas. Visit to Gojeb-Argoba cooperatives in all three woredas. road. Visit to BSF component in Bita Woreda. Meeting with SUPAKS (now KDP) CTA. Travel to Jimma. Sun 1 st Oct Team leader and Socio-economist to Coops and Credit Specialists to Sodo. Sodo. Debriefing for all team members , and planning of remaining fieldwork Mon 2 nd Oct Team leader and Socio-economist to Coops and Credit Specialists to Damot Weyde Tues 3 rd Oct Wolayta Zonal offices, Ofa Woreda, BSF and Sodo Zuria Woredas, and visiting water supply component and Mure cooperatives. Travel to Awassa. Health Post. Visit to Sodo RTC. Drafting of Aide Memoire. Travel to Awassa. Weds 4 th Oct Completion of Aide Memoire and despatch to IFAD. Meetings with SNNPR Government Thurs 5 th Oct officials and other stakeholders. Drafting of Working Papers. Fri 6 th Oct Sat 7 th Oct Preparation of presentations and final arrangements for wrap-up meetings. Meetings with Sun 8 th Oct other key informants. Mon 9 th Oct Wrap-up meeting for SNNPR Government and other Awassa-based stakeholders. Travel to Addis Ababa. Tues 10 th Oct Wrap-up meeting for wider membership of core learning partnership, Addis Ababa. Final team meeting. Weds 11 th Oct Team disperses, team leader to UK.

67

68

APPENDIX 7

Organisations and Individuals Interviewed

Organisation Name and Position Federal and national level MoFED Ato Fisseha, Director MoFED Ato Degene Demissie, International Financial Institutions Commercial Bank of Ethiopia Ato Simon Assefa, Manager SME & Agricultural Loan Division Ethiopian Cooperatives Agency Ato Bedru Dedgeba Ejabo, Deputy General Director IFAD Ato Abebe, Field Presence AEMFI Dr Wolday, Founding Director RUFIP Ato Bahiru Haile, Manager Programme Coordination Office RUFIP Ato Gedion Mekonnen, Monitoring and Evaluation Officer ACDI/VOCA Wzo Brutawit Davit Abdi, Country Representative ACDI/VOCA Ato Hine Hassenu, Head Programme Support Section ACDI/VOCA Ms Elizabeth Farmer, Ethiopia Programme Officer

Southern Region Government PCU Ato Shimekit Gebretsadik BOFED Ato Berigude Bancha Bagaje, Head Ato Mamo Ketti, Dept. Head Ato Legesse Hailemariam, PR Expert Ato Zemenu Gebre, Development Cooperative Expert Ato Shimekit Gebretsadik, PCU Coordinator Ato Demissie Gebre, Dept. Head Ato Alemu Belega, Team Leader Bureau of Agriculture & Rural Development Ato Altaye Aboret, Head, Planning and Civil Service Department Cooperative Promotion Sector Ato Birhanu Assfaw, Expert Cooperative Registration and Legal Affairs Ato Woldegiorgis Demore, Team Leader Cooperative Promotion Department (CPD) Ato Daniel Alemayehu, Acting Head Ato Daniel Tewolde, Saving & Credit Expert Rural Women Affairs Wzo. Belaynesh, Team leader Ato Fekadu Tadesse, Expert Ato Sisay, Expert Bureau of Water Resources Ato Menu Abas, Coordinator Bureau of Health Ato Demissie Bubamo, Environmental Health Team Leader Rural Roads Authority Petros Gordano, Planning and Programme Department Head Rural Roads Authority Ato Sisay Belachew Tadesse, Head, Road Maintenance Division

Regional Institutions OMFI Ato Worku, Acting Head OMFI Ato Worku G./Yohanes, Manager, Planning & Marketing, Research Department OMFI Ato Tamirayehu Meshesha, Auditor, Arba Minch Branch Rural Finance Service Fund (RFSF) Ato Mathewos Rika, Head RUFIP Ato Maru Argaw Unit Head, Rural Finance Intermediation Programme, Awassa

Gamo Gofa Zone Zonal Government Ato Feleke Meka, Cooperative Promotion Coordinator Arbaminch Zuria Woreda Cooperative Promotion Ato Alemayehu Gezahegn, Representative & Input Desk Arbaminch Zuria Woreda Omo Micro Finance Ato Tamirayehu Mersha, Auditor & Acting Head Institute Kola Dega Shara SC Ato Manaye Dercha, Accountant Chencha Woreda Cooperative Promotion & Input Ato Yissak Lema, Cooperative Training Expert Desk Ato Demissie Gebre, Input Loan Expert

69

Eazo Multi-purpose service cooperative Ato Balango Bassa, Vice Chairman Ato Banga Bale, Secretary Gaga Gocha Multi-purpose service cooperative Ato Alemayehu Bassaye, Chairman Ato Gauta Zema, Committee Member Assistant Vet, Chencha woreda Ato Dagmawi Mammo Acting Cooperative Head, Boreda woreda Ato Mateos Meskey Cooperative Organizer, Boreda woreda Ato Tika Wantela Cooperative Training expert, Chencha Woreda Ato Yishak Lemma Cooperative Loan expert, Chencha Woreda Ato Demissie Gebru Cooperative Loan Expert, Chencha Woreda Ato Lamrot Kassaye Cooperative Team Leader, Arba Minch Zuria Ato Alemayehu Gezahegn Woreda

Welayta Zone Zonal Government Ato Gebru Gebre Meskel, Cooperative Expert Zonal Government Ato Badi Jebessa, Animal and Fisheries Resources Department Expert Zonal Government Ato Surafe Seifu, Cooperative Team Leader Zonal Government Ato Dessalegn Dache, Water Resources Development Office Head Zonal Government Ato Melkam Tadesse, Agriculture/Rural Development head Zonal Government Elfnesh Adey CDC Coordinator Sodo RTC Principal Sodo RTC SOCODEP Focal Point Officer Wolaita Zuria Woreda Cooperative Promotion & Ato Petros Dinase, Representative Input Desk Ato Wube Gejerso, Input Expert Damote Woyde Woreda Cooperative Promotion Ato Nega Alemu, Head & Input Desk Ato Aklilu, Expert Girara Multi-purpose service cooperative Ato Tadele Getena, SC Member Ato Matiwos Zema, SC Member & Guard Sake Multi-purpose service cooperative Ato Geletu Mega, Secretary for Control Committee Wachiga Multi-purpose service cooperative Ato Lera Kobote Ex-Chairman (Now Kebele Chairman) Ato Mena Afamo, SC Member Kokate Multi-purpose service cooperative Ato Bogale Shiferaw, Accountant Ofa Woreda Ato Yacob Kaba, Cooperatives Desk Ofa Woreda Ato Bereket Bekele, Cooperatives Expert Yakima Cooperative Ato Mamo Tanga, Chair, and Members Mure Health Post Ato Fisseha Solomon, Nurse

Kaffa Zone Zonal Government Ato Teshager Semerab, Expert in Animal Health, Kefa Zone Zonal Government Ato Tashome Mamu, Expert, Cooperatives Desk Zonal Government Ato Negalign Berhanu, Head Health Department Zonal Government Ato Meseret Adeko, Head, Water Resources Department Gimbo Woreda Ato Alemayu Adare, Cooperatives Accountant Gimbo Woreda Ato Adamsu Tekolla, Assistant Veterinarian Argoba Cooperative Members Decha Woreda Ato Fokaru Mohammed, Cooperatives Expert Baha Cooperative Exec Committee and members Chena Woreda Ato Girma Giji, Animal Health Technician Chena Woreda Ato Tefere Agzo, Cooperatives Team Leader Chena Woreda Ato Kassahun Gezaw, Acting Head, Water Office Chena Cooperative Members Chena Woreda Ato Geramo Takele, Head, Health Office KDP (formerly SUPAKS) Mr Gerard Kiers, Chief Technical Adviser

Derashe Special Woreda Animal Health Team Leader, Derashe Woreda Mekonnen Tilahun Cooperative Team Leader, Derashe Woreda Birru Simalew

70

External PCR Team (met in May 2006) IFAD Phillips Young, Team Leader IFAD Miriam Cherogony, Rural Finance Specialist IFAD Fatima Mohammed Ali, Health Specialist IFAD Carl Lachat, Nutrition Specialist

Participants at Wrap-up Workshops Awassa Workshop OMFI Ato Worku Gebre Yohannes, Planning & business Development Manager Southern Region Roads Authority Ato Petros Godana, Planning and Programming Head CPAIS Ato Berhanu Asfaw, Cooperatives Promotion Expert CPAIS Ato Daniel Alemayehu, Cooperatives Promotion department Head CPAIS Ato Wolde Giorgis Dimore, Cooperatives Regulatory and Legal Affairs Department Head Regional Veterinary Team Ato Sisay Kifle Rural Womens Affairs Team Ato Fekadu Tadesse, Rural Women’s Expert Water Resources Bureau Ato Menu Abas, Coordinator BOFED Ato Mamo Ketti, Head NGOs and Development Cooperation Regional Bureau of Agriculture and Rural Wzo Belaynesh Gelaye Development SOCODEP/BSF Ato Shimekit G/Tsadick, Project Coordinator SRA Ato Shemsu Shafi, General Manager

Addis Ababa Workshop CHF Ms Leslie Gardiner, Program Advisor and Liaison Officer ECA Ato Bedru Dedgeba Ejabo, Deputy General Director

71

72

Appendix VIII

Bibliography

Association of Ethiopian Microfinance Institutions, AEMFI Occasional Paper No. 2, Dr. Wolday Amha, August 2002.

BSF/IFAD (1999) Baseline Survey Technical Document (Technical background of the Woreda baseline survey report and Reference for monitoring and evaluation). August 1999.

Central Intelligence Agency, The World Factbook – Ethiopia, 2005.

FAO, Alemayeju Mengistu, Country Pasture/Forage Resource Profiles, Ethiopia, July 2003.

FAO, Land and Water Division, AQUASTAT, Irrigation in Africa in figures, AQUASTAT Survey 2005.

IFAD (1993) Ethiopia Southern Region Cooperatives Development and Credit Project (SOCODEP) Appraisal Report Volume 1: Main Report & Annexes. Report No. 0498-ET, October 1993.

IFAD (1993) Report and Recommendation of the President to the Executive Board on a Proposed Loan to the Federal Democratic Republic of Ethiopia for the Southern Region Cooperatives Development and Credit Project including a Proposed Loan under the Special Programme for Sub-Saharan African Countries Affected by Drought and Desertification. EB 93/50/R.82/Rev.1, September 1993.

IFAD (1994) Loan Agreement (Southern Region Cooperatives Development and Credit Project) between Ethiopia and the International Fund for Agricultural Development. Loan No. 342-ET, 01 February 1994.

IFAD (1994) Loan Agreement (Southern Region Cooperatives Development and Credit Project) between Ethiopia and the International Fund for Agricultural Development. Loan No. SRS-37- ET, 01 February 1994.

IFAD (1997) Federal Democratic Republic of Ethiopia Southern Region Cooperatives Development and Credit Project (SOCODEP). Mid-term Review Report, 0722-ET, March 1997.

IFAD (1998) Financing Agreement (Southern Region Cooperatives Development and Credit Project – Water Supply, Health and Basic Sanitation Component) between the Federal Democratic Republic of Ethiopia and the International Fund for Agricultural Development. Grant No. BG- 29-ET, 10 November 1998.

IFAD Rural Poverty Portal, Rural Poverty in Ethiopia.

IFAD/BSF (2005) SOCODEP - Water Supply, Health and Basic Sanitation Component Supervision Mission Technical Report, June 2005.

IRIS Center, CGAP, Essays on Regulation and Supervision, Regulating Microfinance in Ethiopia: Making it more Effective, April 2005.

Letson, Perry, The road up: free-market reforms fuel growth of Ethiopia’s co-ops, U.S. Department of Agriculture, Rural Business – Cooperative Service, 2002.

SOCODEP/BSF (2005) Health Water Supply and Basic Sanitation Component Impact Assessment Study Results, December 2005.

73

SOCODEP/BSF Coordination Unit (2003) Southern Region Cooperatives Development and Credit Project Completion Report, September 2003.

The Economist Intelligence Unit, Country Report, Ethiopia, January 2006.

The Ministry of Finance and Economic Development (MOFED) and the United Nations Country Team, Millennium Development Goals Report: Challenges and Propects for Ethiopia, Vol. 1, Main Text, March 2004.

UNDP, Human Development Report, Ethiopia Country Fact Sheet, 2005.

UNOPS (2002) Southern Region Cooperatives Development and Credit Project (SOCODEP) Supervision Report, April 2002.

UNOPS (2003) Southern Region Cooperatives Development and Credit Project (SOCODEP) Supervision Report, February 2003.

UNOPS (2004) Southern Region Cooperatives Development and Credit Project (SOCODEP) Supervision Report, February 2004.

UNOPS (2005) Southern Region Cooperatives Development and Credit Project (SOCODEP) Supervision Report, June 2005.

USAID, Famine Early Warning Service Network (FEWSNET), Ethiopia Food Security Update, March 2006.

USAID, Famine Early Warning Service Network (FEWSNET), Southern Nation, Nationalities and People’s Region, Ethiopia, Livelihood Profiles, January 2006.

World Bank, Ethiopia- Interim Country Assistance Strategy, May 2006.

World Bank, Ethiopia, Well-being and Poverty in Ethiopia, The Role of Agriculture and Agency, Report No. 29468-ET, 18 July 2005.

World Bank, Memorandum of the President of the International Development Association to the Executive Directors on a Country Assistance Strategy for the Federal Democratic Republic of Ethiopia, 24 March 2003, Report No. 25591-ET.

World Press Review, Ethiopia’s Struggle over Land Reform, April 2004, Vol. 51, No.4).

74 PROJECT EVALUATION

Federal Democratic Republic of Ethiopia

Southern Region Cooperatives Development and Credit Programme

Completion Evaluation

November 2008 Via Paolo di Dono 44 - 00142 Rome, Italy Tel: +39 06 54592048 - Fax: +39 06 54593048 E-mail: [email protected] Web: www.ifad.org/evaluation