PCR EVALUATION NOTE

CHAD: SURFACE RUNOFF WATER ENHANCEMENT PROJECT IN , , GUERA AND OUADDAI (PVERS)

1. The Project

Project Code: Project Appraisal Date: Borrower’s Completion P-TD-A00-011 December 1997 Report Ref: NA Total Project Cost (UA): Project Approval Date: PCR Evaluation Note Date: 14.23 million 29 April 1999 August 2009 Loan Amount (UA): Date of Effectiveness: Evaluator Name: 11.9 million 15 September 1999 Awale TAF Grant Amount:(UA) Project Completion Date: Reviewer(s) Name: 0.51 million 31 December 2008 Panel Co-financed Amount (UA): PCR Date: Manager Name: NA May 2009 M.Manai

1.1 Objectives The objectives were to harvest and harness the surface runoff water in four prefectures of Batha, Biltine, Guera and Ouaddai for flood-recession farming, market gardening and livestock watering.

1.2 Project Results. (Expected outputs, outcomes and impact) Nine micro-dams (revised to eight) constructed and operational; ten boreholes (revised to open wells) constructed and equipped; 586 ha of cultivable land developed including 555 ha for flood recession cropping and market gardening area of 31ha; 143 km feeder roads built (revised to 55.5 km) and maintained; rain-fed farming intensifies on 3800 ha; training of beneficiaries (2,400 households) and 95 project staff; establish and strengthen 80 village groups and support them with cereal milling facilities; supervision means established and operational and credit system initiated and operational for agriculture inputs and income generating activities, environmental protection of an area of about 380 ha.

1.3 Activities Civil engineering works including topographical studies, community mobilization, construction of roads, input distribution, selection and agreements with operators for village and farmer organizations, mobilization of field extension workers and implementation of the training and extension and research program, selection and appointment of credit operator and establishment of credit system, erosion control through construction of dykes, bunds, dry- stone walls, and reforestation and grass hedges, maintenance of water points, monitoring incidence of water-born diseases, health education and awareness campaigns, drug supply to health centers and day to day management

1.4 Inputs The inputs consisted of goods and services including project management. These were: civil works contracts, credit for agriculture inputs, pharmaceutical and veterinary products, handicrafts, provision of equipment, tools, staffing, technical assistance and training, vehicles, motorbikes, office facilities, vehicles and motor bikes and staff operating costs.

1.5 Intended beneficiaries and Scope The primary beneficiaries are the communities residing in the four Prefectures of Biltine, Batha, Guera and Ouaddai and the Directorate of Agricultural Engineering and Irrigation in the Ministry of Agriculture (MoA).

1 2. PCR Conclusions and Success Ratings

2.1 Main Conclusions The main conclusions of the PCR contradict the success ratings accorded to project implementation, Bank performance and project results. In the text, implementation performances is described as unsatisfactory but under the overall assessment and rating the Borrower performance is rated as satisfactory. Deficiencies at preparation and appraisal stage and weak supervision in terms of rigor and skill mix reduce the rating of the Bank performance. One of the major impediments to effective implementation reported by the PCR was the frequent interference of the Ministry of Agriculture (MoA) in the choice of project personnel and the poor technical follow-up and supervision of MoA staff. The main instruments for increasing the chances of successful implementation and sustainability were the execution of the credit activity which was cancelled and the farmer group empowerment. Lack of agricultural input credit limited the possible cultivation of 3,800 ha of rain-fed crops and weakness in the development of farmer institutions undermined the potential for the achievement of project objectives especially increased crop production and enhanced food security and sustainability project benefits and services. At project end 93.7 of the total project funds where disbursed. About 96% and 81% of the ADF loan and TAF grant were disbursed respectively. About 79% of government contribution and 100% of the beneficiary contribution were utilized.

2.2 Performance Ratings The PCR rated all categories of the project as satisfactory. The overall performance of project outcomes was rated satisfactory with an overall rating of 2.8 out of 4 points. The implementation performance was assessed as satisfactory with a rating of 2.4 and the Bank performance as satisfactory with a rating of 2.75. This note does not concur with the PCR ratings and evaluated all categories as unsatisfactory across the board. The first three dimensions of project outcomes were evaluated as follows: relevance and Achievement of Objectives was rated as satisfactory with a rating of 2.5, institutional development and sustainability were rated unsatisfactory with a rating of 2 and 2.4 respectively. (See annex 2).

2.3 Lessons Learned The lessons itemized in the PCR are general statements on project management and implementation issues arising and are not relevant or of value to the Bank as lessons. Some are reformulated as follows: a) the importance of strengthening farmer institutions to increase the chances of successful implementation and sustainability, b) the importance of strong project management structure for effective project implementation; c) the fact that it is pointless to include a credit component in the design which is difficult to implement and eventually cancelled

3 Recommendations The PCR made useful recommendations. The Bank was encouraged to support the concept of carrying out feasibility studies as part of project preparation to improve the quality of project design. The Bank should rethink its policy on including agricultural input credit into projects when capacity to implement it is not in place or can’t be outsourced. At negotiations the Bank should insist on appropriate Staffing arrangement and appointment of a suitable project coordinator who can only be replaced with the consent of the Bank. The bank was also encouraged to revisit its supervision guidelines to ensure provision of dedicated quality technical supervision and loan administration service. The Borrower was encouraged to ensure completion of remaining activities to be able to make maximum potential use project benefits and services established to date and to continue maintaining the project infrastructure especially the roads and consider outsourcing government procurement functions and contract management to avoid implementation delays in future.

2 3.1 Future operational plan and potential benefits (sustainability, institutional development and overall success ratings) It is not indicated that future operational plan was prepared.

3. Borrower’s PCR (its inputs to Bank’s PCR) Borrower PCR is indicated in the list of the source of information but there is no statement on the date it was submitted and its quality in providing useful inputs to Bank staff PCR.

4. PCR quality ratings

4.1 Objectivity and soundness The retrospective PCR matrix is incomplete and does not include activities and inputs and also actual verifiable indicators on sector goals. All other aspects were adequately covered.

4.2 Project implementation All aspects of project implementation were covered adequately handled except for the Borrower PCR as part of the reporting requirements.

4.3 Project performance and results Judgment on project results is adequately covered although limited to physical outputs achieved in full, in excess or in part. No M&E system was established for data collection to access project outcomes

4.4 Social and environmental impacts Judgment on social impact is inadequate due to lack of supporting data. Environmentally the project was classified as category II. Mitigation measures described in the appraisal have been followed during the course of implementation.

4.5 Project Sustainability Judgment on sustainability is adequate. Clearly the cancellation credit component and poor government supervision and follow-up on the beneficiaries had negative effects on the anticipated level of project sustainability.

4.6 Bank and Borrower performance Judgment on both the Borrower and Bank performance was not sound. Both were evaluated as unsatisfactory.

4.7 Consistency of the PCR overall rating A number of sub-component indicators under the project outcomes component indicators were not rated. All the unrated sub-items were assessed and rated by the evaluator.

4.8 Analysis and Clarity of conclusions, lessons learned and recommendations Conclusions section is a too short and PCR wrongly concludes that the overall project performance was satisfactory. Lessons listed were a mixture of findings/issues and recommendations. The key recommendations and follow-up actions (Annex 4) are identical to the separate recommendations to the Bank and the Borrower and could have been written in expanded form for wider applicability.

5. Priority of Project for an activity of Performance Evaluation Report

A PPER is not recommended even though there are critical gaps in completing the PCR. Given the importance of agriculture and food security to the Bank and the Borrower and in view of the current food crisis situation, and the fact that only few dams were opened up in the 2008 rainy season cropping season when project was ending, case studies would be more appropriate. A thematic study of this type would enrich the lessons for the Bank’s future

3 operations.

4 Annex 1 PCR QUALITY ASSESSMENT AND RATING

Project Loan No: F/TCH/DEV-RUR/99/30 Title: Prefecture Rural Development Project Country: Sector: Agriculture/Rural

PCR EVALUATION CRITERIA RATING REMARKS (4-point scale) 1. Adequacy of analysis of Project goals, objective and 2 This section was adequately Formulation (including the verifiable indicators, analyzed except for the consistency with appraisal and subsequent revisions) actual verifiable indicators on sector goals related to the overall status of oilseed production and livestock including milk production. The activities and inputs are not included in the matrix 2. Adequacy of analysis of Project execution 3 Analysis and presentation of (including procurement issues, disbursements, all aspects of project Borrower’s reporting, and assessment of monitoring execution adequate. and evaluation achievements) Borrower PCR and financial performance indicators by component and activity missing. M&E system was assessed as inadequate. 3. Soundness of judgments on Project Performance and 3 Judgment on results limited Results (including operating results, economic and to physical outputs realized. financial and related conditions/covenants and their Project development fulfillment, institutional, performance of consultants, outcomes and impact could contractors, suppliers and other parties) not be assessed as flood recession cropping was only partly achieved due to delays in the construction of dams and the fact that input credit activity was cancelled. 4. Adequacy of analysis of social and environmental 3 Analysis of social impact impacts adequate as hard data is lacking. Project environmentally classified as category II and mitigations measures were addressed under civil works and support measures. 5. Soundness of judgments on project sustainability, 3 Judgment on project plan for future project operation’s phase and sustainability adequate. maintenance Lack of proper maintenance and poor technical

5 supervision will most likely put most of the facilities in place into disuse. 6. Soundness of judgments on Performance of the 2 PCR judgments on this not Bank and Borrower. sound. Both the Borrower and Bank performance were evaluated as unsatisfactory (see annex 2). 7. Consistency of Overall rating with individual rating 2 A number of sub-items in components the project outcomes component indicators have not been rated 8. Adequacy of analysis and clarity of conclusions, 2 Conclusions section is too lessons learned and recommendations short. Lessons are mixture of issues and findings. Recommendations and follow-up matrix contains identical recommendations already made to the Bank and Borrower 9. Other (Specify) NA Overall Rating 2.5 Satisfactory

OPEV and Country Department agree/disagree on Project Performance Rating Y/N

6

Borrower’s PCR and inputs to Bank Staff PCR (quality of Borrower’s PCR, reviews of project implementation issues, future operation plan, Borrower’s comments on PCR): Inadequate (See para 3) Conclusion : The quality of the PCR is evaluated as satisfactory with a rating of 2.5 out of 4 points. The project had mixed results. Retrospective matrix of PCR is poor and does not clearly address all elements in the log-frame hierarchy. Reporting on the projects physical performance indicators adequate. Details on project’s financial performance indicators by component and activity missing.

Major Issues of focus in the performance evaluation report: PPER is not recommended

Follow Up Action/Decision: NA

ANNEX 2

7 ASSESSMENT OF IMPLEMENTATION AND BANK PERFORMANCE AND PROJECT RESULTS/OUTCOMES Chad: Lac Prefecture Rural Development Project IMPLEMENTATION PERORMANCE Component Indicator PCR Evaluati Comments Rati on ng Rating (1-4) (1-4) 1. IMPLEMENTATION PERFORMANCE ASSESSMENT 1.1 Adherence to 1 1 Validated. Project was implementation was implementation schedule completed three years behind schedule. 1.2 Adherence to cost schedule 2 3 There was an insignificant cost over-run of UA 0.34 million due to slippage in implementation. The 9th well is being dug with government funds 1.3 Compliance with covenants 3 2 Government complied but with a delay of 17 and conditions months and also difficulties in fulfilling other conditions 1.4 Adequacy of monitoring 3 2 Proper M&E system not established. Quarterly evaluation and reporting reports submitted regularly 1.5 Satisfactory operations 2 2 The credit component which was essential for satisfactory operations was cancelled and not implemented Total Scores 11 10 Average Rating 2.2 2 Unsatisfactory

2. PERFORMANCE OF THE BANK Component Indicator Comments 2.1 Identification 3 3 Bank Identification mission 2.2 Preparation 3 2 Preparation based on feasibility study. Costs of certain activities were underestimated. 2.3 Appraisal 2 2 Appraisal did not adjust the cost underestimation. Project matrix inadequate 2.4 Supervision 3 2 Frequency was adequate but skill mix was lacking and did not include a credit specialist and farmer organizations expert. Total Scores 11 9 Average Rating 2.75 2.25 Unsatisfactory

3. PROJECT RESULTS/OUTCOMES

Component Indicator Comments 1. Relevance and achievement of objectives 1.1 Macro-economic policy 3 3 In line with macro-economic policy in food security and poverty reduction and also consistent with the Bank Strategy 1.2 Sector policy 3 3 Consistent with government sector policy 1.3 Physical (including 2.5 2 About 80% of the physical targets achieved. Due production) to cost escalation a number of activities were cancelled.

8 ANNEX 2 ASSESSMENT OF IMPLEMENTATION AND BANK PERFORMANCE AND PROJECT RESULTS/OUTCOMES Chad: Lac Prefecture Rural Development Project IMPLEMENTATION PERORMANCE Component Indicator PCR Evaluati Comments Rati on ng Rating (1-4) (1-4) 1.4 Financial aspect 2.5 2 No evidence of financial benefits yet 1.5 Poverty reduction, social 3 2 Positive effects reported from revenue generated impact and gender through market gardening with supplementary watering. About 95% of the market gardeners were women. Cancellation of the input credit will affect social impact and incomes negatively 1.6 Environment 3 3 Mitigation measures foreseen implemented 1.7 Private sector development NA NA Other than private contractor and suppliers, private sector participation was not foreseen 1.8 Other (Specify) NA NA Total Scores 17 15 Average Rating 2.8 2.5 Satisfactory

2. Institutional Development 2.1 Institutional framework 3 2 Weak institutional framework of PIU; No including restructuring competent institution was found for the credit component and farmer institutions not strong enough 2.2 Financial and integrated NA 2 Not rated by PCR. No FMIS in place but project systems of management annual accounts and 5 audits submitted some with including audit systems delays 2.3 Transfer of Technology NA 2 Not rated by PCR but limited technology transfer if any 2.4 Staffing by qualified/skilled 3 2 Staffing faced problems. Project manager was personnel (including replaced 3 times and unsuitable TA was recruited turnover), training and and later replaced counterpart staff. Total Scores 6 8 Average Rating 3 2 Unsatisfactory

3. Sustainability 3.1 Continued commitment of 3 3 Validated. Government showed commitment and borrower had funded the cost of the 9th well from own resources 3.2 Environmental policy 3 2 No indication that Government has environmental policy but mitigation measures foreseen in the design were implemented. 3.3 Institutional framework 3 2 Farmer institutions were not properly strengthened. 3.4 Technical viability and 3 2 Project was technically viable and staffing was staffing adequate but it is not indicated if project staff seconded are still linked with project activities 3.5 Financial viability ( NA 2 Not rated by PCR. Budget constraints affected the

9 ANNEX 2 ASSESSMENT OF IMPLEMENTATION AND BANK PERFORMANCE AND PROJECT RESULTS/OUTCOMES Chad: Lac Prefecture Rural Development Project IMPLEMENTATION PERORMANCE Component Indicator PCR Evaluati Comments Rati on ng Rating (1-4) (1-4) including cost-recovery) targets of the micro-dams, wells and feeder roads that were essential to financial viability including cost recovery 3.6 Economic viability 3 3 Validated. PCR judgment based on income from tomatoes and sorghum through cultivation under residual moisture. The influx of Sudanese refugees and NGOs resulted in increased demand for fresh market produce 3.7 Environmental viability 3 3 Environmental mitigation measures were put in place properly 3.8 O & M facilitation (foreign 1 2 Inadequate organization of the beneficiaries and exchange and recurrent cost the unlikely government budgetary support will financing availability,etc) negatively affect the chances of O&M facilitation and cost recovery Total Scores 19 19 Average Rating 2.4 Unsatisfactory 2.7 4. Economic rate of return NA NA EIRR calculated at 11% by PCR against appraisal estimate of 11.6%. No recalculated by evaluator OVERALL TOTAL 42 42

OVERALL RATING (1-4 above) 2.8 2.4 Unsatisfactory

10