The World Bank

Document of The World Bank

Public Disclosure Authorized Report No: ICR2847

IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF-57324)

ON A

Public Disclosure Authorized GRANT

IN THE AMOUNT OF (US$ 14 MILLION EQUIVALENT)

TO THE

SUDAN

FOR A

DECENTRALIZED HEALTH SYSTEM DEVELOPMENT PROJECT Public Disclosure Authorized

December 18, 2013

Health and Population Unit: Eastern and Southern Africa (AFTHE) Human Development Department Sudan Country Department (AFCE4)

Public Disclosure Authorized

CURRENCY EQUIVALENTS

Currency Units: Sudanese Dinar (SDD) Sudanese Pound/Sudanese Guinea (SDG) At Appraisal (October, 2006) SDD 1.00 = US$ 0.0048 US$ 1.00 = SDD 207 At Completion (June, 2013) SDG 1.00 = US$ 0.18 US$ 1.00 = SDG 5.5

FISCAL YEAR October - September

ABBREVIATIONS AND ACRONYMS

CBS Central Bureau of Statistics CDF Community Development Fund CPA Comprehensive Peace Agreement CPAR Country Procurement Assessment Report DHSDP Decentralized Health System Development Project EDS Environmental Data Sheet EIA Environmental Impact Assessment EMP Environmental Management Plan ESMF Environmental and Social Management Framework EOC Emergency 0bstetric Care FFAMC Fiscal and Financial Allocation and Monitoring Commission FM Financial Management FMR Financial Monitoring Report FPIU Federal Project Implementation Unit FPP Final Project Proposal GDP Gross Domestic Product GoS Government of Sudan HIV/AIDS Human Immunodeficiency Virus/Acquired Immune Deficiency Syndrome HMIS Health Management Information System HRH Human Resources for Health IDA International Development Association ITN Insecticide Treated Nets ISR Implementation Supervision Report JAM Joint Assessment Mission LLIN Long-Lasting Insecticidal net M&E Monitoring and Evaluation MDGs Millennium Development Goals MDTF Multi-Donor Trust Fund MoFNE Ministry of Finance and National Economy MoH Ministry of Health MWMP Medical Waste Management Plan NGO Non-Governmental Organization NHA National Health Accounts NHIF National Health Insurance Fund O&M Operations and Maintenance

PDO Project Development Objective PHC Primary Health Care PIU Project Implementation Unit RDF Revolving Drug Fund RH Rural Hospital SDD Sudanese Dinar SDG Sudanese Guinea (Sudanese Pound) SMC Safeguards Management Committee SMoH State Ministry of Health SOE Statement of Expenditure SPIU State Project Implementation Unit SPLA Sudan People’s Liberation Army UNAIDS Joint United Nations Programme on HIV/AIDS UNFPA United Nations Population Fund UNICEF United Nations Children’s Fund US$ United States Dollar VMW Village Midwife WHO World Health Organization

Vice President: Makhtar Diop Country Director: Bella Deborah Mary Bird Sector Manager: Olusoji O. Adeyi Project Team Leader: Isabel Cristina Soares ICR Team Leader: Isabel Cristina Soares

SUDAN

Decentralized Health System Development Project

Table of Content

DATA SHEET A. BASIC INFORMATION B. KEY DATES C. RATINGS SUMMARY D. SECTOR AND THEME CODES E. BANK STAFF F. RESULTS FRAMEWORK ANALYSIS G. RATINGS OF PROJECT PERFORMANCE IN ISRS H. RESTRUCTURING I. DISBURSEMENT GRAPH

1. PROJECT CONTEXT, DEVELOPMENT OBJECTIVES AND DESIGN 1 2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES 10 3. ASSESSMENT OF OUTCOMES 17 4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME 30 5. ASSESSMENT OF BANK AND BORROWER PERFORMANCE 32 6. LESSONS LEARNED 34 7. COMMENTS ON ISSUES RAISED BY GRANTEE/IMPLEMENTING AGENCIES/DONORS 35

ANNEXES:

ANNEX 1: PROJECT COSTS AND FINANCING 36 ANNEX 2. OUTPUTS BY COMPONENT 36 ANNEX 3. ECONOMIC AND FINANCIAL ANALYSIS 57 ANNEX 4. GRANT PREPARATION AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESSES 63 ANNEX 5. BENEFICIARY SURVEY RESULTS 64 ANNEX 6. STAKEHOLDER WORKSHOP REPORT AND RESULTS 64 ANNEX 7. SUMMARY OF GRANTEE'S ICR AND/OR COMMENTS ON DRAFT ICR 65 ANNEX 8. COMMENTS OF COFINANCIERS AND OTHER PARTNERS/STAKEHOLDERS 87 ANNEX 9. LIST OF SUPPORTING DOCUMENTS 87

MAP 90

DATA SHEET

A. Basic Information Multi-Donor Trust Sudan Project Name: Country: Fund – Northern Sudan Project ID: P098483 L/C/TF Number(s): TF-57324 ICR Date: 12/18/2013 ICR Type: Core ICR Grantee: Republic of Lending Instrument: ERL SUDAN Sudan Original Total USD 6.00M Disbursed Amount: USD 13.87M Commitment: Revised Amount: USD 14.00M Environmental Category: B Implementing Agencies: Federal and State Ministries of Health Cofinanciers and Other External Partners: None

B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 02/13/2006 Effectiveness: 01/24/2007 01/24/2007 5/15/2009 6/9/2011 Appraisal: 05/30/2006 Restructurings: 6/21/2012 12/14/2012 Approval: 10/30/2006 Mid-term Review: None None Closing: 12/31/2009 06/30/2013

C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Significant Bank Performance: Moderately Satisfactory Grantee Performance: Moderately Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Moderately Moderately Quality at Entry: Government: Unsatisfactory Unsatisfactory Implementing Quality of Supervision: Satisfactory Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Moderately Satisfactory Performance: Performance:

C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry Learning Review with no No at any time (Yes/No): (QEA): rating Problem Project at any Quality of Yes None time (Yes/No): Supervision (QSA): DO rating before Satisfactory Closing/Inactive status:

D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Health 100 100

Theme Code (as % of total Bank financing) Child health 20 20 Health system performance 20 30 Nutrition and food security 20 Other communicable diseases 20 10 Population and reproductive health 20 40

E. Bank Staff Positions At ICR At Approval Vice President: Makhtar Diop Obiageli K. Ezekewesili Country Director: Bella Deborah Mary Bird Ishac Diwan Sector Manager: Olusoji O. Adeyi Lynne D. Sherburne-Benz Project Team Leader: Isabel Cristina Soares Patrick Mullen ICR Team Leader: Isabel Cristina Soares ICR Primary Authors: Isabel Cristina Soares and Jack W. van Holst Pellekaan

F. Results Framework Analysis

Project Development Objectives (from the Project Appraisal Document) To improve access to basic health services by conflict-affected and underserved populations in four target states while establishing the basis for reform, sustainable financing and development of the decentralized health system.

Revised Project Development Objectives (as approved by original approving authority)

No revisions

(a) PDO Indicator(s)

Original Target Formally Actual Value a/ Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years PDO Indicator 1 Outpatient consultations per person per year in target health facilities Number 0.20 0.30 0.41 Date achieved December 2011 6/30/2013 Comments The target for this indicator was reset to 0.30 at the first restructuring in May 2009. The (including % achievement at project closure was 37% above target achievement) PDO Indicator 2 Percent of pregnant women who attended at least one antenatal care consultation Percent 48 70 65 68 Date achieved May 2009 December 2012 6/30/2013 Comments This indicator was introduced at first restructuring when focus shifted more to basic care (including % for improved reproductive health. The achievement was 4.5 percent above the target. achievement) PDO Indicator 3 Births attended by skilled health staff Percent of births 19 55 63 Date achieved May 2009 6/30/2013 Comments This indicator was introduced at first restructuring when focus shifted more to basic care (including % for improved reproductive health. The achievement was 24 percent above the target. achievement)

(b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised Target approval Completion or Values documents) Target Years Intermediate Number of ITNs distributed Indicator 1 Number 0 180,000 179,100 Date achieved May 2009 May 2009 The target for this activity was based on the estimated number of malaria bed nets that Comments could be procured. It was achieved during the project’s initial phase. Intermediate Percent of households who possess at least one ITN Indicator 2 Number 18 42 42 Date achieved May 2009 May 2009 Comments The target was achieved was achieved during the project’s initial phase. Intermediate Annual per capita US$ value of drugs, consumables and operational support provided to Indicator 3 target health facilities and midwives US$ per capita 0.17 0.40 0.36 per annum Date achieved May 2009 December 2010 6/30/2013 This indicator was introduced at the first restructuring. The achievement for all target Comments states was $0.36 in June 2013 (90% of target), but it had been as high as $0.48 in 2010. Intermediate Health Care Financing studies completed Indicator 4 Yes/No No Yes Yes Date achieved June 2012 June 2012 The National Health Accounts and related documents on health care financing were Comments expected to be available mid-2011. While the task was not completed until mid-2012, the target was fully achieved. Intermediate Rural health facilities constructed and or equipped Indicator 5 Number 0 18 18 Date achieved November 2010 November 2012 The target was achieved under the project’s initial phase (before the first restructuring). Comments These numbers include both Health Centers (HC) and Basic Health Units (BHU) Intermediate Rural hospitals upgraded and equipped for EOC Indicator 6 Number 0 3 9 9 Date achieved December 2011 6/30/2013 Comments The target was fully achieved Intermediate Midwifery schools constructed/rehabilitated Indicator 7 Number 0 4 7 7 Date achieved 6/30/2013 Comments The target was fully achieved Intermediate Number of PHC workers (including midwives) trained Indicator 8 Number 40 200 3,000 3,095 Date achieved May 2009 May 2013 6/30/2013 Comments The final achievement was 3% above the revised target

a/ All baselines are for 2006 (based on a household survey) except for indicators introduced at the first restructuring in May 2009 (as noted in the baseline column and also based on a household survey).

G. Ratings of Project Performance in ISRs

Actual Date ISR No. DO IP Disbursements Archived (USD millions) Moderately Moderately 1 06/27/2008 1.81 Unsatisfactory Unsatisfactory Moderately Moderately 2 08/25/2008 1.81 Unsatisfactory Unsatisfactory 3 04/24/2009 Moderately Satisfactory Moderately Satisfactory 2.38 4 06/29/2009 Moderately Satisfactory Moderately Satisfactory 2.38 5 12/22/2009 Moderately Satisfactory Moderately Satisfactory 4.79 6 06/29/2010 Moderately Satisfactory Moderately Satisfactory 6.21 7 02/16/2011 Moderately Satisfactory Moderately Satisfactory 7.36 8 09/10/2011 Moderately Satisfactory Moderately Satisfactory 8.45 9 01/22/2012 Satisfactory Satisfactory 10.02 10 09/09/2012 Satisfactory Satisfactory 11.69 11 12/31/2012 Satisfactory Satisfactory 11.98 12 06/24/2013 Satisfactory Satisfactory 13.87

H. Restructuring

MDTF-NS ISR Ratings at Amount Oversight Restructuring Disbursed at Restructuring Reason for Restructuring & Committee Restructuring Date(s) Key Changes Made Approved DO IP in USD PDO Change millions Scope of project as appraised was too ambitious compared with implementation capacity and a shortfall in Government counterpart funds. Project focus was narrowed to address the reduction of maternal and child mortality through improvements 5/15/2009 None MS MS 4.28 in access to prenatal care, midwifery training and support at the village level, medical supplies, improved rural hospitals with EOC facilities, and neonatal care. Two indicators for reproductive health made more concise.

MDTF-NS ISR Ratings at Amount Oversight Restructuring Disbursed at Restructuring Reason for Restructuring & Committee Restructuring Date(s) Key Changes Made Approved DO IP in USD PDO Change millions Attention to PHC not abandoned. Additional financing of $6 million Closing date extended by 18 months Closing date extended by 12 6/9/2011 None MS MS 6.21 months Closing date extended by 6 6/21/2012 None SAT SAT 10.86 months Addition of small component in Red Sea (hospital renovation in poor peri-urban area of Port Sudan) and rehabilitation of 6/14/2012 None SAT SAT 11.98 three midwifery schools in North Kordofan, financed by an additional $2 million. Closing date extended by 6 months.

I. Disbursement Profile

1. Project Context, Development Objectives and Design

Project Context.

Post-Conflict Environment. After close to 40 years of civil war between northern and southern Sudan the two parties signed the Comprehensive Peace Agreement (CPA) on January 9, 2005. The CPA defined the roles and responsibilities of the Government of National Unity (GoNU) in the north and the Government of Southern Sudan (GSS) in the south for an Interim Period of six years. Before the end of the Interim Period a referendum would be held in Southern Sudan to decide if the people in the south wanted to be independent from the rest of Sudan. They chose independence. The Republic of Sudan and the Republic of South Sudan became independent states on July 9, 2011.

The Joint Assessment Mission (JAM). The joint mission by the GoNU and the GSS was mandated by the CPA and prepared a “Framework for Sustained Peace, Development and Poverty Eradication”. The mission report concluded, amongst other things, that Sudan’s poor health outcomes were associated with a health care system that is significantly under-funded, unbalanced and inefficient. Preliminary findings from a 2003 health infrastructure survey showed that, overall, 36 percent of health facilities were not fully functioning in North Sudan. The coverage of the primary health care (PHC) network was severely constrained by lack of staff and funds for recurrent expenditure, and by a range of management problems including fragmentation and an inadequate information system. The infrastructure network was large overall, but population-to- facility ratios – around 100,000 per rural hospital and 35,000 per health center – were high. There were very large urban-rural and regional disparities in the availability of services. There was a re-emergence of polio cases in Darfur in 2004, just as Sudan was on the verge of being declared polio-free. This illustrated the inadequacy of the PHC system, including low routine immunization coverage, and the threats facing neighboring countries as well as Sudan, underscoring the need for a general expansion and upgrading of the health sector.1

For the health sector the JAM report’s objective by the end of the six-year Interim Period specified in the CPA in 2011 was to increase coverage of basic health services in Sudan from an estimated 45 percent to 60 percent of the population.

Government Health Strategy. The GoNU vision for the health sector was expressed in the Federal Ministry of Health (GMoH) 25-year Strategy. The main objective was to achieve the Millennium Development Goals (MDGs), contribute to poverty reduction, and improve equity across and within states, and among vulnerable groups.2 The JAM

1 Joint Assessment Mission report: “Framework for Sustained Peace, Development and Poverty Eradication”; March, 2005; Volume I, Synthesis, page 35.

2 Bank Report No. 37811-SD; Final Project Proposal for a “Decentralized Health System Development Project”, October 26, 2006, page 2.

estimated that additional resources of approximately US$ 215 million would be required annually in the next two to three years in order to achieve significant progress towards the MDGs for all Sudan

1.1 Health Sector Context at Appraisal

The Decentralized Health System Development Project (DHSDP) was proposed in the JAM as an important investment. Although health indicators in Sudan in 2006 were often better than Sub-Saharan Africa averages, this masked significant urban-rural and regional disparities caused by conflict, internal displacement, and chronic poverty. For example, while there were about 22 physicians per 100,000 population overall in Sudan, the ratios were considerably lower in poorer and conflict-affected states. The ratio was 13 in Red Sea, 10 in Blue Nile, 5 in Kassala, and 4 in South Kordofan. Similar patterns existed for health outcomes. The 1999 Safe Motherhood Survey found that average under-five mortality was 104 per 1,000 in the then Northern Sudan, compared to an estimate of 162 for Sub-Saharan Africa as a whole. However, under-five mortality in this project’s target states of South Kordofan (147), Kassala (148), Red Sea (165), and Blue Nile (172) was significantly higher than the country average. Nationally maternal mortality averaged about 600 per 100,000 births in 2006, but ranged between 1,400 in Kassala and 170 in Red Sea state. Along with disparities among health indicators between states, there were also substantial inequalities in the availability of health services within states. Many important health indicators in localities in the target states for this project were lower than state averages because the poor and the isolated in those localities had almost no access to effective PHC.

A social and gender assessment in the target states conducted as part of project preparation, concluded that even in situations where adequate primary health services existed, there were major barriers to access facing women, vulnerable groups and the poor. In the four target states, the cost of service fees and drugs, as well as shortages of drugs in many rural hospitals and basic health units, prevented many households from accessing health care. In many cases, women have little say over the use of a family‘s financial resources, and consequently little control over health-seeking behavior for themselves and their children. Cultural norms and domestic responsibilities result in women spending much of their day carrying heavy loads of firewood, water, and foodstuffs long distances, creating both related health concerns, as well as limiting opportunities for seeking health care. These norms also dictate home birthing which, combined with inadequate access to health centers and emergency obstetric care (EOC), and poor post natal care result in high maternal and new-born mortality. These weaknesses in access of large numbers of people in Sudan to PHC provided the core rationale for the project.

The principle of a decentralized health system was already national policy. While the federal Government was gradually increasing its fiscal transfers to the states, primary health care at the state level was still focused predominantly on urban areas. However,

Sudan’s Interim Constitution (2005) made a commitment to provide free emergency and basic health services to all Sudanese. This project supported the pursuit of this goal. 3

Rationale for MDTF Financing. The CPA had specified that a Multi-Donor Trust Fund-National (MDTF-NS) focus on recovery and development of conflict-affected regions, particularly the Three Areas (Blue Nile, South Kordofan and Abyei), as well as the least-developed regions in North Sudan. In light of the growing Government revenues from the oil industry, the project was intended to lay the institutional and technical groundwork for sustainable financing, reform and development of the decentralized health care system. In accordance with agreements when the MDTF-NS was established projects were financed jointly by the GoNU (two-thirds of the cost) and the MDTF-NS (one-third of the cost).

1.2 Original Project Development Objectives (PDO) and Key Indicators

The Final Project Proposal (FPP) proposed a three phase project with the following objective and sources of financing shown in Table 1 below.

To “improve access to basic health services by conflict-affected and underserved populations in four target states while establishing the basis for reform, sustainable financing, and development of the decentralized health system”.4

Table 1: Proposed Project Costs for Three Phases by Sources of Financing

Phase MDTF-NS Government of Sudan Total ($ million) Phase 1 6 13 19 Phase 2 8 16 24 Phase 3 9 18 27 Total 23 47 70 Source: Final Project Proposal for a “Decentralized Health System Development Project”, October 26, 2006; page 8

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification.

3 Sudan’s Federal Government is responsible for policy and vertical public health programs, and overall monitoring and evaluation of health system performance, as well as for tertiary hospital services. The states are responsible for first-referral hospital services and, through the formal sub-divisions in the states called localities which are typically responsible for the provision of services such as water supply, power, basic education and basic health care) for primary health services. The terms primary health care (PHC) and basic health care services are used interchangeably and are understood to encompass primary preventive and curative services as well as first-referral services, usually provided by district/rural hospitals.

4 Final Project Proposal for a “Decentralized Health System Development Project”, October 26, 2006, op cit, page 4. The Oversight Committee approved the project, but financing was approved only for Phase 1.

The PDO was not revised. However, two of the project’s three key outcomes indicators were revised during project’s first restructuring to make them more concise. Other indicators were dropped because they were not measurable. The revisions as well as the reasons for the changes are shown in Table 2. Actions taken at the first restructuring with respect to the three key performance indicators were as follows:

• The first original key indicator which directly addressed the main project objective ("access to PHC") was not changed and remained unchanged throughout the project’s implementation

• The second original key indicator ("percent of patients who do not receive health care due to financial barriers in target areas") was not measurable and replaced ("percent of pregnant women who attended at least one antenatal care consultation) which was measurable and consistent with the original key indicator

• The third original key indicator ("skilled birth attendance in target areas") was not concise and hence there were no data for its measurement; it was therefore modified ("births attended by skilled health staff") to make it concise and consistent with the original key indicator

The intent of changes to the second and third indicators was to make them measurable and concise while maintaining their focus on the measurement of improvements to reproductive health.

The Data Sheet contains information on the targets for the indicators and the achievements towards those targets at the time the project closed.

1.4 Main Beneficiaries

The FPP stated that the project will provide support to the poor, underserved and conflict- affected states of South Kordofan, Blue Nile, Kassala, and Red Sea which at appraisal had a total population of approximately 5.1 million where infant mortality averaged 101 per 1,000 births, under–five mortality was 154 per 1,000 births, and maternal mortality averaged 524 per 10,000 births. All of these mortality rates were higher than national averages. The project activities were initially targeted at 19 localities where the total population was about 1.1 million, although three training centers for midwives in North Kordofan were added towards the end of the project as well as a hospital in a peri urban rural locality in Port Sudan in Red Sea State. 5 While pregnant women and new-born children were the main beneficiaries, the PHC systems in target localities benefited resulting in improved access to PHC for numerous poor households in these localities.

5 These additions brought the number of localities after the Fourth Restructuring to 21 at the project’s close

Table 2: Original and Revised Performance Indicators at First Restructuring (May 2009)

Original Key Performance Revised Key Performance Reasons for Retention and Change Indicators Indicators Objective 1. Improve Access to Primary Health Care Services by Conflict/Underserved Populations Outpatient consultations per Retained Could be regularly measured through person per year in target areas HMIS – outcome indicator

Patient ratio between highest and Dropped Not measurable with available lowest wealth quintile monitoring programs Percent of patients who do not Modified to: Percent of Original indicator not measurable. receive health care due to financial pregnant women who New indicator can be measured barriers in target areas attended at least one regularly – outcome indicator antenatal care consultation Skilled birth attendance in target Modified to: Births attended Original indicator not specific. New areas by skilled health staff indicator is specific and measurable – (percent of total births) a/ outcome indicator Vitamin A coverage in target areas Dropped Not measurable with available monitoring programs Added: Percent of Measurable – intermediate indicator households who possess at least one ITN Number of ITNs distributed Number of pilot initiatives Dropped Original pilots not clearly defined in implemented aimed at reducing the FPP and dropped. A pilot to test financial and gender barriers to incentive payments for village care midwives was introduced. Percent of PHC worker training Modified to: Number of Directly relevant to the objective of objectives in project plan achieved PHC workers (including the project – intermediate outcome midwives) trained indicator

Objective 2. Establishing the Basis for Reform and Development of the Decentralized Health System Added: Measurable – intermediate outcome Annual per capita US$ value indicator of drugs/consumables and operational support provided to target health facilities and midwives b/ National Health Accounts and Retained Measurable – intermediate outcome training completed indicator Percent of PHC health facilities Modified to: PHC Measurable - intermediate outcome included in the investment plan for constructed, rehabilitated indicator target areas rehabilitated, and equipped constructed and/or equipped Rural hospitals upgraded and Measurable - intermediate outcome equipped for EOC indicator a/ Skilled health staff = doctor, nurse, midwife, trained village midwife b/ The per capita value is estimated on the basis of approximately 1 million people in the catchment area for the targeted primary health facilities (based on data in Annex D of the First Restructuring Proposal.

1.5 Original Components6

Component 1. Expand access to primary health care services by underserved populations (FPP estimate US$19.00 million; actual US$8.30 million). This component was directly relevant to the first part of the PDO, namely “improve access to basic health services by conflict-affected and underserved populations in four target states”.

Sub-component 1.1. Improvement in the quality of existing primary health care services in under-served areas. This involved an integrated package of support to targeted health services to be implemented by existing government systems, with technical support from a dedicated government team.

Sub-component 1.2. Expansion of coverage of primary health care services and high- impact interventions in un-served areas. This Sub-component supported expansion of basic health services to improve coverage of conflict-affected and un-served areas using the following strategies. i) In areas with no existing government services, the project financed mobile and temporary clinics managed and supplied by the State MoHs and staffed by government health workers reallocated from better-served areas and States. ii) Another strategy for areas with no government health services was to finance private for-profit and non-profit firms and organizations to provide services on a contractual basis such as in the Blue Nile iii) Prioritizing populations with little or no access to facility-based health services, the project supported the provision of high-impact health interventions directly to communities and households, with particular focus on reducing barriers to accessing care for women and vulnerable groups.

Sub-component 1.3. Pilot experiences to reduce barriers to access to primary health care services. The objective of this Sub-component was to improve the knowledge and experience of the health authorities with possible strategies and interventions to reduce barriers to access to PHC services, particularly by women, vulnerable groups and the poor in general.

Planned pilots at appraisal were to: i) Evaluate the implementation, effect on service utilization, and financial feasibility of subsidies PHC services with a large impact on morbidity and mortality. ii) Measure the cost and impact on service utilization of expanding health insurance coverage to conflict-affected and underserved groups.

6 Annex 1 shows the changes in the components and in their funding following the restructurings of the project.

iii) Assess the effect on health service utilization by women of interventions designed to address gender-related barriers to access.

Component 2. Establish the basis for reform and development of the decentralized health system. (FPP estimate US$50.4 million; actual US$9.94 million) This component was directly relevant to the second part of the PDO, namely “establishing the basis for reform, sustainable financing, and development of the decentralized health system”. However it was also relevant to the first objective because this component financed most of the project’s construction and rehabilitation activity

Sub-component 2.1. Capacity-building and policy development. The objective was to lay the groundwork for reform and development of the decentralized health system through technical assistance and studies on priority systemic issues as well as capacity- building in selected areas.

Studies planned at appraisal were:

i) Health care financing. The project supported technical assistance to provide a stronger information based on current health financing in Sudan, including household out-of-pocket expenditures, National Health Accounts (NHA), and support for the development of a reform strategy to improve financing of basic health services under the fiscal federal system in the context of the Fiscal and Financial Allocation and Monitoring Commission (FFAMC). This work also included an in-depth study of the National Health Insurance Fund (NHIF). ii) Access to pharmaceuticals. Technical assistance and studies examined availability and barriers to access to affordable medication in order to inform the ongoing development and expansion of the Revolving Drug Fund system. iii) Health planning, budgeting and management by target State and Locality health administrations. Capacity-building and training would focus on the four target State MoHs. It was to include institutional assessments to provide recommendations on organizational reform and development, and technical assistance and training in planning, budgeting, management and supervision. iv) Monitoring and evaluation. Capacity building for monitoring and evaluation functions for all three levels of government, including strengthening health management information systems (HMIS) and the capacity of administrators to effectively analyze and use data. Technical assistance at the federal and four target state levels.

Sub-component 2.2. Development of primary health care human resources. This sub- component provided support to implementation of the FMoH human resources for health (HRH) strategy in the four target states, established the basis for improvements in the production, quality, deployment and retention of the PHC workforce. Of particular priority were medical technicians, nurses and midwives.

The project supported the following:

i) National human resources for health (HRH) strategy development. Technical assistance for the development of sector-wide HRH strategies and policies, with a focus on PHC cadres. ii) State PHC human resource development strategies. Technical assistance to the State MoHs for assessment of HRH needs in the four target states, with a focus on PHC workers. iii) Curriculum review and instructor in-service training. Technical assistance to review PHC training programs and curricula which formed the basis for refresher training for instructors. iv) Rationalization and investment in training schools and equipment. In line with State- level HRH strategies, nursing and midwifery schools upgraded through renovation and equipment which was accompanied by rationalization of existing training institutions.

Sub-component 2.3. Investment in primary health care infrastructure and equipment. Upgrading and expansion of the PHC infrastructure in the four target states, focusing on weakest health facilities. PHC facilities included health centers and rural hospitals providing first-referral services, notably emergency obstetric care. The first phase of investment was detailed planning, including assessment of the physical and functional status of PHC facilities, equipment needs, geographical distribution, identification of priorities for rehabilitation, and construction work, including the initiation of architectural studies. It was anticipated that the bulk of rehabilitation and construction would be carried out in phases two and three of the project. There were, however, no second and third phases.

Sub-component 2.4. Project implementation. Financed the personnel and resources necessary to manage the project and coordinate the Project Implementation Units (PIUs) at the federal and state levels.

1.6 Revised Components

The project was restructured four times which twice involved additional financing. Annex Table 2A provides details of the first and fourth restructuring when project changes involved more than an extension of the closing date as in the second and third restructuring.

(a) First Restructuring and Additional Financing. This was approved by the Oversight Committee of the MDTF-NS on May 15, 2009. The actions were stimulated by a concern about the project’s slow progress due to the substantial shortfall in releases of the Government’s funding commitments, the difficulties posed by an overly ambitious project, and the opportunity to support the Government’s “Road Map for Reducing Maternal Mortality and New Born

Mortality in Sudan”, 2009. 7 The main features of this restructuring which included revision of some activities, components and indicators were as follows:

i) Sharpening the project’s focus on child under five and reproductive health services. The trade-off for this focus was to drop a number of activities related to specific diseases and a reduced rehabilitation and construction program for health and training centers at the state and locality levels. However, the objective of improving access to PHC services was not abandoned. Some indicators were dropped because they were not measurable and substituted or modified to measure outcomes more precisely (see Table 2).

ii) Component 1 (“Expand Access to Primary Health Care Services), $2.5 million of additional financing to targeted health facilities for the procurement of medical supplies, operational costs and pharmaceuticals, as well as midwifery kits and supplies for midwives, and refresher training for PHC workers and midwives. iii) Component 2 (“Establish Basis for Reform and Development of the Health System”), $3.5 million of additional financing to system investments to improve maternal and neonatal health care services. This included improving antenatal, delivery and referral services provided by village midwives (VMWs) based on pre-service training, applying the Government’s newly-developed curriculum for skilled birth attendants, technical support and supervision, and piloting of performance-based incentives replaced the piloting to reduce barriers to access to PHC. Investment in PHC training was scaled down and the number of health facilities to be rehabilitated and equipped was limited to civil works and equipment already procured. Technical assistance and studies continued. iv) A project results framework was prepared, and key indicators and targets reflected the change in the project’s focus (albeit still aimed at the PDO of improving access to basic health services), particularly in Component 1. v) The geographic focus of Component 1, as well as the infrastructure and equipment investments of Component 2, would be more narrowly directed to 58 targeted health facilities and their catchment populations totaling approximately one million in underserved parts of 19 poor rural localities in four target states. vi) The closing date was changed to June 30, 2011.

(b) Second Restructuring. Approved on June 9, 2011, the purpose was to extend the closing date to June 30, 2012 due to the delayed implementation in Blue Nile and South Kordofan on account of the civil conflict.

7 Memorandum and Recommendations of the MDTF-NS Technical Secretariat to the Oversight Committee on Proposed Restructuring and Additional Grant to the Sudan Government of National Unity for the Decentralized Health System Development Project (DHSDP) - Grant No. TF057324; May 14, 2009, page 4.

(c) Third Restructuring. Approved on June 21, 2012, to extend the closing date to December 30, 2012, it provided time to prepare the Fourth Restructuring and Additional Financing. (d) Fourth Restructuring and Additional Financing. This was approved on December 14, 2012. The purpose was to:

(i) Consolidate the achievements with VMW training, and expand to a new target state (North Kordofan), that had received IDPs from Blue Nile and South Kordofan, and add the rehabilitation of a peri urban hospital in Rural Port Sudan. (ii) Access by pregnant women to trained village midwives was expanded, all midwives were provided with a package of incentives, and continuous supervision and support to midwives was started based on institutional arrangements already introduced to improve their performance. The rehabilitation of midwifery schools at El Obeid, El Nuhud and Um Ruwaba was added. (iii) The results matrix was changed by increasing the targets for two indicators, namely “percentage births attended by skilled health staff” was increased from 30 to 55 percent, and the “number of health staff (including midwives) receiving training” was increased from 1,000 to 3,000. (iv) The project’s components and sub-components were also formally amended (“consolidated”) as shown Annex 1. The closing date was extended to June 30, 2013. The restructuring increased funding by the MDTF-NS in Phase 1 of the project from $6 million to $12 million (after the first restructuring) and to $14 million (after the fourth restructuring). Government contributions were reduced from $13 million in Phase 1 to $4.24 million. The Government stated, however, that it maintained its overall financial support through regular budget support for many of the project’s activities. Additional details of the project’s restructurings are provided in Annex Table 2A.

1.7 Other significant changes

Following the re-emergence of conflict in 2011/2012 the project could not implement most activities in Blue Nile and South Kordofan States. While the problems in most of Blue Nile subsided allowing the resumption of project activities in 2012, South Kordofan remained largely inaccessible to the project until the project closed.

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry

Project Preparation. As noted above, the concept for this project was based on a detailed examination of Sudan’s health sector in the JAM. The project’s concept paper was reviewed in February 2006. The FPP submitted to the Oversight Committee of the MDTF-NS for funding was developed in consultation with sector partners, namely the Health Thematic Group among development partners in Sudan led by the FMoH, which included MDTF-NS donors, UN agencies, the World Bank, and non-governmental health

sector partners. Project preparation by the Government was based on the best information available on the health sector and its needs. Appraisal included extensive planning and consultation in each of the target states, involving Federal, State and local authorities, including representatives of conflict-affected and underserved populations in the target states, and other sector stakeholders such as non-governmental and community- based organizations. WHO provided assistance to the project’s preparation. DHSDP was in line with MDTF-NS objectives and supported by the Oversight Committee. In addition the Federal and State governments had strong ownership of the project and were committed to its implementation and success. The Oversight Committee approved the FPP on October, 30 2006.

Design. The project’s first objective of improving access to PHC was clear. Evidence showed that access to PHC needed to improve, but the expected extent of improved access resulting from this project was not specified. The second objective was not clear because it depended on establishing a basis for progress on various benchmarks; this basis was not defined.

A signature and successful feature of the project’s design was that State Ministries of Health (SMoHs) were empowered to be responsible for the day to day management and implementation of the project. They were supported by relatively small PIUs in each state working closely with state health authorities at all levels providing oversight and facilitation as well as advice on fiduciary issues.8 The FMoH, through its departments, provided the overall management and policy direction for the project supported by a small Federal PIU responsible for fiduciary matters such as the procurement at the federal level and the management of the special project accounts.

The project’s design at appraisal was influenced by the range of problems facing the health sector in target states which experienced a considerable decline in the quality of health services during the civil war. Consequently the project’s original scope was overly ambitious considering the relatively small size of the total project cost and the very short time (4-5 years) to complete three phases in a country where PHC institutions were weak and fiscal decentralization had barely started. Its scope was trimmed during the first restructuring in May 2009 following lengthy discussions within the Bank and with the Government which started in 2008 The second and third phases of DHSDP were abandoned as separate projects although some of their proposed activities were implemented. 9 Despite the project’s restructuring there was no change in its basic design of empowering Federal and State MoHs with the responsibility for improving access to PHC services.

8 The exception was South Kordofan which had two PIUs, one in the north and the other in former SPLA areas in the south.

9 The Oversight Committee abandoned all phases for MDTF-financed projects in 2008.

Risks. This was an ambitious project with many risks. The risks perceived at appraisal included the inherent institutional risks such as the project’s dependence on the FMoH and the SMoHs, their ownership of the project, the competence of FPIU and SPIU staff, the adequacy of financial reporting, and the timely availability of Government contributions (67 percent of original total project costs). Most of these risks did not emerge as major issues because they were effectively addressed during implementation. However, the anticipated risk of a shortfall in Government counterpart funding became a serious reality. This problem was addressed at the first restructuring. Surprisingly political risks were not anticipated as part of country risks. Nevertheless they emerged in the form of a civil conflict in 2011-2012 in Blue Nile where it slowed down implementation, and in South Kordofan where it brought most implementation to a halt

Quality at Entry. This project was included in a quality at entry learning review of recipient executed trust funds by the Quality Assurance Group (QAG) in the Bank in December 2007. The QAG review commended the task team on strong country ownership of the project, its responsiveness to client demands, rigorous appraisal and review, excellent collaboration with donors and UN agencies, and the relevance of the social and gender assessment. However QAG concluded that the project was ambitious and needed a more realistic time frame for implementation, the project documentation needed to describe the overall program more clearly, and the assessment of country risks was inadequate. In summary, the main problem at entry was the project’s ambitious scope. Consequently the implementation task was too comprehensive to be managed effectively by the weak MoH institutions at the start of the project.

2.2 Implementation10

The project’s implementation faced a number of challenges, particularly during its early years. However, actions by the project team generated good results as the project went through three implementation phases, namely “establishment” (2007-2009), “restructuring” (2009-2011), and “consolidation and expansion” (2012-2013).

Implementation performance reflected these phases. According to the Implementation Supervision Reports (ISRs) the project was rated moderately unsatisfactory for most of 2008 (its first year after effectiveness) because of delayed release of Government counterpart funding and its over ambitious scope. However, it achieved a moderately satisfactory status in April 2009 after which the first restructuring occurred in May 2009 because the Government had advised that its counterpart funding obligations for this and other MDTF-NS projects needed to be substantially reduced. In 2010 construction and rehabilitation accelerated when additional MDTF funds were disbursed and PHC services improved as enhanced facilities became available. The project was rated in the satisfactory range from April 2009 to the time it closed in June 2013.

10 A review of implementation by components is provided in Annex 2

The most important responses to challenges facing the project that led to its ultimate satisfactory rating in the ISR when the project closed are listed below.

Slow implementation at the start was addressed strategically The project faced many challenges during the year after approval. Time was needed to establish institutional arrangements and meet conditions of effectiveness, and the shortfall in the agreed disbursement of counterpart funds from the Government created delays in project performance. Two and half years after effectiveness Government disbursements were only $3.2 million out of a total commitment of $13 million for a planned three year project. MDTF disbursements at that time lagged the planned level by about 50 percent. This funding crisis led to the first restructuring which reduced the Government’s contribution which was eventually only $3.2 million (see Annex 1 – financing table). Funding from the MDTF increased by $6 million to $18 million. Restructuring also reoriented the project’s broad scope to focus more on maternal and under-five death rates which meant that some programs envisaged in the FPP were dropped – among them disease control interventions and piloting of experiences to reduce barriers to access to PHC services. This first restructuring led to an acceleration of implementation – particularly for the infrastructure program to improve PHC services and the training of midwives. The fourth restructuring consolidated the project’s structure into fewer components (see Annex 1) with most of the $2 million additional funds allocated to bolstering referral services, and expanding training and support for village midwives including their training in North Kordofan.

Actions were taken to address institutional weaknesses after project start up. NGOs which had the capacity and experience to support some aspects of the project’s objectives were retained to fill institutional gaps when the project started. In 2008 the DHSDP contracted an international NGO (GOAL) to provide basic health care services in Kurmuk Locality (Blue Nile State) for 15,000 returnees from Ethiopia because of conflict. Also, in the Kauda Locality (South Kordofan) the health training institute lacked capacity and an international NGO (SCF-USA) was contracted to provide training until civil conflict made it impossible to complete this contract.

Adjustments were made when civil war disrupted implementation in Blue Nile and South Kordofan. The project’s progress in improving access to basic health care in Blue Nile and South Kordofan States was significantly delayed between 2011 and 2013 because of violent civil conflict. With the exception of one locality the conflict in Blue Nile State abated sufficiently to allow project activities to resume in 2012. However, by the time the project closed in June 2013 it had still not been possible to resume regular project activities in most of the targeted localities in South Kordofan State. As part of the fourth restructuring additional funds were made available to rehabilitate a hospital in peri-urban “rural” Port Sudan and the rehabilitation of three midwifery schools in El Obeid, El Nuhud and Um Ruwaba to provide training of VMWs from South and North Kordofan. Despite the difficult circumstances, the project was never designated as a project at risk

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization

Design. The FPP did not have a results matrix although key performance indicators were clearly defined.11 At the first restructuring a results matrix was, however, prepared. On the other hand the FPP included a sub-component for designing a comprehensive national M&E system for the health sector (described briefly in Annex 2 of this ICR) which contributed to the development of the M&E design for this project. The official Sudan Household Health Survey (SHHS) for 2006 provided baseline information on state-wide parameters for the DHSDP. The State PIUs were responsible for regular collection of data on these indicators from the participating health facilities and village midwives. Compilation of data from the M&E program was managed by the Federal PIU. As mentioned already, the implementation of this program was extended at first to four target states and towards the end of the project South Kordofan State was added. However, despite diligent work on project indicators by the project team, only one of the indicators provided a direct measure of access to PHC and none of the outcome indicators provided a direct measure of the basis for reform of the decentralized health care system.

Final and intermediate outcome indicators were revised at the first restructuring (see Table 2). For the first objective one original and two revised key performance indicators were relevant and robust including the addition of one discreet outcome indicator (outpatient consultations per person per year) to measure access to PHC. For the second objective there was, by definition, no final outcome indicator because activities such as the NHA and associated training directed toward this objective were only initial steps and hence only intermediate indicators.

While the Ministry established a sound M&E system for the project, the MDTF’s Monitoring Agent (MA) was a constructive partner in the M&E process. The Government’s ICR noted that the MA verified that state PIUs collaborated with the state MOHs in assuring the veracity of the data collected.

Implementation. At the project’s start difficulties were encountered in obtaining the necessary information for project monitoring. Inaccurate baselines stimulated a review of the SHHS for 2006 mentioned above and data collection protocols were established. There were also a number of changes in the M&E specialist in the Federal PIU. Nevertheless, data on the project’s indicators were ultimately collected on a weekly and monthly basis. According to the Federal PIU the collection rate varied between 85 and 100 percent. For health clinics which were not part of the DHSDP the response rate averaged 30 percent but for some the rate was as high as 80 percent. Incentives were paid using Government funds to some locality staff to ensure the assembly and delivery of timely and accurate data.

11 A results matrix was not included as part of simplified procedures authorized under the Bank’s Operational Policy 8.00 dated March, 2007

Capacity to undertake structured analytical, monitoring and evaluation was strengthened. Considerable resources were allocated to improving the capacity of the FMoH and SMoHs to conduct monitoring and evaluation. For example the project funded a study to establish baselines for indicators for the M&E system12 as well as the formulation of an M&E work plan and budget. This provided the basis for a set of health care indicators which became the Health Management Information System (HMIS) for the Federal and State Ministries of Health. In addition a series of annual Maternal Death Reviews was started in 2009. While maternal death rate was not an outcome indicator for the project because of the lack of reliable data, the monitoring of maternal death rates as part of this project was an important start to the longer term evaluation of the project’s achievements. No assessments were found of the quality of the studies, technical assistance and training. However, an assessment of the quality of the construction program was prepared (see Annex 2). There was also a situation analysis of the Maternal Death Review System.13

Utilization. Regular monthly meetings were held in each target State to review the data and monthly reports sent to the Federal PIU. Comprehensive quarterly reports were prepared by the Federal PIU containing a summary, reports on physical progress, achievement by components, reports on financial management and procurement, as well as future work plans and a tabulation of the project’s compliance with the Grant Agreement. Monitoring reports were used by the project’s management, the Ministry, the World Bank and the Monitoring Agent to evaluate progress. They were also used to compare performance among states and localities and against targets and explore explanations. While the rigor of the M&E framework, data collection and management was appreciated by the Government, it should be noted again that there was only one outcome indicator that directly measured progress towards the project’s first objective of access to improved basic health care services. However, indicators that focused on reproductive health provided supporting evidence of progress on the improvement of basic health care services.

2.4 Safeguard and Fiduciary Compliance

Safeguard Compliance. At project appraisal an Environmental and Social Management Framework (ESMF) and a Resettlement Policy Framework (RPF) were chosen for the project as mitigation instruments because at that time the sites and locations for the construction of health facilities were not known. OP4.01 required a Medical Waste Management Plan (MWMP) to ensure the effective collection and disposal of hazardous

12 Government of National Unity (MTDF-N), Federal Ministry of Health Decentralized Health System Development Project (DHSDP), “Baseline and Follow up Assessment of Health Services in Four Northern States-Sudan”, May 2010; prepared by Mustafa Khidir Mustafa Elnimeiri. 13 Federal Ministry of Health (2013), Improving the Sudan Maternal Death Review System: A Situation Analysis, June 2013

medical waste at sites such as hospitals. Neither an ESMF nor an RPF was necessary because there were no environmental, social or resettlement issues to be addressed. A MWMP was completed, approved and disclosed by the Government and the Project Coordinator within six months of project effectiveness with support from Bank supervision missions. A framework for addressing the medical waste issues was designed. The MoH prepared waste management policies and strategies, training programs for environmental health officers, and waste management operational manuals for hospitals. Medical waste incinerators were installed and fenced at all DHSDP- supported facilities. The Federal and State Ministries of Health paid close attention to waste management. It became standard policy and an environmental officer was appointed in the FMoH. Financial Management. The project used a series of four financial management specialists in the Federal PIU during implementation. This disrupted continuity in this function and led to one short period when financial management performance was rated moderately unsatisfactory. However, with this exception, financial management in the Federal and State PIUs was satisfactory. No financial control issues emerged in Bank financial management supervision or Monitoring Agent reviews, and there were no overdue or unacceptable audit reports or interim financial statements. The Government’s ICR (Annex 7) noted, however, that the MA found that the fixed asset register did not capture all details considered necessary for such a register. The FPIU advised that this matter had been rectified when the project closed. Procurement. The project employed qualified procurement officers in all PIUs. Despite some turnover in procurement staff in the Federal PIUs, and with the support of the MA, there were no major problems with the management of procurement activities. By agreement with the Government, at the start of the project Federal PIU procurement staff handled grant-financed procurements, while State PIUs handled government- financed procurements, particularly health facility supplies and consumables. State PIU staff managed the purchase of drugs and supplies for health facilities. The Federal PIU managed Grant-financed procurement of works, goods and consultants. .

2.5 Post-completion Operation/Next Phase

The Government has been a strong supporter of the DHSDP and is satisfied with the results so far. The Ministry of Finance and National Economy (MOFNE) therefore decided to finance a long term program to continue the improvement of access to PHC in Sudan. The Ministry has launched the “Primary Health Care Expansion Project” modeled on the DHSDP including a component (30 percent) for the improvement and new construction of health centers. Annual Government funding for this project is anticipated to be SDG 70 million (US$13 million) although the federal government will fund only infrastructure, medical equipment, drugs, consumables, and training. The States will be expected to finance operating costs. The MoFNE anticipates that bilateral donors will contribute to the project’s cost either directly or through a future multi-donor trust fund. The Bank has been active in laying the groundwork for such support. The Government’s project gives all Sudanese states the same implementation responsibilities as the four target states in DHSDP. Federal and State MoHs will use the institutional

capacity developed under DHSDP to facilitate fiduciary functions in all states as well as provide coordination on policy and technical matters.

In order to broaden PHC beyond the focus areas for DHSDP, and at the same time sustain this project’s progress on reproductive health, there will need to be continued training and capacity building in order to ensure that VMWs provide services at adequate technical standards and that first referral hospitals maintain their capacity to generate electricity, pay for operating costs, have adequate staffing and perform their functions at high standards. In future DHSDP’s sustainability in the context of the Government’s Primary Health Care Expansion Project will be measured by reductions in maternal mortality achieved.

3. Assessment of Outcomes

As noted in Section 1.3 the project development objective was not changed. However, three out of the four key performance indicators for the project were dropped or amended as part of the first restructuring. The first of these three was dropped for lack of data. The reason for changing the second and third indicators was only to make them measurable while maintaining their focus on the measurement of improvements in access to PHC and to reproductive health. Nevertheless, in accordance with the Guidelines14 the following assessment of outcomes will be based on relevance, achievements and efficiency before and after the first restructuring when the changes in indicators were formally approved, and will then evaluate the overall outcome with weights based on the value of disbursements before and after this restructuring.15

3.1 Relevance of Objectives, Design and Implementation

Relevance of Objectives (Rating: Satisfactory before and after restructuring).

Since the project objectives did not change the following assessment of the relevance of objectives applies equally to the project before and after the restructuring. The project’s two part objective of “improving access to basic health services by conflict-affected and underserved populations” and “establishing the basis for reform, sustainable financing, and development of the decentralized health system” was relevant to the stated objectives of the FMoH program to strengthen the capacity of health administrations in localities, achieve the MDGs, and to the Government’s “Road Map for Reducing Maternal

14 “Implementation Completion and Results Report – Guidelines”, OPCS, updated 11/10/2010, states that “For those operations where there was a formal change in PDO/GEO or key associated outcome targets …. the evaluation method for determining the outcome rating (should) take into account both the original and formally revised objectives or targets”, page 25. It is assumed that the same principle should apply to changes in indicators.

15 The first restructuring will be referred to as “the restructuring” unless there is a need to distinguish it from the other project restructurings The three other restructurings did not involve changes in the PDO or the indicators..

Mortality and New Born Mortality in Sudan”. The objectives were also relevant to the MDTF’s focus on recovery and development in conflict-affected regions of Sudan, and the World Bank’s Interim Strategy Notes (ISNs) for FY 07-09 which continued to FY13 and the ISN for FY14-15. The results matrix for the FY07-09 ISN included a strategic objective to “Develop core capacities and components of the health system at the same time as supporting rapid expansion of service delivery and selected high-impact preventive health interventions, particularly in war-affected or marginalized areas”. The objectives remain relevant for the future. The Sudan ISN for FY14-15 makes the commitment to provide “Support to national and local programs for service delivery focusing on key human outcomes” in the second pillar of the strategy.16 The Government’s initiative to fund its “Primary Health Care Expansion Project” reflects the continued substantial relevance at exit of the DHSDP model as a basis for a sustainable and decentralized basic health care system in Sudan. The relevance of the project’s objectives before and after restructuring had only minor shortcomings.

Relevance of Design (Rating: Unsatisfactory before restructuring; Satisfactory after restructuring).

Before and after restructuring the project’s core design, which made the Federal and State PIUs responsible for advice, facilitation and fiduciary activities, continued to be relevant throughout implementation. This approach was consistent with the national objectives of improving and sustaining decentralized basic health services. This aspect of the project’s design was therefore relevant to the project’s objectives before and after restructuring.

However, before restructuring the scope of DHSDP’s activities to “improve access to basic health care services by conflict-affected and underserved populations” was overly complex and ambitious because it covered numerous elements in basic health care services to be implemented in three phases over only 4 to 5 years without a results matrix or adequate funding due to a shortfall in Government disbursements. With no prospects for resolving these problems the best solution was restructuring to, inter alia, seek alternative sources of financing and sharpen the project’s focus on reproductive health and under-five mortality while not abandoning the objective of increasing access to PHC services. In conclusion, before restructuring the relevance of the project’s scope had major shortcomings.

After restructuring, when a results matrix with improved and more concise indicators had been defined, the project’s design features and scope were more relevant to the project’s first objective of improving access to basic health care services. The project’s design to achieve the second objective was defined by a number of studies. While those studies were relevant it was not clear how their results would lead to “reform, sustainable financing and development of the decentralized health system”. In conclusion, after restructuring, the relevance of the project’s design and scope had minor shortcomings.

16 Bank Report No. 80051-SD, “Interim Strategy Note (FY2014-1015) for the Republic of Sudan”, August 30, 2013, page iii.

. Relevance of Implementation (Rating: Moderately Unsatisfactory before restructuring; Satisfactory after restructuring).

Before restructuring the project’s implementation was relevant to the challenges facing PHC as well as to “reform, sustainable financing and development of the decentralized health system” as reflected in a number of achievements such as the distribution of bed nets, the improvement of a number of health facilities leading to increased consultations, and the initiation of a number of studies on reforms in the health sector. On the other hand accomplishments from implementation failed because of the shortfall in Government disbursements weak capacity of the national counterparts and hence the relevance of implementation was significantly reduced. Therefore before restructuring the relevance of the project’s implementation had significant shortcomings.

After restructuring, when the financial arrangements for the project had been changed and project activities reformulated, project implementation became substantially more relevant to improving access to PHC because increased funds were available for financing construction/rehabilitation of health facilities, the delivery of pharmaceuticals, services and training. The FMoH viewed the training of midwives as the most relevant aspect of the project’s implementation because trained midwives improved antenatal care, identified pregnancy problems early and, to a certain extent could arrange referrals to appropriate specialist care, therefore contributing to reductions in maternal deaths. This training was eventually a highly relevant contribution to the project’s implementation and sustainability. After restructuring the relevance of the project’s implementation had only minor shortcomings.

3.2 Achievement of Project Development Objectives (Rating: Moderately Unsatisfactory before restructuring; Satisfactory after restructuring)

The extent to which the project’s two objectives were achieved is evaluated separately.

(a) Improving access to basic health services by conflict-affected and underserved populations in four target states (Rating: Moderately Unsatisfactory before restructuring; Satisfactory after restructuring)

(i) Activities17

Chain of activities. Following the change in the project’s main focus to reproductive health the activities financed by DHSDP’s first objective were achieved through a chain of interrelated activities. The first activity was arranging project management at the Federal and State levels. This was followed by investments in primary health care service and training facilities, the training of health care professionals, and support for a

17 Additional information on project outputs is provided in Annex 2

hierarchy of health care workers such as health visitors, and assistant health visitors who supervised the services at PHC facilities and the activities of village midwives at the locality level to ensure the satisfactory quality of primary health care services. Finally, if emergency obstetric care (EOC) was required then teams at rural and state hospitals, supported in part by the DHSDP, became involved. This was an effective chain of activities. This chain of activities was the same before and after restructuring.

(ii) Outputs

Health and training facilities were improved and increased. Before restructuring the project’s performance was rated moderately unsatisfactory largely due to the fact that project design was too broad and ambitious. Only parts of the program had been fully funded due to a substantial shortfall in Government counterpart funding and weak institutional capacity. After restructuring an additional $3.5 million was available from the MDTF for the rehabilitation and construction program for rural hospitals, health training academies and various VMW training centers were enlarged. Another $2.5 million was available for improving primary health care services along with the financing of medical supplies and equipment at 58 health facilities in targeted poor localities to enhance mother and child health care. After restructuring village midwifery training programs were expanded contributing to increased access to antenatal care and deliveries for pregnant women which were assisted by skilled health staff. This was a substantial contribution to the improvement of primary reproductive health care. There were only minor shortcomings in health and training facilities after restructuring.

Considerable training achieved. Training activities financed by DHSDP were not started until after restructuring. By the final closing date 3,095 health professionals (including village midwives) were trained under this project which enhanced the professional competence and confidence of the MoH reproductive health staff in the four target states. Annex Table 2H shows the increase in coverage of villages which have formally trained village midwives in Blue Nile, Kassala and Red Sea States. For example, of the 393 villages in Kassala State 232 villages (59 percent) had trained midwives in June 2013 compared with 52 villages (13 percent) in 2009. In addition, health facilities had been improved and equipped at 46 locations compared with the target of 58. Given the difficulties this was a substantial result with only minor shortcomings.

Capacity building activities for VMWs. While the Federal MoH already had a policy of support for midwifery programs to reduce maternal mortality, the DHSDP provided additional support for this policy resulting in significant increases in trained VMWs after restructuring.18 Annex Table 2G records the number of VMWs who have graduated from training during DHSDP implementation after restructuring. It shows that 675 midwives graduated from basic training since 2010.19 The Bank’s budget financed a

18 Out of the $6 million additional financing from the MDTF-NS in the first restructuring, $2.8 million was allocated to support for village midwives and their training.

19 There are currently about 15,000 trained village midwives in all Sudan.

study exploring ways to strengthen the VMW system and make it more sustainable through the integration of midwives with women’s groups in communities. It also financed an assessment of the performance of the VMW program in Blue Nile and Kassala states.20 There were, however, no evaluations of the quality of the VMW training program and hence the substantive results have been difficult to evaluate. Nevertheless, the increase in numbers was a substantial step forward.

(iii) Intermediate Outcomes

Table 3 shows substantial results for the project’s intermediate outcome indicators.

• 179,000 insecticide treated malaria nets were distributed. The distribution covered about one-third of the rural population in the four target states and 42 percent had at least one ITN. This was achieved before restructuring. • Investment targets met. The target number of training and PHC facilities rehabilitated/equipped, and number of rural hospitals upgraded/equipped are also shown against the final achievement. In all cases the targets were met. This was achieved after restructuring.

Table 3: Intermediate Outcome Indicators – 21 Localities Item Baseline Targets Achievement 2006 2008/ Nov May June Dec Dec Dec Dec June 2009 2006 2009 2011 2012 2010 2011 2012 2013 No. of ITNs distributed a/ 0 179, 100 180,00 179,100 0 % of households with 1 18 42 42 ITN a/ Training and PHC Facilities Rehabilitated and Equipped - No. of State academies b/ 0 4 4 0 - No. of midwifery schools 0 4 7 4 7 c/ - No. of PHC facilities a/ d/ 0 18 18 18 - No. of rural hospitals 0 3 6 9 4 6 9 upgraded and equipped for EOC

Source: DHSDP Monitoring and Evaluation system and Annex 2. Note: The series of targets reflects earlier targets before the first restructuring and targets following the first restructuring a/ Completed before first restructuring. b/ Support to academies was dropped at first restructuring (to be subsequently financed by the Government). c/ Includes three midwifery schools in North Kordofan (added at fourth restructuring); d/ Includes both Health Centers (HC) and Basic Health Units (BHU).

20 Amel Aldehaib, A Report on Social Assessment of the Potential for Women’s Groups at Locality Levels in Supporting the Role and Work Done by Village Midwives”, 30 June, 2011; and Peter Bachrach, Amel Aldehaib, Mahgoub El Nour and Mohamed El Tom, “Assessment of the Performance of Village Midwives – Blue Nile and Kassala States”, prepared for the Decentralized Health System Development Project, March 2012.

(iv) Outcomes

Before restructuring the project established four outcome indicators (see Table 2); their relevance and achievements can be summarized as follows:

• Outpatient consultations per person per year. There are no data available for this indicator before 2010. It is assumed that because of the delays in project implementation before restructuring that this indicator had significant shortcomings – in line with the assessment of the project in the ISR before restructuring • Patient ratio between highest and lowest wealth quintile. This indicator was not measurable with available monitoring programs and hence it was not rated. The indicator was dropped at restructuring. • Percent of patients who do not receive health care due to financial barriers in target areas. This indicator was not measurable and hence it was not rated. The indicator was replaced at restructuring by “percent of pregnant women with at least one antenatal consultation”. • Skilled birth attendance in target areas. This indicator was not measurable and hence it was not rated. The indicator was replaced at restructuring by “percent of births attended by skilled health staff”.

After restructuring the project defined three outcome indicators for the first objective - namely outpatient consultations per person per year, percent of pregnant women having at least one antenatal consultation, and percent of births attended by skilled health staff. The average results during project implementation for these indicators compared with their targets are shown in Table 4. The following points are relevant to the evaluation of each of these outcomes.

• Outpatient Consultations per person per year in target areas increased substantially but varied considerably between localities. There was variability at the locality level due to seasonality (particularly in Red Sea State), the effects of the security situation (in two of the three target localities in Blue Nile and South Kordofan State), and the availability of drugs and consumables provided by the project. This is the only direct overall measure of increased access to basic health care and shows substantial improvement since December 2010 meeting the target established by the project. However, project documents provided no standard against which to judge the achievement and it is therefore assessed as having minor shortcomings • Percent pregnant women with at least one antenatal consultation increased. Since December 2010 (about 18 months after restructuring), the average incidence of antenatal consultations increased from 59 percent to 68 percent (3 percent below the target) due to increased coverage in Kassala and Red Sea which in turn resulted from the recruitment of midwifery school graduates by contracting and incentives through a pilot program of payments for specific services. Kassala and Red Sea States organized outreach activities. Red Sea State added a Rural Port Sudan locality in its coverage because of its high

incidence of poverty. Overall the improvement to within 3 percent of the target was an achievement with minor shortcomings. • Percent of births attended by skilled health staff. A substantial improvement from a base of 19 percent in 2008/2009 to 63 percent in June 2013. This achievement had no shortcomings.

Table 4: Trends in Outcome Indicators for First Project Objective – 21 Localities a/

Indicator Baseline Target Achievements (2008/2009) Dec 2012 Dec 2010 Dec 2011 Dec 2012 June 2013 Outpatient Consultations 0.15 0.30 0.26 0.31 0.32 0.41 per person per year in target areas Percent pregnant women 48 70 59 59 68 68 with at least one antenatal consultation Percent of births attended 19 55 23 42 52 63 by skilled health staff Source: DHSDP Monitoring and Evaluation system. a/ These data do not cover South Kordofan from end 2011 because the project discontinued collection there due to the insecurity resulting from the civil conflict.

Attribution. The focus localities for this project were among the poorest in the state and typically remote where investments in the health sector other than by the Government and DHSDP were rare. Hence most, if not all, improvements in PHC (including reproductive health services) would have been attributable to DHSDP assistance.

Maternal Death Rates. A higher order outcome indicator for measuring improvements in reproductive health would be changes in the maternal mortality rate. This was not defined as an outcome indicator for the project but information on it was collected in three Maternal Death Reviews financed by the DHSDP in 2011, 2012 and 2013. These reviews were exploratory because of incomplete and fragile data. For example, although notification of maternal deaths to the locality authorities is compulsory by law, not all deaths are reported, and when reported the cause of death is often inaccurate. The data in Table 5 suggest a decline in notified deaths, but in 2012 they are only about two-thirds of expected deaths (based on demographic estimates).

The relatively slow progress in reducing maternal death rates in Red Sea and South Kordofan was said to be due to modest infrastructure, inadequate staffing, poor equipment, variable power and no blood supplies at hospitals, as well as a dysfunctional referral system. It should also be added that one of the findings of the Maternal Death Reviews was that more than 90 percent of all maternal deaths occur among women who had no antenatal care or irregular antenatal care visits. To the extent that the DHSDP increased antenatal care it may have contributed to reductions in maternal deaths.

Table 5: Number of Maternal Mortality Deaths Expected and Notified – Sudan

Indicator 2010 2011 2012 June 2013 Total Percent Total Percent Total Percent Total Percent of of of of Expected/ Expected/ Expected/ Expected/ Notified Notified Notified Notified Number of maternal 420 433 445 445 deaths expected Number of maternal 328 78 251 58 294 66 147 23 death notified - Number notified 105 32 60 24 90 31 40 27 from community - Number notified 223 68 191 76 204 69 107 73 from RH or FHC -Number of maternal 323 98 239 95 263 89 129 88 deaths investigated Source: DHSDP Monitoring and Evaluation system and Progress Report for quarter ending June 30, 2013.

(b) Establishing the basis for reform, sustainable financing, and development of the decentralized health system (Rating: Moderately Satisfactory before and after restructuring)

(i) Activities21

A number of substantial activities were planned to be concluded soon after restructuring but they were not concluded until well after restructuring. They did, however, provide a contribution to establishing the basis for the reform of a sustainable and decentralized health system. The two most prominent were analyses of health care financing including the National Health Accounts (NHA) implemented with technical assistance from the World Health Organization (WHO), and a capacity building program in the Federal and State Ministries of Health implemented with technical assistance from an international consultancy firm.

(ii) Outputs

Health care financing in Sudan. Before restructuring in May 2009 the project contracted WHO to undertake a comprehensive assessment of health care financing in Sudan but there were significant delays in completing the contract and it was finally completed after the restructuring. The work included the preparation of national health accounts, household health services utilization survey, a review of options for the expansion of the NHIF, a review of knowledge, attitudes and practices on health care in Sudanese households, and a summary of the coverage of the comprehensive PHC package. The assessment of the cost of the PHC package by the FMoH was not delivered

21 Additional information on outputs is provided in Annex 2.

but has since been completed. This work had moderate shortcomings before restructuring. Nevertheless WHO delivered its report in June 2012. It was a substantial result complemented by follow up technical assistance and training. The NHA is regularly updated in the Ministry of Health and used in development planning. Overall this work, while delayed well beyond its original delivery date, had only minor shortcomings after restructuring.

Capacity building for planning future health care programs at Federal and State levels. Before restructuring an international consulting firm was retained to provide technical assistance to strengthen the capacity of the Federal and State Ministries of Health to plan, budget and manage health services, strengthen human resources for health strategies, and improve monitoring and evaluation of health systems at state and locality levels. The inception phase of this contract started in August 2008 and implementation was completed after restructuring in January 2010. Activities were aligned with the National Health Policy and the development objectives of the State Ministries of Health. All outputs were developed in working sessions with health professionals in the Federal and State ministries. In the area of policy reform and analysis the substantial improvement in the capacity of the State Ministries of Health has been a major achievement representing the vanguard of a decentralized and sustainable health system. This was also a substantial result with minor shortcomings. Additional capacity building support was provided from other sources including the MDTF-financed Technical Assistance Facility which financed the costs of management training for staff from the FMoH, SMoHs and the Public Health Institute at the University of Washington in Washington State (USA) who would later became trainers of trainers.

Other Studies. A number of other studies relevant to the reform of the health sector were financed by the DHSDP as well as by the World Bank’s operational budget before and after restructuring. These studies were all completed after restructuring. For example, a joint study by the Government, the MDTF-NS and the Bank’s budget was the Public Expenditure Tracking Survey (PETS) which focused on the health sector in six states, including the four target states in this project conducted in 2010 and completed in 2011.22 Another significant study was “Free Health Care for Under-Five Children and Pregnant Women in Northern Sudan: Progress so Far and Recommendations for the Future prepared by a Technical Working Group in the Federal Ministry of Health led by Sophie Witter and completed in September 2010. Following this study a one year pilot in the four target states was launched by the NHIF which incorporates the study’s main recommendations. Another study was a “Brief Assessment of the First Referral System” completed May, 2012 with a concrete proposal for a “first level component” to be included in the second additional financing for the DHSDP (fourth restructuring); however support was not strong enough and the implementation period for the project was coming to an end.23

22 World Bank, Public Expenditure Tracking Survey (PETS) for Northern Sudan – Case Study of the Health Sector”, July 2011

23 Other studies are listed in Annex 2.

On balance the technical assistance and studies intended to provide the basis for reform, sustainable financing, and development of a decentralized health system were useful to the Ministries of Health A salutary characteristic of this project was the substantial and relevant analytical work that accompanied the investment component.

(iii) Intermediate Outcomes

The FPP stated that the implementation of reform and a decentralized health system was anticipated beyond the time frame of the project (page 19) but that DHSDP would provide the “basis for reform”. No outcome indicators were defined in the FPP, but as part of the First Restructuring two intermediate outcome indicators were defined for the “basis for reform”. They were “annual per capita US$ value of drugs/consumables and operational support provided to target health facilities and midwives”, and “completion of National Health Accounts”.

• Increased annual per capita value of support to health facilities and midwives. The value of drugs, consumables and operational support to health facilities and midwives was calculated to be $0.17 per capita at the start of the project. The target at the end of the project was $0.40 per capita. In the event the average achievement for all target states was $0.36 per capita in June 2013 although it had been as high as $0.48 per capita in 2010. While the project established a target for this indicator, it did not establish a rationale for the target nor a norm or adequate level for the indicator against which the achievement could be compared. One way to assess the aggregate allocation is to compare it with the value of drugs distributed for free care by states. The latest data available show that the per capita value of drugs for free distribution to children under five and pregnant women in Blue Nile, Kassala, Red Sea and South Kordofan in 2009 was SDG 1.30, SDG0.68, SDG0.77 and SDG0.98 respectively. It was SDG0.73 in all “northern” States.24 For the four target states these per capita levels were equivalent to $0.56, $0.29, $0.33 and $0.43 at an exchange rate in 2009 of SDG2.9 = $1.0. However, evidence based on data from Red Sea state indicates that the distribution of funds within a state goes predominantly to urban hospitals and only 21 and 6 percent respectively went to rural hospitals and health centers. These data, though for three years earlier than the current data for the project, suggest that the contribution of DHSDP to the resources of rural health facilities to the target states on a per capita basis has been significant and its distribution was almost certainly more effectively targeted to the poorest and needy groups. On balance this intermediate outcome had moderate shortcomings before and after restructuring because until the assessment of the cost of the PHC package by the FMoH is available the relevance of the value of drugs, consumables and operational support to health facilities and midwives assessed by this project will not be known.

24 Federal Ministry of Health (2010), “Free Health Care for Under-Five Children and Pregnant Women in Northern Sudan: Progress so Far and Recommendations for the Future”, prepared by a technical working group of the Federal Ministry of Health led by Sophie Witter, September 2010, Table 9, page 39.

• National Health Accounts. This intermediate outcome indicator required the completion of the National Health Accounts. Annex 2 provides a summary of the achievements under this heading. The review and analysis of health care financing was comprehensive and provided a building block for planning future sustainable decentralized financing of PHC services and hence was a direct and relevant contribution to the project’s second objective. As noted already, the completion of this work was substantially delayed and hence before restructuring it had moderate shortcomings. Nevertheless the NHA was completed and this intermediate objective was achieved with minor shortcomings. However, ultimately the basis for a sustainable financing and development of a decentralized health system is not under the control of the FMoH because the FFAMC and the MOFNE control the formula for allocations to the states for basic health care programs.

3.3 Efficiency (Rating: Moderately Unsatisfactory before restructuring; Satisfactory after restructuring))

The FPP reviewed various approaches to estimating the returns from investments in primary health care, but it did not estimate an expected rate of return from the DHSDP. The efficiency of this project in addressing its first and second objectives is assessed in terms of management and implementation efficiency as well as its cost-effectiveness. The assessment is framed in terms of whether the project provided “value for money”. The analysis of efficiency in this ICR is elaborated in Annex 3.

(a) Efficiency in Improving Access to Basic Health Care

Management efficiency. DHSDP was under the energetic leadership and management of the FMoH and the SMoHs supported by the Federal and State PIUs which improved over time. The continuity of core personnel was satisfactory, except for fiduciary management, and a testament to the overall management of the PIUs. However the cost of project management was 13 percent of total costs compared with the original estimate of 5 percent. The original estimate of management costs might have been unrealistically low for the first phase in view of the legacy of conflict, and there was indeed more conflict in Blue Nile and South Kordofan during implementation than anticipated. While 13 percent is a high percentage compared with similar projects, it should be acknowledged that Sudan is classified as a post conflict country with ongoing conflict in some regions. Costs under these circumstances are known to be high25 On balance, management efficiency had moderate shortcomings before restructuring and minor shortcomings after restructuring.

25 There is no known data base on project management costs for health projects but there are data on Bank supervision costs in Fragile and Conflict-Affected States (FCS). For example, see “World Bank Assistance to Low-Income and Fragile and Conflict-Affected States: An IEG Evaluation”, 2013 which noted that “The unit cost of supervision in Always FCS IDA in FY12 was 18 percent higher than in Never FCS IDA countries”; paragraph 8.25.

Implementation efficiency. The Monitoring Agent noted that project implementation improved from its original weak phase with significant shortcomings (before restructuring) to the final phase of development (after restructuring) when implementation had minor shortcomings. After restructuring the project was focused and effective in pursuing the achievement of the project’s first objective. Almost all the implementation targets for the renovation or construction of primary health care facilities as well as training facilities, with heavy focus on the training of village midwives (VMWs), were achieved. Innovative approaches such as the organization of midwifery training at locality level near beneficiary communities to overcome traditional constraints on young female students travelling unaccompanied to main towns contributed to a high training output and increased numbers of trained midwives in remote villages. Reproductive health care (a basic health care issue) as measured by the incidence of both antenatal care and births attended by skilled health staff improved. Concurrently outpatient consultations per person per year doubled but only after restructuring. The project effectively achieved its targets. Although these were substantial achievements the three year delay at the start of the project before restructuring was not an efficient use of time or resources. In addition there was little progress in South Kordofan because of the civil conflict there. On balance implementation efficiency had significant shortcomings before restructuring but minor shortcomings after restructuring.

Cost-Effectiveness. The overall conclusion of the analysis of the value of the disability life years averted as a result of improved reproductive health and increased access to primary health care in Annex 3 was that for each dollar of cost the project generated two dollar’s worth of benefits because of reduced maternal mortality and the diseases averted through increased access to primary health care. Despite the many assumptions that needed to be made to complete the DALY analysis, results suggest minor shortcomings in efficiency because the present value of benefits is twice the present value of costs. In addition this analysis does not include any assessment of net social benefits from reduced mortality or disability for families and society. In summary, cost effectiveness had minor shortcomings after restructuring. Because the project was not on track with a poor project design, shortfalls in financing and not able to achieve anticipated benefits before restructuring there were major shortcomings in cost-effectiveness during that period.

(b) Efficiency in establishing a basis for reform, sustainable financing, and development of the decentralized health system.

While WHO completed the contract to analyze health care financing, contract implementation was delayed for various reasons including the inclusion of additional activities financed by sources other than DHSDP. Also, one sub-contract with the FMoH for estimating the cost of the basic health care package was not finished and the impact of the NHA on actual allocations of federal funds to the states for primary health care is not clear.. However the FMoH advised that the improved capacity for health policy analysis in the Ministry enhanced the “basis for future reform, sustainable financing, and development of the decentralized health system”. By this measure NHA was effective,

but not necessarily cost-effective – and hence it is concluded that the efficiency of NHA had moderate shortcomings before and after restructuring.

3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory

Relevance. The relevance of this project’s first objective, design and implementation before restructuring was rated as moderately unsatisfactory (see Annex Table 2J for a summary of all ratings based on the text above). Relevance after restructuring was rated as satisfactory. . The project’s relevance to reforming the health sector to put it on a sustainable path to a decentralized system (i.e. the second objective) was also rated moderately unsatisfactory before restructuring and satisfactory after restructuring.

Achievements. Before restructuring it was only possible to rate one of the original four key performance indicators for the first objective because three of them were not measurable which was the reason they were changed at restructuring. The achievement based on the one measurable indicator was rated as moderately unsatisfactory. At exit, using this indicator and two of the other three, the project was rated as satisfactory. With respect to the second objective, considerable capacity building through an intensive training program was achieved before and after restructuring. The project also provided a basis for the reform of the health sector because of better data on health care financing (NHA) and improved capacity for health policy analysis in the FMoH even though there was no definition of exactly what was expected from this part of the PDO. The project’s achievements were rated as moderately satisfactory before and satisfactory after restructuring.

Efficiency. A rate of return for this project was not estimated because of the difficulties in measuring the benefits flowing from the achievements relevant to the first objective. There were, however, moderate shortcomings in the project’s managerial efficiency before restructuring.. On the other hand implementation efficiency after restructuring had minor shortcomings. The cost-effectiveness with which this project has delivered improved basic health care services to pregnant women and new born children could not be directly assessed. However an estimate was made of the disability adjusted life years (DALYs) after restructuring and the project was found to be cost-effective (see Annex 3 for the analytical basis for this conclusion). Before restructuring the project had used 36 percent of funding and shown few results supporting to the conclusion that there were major shortcomings in efficiency before restructuring.

In summary, based on the conclusions above, the project outcome before restructuring is rated as moderately unsatisfactory. After restructuring project outcome is rated as satisfactory. A weighted average outcome for the project’s phases before and after restructuring (weighted by disbursements in each phase) is moderately satisfactory. Annex Tables 2J and 2K provide the basis for this result.

3.5 Overarching Themes, Other Outcomes and Impacts

(a) Poverty Impacts, Gender Aspects, and Social Development

The project targeted the poorest 21 localities in four target states. It supported Government policy that primary health care should be free for all and all health care should be free for the poor. Gender aspects permeated the project from the start. Most of the suggestions in the “Social and Gender Assessment” prepared at project preparation were adopted. After the first restructuring the project was mainly oriented towards benefitting pregnant women. Communities were prominent in project implementation such as in supporting the work of village midwives.

(b) Institutional Change/Strengthening

The project built capacity in the Federal and State MoHs through training programs and empowering the state ministries. The status of village midwives was raised and their success as an institution was reflected in the increased proportion of antenatal consultations and births attended by skilled health staff during the project.

The FMoH used the DHSDP model to obtain additional funds from the GAVI and Global Funds for respectively $16.1 million and $30 million for activities aligned with the DHSDP. Another unexpected impact was that World Bank budget funds and the MDTF- NS Bank–executed funds (allocated to the Technical Secretariat) were attracted for analytical work on the health sector in Sudan.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops

No stakeholder workshop was conducted although regular consultations with beneficiaries were held to seek their feedback on the project during implementation..

4. Assessment of Risk to Development Outcome Rating: Significant

As the Assessment of Program Sustainability prepared by the MDTF-NS states, “replicability of MDTF-NS projects is closely related to their sustainability”26. It is therefore salutary that the Federal Ministry of Finance and National Economy (MOFNE) decided to replicate DHSDP and will finance a long term program to continue the improvement of primary health care in Sudan. The Ministry has already launched the “Primary Health Care Expansion Project” modeled on the DHSDP which includes a component (30 percent of project cost) for the improvement and new construction of

26 Multi-Donor Trust Fund – National Sudan, “Assessment of Program Sustainability”, October 23, 2011, page v.

rural hospitals. Government funding is anticipated to be SDG 70 million annually (US$13 million) although, as mentioned already, the states need to contribute significant operational costs. Nevertheless, institutional capacity to support sustained implementation will be a challenge for this project.

Although it is Government policy to provide free primary health care, reports show that patients are paying a significant part of basic health care costs.27 It is likely that future primary health care services will not be sustainable without some cost recovery through fees and health insurance programs. About 30 percent of the population (mainly employees of the government and private companies in urban areas) is currently covered by health insurance (predominantly the National Health Insurance Fund - NHIF). The NHIF and the Government are assessing ways to increase this percentage in rural areas. DHSDP supported this goal through studies of necessary conditions for health care financing and the costs of universal primary health care coverage.28

At the national level ownership of the DHSDP model is strong, but financial and political risks in Sudan are substantial. On the financial side budgets are fragile and comprehensive health insurance is still a long way from being established and hence even partial cost recovery in poor rural areas will be a huge challenge. Political risks are hard to assess but the most difficult issue is the reliability of increased allocations of the federal budget to the states. On balance the risks to the continued implementation of the DHSDP model with its substantial subsidies are significant.

27 See Sophie Witter, op cit, pages 20 and 123

28 Federal Ministry of Health, Secretariat for Planning, Policy and Research, “A Review of the National Health Insurance Fund”, September 2012, managed by the World Health Organization Health Systems and Services Development Unit. See also Xavier Modol, “A Brief Analysis of the FMOH Universal Coverage by Essential Package of Primary Health “, November 2102; and Xavier Modol, “Care Universal Coverage by Essential Package of Primary Health – Project Proposal”, March 2013

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance

(a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory

The project was strategically relevant to Sudan’s health sector. It was strategic because it was in line with the Government’s health policy, addressed poverty reduction, and was limited to 19 poor localities in four states. The FPP was well prepared and comprehensive with respect to technical issues but too ambitious in terms of the scope of activities with project indicators that were weak and in some cases not measureable. The QAG learning review also questioned the project’s ambitions and added that the FPP did not describe the project clearly. The FPP did, however, provide a sound basis for empowering both Federal and State MoHs to take the responsibility for project implementation. While QAG also noted that the FPP was too sanguine about the Government’s assurances on the payment of counterpart funds, the financing arrangements in the project followed standard practice because it had been agreed formally when the MDTF-NS was established that the Government would contribute twice the MDTF contribution. In the event, for various reasons, the Government was not able to meet its financial commitment to this project which created considerable delays during the project’s first two years.29 On balance the overly ambitious scope within a short time frame, a project description that needed elaboration and the shortfall in the Government’s funding suggest that quality at entry had significant shortcomings.

(b) Quality of Supervision Rating: Satisfactory

The project was competently supervised by TTLs based in Washington during the project’s first phase and from Khartoum for the remainder of the project when the TTL was a staff member of the MDTF-NS Technical Secretariat. Typically there were two to three supervision missions per year. The quality and effectiveness of supervision together with the technical assistance provided at various stages during implementation to the FMoH (e.g. assistance with the project’s annual budgeting) was appreciated by the Federal MoH staff and management. Supervision reports were detailed and the TTL maintained regular contact with the Federal MoH, Federal PIU and State PIUs. Problems or issues were quickly resolved. Senior management of the FMoH appreciated the support and advice from the Bank/MDTF-NS Technical Secretariat. Financial management and procurement were systematically supported and compliance with safeguard policies was closely monitored. Overall there were only minor shortcomings in project supervision, the main ones being the delays in arriving at an agreement within the Bank and between the MDTF Secretariat and the Government on the terms and conditions of the project’s first restructuring.

29 The Final Project Proposal had indicated the possibility of delays in Government contributions (page 63)

With significant shortcomings in quality at entry and minor shortcomings in supervision the overall performance of the Bank and the MDTF-NS was moderately satisfactory.

5.2 Borrower Performance

(a) Government Performance Rating: Moderately Unsatisfactory

The Government, through the FMoH, was a strong supporter of the DHSDP and the FMoH Directorate of Planning was closely involved with the project’s design. But, as it had to prioritize its financial commitment to the MDTF-NS to other MDTF projects (Census and New Currency projects which required substantial funding), the Government was not able to meet all its financial commitments to DHSDP. This slowed progress at the start and was one reason for the First Restructuring. Restructuring substantially reduced the Government’s direct contribution, but the Government asserted that it made contributions to ongoing costs of PHC through its regular budget. This was partially verified, as some of the project’s activities that were supposed to be paid by Government were finally implemented (e.g., rehabilitation of training academies at state level). However, a matter that posed serious problems during implementation was the inadequate professional staffing at DHSDP-improved facilities due partly due to extreme difficulty in deploying qualified staff to more remote areas. These were significant shortcomings in the Borrower’s performance.

(b) Implementing Agencies’ Performance Rating: Satisfactory

The Federal and State PIUs functioned efficiently and provided the project with much energy and excellent facilitation and leadership. At the State level PIU staffing was the same throughout implementation. The Government’s ICR ascribes this mainly to the project’s clear objective. Most of the project’s implementation was done effectively by departments and units of the Ministries of Health at all levels. This resulted from empowering a ministry to be responsible for project implementation. There were minor shortcomings in the performance of the PIUs.

Based on a moderately unsatisfactory performance by the Government and a satisfactory performance by the implementing agencies the overall performance of the Borrower was moderately satisfactory.

6. LESSONS LEARNED

Lessons with general relevance discussed below supplement a number of interesting lessons in the Government’s ICR

Projects overloaded with activities create implementation problems. It is tempting for a government and task managers, when provided with an opportunity to address post conflict development challenges, to load a project with numerous activities which are all seen as priorities. This occurred in this project. Under the weight of Government requests too many activities were planned in a weak institutional implementation environment creating substantial implementation problems. The clear lesson is to keep project objectives and scope limited and consistent with implementation capacity.

Project restructuring should be done as early as possible. Restructuring early to resolve implementation problems is important to give time for a successful response to implementation constraints. Delayed restructuring of a dysfunctional project takes longer because commitments made during a project’s early phases can usually not be undone and that reduces the degrees of freedom for restructuring.

Lessons relevant to this project were as follows:

Flexibility in design is a major advantage. A core feature of the design of this project was the empowerment of the state MoHs by the Federal MoH to manage the project’s activities. This in turn depended on competent PIUs which were empowered to take decisions, make progress at the locality levels and achieve results. It was this flexibility, which the government’s ICR also recognized that, for example, provided the scope to move basic midwifery training from schools in state capitals to locations close to communities to overcome traditional constraints on women moving to cities and towns (away from their villages) for education.

Complexity creates problems. Implementation was expected on a wide range of activities from sector analysis, health facility construction, and developing the capacity to address all diseases posed an overwhelming management challenge for weak health sector institutions in the states and progress was slow. Once the range of activities was substantially reduced at the first restructuring the project’s progress accelerated as it became more focused on improving reproductive health within an unchanged overall set of objectives.

Future sustainability of primary health services will depend on the actions of Federal and State public health institutions responsible. To make affordable primary health care services sustainable Governments will need to provide adequate financial resources for operating and maintaining primary health facilities, establish technical and institutional capacity to provide the services, stimulate fiduciary and risk management, and establish a sustainable health insurance program. Without these elements the sustainable provision of primary health services in Sudan is unlikely.

Strategic studies can provide information for future planning. This project financed many studies which provided the basis for future progress on difficult issues. For example, in parallel with the project’s focus on improving reproductive health it financed a series of maternal death reviews which over time will fill important information gaps on the causes of maternal deaths for which data at present are weak. This information could provide the basis for future investments to reduce the rate of maternal mortality.

7. Comments on Issues Raised by Grantee/Implementing Agencies/Donors

(a) Grantee/Implementing agencies

The FMoH reviewed a draft of this ICR and provided a number of important pieces of technical information. As this ICR was developed the FMoH generously contributed additional information on aspects of the project’s implementation and performance as well as on reproductive health in Sudan.

(b) Cofinanciers/Donors

None

ANNEX 1: PROJECT COSTS AND FINANCING

(a) Project Cost by Component Appraisal Actual/Latest Percentage Components Estimate b/ Estimate of Appraisal (USD millions) (USD millions) 1. Expand Access to Primary Health Care 19.00 7.26 38.2 Services

1.1 Improvement of Quality of PHC in 7.60 3.26 underserved areas 1.2 Expanded Coverage of PHC & High 9.4 4.00 impact interventions in underserved areas 1.3 Pilots to improve access 2.0 0

2. Strengthen Health Sector Reform and 50.4 9.94 19.7 Decentralization

2.1 Policy Development/Sector Capacity 11.5 7.58 Building

2.2 PHC Human Resources Development a/ 10.9 0

2.3 Investment Infrastructure and Equipment 24.5 0 a/ 2.4 Project Management 3.5 2.36 67.4

Unallocated 0.00 0.00 0.00 Total Baseline Cost 70.00 17,20 24.6 Physical Contingencies 0.00 0.00 0.00 Price Contingencies 0.00 0.00 0.00 Total Project Costs 70.00 17.20 24.6 Project Preparation Costs 0.00 0.00 0.00 Total Financing Required 70.00 17.20 24.6 a/ Sub-components 1.3, 2.2 and 2.3 deleted during Fourth Restructuring.; infrastructure costs recorded under sub-component 1.2 after restructuring. b/ Cost of original project. Only first phase implemented. (b) Financing Appraisal Actual/Latest Type of Co- Estimate Estimate Percentage Source of Funds financing (USD (USD of Appraisal millions) millions) Government of Sudan 47 3.2 6.8 Multi-Donor Trust Fund - North 23 14.00 60.9 Sudan Total Financing 70.0 17.20 24.6

ANNEX 2. OUTPUTS BY COMPONENT

This is a summary evaluation of outputs from the project’s two main components and their sub-components. As mentioned in Section 1.6 of this ICR the project’s components were restructured with some additional financing and delays in closing dates at various times during implementation. This annex will therefore assess outputs before and after restructuring. Annex Table 2A summarizes the impact of restructuring on the project’s components. Annex Table 2B summarizes physical outputs by states and the overall training of health professionals achieved aggregated across four states.30

In 2011 and 2012 Blue Nile and South Kordofan experienced severe civil conflict which led to a temporary halt in activities in Blue Nile but with a major impact on the Bao locality. The conflict stopped most implementation in South Kordofan. As a result the PIU in South Kordofan was disbanded and implementation suffered substantially. Therefore results in South Kordofan were often less than the targets and also not recorded from the end of 2011.

Component 1: Expanding access to primary health care services by underserved populations.

This core component squarely addressed the project’s first objective to improve access to primary health care (PHC) services for people in the former conflict-affected areas in four target states. It was originally designed to provide assistance in two domains, namely improving the quality of existing primary health care services, expansion of coverage of primary health care services and high-impact interventions, and piloting experiences to reduce barriers to access to primary health care. The scope of this component’s activities was changed as a result of the project’s first and fourth restructuring. It was renamed and consolidated. The First Restructuring reduced the scope of basic health services covered by this project and sharpened the focus of its activities to “reduce maternal and under-five mortality rate.”

Sub-Component 1.1: Improving the Quality of Existing Primary Health Care Services. Before the First Restructuring the programs for procurement of medical supplies for health facilities, and for the training of primary health care workers and for midwife refresher courses were started. The rehabilitation of primary health care facilities was also started but in the Final Project Proposal most construction costs were included in Component 2. These activities were relevant to the project’s objectives of improving access to primary health care.

30 These are the four target states. Some activities in North Kordofan which were added during the Fourth Restructuring will be mentioned separately.

Outputs

• Annex Table 2B shows the achievements in terms of numbers of health care facilities that benefitted from the distribution of medical supplies, equipment, emergency vehicles, and the construction program. Targets were met. This was a satisfactory performance.

Sub-Component 1.2: Expansion of Coverage of Primary Health Care Services and High-Impact Interventions in Un-Served Areas. During the first two years of project implementation this sub-component financed the purchase of 179,000 insecticide treated bed nets to 100,621 households in the four states implying just under 2 bed nets per household and, assuming 6 member households, the program would have potentially benefitted some 603,700 people predominantly in rural areas. This program was discontinued after the First Restructuring. It is relevant to note that UN agencies such as UNICEF distributed malaria bed nets.

Before the First Restructuring the project also supported programs to control a number of serious diseases prevalent in Sudan such as leishmaniasis, tuberculosis and schistosomiasis. In South Kordofan the project funded investments to support the immunization cold chain. The Government’s contribution to this sub-component was through its policy of providing free basic health services to pregnant women and children under-five. Until the First Restructuring the project supported the supply of drugs, medicines and incentives to PHC facilities. This support from the project was terminated following the First Restructuring in favor of providing more focused primary health care support.

Re-orientation of priorities in PHC and additional funding in the First Restructuring led to a major focus on improving the availability of trained midwives. For example it provided a pilot for performance-based incentive payments for village midwives and a modest increase in the funding for training of PHC workers and midwives. The incentive payments were aimed at ensuring that pregnant women had access to a basic package of quality services during pregnancy and at child birth. Payments were made for the registration of women in the village who are pregnant, upon completion of the third ante-natal visit, upon delivery and two post-natal visits, upon notification of the death of the mother or her neo-natal infant within 40 days if either occurs. Midwives were also expected to raise awareness of the need for maternal health services in villages and to keep accurate and systematic records. If all these performance criteria were achieved a midwife would receive SDG40 per pregnancy which she supported and attended.

The sub-component also financed the purchase of two ambulances – one each for Red Sea (located at Tokker and observed by ICR mission) and Kassala states (located at Kassala) where, under the management of the locality governments, they are typically used to transport patients needing emergency care from rural health centers to state hospitals. Mobile clinics (four for Blue Nile and one each for the other two states) were also procured. They provided urgently needed transport for pregnant women referred to state hospitals for emergency obstetric care.

In Blue Nile State the project funded a performance-based contract for 12 months between the state MoH and an international NGO which led to the provision of basic health care services for 15,000 returnees from Ethiopia who had been displaced because of conflict. A final report on this activity was prepared which showed that the contract’s objectives were achieved.31

Outputs

• Assuming that 70 percent of the population lives in rural areas then the bed net distribution covered about one-third of the rural population in the four target states. On the basis of the distribution to the target area for distribution a coverage was 42 percent was achieved • The provision of additional medical supplies and services to improve primary health care services in un-served areas had a positive impact on a wide array of diseases during the first two years of the project. Data from DHSDP monitoring showed that when these additional supplies were available outpatient attendance increased by 30 percent. • Records from the localities which administer the incentive payments for midwives showed that, while an increase in the number of midwives was slow the rate of increase has accelerated to the extent that by January-April 2013 on average 68 percent of pregnant women received ante-natal care (an increase of 42 percent over the baseline), more than 60 percent of deliveries were assisted by midwives in (an increase of 230 percent over the baseline) but with a wide variation between states reflecting the conflict in 20111/2012 in Blue Nile and South Kordofan (see Annex Table 2E). • In the four target states there has also been a decline in the number of deaths notified as a percentage of the reported pregnancies (see Table 3 in the main text). However, it has not been possible to attribute the decline in the proportion of maternal deaths during child birth to either the increased number of midwives, the incentives given to midwives, or to the quality of the primary health care facilities. • Ambulances were an important part of the referral system to ensure access to PHC in state hospitals in Red Sea and Kassala if emergencies occurred such as for women during delivery. However, mobile clinics were not very successful in providing PHC to remote areas because of the poor quality of rural roads in these areas. In South Kordofan the ongoing conflict made the use of mobile clinics hazardous.

Component 2. Establish the basis for reform and development of a decentralized health system. This was relevant to the second part of the PDO, namely “establishing the basis for reform, sustainable financing, and development of the decentralized health system”. However, a number of activities included in the FPP for this component, such

31 GOAL, (2009), “Delivering Basic Package of Health Services to Chali Area”, April 2008-2009, Final Report submitted to the MoH, Blue Nile State.

as the substantial infrastructure construction program, could have more logically been included in the domain of the first component. This anomaly was addressed to some extent in the project’s restructuring when sub-components were re-arranged.

Sub-component 2.1. Capacity-Building and Policy Development. Four consultancy services and studies were planned at appraisal, namely (a) Health Care Financing; (b) Access to Pharmaceuticals; (c) Health Planning and Budgeting; and (d) Monitoring and Evaluation. Results of these studies are summarized below. The First Restructuring added funding for technical support for and supervision of midwives, and a maternal death audit system. There were also a number of studies financed separately by the MDTF-NS Technical Secretariat (not financed by the DHSDP) which are listed in Annex Table 2C.

(a) Health Care Financing.

The project commissioned WHO to undertake a Health Financing study. It was completed in October 2012 and covered (i) National Health Accounts; (ii) Household Health Services Utilization; (iii) Review of the National Health Insurance Fund; (iv) Comprehensive PHC package; (v) Cost of the PHC package; (vi) Knowledge, Attitudes and Practices (KAP) Study of Household Health Behavior.

(i) National Health Accounts (NHA). The preparation of the NHA was completed using a range of consultants and some additional financing obtained by WHO from sources other than the DHSDP. This work delivered health accounts for all 15 states and provided the federal MoH with a crucial framework and complete data on health expenditures by sources in Sudan.

Outputs

• The NHA has been institutionalized within the Federal Ministry of Health with the support of staff training by WHO. The data are regularly updated by the Health Economics Unit in the Ministry and used in its regular planning and budgeting processes. • The NHA has also provided relevant information to the Fiscal and Financial Allocation and Monitoring Commission (FFAMC) which in the longer run should provide the basis for sustainable financing of a decentralized health system and contribute to more effective central government allocations of funds for health care to the states. • The NHA has led to a demand for the generation of sub-accounts for various specific Government expenditures such as for malaria prevention with implications for facilitating applications for financial assistance from the Global Fund. • Another result is that the NHA has led to a health economics course in the Economics Faculty in the University of Khartoum. • Overall, the NHA was a significant contribution to health sector reform .

(ii) Household Health Services Utilization Survey (2009). The survey was intended to provide data to explore the public’s expenditure patterns for health care services. No formal evaluation of the quality of this survey was found in the project files.

Outputs

• Results from the survey provided data for the construction of the NHA. • Survey information will be relevant to the National Health Insurance Fund’s future development program. The survey provided important information for the health sector reform program. • The survey results changed the Ministry’s understanding of the structure of health care financing in Sudan and the relatively small importance of the Government’s funding of health care. For example the survey showed that 80 percent of annual health care expenditures by households come from “out of pocket” expenditures.32

(iii) Review of the National Health Insurance Fund (NHIF).33 The National Health Policy aims to achieve universal coverage of basic health care to all citizens in Sudan through the state health insurance funds (SHIFs). This review was aimed at assessing current NHIF coverage in the states and suggesting strategies for increasing that coverage.

Outputs

• Four intervention areas were suggested, namely stimulate SHIF incentives to actively pursue increased enrolment, especially in the informal sector, mobilize increased resources for social support, increase access using provider payment mechanisms to increase attractiveness of setting up clinics in under-served areas and to give health care providers an interest in promoting enrolment in health insurance, and develop close-to-client health insurance by expanding the use of delegated collection of contributions and exploring and evaluating the use of mobile phones for payment of contributions and health messages.

• The NHIF has taken account of this study in its assessment of future strategies to extend its population coverage in line with the Government’s policies on free health care.

32 Household Health Services Utilization and Expenditure Survey (2006), Executive Summary, page 2.

33 According to its last Annual Report (December, 2012) the NHIF is currently represented at 1,347 health care facilities in Sudan including 145 facilities in the four target states in the DHSDP

(iv) Comprehensive PHC Package. This document starts by defining the different levels of primary health care followed by a brief review of the major illnesses in Sudan and the main primary health care programs. It then documents significant features of primary health care in Sudan based on a Health Facility Survey in 2010.

Outputs

• The survey found that almost 40 percent of PHC facilities suffered from a shortage of human resources (i.e. qualified staff), only about 60 percent of PHC facilities were connected to a source of safe water, and 70 percent of PHC facilities had no source of electricity. The bulk of the document is a set of tables describing the services theoretically available for certain diseases at different types of primary health care facilities.

• This document alerted the authorities to the weak overall status of the primary health care facilities.

(v) Cost of the PHC Package. This assessment is being prepared by the federal MOH. It has not yet been completed although, apart from providing information for budgeting purposes, its potential impact on increased access to PHC is not clear.

(vi) Knowledge, Attitudes and Practices (KAP) in Sudanese Communities for their Health Behavior.34 This study assessed the knowledge, attitudes and practices towards selected PHC services in 15 communities which suffered from poor living and sanitary conditions, including lack of toilets, and improper solid waste and water disposal.

Output

• The study concluded that “the importance of personal hygiene and sanitary practices increased significantly with educational level and financial status. In addition, health financing is seen by almost all respondents as a burden on the family, as out of pocket expenditure was reported to be the main source of health financing.” • The study recommended that literacy should be enhanced to boost the utilization of health care services, awareness of primary health care and services needs to be promoted, regulatory efforts and intervention programs should be focused on improving the living conditions of the poor and the disadvantaged the location and working hours of PHCs should be made more accessible to underserved communities, the capacity

34 Kassam M. Kasak, Rania Hussein and Nahed Abdelgadeir, “Knowledge, Attitudes and Practices (KAP) in the Sudanese Communities for their Health Seeking Behavior”, September 2011.

of PHCs to deliver high quality services should be improved the efficiency of referral systems should be improved key media agencies and personalities should be trained in primary health care to improve the effectiveness of media messages on PHC strategies and action plans. • While health administrators must have been aware of most of these recommendations, they served to reinforce the urgency of reform in the health sector in order to improve access to PHC services.

(b) Access to Pharmaceuticals.

This assessment was not completed before the First Restructuring and dropped.

The FPP proposed that this contract support "Health planning, budgeting and management in target State and Locality health administrations. Capacity-building and training would focus on the Ministries of Health in four target states. It was to include institutional assessments leading to recommendations on organizational reform and development, and technical assistance and training in planning, budgeting, management and supervision". These activities were all addressed by the consultant company, although the attention to these issues at the locality level was modest

Rather than a broad capacity building exercise for improving access to PHC services (as the FPP had clearly intended) this activity became an exercise in capacity building for planning - but without much focus on policy formulation although it was “aligned to the aims of the National Health Policy or to the development objectives of the State 35 MoHs”.

There were many workshops and training sessions, but no specific description of the content of the training programs in the consultant’s completion report. They were broadly described as "training in planning through workshops and training sessions at the federal and state level "and "planning workshops" at the federal and state levels. The conclusion is that MoH staff and others attending the workshops received training in how to make plans. As a result the consultant’s completion report is largely about capacity building for planning and future plans to address those problems and an investment plan for the state and locality public health institutions. The outcome of the BMB program is difficult to evaluate, but there were a number of outputs.

Outputs

• The analysis and planning under this consultancy culminated in "profiles" of the four states’ health status and health care services as well as "state

35 Mott MacDonald, “Decentralized Health System Development Project – Technical Assistance Component”, Implementation Completion Report – Volume i: Main Text, page 6.

investment plans" with budgets attached for a five year program of capacity building programs that included capacity building for public health staff (achieved mainly in 2011) and a future investment plan for activities in each state such as rehabilitation of state and locality health sector infrastructure. Presumably the state officials prepared the plans but the consultant company did not include them in its completion report. • It is not clear how these investment plans related to the ongoing activities in the project or the future plans of the federal and state Ministries of Health. For example, were they complementary or additional to the investments funded by the DHSDP? • The consultant‘s completion report states that the intention was to "formulate detailed investment plans to develop the decentralized health system, to enhance PHC services delivery and to provide a model of planning-cycle management that can be extended to other Sudanese States." If this is a summary of the terms of reference for the consultancy then the consultant has achieved that goal, but it does not reflect the objective for sub-component 2.1 of the FPP. • The consultant’s completion report provided no information on the quality of the training programs – as evaluated either by the trainees or a third party. There was also no evidence of what plans were produced or their utility.

(d) Monitoring and Evaluation

The Sudan Household Health Survey (SHHS), conducted in 2006. was implemented by the Federal Ministry of Health and the Central Bureau of Statistics (CBS) representing the then Government National Unity (GONU), and the Ministry of Health in Southern Sudan and the Southern Sudan Commission for Census, Statistics and Evaluation (SSCCSE).

Outputs

• This survey provided a baseline data for some of the project’s intermediate and final outcome indicators. However it is necessary for the Government to explain why the baseline for the maternal death rate in the table of project indicators for 2006 is not the same as the maternal death rate shown in the survey results for 2006.

• The PIU, the FMoH and the Bank undertook intensive reviews and consultations of the appropriate M&E framework for the project.

• The FPP for DHSDP included a sub-component for designing a comprehensive national M&E system, and to assist four targeted northern states in its implementation. A comprehensive review of the M&E system was carried out which resulted in the elaboration and adoption of a five- year strategy for strengthening M&E, namely the “Results-oriented

Monitoring & Evaluation Strategy Health Sector of Sudan 2010-2016”, which was formally adopted in August 2009.

Sub-Component 2.2. Developing Primary Health Care Services

Activities proposed were focused on strengthening the competence human resources in the Ministries of Health and in the locality health offices in the four target states with a focus on medical technicians, nurses and midwives. These studies are listed in Annex Table 2C and were completed at a satisfactory standard. The indirect benefits to MoH staff through participation in a study such as the annual Maternal Death Review are shown in Annex Table 2F and are likely to have had a substantial impact.

Sub-Component 2.3 Investments

Following the first restructuring nine rural hospitals were to be partially rehabilitated (maternity ward and surgical unit only), in preference to the rehabilitation of health centers and basic health units, to support the new focus on reducing maternal death rates in the four target localities in the four target states. In the event only seven were rehabilitated (see Annex Table 2B).

Outputs

• A consultant civil engineer reviewed the quality of construction financed from the additional financing included in the first and fourth restructuring program. The review concluded that the “In general the standards being achieved are satisfactory” (May 2013 report – as amended). • On the other hand in a report dated March 2012 the Monitoring Agent for the MDTF-NS noted that there were a number of deficiencies and defects in completed DHSDP civil works which at that time would cost $574,715 to remedy. No information was available about whether these deficiencies and defects had been addressed before the project closed. The lack of a surgical unit at Bados rural hospital in Blue Nile was addressed as part of the Fourth Restructuring. Minor issues at other sites have been addressed by the MoH.

Annex Table 2A: Summary of Project Restructuring History in DHSDP

Item/Component Final Project Proposal/Initial Grant First Restructuring/First AF of $6 million Fourth Restructuring/Second AF of $2 (October 26, 2006) (May 15, 2009) million Support improvement and expansion in basic health Focus narrowed to support for Sudan’s (December 14, 2012) service delivery targeting conflict-affected and poor National Strategy for Reproductive Health Continued focus on reproductive health with states; and establishing the basis for reform, and the Government’s Road Map for consolidation of achievements under the 1st sustainable financing and development of the Reduction of Maternal and Newborn Mortality AF and contribution to better sustainability decentralized health system. beyond project closure Closing Date 12/31/2009 6/30/2011 6/30/2012 Government/ $13 million/$6 million $3.2 million/$12 million $3.2 million/$14 million MDTF Financing Component 1 Expanding access to primary health care services Component 1 remained unchanged. Component 1 remained unchanged. by underserved populations

Sub-Component 1.1 Improvement in the quality of existing primary Changes in Sub-Component 1.1 included: No changes in Sub-Component 2.1 between health care services in underserved areas, through: the First and the Second AF - Injection of resources into PHC, aiming at - No. of targeted health facilities reduced improving service quality and allowing to 58 – priority to the poorest localities. reduction in consultation fees for patients. - Focus on the free-of-charge PHC package • Integrated package of support to targeted for pregnant women and children under 5 health services; - Performance-based incentives extended • Incentives for relocation of health workers; to village midwives (VMWs) and rural • Rehab & maintenance of infrastructures; hospitals. • Procurement of equipment & - Rehabilitation and equipment shifted consumables. from HCs & BHUs to targeted rural hospitals – including what was needed for provision of comprehensive EOC. - Procurement extended to midwifery kits and supplies for VMWs. Sub-Component 1.2 Expansion of coverage of primary health care Changes in Sub-Component 1.2 included: Changes in Sub-Component 1.2 between the services: - Focus on the free-of-charge PHC package First and the Second AF included: - Mobile and temporary clinics; for pregnant women and children under - Increase in mobile outreach coverage in - Finance private-for-profit and non-profit firms and 5. Kassala and Red Sea States. Inclusion organizations to provide services on a contractual - Contracts with international NGOs of Rural Port Sudan to the targeted basis. (GOAL) re Kurmuk/Blue Nile State, and localities in Red Sea State. SCF-USA re Kauda in South Kordofan High-impact interventions in un-served areas: State closed/discontinued. - Priority disease control interventions (HIV- - Midwife kits, supplies and incentives AIDS, Malaria, Tuberculosis, etc.) - Provision of INTs discontinued Sub-Component 1.3 Pilot experiences to reduce barriers to access to Changes in Sub-Component 1.3 included: Changes in Sub-Component 1.3 – between primary health care services: - Pilots in the FPP were dropped because the First and the Second AF included: 46

- Subsidies for PHC services; of inadequate specification of the pilots - Pilot activities designed to strengthen - Expanding health insurance coverage; in the FPP village women’s groups to increase - Interventions designed to address gender- - Piloting of performance-based incentives demand and utilization. related barriers extended to VMWs and targeted rural hospitals was added. Component 2 Establishing the basis for reform and Component 2 remained unchanged. Component 2 remained unchanged. development of the decentralized health system Follow-up of some of the studies’ recommendations (e.g. Free Health Care study) and complementary study (First level referral system).

Sub-Component 2.1 Capacity-building and policy development: Changes in Sub-Component 2.1 included: Changes in Sub-Component 2.1 included: - TA to provide stronger information based on - VMWs-related assessments were - TA to design a Pilot for free health care health financing (support to NHA, study on the conducted under the First AF – both on for pregnant women and children under NHIF, small household surveys, etc.); the institutional and the community side 5, in three states (following - TA for studies on Pharmaceuticals; – aiming at providing guidance towards recommendations of the NIHF-related - Training in health planning, budgeting and their sustainability. study conducted under 1st AF). management by target states. - TA to conduct an assessment of 1st-level - Capacity building in M&E functions at all three referral system in 2 localities in one of levels of Govt. the targeted states (Kassala). Sub-Component 2.2 Development of primary health care human Changes in Sub-Component 2.2 included: Changes in Sub-Component 2.2 – from the resources: - Training support shifted to/focused on 1st to the 2nd AF – included: - TA to assist the development of a sector-wide VMWs – both on-the-job training and - Rehabilitation/construction and HRH strategy & policies, with focus on PHC pre-service training with new curricula. equipping of Midwifery Schools cadres; - Rehabilitation/construction and expanded to a new state (North - TA to carry out a needs assessment to equipping focused on Midwifery Schools Kordofan ). determine the HRH needs in the four target (in addition to targeted rural hospitals). states, with focus on PHC workers; - TA to review PHC training programs & curricula; - Upgrade/physical renovation and equipment for Nurse and Midwifery Schools. Sub-Component 2.3 Investment in primary health care infrastructure Changes in Sub-Component 2.3 included: Changes in Sub-Component 2.3 – from the and equipment: - Upgrade & expansion of infrastructure First to the Second AF included: - Upgrade & expand PHC infrastructure in the shifted from HCs and BHUs to targeted - Upgrade and expansion of rural four target states – incl. HCs and RHs rural hospitals – with a focus on hospitals expanded to 3 additional providing first referral services, notably provision of emergency obstetric care. hospitals in target states (Red Sea, emergency obstetric care. Kassala and Blue Nile) Sub-Component 2.4 Project implementation: No changes in Sub-Component 2.4 No changes in Sub-Component 2.4 - Finance personnel & resources necessary to manage the project and coordinate project activities at federal and state level.

Annex Table 2B: Physical Outputs and Training – Results Compared with Plans after Restructurings Achievements Red Blue South Physical Output Target Planned Kassala Sea Nile Kordofan Total Component 1 - Expanding Access to Primary Health Care Services by Underserved Populations

Number of health staff Health 520 per benefited from incentive Professionals quarter 88 80 75 190 433 Number of health facilities Midwife Schools 4 1 1 1 1 4 benefited from medical supplies, incentive Health Centers 58 20 8 6 12 46 Basic Health Units 114 40 22 7 30 99 Number of insecticide treated malaria nets distributed Four states 179,100 32,200 30,500 26,900 89,500 179,100 Rural Hospitals 6 1 1 2 2 6

Number of health facilities Health Centers na 2 2 11 15 provided with equipment Basic Health Units na 1 1 14 16 Number of skill laboratories distributed Four states 4 1 1 1 1 4 Branches of National Health Academy completed Four states 4 0 1 1 1 3 Number of Ambulances Kassala and Red distributed Sea states 7 1 1 2 Support for Photovoltaic immunization solar South Kordofan 26 12 12 cold chain refrigerators with equipment Icepack (HF) freezers South Kordofan 17 17 17

Component 2 - Establishing the Basis for Reform and Development of the Decentralized Health System Establish mobile and temporary clinics Four states 11 1 1 4 1 7 Number of mobile cinema Four states 4 1 1 1 1 4 Number of health facilities Rural Hospitals 9 1 1 2 2 9 constructed or rehabilitated Midwifery (including North Kordofan in Schools 7 1 1 1 1 7 totals for rural hospitals and midwifery schools) Health Centers 10 1 2 2 5 10 Basic Health Units 8 2 1 0 5 8 Training Number of VMWs enrolled Four states na 307 203 126 409 1,045 Number of VMWs graduated Four states na 103 84 118 70 375 Number of midwifery kits distributed Four states na 1,419 Number of consumables kits distributed Four states na 32,400 Number of Health Professionals trained Four states na 3,095 48

Annex Table 2C: Consulting Services and Studies Financed by DHSDP a/

Task Title Consultant Cost (US$) Month/Year Completed Consulting Services Medical Waste Management Dr.A.Rahman Elamin 18,000 February 2008 Plan Providing Basic Package of GOAL, Sudan ( INGO) 130,000 April 2009 Health Services in Chali Locality (Blue Nile State) Health Planning and BMB Mott MacDonald 849,000 June 2010 Budgeting - Technical Assistance Package to Four States Development of Financial Final Solution 26,000 July 2012 Management Software for DHSDP Technical Assistance Support Save the Children, USA 130,000 Terminated in for Health Training in South (INGO) May 2012 Kordofan State because of civil conflict in South Kordofan Studies Health Care Financing Study WHO 423,000c/ June 2012 b/ Maternal Death Review Dr.Mohsin S.Khan 46,000 June 2013 Assessment Dr.Igbal Abukarig

Source: PIU in the Federal MoH. a/ A pharmaceutical study planned in the FPP was not implemented after it was decided to focus the project on maternal death and under-five death rates as part of the First Restructuring. b/ This study was a compilation of work explained in the text above. c/ WHO secured additional funds to complete the various components of this study.

The following additional studies related to the DHSDP were conducted under the MDTF- NS-TS Budget (Bank-administered budget). They contributed to the achievement of Sub-Component 2.1 of the project.

a. Funded by the MDTF-N TS Budget for TA/ESW • Study on the Impact of Government Decree on Free Health Care for Pregnant Women and Children Under 5. Conducted in 2010 with Bank TA, final report completed in April 2010. As recommended in the report, the Government decided to conduct a one-year pilot under the NHIF. Bank TA has been provided to design the pilot, which is expected to start soon.

• Health Financing and Service Delivery Study. Conducted in 2010 – data collection done together with the PREM-led PETS in the Health Sector – draft report completed in Oct 2011. Report was reviewed and a final version completed in May 2012. No follow-up due to lack of funding and proximity of closing date. b. Funded by the MDTF-N TS Budget for SPN of the DHSDP • Assessment of the Performance of Village Midwives – Blue Nile and Kassala States. A comparative study for which data were collected in 2011 and a final report completed in March 2012. • Social Assessment of the Potential of Women’s Groups at Locality Level in Supporting the Role and Work Done by Village Midwives. Conducted in the first half of 2011, report completed in June 2011. This assessment was complementary to the VMWs comparative study. Both studies helped to make a few adjustments to the implementation of DHSDP. They were also meant to guide the planned “community component” under the second Additional Financing, which ended up not being possible because there was not enough time to identify a competent National NGO or CBO which could be sub-contracted by the project for this specific purpose). • Brief Assessment of the First Level Referral System. Conducted in October 2012. Report completed at the end May 2012. It included a concrete proposal for a “first” level referral component” and estimated costs to be included in the Second Additional Financing. It was not implemented because the FMOH’s response was not strong enough. There was also limited time – only 6 months for implementation of the Second Additional Financing • Assessment of the Maternal Death Review System. Completed in June 2013. A task force established by the PHC Department to develop a work plan to implement the study’s recommendations

Annex Table 2 D: Summary of Civil Works Construction Achieved by States and Funding Source Amount State Scope of Work Status (SDG) Grant South Kordofan

Initial Financing Rehabilitation and new 5 Health centers (HCs): Tuleshie, construction Nagorban, Kabiela, Wali, Taimeien 5 Basic Health Units (BHUs): Hagar Completed 1,937,496 Hatab, Algania, Kajma Garbia, Jungarow, Abu Bitikh 1st Addtnl.Financing Rehabilitation and new Lagawa and Telody Rural Hospitals construction (RHs) Completed 1,447,684 Rehabilitation Kadugli Midwife Training School (MWTS) Blue Nile Initial Financing Rehabilitation and new 1 HC: Abu Garin construction Completed 334,179 1 BHU: Abu Zur 1st Additional. Rehabilitation Bados and Boot RHs Completed 938,000 Financing Rehabilitation Damazine MWTS 2nd Additional Financing Construction Surgical Theatre at Bados RH Completed 522,693 Kassala Initial Financing Rehabilitation and new 1 HC: Shalakai construction Completed 372,174 2 BHUs: Al Adergani, Algira

1st Additional Rehabilitation Telkok RH Financing Construction Kassala MWTS Completed 1,241,826 2nd Additional Construction Delivery Unit at Hameshkoreib RH Completed Financing 412,172 Red Sea Initial Financing Rehabilitation and 2 HCs: Saloum, Tomala construction Completed 405,983 1 BHU: Kamosawa 1st Additional Rehabilitation Tokker RH Completed 479,284 Financing Rehabilitation Port Sudan MWTS

2nd Additional Rehabilitation and Tagdoum Peri-Urban Hospital: Financing construction Rehabilitation of Delivery Unit Rehabilitation of Maternity Ward Completed 366,350 New Pediatric Ward North Kordofan 2nd Additional Rehabilitation and Rehabilitation of El Obeid MWTS 2,289,814 Financing construction Rehabilitation of Al Nuhud MWTS Completed 582,367 Rehabilitation at Umm Ruwaba 180,233 MWTS Total Grant-Funded 11,510,255

Government/Counterpart Funding South Kordofan Rehabilitation Al Fula Health Science Academy Completed 492,077 Construction Abu Gbiha MWTS Completed 221,908 Construction 8 BHUs: Labu, Um Durien, 6 out 8 BHUs 402,600 Umdehailieb, Dabakar, Libi Shreg, completed Karorak, Kary, Natal Blue Nile Rehabilitation Health Science Academy Completed 344,000 Construction Health Science Academy 65% completed Kassala 807,973 Red Sea Rehabilitation Health Science Academy 74% completed 158,000 Rehabilitation Port Sudan MWTS Completed 148,348

Total Government/CF-Funded 2,574,906

Grand Total (Government and Funded) 14,085,161

Annex Table 2E: Outcome Indicators by Localities in Blue Nile, Kassala and Red Sea States

Outpatient Consultations Antenatal Care Assisted Delivery State/Locality 2010 2011 2012 April 2010 2011 2012 April 2010 2011 2012 April 2013 2013 2013 (per person per year) (percent) (percent) Blue Nile -Bao 0.25 0.25 0.09 0.04 36 59 72 82 34 56 69 79 -Rosaires 0.52 0.61 0.69 0.64 73 91 99 104 50 85 96 102 -Tadamum 0.21 0.38 0.43 0.42 45 80 104 119 51 92 109 119

Kassala -Aroma 0.42 0.48 0.50 0.72 64 85 95 94 27 39 60 110 -Wargar 0.42 0.51 0.54 0.68 100 135 144 62 42 90 124 117 -Talkuk 0.14 0.22 0.28 0.35 8 16 39 47 5 11 34 55 -Hameshkorib 0.13 0.19 0.18 0.16 20 28 56 80 12 37 49 66 -Rural Kassala 0.50 0.23 0.43 0.67 271 148 123 129 65 70 79 97

Red Sea -Ageg 0.46 0.64 0.60 0.45 10 21 33 39 9 14 20 14 -Oleeb-Gonob 0.12 0.21 0.36 0.40 5 17 9 22 6 12 9 15 -Tokar 0.11 0.15 0.18 0.22 7 15 18 26 7 12 16 15 -Tokar City 0.21 0.43 0.52 0.68 13 42 55 78 15 36 58 45 -Rural Port Sudan na 0.02 0.08 0.45 na 89 71 44 na 23 26 26

Source: DHSDP M&E system

Annex Table 2F: Summary of Training Financed by DHSDP

Type of Training 2010 2011 2012 Cumulative May 2013 Health Personnel 58 57 28 143 Midwives - One-year course 47 165 0 212 - Two year course 0 131 65 196 - Refresher 358 372 519 1613a/ Midwife Supervisor 28 32 0 60 Specialized and On the Job Training - Health info/education, health promoters 1 73 0 74 - Maternal Death Review 85 60 193 338 - Safeguards 0 355 25 459b/ Total 577 1,245 830 3,095 Source: DHSDP M&E system. a/ Includes 364 midwife refresher training in May 2013. b/ Includes 79 in safeguards Training in May 2013

Annex Table 2G: Village Midwife Graduations Financed by DHSDP by States

State Number of Village Midwives Graduated Under Training 2010 2011 2012 2013 Total Blue Nile 47 36 35 118 Kassala 103 103 Red Sea 34 50 84 South Kordofan 30 40 70 40 North Kordofan 300 183 Total 47 137 66 125 675 223

Source: DHSDP M&E System

Annex Table 2H: Villages Covered by Trained Village Midwives by States and Localities

State/Locality Number Villages Covered by Trained Mid Wives Increase in of 2009 2010-2013 2013 Percentage Villages Number Percent Number of Number Percent Coverage Villages Added Blue Nile -Bao 193 26 13.5 16 42 21.8 8.3 -Rosaires 175 135 77.1 19 154 88.0 10.90 -Tadamun 29 20 69.0 8 28 96.6 27.6 -Demazine 114 98 86.0 11 109 95.6 9.6 -Geisan 110 65 59.1 24 89 80.9 21.8 -Kurmuk 71 36 50.7 6 42 59.2 8.5 Total 692 380 54.9 84 464 67.1 12.1 Kassala -Tarkuk 57 14 24.6 37 51 89.5 64.9 -Hamishkorieb 85 0 0 56 66 77.6 77.6 -Wagar 53 7 13.2 58 65 122.6 109.4 -Aroma 92 12 13.0 7 19 20.7 7.6 -Rural Kassala 106 19 17.9 12 31 29.2 11.3 Total 393 52 13.2 180 232 59.0 45.8 Red Sea -Gobob & Alieb 45 9 20.0 28 37 82.2 62.2 -Tokar 25 7 28.0 11 18 72.0 44.0 -Ageig 16 2 12.5 8 10 62.5 50.0 Total 86 18 20.9 47 65 75.6 54.7 Source: DHSDP M&E System

Annex Table 2J: Project Ratings Before and After First Restructuring Issue Criterion/Indicator Before After Restructuring Restructuring Rating Points Rating Points Relevance Objective Satisfactory 5 Satisfactory 5 Design Unsat. 2 Satisfactory 5 Implementation Mod. Unsat. 3 Satisfactory 5 Unweighted Average Mod. Sat. 3.3 Satisfactory 5.0 Efficacy Objective (a) Outpatient Consultations Mod. Unsat. 3 Satisfactory 5 Patient Ratio Not Ratable Patients with Fin. Barriers Not Ratable Skilled Birth attendance Not Ratable One ANC Visit Satisfactory 5 Skilled Health Staff at birth Satisfactory 5 Average Mod. Unsat 3 Satisfactory 5.0 Objective (b) Per Cap. Value of Support Mod. Sat. 4 Mod. Sat 4 Sustainable Health System Mod. Sat. 4 Satisfactory. 5 Unweighted Average Mod. Sat 4.0 Satisfactory 4.5 Objectives (a)&(b) Unweighted Average Mod. Sat 3.5 Satisfactory 4.8 Efficiency Objective (a) Management Efficiency Mod. Sat. 4 Satisfactory. 5 Implementation Efficiency Mod. Unsat 3 Satisfactory 5 Cost Effectiveness Unsat. 2 Satisfactory. 5 Unweighted Average Mod. Unsat 3.0 Satisfactory 5 Objective (b) Sustainable Health System Mod. Sat. 4 Mod.Sat. 4 Objectives (a&b) Unweighted Average Mod. Sat. 3.5 Satisfactory 4.5

Outcome a/ Unweighted Average Mod. Unsat Satisfactory a/ Overall outcome ratings are according to OPCS Guidelines

Annex Table 2K: Calculation of Weighted Average Outcome Rating for Split Evaluation Item Against Against Overall Comments Original Revised Indicators Indicators 1 Rating Mod.Unsat Satisfactory 2 Rating Value (points) 3 5 Table 2J 3 Weight 36% 64% 100% 4 Weighted Value (points)a/ 1.08 3.2 4.28 5 Final Rating Mod.Sat. a/ Percent disbursed before and after indicators were changed at first restructuring

ANNEX 3. ECONOMIC AND FINANCIAL ANALYSIS

The Final Project Proposal presented an overview of different approaches to an economic analysis of the project. It described effectiveness of a PHC approach, cost-effectiveness, cost-benefit analysis, positive externalities, and equity. With respect to financial analysis it described aggregate health sector expenditure requirements, per capita health expenditures requirements, federal government health expenditures, functional health expenditures, state government health expenditures, and international comparisons of expenditures on health. The material was a review of various measures for estimating the benefits and costs of improving primary health care.

The review resulted in no conclusions on the likely rate of return from or the economic efficiency of the DHSDP. It emphasized, however, that international experience with economic and financial analysis of the efficiency of investments in primary health care typically shows substantial positive returns.

This annex provides an analysis of the project’s efficiency using the available data. However it makes no pretense of being comprehensive. Rather it provides an assessment of managerial and implementation efficiency and an analysis of the cost-effectiveness of the project’s most important objective of improving reproductive health and improving access to primary health care in Blue Nile, Kassala and Red Sea states.

Managerial Efficiency

The DHSDP was under the leadership of the FMoH and the SMoHs supported by the Federal and State PIUs. The continuity of core personnel (mainly at State level) before and after restructuring was salutory as was fiduciary management. On balance, management efficiency had moderate shortcomings before restructuring and minor shortcomings after restructuring when it made good use of earlier technical assistance to improve institutional capacity. However the cost of project management was 13 percent of total costs which compares with the original estimate of 5 percent. The original estimate of management costs might have been unrealistically low in view of the legacy of conflict and there was indeed more conflict in Blue Nile and South Kordofan during implementation. While 13 percent is a high percentage compared with similar projects, it should be acknowledged that Sudan is a post conflict state where costs of project supervision are known to be relatively high.

Implementation Efficiency.

The Monitoring Agent noted that project implementation improved from its original weak phase (when implementation efficiency had significant shortcomings) to the final phase of development (when implementation efficiency had minor shortcomings).

After the first restructuring in 2009 the project focused mainly on pursuing the achievement of the project’s first objective.. All the planned health facilities were either renovated or constructed, and the total training objectives were achieved with heavy focus on the training of VMWs. Almost all the implementation targets for either the renovation or construction of primary health care facilities as well as targets for training facilities, with heavy focus on the training of village midwives (VMWs), were achieved. Outpatient consultations per person per year doubled. Reproductive health care (a basic health care service), as measured by the incidence of both antenatal care and births attended by skilled health staff, improved during project implementation.

Innovative approaches to improve efficiency involving the organization of midwifery training in localities close to target communities to overcome traditional constraints on young female students travelling unaccompanied to main towns contributed to a high training output and increased the supply of trained midwives in remote villages. The project effectively achieved its targets. This was a substantial achievement.

However, progress in South Kordofan was set back because of civil conflict, although it was compensated by the rehabilitation of three village midwife training centers in North Kordofan.

Cost-Effectiveness One approach to the assessment of cost effectiveness is to estimate the cost of death and disease averted. The cost averted is a benefit of a protocol or intervention that is used to avert a disease. An example of this is the use of “disability-adjusted life years” (DALYs) averted due to interventions financed by this project. After valuing the DALYs to obtain an estimate of the benefits from the project they can be compared with the cost of the project. This approach is used below for three of the four target states. South Kordofan was excluded because of the substantial interruption to implementation in that state due to the civil conflict.

DALYs are defined as the sum of the present value of future years of lifetime lost through premature mortality of patients and the present value of years of a future lifetime adjusted for the average severity (frequency and intensity) of any mental or physical disability inflicted on patients by a disease or injury.36 37

36 See JA Fox-Rushby and K. Hanson; “Calculating and presenting disability adjusted life years (DALYs) in cost-effectiveness analysis”; Health Policy and Planning; 16(3): 326-331

37 The concept of the DALY was first introduced in the World Development Report in 1993 in collaboration with the World Health Organization.

The equations for the future years of lifetime lost through premature mortality (YLL – i.e. equation 1) and years of future lifetime adjusted for mental or physical disability (YLD – i.e. equation 2) are shown below. 38

YLL[r,K,β] = { ( )( )[-(r+ )( + ) 1] ( ) [-(r+β)a-1]}+ ( )푟푎 퐾퐶푒 − 푟+훽 퐿+푎 − 푟+훽 푎 1−퐾 (1- ) ...... (1) 2 푟+훽 푎 훽 퐿 푎 − − 푒 푟 −푟퐿 Where푒 K = patient age weighting modulation factor; C = constant; r = discount rate for the value of life over time; a = age of patient at death; β = parameter from the age weighting function; L = standard expectation of life of a patient at age a.

YLD[r,K,β] =D{ { ( )( )[-(r+ )( + ) 1] ( ) [-(r+β)a-1]}+ ( )푟푎 퐾퐶푒 − 푟+훽 퐿+푎 − 푟+훽 푎 1−퐾 (1- )}.(2) 2 푟+훽 푎 훽 퐿 푎 − − 푒 푟 −푟퐿 All푒 symbols have the same meaning in the two equations except that in the second equation L = duration of disability and D = the disability weight.

Equation 1 generates the number of DALYs averted due to mortality at a specific age. For the DHSDP it is maternal mortality because activities this project focused on reducing maternal mortality by providing antenatal care and skilled birth attendants to pregnant women. Baselines and achievements resulting from these activities in three of the project’s four target states were provided in Table 4 of the main text. 39 Assumptions used in calculating the DALYs averted for these activities are provided below.

Antenatal Care (“at least one antenatal consultation”). The baseline of pregnant women in 2008 who had at least one antenatal consultation was 25,650. This represented 41 percent of the estimated number of pregnant women in the three target states. At the project’s close the number had increased to 44,771 which was 60 percent of the estimated number of pregnant women in the three states analyzed here. It is assumed that the increase in the number of pregnant women seeking antenatal consultations was attributable to the project. The expected maternal deaths among these women without the project would be the number of deaths averted because of the antenatal consultations financed by the project.

Maternal mortality rates have been difficult to measure in Sudan because the majority of deliveries take place in mothers’ homes and a significant proportion of deaths are not reported. The official average maternal mortality rate for all Sudan is about 0.7 percent, but the actual rate is almost certainly much higher in the poor rural localities targeted by

38 Murray, Christopher J.L. and Alan D. Lopez (ed); “The Global Burden of Disease”; Harvard University Press; August 1996

39 The three states are Blue Nile, Kassala and Red Sea. South Kordofan was excluded because of the negative impact of the hostilities in that state on project implementation results.

this project. It is therefore assumed for this analysis that antenatal care averted deaths among the increased number of pregnant women who sought antenatal care during the project was 1 percent.

Skilled Health Attendants (“births attended by skilled health staff”). The baseline of pregnant women in 2008 who had skilled attendants at delivery was 13,358 which represented 23 percent of the estimated number of women who delivered babies in the three target states. At the project’s close the number had increased to 33, 249 which was 46 percent of the estimated number of women who delivered babies in the three states. As argued above, it is assumed for this analysis that the presence of skilled birth attendants for pregnant women averted deaths among 1 percent of the increased number of pregnant women who received such support at delivery during the project.

Access to Basic Health Care (“outpatient consultations”). The baseline in 2008 was 353,598 consultations which was 23 percent of the population in the three states. When the project closed the number of annual consultations had increased to 739,365 which was 34 percent of the population. The increase of 385,467 consultations is assumed to have been attributable to the project. It is also assumed that most patients did not have serious diseases that caused continued disability. In other words they had diseases that were cured within a year with no significant subsequent disability effect. However, serious diseases are prevalent in Sudan and particularly in the target localities for this project. Malaria, for example, is a major problem with an incidence of about 28 percent in all Sudan40 and its incidence may be higher in rural areas of Blue Nile State where its debilitating effects could linger for many years. Nevertheless, malaria and other serious diseases have been successfully addressed in Sudan in the past and chronic disabilities have been averted with adequate access to primary health care. It is assumed that this performance would have continued without the project, but that 1 percent of consultations funded by DHSDP were serious with the potential for prolonged disability for patients for an average of 15 years. It is assumed that consultations at primary health care facilities funded by DHSDP averted these periods of disability. This is possibly a conservative assumption given the shortage of health professionals in rural area of Sudan.

On the basis of the analytical approach and assumptions outlined above for calculating YLL for antenatal care and skilled health staff assumptions for equation 1 were as follows:

K = 1 (implies that each year of life has the same value); C = 0.1658 (standard value for Global Disease Burden assessments by WHO); r = 0.03 (standard value for Global Disease Burden assessments by WHO); a = 32 (based on the average age of pregnancy being between 15 and 49); β = 0.04 (standard value for Global Disease Burden assessments by WHO); L = life expectancy for women in Sudan at age 32 (i.e. 33 years);

40 See Safa I. Abdalla, Elfatih Malik and Kamil M. Ali, “The Burden of Malaria in Sudan: Incidence, Mortality and Disability Adjusted Life Years”, Malaria Journal; 2007, 6:97

e = 2.72 (an approximation of the value of the natural base); and D = 1 (no disability factor).

For the purpose of calculating YLD for outpatient consultations all assumptions are the same as for YLL except that D = 0.2 (disability weight which reflects disabilities caused by ailments such as congestive heart failure, cataracts, drug use, low vision, malaria, dengue fever, diphtheria and measles).41 a = 25 (age of onset of disability); L = 15 (duration of disability in years).

Results of the calculation of DALYs for the three outcome indicators are presented in Annex Table 3-1 and the calculation of the present values of benefits and costs are shown in Annex Table 3-2

The overall conclusion on the basis of the assumptions set out above is that the present value of the disability life years averted valued at the per capita gross national income in Sudan in 2008 ($1,270) is estimated to be $30.7 million.42 The discounted value of all project costs is calculated at almost $14 million. In other words for each dollar invested in the project the benefits from improve reproductive health and increased access to primary health care alone generated two dollar’s worth of benefits in terms of the effects of maternal mortality and diseases averted. While many assumptions have been made in this analysis they have been on the conservative side and the result is therefore robust in terms of unexpected lower performance parameters. In addition the social benefits of reduced maternal mortality, less disabilities and illness in the population as a result of improved access to PHC services have not been assessed. However there can be no question that the net social benefits of reduced maternal mortality rates and improved access to PHC are positive and represent additional benefits above those estimated in this analysis.

41 See World Health Organization, “Global Burden of Disease 2004 Update: Disability Weights for Disease Conditions”, Table 1.

42 Assessing what Sudan’s per capita income would have been in 2008, without the benefits of the value of its share of oil exports originating from South Sudan is hazardous. As a proxy it is assumed to be equal to the current value of Sudan’s per capita income. The estimated per capita income for Sudan in 2008 (which included South Sudan) in current terms was close to $1,400.

Annex Table 3-1: Basis for Estimates of Disability Adjusted Life Years

Activity Baseline Final Difference Number of YLL Total YLD per Total Total Result Deaths or per YLL Disability YLD DALYs Disabilities Death DALYs Averted DALYs Averted Averted Averted Averted Averted (number) DALYs One ANC 21.207 44,771 23,564 235.6 25.30 5,961 5,961 Consultation Health Worker 11,895 33,249 21,354 213.5 25.3 5,402 5,402 Attending Outpatient 376,560 739,365 362,805 3,628.1 3.54 12,829 12,829 Consultations Total 24,192

Annex Table 3-2: Basis for Cost-Effectiveness Analysis

Dates for MDTF Government Total Funds Discounted Present Present Value ISRs Funds Funds Disbursed Project Costs Value of of DALYs Disbursed Disbursed (at 10% pa) Discounted (based on Costs GNI $1,270 per capita per year in 2008) a/ ($ million) 6/7/2008 1.81 0.91 2.72 2.72 2.72 4/24/2009 0.57 2.29 2.86 2.59974 5.31974 12/22/2009 2.41 2.41 2.09188 8.22972 6/29/2010 1.42 1.52 1.17292 8.58454 2/16/2011 1.15 1.15 1.0074 10.49624 9/10/2011 1.09 1.09 0.82404 11.31668 1/22/2012 1.57 1.57 1.13511 12.45179 9/9/2012 1.67 1.67 1.10554 13.55733 12/31/2012 0.29 0.29 0.18879 13.74612 6/24/2013 1.89 1.89 1.17369 14.01911 30.72

Totals 13.87 17.07 a/ Present value of DALYs averted is 24,192 (see Annex Table 3-1)

ANNEX 4. GRANT PREPARATION AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESSES

(a) Task Team members Responsibility/ Names Title Unit Specialty Lending/Grant Preparation Patrick M. Mullen Senior Health Specialist SASHN Original TTL Economist on Verdon S. Staines Senior Economist AFTSE appraisal mission

Supervision/ICR Sr. Financial Management Financial Nestor Coffi AFMNE Specialist Management Mohamed Yahia Ahmed Said Financial Financial Management Specialist MNAFM Abd El Karim Management Jamal Abdulla Abdulaziz Senior Procurement Specialist MNAPC Procurement Evarist F. Baimu Senior Counsel LEGAM Legal issues Consultant, Sr. Procurement Antonio J. Cittati LCSPP Procurement Specialist Consultant, Sr. Procurement Francesco Sarno MIGOP Procurement Specialist Peter D. Bachrach Consultant AFTHE Planning Richard Coppinger Consultant AFTHE Civil Engineer Yasser Aabdel-Aleem Awny Cluster Leader MNSHD Health Sector El-Gammal Abdelmonem Osman Kardash Consultant AFMSD Safeguards Limya Abdelaziz Consultant MNSHD M&E Tomo Morimoto Operations Officer AFTHW Selma Ahmed Siddig Team Assistant AFMSD Maisa Nurein Team Assistant AFMSD Aimnn Hassan Team Assistant AFMSD TTL after Patrick Isabel Cristina Soares Senior Operations Officer AFMSD Mullen Enas Mohammed Suleiman Team Assistant AFMSD

(b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending

Total: 0.00 Supervision/ICR

Total: 0.00

ANNEX 5. BENEFICIARY SURVEY RESULTS

None

ANNEX 6. STAKEHOLDER WORKSHOP REPORT AND RESULTS

None

ANNEX 7. GRANTEE'S IMPLEMENTATION COMPLETION AND RESULTS REPORT

Government of the Sudan

Decentralized Health System Development Project (DHSDP)

IMPLEMENTATION COMPLETION AND RESULTS REPORT

30 June 2013

ABBREVIATIONS AND ACRONYMS MDG Millennium Development Goals CPA Comprehensive Peace Agreement JAM Joint Assessment Mission PHC Primary Health Care NGO Non-Governmental Organization PDO Project Development Objective ITN Insecticidal Treated Net FMoH Federal Ministry of Health HRH Human Resource for Health PIP Project Implementation Plan SMoH State Ministry of Health HC Health Center BHU Basic Health Unit VMW Village Midwives FPIU Federal Project Implementation Unit SPIU States Implementation Unit SPLM Sudan People Liberation Movement GoNU Government of National Unity QAG Quality Assurance Group NHA National Health Account TA Technical Assistance HIS Health Information System RH Reproductive Health FO Financial Officer FMRs Financial Management Reports OPC Outpatient consultations ANC First antenatal visits AD Assisted delivery by skilled health staff LLIN Long Lasting Insecticidal Nets MDR Maternal death review PER Public Expenditure Review PETS Public Expenditure Tracking Survey HSS Health System Strengthening GAVI Global Alliance for Vaccines and Immunization GF Global Fund TBA Traditional Birth attendant FHU Family Health Unit FHC Family Health Centre

Table of Contents

1. Project overview Country Sudan Project title Decentralized Health System Development Project (DHSDP) Project Development To improve access to basic health services by conflict-affected Objective and underserved populations in target states while increasing the capacity of the decentralized health system to establish the basis for health sector reform, sustainable financing, and development. Sector Health Location Along with some activities at the level of the Federal Ministry of Health, the project targets the conflict-affected and poor states of Southern Kordofan, Blue Nile, Kassala, and Red Sea, covering a population of approximately 5.1 million. The target states include the Three Areas. Implementing Federal Ministry of Health (FMoH) and four State Ministries of Agency Health (SMoH) Implementing Three phases over a period of 4.5 years (54 months) .Project Period: closing date extended to June 2012 Brief Description Depending on different contexts within each target state, the project supported strategies, sometimes on a pilot basis, for improving access to basic health services by underserved populations, including injecting resources into existing underfinanced public sector health services, reducing financial and social barriers to care, creating mobile and temporary clinics, contracting for-profit and non-profit private organizations and firms to support service delivery, and providing high-impact health interventions directly to households. The project also started investing in key inputs to the PHC system in the four target states, developing state and local health administration capacity, human resources, infrastructure and equipment. At the same time, the project supported technical assistance to improve knowledge and allow the government to assess options for

addressing important systemic issues, particularly sustainable financing of PHC services under the decentralized system. Special attention was given to building capacities in monitoring and evaluation. Developing the decentralized health system in the target states on a sustainable basis was a long-term endeavor and the project supported the initial steps, through pilot experiences, technical assistance, studies, and capacity-building, in order to lay the groundwork for sector-wide reform. Finally project contributed to reduction of maternal mortality in four targeted states by supporting reproductive health strategies. Total Project Cost: Breakdown of project costs: Amount (US$ Million)

MDTF-N GoNU Total

Total project cost 23 47 70 Phase 1 initial budget as per 6 13 19 2006 Restructured Budget (phase 17 34 51 2 and 3)

2. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. Country context: The Government’s vision for the health sector in Sudan- described by the 25 year Strategy - is to fulfill the MDGs, contribute to poverty reduction, and improve equity across and within states, and among vulnerable groups. The signing of the Comprehensive Peace Agreement (CPA) in 2005 has provided unprecedented opportunities for development initiatives in the country and a Joint Assessment Mission (JAM) has determined the developmental priorities to be addressed by the country and its donors. A number of national action plans aimed at promoting the well-being of children and women in Sudan also existed. 2. In 2005, the context in Sudan was characterized by: (i) a decentralized structure but with weak institutional arrangements between Federal, State, and Locality levels; and (ii) a dilapidated and poorly funded health system in need of both policy and structural reform. The JAM had estimated that additional resources of approximately US$ 215 million would be required annually in 2006-08 to achieve significant progress towards the MDGs over all of Northern Sudan. The newly signed CPA and the potential for dramatic growth in Federal Government revenues (based on oil revenues) held the

prospect for reversing the deterioration of the PHC system and for financing its development and expansion, particularly in poor and conflict-affected areas. At the same time, with significant co-financing by the Government, the DHSDP project itself represented a mechanism for increasing public financing of basic health services in the target states 3. Results from the 2006 Sudan Household Survey indicated that progress toward achievement of the Millennium Development Goals (MDGs) was slow: (i) the maternal mortality ratio was estimated at 1,107 per 100,000 live births (638/100,000 in the Northern states); (ii) infant mortality was 81/1000 live births (71/1,000 in the Northern states); and (iii) the under-5 mortality rate was (102/1,000 in the Northern states). Furthermore, communicable diseases including vaccine preventable constituted a major burden of disease, and the health system, disrupted after years of conflict, was unable to provide essential primary health services to the population, especially those vulnerable. In the four target states of the project, there were large deficiencies in health services, as staff were poorly allocated, paid and motivated and often have insufficient technical knowledge and skills, while the supply of pharmaceuticals was uncertain, and basic equipment was lacking. In many areas of the target states, there was a basic lack of health facilities and services, with limited support from non-governmental organizations (NGOs). A social and gender assessment in the target states done as part of project preparation showed that even in situations where a supply of health services exists, there are major barriers to access by women, vulnerable groups and the poor in general. 4. The project paper also identified a number of systemic issues, of which the most important were the following: • Inadequate coverage and inequitably distributed primary health and first level referral maternal and neonatal services; • Inadequate availability of health human resource, their weak capacity and inequitable distribution between geographical regions; • Poor infrastructure and inadequate supplies and logistics (medical supplies and equipment, cold chain, transport and communication) for primary health care facilities network. • Low level of spending on health and inefficiency in utilization of available resources; • Weak governance and management systems, including the health information and health financing, especially at state and locality (district) levels. 1.2 Rationale for MDTF Involvement 5. As specified in the Comprehensive Peace Agreement (CPA), the Multi-Donor Trust Fund (MDTF) was expected to focus on recovery and development of conflict-affected regions, particularly the Three Areas (Blue Nile, Southern Kordofan/Nuba Mountains, and Abyei), as well as the least-developed regions in North Sudan. The proposed project consequently targets four conflict-affected and poor states, supporting improvement and expansion in basic health service delivery. 6. At the same time, the National MDTF can be considered as a mechanism to support policy reform in a context of increased dialogue between the Government and international partners, in particular with the intention of leveraging growing Government

fiscal resources towards pro-poor spending. In this context, the project was intended to improve knowledge and capacity to lay the groundwork for sustainable financing, reform and development of the decentralized health care system. 3. Original Project Development Objectives (PDO) and Key Indicators (as approved) 2.1 Project Development Objective (from Project Appraisal Document) 7. To improve access to basic health services by conflict-affected and underserved populations in four target states while establishing the basis for reform, sustainable financing, and development of the decentralized health system.

2.2 Revised Project Development Objective 8. The project development objective was not revised.

2.3 Main Beneficiaries and Benefits 9. The project was intended to provide support to the poor, underserved and conflict- affected states of Southern Kordofan, Blue Nile, Kassala, and Red Sea, which have a total population of approximately 5.1 million. These states include the Three Areas. Health indicators in these states, including service utilization and health outcomes, are consistently lower than overall Northern Sudan averages. Populations expected to benefit from the project included: 1) poor people living in underserved areas in the target states, 2) populations with little or no access to facility-based health services, with focus on vulnerable groups 3) Women and children were specifically targeted.

2.4 Key performance indicators (table 1)

Original key performance indicators Revised Key Indicators Component 1. Expanding access to Project Outcome Indicators primary health care services by underserved populations Outpatient consultations per person per outpatient consultations per person per year year in target areas; ratio between highest and lowest wealth quintile % patients who do not receive care due % pregnant women who attended at least one to financial barriers in target areas antenatal care consultation Skilled birth attendance in target areas births attended by skilled health staff (% of total births) (skilled health staff = doctor, nurse, midwife, trained village midwife) % households who possess at least one ITN Vitamin A coverage in target areas number of ITNs distributed

Number of pilot initiatives number of PHC workers (including midwives) implemented aimed at reducing trained financial and gender barriers to care Component 2. Establishing the basis Intermediate Outcome Indicators for reform and development of the

decentralized health system National Health Accounts completed National Health Accounts completed

% of PHC health facilities included in annual per capita USD value of the investment plan for target areas drugs/consumables and operational support rehabilitated/constructed and equipped provided to target health facilities & midwives (5) % of PHC worker training objectives in project plan achieved

* Baseline = 100. This relative measure is used until baseline estimates are available from the 2006 Sudan Family Health Survey and/or baseline surveys implemented by the project

2.5 Original Components 10. The project was designed to include two principal components and seven subcomponents Component 1. Expanding access to primary health care services by underserved populations 11. The objective of this Component is, in the immediate term, to improve access to primary health care services and high-impact health interventions by conflict-affected and underserved populations in the target states. Sub –components: • Improvement in the quality of existing primary health care services in underserved areas. In underserved areas with some functional government health services, the project supports injection of resources into the existing PHC system, improving service quality and allowing reductions in consultation fees. • Expansion of coverage of primary health care services and high-impact interventions in un-served areas. This Sub-component supports expansion of basic health services to improve coverage of conflict-affected and un-served areas. • Pilot experiences to reduce barriers to access to primary health care services. The objective of this Sub-component is to improve the knowledge and experience of the health authorities with possible strategies and interventions to reduce barriers to access to primary health care services, particularly by women, vulnerable groups and the poor in general.

Component 2. Establishing the basis for reform and development of the decentralized health system 12. The objective of this Component is to increase the capacity of the decentralized health system to establish the basis for sustainable financing, reform and development. This involves three Sub-components. Sub –components: • Capacity-building and policy development: the objective of this Sub-component is to lay the groundwork for reform and development of the decentralized health 71

system through technical assistance and studies on priority systemic issues as well as capacity-building in selected areas. This is to set the basis for reforms in health care financing, access to pharmaceuticals, health planning, budgeting and management by target State and Locality health administrations and monitoring and evaluation. • Development of primary health care human resources: this Sub-component provides support to implementation of the FMoH human resources for health (HRH) strategy in the four target states, establishing the basis for improvements in the production, quality, deployment and retaining of the PHC workforce. Of particular priority are medical technicians, nurses and midwives. The supported activities includes: a) development of a national human resources for health (HRH) strategy, b) development of state PHC human resource development strategies, c) review of curricula and in-service training of instructors, d) rationalization and investment in training schools and equipment. • Investment in primary health care infrastructure and equipment: this Sub- component upgrades and expands the PHC infrastructure in the four target states, focusing on the areas where the network of health facilities is weakest. • Project implementation: this Sub-component finances the personnel and resources necessary to manage the project and coordinate project activities at the federal and state levels.

2.6 Revised components: 13. The project was re-structured in 2009, but the project development objective, components and implementation arrangements remained unchanged. 4. Key Factors Affecting Implementation and Outcomes 3.1 Project preparation 14. Preparation of the final project paper started in April 2006 and the first thematic group meeting was held on May 2006. The project preparation involved analytical work, consultations in each of the states, and national level workshops. The preparation process was driven by the government's demand and the proposal was developed in consultation with sector partners; in particular the Health Thematic Group, led by the FMoH, and included the MDTF donors, UN agencies, the World Bank, and non-governmental sector partners. The decisions were incorporated into a Project Implementation Plan (PIP). The project was approved by the MDTF-NS Oversight Committee in October 2006; the Development Grant agreement was signed in 29th October 2006; and the project became effective in 24th January 2007, with a closing date of 31 December 2009. 15. The project design was based on a number of studies including the Sudan Household Health Survey, conducted on 2006 and provided baseline information for the project. A social and gender analysis was conducted on 2006 to examine and identify the needs of women and other vulnerable groups, in the context of each of the 4 target states, and provide community generated suggestions on how the Federal Ministry of Health (FMoH) and the State Ministry of Health (SMoH) can address these problems through

targeted health interventions, in the context of the MDTF Decentralized Health system Development Project. The project design was also built on the findings of the Country Economic Memorandum 2003 and the JAM mission report 2005.

3.2 Project design 16. Drawing on experiences from post conflict countries, the initial project sought to improve access to basic health services in the short term by: (i) providing mobile clinics and contracting out services to NGOs to bypass public sector capacity constraints; and (ii) emphasizing high-impact health interventions focused on households. The project also recognized that: (i) post-conflict situations can provide an opportunity to undertake reforms which might otherwise be difficult to undertake; and (ii) experience had demonstrated the need to work in parallel on system development and financing. The initial project design was an ambitious project with a dual objective attempting to balance a focus on improving services in conflict-affected areas in the short term with laying the ground for sustainable financing and development of the health system in the target states in the medium to long-term. The prospect of growing and significant government health spending in Sudan was seen as an opportunity to address financing and system development and reform issues at that point in time.

17. The total proposed project cost was US$ 70 million, of which the Government was expected to finance US$ 47 million and the MDTF to finance US$ 23 million. The cost for Phase 1 was estimated at US$ 19 million, of which the Government was expected to finance US$ 13 million and the MDTF US$ 6 million (Table 2). The project was anticipated to extend for 4.5 years. Table 2. Proposed project costs by source of financing (US$ million) MDTF Government Total Phase 1 6 13 19 Phase 2 8 16 24 Phase 3 9 18 27 Total 23 47 70

18. The project was restructured starting from January 2010 due to many issues arising during the first 2 years of implementation. The DHSDP has essentially came to involve two distinct project phases (i) from 2007-2009, the project represented a broad health system development agenda that aims at achieving short term objectives of expanding the health services to underserved communities with strategic objective of establishing a basis for reform and development of the decentralized health system. (ii) from 2010- 2013, the restructured project focused almost exclusively on supporting the implementation of the Road Map for Reproductive Health. During the initial project period, Sudan built on the National Reproductive Health Strategy (2006-10) and adopted the Road Map for Reducing Maternal and Newborn Mortality in Sudan (2009). The Road Map provides a comprehensive strategy for the period 2010-15 to reduce the current levels of maternal and neonatal mortality and morbidity in line with the MDG

health objectives. Based on these documents and a concern that the initial project was not sufficiently focused, a restructuring of the project began in end of 2009. Other changes accompanied this strategic change: (i) a significant works program at health facility level was stopped; (ii) the Government financing requirements were relaxed. 19. The restructuring and additional financing has sharpened the focus on improving basic maternal and child health services in rural areas of the four target states. Specifically, this involved: (i) supporting the government initiative to provide essential inputs for basic child health services free-of-charge in targeted health facilities; and (ii) intensifying investment in training, equipment and inputs to improve maternal and neonatal health care services in targeted health facilities and communities. The focus on results in terms of improved health services included implementation through public sector performance-based contracting strategies and support to improvements in monitoring and evaluation. Changes were made to project activities, geographic scope, financing, and key indicators and targets with focus on the following initiatives: 1. Free health care services. The Government's efforts to provide free health care for children under five and for mothers requiring caesarian sections was already a focus of the project and had demonstrated results (in terms of increased outpatient consultations). Activities included (i) the provision of drugs, laboratory supplies, and other consumables; and (ii) the payment of performance incentives for Health Center (HC) and Basic Health Unit (BHU) staff in remote areas. 2. Reduction of maternal and child mortality. The Road Map adopted in July 2008 provides a picture for the current situation that reveals unsatisfactory progress, and a detailed approach for the way forward. The focus was on two key areas: • support for the Village Midwives (VMW) in the form of training (basic and refresher), supplies and equipment (VMW kits and renewal, consumables, etc.), supervision (from the Locality through the Health Visitor and from the SMoH), and the payment of performance incentives; and • investments in strengthening the basic and comprehensive emergency obstetric capabilities of health facilities and the physicians who staff them. 20. The proposed additional financing was also designed to take advantage of the proven implementation capacity established at the federal and state levels, building on project investments to date in order to maximize development impact. 3.3 Project implementation 21. A Federal Project Implementation Unit (FPIU), States Implementation Units (SPIUs) and a unit at the former SPLM areas were established for the project implementation at federal and states levels. The SPIUs included one unit in each of the Red Sea, Kassala and Blue Nile states and two units in South Kordofan. Federal and State MoHs emphasized that the Project technical work was being done by the relevant departments and staff of the ministries. In that respect, the PIUs were acting as coordination rather than implementation units. A Project operational manual which includes M & E, Financial and procurement systems was operationalized.

22. Initial Deposit of funds by the government was satisfactory since the project was not ready to disburse large amounts of funds. With the growing capacity of the federal and state project management's ability to absorb the funds, potential achievement of overall project objectives was undermined by insufficient government funding which posed significant problems for both the amounts required and the timing. Unsteady flow of funds from the Government of National Unity (GoNU) negatively affected the implementation and effectiveness of the signed contract. By 2008, lack of funding has severely hampered planned activities. 23. In 2008, when Government co-financing was available, the project has operated effectively. The activities for which funding has been available (both grant and government), have been implemented satisfactorily. However, these activities constituted only parts of a broad and ambitious program that has not been fully funded. A QAG review in 2008 rated project performance as moderately unsatisfactory largely due to the fact that project design was too broad and ambitious in comparison to the activities that had received confirmed funding. By the end of 2008, the project was rated as moderately satisfactory due to successful completion of a number of procurement activities that have accelerated project implementation as well as receipt of significant government counterpart funding end-2008. The procurements included bed nets, drugs, medical supplies, TA and the NHA contract in addition to incentives to health facilities. In addition, the project was able to demonstrate significant increases in outpatient consultations by making drugs, consumables, and incentives available to the Health Centers and Rural Hospitals. 24. Several other problems, linked essentially to the context in the targeted states and the complexity of the project constrained project implementation. Challenges included: 1. The landscape for provision of basic services and the need to work in remote and often inaccessible areas, the shortage in health workers and the traditional and cultural barriers 2. Contracting NGOs to provide high impact services was only possible in a small scale and was constrained by the low capacity of national NGOs. 3. The major reform and capacity-building efforts proved to be more difficult than envisioned and required longer time, Slow policy development process, low FMoH and SMoH institutional capacity and slow implementation of the local health system plans and formation of locality teams posed additional constraints that slowed the health system reform component of the project. 4. Flare-up of the Conflict again in South Kordofan and Blue Nile states. 25. The project has contracted two international NGOs. The first was GOAL to provide basic health services for returnees in Kassala State. The contract continued for one year, until a fixed health facility was established in the area. Another contract was signed with Save the Children organization to provide basic training for nurses as part of the Academy of Health Science activities in south Kordofan. International NGOs proved to have the ability for quick delivery of the services in addition to contributing more resources for the project. Despite its limited scale, the experience of partnering with INGOs needs to be explored as an example of a successful Public Private Partnership

model that can inform development of service delivery modalities in underserved and conflict affected regions such as Darfur. 26. In November 2009, the MDTF-NS Oversight Committee approved an additional grant in the amount of USD 6 million for an 18 month period with the closing date of June 30, 2011. A work plan for the eighteen month period of the additional financing was prepared in January 2010. The DHSDP was extended to close on June 2013. The extension was intended to provide flexibility to the project in disbursing the remaining funds and in consolidating the project’s achievements, while continuing to address issues related to sustainability. 27. The DHSDP implementation units at state level remained stable throughout the 6 years of implementation. With very few exceptions (e.g. Kassala), the project officers and other staff remained the same since the beginning of the project, thus ensuring institutional memory and consolidation of capacity building. However, at federal level, with the exception of the Project Coordinator – which remains the same since the beginning of the project – a high turnover of staff has characterized key positions such as those of procurement officer, finance management officer and M&E officer. The concerns were losing institutional memory, the demands of additional training efforts and delay in implementation of activities. 28. In 2010-2012 the project was consistently showing ratings of moderately satisfactory to highly satisfactory. DHSDP’s average rating has been ‘Satisfactory’. The best rated parameters have been Project management and procurement administration, and the worst rated parameter has been provision of counterpart funds. 29. Implementation of the initiatives to strengthen the health sector was very slow, affected by FMOH institutional capacity. The initiatives included the NHA and capacity building on health planning, budgeting, management, M&E and human resource for health. Implementation was done through the WHO and FMOH planning and health economic directorates with Bank TA. The documents of the capacity building component had no clear road map for impregnation. The TA Component report was submitted to the project by the Consulting firm after a long delay. But the TA provided neither sufficient quality control for the documents produced nor a clear roadmap to sustain these efforts; as a result, the feasibility of implementing their approaches and tools on the ground remained unclear 30. Conflict in South Kordofan and Blue Nile erupted in the middle of the DHSDP implementation and resulted in suspension of activities in the two states. Many areas and targeted facilities became inaccessible to supervision. Targets were compensated by scaling implementation in stable areas/ other states and resuming project activities in the conflict affected states whenever security situation allowed. 3.4 Monitoring and evaluation 31. Baselines indicators were survey-measured indicators taken from the 2006 Sudan Household Health Survey. An M&E officer has been appointed by the FMoH and was producing regular and comprehensive reports. Results chain development and monitoring framework development were assisted by the bank in early 2008. A number of problems were encountered in carrying out that exercise: (i) some information expected to be available through the routine health information system at national level was not; and (ii)

target setting was difficult at State level. Reporting of baseline indicators that can be measured by the state MoHs' routine information systems was identified as an area to work on. In general, the M&E during the first 3 years lagged with a lack of specific results. 32. Starting from 2010, the quality of M&E information has improved because of a number of measures. Each State PIU was responsible for reporting on state-level project activities and on the state-level achievement of the intermediate outcomes. SPIUs in collaboration with the state Health Information System (HIS) and RH personnel collect disaggregated data and analyze it by locality. This information include: 1) essential information on the total population by locality, from which both the number of expected pregnant women and the number of households can be calculated. This data provided the denominators for the key project performance indicators; 2) detailed information in the targeted localities on the number of outpatient consultations, the number of pregnant women attending at least one ANC, and the number of births attended by skilled health staff. This data provided the numerators for the key project performance indicators. 33. The M&E function for DHSDP had always rated as satisfactory since 2008 to MDTF- N closure on 30 June 2013. Turnover of M&E officers posed a problem and has affected the quality of reporting. According to the monitoring agency; the State DHSDP offices were instrumental in collaborating with the SMoH to assure the completeness and quality of the data from the village midwives and health facilities, while the organization of regular meetings of SMoH staff, DHSDP personnel, and Bank’s Task Team provided a forum for effective technical support for M&E, maintaining consistent management of PDO data, and exchanging views among the project implementers.

3.5 Procurement, Disbursement, and Financial Management Financial management: 34. The primary responsibility for the management of project funds, including management of the two accounts, one receiving MDTF funds (Special Account) and the other receiving Government contributions (Project Account), rested with the Financial Officer (FO) of the Federal Project Implementation Unit (FPIU) under the overall supervision of the Project Coordinator. The Federal PIU generates Financial Management Reports (FMRs) and submits them to the MDTF, FMoH and MoFNE. Accounting software was made operational. As a result of in-availability of Government funding 2008-2009, there were delays in paying for delivered works and services. After restructuring of the project, funds were made available for pending payments to contractors. 35. According to the Monitoring Agency: the financial management rating was satisfactory in 12 quarters, moderately satisfactory in 9 and only one moderately unsatisfactory over the life of the project. The change in rating was mainly impacted by changes in key finance staff and finance managers. The project life cycle has seen four finance managers with some delays in replacement of finance managers. Financial reporting has been mostly timely and of good quality, except during transition which also affected timeliness of some audits. Fixed assets management was not prioritised as a key financial management issue until much later in the life of the project. As a result, the

fixed asset register maintained by the project does not capture all details that are considered necessary for a proper fixed asset register such as the cost of the assets, serial numbers of equipment and dates of acquisition. Procurement: 36. The project employed qualified Procurement Officers to carry out procurement functions. These Procurement Officers were part of the PIUs (at least one for each PIU). The ToRs of the PIUs’ Procurement Officers included training MoH staff by working with them. Upon establishment, the state PIUs were faced by lack of qualified candidates to work as procurement consultants and the projected has opted to select and then train its own personnel. It is worth noting that a large part of the important procurements were handled by the Federal PIU, with the exception of performance-based contracts with the VMWs, and RHs, as well as for recruitment of site engineers to supervise the civil works, which were directly handled by the State PIUs. By the end of 2010, procurement system has successfully complied with all the internal controls, record keeping and implementation procedures in the agreed provisions of the grant agreement. The project procurement system has benefited from: • The centralized procurement function with experienced personnel combined with early technical assistance and continuing W.Bank support • Close oversight of the civil works component including recruitment of local supervisors and periodic assistance (FMoH and W.Bank), • Identification of bottlenecks and prompt remedial actions, comprising regular missions and action plans, timely termination of failed contracts and rebidding, restructuring of the project to eliminate the co-funding requirement • Biweekly, monthly, quarterly, field, ad hoc monitoring and independent procurement review by MA 5. Assessment of Outcomes 37. A summary of the project achievements has shown that until June 30th, 82% of the indicators, 78% of the outcome indicators and 85% of the intermediate outcome indicators were fully achieved. 4.1 Outcome indicators 38. Since the approval of the second Additional Financing in December 2012, two PDO targets have been modified, namely the births attended by skilled health staff (from 30% to 55%) and no. of PHC workers trained (from 1,000 to 3,000). The following table presents the revised targets and the project’s results as of April 30, 2013:

Table 3: Status of the key performance indicators for project outcomes Baseline Targets* Progress Orig. Rev. Rev. Rev. Initial Additional Nov May Jun Dec Dec Dec Dec May Rural areas of the four target states (2006) (2008-09) 2006 2009 2011 2012 2010 2011 2012 2013 Outpatient consultations per person per year 0.20 0.16 0.30 0.30 0.30 0.30 0.26 0.31 0.32 0.41 % pregnant women attending at least one ANC 59% 48% 70% 70% 65% 65% 59% 59% 68% 68% Births attended by skilled health staff (% of total births)49% 19% 55% 55% 30% 55% 23% 42% 52% 63% % households who possess at least one ITN 18% 42% Cumulative no. of PHC workers (incl. midwives) trained0 40 200 200 1000 3000 577 1780 2425 3095 *The original targets (DGA 2006) cover the period 2007-09; the revised targets (2009) cover the period Jan 2010-June targets2011; the (2011) revised cover the period July 2011-Dec 2012; and the revised targets (2012) cover the period Jan-June 2013.

38. Additional details on the key outcome indicators are presented below: • Outpatient consultations (OPC): Monthly data from the Health Management Information System (HMIS) indicate considerable variability at locality level due to: (i) seasonality (particularly in Red Sea State); (ii) the effects of the security situation (particularly in Blue Nile State); and (iii) the availability of drugs and consumables provided by the project. • First antenatal visits (ANC): The rates of first ANC lagged early but have steadily (and sometimes sharply) increased over the past year. Since December 2011, first ANC has increased from 59% to 68%, due in part to increased coverage in Kassala and Red Sea States, resulting from the recruitment of Midwifery School graduates by the project (by contracting and incentives) and by the States (through the payment of minimal salaries). In addition, Kassala and Red Sea States have organized outreach activities, and Red Sea State added Rural Port Sudan Locality with its significant population. • Assisted delivery by skilled health staff (AD): There a continuous upward trend that is generally positive across the states. Table 4: State-Level Progress in Assisted Delivery (Before and After the Introduction of the VMW Initiative) Jan-Jun Jul-Dec Jan-Jun Jul-Dec Jan-Jun Jul-Dec Jan-Apr States 2010 2010 2011 2011 2012 2012 2013 Blue Nile 29% 61% 74% 81% 88% 90% 98% Kassala 17% 31% 29% 52% 63% 59% 79% Red Sea 5% 9% 10% 18% 16% 20% 19% Southern Kordofan 14% 21% 21% Total 19% 28% 36% 48% 53% 52% 63%

• Number of PHC workers trained. Data on the number of PHC workers trained (including midwives) show that this objective has been largely surpassed. 4.2 Intermediate outcome indicators 39. The project has made satisfactory progress on the five intermediate outcome indicators, which are presented in table 5:

Table 5: Status of the key performance indicators for intermediate project outcomes Baseline Targets Progress Orig. Rev. Rev. Rev. Initial Additional Nov May Jun Dec Dec Dec Dec May Rural areas of the four target states (2006) (2008-09) 2006 2009 2011 2012 2010 2011 2012 2013 Number of ITNs distributed 0 179,100 Annual per capita USD value of operational support to 0 0.17 0.40 0.40 0.40 0.48 0.47 0.41 0.36 targeted health facilities and midwives National Health Accounts completed No In progress Yes Done No.of training/PHC facilities rehabilitated & equipped No. of state health academies 0 4 4 No. of midwifery schools 0 4 7 4 7 No. of PHC facilities 0 18 18 18 18 18 No. of rural hospitals upgraded/equipped for EOC 0 6 6 9 0 4 6 9

• Distribution of Long Lasting Insecticidal Nets (LLIN): 179,100 LLINs were distributed during the initial financing (2007-09). • Per capita value of operational support: The per capita value of support to targeted health facilities and midwives over the period January 2010-March 2013 has averaged US$0.36 per year. • WHO’s final report on the National Health Accounts (NHA) study was submitted to the DHSDP in May 2011. Other studies to inform the development of the health financing policy are still ongoing with support from other development partners. 40. Civil works. Construction/Rehabilitation of 18 PHC facilities was completed during the first phase of the project. Construction/rehabilitation of 4 midwifery training schools and upgrading of operating theatres and maternities in 6 rural hospitals was completed during the 1st additional financing. For the 2nd additional financing, 3 additional midwifery schools (in North Kordofan State) and 3 rural hospitals (in Blue Nile, Kassala, and Red Sea States) have been or are being renovated; they are scheduled to be completed (with one minor exception) by June 30, 2013 41. Maternal death review (MDR) was established to audit maternal death at State and locality levels to identify exact causes of maternal death, and to measure effectiveness of interventions aimed at reduction of maternal deaths. The Project has supported the implementation in the 4 target states by paying operation cost for maternal death notification and investigation in addition to training of health cadres and community leaders on reporting and investigation procedures. Recently, the project has assisted the FMoH to assess the MDR system through a study carried out by an International and National consultants. The final MDR evaluation report was submitted on June 18, 2013 42. Though not an intermediate outcome, the Maternal Death Reviews carried out by the target states have become important indicators for the project, providing both explanatory factors for some of the results and potential impact indicators over time. Annex 2 presents the MDR results to date, while the table below summarizes the overall results:

Table 6: Number of Maternal Deaths Notified by Locality 2010 2011 2012 Apr 1 3 Maternal Deaths by Locality Total % Total % Total % Total % Total No. of maternal deaths expected per year 420 433 445 445 No. of maternal deaths notified 328 78% 251 58% 294 66% 103 23% No. reported from the community 105 32% 60 24% 90 31% 30 29% No. reported from the RH or FHC 223 68% 191 76% 204 69% 73 71% No. of reported maternal deaths investigated 323 98% 239 95% 263 89% 99 96%

43. Safe guard was introduced as a package of service in the targeted health facilities. As of May 2013, the project has: (i) trained 465 (or 139% of planned staff) on proper environmental health at facility level; (ii) provided protective equipment for health care workers/ health care facilities; and (iii) installed incinerators for 45 Basic Health Units/ Health Centers and 3 Rural Hospitals (or about 22% of all facilities assisted by the project).. A bank assessment for MDTF safeguard components has rated the project as satisfactory. 44. The National Health Accounts (NHA), developed by the FMoH with technical support from the WHO, is the country’s first effort and is expected to serve as an important tool for evidence-based decision making and planning. The report comprises: (i) the main text; (ii) the household health expenditure and utilization report; (iii) an in- depth study of national and state public sector health insurance; and (iv) the costing of targeted subsidies for PHC services. 4.3 Project outputs. Project outputs are summarized by year in Annex 2. 6. Other - unintended outcomes: 5.1 DHSDP Complementary activities: 45.Integrated within the MoH and implemented through its departments, the project has contributed to MoH capacity development through: • Attraction of qualified project staff and national and international experts who participated in development of local staff through training and transfer of knowledge. • Building the Projects and Development Departments at state and national levels in terms of physical infrastructure, oversight capacity and establishment of guidelines, systems, strategies and curricula. Capacity development included areas of procurement and financial management and civil work implementation. • The project has drawn the government attention to a number of important issues including environmental and waste management and maternal health issues. 46. Studies have been carried out with the main purpose of providing the FMoH with evidence for strategy and policy reform. Funding of the studies was mixed: DHSDP funding (NHA), combined DHSDP and MDTF-National Secretariat funding (Free Health Care, PER/PETS, and VMW study), and internal W.Bank financing (CSR). In response to Sudan government request, the bank has supported the following studies, complementary to the DHSDP: 1) the design, planning and implementation of the free care initiative in order to enhance its sustainability and impact.2) Technical Assistance to

support the strengthening of the Health Information System (HIS) with a view to improving results framework in the health sector, and to address the weakness of the existing Health Information System. The main focus of this TA was to develop a health sector-wide M&E framework based on the five-year M&E strategy and its state implementation plans developed earlier as a TA component of DHSDP. 3) Health Financing & Service Delivery (PER) and Public expenditure tracking survey (PETS) merged in one study. 4) Evaluation of the village midwife initiative to conduct a comparative study of various incentive schemes to providing the federal and state level ministries with detailed information and evidence on VMWs performance-based arrangements, as well as suggestions for subsequent policy and strategic decisions. 5) a Health Status Report (the previous one is dated August 2003), to present an overview of the Health Sector in Sudan, integrate the findings and recommendations of the studies above, and contribute to the next 5-year strategy. 5.2 Opinion of the DHSDP state level implementers about the project: 47. The project is seen by people involved in implementation as relevant because 1) it addresses health problems in areas affected by long standing conflict and have seen no or little health interventions in decades 2) it addresses priority issues and community needs based on reliable assessments and studies 3) the project is integrated within the ministry of health plans and implementation structures generating ownership and acceptance. 48. Issues facing implementation included: • Building the locality health system has proved to be a difficult intervention. Implementation was slow, suffering from lack of qualified personnel at locality level. As a result, the locality health system development has lagged behind, affecting overall project achievement and sustainability. • The project was faced by resistance from the local community in some areas, which required additional advocacy and trust building efforts from implementers. Successful involvement of the local partners including the community has contributed to better tailoring of the activities and cascading of the outputs with securing additional resources. • Reasons behind stability of SPIUs personnel throughout the project life cycle were identified as 1) the clear project objectives, plans and budget 2) feelings of contribution and making a difference to the local community 3) relatively good financial incentives and career development opportunities

• Other challenges included: shortages in human resources which affected operationality of rehabilitated /equipped health facilities and weakness of road and communication infrastructure.

7. Post-completion Operation/Next Phase 6.1 Institutional sustainability: 49. FPIU and SPIUs are integrated in MoH. This has contributed to alignment with other MoH activates, training of MoH staff and intuitional capacity building in terms of development of management systems, processes and software (examples include

procurement and financial management capacities ). State RH programmes represented key implementers which contributed to strengthening the RH management capacity at state level. Recruitment of qualified project personnel has contributed additional capacity building through consultation and transfer of knowledge.

50. The PIUs office spaces are MOH owned facilities. Other project assets will be dealt with according to the MDTF policy. Technical sustainability: 51. Many interventions have required preparatory capacity development and policy work that contributed to the MoH institutional development. Example is the development of TORS and guidelines for waste management to introduce incinerators, improvement of the integrated health information system and advocacy for RH interventions. Financial sustainability: 52. A sustainability framework was prepared where SPIU were requested to identify potential sources of funds to continue project activities. Currently, the Government has agreed to allocate 100 million SDG43 (or US$ 22.7 million at the current exchange rate of 4.4) annually over the period 2012-16 for a national project to develop universal coverage of the population with an essential package of primary healthcare services. The project is intended: (i) to expand coverage of a basic package of primary health care services to achieve the current national and international targets; and (ii) encourage other development partners to participate in the health sector using the DHSDP experience as a model.

53. The DHSDP was the first comprehensive health system development project in Sudan in decades. Building on its experience, the FMOH managed to get GAVI HSS and GF HSS Grants. GAVI-HSS began in 2008 to support the federal level and all 15 states on specific issues with a budget of US$ 16.1 million. The Global Fund HSS began in 2009 with a total estimated budget of approximately US$ 30 million over five years. Together, these three funds share the same development objectives and provide an opportunity for a coordinated and harmonized approach for: (i) strengthening institutional capacity through health sector reform and health systems development; and (ii) improving basic health service delivery. In addition, FMOH has sought to ensure synergy between the three projects in strengthening health administration at central, state, and locality levels and developing the capacities of those institutions with special roles in training (Academy of Health Sciences) or training and research (Public Health Institute) or other areas. FMOH is also scaling up the implementation skills gained under DHSDP in the areas of implementation planning, procurement, and financial management to implement the new funds.

43 Of which 20 million SDG would be for drugs and consumables to be made available through NMS.

8. Assessment of Risk to Development Outcome 54. The initial project design was ambitious. Project implementation was challenged by the weak MoH capacity and inadequate financial resources for the project. To combat this risk: (i) the project was substantially reduced after 3 years to focus on maternal health; 2) close supervision and quick implementation of corrective measures, and institutional capacity building approaches were adopted. Establishment of state project implementation units (and the stability of project staff since 2007) has contributed to ensure that project implementation risk remains low. 55. Political instability has been present in Sudan for quite a while. The recent separation of the South and the continuous conflict in Darfur and the border areas with the South remains a risk and poses security related problems. The project (which targets also South Kordofan and Blue Nile States) has adjusted to the insecurity situation, by finding and implementing alternatives in safe areas in the same states, as well as in other targeted states (namely the Eastern States of Kassala and Red Sea). Moreover, activities suspended due to insecurity resumed whenever possible (e.g. in Blue Nile State, most of the activities suspended in early Sept 2011 due to insecurity have already resumed). 56. Government commitment, the detailed financial sustainability framework prepared by the project- including a detailed maintenance and running costs related to the project inputs, and identifying potential funding sources (other than the project) will ensure that the project achieves its objectives and lays the groundwork for maintaining many (but not all) of the essential components of the project 9. Lessons Learned 57. Policy development is more effective when it is integrated into ongoing activities and when it combines evidence and recommendations with the capacities of the potential users. National level investments in policy development (Management capacity building, NHA) yielded only meager results, but MDTF-funded analysis in specific areas linked to project implementation (Free Health Care, VMW, MDR, public expenditure review, etc.) were more likely to advance the policy agenda. However, implementation of the results of this analysis was often constrained by limited institutional capacity to advance the reform agenda. 58. Managing Bank and Grantee expectations is essential for effective project design and implementation. Although the PDOs were appropriate and the project was justified in addressing service delivery and policy reform, the initial project, as designed was too ambitious and complex with respect to both the proposed activities and policies to be developed. Only when the focus was sharpened, state-level responsibilities made clear, and project activities designed to promote the existing reproductive health strategy and the road map for reducing maternal mortality did the project begin to make an impact. 59. Implementation of even a relatively simple project design turns out to be more complex than initially envisioned. Improvements in the delivery of services at the community level take place in a complex environment characterized by three types of issues: (i) community norms and values (e.g., the preference of women for TBAs and home deliveries, the difficulties of organizing and financing referrals, etc.); (ii) the individual capabilities and motivations of

service providers living and working in the communities; and (iii) health systems strengthening (e.g., physical and financial accessibility; financing; training and supervision; organization of supplies and consumables; and monitoring and evaluation of the impact on the women in the community. Though the project made progress on several of these issues, it could not address them all. 60. Comparable combinations of inputs may not have equivalent results. The combination of training, equipment, consumables, incentives, and supervision, which contributed to the VMW’s improved performance, was not nearly as successful at the FHU, FHC, and Rural Hospital levels. Given that maternal mortality cannot be solved at community level, the correct combination of inputs for strengthening the health system and improving health service delivery requires careful additional analysis and appropriate measures. 61. Flexibility was an important contributing factor to the project’s success. Even after the project was restructured, it continued to adapt to the evolving situation: (i) midwifery training schools were moved from the capital to the communities to overcome local resistance to sending females outside the communities; (ii) outreach activities were organized to compensate for the initially slow production of village midwives; (iii) support for the Maternal Death Review process; and (iv) waste management was recognized as a priority with appropriate measures financed in all project sites. In each case, the measures implemented by the project were rapidly scaled up nationally. 62. Easily measured indicators which are reported regularly are far more useful than more complex indicators available only periodically. Comparing the lack of data for monitoring the original grant with the monthly PDO results for the two additional financings, it’s clear that regular use of available data allowed for close supervision and supported the project’s flexibility. 63. Project sustainability has both institutional and financial components. The integration of project coordination within FMoH and SMoH was critical for project performance during and will be essential for institutional sustainability after project closing. Financial sustainability for continuing the project’s benefits will be almost impossible without a formal and transparent budgeting and disbursement system in the State Ministries of Health.

Annex 1. Project Costs and Financing Annual Receipts and Expenditures (in '000 SDG) by source and use of funds 2007 2008 2009 2010 2011 2012 2013 Total % Receipts 3 678 7 820 6 720 11 449 9 307 9 373 48 347 MDTF 1 746 3 281 5 628 5 226 7 019 7 490 30 390 63% GONU 1 932 4 539 1 092 6 223 2 288 1 883 17 957 37% Payments 2 665 6 130 7 266 8 878 9 297 7 264 41 500 86% Works 26% MDTF 1 946 952 2 388 1 932 7 218 GONU 157 1 056 33 1 649 662 194 3 751 Goods 34% MDTF 687 1 866 524 2 456 2 424 699 8 656 GONU 884 1 469 1 471 1 119 43 468 5 454 Services 25% MDTF 363 949 2 758 2 174 1 825 2 371 10 440 GONU 118 4 8 0 130 Operating 14% MDTF 18 4 9 299 1 738 1 290 3 358 GONU 556 786 407 225 209 310 2 493 Source: Annual audits

Annex 2. Outputs by Component For 2007-09, see the annual summary of expenditures, the annual audits for 2007- 08, and the project reports; for 2010-12, see the summary annex in the AM which will be available after the Thursday workshop Annex 3. Beneficiary Survey Results A beneficiary survey was not done, but assessments were done and summarized. They could be mentioned here. Annex 4. Stakeholder Workshop Report and Results Is this something to consider for the final mission in late-May/early June. Annex 5. Comments of Co-Financiers and Other Partners/Stakeholders Do we want to interview other MDTF partners about the effects of the project. Annex 9. List of Supporting Documents

ANNEX 8. COMMENTS OF COFINANCIERS AND OTHER PARTNERS/STAKEHOLDERS

None

ANNEX 9. LIST OF SUPPORTING DOCUMENTS

Bachrach Peter, Amel Aldehaib, Mahgoub El Nour and Mohamed El Tom, “Assessment of the Performance of Village Midwives – Blue Nile and Kassala States”, prepared for the Decentralized Health System Development Project, March 2012.

Central Bureau of Statistics, Household Health Services Utilization and Expenditure Survey (2006)

Daniel R. Gross and Abdelmoneim Kardash (Consultants); “Environmental and Social Safeguards Performance Review”; August, 2012

Eldihab Amel, A Report on Soicial Assessment of the Potential for Women’s Groups at Locality Levels in Supporting the Role and Work Done by Village Midwives”, 30 June, 2011

Federal Ministry of Health (2010), “Monitoring and Evaluation Framework”, Draft, prepared by Donald Whitson, June 23, 2010.

Federal Ministry of Health (2010), “Free Health Care for Under-Five Children and Pregnant Women in Northern Sudan: Progress so Far and Recommendations for the Future”, prepared by Technical Working Group led by Sophie Witter, September 2010.

Federal Ministry of Health (2013), Improving the Sudan Maternal Death Review System: A Situation Analysis, June 2013

Federal Ministry of Health, General Directorate of Health Planning (2011), Policy and Research Department of Health Economics, Research & Information; “Sudan Households Health Utilization & Expenditure, Survey in Northern States 2009”, April 2011

Federal Ministry of Health and World Health Organization (2012), “End of Project Report, Decentralized Health System Development Project (Health Financing)”, October 2012

Federal Ministry of Health and World Health Organization (2012), “A Review of the National Health Insurance Fund – Does Context, Scheme organization, and Health Infrastructure Matter?”, September 2012

GOAL (2009), “Delivering a Package of Health Services to Chali Area”, April 2008- April 2009, Final Report submitted to MoH Blue Nile State

Government of National Unity (MTDF-N), Federal Ministry of Health Decentralized Health System Development Project (DHSDP), “Baseline and Follow up Assessment of Health Services in Four Northern States-Sudan”, May 2010, prepared by Mustafa Khidir Mustafa Elnimiery.

Government of Sudan, Federal Ministry of Health, Primary Health Care General Directorate, Mother and Child Health Directorate, “Road Map for Reducing Maternal and Newborn Mortality in Sudan, 2010-1015”, December 2009

Kassam M. Kasak, Rania Hussein and Nahed Abdelgadeir, “Knowledge, Attitudes and Practices (KAP) in the Sudanese Communities for their Health Seeking Behavior”, September 2011.

Ministry of Welfare and Social Security, National Health Insurance Fund, “Summary Report on NHIF Performance2012”, December 2013

Modol, Xavier “A Brief Analysis of the FMOH Universal Coverage by Essential Package of Primary Health “, November 2102

Modol, Xavier “Care Universal Coverage by Essential Package of Primary Health – Project Proposal”, March 2013

Mott MacDonald, “Implementation Completion Report on Technical Assistance to Decentralized Health System Development Project, Sudan”; Volumes 1 and 2; June 2010

Multi-Donor Trust Fund -National Sudan (2011), “Assessment of Program Sustainability”, October 23, 2011 World Bank (2006) Report No. 37811-SD, “Final Project Proposal – Decentralized Health System Development Project”, October 26, 2006

Save the Children (USA), “Intensive Training of Medical Personnel at Hakima Health Training Institute”, Second Quarterly Report (May1-July 31, 2009.

World Bank (2009), “Memorandum and Recommendations of the MDTF-NS Technical Secretariat to the Oversight Committee on Proposed Restructuring and Additional Grant to the Sudan Government of National Unity for the Decentralized Health System Development Project (DHSDP)” - Grant No. TF057324; May 14, 2009 (First Restructuring including an additional grant of $6 million)

World Bank (2011), “Proposed Restructuring of Decentralized Health System Development Project” (MDTF-NS Grant No. TF057324), June 9, 2011 (Second Restructuring)

World Bank (2011), Public Expenditure Tracking Survey (PETS) for Northern Sudan – Case Study of the Health Sector”, July 2011

World Bank (2012), “Proposed Restructuring of Decentralized Health System Development Project” (MDTF-NS Grant No. TF057324), June 21, 2012 (Third Restructuring)

World Bank (2012), “Proposed Additional Grant in the Amount of US2.0 million to the Government of Sudan for the Decentralized Health System Development Project” (MDTF-NS Grant No. TF057324), December 14, 2012 (Fourth Restructuring)

World Bank (2013), Report No. 80051-SD, “Interim Strategy Note for the Republic of the Sudan”, August 30, 2013

To 25˚E25˚E Aswan Lake 35˚E To ARABARAB RREPUBLICEPUBLIC Hurghada Nasser SUDAN OFOF EGYPTEGYPT Lake Nubia SELECTED CITIES AND TOWNS WadiWadi HalfaHalfa WILAYAH (STATE) CAPITALS NubianN u b i a n Red NATIONAL CAPITAL LIBYALIBYA DesertD e s e r t S a h a r a D e s e r t RIVERS Sea MAIN ROADS 20˚N NORTHERNNORTHERN REDRED SEASEA RAILROADS KermaKerma PortPort SSudanudan

WILAYAH (STATE) BOUNDARIES DongolaDongola

INTERNATIONAL BOUNDARIES NiN i llee TokarTokar MeroweMerowe HaiyaHaiya El‘AtrunEl‘Atrun KaroraKarora This map was produced by the Map Design Unit of The World Bank. AtbaraAtbara The boundaries, colors, denominations and any other information shown on this map do not imply, on the part of The World Bank Ed Damer Group, any judgment on the legal status of any territory, or any RIVERRIVER EdEd DDameramer endorsement or acceptance of such boundaries. ar NILENILE ow A H t i b 0 200 Kilometers d a r a ile a W NileN

0 200 Miles ERITREAERITREA KASSALAKASSALA To KHARTOUMKHARTOUM Dese NORTHERNNORTHERN Kassala KassalaKassala DARFURDARFUR KHARTOUMKHARTOUM 15˚N 15˚N

ALAL JAZIRAHJAZIRAH W

NORTHERNNORTHERN W h CHADCHAD h Wad Medani

i t WadWad MMedaniedani To e Al Qadarif Abéche WESTERNWESTERN KORDOFANKORDOFAN N AlAl QQadarifadarif i l e Geneina DARFURDARFUR GeneinaGeneina Al Fasher Sinnar ALAL QADARIFQADARIF AlAl FasherFasher SinnarSinnar El Obeid ElEl OObeidbeid KostiKosti Rabak B ETHIOPIAETHToI OPIA lue RabakRabak Ni Dese EnEn NNahudahud le WHITEWHITE SINNARS I N N A R Nyala NILENILE NyalaNyala Ed Damazin EdEd DDamazinamazin AlAl FulaFula EdEd DDa‘eina‘ein SOUTHERNSOUTHERN KORDOFANKORDOFAN BLUEBLUE SOUTHERNSOUTHERN MugladMuglad Kadugli To KadugliKadugli Malakal NILENILE DARFURDARFUR To B Sarh a SUDAN h r 10˚N e l 10˚N 'Arab ABYEI AbyeiAbyei

To IBRD 33489R1 CENTRALCENTRAL Wau

JULY 2011 AFRICANAFRICAN SOUTHSOUTH SSUDANUDAN REPUBLICREPUBLIC 20˚E 25˚E25˚E 30˚E 35˚E