External Evaluation of Phase I of the Pbf Showcase Project in Borena
Total Page：16
File Type：pdf, Size：1020Kb
» EVALUATION REPORT MARCH 2019 EXTERNAL EVALUATION OF PHASE I OF THE PBF SHOWCASE PROJECT IN BORENA Workie Mitiku Eshete Yilma February 18, 2019 EXTERNAL EVALUATION OF PHASE 1 OF THE PBF SHOW CASE PROJECT IN BORENA ACKNOWLEDGemeNTS ACKNOWLEDGEMENTS The evaluation team would like to acknowledge several colleagues from Cordaid Head Office and Cordaid Addis Ababa Office for their inputs, support and guidance throughout the evaluation process. In particular, the team is grateful to Inge and Maarten from Cordaid head Office and Fikremariam Gezahegn from Cordaid Addis Ababa Office for their valuable comments on the evaluation design, tools, and report preparation, all of which improved the quality of this evaluation report. We also acknowledge Nurky Ibrahim and Gelgelo Hallke from Cordaid Yabelo Borena Office. Without their support, vital appointments with key informants at zonal, woreda, facility and community levels and data collection would not have been successful. Finally, we are very grateful to all the key informants and focus group discussion participants at the federal, regional, zonal, woreda, facility, and community levels for their open and honest views on the performance of the project. 2 MARCH 2019 © CORDAID CONTENTS EXTERNAL EVALUATION OF PHASE 1 OF THE PBF SHOW CASE PROJECT IN BORENA CONTENTS acknowledgments ........................................................................................................2 acronyms. 5 executive summary ...................................................................................................... 6 1. introduction ........................................................................................................... 8 2. purpose and scope of evaluation ................................................................................. 9 3. methodology .........................................................................................................10 4. findings ................................................................................................................ 12 4.1 Qualitative findings (Evaluation Question No.1) ........................................................... 12 4.2 Quantitative Findings (Evaluation Question No.2). 16 5. conclusions and lessons learnt ..................................................................................23 6. recommendations ....................................................................................................25 annexes Annex A: Figures on quality scores by Health Facility ..............................................................26 Annex B: Tor for an external evaluation of phase I PBF showcase Project in Borena Zone ...................28 Annex C : Questionnaire for the evaluation ..........................................................................32 Annex D: List of people consulted .....................................................................................37 Annex E : References .................................................................................................... 41 MARCH 2019 © CORDAID 3 EXTERNAL EVALUATION OF PHASE 1 OF THE PBF SHOW CASE PROJECT IN BORENA CONTENTS list of tables Table 1: Evaluation Design Matrix .....................................................................................10 Table 2: List of Woreda Health offices and Health Facilities Visited ...............................................11 list of figures Figure 1: Outpatient consultations including under-five children for all 8 health centers, 2008-2010 ...... 17 Figure 2: Consultations including under-five children by health center, 2008-2010 ........................... 17 Figure 3: First ANC visit within 16 weeks of Pregnancy for all 8 health Centers, 2008-2010 ................. 17 Figure 4: First ANC visit within 16 weeks of Pregnancy by health Center, 2008-2010. 17 Figure 5: ANC Visits (4 times) for all 8 health centers, 2008-2010 ................................................ 18 Figure 6: ANC visits (4 times) by Health center, 2008-2010. 18 Figure 7: Skilled Deliveries excluding Caesarean section for all 8 Health Centers, 2008-2010 ............... 18 Figure 8: Skilled Deliveries excluding Caesarean section by health center, 2008-2010 ....................... 18 Figure 9: Postnatal care visit (1) for all 8 health centers, 2008-2010 ............................................ 19 Figure 10: Postnatal care visit (1) by health center, 2008-2010 ................................................... 19 Figure 11: Beneficiaries of long term Family Planning Methods for all 8 Health Centers, 2008-2010 ....... 19 Figure 12: Beneficiaries of Long term Family Planning Methods by Health Center, 2008-2010 .............. 19 Figure 13: Cases of Sexually Transmitted Infections treated for all 8 Health Centers, 2008-2010. 20 Figure 14: Cases of Sexually Transmitted Infections treated by health center, 2008-2010 ...................20 4 MARCH 2019 © CORDAID ACRONymS EXTERNAL EVALUATION OF PHASE 1 OF THE PBF SHOW CASE PROJECT IN BORENA ACRONYMS ANC: Antenatal Care CBO: Community Based Organizations CBHI: Community Based Health Insurance Cordaid: Catholic Organisation for Relief and Development Aid DHIS: District Health Information System FMOH: Federal ministry of Health FDG: Focus Group Discussion FGB: Facility Governance Board HEP: Health Extension Program HMIS: Health Management Information System HSTP: Health Sector Transformation Plan ORHB: Oromia Regional Health Bureau PBF: Performance Based Financing WHOs: Woreda Health Offices ZHO: Zonal Health Office MARCH 2019 © CORDAID 5 EXTERNAL EVALUATION OF PHASE 1 OF THE PBF SHOW CASE PROJECT IN BORENA EXecUTIVE SUmmary EXECUTIVE SUMMARY Background equity through remoteness bonus. The evaluation team is Cordaid has been financing and implementing Phase I of the impressed with the extraordinary achievements of the Performance Based Financing (PBF) showcase project in project and found out that the PBF approach was effective Borena, Ethiopia, starting on May 2015 running through June and game changing. Because of the dependency of more 2018 in 9 health facilities (Yabelo hospital and 8 health centers) payment to increased results in health outcomes, the in the four woredas of the Borena zone. The main objectives of financial incentives to the staff, the autonomy given to the PBF showcase project were improving the quantity (cover- health mangers to make decisions on the use of the subsidy age), quality of health services and the health information of the project significantly improved the behaviors of the system through the implementation of a new approach, and health workers and managers motivating them to achieve introducing output/performance-based financing, shifting more health outcomes. away from the conventional input-based financing. The project has also created effective health information Evaluation Purpose system through the establishment of an independent body The general objectives of the external evaluation was to for the verification of reported data, with the consequences document lessons from the implementation of the showcase of health facilities losing money for reporting incorrect data, project and generate evidences / findings for the scale-up of the creating an accurate health information system in the program to other zones and regions in Ethiopia. Specifically, project area. the evaluation assessed: b. Extent of progress on quantity and quality of health 1. the consistency and effectiveness of the PBF approach/six services/Quantitative Findings: elements and 2. to what extent Phase I of the PBF showcase project has Relevance: The evaluation found out that the PBF project and improved the quantity, quality, equity of health services its approach is highly relevant to the implementation of three and health information system. transformation agenda of the health sector transformation plan (HSTP): Transformation in equity and quality of health care since it Evaluation Methodology works in hard to reach area and enhances quality, Information Revolution The external evaluation used a mixed methodology including: by putting in place checks and balance system through its a. Review of existing project documents; sample of the independent verification process ,Caring, respectful and compas- monthly quantitative reports, verification reports, quarterly sionate health workforce through the promotion of autonomy and quality assessment reports, sample of contracts, sample of financial incentive mechanisms. health facility business plan, the Project Implementation Manual, baseline data and internal review reports and other Effectiveness: The project uses 21 indicators for health centers relevant PBF documents; and 18 indicators for the hospital to measure the progress b. Key informant interview (KII) with Cordaid Addis and Yabelo in quantity of preventive and curative services over time. Office, FMOH, Borena Zone Health Office, Oromia Regional The evidences show that at the health center level, significant Health Bureau, Federal Ministry of Health, four Woreda progress has been made in quantity of health services in Health Offices, Yabelo Hospital, seven health centers and all of the eight health centers covered under the program. CBOs; For example, in some of the health centers such as Did - Yabelo c. Focus Group Discussions (FGDs) with direct beneficiaries of and Dikale, health service utilization in outpatient consulta- the target population; and tion including children under five has increased seven-fold d. Quantitative data analysis