• Abstract and Figures
• Public Full-text

EUROPEAN COMMISSION DELEGATION OF THE EUROPEAN COMMISSION IN NIGERIA CO-FINANCING WITH CATHOLIC AGENCY FOR OVERSEAS DEVELOPMENT (CAFOD), UK FOR NIGERIA

Improving Primary Health Care for Rural Poor Communities in Northern Nigeria

(Contract No. ONG-PVD/2006/119-131) FINAL EVALUATION REPORT

Conducted by: HIRAM Consulting Limited, PLOT 121 Golden Spring Estate, Duboyi District, Abuja Report written by: Adeniyi Olaleye, Olayinka Falola-Anoemuah, Patricia Suswam and Doris Ogbang

This project is implemented by This project is funded by Catholic Agency for Overseas Development The European Commission (CAFOD) & Catholic Secretariat of Nigeria

1st February 2007- 31st January 2011

1 | P a g e TABLE OF CONTENTS TABLE OF CONTENTS...... 1 List of tables...... 4 List of figures...... 4 List of Acronyms...... 5 Executive Summary...... 7 1.1 Introduction...... 10 1.2 Objectives and purpose of the final evaluation:...... 12 2. Methodology...... 13 3.0 Major Findings and Analysis of the Programme Outcomes...... 16 3.1 (Objective 1): To increase the capacity of 63 Catholic-Church run primary health providers to manage and deliver high quality, sustainable health care services in order to reduce the incidence of maternal and childhood illnesses, and other preventable diseases, including HIV/AIDS, malaria and waterborne diseases.16 3.1.1 Training of Staff...... 16 3.1.2 Provision of Essential Capital Items and PHC Equipment...... 20 3.1.3 Technical support to PHC staff...... 22 3.1.4 Provision of financial supports for running cost to selected facilities...... 24 3.1.5 Development of a gender focal points network...... 25 3.2 (Objective 2): To increase the level of community participation in, and ownership of primary healthcare in order to promote sustainable, healthy lifestyles...... 26 3.2.1 Community mobilization activities...... 27 3.2.2. Establishment of Village Health Committees (VHC)...... 29 3.2.3 Training of Village Health Workers and TBA...... 30 3.2.4 Publication and distribution of IEC materials...... 31 3.2.5 Participatory Health Assessment by Rural Communities...... 32 3.2.6 Awareness raising workshop on HIV & AIDS for religious leaders...... 32 3.3. (Objective 3): Improved practice in primary health care as a result of increased collaboration, sharing, coordination and learning between/amongst a diversified range of health actors and stakeholders...... 33 3.3.1 Production of newsletters and documentation on project learning...... 34 3.3.2 Exchange visits to other PHCs within the programme...... 34 3.3.3 Learning and sharing meetings...... 36 3.3.4 Collection, production and dissemination of up-to-date health information materials...... 37 3.3.5 On-going/Regular dialogue with other PHC and HIV practitioners and stakeholders...... 37

2 | P a g e 3.4.1 Creation of Health Information and Advocacy Unit in the Health Department of the CSN...... 39 3.4.2: Facilitation and support to the CSN Health Think Tank...... 40 3.4.3: Advocacy Training and Divulgation of CAFOD Advocacy Tool...... 40 3.4.4: Development and implementation of health advocacy strategies...... 41 3.4.5: Active Engagement with State Health Decision Makers and other Non-State Key Actors...... 41 4.0 Short term impact of the EC/CAFOD PHC programme...... 42 4.1 PHC’s capacity to provide sustainable quality primary healthcare service...... 43 4.2 Client Satisfaction...... 44 4.3 Changes in health behaviour and incidence of communicable diseases...... 45 5.0 STRENGTHS, KEY CHALLENGES AND RECOMMENDATIONS...... 47 5.1 STRENGTHS OF THE PROGRAMME...... 47 5.2 KEY CHALLENGES...... 47 5.3 RECOMMENDATIONS...... 48 APPENDICES...... 51 APPENDIX 1: PROGRAMME RESPONSES TO RECOMMENDATIONS...... 52 APPENDIX 2: EVALUATION TERMS OF REFERENCE...... 58 APPENDIX 3: ANALYSIS OF THE PHC CHECKLIST...... 64 APPENDIX 4: CLIENT SATISFACTION...... Error! Bookmark not defined.

3 | P a g e List of tables Table 1: Number of persons interviewed (planned versus actual)...... 14 Table 2: Trainings/workshops conducted...... 18 Table 3: Capital items and equipment planned and distributed...... 21 Table 4: Program accompaniers and advisor recruited with appointments and disengagement dates23 Table 5: Summary of Village Health Committees activities and involvement of women...... 30

List of figures Figure 1: Actual number of persons trained versus targets...... 17 Figure 2: Sustainability and dependence of Facility on EC CAFOD Funding...... 25 Figure 3: Trend in number of clients for ANC and Delivery services in 6 selected facilities*...... 44

4 | P a g e List of Acronyms

AIDS Acquired Immune Deficiency Syndrome ANC Ante-Natal Care ART Anti-Retroviral Therapy ARV Anti-Retro- Viral BCG Bacillus Calmette-Guerin CAFOD Catholic Agency for Overseas Development CHEW Community Health Extension Worker CSN Catholic Secretariat of Nigeria CSOs Civil Society Organizations CWC Child Welfare Clinic DPT Diphtheria, Pertusis and Tuberculosis DRACC Divine-Love Retreat And Conference Centre EC European Commission EC-ACP European Commission Africa, Caribbean and Pacific FCT Federal Capital Territory FGD Focus Group Discussion FGN Federal Government of Nigeria FMC Federal Medical Centre FMoH Federal Ministry of Health HBC Home-Based Care HIV Human Immuno-deficiency Virus HMIS Health Management Information System IEC Information, Education and Communication IMCI Integrated Management of Childhood Illnesses JCHEW Junior Community Health Extension Worker KII Key Informant Interview LGA Local Government Area MDG Millennium Development Goals M&E Monitoring and Evaluation MoH Ministry of Health MVS Multi-Vitamins Supplement NGO Non-Governmental Organization NPHCDA National Primary Health Care Development Agency NSA Non-State Actors PLHIV People Living With HIV/AIDS PRA Participatory Rapid Appraisal PPRHAA Peer and Participatory Rapid Health Appraisal for Action SACA State Action Committee for HIV/AIDS SOP Standard Operating Procedures SPSS Statistical Package for Social Scientists TB Tuberculosis TBA Traditional Birth Attendant TOR Terms of Reference 5 | P a g e TT Tetanus Toxoid UN United Nations UNICEF United Nations International Children Education Fund USD United State Dollar VHC Village Health Committee VHW Village Health Worker

6 | P a g e Executive Summary In response to the challenges of primary healthcare delivery system in Nigeria, the European Commission (EC) and Catholic Agency for Overseas Development (CAFOD) developed a programme to improve the quality of Primary Health Care (PHC) services. The programme titled ‘Improving Primary Health Care for Rural Poor Communities in Northern Nigeria’ (popularly called EC/CAFOD PHC Programme) spanned from 1 February 2007 to 31 January 2011 and was implemented in partnership with the Catholic Secretariat of Nigeria (CSN) and the 19 Catholic dioceses that make up the three northern Ecclesiastical Provinces of Abuja, Jos and Kaduna. In terms of civil administration, the programme covered 63 PHC facilities in 18 states and the Federal Capital Territory of Abuja.

In March, 2011, CAFOD/Nigeria commissioned HIRAM CONSULTING LIMITED to conduct a final evaluation of the EC/CAFOD PHC Programme. The main objective of this evaluation was to determine the extent to which the programme has achieved its stated objectives. The final evaluation, which was conducted between 7th of April and 30th of May, 2011, used a combination of qualitative and quantitative methods to elicit relevant information from different sources. Ten PHC facilities (Ankpa, Agagbe, Adikpo/Vandeikya HBC, Yakoko, Jada, Namu, Mandella, Fuka, Dongo Kurmi and Abuja HIV) were selected for field visit through stratified and systematic random sampling methods. In all, a total of 22 KII and 20 FGD were conducted. In addition, 53 clients exit interviews were conducted and the facility check list was administered in 9 out of the 10 selected facilities (excluding Abuja HIV).

The key findings of the evaluation are presented under each of the four programme objectives as follows:

Objective 1: To increase the capacity of 63 Catholic-Church run primary health providers to manage and deliver high quality, sustainable health care services in order to reduce the incidence of maternal and childhood illnesses, and other preventable diseases, including HIV/AIDS, malaria and waterborne diseases.

To enhance the capacity of the PHC staff, a total of 27 training/workshops were conducted through which a total of 809 participants were trained (79% achievement of the target). Generally, the training contents were appropriate and relevant to the objectives of the EC/CAFOD PHC Programme. About 97.6% of various items and equipment proposed to be distributed to the PHC facilities (830 out of 850 proposed) were given out by the end of the programme, based on needs. Similarly, a total of 369 out of the planned 810 visits (46%) were carried out by both Provincial and Technical Programme Accompaniers within the programme implementation period. Among the 20 “vulnerable” PHC facilities that were given financial support for running cost, the overall dependency on the EC/CAFOD grant reduced from 38% in 2007, to 36% in 2008 and to 30% in 2009.

7 | P a g e There were both oral (through interviews) and documented evidence to show that the capacity of the PHC facilities to deliver high quality PHC service has improved. PHC staff interviewed found the trainings useful and reported that the knowledge and skills acquired had helped them to deliver quality services to their clients. Noticeable improvement attributable to the trainings included: improvement in data collection, record keeping and reporting; improvement in quality of drug prescription/use and better obstetric practices, among others. The items and equipments provided also improved the quality and efficiency of services being provided in the facilities.

Objective 2: To increase the level of community participation in and ownership of primary healthcare in order to promote sustainable, healthy lifestyles

The data showed that 74 PHC staff, Health and HIV Coordinators were trained on Stepping Stones methodology (82% achievement) while 50 PHC staff were trained as trainers (83% of the target). Village Health Committees existed in 120 out of the 291 PHC stations with females constituting about a third (34%) of the membership. Out of the 10 facilities visited, VHC were established and effective in eight facilities. About 209 TBAs (66% of target) were trained on basic obstetric care, identification of pregnant women at risk and refer same, use of IEC materials and participatory methodologies. Similarly, a total of 81 participants attended the workshops on the Use of IEC and Participatory Methodologies, representing 77% of the target. A total of 4 rural health assessments were carried out as hands-on-training in Mandella, Zawan, Zambina and Zonkwa. On HIV awareness, 110 priests and religious attended the awareness workshops of which about half of the participants were priests.

PHC facilities conducted HIV/AIDS awareness campaign and mobilization which created positive impact through increased awareness and changes in behaviours of the community members and resulted in increased community participation and ownership. There were reports that PHC facilities produced IEC materials and translated some of the IEC into local languages. Thirty (30) local authorities became more aware of key health issues for rural communities, suggesting 100% achievement of the set objective.

Objective 3: Improved practice in primary health care as a result of increased collaboration, sharing, coordination and learning between/amongst a diversified range of health actors and stakeholders

Two drafts of newsletters were made but they were not finalised before the programme ended. Eleven (30.6%) of the proposed 36 PHC facilities participated in the exchange visit for learning and sharing with 65 staff as beneficiaries. Thirteen (13) meetings were held between programme/partner representatives and other health stakeholders between February 2007 and January 2011 out of 15 planned (87% of the target). Key health information materials including HMIS forms and standards treatment protocols were accessed from relevant government agencies, reproduced and distributed to the PHC facilities. The 10 PHC facilities visited reported that they had linkages and collaboration with either or both local and state government departments of health.

8 | P a g e The PHC staff commended the use of the exchange visit approach which exposed them to additional knowledge and skills and enhanced their primary healthcare practices. The adaptation and use of the FMoH data forms also enhanced the collaboration between government staff and the church health teams.

Objective 4 : To build the capacity of the Catholic Church on health policy and advocacy issues in order to engage with government and other key stakeholders in the health sector with the aim of influencing the development and monitoring of pro-poor health policies in Nigeria.

Activities conducted under this objective included identification and training of advocacy champions in the 3 provinces, meetings of the Health Think Tank in order to develop advocacy strategy for Catholic Church and joint campaigns of health stakeholders on health issues. However, other planned activities such as the creation of Health information and Advocacy Unit in the Health Department of the CSN, development and implementation of health advocacy strategies and active engagement with state health decision makers and other non-state key stakeholders could not be implemented due to various reasons.

Short Term outcomes of the EC/CAFOD PHC Programme All of the 9 facilities (100%) visited offered a range of basic primary healthcare services including: ANC, Immunization, Health education, Nutritional education, treatment of minor illnesses for adults and children, HIV counselling and HIV testing. Six of the facilities (66.7%) offered labour/delivery services while a third of the facilities (33.3%) provided postnatal services. Number of pregnant women receiving appropriate antenatal care and number of deliveries assisted by health professionals and trained birth attendants increased across the facilities. Majority of the PHC clients interviewed rated the facilities high on all of the criteria used. Similarly, majority of the community leaders, TBA, village health committee and PHC staff interviewed believed that the prevalence/incidence of most preventable diseases has decreased in the community over the past few years.

Strengths, Key Challenges and Recommendations The strengths of the EC/CAFOD PHC programme were identified to include the community -focus and bottom-up approaches adopted in the design and implementation of the programme that was built on the existing structure of the Catholic Church in Nigeria. The key challenges of the programme included its weak monitoring and evaluation system and inadequate allocation of funds for M&E activities; high staff turn-over; very wide coverage of the programme and non-existence (and usage) of SOPs and job aids at the PHC level. Recommendations for future programme include setting up of an effective M&E system at the programme start up, a review of staff recruitment process, a reduction in area of coverage and focus on two or three related PHC services areas and capacity building for CSN and health coordinators for monitoring and supervision of the programme.

9 | P a g e 1.1 Introduction In spite of her improved health status in the recent years, Nigeria still has one of the poorest health indices in the world. For example, the life expectancy at birth in the country as at 2006 1 was estimated at 46 years but it increased to 48.4 in 20102. Infant and under-five mortality rates were 99 in 2006 and 91.54 in 2010; and 191 in 2006 reduced to 138 in 2009 per 1000 live births3. These indices are worse than the average for sub-Saharan Africa. The major causes of morbidity and mortality are preventable and curable diseases. The World Health Organization4 ranked Nigeria’s overall health system performance 142nd out of 191 member states. The high level of mortality in Nigeria is a reflection of its weak and inefficient healthcare delivery system. Major challenges confronting healthcare delivery system in Nigeria include poor medical infrastructure and inadequate capacity of healthcare providers to manage and deliver high quality, sustainable health care services, especially at the primary healthcare level which constitutes about 93.6% of formal health facilities in Nigeria5.

In response to the challenges of primary healthcare delivery in Nigeria, the European Commission (EC) and Catholic Agency for Overseas Development (CAFOD) developed a programme to improve the quality of Primary Health Care (PHC) service delivery in Nigeria. The programme titled ‘Improving Primary Health Care for Rural Poor Communities in Northern Nigeria’ (popularly called EC/CAFOD PHC Programme) spanned from 1 February 2007 to 31 January 2011 and was implemented in partnership with the Catholic Secretariat of Nigeria (CSN) and the 19 Catholic dioceses that make up the three northern Ecclesiastical Provinces of Abuja, Jos and Kaduna. In terms of civil administration, the programme covered 63 PHC facilities in 18 states and the Federal Capital Territory of Abuja.

The EC/CAFOD PHC programme was developed in collaboration with Catholic Secretariat of Nigeria

1 UNAIDS/WHO. 2008. Epidemiological fact sheet on HIV and AIDS. Nigeria update. UNAIDS/WHO Working Group on Global HIV/AIDS and STI. Switzerland 2 UNDESA (2009d) Barro and Lee (2010), UNESCO Institute for Statistics (2010a), World bank(2010g) and IMF (2010a): Human development Report 2010 3 CIA World Factbook (20110.

4 WHO (2010) World Health Report. Geneva

5 National Primary Health care Development Agency (2007). Health Facilities Survey 10 | P a g e (CSN) in recognition of the efforts of the Catholic Church in Nigeria in primary healthcare delivery to the poor people, especially those living in rural areas of northern Nigeria. The programme was built on the premise of ensuring equal opportunities of access and improved quality of PHC services in rural areas where there is little or no coverage by the public services. Improved service delivery is critical to poverty alleviation which lies at the very heart of the EC programming and more broadly at the centre of EC-ACP cooperation. Records show that Nigeria, despite being one of the highest producers of petroleum in the world, remains a very poor country where 70.2% of the population live below the poverty line of 1 US dollar/day. The country still features on the list of the United Nations Least Developed Countries, ranking 142 in the UN Human Development Index (2010). Consequently, many Nigerians are yet to have access to adequate services that meet their basic needs in areas such as health, water and sanitation, and education.

The EC/CAFOD PHC programme was also in line with the Millennium Development Goals (MDGs). The nature of the project and its key aim to improve health status of rural poor people via the promotion of primary health was to contribute directly to the achievement of three MDGs, specifically: MDG 4 - Reduce child mortality MDG 5 - Improve maternal health MDG 6 - Combat HIV/AIDS, malaria and other diseases

The programme also adopted gender sensitive approach in PHC practices to ensure that it contributes to MDG 3 (Promote gender equality and empower women) and at the same time incorporated the building of civil society networks of Non-State Actors (NSAs) and promote closer collaboration between civil society and public authorities. The specific objectives of the programme were: 1. To increase the capacity of 63 Catholic-Church run primary health providers to manage and deliver high quality, sustainable health care services in order to reduce the incidence of maternal and childhood illnesses, and other preventable diseases, including HIV/AIDS, malaria and waterborne diseases. 2. To increase the level of community participation in and ownership of primary health care in order to promote sustainable, healthy lifestyles. 3. To facilitate, capture and share learning between Catholic Church, government, other private PHC providers and other stakeholders in order to strengthen health networks and promote good 11 | P a g e practice in primary health care. 4. To build the capacity of the Catholic Church on health policy and advocacy issues in order to engage with government and other key stakeholders in the health sector with the aim of influencing the development and monitoring of pro-poor health policies in Nigeria. The EC/CAFOD PHC programme used holistic approach to healthcare delivery. On the one hand, it made efforts to directly contribute to the improved provision and responsiveness of health services in low-income communities, and on the other hand, it carried out activities geared towards building advocacy and lobbying skills and initiatives for developing and monitoring policies that enhance equal access to health services. The expected outcomes as a result of the interventions of the programme included the following:  Strengthened capacity of 976 PHC staff in management and provision of a more comprehensive range of effective and appropriate primary health care services that meet the health needs of rural people living in northern Nigeria.  Community members benefit from healthier lifestyles as a result of behaviour change and taking greater ownership of their own primary healthcare.  Improve practice in primary health care as a result of increased collaboration, sharing, coordination and learning between/amongst a diversified range of health actors and stakeholders.  The health structures of the Catholic Church are more pro-active in engaging with the government, receive greater government recognition and support, and are able to contribute to the formulation, implementation and monitoring of health policies, which reflect the needs of the rural poor in Nigeria.

1.2 Objectives and purpose of the final evaluation: The main objective of this evaluation was to determine the extent to which the programme has achieved its stated objectives. The evaluation assessed the appropriateness and effectiveness of the design and implementation of the programme and how the programme has built the capacity of 63 Catholic Church-run PHC providers to manage and deliver high quality, sustainable healthcare services. See the Terms of Reference (TOR) for the evaluation in appendix 2. This evaluation will enable CAFOD and its partners to take stock of achievements of the project and provide a learning opportunity to improve future programming. The results of the evaluation will be useful as an advocacy

12 | P a g e tool by the Catholic Church to mobilize resources locally for the continuity of the programme activities and provide major inputs for designing of similar interventions in future.

2. Methodology The final evaluation for EC/CAFOD PHC programme was conducted between 7th of April and 30th of May, 2011. A combination of qualitative and quantitative methods was used to elicit relevant information from different sources. The qualitative methods included focus group discussions (FGD) and key informant interviews (KII) while quantitative method involved the use of semi-structured (exit) interviews and facility checklist. The sampling procedure adopted involved stratified and systematic random methods. To select the facilities to be visited for the evaluation, the 63 facilities were stratified by provinces (Abuja, Jos and Kaduna) and a systematic process was used to select 10 facilities. At the end of this process, the following facilities were selected: Ankpa, Agagbe, Adikpo, Yakoko, Jada, Nassarawa, Kura falls, Malumfashi, Abuja HIV and Mafo fadiya. However, due to logistic reasons and peculiar circumstances of some of the selected facilities, the following replacements were made: Adikpo PHC with Adikpo/Vandeikya HBC; Nassarawa with Namu; Kura falls with Mandella; Malumfashii with Fuka and Mafo fadiya with Dogon kurmi. The replacements were made purposively based on advice from CAFOD staff to ensure coverage of all programme components while the provincial spread of the sample was still maintained.

The range of persons interviewed with KII included CAFOD Programme Manager, CSN Director of Church and Society, CSN HIV Coordinator, Diocesan Health Coordinators, PHC Managers, Community leaders, Diocesan HIV Coordinator and FGD with group of PHC staff in 9 facilities, group of VHC in 5 facilities and group of TBA in 6 facilities. In all, a total of 22 KII and 20 FGD were conducted. In addition, 53 clients exit interviews were conducted and the facility check list was administered in 9 out of the 10 selected facilities (excluding Abuja HIV). Table 1 shows the breakdown of the interviews conducted.

Table 1: Number of persons interviewed (planned versus actual) Respondents Method Planned Conducted Remarks CAFOD staff KII 3 1 2 other key staff had disengaged CSN staff KII 2 2 The 2 persons interviewed were not the staff who implemented the program

13 | P a g e directly. Diocesan Health KII 10 7 Shendam HC doubled as Namu PHC Coordinators manager, Vandeikya and Agagbe have same HC while Jos HC was not available for interview PHC Managers KII 10 9 Abuja HIV has no PHC manager designate HIV Coordinator KII - 1 At Abuja HIV PHC Staff FGD 10 groups 9 groups PHC staff not available at Abuja HIV Community KII 10 4 Not available for interview at 6 sites Leaders (Abuja, Mandella, Jada, Namu & Dogon kurmi). Community leaders at Jada and Yakoko declined interview. TBA/VHW/VHC FGD 10 6 Not available for interview at 4 sites (Mandella, Abuja HIV, Fuka & Namu) PHC Clients Exit 90 53 Number able to interview among clients interview attending clinics on evaluation days.

In order to permit triangulation of information from multiple sources, collection of data involved three basic approaches including: desk review of project documents, direct interviews and observations during site visits. The evaluation process which lasted for about two months involved different stages including: a. Meeting with the CAFOD Programme Manager at CAFOD office in Jos on 07 April, 2011 for briefing on the project activities, explanations on TOR, agreement on evaluation methodology, sampling process and signing of Contract letter. The work plan and itinerary for the evaluation were also adopted during this meeting. See annex 2. b. Desk review of project documents and development of data collection tools: An extensive review of relevant projects’ documents was conducted to familiarize with the project implementation strategies, areas of focus, routine monitoring reports and findings of previous reviews. The list of documents reviewed included: the Programme proposal and log-frame, evaluation TOR, various reports (partners meeting, quarterly narrative, final narrative, workshop /training), 2009 midterm review report and the report of 2005 CAFOD PHC review. Based on the results of the review, a set of data collection tools was developed and shared with CAFOD staff for comments and inputs. The tools were later finalized after incorporating relevant comments/inputs from CAFOD staff. c. Site visit and data collection: The actual site visits and data collection exercise were conducted between 2nd and 13th of May 2011. Two evaluation teams were formed with each team

14 | P a g e (comprising a consultant and a field assistant) visiting 5 facilities. Activities conducted during site visit included KII and FGD interviews with different range of respondents as listed above, exit interviews with randomly selected clients who received treatment at the facility on the evaluation day using semi-structured questionnaire and assessment of staff, equipment and commodities currently available in the facilities. All KII and FGD were tape recorded. Each team moved from one facility to the other, ensuring that data collection was completed at a facility before moving to another. d. Data analysis and report writing: After the site visit, all data collected by the two teams were collated. Quantitative data (exit interview and facility checklist) were entered into computer and analyzed with SPSS and EXCEL respectively while the qualitative data (recorded KII & FGD interviews) were transcribed verbatim and analyzed thematically. The evaluation report was written in line with the format recommended by CAFOD in the TOR.

Limitation of the evaluation methodology The methodology adopted for this evaluation was influenced by the following considerations: 1. The evaluation used more of qualitative method because there was no structured baseline evaluation before the commencement of the EC/CAFOD programme. Hence, it was difficult to assess level of achievement of some targets in quantitative terms because there were no benchmarks on which comparison could be made. 2. The scope of the evaluation was limited in terms of sample size, range of persons interviewed and activities carried out at the sites because very limited budgetary provision was made for the evaluation. 3. Data collection at CSN and CAFOD offices was limited because most of the key staff that implemented the programme had disengaged before the evaluation. However, these limitations did not significantly affect the quality and findings of the evaluation.

3.0 Major Findings and Analysis of the Programme Outcomes The key findings of the evaluation are presented under each of the four programme objectives in order to assess the level of achievement of each objective and evaluate the relevance, effectiveness and impact of interventions carried out.

15 | P a g e 3.1 (Objective 1): To increase the capacity of 63 Catholic-Church run primary health providers to manage and deliver high quality, sustainable health care services in order to reduce the incidence of maternal and childhood illnesses, and other preventable diseases, including HIV/AIDS, malaria and waterborne diseases. The first objective of the EC/CAFOD PHC programme was pursued using four main approaches: (1) Training of staff (2) Provision of essential capital items & equipment (3) Provision of technical supports to the PHC staff (4) Provision of running cost to selected facilities. Each of these approaches is evaluated below:

3.1.1 Training of Staff The programme proposed to conduct a total of 31 trainings/workshops where a total of 1030 persons will be trained. The break-down of the training schedule showed that 63 PHC managers, 20 diocesan health coordinators and 10 HIV coordinators were expected to be trained on managerial issues while 378 PHC health staff were to be trained on PHC best practices. The 2011 draft final narrative report showed that a total of 27 training/workshops were conducted through which a total of 809 persons were trained (note that in many instances, same person attended more than one training) representing 79% achievement. Figure 1 shows the number of persons trained against what was planned while table 1 shows the breakdown of the types of trainings conducted and total number of persons trained.

16 | P a g e Figure 1: Actual number of persons trained versus targets

Generally, the training contents were appropriate and relevant to the objectives of the EC/CAFOD PHC Programme. Adequate number of days was allocated to each training and delivery/facilitation methods adopted were appropriate for the audience. However, in some cases, the training modules standards were higher than educational levels of some of the participants which affected the rate of understanding and ability to step down the training to other staff. This was because some PHC facilities in rural areas were unable to attract staff with required educational qualifications. For example, Fuka PHC Manager reported that she sometimes found it difficult to get staff with required educational qualifications to attend some trainings. This resulted in a situation where staff with less educational qualifications was sent to attend some trainings or a situation where same staff attending almost all of the trainings. It is important for the PHC and dioceses to develop strategies of attracting staff with required educational qualifications to make trainings more effective. During the interviews, all of the respondents including diocesan health coordinators, PHC staff and managers confirmed that they attended different types of training under the programme either directly or through a step-down. The most frequently mentioned among the trainings attended by the respondents were trainings on HMIS, rational drug use, programme and financial management, gender and health care provision and stepping stones methodology. 17 | P a g e The respondents found the trainings useful and reported that the knowledge and skills acquired had helped them to deliver quality services to their clients. The trainings had also improved the over-all ability of the PHC to deliver sustainable quality primary health care services in their communities. Some of the significant changes in practices of the PHC facilities attributable to the trainings were enumerated by the PHC staff to include the following: i. Improvement in data collection, record keeping and reporting: All of the 9 PHC facilities (excluding Abuja HIV) visited by the evaluation team had basic records (including registers for ANC, Delivery, Postnatal, Immunization, HIV testing etc) in place and were updated. Assessing the benefits from the HMIS training, a PHC staff said: “ When the project (EC/CAFOD Programme) started, there was no formal way of writing a report. We had challenge in terms of knowing what to report because there was no structured reporting format,...there was no data collection. After M&E (HMIS) training a tool was provided which has the indicators which guided us in writing our reports and helps to make it clearer and easier to put together” Another PHC staff reported “The training of the health management information and PPRHAA has helped me to be able to keep proper record and statistics of the patients here”. ii. Improvement in quality of drug use: All PHC managers interviewed confirmed attendance of training on rational drug use. The training has changed the practice of poly-pharmacy to accurate prescription based on diagnosis. Most PHC reported that they started using standard manual for prescription after the training. “There was training on rational drug use in Jos. Which help us a lot because we used to just prescribe many drugs to patients thinking we are raising money for the clinic. But we were made to understand that except in a critical condition, we are not supposed to prescribe two antibiotics at the same time for a patient”- A PHC manager

Table 2: Trainings/workshops conducted S/N Name of Type of participants No of Date training training/workshop participants conducted 1 Health management HC, PHC Managers, 86 June 18-21, 25-29; information systems HIV Coordinators July 2-5 2007 2 Health & HC, HIV 93 24 Sept- 2- Organizational Strategic Coordinators,& PHC November 2007 Planning Managers

18 | P a g e S/N Name of Type of participants No of Date training training/workshop participants conducted 3 Core Management Health Coordinator & 66 February 6 2008 Function Training PHC Managers 4 Basic financial HC, HIV Cord, PHC 81 April 1 - 17 2008 management Training Managers & Financial Officers 5 Stepping Stones HIV Coordinators & 48 June 19 -19; July 7 - Training of Trainers’ PHC staff 17 2008 workshops 6 Rational Drug Use and HC, PHC Coordinators, 84 Aug 11 -15 & Sept. Treatment Protocol PHC Managers & PHC 1 - 5 2008 Staff 7 Peer and Participatory HC, HIV & PHC 26 November 28 - Rapid Health Appraisal Coordinators, Provincial December 3 2008 and Action (PRHAA) Health Advisors & PHC Staff 8 Awareness Raising Both Male and female 110 January 14-15; 20-21 Workshops on Religious Leaders 2009 HIV/AIDS for Religious Leaders 9 Staff/Human resource Health Coordinators, 78 March 23-27; July 6- and team management PHC Managers and 10 2009 Provincial staff 10 Natural Family Nurses and women 70 June 22 - 26 2009 Planning & community leaders Reproductive Health 11 Financial Sustainability HC, HIV Coordinator, 61 April 20 - 21 2010 strategy training PHC Managers & Health Advisor 12 Training of Trainers Nurses, SCHEWs, 50 June 14 - 18 2010 workshop on TBA and JCHEWs and TBAs and community obstetric. VHWs 13 Gender and Health Care HC & PHC Health staff 37 July 19 - 23 2010 Provision 14 Community TBA VHWs and TBAs 195 July, Dec. 2010; Training January2011 15 Drug Revolving Fund in PHC Managers 29 December 7 - 10 Primary Health Care 2010 16 Stepping Stones HC, HIV Coordinators 74 October 22- 25; Nov. Awareness Workshop & PHC Managers 19 -29 2007 Source: Compiled from Annex VI Final Narrative Report iii. Better obstetric practices: PHC staff and TBAs reported that they now have improved knowledge and skills in conducting antenatal care and taking deliveries. The training of TBA

19 | P a g e has enhanced referral network between the TBA and PHC, thereby reducing maternal mortality due to complications during home deliveries. iv. Other changes in PHC practices associated with the trainings received as mentioned by the PHC staff and managers include: better project management style, effective pre and post HIV test counselling, better financial management, improved working relationship and better leadership styles, better drugs stock taking and use of essential drugs for prescription, cost effective drug procurement and prescription for clients and effective community mobilization to utilize PHC services. However, despite the numerous benefits and positive changes in PHC practices associated with the trainings/workshop conducted, the following limitations were observed: a. Due to low educational level, some PHC staff could not benefit maximally from the trainings. Some of them could not express themselves and their ability to assimilate training contents, implement the knowledge or step it down to other staff is in doubt. b. Many of the staff trained under the programme were no longer in the facilities due to high staff turn-over in search of greener pasture. Some of the new staff employed to replace those that resigned had not been trained. c. Despite reported implementation of rational drug use by all PHC managers, no standard protocol was sighted in consultation rooms of the PHC facilities visited. d. Although HMIS was in place in all facilities visited, one of the facilities (Fuka) was using hand ruled note books for essential records such as ANC, delivery, immunization, drug dispensing and laboratory records due to stock out of standard forms and registers while Abuja HIV was unable to produce their M&E forms and registers for sighting by the evaluation team.

3.1.2 Provision of Essential Capital Items and PHC Equipment Another capacity building approach adopted by the EC/CAFOD PHC programme was the provision of essential capital items and equipment to the PHCs. The 2011 final programme narrative report showed that a total of 850 of 10 different items and equipments were planned to be distributed to some selected facilities based on need. However, analysis in table 3 shows that a total of 830 items and equipment were given out, representing 97.6% achievement of the target. During the site visit, PHC managers and staff confirmed that various items and equipment were received from CAFOD and were sighted by the evaluators. In all cases, the items and equipments had

20 | P a g e been installed and were being used to deliver sustainable services to their clients. There were verbal evidences to show that the items and equipments had improved the quality and efficiency of services being provided in the facilities. Various instances of improvement in service delivery as due to these items and equipments were mentioned by the PHC managers and staff. For instance: i. Solar panel installed at Namu PHC has increased the level of patronage of the facility. One of the PHC staff reported that: “The solar energy has brought about a total change to the facility. Patients are happy with the fact that there is 24 hours light in the hospital. So, they prefer to come here”

Table 3: Capital items and equipment planned and distributed S/N Items and Equipments Pre Plan Revised Number Plan vs Implementation Prior to distributed actual (%) Plan Implementation 1 Vaccine Cold box 60 60 53 88 carrier 2 Microscope 30 30 30 100 3 Adult weighing scales 30 30 30 100 4 Manual infant suction 30 30 30 100 machine 5 Autoclave 30 30 25 83.3 6 Electric refrigerators 20 10 10 100 7 Solar Vaccine 20 15 9 60 refrigerators 8 Solar Lighting systems 10 10 8 80 9 Bicycles 300 300 300 100 10 Delivery and health 335 335 335 100 Kits TOTAL 865 850 830 97.6

ii. Provision of autoclave in Adikpo/Vandeikya HBC has improved sterilization procedure and reduced the possibility of infections to patients. iii. Provision of Microscope in Fuka has improved the quality of diagnosis and enhanced ability of the staff to attend to more patients. iv. Distribution of health (delivery) kits to Dogun/Kurmi PHC and TBA has promoted best delivery practices, minimised infections and reduced maternal and neonatal mortality.

21 | P a g e v. Vaccine cold box carriers distributed to Dogun/Kurmi PHC have improved vaccination activities as cold chain is adequately maintained and increased uptake of immunization. vi. Provision of bicycles has aided community outreach activities at Adikpo/Vandeikya HBC, Dogon/Kurmi and Mandella and has enabled the PHC staff to take health services to hard-to- reach communities. Few challenges reported in relation to the provision of capital items and equipments include: a. Late distribution of some equipment to the PHC. For instance, the last batch of bicycles was distributed in April 2011 while some dioceses had not collected their allocations from CAFOD office as at the time of this evaluation. b. Some PHC facilities (for example, Mandella) did not understand the criteria for selecting facilities that were given capital items/equipment and felt excluded. Early procurement of equipment is recommended to ensure optimal use during the life of the programme and criteria for selection of benefiting facilities should be clearly explained to all stakeholders.

3.1.3 Technical support to PHC staff The capacity building approach of the EC/CAFOD PHC programme also made provision for the recruitment of 3 Provincial programme accompaniers, 2 Technical Programme Accompaniers for HMIS and Finance, 1 Programme Administrator and 1 Health and Policy Advisor to provide technical supports to the PHC staff in specialized areas such as HMIS, finance, gender and advocacy. Desk review of project documents showed that three Provincial Accompaniers, two Technical Accompaniers, one Programme Administrator and one Health Policy & Advocacy Advisor were recruited in the course of the programme implementation. However, most of these staff disengaged their services at different times before the end of the programme. Table 4 shows dates of recruitment and disengagement of Program Accompaniers and Advisors. During the interview, the CAFOD Programme Manager attributed the high technical staff turn-over to inability of some of the staff to successfully go through probation period, personal reasons and securing new jobs towards the end of the contract with the staff. As at the time of this evaluation, only the Programme Manager was still on ground at the CAFOD office. Table 4: Program accompaniers and advisor recruited with appointments and disengagement dates S/N Position First Appointment Second Appointments

22 | P a g e Date of Date of Date of Date of Appointment Disengagement Appointment Disengagement 1 Provincial April, 2007 January 2011 - Accompanier (Abuja) 2 Provincial April, 2007 July 2009 January 2010 January 2011 Accompanier (Jos) 3 Provincial April, 2007 July 2009 - Accompanier (Kaduna) 4 Technical April, 2007 July 2007 May 2008 June 2010 Accompanier – Finance 5 Technical April, 2007 January 2008 May 2008 May 2009 Accompanier - HMIS 6 Programme April, 2007 November May 2008 January 2011 Administrator- 2007 CAFOD 7 Health Policy and April, 2007 October 2007 August 2008 December 2009 Advocacy Advisor- CSN

In terms of effectiveness and impact, this approach of capacity building appeared not successful. Most PHC managers complained of not benefiting enough from technical supports of the Accompaniers as they only had them in their facilities only a few times through-out the duration of the programme. Desk review of the end of project narrative report revealed that a total of 369 out of the planned 810 visits (46%) were carried out by both Provincial and Technical Programme Accompaniers within the programme implementation period. The reasons for the low performance of the Technical Accompaniers could be attributed to high staff turn-over. However, both the PHC staff and managers commended the Programme Manager for doing a good job in regular visiting and providing technical supports and guidance in programme implementation and management. Staff recruitment process and condition of service for technical staff should be reviewed for future programming in order to attract and retain qualified personnel.

3.1.4 Provision of financial supports for running cost to selected facilities Another capacity building approach adopted by the EC/CAFOD PHC programme was the provision of running cost to 20 selected “vulnerable PHC clinics”. The aim of this approach was to increase the financial capacity of the selected PHCs to continue to provide sustainable PHC services to

23 | P a g e the community by giving them financial support to meet the overhead cost including cost of local travel, drugs and consumables. Five out of the 20 PHC clinics (Abuja HIV, Agagbe, Ankpa, Yakoko and Namu) that received financial support were visited by the evaluation team. The PHC Managers of the five facilities confirmed that the financial support from EC/CAFOD Programme helped them to carry out various activities (such as outreaches and community mobilizations) and meet their overhead costs.

Performance indicators to measure success of the financial support approach stipulated that 75% of partner PHCs improved their financial management and CAFOD’s contribution to core costs of partners has reduced by 60%. The final narrative report revealed that the overall level of partners’ dependency on the EC/CAFOD grant reduced from 38% in 2007, to 36% in 2008 and to 30% in 2009. The analysis further showed that 13 vulnerable PHC clinics were able to reduce their grant dependency, two showed no difference in dependency while six showed increase in their dependence on the grant. Generally, average dependence on EC/CAFOD funds by facilities was less than 50%. There was a slight drop of dependence on EC/CAFOD from 49% to 44%, while facilities like Abuja HIV depended solely on EC/CAFOD from the beginning to the end of programme. Other PHC facilities like Namu dependency dropped to almost zero by the end of the programme.

Figure 2: Sustainability and dependence of Facility on EC CAFOD Funding

24 | P a g e 3.1.5 Development of a gender focal points network The programme planned to establish a gender focal point network at provincial level to work with a cluster of PHC providers from each diocese to encourage the discussion and implementation of models of best practice on gender and healthcare both inside the PHC facilities and in the outreach work of PHC staff. Gender focal points were expected to play a specific role regarding the identification, development and implementation of gender sensitive approaches in PHC practice which was to lead to the development of guidelines on models of best practices on gender and health care in the specific context within which partners operate.

An attempt to develop and implement gender sensitive approaches in PHC practice started with the training of 37 PHC staff on Gender and Healthcare Provision in the context of primary healthcare to raise awareness on gender issues. Realising the strong influence of men on nutritional intake of women, some facilities reported that they targeted males (especially husbands) in nutritional education for pregnant women as part of their outreach work. Some other facilities said they tried to be gender sensitive by separating female and male wards. Few others reported that they ensured that staff of same gender as the clients attend to some private medical issues.

However, there were no specific guidelines on models of best practices on gender and health care in any of the facilities visited and there was no record to show that gender focal points were established at provincial level. A PHC staff said: “There is no specific policy towards that issue but we are being encouraged to ensure gender balancing between male and female to equally reflect in that”. The level of gender sensitivity in primary healthcare delivery was low in the facilities visited. A PHC staff reported that “Both male and female nurses receive labour and delivery without any complaints from the patients”. There were reported cases of patients complaining about gender insensitivity in service delivery in some facilities. A male JCHEW recounted his experience: “.... like in ANC I remember some weeks back when this woman (referring to a female CHEW) was not around I was in charge of this place, some women were complaining that why should a man come and see...... I tried to educate them that either a man or a woman we both received the same training. That anything we discuss will just end here they will not hear it anywhere. But I don’t think there is anything set aside to cater for that problem”

The observed misunderstanding of PHC staff on gender issues and misapplication of the gender sensitive approach are attributable to non existence of the guidelines on models of best practices on gender and health care and weak follow up and technical support after the sensitization workshop. The development of 25 | P a g e the guidelines on models of best practices on gender is urgently required to help the PHC in application of gender sensitive approaches.

3.2 (Objective 2): To increase the level of community participation in, and ownership of primary healthcare in order to promote sustainable, healthy lifestyles

It was expected that community members would benefit from healthier lifestyles by taking greater ownership of their own primary healthcare and participate actively in its activities. To achieve the second objective of the EC/CAFOD PHC programme which focused on community participation and ownership, the following approaches were adopted:  Community mobilization activities  Establishment of village health committees  Training of village health workers and TBA  Publication and distribution of IEC materials  Participatory Health Assessment by Rural Communities  Awareness raising workshop on HIV & AIDS for religious leaders

3.2.1 Community mobilization activities

Community mobilization for health care services was concerned with encouraging people to feel committed to working together for the benefit of themselves and their neighbours. To achieve effective community mobilization, EC/CAFOD Programme conducted different trainings (including stepping stones, IEC and Participatory methodologies and awareness raising workshops on HIV/AIDS and Human Sexuality) targeting PHC staff, religious leaders and community members. This was to enhance their capacity in community mobilization and create awareness on HIV/AIDS and primary health care services (as reported in section 3.1.1). The trainings aimed at promoting healthier lifestyles, improving communication and relationships, and bringing about behaviour change among the target groups and the wider communities.

According to the EC/CAFOD programme log frame, 90 PHC staff, Diocesan Health Coordinators and Diocesan HIV Coordinators were expected to be trained in the Stepping Stones methodology, 60 PHC staff were to be trained as trainers of Stepping Stones methodology while at least 1200 community

26 | P a g e members from 75 communities would participate in Stepping Stones workshops. However, desk review of reports showed that 74 PHC staff, Health and HIV Coordinators were trained on Stepping Stones methodology (82% achievement) while 50 PHC staff were trained as trainers (83% of the target). The number of communities and community members that participated in Stepping Stones workshops could not be ascertained. The log frame also listed the expected outcome of this activity to include 40% increase in number of PLHIV support groups established in communities where Stepping Stones is being used and 30% increase in number of volunteers working with PLHIV communities where Stepping Stones is being used. The achievement of this outcome could not be assessed because there were no baseline figures on the indicators.

Apart from the trainings, PHC facilities embarked on HIV/AIDS awareness campaign and mobilization for other PHC services including childhood diseases, immunization for babies and vaccination for pregnant mothers using Stepping Stones methodology. A PHC staff summarized community mobilization activities as: ‘critical areas of positive health impact were nutrition, immunization of children against preventable diseases, malaria prevention through the use of mosquito nets, quality drinking water using water-guard, improved sanitation both personal (e.g. hand washing, bathing) and environmental (by keeping surrounding devoid of over-grown grasses and stagnant waters)”.

Continuous engagement with the communities on HIV/AIDS has created positive impact through increased awareness and changes in behaviours of the community members. For instance, the end of programme narrative report indicated that families within Vandeikya/Adikpo HBC catchment now provide care and support to members who are HIV positive and PLHIV no longer hide their HIV sero- status. Namu PHC also reported an increase in awareness of its local communities on HIV voluntary counselling and testing and on prevention of mother-to-child-transmission. This was evident by 39% increase in number of pregnant women tested (from 300 in 2008 to 764 in 2009). Increase in uptake of HIV testing by pregnant women also has implication for the HIV status outcome of their babies because those whose status was discovered to be positive will have access to preventive care/treatment and they are most likely to deliver HIV free babies.

The community mobilization efforts had also resulted in increased community participation and ownership. For example, Ankpa PHC reported that a community provided a piece of land and made financial contribution to construct a PHC outreach centre where the PHC staff and VHWs can provide 27 | P a g e primary healthcare services. Similarly, Bitako community under Yakoko PHC built a health post which is now being used as mother and child clinic as well as providing other primary health care services.

Community enlightenment on environmental sanitation was conducted to improve the hygiene practices of the community members and to reduce incidences of communicable diseases such as diarrhoea, malaria, cholera and skin infections. During the interviews with the PHC staff, they emphasized the successes they have recorded through sanitation campaigns to include improvement in environmental sanitation of the targeted communities, better hygiene behaviour and safer lifestyle of the people. Other achievements included community behavioural change such as increased utilization of PHC services (e.g. immunization uptake, antenatal clinic attendance), access to quality drinking water through the use of water-guard (courtesy of Catholic Relief Services), increased participation in Village Health Committee and reduction in harmful cultural practices. The knowledge acquired from the stepping stones training was very useful in community mobilization activities. Some comments during FGD and KII were: ‘ Community members now take the issue of sanitation serious. When we go back to some of these villages we see that the hygiene level has increased’, a PHC staff at Fuka.

‘With the use of Stepping Stone approaches the Benue programme communities have reduced the level of men cheating on their wives and wife inheritance is reduced due to improved knowledge of HIV’- Adikpo/Vandeikya HBC member.

3.2.2. Establishment of Village Health Committees (VHC)

It was planned that every PHC would establish at least one VHC to represent and mobilise the local community. Out of the 10 facilities visited, VHC had been established and were effective in eight (80%) PHC facilities while two facilities (Mandella and Abuja HIV) had no VHC in place as at the time of evaluation. The Mandella PHC Manager reported that establishing VHC was not possible because the recent ethno-religious crisis had resulted in suspicion among community members. Abuja HIV on the other hand established no VHC but worked directly with the community.

Interviews with the PHC staff and VHC members revealed that VHC, where they existed, had been effective in representing and mobilising the local community to create awareness for and uptake of the PHC services. Regular meetings were held between VHC and respective PHC staff to discuss community health issues such as outbreak of childhood infectious diseases (e.g. measles, diarrhoea), 28 | P a g e plan strategies for provision of specific health services like outreach immunization activities, decide on community outreach dates and information transmission to the communities. Other functions that were mentioned included the notification/reminder of the community members on dates of health post clinics; and participation in tracking clients who need a follow up in the facility.

The PHC Managers and staff reported during FGD and KII interviews that VHC had been very helpful in mobilizing community members to utilize healthcare services and provide feedback on clients’ satisfaction and complaints to the facilities. ‘The VHC members carry out some supportive activities such as give health talk, support in weighing children during CWC and also confirmed that the VHC provides information on outbreak of infectious diseases in the community’- Yakoko PHC manager reported.

One of the expected outcomes of this activity was that there would be an increase in participation of women in VHC and community health promotion initiatives. The desk review of reports showed that there was an increase of about 45% between 2007 and 2010 in the proportion of women participation in the VHC. The end of project narrative report further showed that VHC existed in 120 out of the 291 PHC stations (representing 41%) while females constituted about a third (34%) of the membership. The average yearly meeting of the VHC was about 6 times (once in two months).

Table 5: Women participation in Village Health Committees Province Number of Presence Composition of # of outstations of VHC VHC meeting Male Female held Abuja 135 47 212 96 446 Kaduna 94 42 115 68 98 Jos 62 31 103 60 108 Total 291 120 430 224 652

One major challenge encountered was the misunderstanding of the concept and functions of the VHC by some members. There were reported cases of VHC not being active because they were demanding that they should be paid whenever they hold meetings. VHC members should be adequately enlightened to know that participation in the committee is a voluntary service (that attracts no monetary gain) to engender community ownership of the programme. The concepts of voluntarism, dedication

29 | P a g e and community ownership should be emphasised as part of orientation when new members/committees are being inaugurated.

3.2.3 Training of Village Health Workers and TBA

Realizing the general acceptance and high patronage of TBAs in most rural communities in northern Nigeria, EC/CAFOD PHC programme mobilized and trained a pool of VHWs and TBAs to improve their skills on basic obstetric care, identification of pregnant women at risk and refer same, use of IEC materials and participatory methodologies conducted in the local languages. Thirty six (36) PHC staff and 14 TBAs were trained as trainers which later stepped the training down to 195 TBAs and VHWs across 16 PHC facilities. The number of TBA trained represents a 66% achievement of the 315 target.

The training improved the knowledge and skills of TBAs in conducting safe home deliveries using sterile procedure. The approach by which the PHC staff were trained to step it down to TBAs using local dialect facilitated better understanding. In appreciation of the training a TBA in Agagbe said, ‘the training has not only built our knowledge and skills but has given us confidence in our practice’ The improvement in the skills of the TBAs as a result of the training were listed by both the TBAs and PHC staff to include: improved knowledge on home delivery practices by the TBAs; identification of complications during pregnancy, labour and delivery for referral to the PHC; decrease in maternal and neonatal deaths; counselling of mother on antenatal services attendance; nutrition and appropriate dressing in pregnancy and personal and environmental hygiene.

In addition to the training, VHWs and TBAs were supported with relevant items and equipment such as health and delivery kits, IEC materials and bicycles to enhance their work in the community. The TBA, VHW and PHC staff expressed appreciation and satisfaction with the supply of the health and delivery kits during FGD and KII interviews. Most of the persons interviewed on the activities of TBA reported that the training of TBA had improved the quality of their service delivery. A VHC member in Agagbe reported that: ‘Before EC/CAFOD the TBAs were using the edge of a piece of unwashed bamboo stick to severe the cord from the baby (mother) and dress the stump with sand. However, this is now history’.

3.2.4 Publication and distribution of IEC materials

30 | P a g e The programme planned to train 105 persons on the Use of IEC and Participatory Methodologies and produce Health/Training Kits and IEC materials that would be appropriate to the needs of the target groups and translate the IEC materials into 4 local languages. The programme reports showed that a total of 81 participants attended the workshops on the Use of IEC and Participatory Methodologies, representing 77% of the target. Most PHC facilities visited during the evaluation exercise (Vandeikya/Adikpo HBC, Fuka, Namu, Ankpa and Yakoko) reported that they produced IEC materials and translated some of the IEC into local languages. Namu PHC was provided with technical and financial supports by the programme to produce IEC materials on HIV/AIDS which was used during World AIDS Day. The initiative and message came from the support group members during an exercise on assessing the quality of life of members and quality of HIV programme.

In 2010, 500 copies of Health/Training Kits and IEC materials for TBAs developed by the National Primary Health Care Development Agency were reprinted in English and distributed to 19 dioceses. Thirteen PHC facilities namely Ankpa, Gambar, Namu, Vandeikya, Agagbe, Mutum Biyu, Takum, Yakoko, Shuwa, Mafo Fadiya, Fuka and Malumfashi used the IEC materials to train VHWs and TBAs in the communities.

3.2.5 Participatory Health Assessment by Rural Communities

The programme planned to complete 6 participatory rural communities’ health assessments to inform the programme advocacy strategy and expected that at least 30 local authorities become more aware of key health issues for rural communities. This was aimed to generate models of best practices within PHC service provision and most importantly to identify key priorities to take forward in terms of advocacy with local and state governments.

For quality results, a decision was taken during partners meeting to adopt the Catholic Secretariat of Nigeria Peer Participatory Rural Health Assessment (PPRHAA) training methodology conducted for Lagos province. This assessment tool involved two stages: the training of Diocesan Health Coordinators from selected dioceses and second, the actual assessment of selected rural communities and PHCs by the trained health coordinators and their staff.

31 | P a g e The desk review of reports indicated that training for Diocesan Health Coordinators and PHC managers was conducted in 2010. During the training, 4 rural health assessments were carried out as hands-on- training in Mandella, Zawan, Zambina and Zonkwa. The assessment critical areas were: (i) potential barriers to access (ii) satisfaction with care (iii) drug availability, quality and information (iv) staff attitudes and behaviour; (v) environment and hygiene (vi) community participation in the affairs of the facilities using focus group discussion methodology with the communities. Forty-four persons (20 males and 24 females) participated in the assessment. The end of project narrative report further showed that 30 local authorities became more aware of key health issues for rural communities, suggesting 100% achievement of the set objective. The key achievement of the hands-on-experience on PPRHAA was that the participating communities were able to identify strengths and weaknesses of health facilities in their communities.

3.2.6 Awareness raising workshop on HIV & AIDS for religious leaders

In order to effectively provide the required support to the PLHIV and their families, the EC/CAFOD PHC programme adopted and conducted workshops for the priests and the religious on HIV and AIDS to improve their capacity in contributing to the fight against the pandemic. The workshops’ focus was on mobilising communities to create awareness on HIV and other health related issues. The expected outcome of the awareness workshop was that 60 % of religious leaders attending the HIV and AIDS awareness raising workshops become committed to fighting HIV and AIDS stigma in their communities.

Desk review of the programme reports indicated that a total of 110 priests and religious attended the awareness workshops of which about half of the participants were priests. Although there was no record of the actual number of religious leaders demonstrating commitment to HIV/AIDS fights, some of the priests who benefited from the workshop have reported demonstrable changes in their sermons by using correct and sensitive messages and providing counselling to church members and communities. The end of programme report quoted a priest from Kaduna Archdiocese who sent a text message saying; “I have come to understand myself much better and I am responding with compassion and empathy to those who come to tell me that they are HIV positive. Before I used to treat them with some level of judgement and sympathy which was not helping me to help them as a spiritual counsellor.”

32 | P a g e Though the support for HIV activities was limited to 3 facilities (Abuja HIV, Adikpo/Vandeikya HBC and Zawan), workshops and trainings related to HIV issues were extended to other facilities as well. It is important to note that the PHC staff through their dedication and commitment to support the PLHIV engaged the state and local governments through linkage and collaboration in the HIV activities. This led to the PHCs (for example, Anpka, Jada, Namu, Fuka) benefitting enormously by having access to HIV kits, cold boxes, mosquito nets and FGN forms for the returns of HIV test results.

3.3. (Objective 3): Improved practice in primary health care as a result of increased collaboration, sharing, coordination and learning between/amongst a diversified range of health actors and stakeholders

In order to effectively achieve objective three, the Programme Accompaniers conducted an initial assessment of the health information needs of the project during visits to the dioceses. The findings of the assessment provided a direction on areas of programme priorities. The approaches employed under this objective were:  Production of newsletters and documentation on project learning  Exchange visits to other PHCs within the programme.  Learning and sharing meetings  Production and sharing of best practices  Collaboration between Church run PHC and public PHC.

3.3.1 Production of newsletters and documentation on project learning

The programme was expected to publish 3 newsletters as written forum for exchange of ideas, knowledge and information. This was expected to be a continuation or follow-on of an already produced bi-annual Newsletter by CAFOD partners. Two editions of newsletters were planned for year 3, with a third edition for the extension period. However, programme annual reports indicated that 2 drafts were made in the second year of the programme, they were not finalised before the programme ended. Similarly, the 6 case studies on examples of best practice that were to be produced and widely shared could not be achieved.

33 | P a g e The challenge faced with the production of newsletters and case studies could be attributed to the high staff turn-over and the fact that the task was not assigned to any specific staff or sub-group but remained the responsibility of all the players in the programme. Such assignment should be assigned to a specific committee made of staff members with required skills and commitment.

3.3.2 Exchange visits to other PHCs within the programme

Exchange visits approach was aimed to promote sharing, learning and to give the opportunity to the programme supported staff to explore possibilities for further collaboration and networking among PHC providers in the health system. It was expected that at least 36 PHC providers would take part in exchange visits to other PHCs within the programme. For the PHC staff to achieve this, during the February 2008 partner’s learning and sharing meeting, partners identified a number of opportunities for exchange learning, which included community mobilisation and participation, TBA training and home- based care, HMIS, rational drug use and drug management .

The Programme Accompaniers also in year 3 introduced 11/2 days intra-diocesan sharing and learning meetings with the PHC in each diocese as a part of diocesan monitoring visit. These meetings assumed popularity hence the PHC staff appreciated the opportunity they had to practice in the real life context. Eleven (30.6%) of the proposed 36 PHC facilities participated in the exchange visit for learning and sharing with 65 staff and community members (VHC, VHWs and TBAs) as beneficiaries.

These best practices shared with the visiting teams included drugs and data management, community and maternal and child health activities at Maiduguri diocese; a good community mobilization methodology for health care activities and data collection at Yakoko PHC. A very laudable concept that arose from the exchange visits was the collective initiative applied by the matrons and health coordinators to carry out an objective assessment of Potiskum PHC. The outcome of the assessment was technical supports provided to Potiskum PHC on reactivating the parish health committee, encouraging community participation in health activities, quality data management and staff motivation.

34 | P a g e During the final evaluation process, Agagbe, Vandeikya, Fuka, Namu and Yakoko also confirmed their participation in the inter and intra-diocesan exchange sharing and learning exercise. Agagbe PHC visited Adikpo/Vandeikya HBC and Fuka; Vandeikya/Adikpo made exchange visit to Namu PHC and Fuka PHC. The PHC staff highly commended the use of the exchange approach as they said they were exposed to additional knowledge and skills which has enhanced their primary healthcare practice.

However, despite the opportunity provided by the exchange visits for PHC staff to share experiences and learn better ways of doing things, only 36% of the target was achieved. PHC staff in some facilities complained that they did not have opportunity of participating because they did not visit another facility.

3.3.3 Learning and sharing meetings The learning and sharing meetings approach was designed to provide the opportunity to identify, develop, share and promote models of best practices in the EC/CAFOD programme. This was aimed to focus on direct related health issues, health systems and management. Thirteen (13) meetings were held between programme/partner representatives and other health stakeholders between February 2007 and January 2011 out of 15 planned (87% of the target). The meetings included: 5 Programme coordination, learning and sharing meetings; 4 National Diocesan Health Coordinators learning and sharing meetings and 4 Provincial Health Coordinators learning meetings. Learning and sharing sessions were held during these meetings with the health coordinators and PHC managers. Some of the best practices shared during learning and sharing meetings were listed in programme reports to include:  The best use of data and plotting of disease trends and client progress each month on a wall chart by Fuka, Ankpa, and Shuwa PHCs.  The involvement of community health workers as community contact persons in villages for mobilization in Yakoko and Vandeikya PHCs. Ikot-Ekpene an ex-beneficiary also participated in sharing their garnered experience.  Training of community TBAs by Ikot-Ekpene and the resultant maternal mortality reduction. This huge success attracted the current PHC partners who requested for the training manual and indicated to visit the facility for interaction with the TBAs and trainers.  Water filtration method carried out by Kuru was also one of the best practices that was mentioned by all PHC staff during the final evaluation interview. However, the PHC staff and

35 | P a g e VHWs in Makurdi, Minna, Kaduna and Shendam dioceses used the water-guard treatment initiative to educate the clinic beneficiaries in purifying their water. This they said was free and easily accessible since it was being leveraged from Catholic Relief Services.  Linkages and collaboration with Local and State governments, and other private health facilities also proved to be best practices as these enhanced effective service delivery through easy access to immunizations, TB and leprosy treatment, access to effective 2-way referral and continuum of care in the communities and participation in trainings – shared by Mandella, Fuka, Agagbe, Ankpa, Yakoko, and Vandeikya and Adikpo/Vandeikya HBC. The advantages of this practice include cost effectiveness, continuous availability of services (e.g. immunization) and follow up on clients through a 2-way referral etc.

3.3.4 Collection, production and dissemination of up-to-date health information materials

During programme implementation, key health information materials including HMIS forms and standard treatment protocols were accessed from relevant government agencies, reproduced and distributed to the PHC facilities. The Programme also took advantage of a workshop organised by the Federal Ministry of Health in 2007 to access, reproduce and distribute the forms in 2008 to fast-track the process for the PHC staff rather than allow them wait endlessly at the LGA offices for collection.

Standard Treatment Protocols were also accessed and printed for distribution to PHC staff to enhance quality of service being provided to the clients. The protocols included: World Health Organization (WHO) Good Prescription Manual for health care workers and the IMCI flow chart for use in training PHC staff and display in outpatient department. A copy of syndromic management of Sexually Transmitted Infections (STIs) was also obtained but there was no permission for re- printing and distribution.

The adaptation and use of the FMoH data forms has enhanced the collaboration between government staff and the church health teams through the remittance of data and other information. The 2010 annual programme report noted that the use of FMoH health forms had improved the data reporting rate while the PHC mangers of Ankpa, Namu, Agagbe, Fuka and Yakoko reiterated during interviews that

36 | P a g e government health staff appreciated their prompt submission of accurate data which is accorded acknowledgement and adequate support through supplies of vaccines and TB and Leprosy drugs.

3.3.5 On-going/Regular dialogue with other PHC and HIV practitioners and stakeholders

The strategic location of the church-run PHC in the rural communities has further created avenues for more collaboration with other health actors in the community such as private and community owned health facilities. For record and reference purposes, one of the key indicators was to document such on- going/regular dialogue with other PHC and HIV practitioners and stakeholders. In order to achieve this objective CAFOD and partners have made series of visits to national and state government health ministries and other stakeholders in health care. The programme annual report of 2009 indicated that CAFOD made visits to FMoH in Abuja in June 2008, shared their training standards and requested for materials on IMCI. Other visits were made to UNICEF office at Abuja and Jos university teaching hospital’s gynaecological unit to seek and request for training materials for essential obstetric care and IMCI. These visits helped in establishing linkages and raised awareness of the EC/CAFOD programme among other key stakeholders. The approach also proved to be cost effective as some standard documents were received and others were reprinted with the permission as mentioned in section 3.3.4 above.

On-going dialogue was very apt because of the need to ensure coordinated health services for the clients within the catchment areas. Most facilities reported that collaboration with other PHCs in their communities was through a 2-way referral and joint training. For example, Namu PHC organized a step down training of rational drug use to staff of other PHCs in the community. Vandeikya, Agagbe, Namu, and Fuka PHCs reported that they trained staff of other private PHCs on HIV counselling and testing and also provided them with test kits. ‘ We also collaborate with private hospitals and clinics ...... where we link through a 2-way referral process for HIV pre/post counselling and we refer to them for ARVs and other critical ailments’- A Vandeikya PHC staff. ‘Other clinics refer patients here for HIV counselling and testing, and the HIV positive patients for Anti-retroviral drugs. Sometimes, we refer cases that are beyond our control to other government hospitals’- A PHC staff at Fuka

37 | P a g e It was confirmed during the final evaluation interviews with PHC staff and managers that all of the facilities visited had linkages and collaboration with either or both Local and State Government Departments of Health. These collaborations had led to some PHCs facilities being designated as vaccine collection points or TB (Fuka and Yakoko) and leprosy (Jada) treatment centres. Some PHC facilities (Fuka, Mandella and Ankpa) also benefited from such collaboration by receiving capital items such as solar vaccine refrigerators, vaccine carriers and bicycles from their local Department of Health.

PHC facilities also maintained collaboration with their counterparts in government and private health sector through attendance of meetings, sharing of information/documentation and promotion of joint initiatives at the local, state and national levels. For example, Kaduna province organized a Health Week in November 2008 focusing on malaria and other health management issues. The objective was to enhance learning and sharing among PHCs. In attendance were 35 PHC providers from 7 dioceses of the province. During the FGD and interview with the PHC providers, all the 10 sites expressed satisfaction with the level of collaboration and linkages with state and local governments and other private health stakeholders. The on-going dialogue clearly exhibits evidences of a healthy relationship. It is cost effective (PHC received various items and knowledge at no costs) and has potentials for sustainability and so should be continued even after the EC/CAFOD support. 3.4: (Objective 4) : To build the capacity of the Catholic Church on health policy and advocacy issues in order to engage with government and other key stakeholders in the health sector with the aim of influencing the development and monitoring of pro-poor health policies in Nigeria.

The fourth objective of the EC/CAFOD PHC programme was to build the capacity of the Catholic Church in health policy and advocacy issues to influence the development and monitoring of pro-poor health policies. The advocacy component of the programme was developed to meet the challenges and gaps identified by the need assessment conducted on the primary health care delivery of the Catholic Church prior to the commencement of the programme. The planned activities to achieve this objective were:  Creation of Health Information and Advocacy Unit in the Health Department of the Catholic secretariat of Nigeria  Facilitation and support to the CSN Health Think Tank  Advocacy training and divulgation of CAFOD advocacy tool

38 | P a g e  Development and implementation of health advocacy strategies  Active engagement with state health decision makers and other non-state key actors

3.4.1 Creation of Health Information and Advocacy Unit in the Health Department of the CSN

The EC CAFOD programme planned that creation of Health Information and Advocacy Unit in the Health Department of the CSN should be one of the activities under objective four of the programme. The unit was not created as a stand-alone but there were some activities carried out by CSN while there was an Advocacy Advisor for the programme. The Advocacy Advisor was engaged on the programme for less than a year. This really left a gap in the implementation of the advocacy component of the programme. A lot of the activities for advocacy at the CSN level were on ad-hoc basis and there were no detailed records of the events. This gave rise to the fact that some of the planned indicators for this activity were not achieved. For instance, a health advocacy strategy was not defined and implemented for the Catholic Church; the Health Information and Advocacy unit was not formally created and thereby could not remain effective beyond the duration of the project. The Health Think Tank of the Catholic Secretariat that was also meant to be producing widely circulated policy and position papers on health services was not also functional. It is recommended that this section of the programme should be managed and strengthened with resources ploughed into recruitment and maintenance of skill staff to run the unit.

3.4.2: Facilitation and support to the CSN Health Think Tank There was no record of formation of a Health Think Tank at the CSN. However, there were some advocacy activities organized by the CSN while the engaged Advocacy Advisor was still in office. This is also one of the gaps identified in the implementation of the EC/CAFOD programme by the CSN. More efforts need to be put into planning such laudable strategy in future.

3.4.3: Advocacy Training and Divulgation of CAFOD Advocacy Tool In the course of the implementation of the programme, one advocacy training was organised by CSN out of the five planned. It was facilitated by CSN’s Health Policy & Advocacy Advisor, and two consultants. It targeted selected diocesan health coordinators from the 9 Catholic provinces, selected PHC leaders, and staff of the Catholic Secretariat of Nigeria. Two traditional rulers, a journalist and one of CAFOD’s Programme Accompaniers also participated. In all, there were 28 participants made up of 15 women and 13 men. The training methodology and content were technically appropriate and useful in equipping the participants for appropriate advocacy actions at their dioceses, facilities and 39 | P a g e communities. The participants also developed advocacy issues to be followed up after the training. Evaluation of this component of the programme revealed that lack of continuity in the work of the Advocacy Advisor constituted a major challenge to follow up of the activities. Hence, the expectation of having at least 6 examples of advocacy initiatives with local/state authorities and evidence of joint campaigns with other health stakeholders (as contained in the log frame) was not feasible at the end of the programme.

In spite of these major challenges in implementing objective four, CSN HIV Coordinator during the evaluation said some of the dioceses and facilities reported some level of positive outcome of their advocacy activities carried out. For instance, Namu PHC became an ART outstation of OLA (Catholic) Hospital Jos, a government approved ART provider thus Namu’s HIV clients (1,884 women, and 1,003 men) were currently accessing ART in Namu rather than having to travel 250kms to Jos every month; Vandeikya/Adikpo HBC programme made a similar arrangement with the Federal Medical Centre (FMC) in Benue State. Bali PHC in Jalingo diocese also paid an advocacy visit to their local government and it resulted in improving the infrastructure and provision of equipment to the facility especially laboratory services.

This indicated that given adequate human resource, the advocacy strategy would have been more effective in mobilizing better support for health services for the poor at the communities.

3.4.4: Development and implementation of health advocacy strategies Since this activity was linked directly to the functionality of other components of the advocacy strategy, like creation of a function Health Advocacy Unit with a Think Tank, regular advocacy, trainings and meetings which were dismally implemented in the project, the development and implementation of advocacy strategy for the Church was not implemented. It is recommended that in programme planning, activities that are directly linked to other primary activities should be planned as sub activities and not stand alone. Efforts should also be made to ensure implementation of both sub and primary activities in order to achieve the desired results.

3.4.5: Active Engagement with State Health Decision Makers and other Non-State Key Actors In the course of the programme implementation, two (2) health stakeholders meetings were held by the CSN out of the five (5) meetings planned. The meetings aimed at increasing engagement with state health decision makers and other key non-state actors in the provision of pro-poor health services at the

40 | P a g e facilities and communities. A number of successes were recorded as a result of the advocacy meetings and activities. These included: i) Health stakeholder meetings provided a valuable opportunity to highlight the contribution made by the Catholic health services to health care delivery throughout the country, and especially to primary health care. NPHCDA also expressed readiness to work closely with the Catholic health services and to provide any necessary support. It also created a forum where the issue of the need for health systems reform in order to discourage ‘skills flow’ and encourage ‘retention’ was discussed with government actors. ii) In Kogi state, the dioceses of Idah and Lokoja increased their level of partnership with the communities as well as with the state government. This resulted in the allocation of forty million naira for the improvement of Catholic health facilities including a Catholic Midwifery Training Institution in Idah Diocese; supply of drugs and the secondment of doctors. This was made possible because of sharing of data with the government and making advocacy visits which demonstrated that the Catholic Church contributed 60% to the human resources development in Kogi state. iii.) CSN’s Health Policy and Advocacy Advisor was successful in lobbying the Federal Ministry of Health to include the Catholic health facilities in the distribution of HIV test kits. This resulted in the allocation of 8,000 HIV test kits which were distributed to the health facilities most in need of these items. iv.) The Shuwa PHC had close engagement with the local government and traditional chief of the community to advocate for resource allocation to the centre. The Chief of Shuwa was invited to participate in the advocacy training organized by CSN in Abuja in June/July 2009. After the training he (the chief) became a strong advocate for the PHC with the local government authority and the Adamawa state government, and this has resulted in increased support to the PHC, especially its maternity services. v.) Namu PHC organised an advocacy visit to the local government chairman and the director of Primary Health Care services to press for its inclusion in health resources allocation. This resulted in the local government officials visiting the facility and assessing the services provided. vi.) Vandeikya/Adikpo HBC programme made a similar arrangement with the FMC in Benue State. After active lobbying by local community leaders, the FMC agreed to provide ART access through Vandeikya/Adikpo HBC, thus saving the programme’s HIV clients (399 women, and 216 men) a 200km journey to the state capital.

41 | P a g e vii.) Bali PHC in Jalingo diocese paid an advocacy visit to their local government. It resulted in improving the infrastructure and provision of equipment to the facility especially laboratory services.

4.0 Short term impact of the EC/CAFOD PHC programme

Attempt was made to evaluate the impact of the EC/CAFOD PHC programme on: a. PHC’s capacity to provide sustainable quality primary healthcare service b. Clients satisfaction c. Changes in health behaviour and incidence of communicable diseases among the community members The evaluation team also attempted to evaluate the EC/CAFOD programme performance indicators related to the improvement in practice and capacity of the PHC to deliver sustainable primary healthcare services.

4.1 PHC’s capacity to provide sustainable quality primary healthcare service

The facility checklist was used to assess the level of availability of basic staffing, equipment and drugs in PHC context, as well as the level of clientele for various services in the 9 PHC facilities visited (excluding Abuja HIV site). The objective was to determine whether or not the PHC had minimum capacity (in terms of staff, equipment and drugs) required to deliver quality primary healthcare services.

The programme performance indicator stipulated that 70% of partner PHC would deliver improved quality and increase the number of services being provided. In terms of types of services being provided, all of the 9 facilities (100%) visited offered a range of basic primary healthcare services including: ANC, immunization, health education, nutritional education, treatment of minor illnesses for adults and children, HIV counselling and HIV testing. Eight of the nine facilities (88.8%) offered STI management, community mobilization and referral/follow up services while six of the facilities (66.7%) offered labour/delivery services as well as family planning services. Only a third of the facilities (33.3%) provided postnatal services while five facilities (55.6%) offered growth monitoring services for children. See table 1 of appendix 3 for further details.

42 | P a g e The assessment of medical equipments available in the facilities showed that basic equipments for ANC and child welfare services were available in most of the facilities. About half of the facilities had basic equipment for delivery. However, only two of the facilities had equipment for assisted vacuum delivery while feeding tube for sick babies was not available in all facilities visited. About seven out of the nine facilities had basic laboratory equipment. While all of the facilities had standard consultation rooms, only two facilities had SOP or prescription manual. (See tables 2 – 6 in appendix 3 for full analysis of equipment availability).

In terms of availabilty of drugs and supplies in the facilities, all of the facilities had antacids, vitamins, general pain relievers, antibiotics and antiseptics in stock. Most of the facilities (8 out of 9) had anti- pruritus (anti-itching drugs) and vaccines in stock. Seven out of the nine facilities had vaginal passeries (vaginal insertion drugs), anti haemorrhagics (Vitamin K for arresting bleeding), infusions (dextrose) and blood tonics/tablets. Mineral supplements were in stock in six facilities while five facilities had laxatives, anti-diarrhoeals, anti-fungals, labour inducing drugs and pain relief ointments. Table 7 in Appendix 3 provides more details. Evidence exists to show that the number of pregnant women receiving appropriate antenatal care and number of deliveries assisted by health professionals and trained birth attendants had over the years increased across the facilities. For example, in six facilities (D/Kurmi, Fuka, Jada, Namu, Mandella and Yakoko) the number of pregnant women who received ANC increased from 2507 in 2006 to 3117 in 2010, representing 24% increase. Similarly, number of deliveries in five facilities (excluding Fuka) increased from 171 in 2006 to 373 in 2010, representing 118% increase. Figure 13 below shows the trend (2006-2010) in number of clients receiving ANC and delivery services.

Figure 3: Trend in number of clients for ANC and Delivery services in 6 selected facilities*

43 | P a g e Table 10 in Appendix 3 gives more details of the trend in clientele for basic PHC services across the facilities visited

4.2 Client Satisfaction The level of clients’ satisfaction with the quality of service being provided by the PHCs under EC/CAFOD programme was assessed through exit interviews with 53 randomly selected clients who had come to receive services at the selected facilities on the day of evaluation. The demographic characteristics of the respondents showed that close to three-quarter (73%) of them were females and majority (81%) were Christians. A quarter of the respondents were visiting the facilities for the first time while others (75%) were returning patients. Two-third of the respondents were in the facilities to seek healthcare services for themselves while 30% and 4% accompanied their children and spouses respectively. About two-third (66%) reported that they did not wait long in the facility before they were attended to, 23% said that they waited for between 1 and 3 hours while 11% waited for 4 hours or more. More than half (54%) of the respondents rated the quality of services received at the facility on the day of interview as good, 40% said it was very good and 4% described the service as excellent.

The returning clients among the respondents were asked to describe the quality of service in the facilities based on efficacy of treatment, friendliness of staff, cost of treatment and drugs, waiting time in the facility, availability of qualified staff and availability of drugs and equipment. The results showed that overwhelming majority of the respondents rated the facilities high on all of these criteria. (See details in appendix 4).

44 | P a g e About 79% of the clients believed that the incidence of preventable diseases was decreasing in their communities. Majority of the clients attributed the decrease to increase in uptake of immunization, mass distribution of ITN, health education and hygiene campaigns by the PHC staff. Almost all (98%) of the clients would recommend the facility to others.

4.3 Changes in health behaviour and incidence of communicable diseases

There were oral evidences to show that the community mobilization activities of the PHCs had resulted in positive change in health behaviours of the community members which had led to reduction in incidences of communicable diseases such as diarrhoea, malaria, including HIV. It had also led to increase in uptake of basic obstetric and maternal and child welfare services.

The results of the interviews held with the community leaders, TBA, village health committee and PHC staff showed that majority of them believed that the prevalence/incidence of most preventable diseases had decreased in the community over the past few years. Although, there was no written evidence to support their claims, the respondents explained the reasons for their belief in different ways: “They are decreasing because the people are enlightened on taking their sanitation seriously. The people are aware of the importance of proper hygienic lifestyle. Cutting of grass in their surroundings has help reduce mosquitoes”- A PHC manager

“For me I think it is reducing because when I came two years back a lot of people were coming in with fever and convulsion and other similar things like that and I think the issue of treated bed nets government distributed to the community has also contributed as people are now making use of the nets”- A JCHEW

“Why I said it has reduced is because the community is aware, we give them health talk. They are also aware about their drinking water, personal hygiene at home, keeping their surroundings clean. So, the area of malaria has drastically reduced”- A CHEW

“I don’t know about that. Sometimes because I stay in the pharmacy if I look at the record of the drugs we give out the one of diarrhoea is still intact which means they are not demanding

45 | P a g e for them. It is decreasing because of the good drugs we have for the treatment of such diseases”- A Pharmacy Assistant Similarly, the PHC staff believed that the reduction in the incidence of HIV infection in the community is attributable to their community outreach, mobilization and awareness campaigns: A PHC staff said: “Because of the awareness on HIV/AIDS, people are more careful about their sexual lifestyle than before, HIV infection is reducing, the turn-up for HIV testing is high, people now know that they can leave with an HIV infected person without contracting it. The people here now know how to keep a hygienic lifestyle”- A CHEW However, some of the respondents had different opinions. A few of the PHC staff said there was no concrete evidence to show that incidence of some diseases such as malaria is decreasing. They observed that incidence of malaria tended to fluctuate with weather conditions, increasing during raining season. The statistics on disease pattern from the PHC records also showed a mixed pattern.

5.0 STRENGTHS, KEY CHALLENGES AND RECOMMENDATIONS

5.1 STRENGTHS OF THE PROGRAMME  Community focus: The design and implementation of the programme focused on community development and meeting the needs of people in remote communities who had little or no access to government infrastructure.  Bottom up approach: The programme allowed the PHC and community members to make decisions based on their needs and contexts.  Building on the existing church structure: The use of the existing church structure (CSN, Province, Diocese and PHC facilities) helped in reaching large numbers of people in the community and resulted in the Catholic Church institutional strengthening.

5.2 KEY CHALLENGES i. Weak Monitoring and Evaluation system for the programme: Lack of structured baseline evaluation before the commencement of the EC/CAFOD PHC programme made the assessment of the achievements of the programme in quantitative terms difficult since there were no benchmark figures to some key performance indicators. Similarly, the data collection tools and reporting formats were not structured to routinely monitor the progress of the key indicators as

46 | P a g e listed in the log frame. Narrative quarterly report without a summary data on key indicators was not adequate. ii. Inadequate funding of M&E activities: It was also observed that very limited budgetary allocations were made for M&E activities, including the final evaluation exercise. Between 10% -15% of the total programme grant should have been devoted to M&E for effective system that will demonstrate results and ensure learning. iii. High staff turn-over: There was a high staff turn-over at all levels of the programme implementation. Technical Accompaniers and Health Advisors were recruited but not retained for long period and there was also high staff turn-over at the PHC facilities. Most of the key staff that implemented the programme at CSN, province and CAFOD levels had disengaged as at the time of the evaluation. The amount and quality of data generated for the programme evaluation would have been more if the staff were still available for interview. iv. Wide scope of the programme: The wide scope of the programme in terms of the geographic coverage (63 PHC facilities spread across 19 northern states and FCT in Nigeria) and intervention areas (all primary health care service areas) had negative implication for the coordination, supervision and impact of the programme. There were reported cases of communication problems and feedback system between CAFOD and partners, late disbursement of funds, irregular monitoring and supervision visits and poor visibility of the impact of the intervention due to large area of coverage.

5.3 RECOMMENDATIONS Based on the assessment of each objective in section three, the following recommendations are proffered to improve quality of future programming.

Objective 1:

1. It is important for the PHC and dioceses to develop strategies of attracting staff with required educational qualifications to make trainings more effective. 2. Early procurement of equipment is recommended to ensure optimal use during the life of the programme. 3. Criteria for selection of benefiting facilities for different supports should be discussed in a participatory process with all stakeholders.

47 | P a g e 4. Staff recruitment process and condition of service for technical staff should be reviewed for future programming in order to attract and retain qualified personnel. 5. The development of the guidelines on models of best practices on gender is urgently required to help the PHC in application of gender sensitive approaches. Objective 2

6. VHC members should be adequately enlightened to know that participation in the committee is a voluntary service (that attracts no monetary gain) to engender community ownership of the programme. The concepts of voluntarism, dedication and community ownership should be emphasised as part of orientation when new members/committees are being inaugurated. Objective 3

7. The challenge faced with the production of newsletters and case studies could be attributed to the fact that the task was not assigned to any specific staff or sub-group but remained the responsibility of all the players in the programme. Such responsibility should be assigned to a specific committee made of staff members with required skills and commitment. 8. The collaboration and linkages with state and local governments and other private health stakeholders was a cost effective approach (PHC received various items and knowledge at no costs) and has potentials for sustainability. This should be continued even after the EC/CAFOD support. Objective 4 9. It is recommended that in programme planning, activities that are directly linked to other primary activities should be planned as sub activities and not stand alone. Efforts should also be made to ensure implementation of both sub and primary activities in order to achieve the desired results. Cross cutting issues 10. The importance of M&E to demonstrate programme achievement and learning cannot be over emphasized. Future programme should make M&E a priority by setting up an effective M&E system with relevant benchmark data, user friendly data collection and reporting formats that routinely track programme performance indicators and allocation of adequate funds for key activities such as monitoring visits and evaluation exercise. 11. It is recommended that the current staff recruitment system should be reviewed to identify reasons for high staff turn-over, especially for highly skilled technical staff. Establishment of a 48 | P a g e strong human resources unit and competitive working condition is recommended to ensure that highly qualified staff are attracted and retained through-out the duration of future programme. CAFOD should also provide technical support and guidance in staff recruitment process by the partners. 12. It is strongly recommended that the scope of future programme should be limited to two to three PHC service area(s) and few states. For example, future programme can focus on immunization, drug provision and maternal and child health in four to six states for effective coordination and maximum impact. 13. In order to further strengthen the Church structure and ensure an effective and sustainable monitoring and supervision system, technical capacity of the CSN and health coordinators should be built to perform this role with adequate fund allocation for the purpose. 14. The key to the sustainability of the programme in the community lies with the VHC. Selection of VHC members should be done collaboratively to ensure that credible and dedicated persons are chosen as members. The capacity of the VHC members should be strengthened to play significant role in the design and implementation of the community mobilization activities of the PHC and to be able to mobilize resources within the community to continue to implement the activities after EC/CAFOD funding.

49 | P a g e APPENDICES

50 | P a g e APPENDIX 1: PROGRAMME RESPONSES TO RECOMMENDATIONS RECOMMENDATIONS PROGRAMME RESPONSE

Objective 1:

1. It is important for the PHC and dioceses to i. This will be implemented by CSN, develop strategies of attracting staff with dioceses and PHC facilities. required educational qualifications to make ii. CAFOD will build into its future trainings more effective. programme technical support to partners on recruiting qualified staff. 2. Early procurement of equipment is iii. CAFOD will ensure this is recommended to ensure optimal use during observed and early need the life of the programme. assessment for any capital cost is done before the commencement of programme. 3. Criteria for selection of benefiting facilities iv. CAFOD will work with its future for different supports should be discussed in a partners and beneficiaries during participatory process with all stakeholders. baseline survey to identify specific need and develop guidelines for selection based on available resources and impact 4. Staff recruitment process and condition of v. CAFOD and its partners will service for technical staff should be reviewed consider this in all its future staff for future programming in order to attract and recruitment retain qualified personnel.

5. The development of the guidelines on models vi. CAFOD will focus on developing of best practices on gender is urgently this guideline in its future required to help the PHC in application of programme with the partners gender sensitive approaches.

Objective 2

6. VHC members should be adequately vii. Dioceses, PHC facilities and enlightened to know that participation in the Village Health Committees (VHC)

51 | P a g e RECOMMENDATIONS PROGRAMME RESPONSE

committee is a voluntary service (that attracts will develop and train VHC no monetary gain) to engender community members on their roles ownership of the programme. The concepts of viii. CAFOD will provide technical voluntarism, dedication and community support to achieve this by sharing ownership should be emphasised as part of experiences of how it has work in orientation when new members/committees other places and the National are being inaugurated. Primary Health Care minimum ward packages for Community Development committee’s roles. Objective 3

7. The challenge faced with the production of ix. CAFOD and its partners will newsletters and case studies could be considered in future programme attributed to the fact that the task was not learning assigned to any specific staff or sub-group but x. CAFOD Nigeria will seek support remained the responsibility of all the players from the programme effectiveness in the programme. Such responsibility should and learning team in London for be assigned to a specific committee made of local and global sharing of best staff members with required skills and practices commitment. 8. The collaboration and linkages with state and xi. CAFOD and its partners are local governments and other private health committed and will continue to stakeholders was a cost effective approach keep this close collaboration with (PHC received various items and knowledge the government and other at no costs) and has potentials for stakeholders sustainability. This should be continued even xii. CAFOD will ensure linkage with after the EC/CAFOD support. government agencies and other international and national agencies so that they understand and support CAFOD’s work in Nigeria Objective 4 9. It is recommended that in programme xiii. Not agreeing with this planning, activities that are directly linked to recommendation as most of the 52 | P a g e RECOMMENDATIONS PROGRAMME RESPONSE

other primary activities should be planned as activities of objective/outcome 4 sub activities and not stand alone. Efforts were directed at National level should also be made to ensure implementation advocacy (strategic levels) by the of both sub and primary activities in order to Strategic partner, which was achieve the desired results. Catholic Secretariat of Nigeria while outcomes 1-3 were mostly linked to the primary activities (operational levels) of the programme. Cross cutting issues

10 The importance of M&E to demonstrate xiv. CAFOD and its partners will in . programme achievement and learning cannot future programming conduct be over emphasized. Future programme baseline surveys, document should make M&E a priority by setting up an findings, set measureable effective M&E system with relevant indicators from result of the survey benchmark data, user friendly data collection and agree on monitoring and and reporting formats that routinely track evaluation plans/tools before the programme performance indicators and commencement of programme allocation of adequate funds for key activities implementation. such as monitoring visits and evaluation xv. Roles of each stakeholder in exercise. monitoring and evaluation will be agreed and documented. xvi. Programme budgets will include adequate monitoring and evaluation costs and especially collecting of beneficiaries voice on change to quality of life and how holistic the programme is in meeting their needs 11 It is recommended that the current staff xvii. Same as (V) above . recruitment system should be reviewed to 53 | P a g e RECOMMENDATIONS PROGRAMME RESPONSE

identify reasons for high staff turn-over, especially for highly skilled technical staff. Establishment of a strong human resources unit and competitive working condition is recommended to ensure that highly qualified staff are attracted and retained through-out the duration of future programme. CAFOD should also provide technical support and guidance in staff recruitment process by the partners.

12 It is strongly recommended that the scope of xviii. CAFOD will in its future . future programme should be limited to two to programme focus on three three PHC service area(s) and few states. For Strategic areas: Health with example, future programme can focus on specific emphasis on Maternal, immunization, drug provision and maternal newborn and child health, and child health in four to six states for Governance as it is linked to health effective coordination and maximum impact. and Gender xix. CAFOD will be guided by national indicators of factors driving poverty in Nigeria, most disadvantaged groups, where impacts can be made and shared and the availability of funds xx. Three states with bad indicators and one with good indicators will be targeted for impact and cross learning opportunities 13 In order to further strengthen the Church xxi. CAFOD will work more at . structure and ensure an effective and strategic by building the capacity sustainable monitoring and supervision of the catholic Secretariat, system, technical capacity of the CSN and Provincial and Diocesan structures 54 | P a g e RECOMMENDATIONS PROGRAMME RESPONSE

health coordinators should be built to perform xxii. CAFOD will build the capacity of this role with adequate fund allocation for the the local church and enable it purpose. generate local resources that will support its development work. This will be rooted in the catholic social teaching and CAFOD LiveSimply principles of social development approach 14 The key to the sustainability of the xxiii. Same as in action vii & viii above. . programme in the community lies with the VHC. Selection of VHC members should be done collaboratively to ensure that credible and dedicated persons are chosen as members. The capacity of the VHC members should be strengthened to play significant role in the design and implementation of the community mobilization activities of the PHC and to be able to mobilize resources within the community to continue to implement the activities after EC/CAFOD funding.

55 | P a g e APPENDIX 2: EVALUATION TERMS OF REFERENCE 1) Background to CAFOD The Catholic Agency for Overseas Development (CAFOD) is the official development and relief agency of the Catholic Church in England and Wales, and part of the global Caritas network, a confederation of over 150 Catholic aid agencies. It grew from an initiative taken by Catholic women and was set up as a charity in 1962 by the Bishops of England and Wales, with the task of expressing the concerns of the Catholic community for the needs and problems of the developing world. CAFOD’s mission is to promote human development and social justice in witness to Christian faith and gospel values.

2) The Primary Health Care Programme CAFOD has been working in Nigeria since the 1970s with Catholic Church as its major partner. The Primary Health Care (PHC) Programme was launched in 1999 to provide support to a network of Catholic- run Primary Health Care projects in the North of the country. There have been several subsequent phases of the Programme, with a continued focus on strengthening the network of the partners and building capacity on health related issues. Programme reviews have been undertaken in 2003, 2005 and 2009.

The initiative entered its current phase in 2007 based on the recommendation of the 2005 review. A co-funding grant was secured in 2006 through the support of the European Commission, for a 3-year programme entitled ‘Improving Primary Health Care for Rural Poor Communities in Northern Nigeria’ (Contract No. ONG-PVD/2006/119-131). Implementation began on 1st February 2007, following the signing of a project contract between the EC and CAFOD on 20 December 2006. A further 1 year extension was granted by the EC which ended in January 2011.

The purpose of the current programme is to strengthen Catholic Church Health structures, focusing in particular on primary health-care facilities to enable them to provide high quality, sustainable healthcare services that meet the needs of poor rural communities in the 19 dioceses that make up the three northern Ecclesiastical Provinces of Abuja, Jos and Kaduna.

The specific objectives include:

i) To increase the capacity of 63 Catholic Church-run primary Health care (PHC) providers to manage and deliver high quality, sustainable Health care services. ii) To increase the level of community participation in and ownership of primary Health care in order to promote sustainable, healthy lifestyles. iii) To share learning between PHC providers in order to strengthen Health networks and promote good practice in primary health care. iv) To build the capacity of the Catholic Church in Health policy and advocacy issues to influence the development and monitoring of pro-poor health policies. In terms of civil administration, the programme covers facilities in 18 states and the Federal Capital Territory of Abuja. In addition to the 63 primary health care facilities covered by the programme,

56 | P a g e another important focus for capacity building is the management structures at diocesan and provincial levels.

3) Purpose of the evaluation The Evaluation will focus on the effort of the current programme in building the capacity of 63 Catholic Church-run PHC providers to manage and delivery high quality, sustainable healthcare services.

The evaluators will assess the EC/CAFOD Primary Health Care Programme in order to:

a) Enhance accountability to stakeholders and European Commission. b) Capture lessons to help improve CAFOD’s decision-making and work in responding to the health needs of Nigerians c) Guide future decisions on CAFOD’s long-term strategy in Nigeria. 4) Evaluation Criteria The evaluation will assess the programme according to the following criteria: i) Relevance/appropriateness: Assess whether the programme inputs, activities and outcomes are in line with local needs and priorities. Will seek to answer questions like:  How effective were CAFOD, CSN and the 19 Dioceses from the 3 Northern Catholic Ecclesiastical Provinces in assessing and analysing the needs and context?  To what extent were past lessons or recommendations taken into account?  How relevant was the programme in addressing the needs and priorities of the local communities including specific gender needs and those affected or living with HIV.?  Did the programme take the local wider issues and context into account (e.g. culture, gender, etc.)?  Were the staffing and management arrangements appropriate for effective programme delivery, building the capacity of CSN and the 19 Dioceses from the 3 Northern Catholic Ecclesiastical Provinces, and strengthening the relationship between CAFOD, CSN and the 19 Dioceses from the 3 Northern Catholic Ecclesiastical Provinces  Were intended beneficiaries able to participate effectively to inform programme design and implementation?  How could relevance and appropriateness be improved? ii) Sustainability: Assess whether PHC programme activities were carried out in a way that takes longer-term and interconnected health problems into account. Will seek to answer questions like:  How has this programme built upon, supported and developed existing programmes or structures (partner’s or government)?  How has the programme supported or disrupted communities’ or partners’ ability to support themselves?  How does this programme fit with CAFOD’s plans for longer-term support in Nigeria  How could sustainability be improved?

57 | P a g e iii) Coherence: Assess whether there is consistency with relevant policies (such as National Primary Health Care, National Health Policy, HIV and AIDS etc).  To what extent have national health strategies been taken into account?  Have the different aspects of the programme complemented or contradicted each other? iv) Coverage: Assess whether the major population groups facing poor health indices are reached and supported. Will seek to answer questions like:  How successful was the programme in reaching the poorest, most vulnerable or disadvantaged?  Have programme activities reached areas not covered by other agencies?  How could coverage have been improved? v) Efficiency: Measures the qualitative and quantitative outputs achieved in relation to the inputs and compares alternative approaches to see whether the most efficient approaches were used. Will seek to answer questions like  How efficient was this programme (effectiveness in relation to budget)?  What factors affected the efficiency of the overall outcome? (E.g. political context, logistics, working with local partners, staff capacity, monitoring systems, procurement policies, transport, finance procedures).  Were some locations or components more efficient than others, if so why?  How could efficiency be improved? vi) Effectiveness: Measures the extent to which activities have achieved their purpose or whether this can be expected on the basis of the outputs.  To what extent did the programme achieve its goals and objectives? What are the main factors that have facilitated or constrained the achievement of these?  Have the levels of success varied between locations? What are the reasons for this?  How effective were CAFOD’s, CSN’s and the 19 Dioceses’ systems of ongoing analysis and monitoring?  To what extent did CAFOD co-ordinate with other NGOs, CARITAS organisations, partners, donors, government?  To what extent were partners involved in field-based co-ordination mechanisms (with other Caritas agencies, NGOs, the government), how effective were these?  How successful has the programme been in delivering Primary health care outcomes in accordance with WHO or Federal Ministry of Health PHC Principles? (Including the Alma Ata declaration).  How could effectiveness be improved? vii) Impact: looks at the wider effects of the Programme (social, economic, technical and environmental) on individuals and groups (gender, age groups, communities and institutions). Impacts can be intended and unintended, positive, negative, macro (sector) and micro (household). Will seek to answer questions like  Did CAFOD/CSN and the 19 Dioceses have wider effects on individuals and groups? If so why did these impacts arise?  Have the impacts varied between locations, if so why? 58 | P a g e  How would the impact have been varied if CAFOD / CSN and the 19 Diocesan programmes were different?  How could the impact of the programme be improved?

5) Outputs The evaluation report (in MS Word format) should not be more than 30 pages and should articulate findings, draw conclusions and make recommendations. The report (including annexes where needed) will be in clear, plain English and will outline the main findings and recommendations.

The report will cover the whole EC/CAFOD programme components and must be submitted on or before 31st May 2011.

The following headings are suggested for the report:

 Executive Summary  Introduction  Methodology  Findings/Analysis/Outcomes addressing issues raised in the TOR. This could be organised by reference to the Programme’s four objectives and then for each objective there could be an analysis of the relevance and effectiveness of each activity (workshops, accompaniment visits, etc)  Conclusion and Recommendations with a section dedicated to drawing out specific lessons with suggestions addressed to the relevant institution(s) for taking forward lessons learned  Appendices, to include evaluation terms of reference, maps, beneficiary research and bibliography. (All material collected in the undertaking of the evaluation process should be lodged with CAFOD prior to termination of the contract) The report and all background documentation will be the property of CAFOD (as the contracting organisation) and will be disseminated and publicised as appropriate by CAFOD.

6) Intended users of the evaluation  European Commission  CAFOD particularly staff in Nigeria and ID  CSN  The 19 Dioceses from the 3 Northern Catholic Ecclesiastical Provinces 7) Key persons specification It is anticipated that the evaluation will be conducted by a national consultant who will have the following experience and skills:

 At a least a Master degree in Public Health.  A good understanding of primary health care in the context of northern Nigeria  Familiarity with the current programme, the context and the health structures and services of the Catholic Church  Sound experience in participatory review and evaluation methodologies

59 | P a g e  Excellent facilitation, analytical and report writing skills  Fluent written and spoken English  At least 5 years relevant experience of evaluating Health programmes, especially Primary Health Care and working in Public health and development.  Ability to analyse and synthesise in writing relevant information relating to public health data  Ability to work respectfully with national NGO partners Desirable:

 Experience of working with Health programmes and faith based international agencies and faith based national NGOs 8) Evaluation methodology Approach

The evaluators will propose the methodology for the evaluation, however it should:

 Use internationally accepted guidelines of evaluating health programmes  Ensure good representation  Use participatory approaches and enable feedback from participants  Include field visit to partners and beneficiaries  Organise a one day stakeholders consultative meeting

Timeframe

It is anticipated that the evaluation will require approximately 35 days work, in April to June 2011

Start Date End date Activity

7th April 2011 7th April 2011 Briefing on programme and agreeing on evaluation methodology and timeline

8April 2011 12 April 2011 Desk review and development of evaluation tools

Protocol and Instruments Development

13 April 2011 12 May 2011 Programme Evaluation/Field trip/stakeholders consultation meeting

First draft report writing

19th May 2011 19th May 2011 Submission of 1st draft report

20th May 2011 24th May 2011 Evaluation reference group discuss report

25th May 2011 25th May 2011 Feedback on draft report discussed with evaluators

60 | P a g e 26th May 2011 30th May 2011 Final Report writing

31st May 2011 31st May 2011 Submission of final report

Process

 Initial meeting in Jos to review background information to inform the evaluation proposed methodology  Write-up methodology and timeline  Desk based review of key documents  One day stakeholder’s meeting  Identify programme areas/partners to visit  Field visits – interviews/ focus group discussion with stakeholders: beneficiaries, CSN, Diocesan Health Coordinators/PHC leaders  In-country presentation of preliminary findings to partners  Produce draft evaluation document  Presentation of draft report to CAFOD staff and reference groups to make comments  Incorporation of comments received and preparation of the final report

61 | P a g e APPENDIX 3: ANALYSIS OF THE PHC CHECKLIST Table1: Type of services being provided by the PHC facilities visited

Key Services Facilities Providing Service Total % of Number total sites ANC Agagbe, Ankpa, D/Kurmi, Fuka, Jada, 8 89 Namu,Mandella,Yakoko Labour & Delivery Agagbe, D/Kurmi, Jada, Namu, Mandella, 5 56 Post Natal care Fuka, Namu, Mandella 3 33 Immunization Agagbe, Ankpa, D/Kurmi, Fuka, Jada, 8 89 Namu,Mandella,Yakoko General Lab screening Agagbe, Ankpa, D/Kurmi, Fuka, Jada, Namu, 9 100 Vandeikya,Mandella,Yakoko HIV counseling and Agagbe, Ankpa, D/Kurmi, Fuka, Jada, Namu, 9 100 testing Vandeikya,madella,Yakoko Growth Monitoring Ankpa, D/Kurmi,Fuka,Madella,Yakoko 5 56

Most of the 9 facilities visited provided the key PHC services. All of nine facilities (100%) provided general lab screening, HIV counseling and HIV testing services. 8 of them (89%) provided ANC and Immunization while 5 of the 9 (56%) provided labour and delivery services. Though Adikpo/Vandeikya PHC did not offer ANC or take deliveries in the facility, they reported that they offered ANC in the communities and refer deliveries to their affiliated clinic. Also, Yakoko offered delivery services only in their affiliated hospital not in the PHC. Five of the facilities reported that they offer growth monitoring services but on investigation, it was found that it was only weight and height that were being measured. Only three of the facilities provided post natal services.

Table 2: Availability of basic items and equipment for ANC services

Key Equipment Facilities Where Equipment is available Total % Number Sphygmomanometer Agagbe, Ankpa, D/Kurmi, Fuka, Jada, 8 100 & Stethoscope Namu,Mandella,Yakoko Foetal stethoscope Agagbe, D/Kurmi, Jada, Namu, Mandella,Yakoko 6 75 Vaginal examination Agagbe, Ankpa, D/Kurmi, Fuka, Jada, Namu, 1 13 equipment Vandeikya, Mandella,Yakoko Immunization Ankpa, D/Kurmi, Fuka, Jada, Mandella,Yakoko 6 75 equipment and vaccine Couch with Pillow Agagbe, Ankpa, D/Kurmi, Fuka, Jada, Namu, 8 100 mandella,Yakoko

62 | P a g e Growth Monitoring Ankpa, D/Kurmi,Fuka,Mandella,Yakoko 5 63 equipments

Most of the 8 facilities that offered ANC services had the key ANC equipment available at the time of visit. Of the 8 facilities, 8 (100%) of them had a stethoscope, sphygmomanometer and a couch with pillow. 6 of them had a foetal stethoscope and immunization equipment with vaccine available in the ANC room. 5 had growth monitoring equipment (weight and height scales only). However, only 1 facility (Dogon Kurmi) had vaginal examination equipment on ground.

Table 3: Items and equipment availability in the Labour Rooms

Key Equipment Facilities Where Equipment is available Total % of Number total sites Bed with mattress Agagbe, D/Kurmi, Jada, Namu,Mandella 5 100 Foetal stethoscope Agagbe, D/Kurmi, Jada 3 60 Vaginal examination Agagbe, D/Kurmi, Jada, Namu, Mandella 5 100 equipment Running water Jada 1 20 Mackintosh Agagbe, D/Kurmi, Jada, Namu,Mandella 5 100

Of the 5 facilities that were offering labour and delivery services in the PHCs themselves, all 5 had a bed with matress, mackintosh and vaginal examination equipment available in the labour room. 3 of the 5 had a foetal stethoscope in the labour room but only Jada had running water in the labour room.

Table 4: Items and equipments availability in the Delivery Rooms

Key Equipment Facilities Where Equipment is available Total % of Number total sites Normal delivery kit Agagbe, Jada, Mandella 3 60 Equipment for assisted vacuum Agagbe, D/Kurmi 2 40 delivery Equipment for new born care and Agagbe, D/Kurmi, Mandella 3 60 neonatal resuscitation Surgical set for minor procedures Agagbe, D/Kurmi, Jada, Namu, 5 56 like episiotomy,circumcision Mandella stitching Lamp/torchlight for delivery Agagbe, D/Kurmi, Jada, Mandella 4 44 Mucus extractor for babies Agagbe, D/Kurmi, Namu, Mandella 4 44

Table 5: Items and equipment availability in the Laboratories

63 | P a g e Key Equipment Facilities Where Equipment is Total % of available Number total sites Reagents for essential lab Agagbe, Ankpa, D/Kurmi, Fuka, 7 78 investigations Jada, Namu, mandella Binocular microscope Agagbe, Ankpa, D/Kurmi, Fuka, 7 78 Jada, Namu, Mandella Specimen containers (assorted) Agagbe, Ankpa, Fuka, Jada, Namu, 6 67 Mandella Refridgerator Ankpa, D/Kurmi, Fuka, Jada, 6 67 Namu, Mandella

Table 6: Items and equipment availability in the consultation rooms

Key Equipment Facilities Where Equipment is Total % of available Number total sites Sphygmomanometer & Stethoscope Agagbe, Ankpa, D/Kurmi, Fuka, 6 67 Jada, Namu, Thermometer Agagbe, D/Kurmi, Jada, Namu, 6 67 Mandella Vaginal examination pack D/Kurmi 1 11 Couch with Pillow Agagbe, Ankpa, D/Kurmi, Fuka, 6 67 Namu, Mandella SOPS & Manuals Ankpa, D/Kurmi 2 22 Trolley / wheel chair D/Kurmi, Fuka, Namu, 4 44 mandella In the consultation rooms, 67% (6 out of 9) of the facilities visited had a sphygmomanometer, stethoscope, thermometer, a couch and pillow. Less than half of them (4 out of 9) had a trolley or wheel chair. Only Dogon Kurmi had a vaginal examination pack and only Ankpa and Dogon Kurmi had SOPs and Manuals available. Jada however had a vaginal examination pack and a trolley/wheel chair but these were kept in another room due to lack of space in the consultation room.

Table 7: Availability of essential drugs and supplies

Essential Drugs/Supplies Facilities Where Drug is available Total % of total Number sites Antacids Agagbe, Ankpa, D/Kurmi, Fuka, Jada, 9 100 Namu, Vandeikya,Mandella,Yakoko Laxatives D/Kurmi, Jada, Fuka, Mandella, 5 56 Yakoko Anti-diarrhoeal Agagbe, D/Kurmi, Namu, Vandeikya, 5 56 Mandella Anti-haemorrhages (vitamin Agagbe, D/Kurmi, Fuka, Jada, Namu, 7 78 K) Mandella, Yakoko

64 | P a g e Dextrose Agagbe, Ankpa, D/Kurmi, Fuka, Jada, 7 78 Namu, Vandeikya,Mandella,Yakoko Vitamins Agagbe, Ankpa, D/Kurmi, Fuka, Jada, 9 100 Namu, Vandeikya,Mandella,Yakoko Anti-anaemia (blood Agagbe, Ankpa, Fuka, Jada, Vandeikya, 7 78 tonic/tablet) Madella, Yakoko Antiseptics and disinfectants Agagbe, Ankpa, D/Kurmi, Fuka, Jada, 9 100 Namu, Vandeikya, Mandella,Yakoko Labour inducing drug Agagbe, D/Kurmi, Jada, Namu, Yakoko 5 56 Antibiotics (Various) Agagbe, Ankpa, D/Kurmi, Fuka, Jada, 9 100 Namu, Vandeikya, Mandella,Yakoko General pain relievers Agagbe, Ankpa, D/Kurmi, Fuka, Jada, 9 100 Namu, Vandeikya, Mandella,Yakoko With respect to availability of drugs and supplies, most of the facilities had essential drugs in stock. All the facilities had antacids, vitamins, general pain relievers, antibiotics and antiseptics in stock. 78% had vitamin K (an anti-haemorrhagic), infusions (dextrose) and blood tonics while 56% had laxatives, antidiarroeals and labour inducing drugs available.

Table 8: Availability of standard forms and registers (HMIS tools)

Key Documentation Facilities Where Total % of total Registers/Records are Number sites available of sites OPD Register Agagbe, D/Kurmi, Jada, 6 67 Namu, Vandeikya, Mandella ANC Register Ankpa, D/Kurmi, Fuka, 5 56 Namu, Mandella Delivery Register Agagbe, D/Kurmi, 3 33 Mandella Drug stock card D/Kurmi, Mandella 2 22 HCT register Namu, Vandeikya, Yakoko 3 33

An assessment of the availability of data collection tools (forms and registers) across all the sites visited left much to be desired, though for those who had, most of them were updated`. 6 of the sites had an OPD register (all 6 were updated) 5 sites had ANC registers (all 5 updated). 3 sites had delivery registers (all 3 updated). 2 sites (D/Kurmi and Mandella) had drug stock cards (both were updated). Only Namu, Vandeikya and Yakoko had standard HCT registers and only that of Namu was updated. Some other forms and registers were being kept but these varied widely across the sites. Fuka was using hand ruled notebooks for their records.

Table 9: Availability of Staff

Key Staff Facilities Where Staff is available Total % of total Number sites 65 | P a g e of sites Community Health D/Kurmi 1 11 Officer CHEW Agagbe, Ankpa, D/Kurmi, Fuka, Namu, 8 89 Vandeikya,Mandella,Yakoko JCHEW Ankpa, D/Kurmi, Fuka, Namu, 7 78 Vandeikya,Mandella,Yakoko Nurse/Midwife Agagbe, Ankpa, D/Kurmi, Fuka, Jada, 6 67 Namu Medical Records Ankpa, Namu, Mandella 3 33 officer Pharmacy Agagbe, Ankpa, Namu 3 33 technician/assistant Medical officer or Namu, Mandella 2 22 NYSC doctor Lab technician Ankpa, D/Kurmi, Fuka, Jada, Mandella 5 56 Lab scientist Jada 1 11 Lab assistant Agagbe, Ankpa, Mandella 3 33

Most of the facilities had a CHEW and JCHEW available (8 and 7 of the sites respectively) 6 of them had a nurse/midwife while 3 had a pharmacy technician and a medical records officer. 6 had at least a lab staff (lab scientist/technician or assistant). Only Namu and Mandella had a Medical officer/NYSC doctor and only Dogon Kurmi had a community health officer.

Trend in level of Clientele for basic PHC services

An analysis of the annual clientele for the key PHC services was done. However, data for some of the years was not available. This was due to non-collection of the data by the consultant for some sites (Agagbe and Ankpa) or unavailability of the data at the facilities). Analysis of the annual clientele for ANC showed that for most of the sites, the clientele remained fairly constant over the years. Ankpa however showed a marked increase (116%) in ANC clients in 2010 compared with 2006 while for Agagbe, there was a decline of 50%. Clientele for labour and delivery services was also fairly constant through the years. Mandella showed a steady increase over the years while clientele for Agagbe also declined by about 50%. Post-natal care clientele for the 3 sites for which data was available (Fuka, Namu, Mandella) showed significant increase through the years (over 100% for all 3 sites).

66 | P a g e Table 10: Trend in clientele for basic PHC services (2006- 2010)

Key ANC Labor/Delivery HIV Testing Immunization Services Year 06 07 08 09 10 06 07 08 09 10 06 07 08 09 10 06 07 08 09 10

Agagbe 534 _ _ _ 267 113 _ _ _ 57 58 _ _ _ 57 494 _ _ _ 1698

Ankpa 2234 _ _ _ 4834 ______95 3542 _ _ _ 3783

D/Kurmi 121 84 79 410 296 34 23 93 61 110 ______55 37 36

Fuka 345 570 488 684 567 ______957 1151 944 995 999 Jada 85 95 90 75 87 10 15 8 6 5 60 117 59 78 128 1020 950 850 680 410

Namu 1260 1448 1110 1217 1348 _ 85 103 64 82 260 240 210 280 360 98 104 126 211 120

Adikpo/ ______V/kya 1040 944 2541 271 125

Mandella 57 71 84 83 98 31 40 39 46 87 _ _ _ _ _ 166 182 1026 1178 1238

Yakoko 639 562 540 601 721 96 84 127 92 89 _ 125 833 861 1096 5310 8081 8820 4312 5069 For immunization services, clientele for most of the sites also increased significantly but Dogon- Kurmi and Fuka remained fairly constant, while Jada’s clientele reduced progressively through the years. Of the 5 sites for which HIV testing data was available, Adikpo/Vandeikya HBC showed a drastic drop in clientele in 2009 and 2010 while Yakoko conversely showed a marked increase in clientele through the years. Jada, Namu and Agagbe remained fairly constant.

67 | P a g e 68 | P a g e

Load more

  68

Copy Link