World Health Organization African Programme for Onchocerciasis Control
Evaluation of the Sustainability of the Metekel Zone CDTI Project, Ethiopia May/June, 2009
Dr. Johnson Ngorok, Team Leader Dr. Margaret K. Akogun Dr. Tekola Endeshaw Mr. Abraraw Tesfaye Mr. Birhanu Melak Mr. Tamrat Belete Table of Contents
Page
List of Abbreviations…………………………………………………………………...... 3
Acknowledgements…………….……………………………….……………………..….4
Executive Summary…………..……………………….……………………………...…..5
1.0 Introduction…………………………………………………………………………….7
2.0 Methodology…………………………………….……………………………………..8 2.1 Sampling……………..…………………………………………………… 8 2.2 Protocol…………………...... ……………………..9 2.3 Performance of the Instrument………………………………………….10 2.4 Limitations…………………………………………………………………11 2.5 Team Composition………..………………………………………………12
3.0 Findings and Recommendations………………………………………………………………………...14 3.1 Zonal Level………………………………………….……………………………..14 3.2 District Level………………………………………………………….…….23 3.3 Front Line Health Facility Level………………………………………………………………………....31 3.4 Community Level……………………………………………………………….………...38
4. Conclusions………………………………………………………………………….…..44 4.1 Grading the Overall Sustainability of the Project……………………………………………………………………….44
5.0 Feedback/sustainability plan development workshop………………………….49
6.0 Advocacy visits and debriefing……………………………………………………...51
Appendices:
Appendix 1: Evaluation Time Table Appendix 2: Programme for Feedback/Sustainability Planning Workshop Appendix 3: SWOT Analysis Appendix 4: List of Respondents Appendix 5: List of Documents Reviewed Appendix 6: Sustainability Plans.
2 List of Abbreviations
APOC African Programme for Onchocerciasis Control CDD Community Directed Distributor (of ivermectin) CDTI Community Directed Treatment with ivermectin EPI Expanded Program on Immunization FLHF Front Line Health Facility HSAM Health Education/Sensitisation/Advocacy/Mobilization IEC Information, Education, Communication MOH Ministry of Health NGDO Non-Governmental Development Organization NOCP National Onchocerciasis Control Program NOTF National Onchocerciasis Task Force REMO Rapid Epidemiological Mapping of Onchocerciasis SWOT Strengths, Weaknesses, Opportunities, and Threats TCR Therapeutic Coverage Rate ToT Training of Trainers WHO World Health Organization ZOTF Zonal Onchocerciasis Task Force ZHDH Zonal Health Desk Head
3 Acknowledgements
The Evaluation Team for Metekel Zone CDTI would like to thank APOC Management for the opportunity to be of service to the people of Ethiopia. The team appreciates the logistical support and administrative arrangements that enabled the evaluation be conducted without much difficulty.
In particular, the team would like to acknowledge the following persons and institutions for their help: . The Director and the staff of the African Programme for Onchocerciasis Control (APOC) in Ouagadougou. . WHO Country Office, Ethiopia, in particular Mr Tamrat Belete for in-country logistical arrangements. . The Carter Centre Country Representative for stepping in to take up the role of the MoH which was in the midst of a restructuring exercise. . The Metekel Zone officials (Zonal Administrator, Zonal Oncho Coordinator and the Head of the Health Desk) for the valuable support provided. . The District Health Teams of Pawi and Dangur districts for the arrangements and making available the documents required for review. . The representatives of Mandura and Guba districts for participating in the feedback and sustainability planning workshop. . The Heads of the six FLHF visited for participation and mobilising the communities for the evaluation. . The community leaders, community directed distributors and community members of the twelve communities visited. . The scout, Mr. Abraraw Tesfaye, for planning the evaluation.
4 EXECUTIVE SUMMARY
Metekel zone, located in the north-western part of Ethiopia, is one of the three zones in Benshangul Gumuz People’s Regional State. The zone is administratively divided into seven woredas (districts) and 125 kebeles (village/group of villages). Of the seven districts, four i.e. Dangur, Guba, Mandura and Pawi are oncho endemic. CDTI project was initiated in Metekel in 2004 and is being implemented in the four endemic districts.
The evaluation was carried out at four levels; zonal, district, FLHF and community. Overall, the project performance was rated for each indicator per level as in the table below.
&
n p g o i t d i ® e n r s h i s e g n i g e g o t e s r e a i n r v a g a i n t p o c z r i i r r e t i a s n M n i t e v e n r g d i n i n a n c A v p t e e a a a a n t o e S c R o v l u e r r i n P L M S M T H I A T H F C A Zone 3.2 3.3 3.0 3.8 3.3 3.7 2.5 3.0 2.3 4.0 3.21 District 3.3 3.5 3.2 3.5 3.2 3.8 2.7 3.8 2.8 4.0 3.38 FLHF 2.8 3.0 2.8 3.5 3.2 3.7 2.3 3.5 2.2 4.0 3.1 Comm 3.3 2.7 3.3 3.5 2.3 - - 3.7 2.0 3.0 2.97 Overall average 3.20
Using the instruments as a “lens” for assessing the “Aspects of evaluation”, the evaluation team found out the following
Integration; There was strong integration of CDTI activities into the overall health programmes. Oncho control activities were included in both the annual operational and the five year strategic plans; transport and materials were shared across different programmes and health staff carried out support supervision in an integrated manner.
Resources; The integrated use of vehicles, equipment and other materials provide a good basis for sustainability of the programme. Although government direct funding commitment for oncho was not clearly spelt out, funds from government and other sources were made available for oncho control when activities for the different programmes were conducted in an integrated manner. The commitment of the NGDO partner to continue providing financial support reinforces the sustainability of the project
5 Efficiency; Efficiency of the use of resources was achieved by integration, conducting training sessions in cost free venues and limiting the duration of trainings. However, supervision, trainings and HSAM activities were not targeted; they were carried out routinely every year. These potentially waste limited resources.
Simplicity: Simplicity was demonstrated in several ways, for example; delivery of mectizan from FMOH to the community level using existing structures and procedures, delivery of reports by CDDs to the FLHF and onwards as part of routine activities and without using extra resources; trainings being carried out in premises owned either by the government or the community.
Attitude of staff: The health staff and the CDD expressed great interest and willingness to continue carrying out CDTI activities after APOC project phases out. The health staff considered CDTI as part of their routine duties. For the CDDs, this was despite the fact that they were not receiving financial or material support from their communities.
Community Ownership: Although the involvement of CDDs and community supervisors was commendable and promoted sustainability, the community leadership and the rest of the community members did not show active involvement and ownership of the programme.
Effectiveness: Full geographical coverage and therapeutic coverage rate of over 65% were achieved during the past three years.
Therefore on the basis of the following: Aspects; five of the aspects were helping while two were partially helping the sustainability of the project. Critical elements, All the five critical elements for sustainability were present. Quantitative; The average score is far greater than 2.5 the evaluation team concluded that Metekel Zone CDTI project, in spite of some imperfections, meets the description of being “FULLY SUSTAINABLE”
6 1.0 INTRODUCTION
Project location Metekel zone, located in the north-western part of Ethiopia, is one of the three zones in Benshangul Gumuz People’s Regional State. The zone is administratively divided into seven woredas (districts) and 125 kebeles (village/group of villages) and has a total surface area of 259,932 km land mass. The climate of the zone is typically tropical with two distinct seasons i.e. the rainy and cool season from June to October and the hot and dry season from November to May with temperatures ranging from 25 to 42 Degrees Celsius. The topography of land consists of low land (82%), semi highland (10%) and highland (10%). The economic mainstay of the inhabitants is agriculture involving both crop farming and the herding of animals.
Gilgel-Beles is the zonal capital and the seat of the zonal administration.
Onchocerciasis Control in Metekel Before the initiation of the CDTI project, oncho treatment in Metekel was clinic based. REMO survey conducted prior to initiating the CDTI project revealed that onchocerciasis was endemic in four woredas of Benshangul Gumuz People’s Regional State; Dangur, Guba, Mandura and Pawi. Apart from Pawi district which was found to be oncho hyperendemic, all the other three districts were oncho mesoendemic. CDTI project was consequently initiated in Metekel by APOC in 2004 in the four districts. Of the four, three of them (Dangur, Guba and Mandura) are located in Metekel zone while Pawi is a special woreda which also has the status of a zone. The four districts have a total of 145 villages
The health infrastructure in Metekel CDTI project area consists of 1 zonal hospital, 4 health centres, 19 health stations (18 government and one NGO) and 11 government health posts.
The Carter Centre is the NGDO partner.
7 2.0 METHODOLOGY
2.1 Sampling Sampling was carried out by the scout using primary and secondary criteria in line with APOC Guidelines. Primary criteria considered the geographical and therapeutic coverage rates of the study areas. In this project, the onchocerciasis-endemic districts, the FLHFs as well as the communities had all achieved a geographical coverage of 100%. Therefore, sampling considered therapeutic coverage rates and the level of onchocerciasis endemicity which varied across districts, FLHFs and communities. The secondary criteria considered accessibility and the convenience of working in the potential study areas.
Consequently, two districts i.e. Pawi and Dangur were sampled for the study. Pawi was hyper-endemic while Dangur was meso-endemic. To complete the study samples, three health posts and two communities under each of the health post were sampled.
Sampling of Districts (Woredas) The four districts in the zone i.e. Dangur, Pawi, Guba and Mandura were considered for sampling. Two of them i.e. Guba and Mandura were excluded on the basis of the secondary criteria. Guba district is located 150 kms away from the zonal capital and 188 kms from the headquarters of the evaluation team which made it rather inaccessible considering the time available for the evaluation. On the other hand, inter-group clashes had been reported in Mandura district which potentially posed security risks. Consquently, the remaining two districts i.e. Dangur and Pawi were sampled.
Selected District Therapeutic Coverage Rate Dangur 69% Pawi 74%
Sampling of Front Line Health Facilities (FLHFs) The FLHFs were sampled using the primary and secondary criteria. Instead of the recommended two FLHF per district, three FLHF were selected for each of the two districts in order to make up for the required six districts. After excluding some FLHF on the basis of accessibility, three FLHFs were randomly sampled for each of the two districts as shown in the table below.
8 District Selected FLHF Therapeutic Coverage Rate Dangur Manbuk Health Centre 70% Gublak Health Post 77.7% Burj Health Post 51% Pawi Felege-selam Health Centre 75% Ketena 2, Village 131 Health Post 78.3% Ketena 1, Village 7 Health Post 73%
Sampling of communities Lastly, two communities from the catchment areas of each of the six FLHFs were randomly selected after excluding the remotely located and inaccessible ones. A total of twelve communities were therefore sampled as detailed in the table below.
FLHF Community Therapeutic coverage rate Manbuk Health Centre Ketena 1 65% Ketena 3 79% Gublak Health Post Gublak Village 79% Kola Village 72.3% Burj Health Post Burj Village 46.1% Asama Wuha Village 35% Felege-selam Health Centre Felege-selam Town 76.3% Medin Village 57% Ketena 2, Village 131 Health Post Village 131 84% Village 29 74.2% Ketena 1, Village 7 Health Post Village 6 76.4% Village 7 71%
2.2 Protocol
. Research Question: How sustainable is the Metekel CDTI Project? . Design: Cross-sectional, descriptive. . Population: The Metekel CDTI project, its NGDO partner (Global 2000/The Carter Center), the staff involved in onchocerciasis control at woreda and FLHF levels, the project communities, with their leaders and CDDs. . Instrument: * A record sheet, structured as a series of indicators of sustainability. The indicators were grouped into nine/ten categories/ groups. These groups represent critical areas of functioning of the program. * The instrument assesses sustainability at four levels of operation.
9 * The instrument guides the researcher to collect relevant information about each indicator from a variety of relevant sources. . Sources of information: ∗ Documentary evidence and observations. ∗ Verbal reports from persons interviewed. . Analysis: * Data from all sources was aggregated, according to level and indicator. * A qualitative summary of the situation regarding each indicator at each level was made. This was aggregated and summarized for each category of indicator for each level. * Based on the information collected, each indicator was graded on a scale of 0-4 in terms of its contribution to sustainability. * The average ‘sustainability score' for each group of indicators was calculated, for each level. * Finally an overall assessment of sustainability was made, by considering the 7 aspects and 5 critical areas of sustainability. . Recommendations: ∗ These were strictly based on the findings of each program evaluated.
2.3 Performance of the instrument
Generally, the four questionnaires are well-designed and appropriate instruments for evaluating the sustainability of CDTI programme at the different levels.
However, the team noted the following possible areas of improvement: • There are repetitions of information required for the indicators within the same instrument; for instance, integration in planning, support activities as well as monitoring and supervision. Although this is necessary, it was sometimes irritating to respondents when they were asked the same question several times. Interviewers should preferably avoid repeating questions if information for this had been collected earlier. • The guidelines for conducting the feedback and sustainability planning workshop are not clear and comprehensive. The template needs to be updated; for instance, there are no columns for the sources of funding.
10 2.4 Limitations
During the course of the evaluation exercise, the evaluation team encountered the following limitations: • The main medium of exchange in Ethiopia is Amharic. The external team members had problems in communicating directly with respondents whose abilities to speak in English was limited. This constrained and prolonged the process of data collection. On the other hand, most of the documents were also written in Amharic which made the verification process slow. • The erratic electricity supply slowed down data collation and entry. On some days, the team changed location and once had to travel to another town located one and half hours away. Time spent travelling was lost and the team had to adjust its working schedule to take advantage of the availability of electricity. • Bad roads and distances between communities meant longer working hours and more days for data collection. The stress and fatigue, as a result, sometimes impacted on the input and output of team members. • The coincidence of the evaluation exercise with the bazaar/trade fair in Pawi district, a social occasion which comes up once every two years, limited the number of community members and district officials the team was able to meet as most of them went for the trade fair. • The visit of a renowned political leader to Dangur on the day the evaluation team was visiting the district made it impossible for the advocacy visit with the Dangur District Administrator to take place. In addition, some key district officials were unavoidably absent from their offices. • Due to the need for security and other briefings in Addis-Ababa prior to departure for the projects evaluation sites, the team arrived to the project area two days later than planned. The programme and appointments had to consequently be rescheduled.
11 2.5 Team composition
Dr Johnson Ngorok Sightsavers International Nairobi, Kenya E-mail: [email protected] Tel: +254 722 567897
Dr Margaret K. Akogun Department of Psychiatry Jos University Teaching Hospital Jos, Nigeria Email: [email protected] Phone: +234 8039695047
Dr Tekola Endeshaw The Carter Centre Addis-Ababa, Ethiopia Email: [email protected] Tel: +251-1-91 1172856
Abraraw Tesfaye Organization for Social Science Research in Eastern and Southern Africa (OSSREA) Addiis-Ababa, Ethiopia E-mail: [email protected] Tel: +251-91 1315196
Berhanu Melak The Carter Centre, Bahir-Dar Office, Ethiopia E-mail: [email protected] Tel: 058-220-55149(office); 09-18-77-0764(mobile)
Mr Tamrat Belete
12 African Programme for Onchocerciasis Control World Health Organization Addis-Ababa, Ethiopia E-mail: [email protected] Tel: +251-91 1731248
13 3.0 FINDINGS AND RECOMMENDATIONS
3.1 Findings at the Zonal Level
Metekel Project: Sustainability at Zonal Level
4.5 4 4 3.7 3.8 3.3 3.3 ) 3.5 3.2 4
/ 3 3 (
t 3 h 2.5 g i 2.3
e 2.5 w
e 2 g a r
e 1.5 v
A 1 0.5 0
g n ip n n M e ls s e in io h io a c a e g n t s s iz A n ri c a a r i t S a e r r n r e rv c H in t u e la g d e e a o v P te a M g F s o n e p in M e C I L u n rt R S i o a p n g r s a in T n m r a u ito r n T H o M Indicators
Planning (3.2: Highly) There was a Five Year Strategic Plan, 2006-2010, and yearly Operational Plans. Both plans contained CDTI activities under malaria and other vector borne disease control programme. In addition, there was a detailed CDTI Activity Plan (in Gantt chart) which included all the key CDTI activities. The yearly plans did not vary from year to year and were not targeted to specific needs of each year.
There was a Zonal Oncho Task Force which comprised the Head of Zonal Health Office, the Head of Zonal Drug Store, The Carter Center, the Zonal Oncho Control Coordinator and representatives of the zonal administration. The task force did not have structured and regular meetings but the team was able to see the minutes of al least one of the meetings that had been held. All partners were clear about their roles; both APOC and
14 The Carter Center provided financial and technical support while the government was responsible for the overall implementation of the programme.
Although there had been discussion on sustainability during annual planning meetings, no post APOC sustainability plan had been developed. However, the members of the zonal health team were aware that APOC was reducing its funding and that government was required to take over CDTI activities at the end of the fifth year of the project.
Recommendation: Planning Implementation
Priority: High 1. The annual CDTI plan should target specific needs and problems. Indicators of success: i) Annual CDTI plans are based on 2. Zonal oncho task force ought to identified needs and problems. hold at least two meetings, before ii) Semi-annual ZOTF meeting minutes. and after drug distribution. Responsible person: ZOTF members
Time frame:
For No. (1) May-June Annually For No. (2) December and May Annually
Integration (3.7: Highly) Oncho was planned and managed under the malaria and other vector borne diseases control programme. There was strong integration of CDTI activities with other health programmes. The zonal oncho coordinator combined a number of CDTI activities during supervisory visits to the districts. During their supervisory visits, the health staff of the various programmes carried check lists, including the CDTI check list and supervised several health programmes.
The annual health plans integrated CDTI activities. However, there were no trip reports showing the actual integration of activities during supervisory visits.
15 Recommendation: Integration Implementation
Priority: Medium Trip reports on integrated supervisory visits should be written and filed. Indicators of success:
Written supervisory trip reports showing integrated activities.
Responsible person: ZOC, other members of zonal health team
Time frame:
One week following completion of supervisory visit.
Leadership (3.3: Highly) There is a focal person for oncho control programme who is also responsible for coordinating the activities of malaria and other vector borne diseases control programme. It was evident that the Zonal Oncho Coordinator and the Head of the Health Desk were committed to CDTI and had taken full ownership of the programme. The Zonal Oncho Task Force, which also included other zonal sector leaders and is chaired by the Zonal Administrator, ensured that the zonal leadership was involved in overseeing CDTI activities. The members of the zonal task force interviewed were fully aware of the successes achieved and problems encountered during CDTI implementation and discussed these in detail with the evaluation team. Responsibilities for CDTI had been properly delegated to the district level, an observation that was further confirmed by district health team members.
Recommendation: Leadership Implementation
Priority: Medium Strengthen the involvement and Indicators of success: commitment of the zonal leadership to continue supporting CDTI Programme Minutes of ZOTF show the involvement of zonal leaders.
Budget allocation for CDTI
Responsible person: ZOC, other members of zonal health team
Time frame: Dec and May Annually
16 Monitoring and Supervision (3.0: Highly) Supervision was integrated and the staff at the zonal level supervised those at the district level although they too occasionally supervised the FLHFs and the communities, especially so in cases where low coverage rates had been experienced. The district health team members were empowered to carry out CDTI activities, even though there was only one health staff assigned for oncho control and other vector borne diseases at the zonal level. This was mainly a result of TOTs conducted in collaboration with the NGDO partner and the supportive supervisory activities carried out by the zonal health team.
Records including treatment summary sheets, inventory of equipment and financial records were filed. However, the records were sometimes hand written which compromised quality. The annual technical report, which was prepared by the NGDO partner, was not available at the zonal level. The technical report is of good quality. Supervision was integrated with other health programmes and was conducted on a quarterly basis. CDTI check list was used but supervision, which was intensified during drug distribution, was not targeted to specific needs. There were no reports of supervisory visits and it was therefore rather difficult to assess problem solving during supervisory visits and follow up action on recommendations from earlier visits. Efficiency in the use of resources for supervision was achieved mainly through integration.
Recommendation: Monitoring and Implementation Supervision 1. Supervision should target specific Priority: High needs and problems identified mainly at the district level. Indicators of success: i) Reports on supervisory visits show 2. Zonal health team’s supervision to evidence of targeting needs and the FLHF and community levels problems should take place only for “spot ii) Good quality of project records. checks”. Responsible person: Zonal Health Team 3. Project records should be typed members and filed properly Time frame: 4. Reports on supervisory visits, recommendations and follow up i) One week after each supervisory visit. action should be written and filed. ii) Ongoing
17 Mectizan (3.8: Highly) Mectizan requirement was determined at zonal level based on updated census data from the previous year. A pharmacist from the zonal health desk collected mectizan® from the FMoH using resources availed by the government and stored in the zonal drug store before collection by the district teams.
Although there was sufficient supply of mectizan® in the zone, there were incidents of temporary shortages in some districts which arose due to misallocation of the drug between districts. This was however managed promptly.
The communities visited had decided on the timing for drug distribution to be from January to March based on the migratory pattern of some members of the communities, the farming season and the timing of major festivals such as Easter. However, the arrival and distribution of the drug took place rather late and extended into May and June in some communities. This was attributed to unavailability of the drug at FMOH stores at the required time.
Overall, the procurement and distribution of mectizan is controlled within government drugs and supplies management system.
Recommendation: Mectizan Implementation
Priority: High 1. Due attention should be given to the allocation and delivery of Indicators of success: mectizan to the districts based on i) No reports of drug shortages in the updated and reliable census data. districts. ii) Drug distribution taking place at the 2. The delay in the delivery of time decided by the communities. mectizan ought to be brought to the attention of FMOH. Responsible person: ZOC, FMOH, NGDO, APOC.
Time frame:
For No. (1) December Annually For No. (2) January to March Annually
18 Training and HSAM (3.3: Highly) The zonal oncho coordinator was trained as a trainer at the national level. In turn, the zonal oncho coordinator, with the support of The Carter Center, conducted training for district health teams annually. The evaluation team observed that the district health team members were sufficiently empowered. The zonal health team was not involved in training staff at the levels below the district. However, training needs assessment to identify knowledge and skill gaps was not carried out and therefore training was not targeted. The trainings combined previously trained and newly recruited staff members without an initial basic training for the latter. In order to contain costs, the zonal coordinator traveled to districts for the training which took place within the premises of the districts headquarters and lasted for not more than one day.
The members of the zonal oncho task force were sensitized on the programme but this was carried out routinely without analyzing knowledge gaps. The zonal leaders were aware of CDTI programme and were supportive to the health team.
Recommendation: Training and HSAM Implementation
Priority: High Target training and HSAM activities based on knowledge and skill gaps. Indicators of success: Targeted training and HSAM carried out.
Responsible person: Zonal Health Team, ZOTF
Time frame:
As need arises.
Finance (2.3: Moderately) In the five year strategic plan, CDTI activities were included and budgeted for. However, the one year operational plan did not specify the budget for oncho. Instead, there was a budget allocated for malaria and other vector borne diseases which could be used for CDTI activities in line with the practice of integration. The zonal coordinator had estimates of funds pledged by partners for the current year: WHO-APOC, The Carter Center and the Government, but there was no projection of the estimated income for the following years.
19 Budgeting is carried out at the zonal cabinet level which includes the Head of the Health Desk. The budgetary contributions of the government, the NGDO and WHO-APOC were clearly spelt out for 2009 and the previous years. According to the Zonal coordinator, government funding had been increasing but we could not verify this assertion. Because of the integrated nature of budgeting, we could neither isolate government disbursement for oncho activities nor verify whether the disbursement for oncho was increasing or not.
Project management was aware of budgetary deficit but this had not been quantified. The deficit was however managed using integrated funds. The Carter Centre confirmed, through telephone interview with the Country Representative, that they would continue supporting the programme for the next three years, but there was no written agreement to confirm this commitment.
The control system for expenditure against budget was adequate; requisition was made against budgeted activities, approval was obtained and residual balances were calculated by finance department and reports availed to the zonal oncho coordinator.
Recommendation: Finance Implementation
Priority: High 1. Within integrated budgeting’ funds committed for oncho should be Indicators of success: clearly spelt out. i) Oncho specific budget and 2. Expenditure analysis ought to show expenditures shown. clearly the expenditure on oncho ii) Clearly indicated budget deficit with activities. mechanism of bridging the gap. 3. Budgetary deficit should be iii) A written agreement with NGDO analysed and quantified; possible partner. sources of bridging the gaps indicated. Responsible person: ZOTF, NGDO, 4. A written commitment of support WHO/APOC from The Carter Centre should be obtained. Time frame: By July Annually
Transport and Other Materials (2.5: Moderately) There is one Toyota 4 WD pick up provided by APOC which, although quite old, was still functional. Additionally, there was a Toyota Land Cruizer for the Zonal Health Desk
20 provided by government which is available for use by all health programmes including CDTI activities. The following equipment were also available: desk top computer, printer, photocopier, fax machine, LCD projector, television, generator, radio communication device and training/HSAM materials. Apart from the photocopier, all others were still functional though old and needed replacement.
Maintenance of vehicles and equipment was integrated but there was no scheduled routine maintenance. Travel authorization was routinely obtained but the log books were not filled.
Management was aware of the need for replacement of vehicles but this had not been planned. The head of the zonal health desk indicated that there is a possibility of replacing equipment but not vehicles; as there is integration, vehicles from other programmes can be used for CDTI activities.
Recommendation: Transport and Implementation materials Priority: Medium 1. APOC to replace the current vehicles and equipment. Indicators of success: i) Vehicles and equipment replaced. 2. Include replacement of equipment ii) The strategic plan includes future in the next strategic plan, 2011- replacement of equipment. 2015. iii) Transport availability for oncho programme. 3. Ensure availability of reliable means of transport for oncho Responsible person: ZOTF, programme. WHO/APOC, FMOH
Time frame:
APOC: December 2009. ZOTF/FMOH: June 2015
Human Resources (3.0: Highly) The zonal oncho coordinator was the only staff at this level who was actively involved in CDTI activities. The coordinator was knowledgeable and skilled in the key areas of CDTI programme but seemed to have a heavy work load. We noted the strong commitment of the coordinator which was also confirmed by the district health teams.
21 However, the coordinator would need capacity building in computer skills and data management.
The coordinator had been in post for over five years and there had been no new health staff for the programme during this period.
Recommendation: Finance Implementation
Priority: Medium 1. Training and active involvement of other members of zonal team in CDTI Indicators of success: programme. i) Active involvement of other members of the zonal health team in CDTI. 2. Training the zonal oncho coordinator ii) Improved computer and data in computer skills and data management skills. management. Responsible person: ZOTF, NGDO
Time frame: By December 2009
Coverage (4.0: Fully) The GCR was 100% in 2007 and 2008. In 2006 the GCR was 96%; this was attributed to the intergroup conflicts which hindered treatment in some of the communities in one of the four oncho endemic districts (Mandura district).
The TCRs achieved during the preceding three years were: 66.9% (2006), 71% (2007) and 70.5% (2008) giving an average TCR of 69.5% for the three years
Recommendation: Finance Implementation
Priority: Medium Maintain the TCR coverage and aim to achieve even a higher TCR. Indicators of success:
Increased TCR.
Responsible person: ZOTF
Time frame: By May 2010
22 3.2 Findings at the District Level
Sustainability at District Level
4.5 4 4 3.8 3.8 3.5 3.5
) 3.5 3.3 3.2 3.2 4 / (
t 3 2.8 2.7 h g i
e 2.5 W
e 2 g a
r 1.5 e v
A 1 0.5 0
n e g n ip n a n c ls s e in io h io z io e g n t s s ti t n ia c ra n ra r i c a a r r e a g e rv e iz in te u v l e d e il F a o o P t a p M b s In e o m e C L u t R S M r g g po n n n s a ri i n o in m it a ra u n r T H o T M Indicators
Planning (3.3)-Highly There was a five year strategic plan from 2006 up to 2010 for overall health programmes. Onchocerciasis control was included in the five year plan with a specific line funded by APOC. There were yearly health plans which included malaria and other vector borne diseases. Onchocerciasis control programme was part of the yearly plan. There were also detailed annual activity plans on Gantt Charts posted on the wall and a comprehensive plan showing 15 CDTI activities. The plans were drawn by the District Health Team in consultation with the zonal onchocerciasis control programme coordinator.
Although yearly plans considered the community migratory patterns (migration due to search for seasonal work in other areas and pastoral community), availability of mectizan® at the required time remained a constraint even after the districts had informed the zonal health desk on the appropriate distribution time.
23 Integration (3.8)-Highly There was an integrated annual operational health plan which included CDTI activities and other vector borne diseases such as Malaria and other health programmes like EPI. CDTI activities such as monitoring, supervision, HSAM were combined during trips. Reports also indicated that CDTI activities were combined with those of malaria control, EPI and health extension services. However, the evaluation team did not see any trip reports to confirm this.
Recommendations: Integration Implementation 1. Documentation should be made of Priority: HIGH every activity carried out. Indicators of Success: 2. Trip reports should be documented • Documentation of all reports Who to take action: District staff
Deadline for completion: Continuous
Leadership (3.5)-Highly There were focal persons for CDTI activities in all district health offices who were also responsible for control of malaria and other vector borne diseases. All district health teams visited, did initiate key CDTI activities such as planning, monitoring, supervision and training in consultation with the zonal onchocerciasis coordinator. The district health offices remarked that they were well aware of the temporal support of APOC and they would find ways of continuing with the CDTI activities after APOC withdraws.
Monitoring and Supervision (3.2)-Highly In most cases, the district health teams routinely supervised the FLHF staff on onchocerciasis activities, while in some cases; supervision was reported to be targeted in certain situations. It was particularly noted that communities that were near the district headquarters, the district health team worked alongside the FLHF staff to supervise drug distribution. This was seen as necessary for spot checking and problem solving. In situations where the communities were remotely located and difficult to reach, the FLHF staff were mainly involved in spot checking.
There was onsite management of problems when they arose. For example, shortage of mectizan® in some communities was tackled by transferring from other communities
24 which had excess mectizan®. Shortages were experienced due to migratory patterns. In such cases, reports were made to the districts who further contacted the zonal level for additional supplies.
This difference in the style of supervision can be explained by distances and accessibility of the communities.
The district health teams, with the involvement of the FLHF staff, participated in solving community problems. This included discussion of problems with CDDs, household heads, community leaders and community supervisors. The district teams worked jointly with the FLHF staff in the implementation of CDTI activities at the community level.
A system of publicly recognizing and rewarding the best performing health workers was reported at the district level. This applied generally for all health workers who successfully carried out CDTI activities.
No evidence was found regarding follow up actions to implement recommendations of the previous monitoring visits. What was clear though was that problems were solved on the spot during supervisory visits and completed checklists were filed as a reference.
The channels of reporting were found to be within government system. It was reported that all health activities were sent on quarterly basis. However, CDTI reports were sent to the zonal health desk and onwards to the national level immediately after completion of Mectizan distribution.
Recommendations: Monitoring and Implementation Supervision 1. Supervisory activities should be Priority: HIGH targeted and based on identified needs Indicators of Success: 2. Feedbacks should be constantly given • Practice of targeted supervision to the lower level • Evidence of written feedback Who to take action: District staff Deadline for completion: Continuous
25 Mectizan (3.5)-Highly Mectizan® ordering, storage and delivery was controlled within the government system. The ordering system involved mectizan® collection from the FMoH and delivery to the zonal health desk for onward distribution to the districts and FLHFs. In general, the system appeared simple, effective and efficient. The mectizan® order is based on previous years’ census update and quantification parameter. Nevertheless, the evaluation team did not find any mectizan® order form at the district level.
In most cases, allocated mectizan® did not reach the districts drug store for onward delivery to the communities at the time requested. Most communities’ preferred distribution to take place from January to March. However, mectizan® distribution had been taking place between March to May. Problems of mectizan® misallocation were reported. This resulted in shortages of mectizan® in some districts during distributions. In such cases, the zonal onchocerciasis coordinator was contacted who promptly delivered mectizan® retrieved from other districts which had received excess. The problem of shortage was therefore internal to the zone.
Recommendations: Mectizan Ordering Implementation and Supply 1. Standard Mectizan order forms should Priority: HIGH be in place Indicators of Success: • Order forms in place 2. Mectizan should arrive on time at the • Mectizan being distributed on time community level • Absence of shortages Who to take action: District staff 3. Accurate allocation of Mectizan should Deadline for completion: During be carried out planning and distribution
Training and HSAM (3.2)-Highly Trainings were carried out routinely for both new and previously trained personnel i.e. they were not targeted. The district health teams conducted training for FLHF staff in a central location within the district headquarters. FLHF staff in turn conducted training for CDDs and community supervisors in batches when they came to collect drugs.
26 The evaluation team was able to see lists of participants and letters of invitation for FLHF staff to report for training. The trainings were carried out with the assistance of the zonal onchocerciasis coordinator in collaboration with The Carter Centre.
Resources for training were used efficiently; training lasted for one day and participants returned on the same day; training sites in the district headquarters were used to save on rental costs. However, the number of participants was not regulated which seemed inefficient. HSAM activities were also reported to be carried out routinely for district leadership on annual basis.
Political leadership was involved in social mobilization in community gatherings such as religious, schools and political rallies etc. In Pawi district there were reports of very high turnover of political leaders which necessitated yearly social mobilization.
HSAM activities were planned as evidenced by annual health plans including onchocerciasis activities but no schedule of HSAM activities made.
HSAM was mainly carried out at the district level but extended to lower level in cases where serious problems, such as lower coverage or refusals, were identified but this was done in collaboration with FLHF staff.
There was evidence that people at different levels were cooperative. Community’s demand for Mectizan® was found to be high and there was increased coverage. District leaders were also involved in health education and social mobilization. We did see a letter from the district health office requesting the district administrator to write to the Kebele (community/group of communities) administration to support CDTI activities.
Recommendations: Training and Implementation HSAM 1. Training and HSAM should be targeted Priority: MEDIUM and based on needs Indicators of Success: • Targeted evidence of training and 2. Detailed HSAM schedule should be HSAM activities readily available Who to take action: District staff Deadline for completion: During planning and preparation phase
27 Finance (2.8)-Moderately There was a specific budget for CDTI activities funded by the three partners i.e. APOC, TCC and the Government (evidence was presented to the evaluation team) for the period 2006 – 2009; but this did not spell out the detailed specific activities. In addition, in their annual health plan, there was a budget for malaria and other vector borne diseases control programme which was sometimes used for CDTI activities as well. It was however difficult to separate out the specific budget for CDTI activities because of the integrated nature of planning and budgeting. Besides, the staff at this level did not seem to have a clear estimate of expected funds for the coming year and were not therefore actively using the available information for planning. The main cost containment strategy at this level was integration of various health programmes and conducting supervision.
The government contribution in the CDTI specific budget showed an increase although there was a decline in 2008. It was however difficult to assess whether government contribution for CDTI activities in the overall health budget was increasing because of the integrated nature of budgeted activities.
In some districts, disbursement information was obtained and did show that funds disbursed by government were increasing each year. However, in one of the districts, the evaluation team could not verify the amount of funds disbursed for CDTI activities and also whether or not the contributions were increasing yearly.
The process of fund disbursement follows the following procedures: • Initiation of payment request memos from any one of the health staff for authorization as per the action plan • Endorsement of the request by the Head of Health Office and finally • Forwarding the request for fund disbursement to the Head of Finance Department in the pool system Evidence of such a request was seen.
There was regular calculation of residual balances. District Finance Office prepared reports on year to date expenditure against budget and presented this to the health office. The evaluation team examined this report.
28 Recommendations: Finance Implementation 1. There should be clear estimate of Priority: HIGH expected income at the district level Indicators of Success: • Clearly estimated income in place 2. Estimated income should be based on • Funding gaps are quantified quantified gaps based on available sources • Funds from other sources are 3. District management team should clearly identified actively look for other sources of funding Who to take action: District staff Deadline for completion: Continuous
Transport and Other Material (2.7)-Moderately All APOC donated motorbikes were not functioning. However, there were several other motorcycles and vehicles from other health programmes which were available to be used for CDTI activities. There were adequate training materials which were printed with support from several partners; FMoH, The Carter Center, WHO/APOC and Lions Club International.
Maintenance of vehicles was carried out with government funds. However, constraints arising due to inadequate government funds were sometimes experienced. Request of funds for vehicle maintenance is made by the health team through the finance office. Whenever there were serious transportation problems, staff members used public transport and sometimes walked on foot to carry out CDTI activities.
It was reported that a system of authorization for use of vehicles was in place. However, the approval was sometimes not formal. In the use of motorbikes, verbal approvals were practiced. Log books were not filled.
The team observed that the vehicles were old and maintenance was becoming costly. Even though the management at the districts level was aware of the need to replace the motorbikes and other vehicles, there were no specific plans from the government. During advocacy visits, the district administrators pledged to include provision for replacement of the motorbikes in the next strategic plans after consulting with the regional administration. The immediate solution was however seen in the integrated use of available transport.
29 Recommendations: Transport and Implementation Other Materials 1. There should be plans for replacement Priority: HIGH and repair of vehicles Indicators of Success: • Availability of written commitment 2. Log books for movement of vehicles for replacement of vehicles should be properly filled • Availability of maintenance schedule for transports • Properly filled vehicle movement log books Who to take action: District staff Deadline for completion: Continuous
Human Resources (3.8)-Highly There is evidence of staff stability. Most of the staff in the district had been in post for periods ranging from 4 - 7 years and had undergone at least one form of in-service training on CDTI activities and other health programmes.
The health staff interviewed at the districts expressed satisfaction with their work; they were highly motivated and expressed great commitment to continue with CDTI activities. In the previous year, all health personnel were awarded a certificate of appreciation and five of them were in addition received special awards. Rewards varied across districts and included; certificates, monetary rewards, public recognition etc. This was a big motivation for health staff.
Salaries were reportedly paid regularly but per diems were paid when funds were available. Additionally, the alleviation of the suffering of the communities they serve was reported as a source of motivation.
Coverage (4.0)-Fully Geographic coverage rates of 100% were achieved in each of the three preceding years (2006 – 2008) and therapeutic coverage rates achieved were all above the 65% minimum recommended by APOC.
Recommendations: Coverage Implementation 1. Staff should be encouraged to keep up Priority: LOW with high coverage level Indicators of Success: High coverage Who to take action: District staff Deadline for completion: Continuous
30 3.3 Findings at the FLHF Level
Sustainability at FLHF
4.5 4 4 3.7 3.5 3.5
) 3.5 3.2 4 /
( 3
t 3 2.8 2.8 h g i 2.3 e 2.5 2.2 W
e 2 g a
r 1.5 e h v
A 1 0.5 0
g n p n n n e s s e n o i o a o c l e g i ti h i z ti n ia c a n a rs is ti a a r r r n r e v c z n te u e la g d r e ili i a o v P te a e b F s o n e p M o m e C I L u rt R S M o n g g p a n n s ri i n m o in a u it a r H n r T o T M Indicators
Planning (2.8)-Moderately There were no specific written CDTI plans in all the six FLHFs. However, there was a written annual plan for all health programmes in five of the six FLHFs visited and CDTI activities were “presumably” integrated in one of them. Note of mention is the fact that plan of CDTI activities were normally drawn at the district level and not at the FLHF but distributed to the FLHF to fit in their schedules.
Recommendation: Planning Implementation FLHF staff should be trained and Priority: High empower to make their own plan of Indicators of success: activity CDTI plan in place at FLHF level
Who to take action: District and FLHF staff
Deadline for completion: At the beginning of each fiscal year.
31 Integration (3.7)-Highly In all the FLHFs visited, the staff combined several CDTI activities in a single trip. Similarly, CDTI activities were also combined with activities of other health programmes such as malaria, EPI, TB control and MCH. However, for communities which are far away, the health extension workers were reported to combine CDTI activities with 17 health packages as part of integrated programme activities in the communities. In some instances, CDTI activities were carried out separately for communities close to health centres. There was no written evidence of integrated activities for verification.
Recommendations: Integration Implementation 1. FLHF staff should be encouraged to Priority: HIGH maintain integration of activities. Indicators of Success: • Integrated activities conducted 2. Written evidence of integration should • Written evidence of integration in be in place and FLHF staff have to be place trained Who to take action: Both District and FLHF staff Deadline for completion: Continuous
Leadership (3.0)-Highly For the majority of the FLHFs visited, there was strong ownership and the health staff saw the CDTI programme as their own. However, in one health facility, there was uncertainty of the roles of FLHF staff and considered the CDTI activities are mainly the responsibility of the district health office and he saw the FLHF staff as playing a supportive role.
The local leaders were involved in carrying out some form of social mobilization in conjunction with the FLHF staff in support of CDTI and other health programmes in the communities.
Monitoring and Supervision (2.8)-Moderately The FLHF staff integrated several supervisory activities with other health programmes such as HIV/AIDS, malaria control, TB etc during community visits. This was especially so with the health extension workers whose routine work involved frequent interaction with the community members. However, there were no check lists for verification. Supervision was reported to be carried out routinely by the FLHF staff most of the time and intensively at the time of drug distribution in all communities because some of the
32 CDDs and supervisors were not educated enough to perform and record adequately. Targeted supervision was reported only in a few cases when there were problems especially with remote communities and when there was scarcity of vehicles.
Some of the problems identified were dealt with by the FLHF staff on the spot by providing information whenever possible. Examples of such problems included registers not being up to date, duplication of registration and refusal of treatment. There were no reports of other community problems which the FLHF staff passed on to the community leaders for them to deal with. It appeared that the FLHF staff attempted to solve all the problems themselves. Problems and recommendation for further follow up were not documented and therefore verification was rather difficult by evaluation team
Data from community registers and APOC reporting forms indicted the name of the household members, the amount of drugs family members received, therapeutic coverage, number of community supervisors and CDDs trained. Completed register books were sometimes submitted by the CDDs through the community supervisors after drug distribution. In some cases, the CDDs submitted directly to FLHF staff. Community supervisors where sometimes reported to fill the summary forms and submitted them to the health facility staff in other instances the summary reports were filled by the FLHF staff who in turn further compiled for their catchments areas and submitted to the district office. The reporting is carried out within the health system in an integrated manner.
The FLHF acknowledged the important services provided by the CDDs to their communities, but expressed disappointment over the fact that CDDs services were acknowledged only by encouragement by community members. Despite the situation the CDDs were willing to continue serving the communities.
Recommendations: Monitoring and Implementation Supervision 1. FLHF staff should be encouraged to Priority: HIGH use and document checklists of activities Indicators of Success: 2. Supervision should be targeted • Documentation of activities 3. FLHF should empower community • Targeted supervision leaders to solve community problems • Empowered community CDTI related activities • CDDs recognized 4. A system of recognition of CDDs Who to take action: FLHF staff should be in place Deadline for completion: Continuous
33 Mectizan® (3.5)-Highly Reports show that FLHF staff were not used to estimating Mectizan® requirement for their communities. Therefore there were no order forms seen at this level. Mectizan ordering was made by the district health team based on community updated census reports of the previous year.
Reports also showed that in the past two years, Mectizan was not made available at the time requested by the communities i.e. between December to March (the time after harvest when most people are around). Instead, the drug was received in March/April well into the farming season and when some communities migrated in search of work or pasture. This was true especially for communities settled in Pawi district.
Temporal shortages of mectizan® was reported in the six FLHF visited in the 2008 distribution due to misallocation of mectizan® to districts. This problem was reported to the district health team and zonal level and additional drugs were promptly mobilized from other districts.
The method of mectizan® delivery was controlled within the Government system and appeared simple and effective. The system involved the FLHF staff collecting the drug from the district using any available means of transportation and delivering it to the CDDs directly or through the community supervisors or health extension workers.
Recommendations: Mectizan Implementation 1. FLHF staff should be involved in Priority: MEDIUM estimation of Mectizan supply Indicators of Success: • FLHF staff involvement Mectizan 2. Timely supply of Mectizan as agreed supply estimation by the community • Timely arrival of Mectizan to the community according to 3. Accurate allocation of Mectizan to community request communities based on communities • No shortage at community level requisition Who to take action: FLHF staff Deadline for completion: During planning and distribution time
Training and HSAM (3.2)-Highly Training of CDDs and community supervisors takes place routinely every year before drug distribution and is not targeted to identified knowledge or skills gaps.
34 Efficiency in training was observed. Training sites were within the community, the training lasts less than half a day and did not have a monetary cost to the programme. The evaluation team observed that there were adequate training materials.
HSAM activities were carried out routinely at the health facility during community meetings, in schools and during community visits by health extension workers. There was no indication that HSAM activities being carried were addressing information gaps or based on objective identification of needs among decision makers.
Effectiveness of HSAM activities was evidenced by the increasing demand for mectizan® and improvement in the coverage rate. No training records or checklist of activities were available for verification.
Community leaders were not actively involved in HSAM activities.
Recommendations: Training and Implementation HSAM 1. Training and HSAM activities should be Priority: HIGH targeted according to identified Indicators of Success: knowledge gaps • Training and HSAM activities targeted and recorded 2. Training and HSAM activities should be • Community leaders involve in recorded and properly documented HSAM Who to take action: FLHF staff 3. FLHF should encourage community Deadline for completion: Continuous leaders involvement in HSAM
Finance (2.2)-Moderately There was a one year health budget for all health activities at the health centres to cover operational cost but none for the health posts visited; this was because there was no system of budget allocation at this level. However, the health centres budget did not specify CDTI activities. As there was no specific budget for CDTI activities in the health post, there was equally no disbursement of funds. The cost of carrying out CDTI activities was minimal and was absorbed in other programmes. Sometimes the use of personal funds was reported by the FLHF staff while general funding was being awaited.
35 CDTI activities were integrated and carried out as part of overall health programmes by the FLHF staff. For example, CDD training lasted for half a day and did not have any monetary cost; supervision was carried with no per diem payment. However, when performing other programme activities e.g. Malaria control, HIV/AIDS control and EPI in which CDTI activities were integrated, per diem was paid.
Recommendations: Finance Implementation 1. FLHF should prepare their own annual Priority: MEDIUM plans for specific CDTI activities and have Indicators of Success: clear estimates of CDTI activities. • Availability of budgeted activities Who to take action: FLHF staff Deadline for completion: Every year
Transport and Other Material 2.3)-Moderately The motorbike provided by APOC for CDTI activities was deployed at the health centers but at the moment not functional. There was also one motorcar and four other motorbikes provided by government for other health programmes in the six health facilities. These were also used for CDTI activities. The transports available were partially maintained with the Government sources, but no plan of replacement was yet in place by the Government. Motorcycles were maintained within the district vehicle maintenance system. When vehicles broke down, the FLHF staff requested the district finance department to facilitate maintenance through the district health office which approves the request. Costs of maintenance were met from funds provided by government and other programmes.
Training and HSAM materials were seen and reported to be enough.
Any available means of transport, including walking on foot, were used as coping mechanisms whenever motorbikes were broken down.
The vehicles in the health facilities were pooled, used in an integrated way between programmes and managed under the control of the Heads of the health facilities. In most cases, trip authorizations were provided verbally and there were no log books at all. No maintenance schedules were in place at the FLFH but all repairs and maintenance costs were covered by Government funds.
36 According to The Carter Centre, their current commitment to Onchocerciasis programme extends for another three years beyond which no plans are currently in place. During these three years, their financial input in likely to be in the range of US$10-12,000 annually. This will finance only recurrent expenditure and not capital equipment.
Recommendations: Transport and Implementation Other Materials 1. Written trip authorization and filling of Priority: HIGH log book should be practiced Indicators of Success: 2. Regular system of vehicles • Documented trip authorization and maintenance schedule should be in place properly filled log books • Vehicle are in good condition Who to take action: District and FLHF staff Deadline for completion: Continuous
Human Resources (3.5)-Highly The staff in most of the FLHF was stable except for a few cases. They were reported to have been working at the same health facility for more than two years. Where there were transfers, this was internal within the district and the transferred staff had been trained in CDTI activities. All health staff had attended in-service training for at least two to three times. In some health posts, staff seemed to lack skill and self-confidence in carrying out CDTI activities.
Recommendations: Human Resources Implementation 1. New staffs should be adequately Priority: MEDIUM trained Indicators of Success: Availability of skilled staff at all health facilities Who to take action: District staff Deadline for completion: Continuous
Coverage (4.0)-Fully Geographic coverage rates achieved was 100% for three years (2006 – 2008) and therapeutic coverage rates were all above the 65% APOC recommendation.
Recommendations: Coverage Implementation 1. Staff should be encouraged to keep up Priority: MEDIUM with high coverage level Indicators of Success: High coverage Who to take action: FLHF staff and CDDs Deadline for completion: Continuous
37 3.4 Findings at the Community Level
Sustainability at Community Level 4 3.7 3.5 3.5 3.3 3.3 3
) 3 2.7 4 / (
t 2.5 2.3 h g
i 2 e 2 w
e
g 1.5 a r e v 1 A 0.5
0 p s g i g n e e e n h in a M c g i s r z A n rc n r o ti u ra n e it S a e a n c H in o v l w n e F s P o M e o O M R C & n p a i m h u rs e H d a e L Indicators
Planning and Management (3.3: Highly) CDDs decided on the visiting times for drug distribution (mornings and evenings), in consultation with other community members, based on the availability of the community members. The drug distribution often started at a central location but later on the CDDs carried out house to house distribution whenever the turn up to the central location was lower. This appeared to have increased the work load of CDDs although they did not complain. Besides drug distribution, CDDs also carried out the initial and subsequent annual census updates and health education activities.
In most cases, there were no specific problems reported except during the first two years of the programme when there were refusals as a result of severe side effects. The CDDs mainly worked with FLHF workers to solve this problem. There was little evidence of community leadership helping CDDS to solve problems.
38 Recommendation: Planning and Implementation Management Priority: Medium 1. Scale up the mobilization of the communities to gather at central Indicators of success: locations for drug distribution. i) Reduced house to house visits made by CDDs. 2. CDDs need to work more with community leaders to solve problems. ii) Community leaders more involved in problem solving.
Responsible person: FLHF staff, Community supervisors, Community leaders, CDDs
Time frame: By December 2009.
Leadership and Ownership (2.7: Moderately) The community leaders were not sufficiently involved in mobilizing community members for meetings and for drug distribution. As a result, the turn up for meetings and drug distribution was reportedly low in some communities. It was also reported that community leaders, in conjunction with FLHF staff, were involved in the selection of CDDs in some communities. This was not in line with the CDTI approach and may have contributed to the relatively low community ownership.
There was low therapeutic coverage in the early years of the programme but since then, community demand for the drug increased as its benefits became apparent resulting in improved coverage rates. In a few cases, the CDDs went back and carried out house-to- house distribution when coverage rates were low. However, the involvement of community leaders was found to be inadequate in most of the communities visited.
The community members did not provide incentives, either material or financial, to CDDs except for appreciation and encouragement. The CDDS did not demand for incentives and were happy to continue volunteering their services to their communities. Members of the community were not involved in self monitoring of CDTI activities though in some cases community leaders oversaw drug distribution. Community ownership of the programme was generally low.
39 Community members interviewed could name the main benefits of taking mectizan® and were sufficiently motivated to continue taking the drug for a long time.
Recommendation: Planning and Implementation Management Priority: Medium 1. Selection of CDDs should be carried out with the full involvement of Indicators of success: community members. i) New CDDs selected by community members. 2. There is need for sensitizing ii) Community leaders more involved in community leaders to take an active CDTI role in CDTI. iii) Communities involved in self monitoring activities and supporting CDDs 3. Strengthen community ownership of Responsible person: FLHF staff, the programme Community supervisors, Community leaders
Time frame: December 2009 to June 2010.
Monitoring (3.3: Highly) The activities of CDDs were supervised by the FLHF staff. In most cases, CDDs directly submitted the registration books to the FLHF staff who compiled summary reports for the communities in their catchment areas. In a few cases, the CDDs submitted the registration books to community supervisors to fill summary forms and submit to the FLHF staff depending on the level of literacy of the CDD.
Regarding transport for the delivery of reports to the FLHF, the CDDs and community supervisors in most cases arranged their own means of transport.
Recommendation: Planning and Implementation Management Priority: Medium While selecting CDDs, literacy status Indicators of success: Literate CDDs should be considered. selected. Responsible person: FLHF staff, Community supervisors, Community leaders
Time frame: November to December annually.
40 Mectizan (3.5: Highly) There was adequate mectizan® available for all eligible members of the communities even though delays were occasionally experienced. All communities obtained mectizan® including those remotely located. In the case of absentees and temporarily non-eligible persons, some mectizan was kept for them at the FLHF store for about one month. Most CDDs’ records showed accurate census data which was good enough for calculation of the amount of mectizan® needed.
In most cases, CDDs or community supervisors collected mectizan® from the FLHF even for remote communities, but in a few cases the FLHF staff delivered the drug to the CDDs during their visits to the communities for other health programmes. Generally, CDDs and community supervisors walked to collect mectizan® from the FLHF, and sometime did so for quite long distances i.e. more than 10 km, but were apparently happy to continue doing so.
Recommendation: Mectizan Implementation
Priority: High The district or FLHF should make some arrangement for the delivery of mectizan Indicators of success: to remote communities Mectizan is delivered to remote communities by appropriate means.
Responsible person: District and FLHF staff, Community leaders
Time frame: December to January Annually.
HSAM (2.3: Moderately) HSAM was carried out routinely before and during mass drug distribution but did not necessarily address the information gaps. Most of the time, HSAM was carried out by CDDs during house to house visits when community members were available in their homes. However, sometimes it was carried out by the FLHF staff. There was minimal involvement of community leaders in mobilizing and sensitizing community members. HSAM tended to provide information to the communities and encourage them to continue taking mectizan®. However, little attention was given to the contribution of resources to support the work of CDDs.
41 There was a fair level of community ownership of the programme as expressed by the CDDs and community supervisors’ willingness to continue collecting and distributing Mectizan. However, this was not matched with tangible community support for the work of the CDDs.
Recommendation: HSAM Implementation Priority: Medium 1. Encourage community leaders to be Indicators of success: actively involved in mobilization and i) Active involvement of community sensitization activities. leaders in HSAM. ii) Resources contributed by the 2. HSAM should address the issue of communities to support CDDs contribution of resources by the communities to support CDDs Responsible person: FLHF staff, Community supervisors, Community leaders Time frame: November to December annually.
Finance (2.0: Moderately) Community members did not provide financial or material support to CDDs although they did provide moral support. Nevertheless CDDs expressed willingness to continue serving their communities as long as mectizan® was available. Although there was no contribution of supplies (books, pens etc) by the communities, no gaps were experienced. Registration books were provided by the national programme while other stationery was not seen as an issue by the CDDs
In some communities, there was a misconception among the community members that CDDs were working for the government and were receiving some remuneration. This observation was surprising for a project in its fifth year and points to a gap in HSAM activities. It may be attributed to the practice of paying volunteers by other programmes such as polio eradication campaign, EPI etc.
42 Recommendation: HSAM Implementation Priority: Medium Sensitize the community members to Indicators of success: correct the misconception that No community member believes that government is remunerating CDDs. government remunerates CDD Responsible person: FLHF, community leaders Time frame: As soon as possible.
Human Resource (3.7: Highly) The CDD to population rations ranged between 1 CDD to 94 persons to 1 CDD to 280 persons. The households which the CDDs were expected to cover were within walking distances in many cases. In a few cases, they travelled a distance that took up to 40 minutes.
CDDs were knowledgeable and skilled in the key tasks they were expected to perform. There were a few cases of CDD drop-out; and when they dropped out, new ones were recruited and trained. All CDDs interviewed expressed willingness to continue with the distribution of mectizan® as long as the drug was available. They were motivated by the benefits that the drug had brought to their communities.
Recommendation: Human Resources Implementation
Priority: Medium Train more CDDs to increase the CDD to population ratio in line with the national Indicators of success: target Increased CDD to population ration
Responsible person: FLHF Time frame:
December 2010
Coverage (3.0: Highly) The average therapeutic coverage rates for the twelve communities visited were; 64.3% for 2008, and 64.5% for 2007. In both years, the average TCR was below the cut-off point of 65%. This was attributed to: migratory nature of the population; long distance travelled by some CDDs and the timing of distribution which coincided with farming seasons and festivals;
43 In one of the communities where indigenous people live, CDDs were not competent enough to carry out CDTI activities in terms of disseminating information and mobilizing the community members. In addition, census updates did not take account of people who had permanently migrated from their communities.
Recommendation: Human Resources Implementation
Priority: High HSAM sessions should address the constraints to the achievement of higher Indicators of success: TCR observed in affected communities Increased level of awareness and TCR
Responsible person: FLHF, community leaders Time frame:
June 2010
4.0 CONCLUSION: GRADING OF THE OVERALL SUSTAINABILITY OF THE PROJECT
To reach a judgement on the overall sustainability of the Metekel Zone CDTI Project, the evaluation team considered both qualitative and quantitative methods. a) Qualitative method; the team considered the “Aspects” and the “critical elements” of sustainability as outlined in the Evaluation Guidelines.
On “Aspects”, the extent to which the findings were helping, partially helping, blocking or partially blocking the sustainability of the project were considered. Accordingly, the following conclusions were made on each “Aspect”.
44 Aspect Judgement based on whether the findings were helping, partially helping, blocking, partially blocking sustainability Integration Helping Resources Helping Efficiency Partially helping Simplicity Helping Health staff acceptance Helping (attitude of staff) Community ownership Partially helping Effectiveness Helping
Integration: Helping Sustainability There is strong integration of CDTI activities into the overall health programmes. Oncho control activities were included in both the annual operational and the five year strategic plans; transport and materials were shared across different programmes and health staff carried out support supervision in an integrated manner.
Resources: Helping Sustainability The integrated use of vehicles, equipment and other materials provide a good basis for sustainability of the programme. Although government direct funding commitment for oncho was not clearly spelt out, funds from government and other sources were made available for oncho control when activities from different programmes were conducted in an integrated manner. The commitment of the NGDO partner to continue providing financial support reinforces the sustainability of the project
Efficiency: Partially Helping Efficiency of the use of resources was achieved by integration, conducting training sessions in cost free venues and limiting the duration of trainings. However, supervision, trainings and HSAM activities were not targeted; they were carried out routinely every year. These potentially waste limited resources.
45 Simplicity: Helping Simplicity was demonstrated in several ways such as; delivery of mectizan from FMOH to the community level using existing structures and procedures, delivery of reports by CDDs to the FLHF and onwards as part of routine activities and without using extra resources; trainings being carried in premises owned either by the government or the community.
Attitude of staff: Helping The health staff and the CDD expressed great interest and willingness to continue carrying out CDTI activities after APOC project phases out. The health staff considered CDTI part of their routine duties. For the CDDs, this was despite the fact that they were not receiving financial or material support from the community.
Community Ownership: Partially Helping Although the involvement of CDDs and community supervisors was commendable and promoted sustainability, the community leadership and the rest of the community members did not show active involvement and ownership of the programme.
Effectiveness: Helping Full geographical coverage and therapeutic coverage rate of over 65% were achieved during the past three years.
Regarding the”critical elements”, the judgement was limited to “Yes” and “No” depending on whether these elements were sufficiently present to ensure sustainability.
The evaluation team made the following judgement:
Critical element Judgement Money Yes Transport Yes Supervision Yes Mectizan® supply Yes Political commitment Yes
46 Money: Yes Committed funds were disbursed by partners, including government, and utilised properly even though funding constraints were experienced.
Transport: Yes The Toyota 4WD pick up provided by APOC was functional but two of the motorcycles in the districts visited were non functional. However, the pool system of vehicles ensured that alternative means of transport was available for the programme.
Supervision: Yes Supervision was regularly conducted although it was not targeted to address identified problems. This was intensified during drug distribution to provide effective and prompt support to FLHF staff and CDDs.
Mectizan® supply: Yes The existing government system was effective in delivering mectizan to the FLHF. For most communities, the CDDs or community supervisors collected the drug from the FLHF. Only in a few cases did the FLHF staff deliver the drug to the communities during their visits for other health programme activities. For a few remote communities however, some difficulties were experienced in delivering mectizan.
Political commitment: Yes The participation of the Zonal Administrator in the feedback and sustainability planning workshop was an evidence of good political support at the highest level in the zone. The enthusiasm and commitment he showed during the workshop was reassuring. This was also demonstrated by the District Administrator of Pawi during the advocacy visit. However, the political leaders at the community level were neither sufficiently mobilised nor adequately involved in CDTI programme.
(b) Quantitative method: The team tabulated the scores of each of the indicators for the four levels i.e. zone, district, FLHF and community and further calculated the averages for each level and the overall average for the whole project as detailed in the table below.
47
&
n p g o i t d i ® e n r s h i s e g n i g e g o t e s r e a i n r v a g a n p i t o c z r i i r r e t i a s n M n i t e v e n r g d i n i n a n c A v p t e e a a a a n t o e S c R o v l u e r r i n P L M S M T H I A T H F C A Zone 3.2 3.3 3.0 3.8 3.3 3.7 2.5 3.0 2.3 4.0 3.21 District 3.3 3.5 3.2 3.5 3.2 3.8 2.7 3.8 2.8 4.0 3.38 FLHF 2.8 3.0 2.8 3.5 3.2 3.7 2.3 3.5 2.2 4.0 3.1 Comm 3.3 2.7 3.3 3.5 2.3 - - 3.7 2.0 3.0 2.97 Overall average 3.20
This is represented graphically below
Overall Grading of Metekel CDT Project
3.5 3.38
) 3.4 4 / ( 3.3 t 3.21 3.2 h
g 3.2
i 3.1 e 3.1 W
2.97 e 3 g a
r 2.9 e v 2.8 A 2.7 Zonal District FLHF Community Overall Average Level
Therefore on the basis of the following: Aspects; the team found five of the aspects to be helping while two were partially helping the sustainability of the project. Critical elements, the team found that all the five critical elements for sustainability were present. Quantitative; The average score is far greater than 2.5 expected for a third year project.
On the basis of the aforementioned criteria, the evaluation team concluded that Metekel Zone CDTI project, in spite of some imperfections, meets the description of being “FULLY SUSTAINABLE”
48 5.0 FEEDBACK/SUSTAINABILITY PLAN DEVELOPMENT WORKSHOP
A three day feedback/sustainability planning workshop was held from the 27th to 29th May 2009 in Tana Beles hotel in Chagni town. Participants were drawn from the zonal and district level members of staff involved in the CDTI project.
The proceedings of the workshop were as follows:-
Day 1; 27th May The workshop started with the registration of participants and distribution of workshop materials. A welcome address was made by the scout, Mr Tesfaye, who also requested the participants and the evaluation team members to introduce themselves. An opening remark by Sr. Yalmefikier Hika, the zonal onchocerciasis control coordinator, followed after which a presentation on the concept of sustainability, and the seven elements of sustainability was made by Dr Johnson Ngorok, the evaluation team leader. A briefing on the evaluation methodology and the method of scoring, considered as necessary for the participants to understand the evaluation findings, was presented by Dr. Margaret Akogun. The workshop proceedings then turned to its main agenda i.e. feedback presentation on the major findings of the evaluation; highlighting the strengths and weaknesses of CDTI implementation at each of the four levels i.e. zonal, district, FLHF and community. This was done in turn for each of the four levels by the evaluation team members. The presentations were followed by discussions which generated debate on the issues raised. The sessions on the presentations continued until lunch break. The afternoon session opened with a presentation on the SWOT analysis by the Team Leader following which participants were divided into three work groups. The zonal level participants formed one group while participants from the four districts were divided into two groups; one to work on district level issues and the other on issues for the FLHF and community levels. The groups carried out a SWOT analysis for each of the levels building on the feedback presentations made earlier. The first day was concluded with the presentations by the work groups which were followed by plenary discussions. The presentations on SWOT analysis are appended. The first day of the workshop ended at 5.00 pm.
49 Day 2; 28th May The second day started with a review of Day 1 activities by Mr Melak. Participants then returned to their groups for the second group exercise i.e. to work on solutions to address weaknesses and possible ways to mitigate threats/challenges of the SWOT analysis. On completion of the exercise, the plenary was reconvened for the groups to make their presentations; this was followed by discussions.
The third group work discussed resources needed to sustain project activities for the next three years following APOC’s phase out i.e. financial resources, transport, equipment and material resources. This exercise was carried out in two groups; the zonal level and district level. The group work was also concluded with group presentations and plenary discussions.
The fourth group work then turned to the second main agenda of the workshop; developing five year sustainability plans. This was seen as the most important part of the ToR of the evaluation team. The group work was flagged off with presentations on APOCs definition of sustainability and the sustainability planning template. The three key issues to consider i.e. cost containment, integration and mobilizing finance were also discussed. This time, the groups reassembled to their respective districts and zone for each to develop its own five year sustainability plan. The district groups were advised to take cognizance of the FLHF level in their planning. This activity continued for the rest of the day.
Day 3; 30th May Participants continued working on their sustainability plans with the evaluation team members supervising and providing guidance. All the groups completed Year One i.e. 2010 Plan and plans for at least the following two years. Due to time constraint and this being a Friday, the group work could not be completed. Instead, the evaluation team decided to discuss and polish up at least the first year of each of the five plans. The group work was therefore stopped and participants reassembled back to the plenary for presentations and discussion of the plans. Before the presentations of the draft sustainability plans started, a presentation on the “critical elements” of sustainability was made and participants were alerted that the evaluation team would be assessing their plans to ensure that all these elements were included. Presentations and plenary
50 discussions, which took a critical look at the different indicators of the planned work, thereafter proceeded but time could only allow for the discussion of the zonal and two of the district sustainability plans before turning to the official closure of the workshop
Finally, in the closing ceremony, a debriefing of the major findings was presented by the evaluation team leader to His Excellency Mr. Gawi Jane, the Zonal Administrator, who participated in the third day of the workshop. The workshop was thereafter officially brought to a close by the Zonal Administrator who remarked on his awareness and interest in the CDTI project. He pledged for continued support for the program after APOC withdrawal.
6.0 ADVOCACY VISITS AND DEBRIEFING
Advocacy visits and debriefings were conducted at zonal and district levels. Three advocacy sessions took place. The first with the Head of the Zonal Health Desk, the second with the District Administrator of Pawi and the third with the Zonal Administrator, Metekel Zone. Each of these advocacy sessions is discussed in detail below. i) Advocacy visit to Head of the Zonal Health Desk The evaluation team met with Mr Beyene Lire, the Acting Head of the Zonal Health Desk early during the evaluation exercise immediately after interviews with the Zonal Oncho Coordinator and the health team. During the interviews, three major issues that required advocacy with the policy makers had emerged: finance, vehicle and equipment as well as human resource.
The team leader made a briefing on the background of CDTI programme and the objective of the evaluation mission. He emphasised that the CDTI project was designed to be implemented over a five year period during which time full sustainability of the project should have been achieved. APOC funding was to be reduced over the five years while government and other partners contributions were expected to increase.
The Acting Head of the Zonal Health Desk was reassured that mectizan supply would continue and that government is expected to continue delivering the drug to the FLHFs for which vehicles are needed. Similarly, capacity for project implementation had been
51 developed over the years and that a minimum amount of finance was required to sustain the project. The team also presented the observation on staffing at the zonal level as only one health staff member was actively involved in the CDT project.
The Acting Head of the Health Desk gave the following response:- • Regarding finance, he remarked that he was aware of CDTI activities and that they were being carried out properly. He also noted that sensitisation and mobilisation of the communities had been carried out and that the demand for the drug had increased. He reassured the team that oncho activities will continue being integrated into the health programme and that the programme will continue after the end of APOC support. He was confident that management will take care of funding requirement. • With regards to replacement of vehicles, he was less optimistic as these decisions are taken at a higher level. He however promised that he would forward the request to the Federal Government through the Regional Health Bureau. The more practical measure he saw was integrated use of available vehicles for all programmes. • On staffing, the official mentioned that the existing government structure limited the number of staff at this level as part of the government strategy to reduce cost. He argued that although there was a shortage of staffing at the zonal level, there were many extension workers at the grassroots level.
Finally, the Team Leader requested that in the next strategic plan, 2011-2015, budget allocation is made available for the replacement of vehicles and equipment. ii) Advocacy visit to the Pawi District Administrator The evaluation team met with Mr Aregan Alamirew, the Pawi District Administrator in the presence of the members of the district health team.
The team leader gave a similar briefing as with the Head of the Zonal Health Desk and proceeded to present the two advocacy issues i.e. the need for continuing financial support for CDTI and vehicle and equipment replacement.
In his response, the District Administrator mentioned the following:
52 • The reassurance that mectizan supply will continue was welcome and a big relief. He assured the evaluation team that as long as the drugs were available, the district administration will allocate funding to carry out oncho treatment. • On vehicle replacement, he confirmed that if replacement of vehicles and equipment is made this year as per APOC plan, the district will plan for the next replacement in four to five years time.
Finally, he remarked that the demand for the drug was high and that the programme would continue being supported. iii) Debriefing session with the Zonal Administrator. The Zonal Administrator, Mr Gawe Jane, participated in the feedback and sustainability planning workshop on the first and last days. This was impressive and a demonstration of political commitment to the programme at the highest level in the zone.
At the end of the planning workshop, as already referred to, a debriefing session was conducted as part of the closing ceremony.
The Team Leader presented the following debriefing: Background: APOC and the CDTI approach, the objective and indicators of sustainability evaluation.
Key findings were summarized and presented. The strengths are inclusion of CDTI into the five year strategic plan for health programmes, integration of oncho treatment with other health programmes, committed leadership at all levels, knowledgeable and skilled staff with good attitudes and commitment, good supervision leading to empowerment of staff at lower level, simple and effective delivery of mectizan, good financial control, pool system for using the available means of transportation, commitment of NGDO partner (The Carter Centre), and good geographical coverage (100%) and therapeutic coverage (over 65%).
The weaknesses include: targeting of activities to specific needs, lack of structured meetings for ZOTF, absence of reports on supervisory visits, delay in the delivery of mectizan to some communities, misallocation of the drug to the districts, problem of
53 budgeting and identification of gaps, old and malfunctioning vehicles, shortage of staff at the zonal level, insufficient involvement of community leaders reducing the level of community ownership.
Finally, the team leader presented an appeal on behalf of the evaluation team; he emphasized three issues: allocation of some budget for CDTI programme, planning for replacement of vehicles and essential equipment, as well as active involvement of the political leadership in sensitization and mobilization.
In his response, the Zonal Administrator affirmed that the entire Zonal Task Force, the highest policy making body in the zone, was behind the CDTI programme and that he would ensure that in their deliberations and budget discussions, oncho programme would be considered.
54 Appendix 1: Evaluation Time Table
DAY TEAM MEMBERS ACTIVITY Tuesday 12th – Scout • Preparatory activities for all aspects Friday 15th May of the evaluation visit Saturday 16th May Johnson and Margaret • Arrival in Addis • Check in at Atlas Hotel Sunday 17th May Johnson and Margaret • Team building in Addis Monday 18th May All Team • Welcome and briefing at The Carter Centre • Travel to Metekel Zone • Over night at Debre-Markos Town Tuesday 19th May All Team • Arrival in Metekel Zone • Check in Tana Belles Hotel in Chaggni Town • Orientation on the Guidelines and Instruments • Formation of sub-teams • Finalising the evaluation programme • Data collection at the zonal level Wednesday 20th All Team • Data collection at zonal level cont. May • Advocacy visit to Head of Zonal Health Desk Thursday 21st May Sub-team A (Johnson, • Data collection at zone Tekola, Abraraw) • Data collection at Pawi district Sub-team B (Margaret, • Data collection at zone Berhanu, Tamirat • Data collection at Dangur district Friday 22nd May Su-team A • Data collection at Pawi district cont.. • Data collection at Felegesalam Health Centre Sub-team B • Data collection at Dangur District • Data collection at Manbuk Health Centre Saturday 23rd May Sub-team A • Data collection at Felegesalam Town and Medin village • Data collection at Ketena 2, Village 131 Health Post • Data collection at village 131 and village 29 Sub-team B • Data collection at Dangur district cont…. • Data collection at village level Sunday 24th May Sub-team A • Data collection at Ketena 1, village 7 Health Post • Data collection at villages 6 and 7 Sub-team B • Data collection at village level Monday 25th May All Team • Data analysis and completing master copy for each instrument
55 Tuesday 26th May All Team • Planning and preparation for the feedback/sustainability plan workshop. Wednesday 27th – All Team • Feedback/sustainability Plan Friday 29th May workshop • Debriefing of Zonal Administrator • Grading of the whole project Saturday 30th May All Team • Report writing – Monday 1st June Tuesday 2nd June All Team • Travel back to Addis Ababa Wednesday 3rd Johnson and Margaret • Johnson and Margaret return home. June
56 Appendix 2: Workshop Programme
Feedback and Planning workshop for CDTI sustainability From 27- 29 of May 2009 Chagni Town The African Program for Onchocerciasis Control (APOC) Time Title Presenter Moderator
8:30 - 9:00 Registration and Agenda items introduction Mr. Abraraw 9:00 - 9:05 Opening remark Sr. Yalmfikir H. Mr. Abraraw 9:05 - 9:15 Self introduction Participants Mr. Abraraw 9:15 - 9:25 Housekeeping matters Mr. Tamirat Mr. Abraraw
9:25 - 9:45 The objectives of the evaluation and the meaning of sustainability Dr. Johnson Mr. Abraraw
9:45 - 10:00 Evaluation methodology Dr. Margaret Dr Johnson
10:00 - 10:30 Tea break WHO-APOC Organizers Presentations on the Major findings e n
o 10:30 - 11:00
y Zone level findings Mr. Berhanu Mr. Abraraw a
D 11:00 - 11:30 District level findings Mr Abraraw Mr. Berhanu 11:30 - 12:00 Front Line Health Facility Findings Dr. Tekola Mr. Berhanu 12:00 - 12:30 Community Level findings Dr Margaret Mr. Abraraw 12:30- 2:00 Lunch Break WHO-APOC Organizers Presentation on SWOT analysis 2:00 - 2:10 Dr. Johnson Dr. Margaret Group work on SWOT analysis and solutions to weaknesses 2:10 - 3:10 & threats regarding sustainability Participants Evaluation team 3:10 - 3:30 Tea break WHO-APOC Organizers Group work presentation and plenary discussion 3:30 - 5:00 Participants Evaluation team
57 Feedback and Planning workshop for CDTI sustainability From 27- 29 of May 2009 Chagni Town The African Program for Onchocerciasis Control (APOC) Time Title Presenter Moderator Recapitulation 8:30 - 8:45 Participant Dr. Johonson
8:45 -8:55 Presentation on Group work 2 Dr Johnson M.r Abraraw Group work on solutions to project weakness in relation to 8:55 -10:00 sustainability Evaluation team Participants members 10:00 - 10:30 Tea break o WHO/APOC w Mr. Abraraw and T 10:30 - 11: 00
y Group work presentations and plenary discussions Participants Berhanu a 11:00 - 11: 05 D Presentation on Group work 3 Dr. Johnson Mr. Berhanu
11:05 - 11: 10 Presentation on Group work 4 Mr. Berhanu Dr. Johonson Evaluation team 11:10 - 12: 30 Group work on sustainability plan Participants members
12:30 - 2: 00 Lunch WHO/APOC Evaluation team 2:00 - 5: 00 Group work on sustainibility plan cont. Participants members
58 Feedback and Planning workshop for CDTI sustainability From 27- 29 of May 2009 Chagni Town The African Program for Onchocerciasis Control (APOC) Time Title Presenter Moderator Sustainability plan presentations and discussions Evaluation team 8:30 - 10: 00 Participant members Tea break 10:00 - 10:30 WHO/APOC
e Evaluation team 10:30 - 12: 30 e Sustainability plan presentations and discussions cont. Participants members r h
T 12:30 - 2: 00
y Lunch WHO/APOC a
D 2:30 - 3: 30 Debriefing of the findings to zone administration head Evaluation team members 3:30 - 4: 00 Closing remark Zone administration head 4:00 - 4: 30 Administrative issues (per diem payments etc) Mr. Tamirat
59 Appendix 3: SWOT Analysis
ZONAL LEVEL
Strengths • Involvement of task-force in planning CDTI • Integration of CDTI activities • Integration of CDTI activities with the activities of other health programs • Active leadership at Zonal Health Desk • Effective system of Mectizan® procurement and delivery
Weaknesses • Delay in releasing budget • Lack of qualified professional in data collection and management • Health personnel’s minimal computer skills • Failure to note and document feedbacks and subsequent actions taken • Poor documentation system • Failure to write reports on feedbacks to districts • Lack of regular or immediate maintenance for vehicles and office equipment
Opportunities • Availability of budgets for other health programmes • Favourable policies and strategies on inter-sectoral collaboration • Donation of Mectizan® • The arrival of Mectizan® during the dry season
Challenges / Threats • Shortage of budget • Shortage of human power • Limited involvement of political leaders as they are preoccupied by other administrative issues • Shortage of vehicles
DISTRICT LEVEL
Strengths • Participatory planning involving FLHFs staff • Undertaking CDT activities integrated with other health programs • Availability of supportive leadership at woreda health office • Improving therapeutic coverage • Timely delivery of Mectizan
Weaknesses • Shortage of vehicles and essential equipment (computer, printer, photocopier) • Lack of regular maintenance schedule for vehicles and equipment • Failure to replace old vehicles • Failure to solicit other donors • Poor record keeping/filing practices mainly at FLHFs level • Delay in delivery of reports from FLHFs to district and upward • Lack of schedules for training and HSAM activities
Opportunities: • Availably of resources from other government sectors (Education, Agriculture etc) • Favorable government policy regarding integration and inter-sector collaboration 60 • Availability of pack animals to transport drugs and other materials • Increasing awareness and acceptance of the communities regarding ivermectin therapy • The increasing attention of the government to expand roads in the entire country
Challenges / Threats • Governmental budget constraint • Delay in budget release from APOC • Untimely release of fund from TCC • Difficult topography and poor road network of the district • Migratory nature of the inhabitants • Scattered settlement of communities in some districts • Lack of qualified maintenance workers in the locality • Increasing costs of spare-parts and maintenance service
COMMUNITY LEVEL:
Strengths: • Willingness of CDDs to render CDTI service • Involvement of community members in selection of CDDs
Weaknesses: • Low CDD to population ratios in some communities • Minimum involvement of community leaders • Delay in submitting treatment reports from CDDs working in remote communities • Less involvement of community supervisors in carrying out Onchocerciasis treatment
Opportunities: • Acceptance of the service by community members • Communities witnessing the benefits of Mectizan
Challenges / Threats • Governmental budget constraint • Delay in budget release from APOC • Untimely release of fund from TCC • Difficult topography and poor road network of the district • Migratory nature of the inhabitants • Scattered settlement of communities in some districts • Lack of qualified maintenance workers in the locality • Increasing costs of spare-parts and maintenance service
Proposed Solutions to Weaknesses:
ZONAL LEVEL • Timely utilization of allocated budget and reporting • Facilitating training courses on computer skills and data management • Writing and documenting feedbacks after carrying out supervision • Lobbying for better budgetary allocations • Soliciting funds from various donors • Contacting the Regional Health Bureau and lobbying for a change in the zonal structure • Requesting the Regional Health Bureau to allocate budget for procuring vehicles and equipment
61 DISTRICT LEVEL • Lobbying with higher officials regarding budget increase • Directing FLHFs staff to recruit and train more CDDs and community supervisors • Making arrangements for early HSAM and Mectizan distribution for migratory communities • Using pack animals for transporting Mectizan for remote and hard to reach communities • Note and forward feedbacks to FLHFs • Develop training schedule and execute accordingly • Scale up the knowledge and skills of FLHFs on report writing, documentation and forwarding • Increase the knowledge and skills of district staff on report writing and documentation • Using vehicles from other sectors
COMMUNITY LEVEL
• Increasing the number of CDDs as per standard (1:50) • Initiating more involvement of community leaders • Developing documentation and reporting system • Lobbying for voluntary resettlement program for the scattered communities • Promoting integration with other health programs • Provision of training for new health personnel at FLHF level
62 Appendix 4: List of Respondents
ZONAL LEVEL (Gilgel Beles)
Name Responsibility Location 1. Sr. Yelemfikir Hika Zonal Onchocerciasis Control Coordinator Metekel 2. Mr Beyene Lire Acting Head of Zonal Health Desk Metekel 3. Ms Yeshiwork Addisu Zonal Head of Drug Store Metekel 4. Mr Simieneh Alemu Oncho. Focal Person for Zonal Finance Office Metekel 5. Mr Kef Yalew Muteta Zonal Mother and Child Health Coordinator Metekel
DISTRICT/WOREDA LEVEL
6. Mr Aschale Ayena Head of Dangur district Health Officer Dangur 7. Mr Haimanot Gabeyehu Oncho. Control Coord. for Dangur Health Office Dangur 8.. Mr Andargachew Shibabaw Head of Dangur Health Office Dangur 9. Mr Amsalu Bishaw Oncho. Control Coordinator Pawi 10. Mr Misganaw Tenaw Head of Pawi district health officer Pawi 11. Mr Worku Teshale Health Service and programming Team leader Pawi 12. Mr Arega Alamirew District Administrator Pawi
FLHF LEVEL 13. Mr Ashagre Gebeyehu Head of Manbuk Health Centre Dangur 14. Mr Heyelom Berhanu Oncho. Focal person Manbuk Health centre Dangur 15. Mr Berhanu Yilma Head of Felegeselum Health Centre Pawi 16. Mr Tarekegn Lachamo Head of Ketena 1 village 7 Health post Pawi 17. Ms Zemu Adem Health Ext worker, Ketena 1 Health post Pawi 18. Mr Alemu Ejamo Health worker, Felegeselam Health centre Pawi 19. Miss Hezeblanchi Mekunaw Clin.Nurse and Head of Burj Health post Dangur 20. Mr. Adamtachew Birhanu Lab Technician – Gublak Health Post Dangur
COMMUNITY LEVEL 21. Mrs Etenesh Molla Health ext worker, Ketena 2, village Pawi 21. Mr Gashaw Yehunie CDD, Felegeselam town Pawi 22. Mr Nigussie Gebremariam Com. Supervisor Felegeselam town Pawi 23. Mr Esubatew Bitew Com member Felegeselam town Pawi 24. Mohammed Ahmed Com member Ketena 2 village 131 Pawi 25. Mrs Yeshi Gashru CDD, Ketena 2 Village 131 Pawi 26. Mr Tesfaye Abuye CDD, Ketena 1 Village 6 Pawi 27. Mr Markos Arsicha CDD, Ketena 1 Village 6 Pawi 28. Mr Tesema Adda CDD, Ketena 1 Village 6 Pawi 29. Mr Shewelie Meskelu Community member, Ketena 1 village 7 Pawi 30. Mrs Emamu Ayele CDD, Ketena 1 Village 7 Pawi 31. Mrs Kedija Ahmed Community member Ketena1 village 7 Pawi 32. Mr Malade Azene Communityleader/Health supervisor Dangur 33. Umar Nuru CDD Burj community Dangur 34. Mr Abenai Malis CDD Burj community Dangur 35. Mr Teshome Dilnasaw CDD Asama wiha community Dangur 36. Mr. Abere Tizazu CDD Ketena 3 Dangur 37. Mr. Getu Tinsae CDD Ketena 1 Dangur 38. Mr. Ali Mohammed CDD Ketena 1 Dangur 39. Mr. Nigatu Taye CDD Ketena 1 Dangur 40. Mr. Tariku Terefe Community Leader Gublak Village Dangur 41. Mr. Yabes Lango CDD Gublak Village Dangur 42. Mr. Zeleke Roro CDD Gublak Village Dangur 63 43. Mr. Gedlu Tefera CDD Gublak Village Dangur 44. Ms. Tobia Tefera CDD Gublak Village Dangur 45. Mr. Sisay Sayih CDD Kola Village Dangur
64 Appendix 5: List of Documents Reviewed
At zonal level
1) CDTI Treatment Summary forms showing total population, targeted population, population at risk, total population treated, geographic and therapeutic coverage percentages for the fiscal years of 2006, 2007 & 2008
2) Five year integrated strategic plan of 2006-2010 and a one year plan of 2006, 2007 & 2008 at Metekel Zonal Health Desk for all health programmes including CDTI activities
3) Internal office memo for expenditure autorisation, payment request forms, petty Cash and Cheque Payment Vouchers, cash transfer letters, debit and credit advices, general journals and ledger cards
4) Annual technical/performance reports
5) Mectizan requisition letters to FMoH, Mectizan distribution letters to districts, stock record cards, store requisition form, goods receiving voucher, store issue voucher
6) Training manuals, flip charts, posters
7) Travel authorization forms and log books
At district level
1) CDTI Treatment Summary forms from four districts showing districts’ total population, targeted population, population at risk, total population treated, geographic and therapeutic coverage percentages for the fiscal years of 2006, 2007 & 2008
2) Five year integrated strategic plan of 2006-2010 and a one year plan of 2006, 2007 & 2008 at the districts visited for all health programmes including CDTI activities
3) Integrated activity schedule prepared for all health programmes
4) Internal office memo for expenditure autorisation, payment request forms, petty Cash and Cheque Payment Vouchers, general journals and ledger cards
5) Annual technical/performance reports
6) Mectizan requisition letters to FMoH, Mectizan distribution letters to FLHF, stock record cards, store requisition form, goods receiving voucher, store issue voucher
7) Training manuals, flip charts, posters, timetables for health education
At FLHF level
1) CDTI Treatment Summary forms showing total population, targeted population, population at risk, total population treated, geographic and therapeutic coverage percentages for the fiscal years of 2006, 2007 & 2008
2) Community supervisors CDTI summary form, CDDs treatment registers, annual technical/performance reports 65 3) Written verbal document for issuance of Mectizan to CDDs
4) Training manuals, flip charts, posters
66 APPENDIX 6: THE FIVE YEAR SUSTAINABILITY PLAN; RESOURCES
SUMMARY TABLE (Mandura District)
Amount/Numbers Zonal District FLHF Total Sources of funding level level GOV’t NGDO APOC Finance 2010 59,500 59,500 36,933.33 2011 59,500 59,500 2012 59,500 59,500 Human 2010 15 15 Resources 2011 15 15 2012 15 15 Transport 2010 1motor 1motor and cycle cycle equipment 2011 - - 2012 - -
67 SUSTAINABILITY PLAN FOR METEKEL CDTI PROJECT LEVEL Mandura Disrict YEAR 2010
Activity Objectives Time Person Indicators Requireme Cost Sources of funding frame Responsi of success nt MOH NGDO APOC ble Planning CDTI To implement April 2009 Woreda Accomplishm Paper two 260 2700 1300 710 activities with other the CDTI health ent of rim, health service planned department planned Type ink, 650 activities activities (plan & CDTI activities (stencil), for 2010 is focal person carbon 4 rim, 400 developed) Photocopy 3,400 two cans ink
Mectizan collection Carry out MDA Feb1-5/10 Woreda Availability Fuel & 1,500 from ZHD and CDTI focal of mectizan at lubricant distribution to FLHFs person Woreda and Perdieum 2,100 pharmacy FLHFs in Labour 100 tec. time Contingency 150 (e.g. Tyre repairs, maintenance ) Carry out HSAM Sensitizing the Jan16-20/10 Woreda • Strong Manual& 1,400 activities Woreda task health mobilizati stationary, force on CDTI department on & Per diem 17,500 CDTI activities & CDTI advocacy Refreshment focal person 5,000 • Commitm ent of leadership
68 SUSTAINABILITY PLAN FOR METEKEL CDTI PROJECT LEVEL Mandura District YEAR 2010
Activity Objectives Time Person Indicators Requireme Cost Sources of funding frame Responsi of success nt MOH NGDO APOC ble Conduct HSAM To mobilize for Feb 10-15/10 District Increased Stationery 250 at community active health support for level participation of department CDDs from Per diem 12,250 the community & CDTI the and political focal person community Refreshment 3,500 leaders and political leaders Training of new To build January 10- District • Trained Stationery 850 health staff, new Capacity 12/10 health health CDDs and department workers, Per diem 2,100 community & CDTI Communit supervisors focal person y Refreshment 360 supervisor s & CDDS • Improved quality of CDTI activities Supportive To monitor and April 10- District • Integrated Fuel & 3,700 supervision & supervise targeted 15/10 health staffs supervision lubricant monitoring of CDTI activities and effective monitoring CDTI activities at and other health Per diem 4,200 • Evidence of lower level programs problem solving and follow up of previous visits recommenda tion
69 SUSTAINABILITY PLAN FOR METEKEL CDTI PROJECT LEVEL Mandura District YEAR 2010
Activity Objectives Time Person Indicator Requireme Cost Sources of funding frame Responsi s of nt MOH NGDO APOC ble success To facilitate Januray- District Availability Motorbike • Requesting CDTI activities 2010 CDTI of reliable APOC for coordinator transport Maintenance replacement & District and training of old health office materials motorbike • Reproducing Training materials
Maintenance of To ensure the Focal The bike in GOV motorbike functionality of person good the Motorbike mechanical state
70 SUSTAINABILITY PLAN FOR METEKEL CDTI PROJECT, 2010 – 2014 page 1 Level: zone, Year: 2010 Zone Metekel zone
S. Activities Objectives Time Person Indicators of Total Sources of Funding N Frame Responsible Success Requireme cost MOH NGDO APOC nts (Birr) 1 Preparing • To ensure May 2009 Zone CDTI - • Paper 1860 744 930 186 annual health CDTI is coordinator Comprehensive • Photocop plan which included and together with plan which y includes CDTI budgeted for in other zone includes CDTI • Toner activities the health plan health desk is in place (details) • To determine staff members within the set resources period. needed for -CDTI activities budgeted for • Help tracking and funds activities are released running according to the plan
2. Collect • To ensure January 10- CDTI Arrival of • Fuel 5218 5218 - - Mectizan ® Mectizan® 15/2010 coordinator and Mectizan® at • Lubricant and necessary and materials Pharmacy zone ware • Per diem materials from are available on section house within FMoH specified time the specific at zone period warehouse
71 SUSTAINABILITY PLAN FOR METEKEL CDTI PROJECT, 2010 – 2014 page 2 Level: zone, Year: 2010 Zone Metekel zone
S. Activities Objectives Time Person Indicators of Total Sources of Funding N Frame Responsible Success Requirem cost MOH NGDO APOC ents (Birr) 3 Conduct • To reinforce the January Zone CDTI • Task force • station 2300 975 1325 0 advocacy commitment of 2010 (before coordinator, members ary meeting with task force or at the Zone Health are well • Per zone task force members. beginning of Desk head, aware of diem MDA) together with the TCC staff progress of • Refres the hment program. cost • Fund is released • Members monitoring the program 4 Facilitate and • To make sure December CDTI • Appropriat • Fuel 3560 1712 1000 848 support District appropriate 2009 coordinator e trainings • Vehicl level trainings targeted trainings conducted e for health are conducted in all • Per workers districts as diem per the schedule • Staff are knowledge able and skilled in CDTI
72 SUSTAINABILITY PLAN FOR METEKEL CDTI PROJECT, 2010 – 2014 Level: zone, Year: 2010 Zone Metekel zone
S. Activities Objectives Time Person Indicators of Total Sources of Funding N Frame Responsible Success Requirem cost MOH NGDO APOC ents (Birr) 5 Carry out • To ensure CDTI March to Zone CDTI • Supportive • Per 25090 5875 12350 6865 supportive activities are April 2010 coordinator supervision diem supervision in running properly together with is made as • vehicle four districts • To monitor zone staff per the plan and selected activities and • Problems FLHFs and provide support identified communities when deemed and solved necessary on the spot • Follow up of recommend ations of the previous visits 6 Submit a • To disseminate 2nd week of CDTI • A complete - - - - - summary report and share May 2010 coordinator report is to RHB and performance of submitted partners the CDTI timely activities • Actions taken as a result of the reports
7 Conduct post • To assess the 1st week of ZHD head and • Review • Per 12671 2534 8870 1267 MDA review overall May 2010 CDTI meeting diem meeting performance and coordinator conducted • Fuel share the results as per the • Vehicl among districts schedule • To reflect and draw e lessons for future • MDA • notebo activities activities ok coordinated
73 SUSTAINABILITY PLAN FOR METEKEL CDTI PROJECT, 2010 – 2014 Level: zone, Year: 2010 Zone Metekel zone
S.N Activities Objectives Time Person Responsible Indicators of Total Sources of Funding Frame Success Requirements cost (Birr) MOH NGDO APOC 8 Replacement of • To effectively January Zone CDTI Timely arrival • One car Vehicles and run the CDTI 2010 coordinator and ZHD of vehicles and • Photocopier capital equipment activities for head equipment • Desktop computer the upcoming • Printer five years • Fax machine • Communication - - - APOC radio • Generator
74 Metekel Zone CDTI Project
Amount/Numbers Zonal level District FLHF Total Sources of funding level GoV NDGO APOC Finance 2010 50699 50699 17058 24475 9166
2011 50699 50699 17058 24475 9166
2012 50699 50699 17058 24475 9166
Human 2010 8 8 Resources 2011 8 8
2012 8 8
Transport 2010 7 7 7 and equipment 2011 - - -
2012 - - -
75 76