World Health Organization African Programme for Onchocerciasis Control
FINAL REPORT
Evaluation of the Sustainability of the Ebonyi State CDTI project, Nigeria
Ausust 2003
Eleuther Tarimo (Team Leader) Mary Alleman Cyrille Evini Uwem Ekpo Chukwu Okoronkwo
' * t)c**'*""'
1,
r! &? v{r"r'T,;-;l -.i,), .,.(vJ i lt r .. ii,,-"".'_ ! --4 L. Introduction...... ,...... 8 2. Methodology...... 9 2.1 Sampling ...... 9 2.2 Levels and instruments...... 10 2.3 Protocol...... 10 2.4 Team composition ...... 11 2.5 Advocacy visits and 'Feedback/Planning' meetings...... 12 2.6 Limitations ...... 12 3.Evaluation Findings And Recommendations...... 13 3.1 State lrvel ...... 13 3.2 Local Government Area Level ...... 17 3.3 Front Line Health Facility Level ...... 23 3.4 Community Irvel ...... 27 4. Conclusions...... 33 4.1 Grading the overall sustainability of the Ebonyi State CDTI project...... 33 4.2 Grading of project as a who1e...... 36 ANNEXES 37 Interviews ...... 38 Schedule for the Evaluation, advocacy...... 40 Feedback and planning meeting agenda ...... 41 Report of the feedback/Planning meetings ...... 45 Abbreviations/acronyms
APOC African Programme for Onchocerciasis Control CBD Community Based Distributor (of Ivermectin) CBIT Community Based Ivermectin Treatment CDD Community Directed Distributor (of Ivermectin) r : CDTI Community Directed Treatment with Ivermectin FLHF First Line Health Facility fYQ Headquarters HIC Health Centre IVP Health Post HSAM HealthEducation/Sensitization/Advocacy/Mobilization IDP Ivermectin Distribution Programme IEC Information, Education, Communication LGA Local Government Area LOCT Local Government Onchocerciasis Control Team MoH Ministry of Health NGDO Non-Governmental Development Organization NGO Non-Governmental Organization NID National Immunization Day NOCP National Onchocerciasis Control Programme NOTF National Onchocerciasis Task Force PHC Primary Health Care PRS Personnel Research and Statistics RBP River Blindness Programme REMO Rapid Epidemiological Mapping of Onchocerciasis SHM Stakeholders'Meeting SOCT State Onchocerciasis Control Team TCR Therapeutic Coverage Rate WHO World Health Organization ZOTF Zonal Onchocerciasis Task Force Acknowledgements
We would like to thank the following persons for their help: . The staff at the Headquarters of the African Programme for Onchocerciasis Control (APOC) in Ouagadougou: Dr S6k6t6li, Dr Amazigo, Mr Agboton, Dr Noma, Ms Matovu, Mr. Aholou & Mr. Zoure
. His Excellency, Dr Sam Egwu, the Executive Governor of Ebonyi State, Chief F. O. U. Mbam Permanent Secretary, Ministry of Health, and other top policy makers in the State.
. Mr Stephen Orogwu, State Coordinator, Ebonyi State CDTI Project, other SOCT members, drivers and staff of MoH, Abakiliki for undertaking all the arrangements.
. Mrs Chinyere Maduka, Project Administrator, Global 2000 Anambra/Enugu/Ebonyi CDTI Project, for her kind assistance in facilitating arrangements for travels.
. Political and traditional leaders, health workers and community members of areas visited in Ikwo (Ikwo Central & Ikwo South), Izzi (Izzi & Nnodo) and Onicha (Onicha & Ohaozara East) LGAs. Executive Summary
The Ebonyi State CDTI Project, supported by the African Program for Onchocerciasis Control (APOC) since August 1998, is now in its 5th and final year of support. The project was evaluated by an independent team from 4-18 August 2003 using guidelines developed by APOC. The main finding from the evaluation is that CDTI has been a success in the state. There are, however, areas that need improvement to ensure sustainability of the project.
Funds provided by partners (Ebonyi State, APOC and Global 2000) have been adequate to carry out planned activities. The State has released funds for CDTI as follows: $30,000 (1999); $32,000 (2000); $30,000 (2001) and $37,000 (2002). The evaluation team was pleased to note the high level of political commitment to CDTI by the state as evidenced by enthusiasm, knowledge of onchocerciasis and related issues, and support to activities. It is hoped that the state's commitment will be maintained as APOC support declines. The project leadership has actively sought supplementary funds from other sources. For example, for the period 2003-2006 support for a number of activities has been secured from the Ebonyi State Health System Project (assisted by The World Bank). The rolling Health Plan at the state level includes onchocerciasis control. A detailed CDTI annual plan and a "Sustainability Plan of Action" also exist. The sustainability plan was an important input to the Feedb ack/Plannin g meetin g after the ev aluati on.
Health education, sensitization, advocacy, and mobilization (HSAM) has been quite successful. Innovations have been made in the designs of posters, calendars, brochures, and fliers providing CDTI messages. Radio and television jingles are equally used to provide CDTI messages. However, educational materials were sometimes inadequate in the field.
The Mectizan@ supply is within the government system at all levels and is effective and uncomplicated. Supply was reported as good, and rarely were there reports of shortages received. When such reports were received, prompt action was taken. Supervision and training activities for CDTI (e.g. training and HSAM) are well integrated. However, integration of CDTI with activities of other disease control programs remains weak. This is because each program has its own system. The initiative for integration of CDTI and other disease control programs needs to come from the PHC Director.
To enhance efficiency, the State Onchocerciasis Control Team (SOCT) policy in the last two years has been to supervise and train staff only at the next level immediately below (i.e. LGAs), while the Local Government Onchocerciasis Control Teams (LOCTs) have been empowered to monitor, supervise and train FLtIFs staff. However, findings in the field show that there is a gap between this policy and practice; the SOCT and LOCTs at times supervise and train FLFIFs and CDDs. Many supervisory visits are made by the team of six SOCTmembers (6 including the state coordinator) in a month. These visits need to be coordinated and targeted. The roles and size of the SOCT also need to be reviewed in view of the project's policy to increase skills and empower LOCTs to manage CDTI in their LGAs.
The State has plans for replacement of vehicles, equipment, and materials and to provide motorcycles and bicycles to new LGAs and FLHFs, respectively.
All units visited had good therapeutic and geographic coverage. The state has consistently maintained greater than 65Vo therapeutic coverage in each of the 10 endemic LGAs since 2000, except for Ebonyi LGA which reported a therapeutic coverage of 63.77o in 2000. Overall therapeutic coverage for year 2OO2for the state was7l.L%o.
Most LOCTs are technically competent but are too dependent upon the SOCT for initiative to plan and implement annual activities. Very few LGAs have budgets, and the funds released in most cases have been erratic and small. Most LOCTs visited had a written plan containing the key CDTI activities, but no LGA had an overall health plan. Approximately one-half of LOCT staff seem to be transferred to other LGAs every year.
Despite a lack of resources for the CDTI project and delays in the payment of salaries, all LOCT staff expressed satisfaction with their involvement in the program and commitment to CDTI. Often, LOCT staff that have worked hard are commended for their efforts by the LGA during meetings. CDTI does not have an organized process for recognizing and awarding good perfonnance. The evaluation team recommends that such a system based on sustainable rewards (financial and non-financial rewards be initiated for individuals or institutions (e.g. FLF{F staff or village).
The evaluation shows that FLIIFs are the weak link in the CDTI chain, often seen as conduit or post-office. Staff at this level have not been empowered or encouraged to plan, though most are relatively skilled in training, HSAM, monitoring, and supervision for CDTI. Many FLI{F staff are relatively new to the program or to their present positions. Only in a few cases had FLIIF staff been in position for more than 4 years.
It has become an accepted practice by FLIIF staff to conduct training before every distribution cycle. Such training should be more targeted. There appear to be no specific guidelines on supervision which FLHF staff are expected to follow.
Community ownership/leadership is evidenced by selection of and changing of CDDs (in all villages visited). The time and mode of distribution were not always determined by the villages but were generally accepted. Some villages, however, expressed preference for different times from the period they currently receive treatment. Villages are also involved in encouraging compliance with Mectizan@ treatment, providing writing materials, registers and occasionally CDD compensation.
Village members expressed willingness to continue taking Mectizan@ annually, however long it takes and mentioned improvement in sight, clearer vision, passage of worms, and an overall feeling of being healthy as benefits of taking Mectizan@. In some villages, members are aware of the length of time they would be taking the drug while in some they are not.
CDD compensation is an issue everywhere, only about half of CDDs receive compensation from their villages. Recommendations from an independent monitoring in Ebonyi State emphasized the need to address this issue. In response to that recommendation and subsequent mobilization of communities, many villages have decided to give compensation, but the issue remains. CDDs were also directly supported in form of provision of land, meals, and cash. In some cases, villages assist CDDs in transport required to submit their reports, but in other instances, CDDs are left to make their own transport arrangements.
In some villages the CDDs carry out the census update during distribution while in others these activities are performed on separate occasions. CDDs know that the quantity of Mectizan@ collected from the Front Line Health Facilities (FLHF) is based on population data, but the calculation of tablets is not understood The ratio of CDD to the population ranges from l:200 to 1:1,500. Project staff indicated this reflected willingness of villages. In some villages, the settlement pattern is compact, and the CDD does not have to trek long distances. In other places, households are scattered, but the CDD rarely walks more than a 4-kilometer radius to treat village members. However, the evaluation team was informed by some CDDs that they spend long periods to provide treatment. Communities need to constantly be reminded of the potentials of decreasing workload through increasing the number of CDDs. The table below summarizes the scores for the groups of indicators at different levels. The State level has the highest average score while the FLTIF has the lowest. In terms of group of indicators at different levels, "Mectizan@ Supply" and "Coverage" score the highest while "Finances" scores the lowest.
o .o .= c o .o G .9. E z o o o- o o- o- o f f o U) o- a o 06 o- o= f - o at .g o) o oU c o o) L o -c c c o o TL o, o .E G o, o o (U ql I o N (U o o. c c o .= .E C o (U G c c o, (U c o (E G c o '6 o (U E g o o o c l o L I 5 5 l- c lr L T O On the overall grading of the project, the team found that six of the seven "aspects of sustainability" were helping or not blocking the project move towards sustainability; only resources at the LOCT and FLTIF levels and efficiency at all levels was seen as blocking. In relation to "critical elements", it was found that one (political commitment) were not satisfied at the LGA and lower levels but were satisfied at the state level. The Evaluation Guidelines indicate that where "one or two aspects are not fulfilled and one or two critical elements are not satisfied, the project is making satisfactory progress towards sustainability". The evaluation team therefore concludes that the Ebonyi State CDTI Project is making satisfactory progress towards sustainability. Feedback and Planning meetings were organized for state and LGA levels (in two groups of LGAs) Important issues emerging from SWOT analysis both at State and LGAs levels were used as frameworks for the development of sustainability plans. Ten LGAs and One State sustainability Plans were developed and endorsed. INTRODUCTION AND METHODOLOGY 1. Introduction EbonyiStatewascarvedoutoftheoldAbiaandEnuguStatesinoctober,lgg6.Compared to other states, it is a relatively new state. It is located in the south eastern region of Nigeria and is bounded on the north by Benue State, the east by Cross River State, the south by Abia State and the west by Enugu State. The topography of the state consists of an undulating elevation of approximately 500m above sea level in the south and extends to the rugged terrain of the central. The northern part is characterized by uniform landscape with intermittent escapement. The Cross River terrain is mainly an alluvia tableland, which favours most agricultural activities. The vegetation iurangement consists of mangrove forest, sparsely distributed around the Cross River basin at the boundary of the state, with a mixture of the deciduous and evergreen trees. The State is traverse by many rapid and fast flowing rivers and streams flowing into the Cross River and thus serves as a source of breeding for Simulium species (black fly). The state has an estimated population of 1.8 million based on the 1991 national population projections. The inhabitants of the state are the Igbos with over six different dialectic local languages and are mainly farmers with the majority living in rural areas. The state has a total of thirteen Local Government Areas (LGAs), ten of which are endemic for onchocerciasis. The endemic LGAs are Afikpo North, Ebonyi, Ezza North,Ezza South, Ikwo, Ishielu, lvo,lzzi, Ohaozara, and Onicha. The population update/registration shows that there are about 700,000 people living in onchocerciasis endemic villages in the state. Ivermectin distribution started in 1995, when Ebonyi State was part of the old Abia and Enugu States, with the support of Lions Clubs International (LCI) and Global 2000 River Blindness Program (RBP). The program later became known as the Global 2000 RBP Enugu/Anambra./Ebonyi States Project. Community Directed Treatment with Ivermectin (CDTI), supported with funding from the African Programme for Onchocerciasis Control (APOC), corrmenced in Ebonyi State in 1997 and covers the ten endemic LGAs. CDTI is a re-orientation from the Community Based Treatment with Ivermectin (CBTI) strategy pioneered by LCI and Global 2000 RBP. Ebonyi State is currently part of the APOC supported Global 2000 RBP Enugu/Anambra./Ebonyi States project and continues to benefit from the support of LCI. Ebonyi State has a total of 973 communities under CDTI. For the year 2002, a total of ' 4g2,i6o people were treated representing a therapeutic coverage of 7l.l%o and a geographic coverage of I007o. The project is in its fifth year of funding from APOC and is thus due for a sustainability evaluation as agreed by the NOTF representatives and APOC management during their meeting in Abuja, Nigeria in June 2002. 2. METHODOLOGY 2.1 Sampling The sampling was done purposively. The primary criterion used was therapeutic coverage since it is a measure of the performance of the whole system. Other criteria were also taken into consideration, for example levels of endemicity (hyper-endemic areas were preferred) and geographical spread (the sample contained areas that represented the different regions where the project operates). All the LGAs were equally moderately accessible; so accessibility was not taken into consideration. A peculiar feature of the project was the creation of new LGAs from the previous ones; these have been operational since 2OOl. Although these LGAs report separately, they are regarded as districts for convenience and consistency in records kept at the project office. However, in other respects, they are all treated equally as LGAs. Consequently, in the sampling, after the 3 LGAs for evaluation had been selected the 'sub LGAs' were also sampled where there were more than 2 for each old LGA taking into consideration the 'parent' LGA and therapeutic coverage. See tables below. S'N Main LGA New LGAs Geographic Endemicity Accessibili$ LGA overall Sampled? Location Therapeutic Coveraoe(%) 1 lkwo Central East Central Meso Moderate 75.2 Yes South East 2 lzzi Ebyia North East Meso Moderate 68.9 Yes Nnodo lzzi 3 Ohaozara Uburu South West Hyper Moderate 67.2 No Okposi Ugwulangwu 4 Onicha Oshiri South West Hyper Moderate 66.8 Yes lsu/Onicha 5 Ebonyi lshieke West Central Meso Moderate 68.7 No Ozibo 6 lshielu East North West Meso Moderate 68.5 No lshielu lnyaba 7 Afikpo Amoha South East Meso Moderate 72.2 No North Afikpo Ubeyi I Ezza East East Central Meso Moderate 68.8 No South South I Ezza West South West Meso Moderate 67.7 No North Umuoghara lmoha 10 lvo lshiagu South West Meso Moderate 74.9 No Akaeze SAMPLED SITES Main Local New Local Health Facility Village Remarks Government Area Government Area (therapeutic (therapeutic coverage) (therapeutic coverage) coveraoe) lkwo lkwo Central (74.50/0\ Ndufu Alike H/P Enyim Agalegu (80.7%) Health facilities with 1 - 3 (61,7y") Onyikwa (29.40/0) villages were not included in the sampling lkwo South (76.3%) Agalegu H/P (83.2%) Agubata (83.9%) Ohalekwe (57.5%) lzzi lzzi (70.30/o) Ndiegu Obovu H/C ohia (88.3%) As above (76 8%) Ochoboza (71.80/o\ Nnodo (67.7%) Oyege H/C (65.7%) Ndiepete (71.90/o) Ogbagharu (56.8%) Onicha Onicha (66.3%) Amas H/P (69.7%) Umuogodo (74.4V,) As above Umuigaga (58.4%) Oshiri(68.9%) OshiriH/C (66.8%) Agbabil (79.20/0) Owom (62.2%) 2.2 Levels and instruments Level Instrument used State 1 Local Government Area 2 FLI{F 3 Communitv 4 2.3 Protocol . Research question: How sustainable is the Ebonyi State CDTI project? . Design: Cross-sectional, descriptive. . Population: The Ebonyi State CDTI project, its NGDO partner (Global 2000/The Carter Center), the staff involved in onchocerciasis control in its LGAs and Front Line Health Facilities, the project communities, with their leaders and CDDs. . Instrument: x A record sheet, structured as a series of indicators of sustainability. The indicators were grouped into nine categories/ groups. These groups represent critical areas of functioning of the program. x The instrument assesses sustainability at four levels of operation. x The instrument guides the researcher to collect relevant information about each indicator from a variety of relevant sources. . Sources of information: * Documentary evidence and observations. * Verbal reports from persons interviewed. . Analysis: * Data from all sources are aggregated, according to level and indicator. x A qualitative summary of the situation regarding each indicator at each level is made. This is aggregated and summarized for each category of indicator for each level. l0 * Based on the information collectdd, each indicator is graded on a scale of 0-4 in terms of its contribution to sustainability. x The average 'sustainability score'for each group of indicators is calculated, for each level. * Finally an overall assessment of sustainability is made, by considering the 7 aspects and 5 critical areas of sustainability. . Recommendations: * These are strictly based on the findings of each program evaluated. 2.4 Team composition The core team members were the following: 1. Dr Eluether Tarimo (Team leader) * Box 33277 Dar es Salaam, Tanzania * T (home) +255 748 318574 x Email: [email protected] 2. Dr. Mary Alleman * 750 Commerce Drive, Suite 400 * Decatur, Georgia 30030 USA * T (home): l-404-371-1460; T (office): l-404-687-5633 * Email: [email protected] 3. Mr. Cyrille Evini * Helen Keller International, Cameroon * P.O. Box 14227, Yaound6, Cameroon * T (moblle):237 771.07.21; T (office): 237 220.97.71 * Email: [email protected], [email protected], evini [email protected] r 1 4. Mr Uwem Ekpo * Department of Biological Sciences, University of Agriculture, PMB 2240 Abeokta, Nigeria * T (mobile):234 803 335 1706 * Email: [email protected] 5. Mr Chukwu Okoronkwo x National Onchocerciasis Control Programme, Federal Ministry of Health, Room 915, Federal Secretariat, Phase II, Ikoyi - Lagos, Nigeria * T (office):234 I 4821285; T (mobile) 0803 361 9894 x Email: chukoro [email protected] Team members were grouped into three sub-teams for the purposes of field work. Each sub- team was accompanied by one or two SOCT members that served as guides, facilitators, and translators. ll 2.5 Advocacy visits and 'Feedback/Planning' meetings Advocacy visits were paid to relevant people at each level as possible. Debriefing was done at the feedback and planning workshops. Finally, workshops were conducted for relevant officials at the state and LGA levels. During these workshops, the evaluation team gave feedback on its findings and guided the state and LGA teams in the development of 3 - Year post APOC sustainability plans taking into consideration the evaluation findings. 2.6 Limitations The major limitation of the evaluation was the inability of the team to access and determine the score for the therapeutic coverage at community level. The evaluation team discovered that this indicator was biased due to the criteria set for selection of communities as well as LGA and FLI{F for sampling. Ochoboza village was substituted during the field visit phase of the evaluation due to inaccessibility be the evaluation team. Reports of annual financial expenditures (APOC and MOH) by item were not available for review; thus financial efficiency of the project could not be assessed. Intregation seems to be limited to "internal" integration of CDTI. There is a need for a separate indicator related to integration of CDTI with overall PHC. No indicators for "leadership" at State level. Thus no provision is made for assessing this important aspect of sustainability. t2 3. EVALUATION FINDINGS AND RECOMMENDATIONS 3.1 State Level State Level 4 3 2 1 0 .r""" ."c J* *,""= tt""."t" "-"* -.O *op "-" ^-.* "..".*" {'"" ..c Group of indicators PLANNING (Highly 3): The control of onchocerciasis in Ebonyi State is now an integral part of the overall health plan for the state. The state has a Rolling Health Plan for the period 2OO3 - 2005. The Onchocerciasis Control Program is included in this plan as a line item. This means that there will be budgetary provision for onchocerciasis control annually by the state government. In addition, an item, "Onchocerciasis Control Program", with a budget appears in all the Annual Estimates of Ministry of Health (MOH), 2OOO-2003. There is at the State level detailed annual year plan of CDTI activities. For 2003, the plan contains responsible institutions, time schedules, expected outcomes, indicators, cost, and sources of funds. The MoH and GRBP jointly prepared the plan based on agreed areas of support of each partners (APOC/NDGOAvIoH). A specific, "Ebonyi State CDTI Project 2004 - 2006 Sustainability Plan of Action" has been prepared, considered, and approved by the state health management in collaboration with the NGDO. Though well presented, it appears that the LGAs were not actively involved in its development. It is recommended that in the future CDTI planning begin at the LGA level and then progress to the state. 13 MONITORING/SLIPERVISION (Moderate 2.3) : Prior to 2OOl, SOCTs routinely supervised LOCTs, FLFIFs, and even CDDs. In order to enhance efficiency, a policy was devised whereby the SOCT should only supervise the next level immediately below (i.e. LGAs) while LOCTs have been empowered to monitor and supervise the FLIIFs. However, findings in the field show that there is a gap between this policy and practice; the SOCT and LOCTs very often supervise FLFIFs and CDDs. It is recommended that these levels carry out spot checks, if necessary. There are several visits to each LGA in a month. These visits need to be coordinated and targeted. Supervision of other health programs, though not planned, may take place in the field as problems arise. A supervision checklist exists but has no space to indicate findings. The SOCT takes prompt action on problems encountered in the field. Examples of action include investigation and remedial action where it was suspected that quantities of Mectizan@ used were more than expected. There is no process of rewarding successes at the state level. MECTIZAN@ PROCUREMENT AND DISTRIBUTION (Fully 4): Mectizan@ supply is within a government system. Global 2000 takes delivery of Mectizan@ from the NOTF storage facility in Lagos to Enugu. The SOCT collects the Mectizan@ and brings it to Abakaliki from where LOCTs can collect it during training or otherwise. The system is effective, uncomplicated, and efficient. TRAINING AND HSAM (Highly 3): Training by the SOCT in the last two years has moved from training of staff at all levels to training of staff only at the LGA level. This approach enhances efficiency and sustainability. The evaluation team was informed that training is targeted to weak areas based on the results of the supervision checklist used by the SOCT. However findings from the field show that training is routine. A similar approach is used to identify areas for HSAM. SOCT staff review past activities to identify problems that need mobilization and to develop HSAM specific activities to address problems identified. HSAM is an area of great success with extensive activities. Innovations have been made in the designs of posters, calendars, brochures and fliers providing CDTI messages. Radio and television jingles are equally used to provide CDTI messages. INTEGRATION OF SUPPORT ACTIVITIES (Highly 3): Supervision and training activities for CDTI (e.g. training and HSAM) are well integrated. However, integration of CDTI with activities of other disease control programs remains weak. This is because each program has its own system. The initiative for integration of CDTI and other disease control programs needs to come from the PHC Director. t4 FINANCIAL (Highly 3.2')z An adequate budget for onchocerciasis control activities is provided for and disbursed by the state and APOC. The evaluation team was pleased to note the high political commitment to CDTI by the State. The team hopes that this commitment will be maintained as APOC support declines. The table below shows funds released in recent years: State and APOC Contribution Year APOC (US State (US Dollars) Dollars) 1999 * 30,000 2000 * 32,400 2001 * 30,000 2002 29.484 37,000 * = No information as funding was combined for three States (Anambra/Enugu/Ebonyi) Global 2000 is responsible for approval, accounting, and audit of APOC funds. The control system is adequate and efficient. For state funds, the supervising ministry uses the government control system. There is no coordination between the accounting of funds by APOC and the MOH. TRANSPORT AI\D MATERIAL RESOURCES (Fully 3.6): The SOCT has one 4WD vehicle, two desktop computers, a fax machine, a photocopier, TV, video recorder, generator, and overhead projector. Most items are adequate. All transport and office equipment are well maintained. The SOCT 4WD vehicle is used for activities at the project headquarters and to provide support for the next level. Plans for replacement of vehicles, equipment, and materials exist and efforts have been made to secure approval. HUMAN RESOURCES (Fully 3.6): The SOCT has a motivated team of six technical staff headed by the State Coordinator. Five SOCT members have been with the project since inception. The sixth member joined in 2001 and has been trained on CDTI activities. COVERAGE (Futly 4): The state has consistently maintained greater than 657o therapeutic coverage in all the l0 endemic LGAs since 2000, except for Ebonyi LGA which reported a therapeutic coverage of 63 .7 Vo in 2000. Overall therapeutic for the coverage for year 2002 w as 7 I .l%o . 15 Recommendations for the State level Recommendation lmplementation Planning: Priority: HIGH Indicators of success: The planning process for CDTI should begin with Sustainability state plan are complimentary and development of LGA plans. These plans should indicate suoportive of LGA plans clearly the role and the activities at the communities, FLHF Who to take action: and LGA levels. Cost and sources of funds should be SOCT and LOCT clearly indicated. The State level plan should evolve from D eadline fo r c omplet io n : plans and be supportive to the LGAs October 2003 Monitoring and Supervision: PriOriN: MEDIUM Indicator of success: Supervision by SOCT should not be routine. It should be Clear policy and guidelines on supervision, Checklist targeted and coordinated. reviewed Review of supervision checklist. Who to take action: SOCT Deadline for completion: October 2003 Priority: MEDIUM There is the need to develop a process and mechanism for Indicator of success: recognizing and rewarding for good performance, which is Rewards and commendation of performance instituted essential for sustainability. These rewards could be Who to take action: financial or non-financial. Unsustainable rewards should be SOCT avoided. Deadline for completion: December 2003 Financing: Priority: MFIDIUM Indicator of success: There should be coordination between the accounting and More coordination between partners in accounting of auditing of funds provided by APOC and MOH for the funds CDTI project. Who to take action: SOCT and NDGO D eadline fo r c ompletion : December 2003 PriOriN: IIIGII A system aiming ensuring accountability should be put in Indicator of success: place at State level (expenditure by line items) Availability of expenditure documentation by line item Wlrc to take action: SOCT, MoH D eadline fo r completion: November 2003 Human Resources Priority: IIIGII The roles and size of the SOCT should be reviewed in view Indicator of success: of the project's policy to increase skills and empower Roles of SOCT are clear and its size reduced LOCTs to manage CDTI in their LGAs Who to take action: SOCT, MoH D eadline for compl etion: November 2003 Leadership: Priority: HIGH Indicator of success: SOCT should empower LOCTs to initiate and carry out CDTI activities are initiated bv LOCT each year. CDTI activities Who to take action: SOCT, LOCT D eadline for contpletion : November 2003 l6 3.2 Local Government Area Level Scores at LGA level 4 3 o)E oo 1 0 nd "u"u -.."'u "-c ^.-- .*.J .i*s- JAo ^"/ n.""d .o-" Groups of lndicators PLANNING (Moderate2\z Most LOCTs visited had a written plan containing the key CDTI activities, but no LGA had an overall plan that incorporated CDTI into all of the health activities of the LGA. There was no evidence that the written plans were drawn up in a participatory way. Planning for CDTI seems to stand on its own and as mentioned above is not incorporated into overall health plans of the LGAs. Reasons are: o PHC coordinators and LOCT coordinators are transferred often and were new to the LGAs included in this evaluation. o Each disease control program in the LGA is independent and has its own sponsor and therefore its own coordinator, schedule of activities, and resources. o LGAs did not seem to see a need to have an overall plan for health activities. Some LGAs expressed that in the upcoming years they will make an attempt to coordinate LEADERSHIP (Mod erate 2)z Each LGA has a LOCT consisting of a coordinator and several additional people. LOCTs take responsibility for initiating and carrying out CDTI activities at their level and below but are highly dependent upon the SOCT for triggering annual activities and providing important resources such as educational materials, Mectizan@, bicycles, motorbikes, etc. Some LOCTs indicate that they will attempt to secure funds at the LGA level to gain some autonomy. . " t] STJPERVISION/NIONITORING (Moderate 2.3)z Frequency and formatting of reporting by LOCTs are inconsistent, but when written, the LOCTs' reports are sent to LGA staff and the SOCT. In some cases, the PHC coordinator does not receive written reports from the LOCT about CDTI activities. All LOCTs report supervising at the FLIIF and village levels. Most said that they go to FLI{F and village levels as often as once/week during distribution while others supervise less frequently. While not standardized or formal, the LOCTs may monitor for other disease control programs while on supervisory visits for CDTI; as well, transportation devices granted to other disease control programs might be used for supervision for CDTI. It was noted that an effort to integrate disease control program activities would improve the situation. LOCTs take an active role in problem solving at both the FL[{F and village levels. Problems have included refusals and CDD compensation. The LOCT has dealt with these in conjunction with FLI{F staff and villages. Recommendations from a previous monitoring exercise in Ebonyi State, indicated that the issue of CDD compensation needed to be addressed. In response to that recommendation and after intervention by the LOCT, in some cases, villages have decided to give compensation. While the problem solving process seems effective, it is not always documented, and formal feedback is not provided to lower levels. Program successes are sometimes commended by LOCT at the FLIIF and village levels. MECTIZAN@ SUPPLY (Highly 3): In all years, sufficient Mectizan@ has been provided to the LOCTs by the SOCT and in good time. Rarely were there reports of shortages; when they were reported, the LOCT rectified shortages by promptly contacting the SOCT for additional supplies. Requests for Mectizan@ are based on total population or eligible population figures gathered from the census or on the number of people treated in the previous year. Usually, Mectizan@ is brought to the LOCT by the SOCT during training or other visits. In some cases, the LOCTs go to the SOCT to collect their drug. If additional quantities of Mectizan@ were needed, the LOCTs said that they would contact the SOCT for additional supply, although such a need was rarely reported. TRAINING/HSAM (Moderate 2): Some LOCTs train only FLIIF staff while some others report also training CDDs. Training by LOCTs, in conjunction with the SOCT, is routine and appears to occur annually using the same materials each year. In most LGAs, the LOCTs, in conjunction with the SOCT, provided the LGA decision makers with information about CDTI and with an outline of finances needed from the LGA. In a few cases, the LOCTs did not feel the necessity to approach decision makers due to their l8 apparent lack of interest and their lack of response in the past. LOCTs seem to carry out HSAM activities when there is new leadership at the LGA. The LOCTs did not report carrying out HSAM at levels lower than the LGA. In a number of LGAs, there had been a recent turnover in the LGA leadership; thus it was difficult at the time of this evaluation to assess the HSAM's impact. FINANCING/FUNDING (Slightly 1) : In very few cases have budgets been relevant since LOCTs say that if funds were released they would be a fixed amount rather than an amount based on need. The LOCTs do not know the amount of funds that would be released in any given year. Few LOCTs interviewed had received funds from their LGAs. Cost reduction/containment strategies have not been devised by the LOCTs since the release of funds has always been so improbable and sporadic. LOCTs plan to analyse their financial needs and intensify advocacy to obtain funding for CDTI. In the few cases where budgets do exist, only the contribution of the local government is clearly spelled out. Amounts budgeted in most cases have been erratic and small. Management teams at the LGA level are aware that funds for CDTI activities are not forthcoming but have made no plans to obtain funds from other sources. As described above, there are very little funds available at the LGA level to carry out CDTI activities. However, where funds are available, they are used according to LGA financial rules. TRANSPORT AND OTHER MATERIAL RESOURCES (Slightly 1.5): There are some motorcycles available for some LOCTs, but those provided are not suitable for the terrain which will shorten their life spans and create a need for replacement in less than two years. These original motorcycles were bought without consultation with end users prior to purchase. Inadequacy in quantity of motorcycles in some LGAs could be due to creation of new LGAs or due to lack of support from LGAs. Motorcycles supplied for other disease control programs are frequently used for CDTI activities and visa verse. Educational materials are available at the LGA level if they remain in place when LOCT staff are transferred and may suffice for some years to come. LOCTs continue to request financial support for these materials and advocate for CDTI at the LGA level. Running costs and maintenance of motorcycles are mainly met with the personal resources of LOCT staff. Where available motorcycles break down, staff at this level make use of commercial transport. Authorization for use of motorcycles is not required since they are generally allocated to individuals rather than programs. Considering the circumstances in the field, the transport is well managed and shared. l9 LOCTs are aware of the need to replace existing materials and motorcycles in the future. LOCTs say that they will approach the LGAs for support for these items. HIIMAN RESOURCES (Moderate 2.5): More than half of the LOCT staff were judged as being knowledgeable and skilled enough to undertake CDTI planning, training, HSAM, monitoring, supervision, and Mectizan@ ordering. Approximately one-half of LOCT staff seem to be transferred to other LGAs every year. Transfers are government policy and may affect skills and knowledge of LOCT staff; annual CDTI training sessions at LGA level each year should assure that new staff are informed and trained to carry out CDTI effectively Despite lack of resources for the program and delays in the payment of salaries, all LOCT staff expressed satisfaction with their involvement in the program and commitment to CDTI. Often, LOCT staff that have worked hard are commended for their efforts by the LGA during meetings. COVERAGE (Moderately 2.5): Available records indicate that all hyper- and meso-endemic villages are covered with mass treatment. However the team was informed that there have been rare instances where villages have not been covered due to CDD attrition. In general, village-level therapeutic coverage is good. In 2000, 200I, and 2002 the proportion of villages with greater than 657o therapeutic coverage was 747o,727o, and'73Vo, respectively, in the three LGAs sampled. 20 Recommendations for the Local Government Area level Recommendation Implementation Planning: Priority: MEDIaM Indicators of success: LOCT staff skills should be developed in a planning LOCT staff are capable of drawing up plan for workshop. CDTI activities. Who to take action: SOCT Deadline for completion: October 2003 LOCTs with other LGA staff (PHC coordinator, Prioritv: HIGH counsellor for health, etc.) should in a participatory Indicators of success: way draw up integrated plans and budgets for the LGA health plan with budget that integrates all health programs to be implemented at this level. disease control Drosrams is available. Who to take action: PHC coordinator and LGA health counsellor Deadline for completion: October 2003 M o nitoring and S up e rv is io n : Prioritv: MEDIUM Reporting procedures should be developed: Indicators of success: Check list is being used by LOCT. o LOCTs should routinely use a check list for Supervision calendars being used by LOCTs. targeted supervision Written reports always submitted afier supervisory o Supervision calendars should be drawn up and visits. distributed to CDTI implementers FLHF take action based on feedback. o All superttisory visits should result in written Who to take action: reports which should be given to PHC coordinator, LOCT SOCT, and provided to FLHF feedback Deadline for completion: Januarv 2004 Priority: MEDIUM . Efforts should be made to carry out integrated Indicators of success: supertisions Inteerated suoervisions are carried out bv LOCTs. Who to take action: LOCT Deadline for completion: Januart 2004 Training/HSAM: Priority: MEDIUM Indicators of success: LOCT should only train FLHF staffin CDTI Directives to LOCT Who to take action: SOCT Deadline for completion: Januarv 2004 Priority: MEDIAM Training of FLHF should target needed skills and Indicators of success: knowledge LOCTs tailor trainins asenda to traininp needs. Who to take action: LOCT Deadline for completion: January 2004 2t Recommendation Implementation Priority: HIGH tlJAM snould De proper|y pLanned to aclclress tssues Indicators of success: relating to program implementation and most LOCT secures financial support for CDTI imp o r t antly fund r ai s in g activities Who to take action: LOCT Deadline for completion: October 2003 Finances: Priority: HIGH Indicators of success: LOCTs slnuld intensify advocacy towards LGA authorities LOCT secures financial suDDort for CDTI activities with regards to funding CDTI activities. Who to take action: LOCT Deadline for completion: October 2003 22 3.3 Front Line Health Facility Level Sustainability at Front Line Health Facility level .'."" """ -*".] .*ff -y '"""" Group of indicators PLANNING (Slightly 1): There are written plans in some FLFIFs for 2003, but staff are not conversant with the content of such plans. In other FLIIFs, the plans are not written, but staff are knowledgeable about the various CDTI activities to be carried out and when they are to be performed during the year. Those in charge of CDTI are aware that it is part of their duties, but not all competent staff at the FLFIF are involved in CDTI activities. There are no plans for overall health activities at this level because FLIIF staff at this level have not been empowered and encouraged to plan. .. LEADERSHIP (Mod erate 2): Training and Mectizan@ distribution are dependent on information from and activities by the higher levels. This is accepted as the normal system, and there has been no real effort to empower FLI{F staff to take initiative in carrying out CDTI activities. _ 23 MONITORING AND SUPERVISION (Moderate 2): The reporting process is within the government system, but records are not kept at this level. Where supervisory visits are paid, they are not targeted at villages where there are proven problems. Some FLFIF staff pay several visits to the villages while some visit few villages, and some no villages at all due to logistical problems. FLFIF staff rarely turn to other health activities while on CDTI supervisory visits. Problems identified or reported to FLTIF staff are dealt with in conjunction with village leadership, but issues of CDD compensation earlier highlighted still linger. Villages and CDDs doing well with regard to CDTI are encouraged, but this is rarely reported. There appear to be no specific guidelines on supervision which FL[{F staff are expected to follow. MECTIZAN@ PROCUREMENT AND DISTRIBUTION (Moderate 2.5): Requests for Mectizan@ are based on total population or eligible population figures gathered from the census or on the number of people treated in the previous year. Except for a few cases, there are no reports of shortages or late supply at the village level. Mectizan@ distribution takes place entirely within the government system. Program inventory forms exist which track the movement of Mectizan@ from the FLFIF to villages, but these are not properly used in most cases. TRAINING AND HSAM (Moderate 2): CDDs are routinely trained each year before distribution at the FLHF. The training agenda does not appear to focus upon specific issues that arose during previous rounds of treatment. While the period and site of training seem appropriate, the ratio of trainer to trainees is high. FLHF staff have identified problems of refusals and lack of CDD compensation as issues affecting the implementation of CDTI and have made efforts to address them. There are however few cases where these have led to positive changes. It has become an accepted practice by FLFIF staff to conduct training before every distribution cycle. Although FLHF staff are generally capable of conducting the CDD training it has also become accepted practice that LOCTs are involved. FINANCIAL (Slightty 1): The FLFIF does not formally budget for any health activity. However, in some of the written plans that were seen, there were projected expenditures indicated next to planned activities. No funds are released directly to the FLHF for CDTI or other health activities. 24 . - TRANSPORT AND OTHER MATERIAL COSTS (Slightly 1): No official transport exists at this level. FLIIF staff use commercial transport and pay from personal resources to carry out supervision at the village level. IEC materials were insufficient. -. HUMAN RESOLIRCES (Moderate 2)z Most FLF{F staff are relatively skilled in training, HSAM, monitoring and supervision but appear deficient in planning and Mectizan@ ordering/distribution. Most FLIIF staff are relatively new to the program or to their present positions. Only in a few cases had FLIIF staff been in position for more than 4 years. The Local Government Area authorities routinely and often transfer FLIIF staff. New FLIIF staff are trained by the LOCTs during the annual training sessions. - - COVERAGE (Highly 3): - - Available records indicate that all hyper- and meso-endemic villages are covered with mass treatment. However the team was informed that there have been rare instances where villages have not been covered due to CDDs refusal to carry out distribution for lack of compensation. 25 Recommendations at the Front Line Health Facility level Recommendation lmolementation Planning: Priority: HIGH Indicators of Success: 1. The Project should organize management 1. Skills of FLIIF staff built in areas identified training for FLIIF staff involved in CDTI with above emphasis on integrated planning, targeted 2. Number of FLTIF staff trained and involved supervision, record keeping, & Mectizan@ in CDTI activities ordering (especially Mectizan@ inventory). Wo to take action: SOCTs, LOCTs, PHC Coordinator 2. All trained FLHF staff need to be trained on D e adline for completion : see as part CDTI and encouraged to it of their 1. February 2004 routine duties 2.March20[,4 Leadership: Priorin: HIGH Indicators of Success: Higher levels should empower and encourage FLIIF staff initiating and carrying out CDTI staff at this level to initiate activities activities annuallv particularly with respect to planning and Who to take action: conducting CDD training with little or no input LOCTs, PHC Coordinator from the LGA. Deadline for completion : From next distribution cycle Training/HSAM: Priority: HIGH Indicators of Success: l. Training of CDDs should focus on identified 1. Number of new CDDs/existing CDDs needs (lack of skills in certain areas or newly lacking in specific skills trained selected CDDs) 2. FLI{F staff planning and conducting CDD trainins 2. LOCTs should empower and allow FLIIF Who to take action: staff to plan and conduct CDD training. LOCTs, PHC Coordinator D e adline fo r c ompl etion : From next distribution cycle and subsequently Transport and other material costs: Priority: MEDIUM Indicators of Success: Project should supply adequate quantities of Number of IEC materials supplied/ produced IEC materials to this level or mobilize LGA and available at the health facilities authorities to produce needed quantities. Who to take action: SOCTs, LOCTs, PHC Coordinator D e adline for completion : Februarv 2004 Human resources: Priority: MEDIUM Indicators of Success: See recommendation under planning 2. Plan in place and number of new staff that Dl.-. .}.^',1,{ L- r^ i--^,{i receive immediate orientation on CDTI -.,{- ^-^',.i,1- Who to take action: orientation on CDTI to new staff. LOCTs, PHC Coordinator De adline fo r completion : LOCT leader 26 -. 3.4 Community Level Sustainability at community level ^..tt'" €$ a..o"" """ ,."""- ""t" .r"".. .t"'" a."" "-.."t "u,"-'"* Group of indicators PLANNING AND MANAGEMENT (Highly 3): In villages where house to house distribution is used, CDDs choose times and routes convenient to him, although there are complaints that the time used for distribution could have been used for private pursuits. Where centralized distribution is carried out, treatments are at times set out by village chairmen. In some villages the CDDs carry out the census update during distribution while in others these activities are performed on separate occasions. Where there are problems of refusals, village leadership has encouraged compliance with treatment. LEADERSHIP AI\D OWNERSHIP (Highly 3.3): Village leadership is involved in making announcements of arrival of drugs, encouraging compliance with Mectizan@ treatment, providing materials and occasionally CDD compensation. 27 In all cases the distributors were selected or changed by the village leadership. The time and mode of distribution were not always determined by the villages but were generally accepted. Some villages, however, expressed preference for different times from the period they currently receive treatment. Village members express willingness to continue taking Mectizan@ annually and mention improvement in sight, clearer vision, passage of worms and overall feeling of being healthy as benefits of taking Mectizan@. In some villages, members are aware of the length of time they would be taking the drug while in some they are not. MONITORING (Highly 3): CDD reports or registers are received generally on time. In some cases, villages assist CDDs in transport required to submit their reports, but in other instances, CDDs are left to make their own transport arrangements. MANAGING MECTIZAN@ (Highly 3): Usually enough Mectizan@ is collected for all eligibles. Absentees who actively request Mectizan@ after the time of distribution in the village are asked to wait till the next treatment cycle or referred to the FLIIF. Absentees who do not actively seek treatment may not receive follow-up by the CDD. The CDDs know that quantity of Mectizan@ collected from the FLFIF is based on population data, but the calculation of tablets is not understood CDDs collect their Mectizan@ from the FLIIF or the LGA HQs, whichever is closer. In some places, they are assisted with transport by the village leadership while in others they used their private resources. HSAM (Highly 3): Village leadership usually takes on the responsibility of informing and mobilizing the village members of the arrival of Mectizan@ tablets and of occasionally encouraging compliance in the face of refusals. Village members are rarely encouraged to provide resources to offset local costs of distribution. FINANCING (Moderate 2): In many cases, the village provided writing materials and registers. CDDs were also directly supported in form of provision of land, meals, and cash. In every place, and at all levels, the issue of CDD compensation (financial) was expressed as an issue of serious concern. The support seems to be erratic. 28 HUMAN RESOURCES (Moder ate 2.6) z The ratio of CDD to the population ranges from 1:200 to 1:1,500. In some villages, the settlement pattern is compact, and the CDD does not have to trek long distances. In other places, households are scattered, but the CDD rarely walks more than a 4 kilometer radius to treat village members. All CDDs interviewed appear relatively skilled in the distribution of Mectizan@ but are deficient in census taking. New CDDs are trained during the annual CDTI training by FLIIF staff. If necessary, new CDDs can be immediately oriented so to begin CDTI activities prior to formal training. All CDDs interviewed expressed a strong willingness to continue despite the lack of compensation received from their villages. COVERAGE (Hiehly 3): Most households appear to be covered with regard to treatment. The proportion of the 12 villages included in this evaluation with greater than 657o coverafle was 10/12 (84Vo),8112(677o),and6112(507o) in 2000,2001,and2002, respectively. These results can not necessarily be taken as a trend towards decreasing coverage since villages included in this evaluation were purposefully chosen because of high coverage in 2002 and others purposely chosen because of low coverage in2002; thus, the indicator for therapeutic coverage could not be scored as described in the evaluation instrument. The 2002 therapeutic coverage for the 12 villages ranged from29.47o-88.37o. 29 Recommendations for the Community level Recommendations Implementation Planning and Management: Priority: MEDIUM Indicators of Success: CDDs need to be encouraged to carry out CDDs carrying out census updates during census update during distribution. distribution Who takes action: FLHF, Villaee leader D e adl ine fo r c ompl e tion : By next distribution and subsequentlv Leadership and Ownership: Prioity: HIGH Indicators ofSuccess: Village leadership has to be mobilized to Number of villages where village members are encourage and promote increased participation actively involved in supDort of CDTI activities. of village members in the CDTI process Who takes action: FLIIF, Villase leader D e adline for completion : April2004 Managing Mectizan@: Priority: NIEDIUM Indicators of Success: 1. Mectizan@ should be left at the village level 1. Proportion of absentees that were treated 2 for a period of at least 3 months, and plans months after distribution have to be made for mop up treatments to take care of temporarily ineligibles and absentees. 2. Number of CDDs able to determine accurately ouantitv of Mectizan@ needed bv their villases. 2. The capacity of CDDs needs to be built in Who takes action: the correct determination of accurate quantities FLI{F, Village leader, CDD of Mectizan@ that will be required. D e adline fo r c ompletion : 1. From the next distribution 2.March2OO4 HSAM: Priority: HIGH Indicators of Success: Plans need to be made to mobilize villages Plan in place and number of villages providing effectively to support local costs of suDDort for local costs of distribution distribution. Who to take action: FLIIF, Villaee leader D e adline for completion : November 2003 Priority: MEDIIJM Indicators of Success: Credible Community Based Organizations Number of CBOs and associations actively involved (CBOs), religious groups and town based in CDTI and suonortins CDDs. village associations and age groups can be Who take mobilized to provide some support for CDDs. to action: FLHF, Village leader, LOCT leader D e adl ine fo r c o mpl et ion : December 2003 Human resources: Prioritv: HIGH Indicators of Success: 1. Project should encourage and mobilize 1. Number of new CDDs selected and villages to select more CDDs, possibly participating in CDTI along kindred lines. 2. Skills of CDDs updated in census taking, and reliable census data available 30 Recommendations lmnlementation 2. Skills of CDDs on census updating Who to take action: should be sharpened to ensure that correct FLHF, LOCTS information is obtained by the health D eadline for completion : system February 2004 March 2004 3l The table below shows scores by level and by group of indicators. The State Irvel has the highest scores while the FLTIF has the lowest. In terms of groups of indicators, Mectizan@ and Coverage have the highest scores while "Finances" and "Transport" have the lowest, because of the particularly low scores at LGA and FLIIF levels. Scores at all bvels la -e "od -&' .*t'e ,*C ...-' -"'" J ''c ,oo-'€- ,o" od" $d a"" tE=EG----t "a" "r..oo"". at"" lrru I lorrnr I c co*-rv .oo'trow of lndlcators lo I 32 4. CONCLUSIONS 4.L Grading the overall sustainability of the Ebonyi State CDTI project, aspects and critical elements A judgment of the sustainability of the Ebonyi State CDTI was made according to the following "aspects of sustainability" and "critical elements". Aspects of Sustainability : Aspect Judgment: to what extent is this aspect helping or blockine sustainabilitv of this proiect? Integration Not blockins Resources Very much helping at SOCT, Currently blocking at LOCT and FLHF Efficiency Blockins Simplicitv Very much heloinp Attitude of staff Very much heloins Communitv ownership Heloinp Effectiveness Vem much helpinp Integration: Not blocking sustainability Integration at the SOCT level was deemed fair. The beginnings of integration of CDTI activities with those of other disease control programs are seen in the sharing of vehicles, informal assistance with monitoring and supervision, and the use of SOCT staff in immunization activities when needed. At the level of state government, there is some integration of CDTI with other health related activities as part of the state's planning process. There is some sharing of transport and perhaps passive supervision and monitoring at the LOCT level, but there is no formal integration of CDTI activities with those of other disease control programs. Integration is similar at the FLI#, but there is no shared transport. While integration for the project overall was judged to be fair, the lack of integration was not seen as a blocking CDTI activities presently. In the future, the lack of integration could block the potential to make the most of scarce resources, especially at the LOCT and FLFIF levels. Resources: Very much helping at SOCT, Currently locking sustainability at LOCT and FLHF Sufficient resources are available at the level of the SOCT. The project should be sustainable at the state level if the government or other similar and reliable source can maintain the current situation. Resources for CDTI are not nearly as plentiful at the LOCT or FLIIF levels. Sustainability would be more obtainable here, if the state shared its resources and assisted the LOCTs with advocacy to LGA decision makers. Efficiency: Blocking sustainability The financial efficiency of the project overall could not be assessed as records of financial expenditures were not available to the evaluation team. The project was judged to be inefficient in terms of the use of personnel. Supervision and monitoring were found to be excessive, especially with regard to those carried out by the SOCT at all lower levels. Moreover, training activities in the field tended to involve a high trainer to trainee ratio. Simplicity: Very much helping sustainability With the exception of complications in the release of money at the LGA level, the project operates with a strong element of simplicity. Attitude of Staff: Very much helping sustainability Overall, the attitude of staff working in CDTI at all levels was positive. The greatest level of commitment was seen at the level of the SOCT with LOCT staff and FLTIF staff being progressively less committed. It was noted that LOCT and FLIIF staff often work without being paid salaries, and they are transferred often, both of which may negatively impact their commitment to CDTI. The communities expressed a strong commitment. The evaluation team proposed that the commitment and ownership of CDTI at the LOCT and FLI{F might be enhanced through management training, by involving the PHC coordinator in CDTI, by giving FLHF staff job descriptions, and advocating for budget lines for FLFIFs. Community Ownership: Helping sustainability The evaluation team observed that villages are playing a role in CDTI. They are selecting CDDs, changing CDDs when necessary, and some times providing compensation to CDDs. Moreover, the CDDs are committed. The villages included in the evaluation in Ebonyi State are moving towards ownership; the project needs to do all that it can to maintain the demand for Mectizan@ among village members so that full ownership will be realized. Effectiveness: Very much helping sustainability The project is seen as being effective. The project is achieving its overall goal to cover 1007o of meso- and hyper-endemic communities with at least 65%o therapeutic coverage. The project needs to work to maintain this success for the years to come. 34 Critical Elements of Sustainability: Critical Element YesA.{o Money: Is there sufficient money available to undertake strictly Yes necessary tasks which have been carefully thought through and planned? (absolute minimum residual activities). Transport: Has provision been made for the replacement and Yes repair of vehicles? Is there a reasonable assurance that vehicles will continue to be available for minimum essential activities? (note that 'vehicle' does not necessarily imply '4x4' or even 'car'). Supervision: Has provision been made for continued targeted Yes supportive supervision? (the project will not be sustained without it). Mectizan@ supply: Is the supply system dependable? (the Yes bottom line is that enough drugs must arrive in villages at the time selected by the villagers). Political commitment: Effectively demonstrated by awareness Yes at state, No at LGA, of the CDTI process among policy makers (resulting in tangible Moving towards support); and a sense of community ownership of the ownership at villages prograrnme. Money: Yes The project was judged to have enough money to carry out the absolutely essential activities of CDTI. Transport: Yes As according to the budget submitted to government, provision has been made to secure enough motocycles and bicycles for SOCT, LOCT and FLFIF. Supervision: Yes The project has made provisions to continue supervision necessary for CDTI. Mectizan@: Yes The supply system for providing Mectizan@ to the villages is simple and reliable, and Mectizan@ is available in villages early in the year in time for planned distribution. 35 Political Commitment: Yes at state level, No at LGA level, Moving towards a degree of ownership at village level Political commitment is strong at the state level. In contrast, the commitment by the governments of the LGAs is and has been weak. Villages are moving towards ownership of CDTI. 4.2 Grading of project as a whole, conclusion On the overall grading of the project, the team found that six of the seven "aspects of sustainability" were helping or not blocking the project move towards sustainability; only resources at the LOCT and FLFIF levels and efficiency at all levels was seen as blocking. In relation to "critical elements", it was found that two (transport and political commitment) were not satisfied at the LGA and lower levels but were satisfied at the state level. The Evaluation Guidelines indicate that where "one or two aspects are not fulfilled and one or two critical elements are not satisfied, the project is making satisfactory progress towards sustainability". The evaluation team therefore concludes that the Ebonyi State CDTI Project is making satisfactory progress towards sustainability. 36 ANNEXES 3t INTBRVIEWS ProjecUState Level 1. Mrs C. Maduka Project Administrator, Anambra./Enugu/Ebonyi CDTI Project 2. Mr S. Orogwu State Coordinator, Ebonyi CDTI Project 3. Dr I. Echiegu Director, Primary Health Care (PHC) 4. Dr Gabby Idam Former Commissioner of Health 5. Chief F. O. U. Mbam Permanent Secretary, Ministry of Health 6. Mr K. C. Obini Director, Personnel Research Statistics (PRS) 7. Mr O. A. Uduma Head of Department, Finance and Supplies, Ministry of Health (Project Accountant) LGA Level 1. Hon. Dan Ogiji Chairman,Ikwo Central LGA 2. Elder (Mrs.) Oboke Asst. PHC Coordinator, Ikwo Central LGA 3. Mrs. Theresa Ayama LOCT Leader,Ikwo Central LGA 4. Chinyere Olele Asst. LOCT Leader, Ikwo Central LGA 5. Jacob Anyigor LOCT Leader, Ikwo South LGA 6. Mrs. Jane Nworie HOD, Health, Ikwo South LGA 7. Hon. Obasi Nwode Vice Chairman, OhaozaraEast LGA 8. Mr. K. O. Elem Head of Personnel Management (I{PM), Ohaozara East LGA 9. Hon. Neni Nnennaya Okoro Supervisory Councillor, Health, Ohaozara East LGA 10. Mr. Osilo Celestine PHC Coordinator, Ohaozara East LGA 11. Okechukwu Ugwuba Asst. PHC Coordinator, Ohaozara East LGA 12. Elder M. O. Onu Treasurer, OhaozaraEast LGA 13. Mrs. Nnennaya Njoku LOCT Leader, OhaozaraEast LGA 14. Mr Nwangele Raymond LOCT Leader/PHC Coordinator, Nnodo LGA 15. Mr Usulik K. U. PHC Coordinator, IzziLGA 16. Mrs Idenyi Regina LOCT,lzziLGA 17. Mrs F. Nwogbaga Supervisior for health, IzziLGA 18. Chief Eze O. Chairman/Ilead of Personnel Management, lzziLGA 19. Hon Benjamin C. Ebele Chairman Nnodo LGA 20. Prince U. Elebe PHC Coordinator, Onicha LGA 21. Hon Eze Nwankwo Vice Chairman/Supervisor of health, Onicha LGA 22. Mrs Elizabeth Oriabor LOCT member, Onicha LGA 23. Mr Agu L. E. LOCT member, Onicha LGA 24. Mr Igwe Joseph LOCT member, Onicha LGA Frontline Health Facility (FLHF) Level 1. Mrs. Rachel Amira In charge, Ndufu Alike FVC, lkwo Central LGA 2. Mrs. Beatrice Agu Agalegu FIIP, Ikwo South LGA 3. Mrs. Mgbeke Ogbu Oshiri Health Center, Onicha LGA 4. Mrs Nwokpor Sampson Ndiegu Obovu WC,IzziLGA 5. Mr Lazarus Obele Oyede IVC, Nnodo LGA 6. Mrs Ogonna Eze Amas Health Centre, Onicha LGA Community Level 1. Mr. Philip Nwankwo Chairman, Village Committee, Enyim Agalegu 2. Mr. Vincent Nweke CDD, Enyim Agalegu, Ikwo Central 3. Mr. Michael Uchaya Chairman, Village Committee, Onyikwa,Ikwo Central 4. Mr. Innocent Ituma CDD, Onyikwa, Ikwo Central 5. Mr. Uche Nworie Chairman, Village Committee, Ohatekwe, Ikwo South 6. Mr. Augustine Nwafor CDD. Ohatekwe, Ikwo South 7. Mr. Sunday Agbe Chairman, Village Committee. Agubata, Ikwo South 38 8. Mr. Godwin Ezaka CDD, Agubata, Ikwo South 9. HRH, Eze Ikeagwu Aja JP Community Leader, Mgbom 10. Chief John Onwosi CDD and Community Leader, Owom, Onicha LGA 11. MrPeterEzebe CDD,Agbabi l,OnichaLGA 12. Mr Micheal Ogbulaji Community Leader, Ogbagharu, Nnodo LGA 13. Mr Amos Nwizi Community Leader, Ndiepete, Nnodo LGA 14. Mrs Theresa Alo CDD, Ogbaharu, Nnodo LGA 15. Mr Adegbe Fedrick CDD, Ndiogbu-Ndieze, IzziLGA '1 16. Mr Nkwuda Paul CDD, Olua,lzziLGA 17. Mr Nkwuda Chukwu CDD, Olua,lzziLGA 18. Mr Ovu-oba Opoke Community Leader, Ndiogbu-Ndieze,IzziLGA 19. Mr Paul Igboke CDD, Umunogodo, Onicha LGA 20. Mr Isaac Ngwuta CDD, Umuigaga, Onicha LGA 2I. Mr Oti Nwani Community Leader, Umunogodo, Onicha LGA r 1 22. Mr Chukwu Ngwa Community Leader, Umuigaga, Onicha LGA 39 Schedule for the Evaluation, advocacy 40 Feedback and planning meeting agenda Evaluation of Ebonyi CDTI Project August 2003 Feedback/Planning Meetine for State Level at Coqferenqe Hall. Ebonvi State Universitv Teachine tt nrt I r"rmxr. t t - I 2eAueust2003 Tentative Agenda -ur* S/N Activitv Time Facilitator I Openins Praver 09:00 - 09:0lhrs 2. Ooenins ceremonv and welcome 09:01 - 09:l0hrs Director, PHC 3. Introducins the oarticipants 09:15 - 09:30hrs S. Orogwu 4. Presentation: 09:30 - 09:45hrs E. Tarimo r The objective ofthe evaluation o The Evaluation Methodolosv 5 Presentation of Main Findings & Discussions 09:45 - I 1:00hrs r Community level C. Okoronkwo o FLHF level M. Alleman o Local Government level C. Evini o State level U. Ekpo Coffee break ll:00-ll:30hrs 6. Group work (in 3 groups) ll:30- l3:00hrs M. Alleman L SWOT analysis - 'What is the situation regarding sustainability in our project?' 2. 'What could be the solutions to weaknesses regarding sustainability of our proj ect?' Group l: The community and FLHF levels Group 2: The local government level Group 3: The StatelProiect level Lunch break l3:00 - l4:00hrs 1. Presentation of group works followed by plenary 14:00 - l5:00hrs U. Ekpo discussion 8. Discussions: l5:00 - 16:00hrs E. Tarimo o What is a sustainable CDTI Plan? Features? Format? C. Evini 9. Presentation on Ideas for the Sustainability Plan r6.00 - 16.15 S. Oroswu 10. Group work: l6:15 - l8:00hrs M. Alleman o Identification of Basic CDTI Activities o Identification of resources for a 3-Year period o Plannins for 2OO4 Plan of Action 4t .- 2"d Day - lzt|o August 2003 S/N Activitv Time Facilitator I Opening prayer 09:00 - 09:02 2. Opening ceremony and introduction of the chair 09:02 - 09:30hrs The Director, PHC Summary of previous day's proceedings 09:30 - l0:00hrs U. Ekpo Coffee break 10:00- l0:lShrs 4. Presentation of group work, followed by plenary l0:15 - 12:15hrs Chair discussion 5. Group Work: Development of Sustainability Plans for 12:15- l4:,l5hrr C. Evini 2005 &2006 Lunch break 14:15 -15:30hrs 6. Presentation of group work, followed by plenary 15:30 - l6:30hrs Chair discussion 7. The Way Forward 16:30 - 17:45hrs E. Tarimo/Chair Evaluation of Ebonyi CDTI Project August 2003 FeedbacUPtanninq Meetinq for LGA Level (1"t Batch) at Conference Hall. Ebonvi State Universitv Teachino Hospital. Abakiliki. 13'n - 14rn Auqust 2003-08-13 S'N Activity Time Facilitator 1 Ooenino Prayer 09:00 - 09:05hrs 2 Openinq ceremony and welcome 09:05 - 09:30hrs Director, PHC 3 lntroducinq the oarticioants 09:30 - 09:45hrs J. AIi 4 Presentation: 09:45 - 10:'l Shrs E. Tarimo r The objective of the evaluation o The evaluation methodoloov 5 Presentation of the main findings & Discussions: 10:15-11:00hrs . Community level C. Okoronkwo o FLHF level C. Evini . Local Government level U. Ekpo Coffee break 11:00- 11:1Shrs 6 Group work: ( 2 tasks): in 3 groups 11:15-12:15hrs J. AIi / SWOT analysis -'what is the situation regarding sustainability in C. Okoronkwo our project?' / 'What could be the solutions to the weaknesses regarding sustainability on our project?' . Group 1: The community level . Group 2: The FLHF level o Group 3: The local qovernment level 7. Presentation of qroup work, followed bv olenarv discussion 12:15 - 13:1Shrs U. Ekpo Lunch break '13:15-14:30hrs B. Discussion: 14:30 - 15:30hrs S. Orogwu . What is a sustainable CDTI Plan? Features? Format? C. Evini 9. Group work: (by LGA) Development of 1't Year Post APOC Plan - 15:30 - 17:00hrs C. Evini (what rs to be done, why; by whom; by when; indicators; cosf). The plan must fit into the available resources 10. Presentation of group work, followed by plenary discussion & review of 17:00 - 18:30hrs Chair 2004 Plan 42 Day 2 - 14'h August 2003 S'N Activity Time Facilitator 1 Opening Prayer 09:00 - 09:05hrs 2 Opening ceremony and introduction of the Chair 09:05 - 09:30hrs The Director, PHC 3 Summary of previous day workshop proceedings 09:30 - 10:00hrs S. Orogwu 4 Review of 1rt Year Post APOC Sustainability Plan 10:00 - 11:00hrs M. Alleman Coffee break 11:00-11:1Shrs 5 Group work: Development of Sustainability Plans for 2005 & 11:15 - 13:30hrs C. Evini 2006 - (what is to be done, why; by whom; by when; indicators; cosf). The plans must ft into the available resources Lunch break 13:30 - 14:30hrs 6 Presentation of group work, followed by plenary discussion 14:30 - 16:00hrs M. Alleman 7 Review of Plans 1 6:00 - 17:30hrs Chair 8 Endorsement of Plan 17:30 - 17:45hrs Chair 9 The Way Forward 17:45 - 18:00hrs E. Tarimo Evaluation of Ebonyi CDTI Project August 2003 FeedbacUPlanninq Meetinq for LGA Leve! (2nd Batch) at Conference Hatt. Ebonvi State S'N Activitv Time Facilitator I Openinq Prayer 10:00 - 10:05hrs 2 Openinq ceremonv and welcome 10:05 - 10:30hrs Director, PHC 3 lntroducinq the oarticioants 10:30 -'10:45hrs J. Nwanoa 4 Presentation: 10:45-11:1Shrs E. Tarimo . The objective of the evaluation . The evaluation methodoloqy 5 Presentation of the main findings & Discussions: 11:15-12:00hrs . Community level U. Ekpo . FLHF level M. Alleman . Local Government level C. Evini Coffee break 12:00 - 12:1Shrs 6 Group work: ( 2 tasks): in 3 groups 12:15 - 14:00hrs R. Azi ./ SWOT analysis -'what is the situation regarding sustainability in our project?' /What could be the solutions to the weaknesses regarding sustainability on our prolect?' . Group 1: The community level . Group 2: The FLHF level o Grouo 3: The local oovernment level Lunch break 14:00 - 15:00hrs plenarv 7. Presentation of orouo work, followed by discussion 1 5:00 - 16:00hrs J. Nwanoa 8 Discussion: 1 6:00 - 17:00hrs . What is a sustainable CDTI Plan? Features? Format? C. Evini 43 Day 2 - 16th August 2003 S/N ActiviW Time Facilitator 1 Opening Prayer 09:00 - 09:05hrs 2 Group work: (by LGA) Development of 1st Year Post APOC Plan - 09:05 - 11:05hrs J, AIi (what is to be done, why;by whom;by when;indicators; cost). The plan must fit into the available resources Coffee break 11:05-11:1Shrs 3 Presentation of group work, followed by plenary discussion & review of 11:15-12:1Shrs Chair 2004 Plan 4 Review of 1st Year Post APOC Sustainability Plan 12'.15 - 13:1Shrs S. Orogwu Lunch break 13:15 - 14:15hrs 5 Group work: Development of Sustainability Plans for 2005 & 2006 - 14:15 - 16:1 Shrs C. Evini (what rs to be done, why; by whom; by when; indicators, cosf). The plans must frt inb the available resources 6 Presentation of group work, followed by plenary discussion 16:15 - 17:00hrs M. Alleman 7 Review of Plans 17:00 - 1 8:00hrs Chair 8 Endorsement of Plan 18:00 - 18:'l5hrs Chair 9 The Way Forward 18:15 - 18:30hrs S. Orogwu 44 Report of the feedback/Planning meetings Feedback/Plannins Meetings A feedback/Planning meeting was held for members of SOCT. The objectives of the meetings were to: figures, who have responsibility for the CDTI programme? develop a realistic plan to bring about the sustainability of their CDTI programmes State level Members of SOCT, PHC Director, the State Project Accountant, and the State Planning Officers attended the feedback meeting. The meeting started at exactly 10:25 am. Mr John Ali, a member of the SOCT, said the opening prayer. Mr S. Orogwu, State Coordinator, did introductions. The Director, PHC presented his opening address. In his address, the PHC director, welcome the evaluation team back from the field and apologizes for the difficult terrain experience during their field visits. He stated that health was the second priority of the State Government after education. As such, he pledge the support of State to CDTI activities. In particular, he mentioned that the control of onchocerciasis started in 1991. In 1994, government formally recognized it with an official launching in Ezzago LGA under the auspices of Sight First programme, an initiative of Lion Club International. The Mission Coordinator, Dr E. Tarimo made presentation on the Objective and Methodology of the Evaluation. This presentation described the objectives of the evaluation and method used in sampling, and assessing the project Findings and Discussion Presentation of findings by the evaluation teams at various levels of CDTI was presented. The first presentation was by Mr C. Okoronkwo, who presented the evaluation team finding at the Community level. Dr (Ms) M. Alleman then presented the teams' findings at the Front Line Health Facilities (FLI{F). Mr C. Evini then presented the team findings at the LGA level, while Mr U. Ekpo presented the findings at the State level. After the presentation, Dr M. Alleman facilitated the session of questions and answers, arising from the presentation. It noted that the State has performed well in the area of political support for the CDTI, as evidence for funds released. The State was equally commended on maintaining a >65Vo coverage rate. The evaluation team also highlighted areas of problems identified that needed to be solved, to achieve sustainability. This includes, planning, supervision and monitoring, integration, training, and finances. Overall, the evaluation team score the State was being on its way to achieving sustainability. Group Work At the coffee break, participants were combine into a group to undertake SWOT analysis - 'what is the situation regarding sustainability in our Project?' and 'What could be the solutions to weakness regarding sustainability of our Project?' 45 -t Presentation of Group Work The SWOT analysis was presented by Mr John Ali, and is shown below Indicator Strength Weakness Solution Opportunities Threats Solution Planning Planning rs a part of NIL NIL CDTI activities NIL NIL overall health plan rn is in overall the state Health plan (2003 -2006\ The state has been LGA not The approved NiI NIL NIL able to prepare a actively plan when detailed sustainability involved in a implemented plan (2004 - 2006) partlcipatory will help the already approved by approach in plan state to reach the various partners development down to the LGAs Monitoring & SOCTs have There is Supervision and Supervision of There is no Success Supervision empowered the uncoordinated monitoring will other process of rewarding LGAs to monitor and and untargeted be Targeted and prograrnme rewarding process will supervise lower supervision by coordinated success be adopted levels SOCTs Empowerment SOCTs take prompt of LOCT and actions on problems FLHF staffs encountered in the field SOCTs has detailed suDervisory check list Mectizan There is an organised NIL NIL The NIL NIL Procurement effective, involvement of uncomplicated, Global 2000 in efficient Mectizan the procurement Procurement System of Mectrzan Training SOCT now train staff Some degree of Traimng to be NIL NIL NIL at LGA level routine training targeted and specific Trainings are targeted and soecific lntegration Supervision and Integration of Efforts to Provide Weak Initiate training activities for CDTI with appropriately opportunity for integration integration CDTI now well other lntegrate CDTI possible with other with other integrated programmes is with other integration with prograrnmes programmes weak prograrnme in other the PHC prograrnme department Involvement of Sudan United Missron CDTI Finance Adequate budget for NIL NIL Strong political NIL NIL the programme by commitment by StAtC & APOC Govt. Involvement of Global 2000 Global 2000 in funds support beside accounting APOC fund Strong political comlllItment Transport & Availabilrty of No motorcycles Provision of NIL NIL NIL Matenal transport and other at state level motorcycles for resources material loeistics State and LGA Hunan SOCT has technical NIL NIL SOCT to used NIL NIL Resources skilled staffs their shll to empower staffs Staff have remained at lower level with the programme since incephon 46 lndicator Strensth Weakness Solution Oooortunrhes Threats Solution Staff have adequate skills and well motivated Coverage Consistent NIL NIL NIL NIL NIL maintenance of more thar. 65% TCR in all the l0 endemic LGAs for the last 3 vears After the presentation of the result of SWOT analysis, Mr C. Evini gave a presentation on 'What is a sustainable CDTI Plan? Features and Format? He said that the components of a sustainable plan include efficient, effective, and should be based on dependable resources. The second day of the workshop opened with a short prayer, followed by introduction of the chair by Mr S. Orogwu. Mr U. Ekpo made a summary of the previous day's proceeding. This was followed by presentation of Group work on Development of lst Year Post APOC Plan and Sustainability Plans for 2005 &. 2006. The plans address the weakness and threat observed by the evaluation team and solution to be taken to address these weaknesses and threats. These plans were discussed review and endorse for submission to APOC. Report of Feedback/Plannine Meetines for LGA. The Feedback/Planning Meetings for the LGA level were held in two batches, due to the number of expected participants. The I't batch comprises of participants from Ebonyi, Ishielu, lzzi, Ezza North, Ezza South and Ikwo LGAs. The 2nd batch comprises of participants from Onicha, Ohozara, Afikpo North, and Ivo LGAs. In all cases LGA Chairmen, Supervisor Councillors for Health, LOCTs member, HODs Health, and FLIIF staffs, attended the meetings. The meeting provided a forum for the presentation of findings of the evaluation team and the opportunities for feedback. After the opening prayers and introductions, presentation of findings by the evaluation teams at various levels of CDTI were made. The presentation of the evaluation team later formed subject of the SWOT analysis. The SWOT analyses were developed for LGA, FLI{F and Community levels respectively. The analysis addressed the following issues: LGA LEVEL o Lack of planning o Does not trigger own annual activities o lack of interest b decision makers o lack of funds from LGA o Work more efficiently, target training, supervision, monitor FLTM o Lack of encouragement to plan o Work more efficiently o Reward success o Lack of funds for CDTI Community 41 o Need more village involvement in deciding time of treatment o Issue of lack of CDD compensation o Treatment of absentees o Increase ratio of CDD to village population After the presentation of the SWOT analysis, Mr C. Evini gave a presentation on 'What is a sustainable CDTI Plan? Features and Format? He said that the components of a sustainable plan include efficient, effective, and should be based on dependable resources. The second day of the workshop opened with a short prayer, followed by introduction of the chair by Mr S. Orogwu. This was followed by presentation of Group work on Development of l't Year Post APOC Plan and Sustainability Plans for 2005 & 2006. The plan addresses the weaknesses and threats observed by the evaluation team and solution to be taken. These plans were discussed review and endorse by the supervisory councilor healths from each LGA respectively. The Plans will be forward to APOC for consideration. 48