World Health Organization African Programme for Onchocerciasis Control
Evaluation of Sustainability of East Wollega CDTI Project, Ethiopia (Third Year)
May, 2009
OLAMIJU FRANCISCA ABATE TILAHUN FIREW AYATEW 1
Acknowledgements
The Evaluation team is grateful to the following persons and organisations for their cooperation, contributions and assistance towards the successful execution of this assignment
• The Director, Dr. Mrs. Uche Amazigo,and staff at APOC Headquarters in Ouagadougou for making available the necessary financial and logistic requirements for the success of this assignment. • The WHO, Carter Centre and Light for the World offices in Ethiopia for providing support, which contributed to the smooth functioning of the Evaluation Team. • The National Onchocerciasis control programme staff for all the preliminary arrangements and facilitation. • Mr Ensermu Jeldu,the Zonal Onchocerciasis focal person for East Wollega CDTI project and his team, for all their the logistic support. • The Zonal Health Desk Head and his deputy for their kind support especially at ensuring that all the expected participants were around for the three days sustainability planning workshop. We equally thank Zonal Health Desk Head of the West Shoa zone for his participation. • A very special thanks to the Acting Zonal Administrator of the East Wollega Zone, and all the other Woreda Administrators for your time and commitment ,We appreciate the fact that you were present all through the four days planning meeting. • The Woreda Health desk Head and Onchocerciasis focal persons from the 10 CDTI Woredas of East Wollega and West Shoa. We thank you for your commitment. • The female Health Extension Health workers manning all the Health post visited, for their hard work and willingness to share their CDTI data with the Evaluators. • The CDDs, Kebele Administrators, community leaders and community members that provided very useful information that made this Evaluation possible. • The Evaluation team finally appreciates all the drivers and local guides that assisted through this exercise. 2
Abbreviations/Acronyms APOC African Programme for Onchocerciasis Control BPR Business Process Re-Engineering CDC Communicable Disease Control CDD Community Directed Distributor CDTI Community Directed Treatment with Ivermectin EPOC Ethiopia Programme for Onchocerciasis Control FLHF First Line Health Facility HQ Headquarters HC Health Centre HE Health Education HFs Health Facility staffs HEW Health Extension Worker HP Health Post HSAM Health Education/ Sensitization/ Advocacy/ Mobilization IEC Information, Education, Communication LFTW Light for the World MoH Ministry of Health NGO Non-Governmental Organization NID National Immunization Day NOTF National Onchocerciasis Task Force PHC Primary Health Care REM Rapid Epidemiological Mapping of Onchocerciasis RHB Regional Health Bureau TCR Therapeutic Coverage Rate WHO World Health Organization WOFP Woreda Onchocerciasis Focal person WoH Woreda Health Office ZHD Zonal Health Desk ZHDH Zonal Heath Desk Head ZOFP Zonal Onchocerciasis Focal Person 3
EXECUTIVE SUMMARY
The East Wollega CDTI project was launched in November 2004 but actual reporting on distribution started in Year 2005.The 2007 updated census shows that the Zone has a total population of 1,175,853. The districts of the zone are sub divided into 367 kebeles, of which 339 are rural and 28 are urban. The kebeles are further divided into Gare (village).Each village consists of 30-50 households. Villages are responsible for all developmental activities including health.
East Wollega CDTI projects has 10 districts (Woredas). They are Jima Ario, Leka Dhulecha, Diga, Guto Gida, Wayu Tuka, Sasiga, Sibu Sire, Bako Tibe,Boneya Boshe and Wama Hagalo. Recently the Bako Tibe Woreda was moved from East Wollega Zone to become part of West Shewa Zone.This therefore makes East Wollega Zone to have 9 CDTI Woredas while the West Shewa zone has 1 to give a total of 10 CDTI Woredas. In the 10 CDTI Woredas there are 4299 villages and 8478 Community Directed Distributors (CDDs).The Project has 419 trained Health workers involved in CDTI.
APOC has been supporting the project since Year 2005, According to the East Wollega Zonal Onchocerciasis focal persons the project is its fifth year technically but financially they are in their third year. Considering the age of the project and the need to conduct midterm evaluation for CDTI projects by their third year, APOC decided to conduct an evaluation with the following objectives.
The general objective of the evaluation exercise was to determine the sustainability potentials of the East Wollega CDTI project by its third year of operation and assist in developing a plan for sustaining the project post-APOC.
The specific objectives are: To assess the performance of the different groups of indicators of sustainability of CDTI projects in the East Wollega CDTI project To identify the factors that may block or help the sustainability of the project Discuss the outcomes of the evaluation exercise with the relevant stakeholders in the East Wollega CDTI project Develop plans for sustaining the East Wollega CDTI project post APOC In order to carry out the above objectives, a team of three Evaluators was sent to the project.
The Evaluation took place from the 18th – 23rd of May 2009,while the Feedback meeting and the development of sustainability plans took place from the 25th -30th of May 2009.
In carrying out the Evaluation, two Woredas, 4 Health Facilities and 12 villages were randomly selected from all the 10 Woredas .One additional Woreda was equally randomly selected as a reserve one. 4
The evaluation was conducted using four types of instruments developed by APOC. Instruments were used at the 4 levels to access the projects performance regarding the routine activities and processes, there were 6 group of indicators to access this namely; Planning, Integration, Supervision and Monitoring, Mectizan supply, Training and HSAM. Resources provided for the activities were accessed using 3 groups of indicators namely; Financing / Funding, Transport and other material resources and Human Resources. The result achieved were measured using Coverage indicators, both Geographic and therapeutic. The performance of the above indicators as well as the various aspects of sustainability (Integration, Resources, Efficiency, cost-effectively, Simplicity, Health staff acceptance (Attitude of the health staff) and Effectiveness were used in grading the performance. In addition documentary evidence from CDTI data, reports and plans, inspection of capital Equipments, Verbal reports from persons interviewed, Community Meetings and additional insight during feedback meetings provided additional insight to the evaluation team. The five critical elements of sustainability (money, Transport ,supervision ,Mectizan supply and political commitment),the seven aspects of sustainability in the project were qualitatively discussed and results agreed to by the team. The project was graded using these aspects and elements following the Evaluation guidelines. Qualitative descriptions of problems were deliberated upon and recommendations made. The sustainability judgment about the sustainability potentials of the project was therefore based on the quantitative assessment of the average sustainability scores of the groups of indicators as well as the qualitative assessment of the critical elements and aspects of sustainability of the project.
The following are summaries of the findings;
Community Level The community level was judged to have high sustainability potential based on the insights below; The CDDs plan and manage CDTI activities in close collaboration with the Health Facility staffs, kebele leaders and village chiefs . The main activities enumerated by CDDs as their key CDTI activities were; Conducting census updating, Requesting for Mectizan based on census population, Informing communities the specific date of Mectizan distribution and reporting back on treatment to the HFs. The village registers were seen at the Health facility and confirmed that the above activities have taken place. Kebele leaders as well as village chiefs have impressive knowledge of CDTI activities. They know the impact of the disease, eligible population, as well as its benefits. Leadership and ownership of the programme are shown in various ways like selection of CDDs, initiating annual distribution activities, agreement on time and mode of distribution; they also participate in census updating, social mobilization and monitoring Mectizan distribution. 5
Community members have positive perceptions towards CDTI. They have given high value to Mectizan and wish they will be allowed to take Mectizan for unlimited number of years. All community members elaborated that Mectizan has multiple effects on their health. Besides Onchocerciasis, Mectizan kills lice, bugs, relief from persistent itching and expelling intestinal worms. It seems also that the communities decided not to provide any kind of support to CDDs because of the believe that it is a great honor to be appointed to serve their community. Community incentive to CDDs is also not common in communities visited even the CDDs do not expect it. CDDs are willing to continue to distribute Mectizan because they are happy to protect their communities from Onchocerciasis and other diseases Current CDD: Community member ratio is 1:80 CDDs are reporting their activities to health facility staffs, Some CDDs are literate enough to summarize their activities and report it using village summary reporting format, others submit the completed village registers and summary is done by the front line Health facility staff. The weighted average of therapeutic coverage for visited communities is 73% from year 2006 to 2009. Some CDDs did not update their community census accurately. Only one out of many CDDDs met talked about incentive.
Front Line Health Facility Level At the FLHF level sustainability potential was found to be high with a score of 3.5 using the 9 group of indicators. The CDTI programme is functioning efficiently in some of the indicators like finance and transport but the system used was found to be innovative and unique for this project and Evaluators had to discuss further at the indicators and characteristics of the indicators in awarding scores to those indicators.
Written plans for Year 2009 CDTI activities were seen pasted on the wall of most of the Health Facilities visited .The activity plans were written in the local language (Oromifa).No written integrated health plan was however seen at this level. HFs said they worked in close collaboration with the Kebele administrators in carrying out their duties including CDTI activities, however minutes of such meetings were not seen.
The Health Facility staffs are in charge of all the health programmes in their kebele. Each HFs is manned by two female and activities are carried out in an integrated manner. The Evaluators believed them because the CDDs and communities commended their hard work but written evidence on integrated activities was not available. The health facility staffs takes full responsibility of CDTI activities in their Kebeles ,In all the Health post visited, Health staff explains that their annual CDTI activities starts with a meeting with their Kebele administrator. Activities usually discussed and agreed upon include census updating, mobilization and Health Education, date of distribution, training date for CDDs and any other issues of concern that the Health staff might have. The Community structure is very much aware of CDTI and lends support to the Health Facility staff. However no written report of the planning meeting was seen at all the health post visited. The supervisory data is being transmitted entirely within the 6 government system. At all the Heath Facilities visited CDTI data treatment summary for three years were seen pasted on the wall. The summary provided information at a glance on CDTI performance in the communities under each Health Facility. The management structure in operation at the Health Facility makes routine supervision of CDTI activities easy and efficient. Each health Facility in the zone is manned by two female Health Facility Staffs The Health Facility staffs lives in the community where the facility is located .They are appointed for training by their community through the Kebele administrators but are being paid by the government . Supervision of CDTI activities is routinely carried out in an integrated manner alongside other programmes like Environmental sanitation, Malaria, Family Health home visit etc. FLHFs requests for Mectizan from the Woreda with a request letter written and stamped by the health facility staff. Copies of these were seen in all the Health Facilities visited. The CDDs receive their Mectizan allocation from the FLHFs based on the request calculated using updated census. CDDs usually go to the HFs either at home or in the office to collect more Mectizan or return the excess when they complete distribution. A record of Mectizan collected by each CDD was seen but details like Mectizan Batch number, Lot number, and expiring dates were not written in the record.
Training is carried out routinely. FLHFs train CDDs annually in order to refresh their memory on CDTI. All the Health Facility Staffs visited, lacked training and need identification skill. No refreshment nor transport is provided for the CDDs when they come for training at the Health Post. The Health Facility Staff believe is not necessary since CDDs have accepted this work as a voluntary service to their community they eat at home before coming and their homes are not far from the Health post. The Kebeles support the training by ensuring that every community send their CDDs for the training .One striking observation regarding CDD’s training in the Zone is the inclusion of construction of measuring sticks in the training topics. This was evident in the communities as CDDs showed evaluators measuring sticks of different designs made with local materials but accurately calibrated. HSAM activities are usually done routinely. Community awareness of the benefits of Mectizan is high and this has led to high treatment compliance. However new issues like amending distribution time to accommodate migrant farmers, or sustained compliance when signs of Onchocerciasis disappears are not addressed .HFs lack skill to identify what the objective of any round of HSAM should be.
The health post don’t manage budget. The FLHF workers do not get additional field allowance from Government they only depend on their salary which according to them is reasonable. This comment by one of the HFs summarizes this observation; “In managing our Health Post, most of the materials we use are provided by the Woreda, we only use it as instructed by them to serve our people, because is our community that appointed us. Little things like pen or pencil for our activities we can buy from our salary - Alganesh Degago, Head of Loko Health post -Guto Gida, Woreda. Evaluators were also informed that when the Woreda calls them for additional meetings outside their routine activities, their transport cost is usually reimbursed. This Government/Community system used in managing the Health Facilities helps the HFs function effectively. 7
There is also no transport provision at this level since all health staffs lives in the community where their facility is located. They go on foot in carrying out their activities since activities are integrated and communities they serve are quite close. When they go to the Woreda for other health activities they equally transmit CDTI data. Training/HSAM materials are available. They are provided by the Woreda. Training/HSAM was sufficient and already translated in the local language. Front Line Health Facility staffs managing the Health Facilities are quite stable as they were appointed by their communities. They are efficient and hard working. Evaluators were informed that one of the criteria for their appointment is that they remain and continue to work for the community even after marriage. In all the FLHFs visited, geographic coverage for the last three years is 100% and the therapeutic coverage is greater than 65%.
Woreda Level At the Woreda level sustainability potential was found to be high with a score of 3.3 using the 9 groups of indicators At the Woreda a CDTI plan which is part of the overall integrated annual health plan exists. CDTI activities are integrated more closely with Communicable disease like Malaria, TB, Polio and other infectious diseases at the two Woredas visited. The Onchocerciasis focal persons are in charge of all communicable disease at the Woredas visited. According to head of the health office, planning is participatory involving all key stakeholders and approved by the Woreda council, However Evaluators did not find evidence to confirm partners’ participation.
Managers at this level said their staff combines two or more tasks on a single trip and also combines activities with other health programs. But no written evidence was seen to confirm how CDTI is implemented in an integrated manner with other health activities The Woredas initiates key CDTI activities evidenced by the presence of a detailed timetable stating the time that each CDTI activity will take place. There is a focal person in charge of CDTI in the Woredas visited. These focal persons are also responsible for other health activities of their respective Woredas.
The CDTI activities are supervised and reported through the government reporting system. CDTI supervision checklist was seen but is not integrated with other health activities and there is no written supervision report. Mectizan orders are based on needs and requests from FLHFs and it is also based on the census report of CDDs. Normally, the Woreda Health Office collect Mectizan from Zonal Health Desk. Mectizan is available on time and adequate. No shortages were reported in the 2009 distribution, But in 2008 there was little shortage of Mectizan in both Woredas and was resolved by getting additional tablets from the zone.
Mectizan collection, storage and delivery to lower level is within government system and it is simple, effective, uncomplicated and integrated. Transportation cost is covered from the government as Mectizan is transported in integration with other drugs. 8
Training at Woreda is conducted routinely and not targeted at specific needs of the staff. Training topics does not seem integrated with other health activities. Training is given to both new and old staff every year. Political leaders are well oriented about the CDTI Program and they claim to be supporting the programme using the pool system. There was however no documentary evidence to show Governments funding for specific CDTI activities from the Health desk pool fund. Income statement and ledger showing the amount of money transferred for the program from APOC and Light For the World as well as expenditures were seen at Zonal Finance unit. Bank Statements were also seen as well as retirement record of disbursed fund. The management is aware that APOC funding will stop after five years. But there is no evidence that the management have initiated any sort of plan to mobilize funds from dependable sources. They are confident that since CDTI is one of the health activities, the program will continue without interruption using government.
There are about 8 motorcycles available for integrated health programmes at the two Woredas visited, two of which were provided by APOC five years ago. The APOC motorcycles are however getting old and not functioning effectively due to difficult train and rough roads Government regulations permit that the government budget be used for the maintenance of donated capital equipments, accordingly, the motorcycles seem to be maintained on a regular basis. The Woreda Health Offices approves movements of the motorcycles.
All available transportation resources are put together in a pool to be used for all health related program. Trips are properly authorized by the Woreda Health Offices. The control mechanism is strong. According to the Woreda Health desk head, the Woreda is unable to replace the motorcycles because government budget allocated for the health activities is very minimal. Trained staff available for CDTI activities at this level are committed but inadequate and do not have enough skills on need assessment identification, targeted HSAM, data management /Reporting. The two Wereda has achieved and maintained 100% geographic coverages over the past three years. Average therapeutic coverage over the past three years was 81%.
Zonal Level At the Woreda level sustainability potential was found to be moderate and had the least score of 2.8 using the 9 groups of indicators.
The Zonal Health Unit has a written detailed integrated health plan with a section on Onchocerciasis. In addition there is a separate plan for Onchocerciasis Control activities. Which was said to be is reviewed annually with partners. Minutes of planning meetings were however not seen.
The pool system introduced by the Government in year 2004 is used in planning and implementation of all health programmes. The pool system is a policy for integrated use of money, vehicle and equipment .This year the Business Process Re-Engineering 9
(BPR) which is integration of manpower for effective and efficient implementation of health activities was added to give an integrated package for Health.
All Partners roles are clear and well defined at this level. The key partners identified were APOC, the NGDO (Light for the world), NOTF and the Zonal Health Unit. Sustainability plan have not been developed for post APOC period. The Head of the Health desk at the zone believes that since CDTI is an activity implemented in an integrated manner in a government system, there is no doubt about its sustainability. There remains a concern that with pool funding method of budgeting and fund disbursement and without a detailed Post APOC, sustainability plan, CDTI may receive very little support compared to what it will require to implement its very important activities.
Integration is said to be taking place at this level in the implementation of CDTI activities as well as other health programmes, however no written reports providing what was achieved in an integrated manner for all the programmes involved was seen. Leadership at the Zonal level are aware of the CDTI process .They equally have information on the progress and success achieved, but new emerging issues that is identified with effective supervision is unknown by the Zonal leadership. Delegation of duties is difficult because they do not have enough staff.
CDTI activities are supervised by the Zonal focal person. Monitoring and supervision activities are carried out using supervisory checklist. Filled out supervisory checklist were seen, but there was no supervisory report. Filed copies of APOC technical report for the past two years were equally seen. The zone lost soft copies of their summary report for the first 2 years of programme implementation due to computer virus. Detailed inventory of Mectizan as well as fund management records were not with the Onchocerciasis focal person because Drug and fund management for the whole zone is handled centrally by another unit.
Mectizan is managed by the Pharmacy department and Funds by the finance department. Mectizan supply is controlled within government system. Since the inception of the CDTI programme, the Zonal staff facilitates training of the Woreda Staff as well as orientation of a new staff posted to the health unit that might be involved in CDTI activities implementation. Training is carried out routinely at the Zonal level and for the Woredas. It is not based on need assessment and not targeted. Staff at this level and the level below believes that the purpose of training is to refresh their memory. The Zonal level managers believe they can’t do much about advocacy as their Health package budget is fixed and for any amendment to be done, it will have to involve advocacy at the higher level. At the lower level they believe they can advice the Woredas administrators to give Onchocerciasis a priority attention. Other HSAM activities are carried out routinely, the reason for this is because Health managers do not have sufficient skill to identify new Health Education focus. Secondly they do not have enough manpower to carry out spot check supervision and identify these needs. It therefore makes it difficult to know the new focus for HSAM. 10
Amount budgeted and released by APOC and Light for the world for Onchocerciasis activities were seen with the Zonal Accountant. Government Budget details for the pool financing of Health activities including CDTI were not seen at the Zone. APOC and Light for the World letters of agreement with the Zone were equally unavailable .They said they are yet to receive fund from Light for the World and APOC for year 2009. There was no evidence to show that Zonal managers are aware of the total fund that will is available to him this year and what they hoped to get next year. Therefore cost reduction /containment strategy is difficult.
The zone has one project vehicle donated by APOC. It is about 5 years old and still functional but according to the Zonal Focal person for Onchocerciasis is Inadequate for the 9 Woredas they have to supervise. The health unit has 3 additional vehicles which can be borrowed for CDTI activities. They also have a one desk top computer, one printer and one Xerox photocopier. All of these are about 5 years old and functional. The only challenge they have is with the photocopier, which they complained that its ink is very expensive to replace.
There is no written replacement plan for transport and other materials resources. There is equally no written commitment from any partner to replace the vehicles or other material resources. The reason for this is Poor programme management skills and the believe that Government has a very lean budget and unable to replace the vehicles or motorcycles. Due to the BPR system adopted by the Government, Staffs in the health unit are now in pool and are involved in all the health issues in the unit. However there are two staff dealing more with CDTI, the Focal person and his assistant. They are committed and quite stable as they have been on the job for the past three years They are overloaded with different activities and therefore could not perform efficiently the need assessment and data management activities needed for CDTI activities.
The zone has had good geographic coverages over the past three years. In Year 2008, Geographic coverage was 99.7%.This was due to shortage of drugs in two Woredas. Year 2006 and 2007 geographic coverage was 100%. Average zonal therapeutic coverages was 74.5 for the past three years.73.4% in Year 2008, 74.6% in 2007 and 75.4% in 2006.
Conclusion Based on the seven aspects of sustainability and in line with the guideline for grading the whole project using the five critical elements of sustainability ,the Evaluation Team concludes that the East Wollega CDTI project is MAKING SATISFACTORY PROGRESS TOWARDS SUSTAINABILITY. The quantitative score of 3.3 for the pooled groups of indicators at the four levels supports the above qualitative score. 11
Way forward: As the East Wollega CDTI project is almost completing its fifth year technically there is the need to improve on the project’s Efficiency and availability of Resources.
1. EFFICIENCY: The project needs to be more efficient in Training, HSAM and documentation.
A) Training: There is the need to ensure that training activities at all levels are carried out in an efficient manner. Training should be targeted. Based on need assessment identified using competency /skill monitoring strategies like pre/post test ,supervision, evaluation of reports etc. The training should therefore address that need and report of such training should be documented. Training on CDTI for all the Health Staff in the pool should also be considered in view of the BPR scheme.
B)Health Education, Sensitisation ,Advocacy and Mobilisation(HSAM) The Zone need to intensify advocacy at all levels in order to mobilize enough resources to sustain this CDTI project. The sustainability plan which has been prepared is an available tool that can be utilized. Health Education should be targeted to meet information and knowledge gaps. E.g. High number of refusals and Absentees in an area is a possible indication of Mectizan benefit information gap. Other gaps can be identified through effective supervision and spot checks.
C) Documentation Documentation at the Zone is poor. It is very important therefore that the Zone update its technical and financial records, Efforts should be made to ensure that this updating is completed before December 2009.Copies of the unavailable records should be requested for from the relevant partners. Documentation will enhance the Zones efficiency especially in monitoring their progress and planning.
2. Resources: Human and Financial resources available at the Zone is inadequate
A) Human Resources: The available trained human resource at the Zone and Woreda is insufficient for efficient management of CDTI activities. There is also capacity gap in the area of reporting and need assessment among the available ones. The Zone should therefore increase available human resource to improve their capacity.
B) Financial Resources: The Zonal Government should fund CDTI activities by increasing its budgetary allocations and ensuring timely releases for its implementation. Evidence of Government actual release from the Health budget for CDTI activities should be clearly stated and documented. 12
TABLE OF CONTENTS
Acknowledgements------1 Abbreviations/Acronyms------2 Executive summary------3 Table of Contents------12 1.0 INTRODUCTION------14 2.0 METHODOLOGY------15 2.1 Objective of the Evaluation------15 2.2 Sampling------15 2.3 Instrument used for Evaluation------15 2.4 Sources of Information------16 2.5 Analysis------16 2.6 Constraints------16 2.7 Team Composition------17 3.0 EVALUATION FINDINGS------18 3.1 Sustainability at project (zonal level)------18 3.1.1 Recommendation for the Zonal Level------23 3.2 Sustainability at the Woreda Level------26 3.2.1 Recommendations at the Woreda Level------29 3.3 Sustainability at the Front line Health Facility------32 3.3.1 Recommendation at the Front line Health Facility Level------35 3.4 Sustainability at the Community Level------37 3.4.1 Recommendation for the Community Level------40 4.0. Comparative Analysis of the Sustainability of the Four Levels------41
4.1. A quantitative judgement, based on the grades given to individual indicators--41
4.2. Qualitative judgment of project------43
4.3. Judgment using the five key aspects of the project ------44 13
4.4 Conclusion------45 5.0 Feedback/Planning Meetings------46 Annexes ------49 14
1.0. INTRODUCTION
East Wollega is one of the 17 zones of Oromia Regional State. It is located in the western part of the country. The capital town of the zone is Nekemte, which is 331 kilometers from Addis Ababa (capital city of Ethiopia) .The catchment areas of East Wollega zone extends from Gibe river to Didesa river having with varied topography. East Wollega zone is bounded by Amhara regional state and Horo Gududru Zone in the North, West Shoa in the East, Jima zone in the South East, Illubabor in the South West, West Wollega zone and Benshangul Gumuz regional State in the West. The altitude ranges from 500 to 2600 meters above sea level. Agro ecologically, it is divided into lowland, middle land and highland areas. Annual rainfall varies from 1200-2500mm. The temperature reaches a daily maximum of 28 degree centigrade. There are four big rivers in the zone , namely Gibe which is between West Shoa and East Wollega zones, Didesa, Anger and Uke. The zone has many other small rivers and streams that drain into the basin.The climate is characterized by distinct rainy and dry seasons. The rainy season is from June to September. Harvesting is from November to January. The best time for distribution is in the months of February to March. There is good telecommunication access and fairly good road to the districts and their capitals. Some villages can be reached on motorcycle while others are only accessible on foot or on the back of animals.
According to 2007 census, the total population of the zone is 1,175,853. Among this, a total of 782,236 persons are at risk of Onchocerciasis in the CDTI area. Likewise, 11% of the populations are urban and 89% are rural residents. The districts of the zone are sub divided into 367 kebeles, of which 339 are rural and 28 are urban. The kebeles are further divided into Gare (village).Each village consists of 30-50 households. Villages are responsible for all developmental activities including health.East Wollega CDTI projects has 10 districts (Woredas). They are Jima Ario, Leka Dhulecha, Diga, Guto Gida, Wayu, Tuka, Sasiga, Sibu Sire, Bako Tibe,Boneya Boshe and Wama Hagalo. In line with this, the CDTI project areas consist of 4299 villages and 8478 Community Directed Distributors (CDDs).
The East Wollega CDTI project launched around November 2004 but actual reporting on distribution started in Year 2005.The 2007 updated census shows that the Zone has a total population of 1,175,853. The districts of the zone are sub divided into 367 kebeles, of which 339 are rural and 28 are urban. The kebeles are further divided into Gare (village).Each village consists of 30-50 households. Villages are responsible for all developmental activities including health East Wollega CDTI project has 10 districts (Woredas). They are Jima Ario, Leka Dhulecha, Diga, Guto Gida, Wayu, Tuka, Sasiga, Sibu Sire, Bako Tibe,Boneya Boshe and Wama Hagalo. Recently the Bako Tibe Woreda was moved from East Wollega Zone to become part of West Shoa Zone. 15
2.0. METHODOLOGY
2.1. OBJECTIVE OF THE EVALUATION
2.1.1. The general objective for the evaluation exercise was to determine the sustainability potentials of the East Wollega CDTI project by its third year of operation and assist in developing a plan for sustaining the project post-APOC.
2.1.2 .The specific objectives are: To assess the performance of the different groups of indicators of sustainability of CDTI projects in the East Wollega CDTI project To identify the factors that may block or help the sustainability of the project. Discuss the outcomes of the evaluation exercise with the relevant stakeholders in the East Wollega CDTI project Develop plans for sustaining the East Wollega CDTI project post APOC
2.2. SAMPLING The East Wollega Zone has a total of 10 CDTI Woredas including Bako Tibe which was recently became part of West Shoa Zone. Thus, two Woredas, 4 Health Facilities and 12 villages were randomly selected from all the 10 Woredas .One additional Woreda was equally randomly selected as a reserve one. The Woredas, Health Facilities and Villages selected are as shown below:
Table 1: Summary of selected Woredas, FLHFs and Villages of East Wollega CDTI Project Woreda FLHF Village Bedo Health Post Wanebo Legetsebela Tumee Leka Dulecha Kewissa Health Post Mayibasi Kambi Lafto Kitiesa Health Post Haro Jeni Guto Gida. Daleti Loko Health Post Burka Woligelti Derartu Bua Oda
2.3. INSTRUMENTS USED FOR THE EVALUATION The evaluation was conducted using four types of instruments developed by APOC. Instrument 1 : For National and Project Level Instrument 2 : For Woreda/ District Level Instrument 3 : For Front Line Health Facility Level Instrument 4 : For Community Level 16
Instruments were used at the 4 levels to access the projects performance regarding the routine activities and processes. The following were the Indicators used; Planning, Integration, Supervision and Monitoring, Mectizan supply, Training and HSAM
Indicators used to check for resources provided for the above activities were financing / Funding, Transport and other material resources as well as Human Resources. Result achieved were measured using the Coverages, both Geographic and therapeutic. The performance of the indicators as well as the various aspects of sustainability (Integration, Resources, Efficiency, cost-effectively, Simplicity,Health staff acceptance (Attitude of the health staff and Effectiveness were used in grading the performance
2.4. Sources of information Documentary evidence from CDTI data, reports and plans Inspection of capital Equipments Verbal reports from persons interviewed. Community Meetings Additional insight during feedback meetings
2.5. ANALYSIS At the end of field activities, the information collected on each indicator was graded on a scale of 0-4 (worst to best), in terms of its potential contribution to sustainability. The average 'sustainability score' for each group of indicators was calculated, for each level, and a graph was plotted. Summary statistics for the scores were calculated for each level, and for each group of indicators, tables and graphics of these results were presented at feedback workshop. The quality of the overall project was also assessed using the different aspects and critical elements of sustainability present in the project. The five critical elements and the seven aspects of sustainability in the project were qualitatively discussed and results agreed to by the team. The project was graded using these aspects and elements following the Evaluation guidelines. Qualitative descriptions of problems were deliberated upon and recommendations were made. The sustainability judgment about the sustainability potentials of the project was therefore based on the quantitative assessment of the average sustainability scores of the groups of indicators as well as the qualitative assessment of the critical elements and aspects of sustainability of the project.
2.6. CONSTRAINTS The following were some of the major challenges faced by the team: Insufficient Number of Evaluators: The East Wollega team had three evaluators instead of the expected six. This made it difficult to have a sub team and increased the workload of the available team members. Translation issues: The team members from Addis could speak Armaric which is an official language in Ethiopia but in the Zone the official language was Oromifa which they could not understand and couldn’t speak. This made the team involve more local guides during data collection. 17
Power Failure: For some days during the Evaluation there was no power for data entry even in the hotel. This slowed down data entry.
2.7. TEAM COMPOSITION
S/No Name Designation Address 1. Mrs Francisca Executive Director, MITOSATH Olamiju MITOSATH/Nigeria-Team 605, Hospital place, Leader opposite Green valley Suites, GRA,Jos Plateau State, Nigeria. [email protected]. +234-8033318085 2. Abate Tilahun Programme Manager, Carter +251-911462483 Center, Addis Ababa [email protected] Ethiopia-Team member 3. Firew Ayalew Data Manager Carter Center, +251-911462483 Addis Ababa [email protected] Ethiopia-Team member 18
3.0 EVALUATION FINDINGS
3.1 Sustainability at Project (Zonal) Level
Figure 1. Average Performance of Each group of indicator in the entire Project (Zonal level) 4 4 4 3.5 3.5 3.5 3.1
t 3 3
h 3 g i 2.3 2.3 e 2.5 2 2 W
e 2 g
a 1.5 e v 1 A 0.5 0
g n p g n e rt n e e n o i in a M c o a g g i ti h r iz A n a a n a rs o t S a sp m r r n r e it c H n n u e e la g d n e i a H v v P te a o M & F r o o In e M g T .C .C L in T in G ra T Group of indicators
Planning (moderately 2.3) The Zonal health unit has a written detailed integrated Health plan with a section on Onchocerciasis. In addition, there is a separate plan for Onchocerciasis Control activities. The Health plan is reviewed annually. Monthly and quarterly reports were not seen. The head of the Health desk said planning is done annually by the Health Committee .Minutes of planning meetings were however not seen. The pool system introduced by the Government in year 2004 is used in planning and implementation of all health programmes. The pool system is a policy for integrated use of money, vehicle and equipment .This year the Business Process Re-Re-Engineering (BPR) which is integration of manpower for effective and efficient implementation of health activities was added to give an integrated package for Health.
All Partners roles are clear and well defined at this level. The key partners identified were APOC, the NGDO (Light for the world), NOTF and the Zonal Health unit. According to the Zonal Onchocerciasis focal person he said; “We ensure that all our partners are involved in the annual planning of CDTI activities because it is a partnership” However minutes of the planning meetings were not seen to confirm this claim. Light for the world employed a focal person for Onchocerciasis, who is based in Nekemt.His main responsibility is to provide technical assistance to the Onchocerciasis focal persons for the East and west Wollega projects. 19
Sustainability plan have not been developed for post APOC period. The Head of the Health desk at the zone believes that since CDTI is an activity implemented in an integrated manner in a government system ,there are no doubt about its sustainability. There remains a concern that with pool funding method of budgeting and fund disbursement and without a detailed Post APOC, sustainability plan, CDTI may receive very little support compared to what it will require to implement its very important activities.
Integration (Highly 3.5) Integration is taking place at this level in the implementation of CDTI activities. According to the Onchocerciasis Focal person for the zone he said field outing like training is equally used for activities like HSAM. In addition integration is taking place in the implementation of other health programmes in the zone. The opportunity of being in the field for Onchocerciasis activities for instance is equally used to implement other health programme like Malaria, Family Health or Environmental sanitation activities. The Head of the Health desk said; “We most times release funds for monitoring and supervision of communicable diseases. During this activity supervision is done by responsible officers in the unit for more than 3 or 4 diseases. It is the pool system joined with the new government policy called BPR.” No written detailed activity work plan and reports were seen to show the effectiveness of the pool system.
Leadership (Highly 3.0) Leadership at the Zonal level is effective and aware of the CDTI process .They equally have information on the progress and success achieved. The leadership is aware of some of the problems but new emerging issues that is identified with effective supervision is unknown by the Zonal leadership. Delegation of duties to other staff at the Zonal level is difficult because they do not have enough staff but active delegation takes place at the lower level.
Monitoring and Supervision (Highly 3.1) The Zonal focal person supervises activities at the Wereda level. Hard and soft copies of treatment data and activity reports from the different Wereda for the past 2 years were seen .Filed copies of APOC technical report for the past three years were equally seen. The zone lost soft copies of their summary report for the first 2 years of programme implementation due to computer virus. Detailed inventory of Mectizan as well as fund management records were not with the Onchocerciasis focal person because Drug and fund management for the whole zone is handled centrally by another unit. Mectizan is managed by the Pharmacy department and Funds by the finance department. Zonal staff supervises mainly the Wereda activities and have already empowered the lower staff to supervise the activities at their level and the levels below them. Supervisory visits are usually carried out during Mectizan distribution. It is however difficult to estimate the number of visits because supervision of health programmes is carried out in an integrated manner using resources available to the Health unit. 20
Monitoring and supervision activities are carried out using supervisory checklist. Filled out supervisory checklist were seen, but there was no supervisory report. Problems identified using monitoring system are addressed as they arise by the Zonal managers, some are identified by them and others passed on to them by the lower level. Successes are noted and staffs are commended verbally. According to the Head of zonal Health desk, most of the staff identified to be doing well are usually appointed to represent the Zone at meetings or recommended for a course. Those not doing well are usually rebuked and encouraged to improve.
Mectizan® Procurement and Distribution (Highly 3.5) Mectizan supply is controlled within government system. The Zonal Onchocerciasis focal person collects the drugs from the National Onchocerciasis office of Federal Ministry of Health of Ethiopia and hands it over to the Zonal Pharmacist. The resources used in the collection of Mectizan are from Government because they combine Mectizan drug collection with the collection of other drugs allocated to their Zone. The Zonal pharmacist is in charge of the release of Mectizan to the different Woredas with approval from the Zonal Onchocerciasis focal person. A very efficient inventory system called stock order is used for Mectizan flow .Stock order module 19 and 22 exist at all levels including the Health Facility. At every level stock order module 19 is used to receive Mectizan and stock order module 22 is used to issue it out. The stock order however does not capture the batch and Lot numbers as well as expiring dates. When the Zonal Pharmacist was probed further his response was; “It takes a lot of time to begin to enter those details but I usually check the expiration dates”. Leftover Mectizan is usually returned by the CDDs through the Health Facility to the Woreda and then handed over to the Zonal office. The Pharmacist then receives it from the Zonal Onchocerciasis Coordinator and enters it in his record against the returning Woreda. There was shortage of Mectizan during Year 2007 distribution due census problems but was re solved by getting left over drugs from other Zones.
Training & HSAM (Moderately 2.0) Since the inception of the CDTI programme, the Zonal staff facilitates training of the Woreda Staff as well as orientation of a new staff posted to the health unit that might be involved in CDTI activities implementation. Lower level personnel have already been empowered to conduct training at their levels and the levels below them. Training is carried out routinely at the Zonal level and for the Woredas. It is not based on need assessment and not targeted. Staff at this level and the level below believes that the purpose of training is to refresh their memory.
Training is fairly integrated as staff at this level claim to train on CDTI during other programme trainings. There was however no evidence to prove this. 21
At the Zonal level managers believe they can’t do much about advocacy as their Health package budget is fixed and for any amendment to be done, it will have to involve advocacy at the higher level. At the lower level they believe they can advice the Woredas administrators to give Onchocerciasis a priority attention .Presently this is not yet actively implemented. Other HSAM activities are carried out routinely, the reason for this is because Health managers do not have sufficient skill to identify new Health Education focus. Secondly they do not have enough manpower to carry out spot check supervision and identify these needs. It therefore makes it difficult to know the new focus for HSAM. Obvious Health Education issues like community poor understanding of key CDTI messages written in English and Armaric was identified. According to the Zonal focal person, “We have addressed the problem because our CDTI booklet and posters have been translated into Oromifa language with APOC funds and my people like it a lot, you will see for yourself when you go to the field” Evaluators confirmed this when communities were visited. Some community leaders even showed us their copy which they kept very well and use in sensitizing their communities.
Finance (Moderately 2.0) In the Zone, amount budgeted and released by APOC and Light for the world for Onchocerciasis activities were seen with the Zonal Accountant. Budget details and letters of agreement were unavailable at the Zone and was said to be with World Health Organization office in Addis Ababa. According to record available at the Oromia regional Health Bureau the sum of $30,754 was approved by APOC support for support of CDTI activities in East Wollega Zone for year 2009.out of this $21,528 was said to have been released to the field but as at Evaluation time, the project was yet to receive the fund. Similar delays were encountered regarding funds released by Light for the world through the Oromia region. The delay was said to be due to some bureaucratic procedures at the region which partners are aware of and making effort to address. Evidence of funding for CDTI activities from Government was not seen. It was said to be embedded in a pool called Health Budget. And according to the Zonal head it is difficult to state. His Comment; “Sometimes the Health unit will release funding for Monitoring and supervision of Health activities (Malaria, TB, Onchocerciasis)how do i then tell you how much of that fund will be charged to Onchocerciasis? It is difficult. It is Government policy. We can’t change it”. There was no evidence to show that Zonal managers are aware of the total fund that will is available to him this year and what he hopes to get next year. Therefore cost reduction /containment strategy is difficult. There is no evidence to show Government Contribution to CDTI activities. There is however claim that they are using the Health Budget in an integrated manner to cover CDTI expenses like fuelling and maintenance of vehicles. There is no written document to enable an assessment of budget deficit and how to address it. It has been like that from the programme inception till date. 22
At the zonal finance department requisitions and approvals signed by the Head of the Health Zone were seen. Ledgers and retirements were seen as well. But it was difficult to know whether or not budget lines were followed because budget breakdown and letters of agreement were unavailable.
Transport and other Material Resources (Moderately 2.3) The zone has one project vehicle donated by APOC. It is about 5 years old and still functional but according to the Zonal Focal person for Onchocerciasis is Inadequate for the 9 Woredas they have to supervise. The health unit has 3 additional vehicles which can be borrowed for CDTI activities. This he said is only available when the programmes that owns them are not in the field. They also have a one desk top computer, one printer and one Xerox photocopier .All of these are about 5 years old and functional. The only challenge they have is with the photocopier .They complained that the photocopiers ink is very expensive to replace, so they have not been using it .According to the head of the Health desk; “We like the photocopier because it can print on wider papers but the ink is expensive, we will appreciate it if APOC can give us another that is more universal and cheaper to maintain and use, that will complement this one.” The zone believes that Government budget is too small to buy capital equipments and therefore rely on donors. But they are committed to efficient use and good maintenance using government funds. The Head of Health desk, said the zone in view of sustainability, will consider suggesting to Government to buy some of these needed capital equipments as from next year and see if it will be successful. Capital equipments donated by APOC are well maintained using Government funds. When maintenance issues are identified, it is usually reported to the head of the health desk who then issues instruction for it to be repaired using zonal funds. No maintenance record was seen but all APOC donated capital equipments seen were still functional after 4 years. The four vehicles that the health desk has are all functional .The management of all the vehicles is the responsibility of the zonal head of the health desk. Vehicles are authorized for use with an authorization letter and an outlet paper. The outlet paper according to the Onchocerciasis focal person shows where the vehicle is going to, purpose, no of days, no of occupants and stipulated return date. All the vehicles are in a pool and assigning depends on the activity to be done and vehicle availability. If there is a diversion in the authorized use of the vehicle appropriate disciplinary action is usually taken against the defaulting staff. Copies of trip authorization letters and log books were however not seen. There is no written replacement plan for transport and other material resources. There is equally no written commitment from any partner to replace the vehicles or other material resources. The reason for this is poor programme management skills and the believe that Government has a very lean budget and unable to replace the vehicles or motorcycles. 23
Human Resources (Highly 3.0) Due to the BPR system adopted by the Government, Staffs in the health unit are now in pool and are involved in all the health issues in the unit. However there are two staff dealing more with CDTI, the Focal person and his assistant. They are overloaded in different activities and it is difficult for them to be very effective and efficient in CDTI. They are quite stable as they have been on the job for the past three years. They are equally committed. However, they could not perform properly the need assessment and data management activities.
Coverage (Fully 4.0) The zone has a good geographic coverage over the past three years. In Year 2008, Geographic coverage was 99.7%.This was due to shortage of drugs in two Woredas (Diga with geographic coverage of 99.3 and Wayu Tuka with geographic coverage of 98.8%).The reason for the shortage was incorrect census. According to the Onchocerciasis focal person, the Census has since been updated and enough Mectizan supplied to them during year 2009 distribution. Year 2006 and 2007 geographic coverage was 100%. Therapeutic coverage has been good, above 65%. Average zonal therapeutic coverages was 74.5 for the past three years.73.4% in Year 2008, 74.6% in 2007 and 75.4% in 2006.
3.1.1: Recommendations for the Project (Zonal) level
Table 2:Recommendations for the Project (Zonal) level
Recommendation Implementation Planning: Priority: HIGH Develop a short and long term CDTI Indicators of success: sustainability plan with cost reduction and Existence of CDTI sustainability plan containment strategies. Existence of complete CDTI information for effective planning and monitoring. Put in place proper documentation of past and present CDTI information like Who to take action: Treatment reports, supervisory visits, ZOFP,LFTW,WHO letters of agreement, Government Deadline for completion: contribution record etc July 2010
Integration of support activities: Priority: HIGH Put in place documentary evidence to Indicators of success: show how effectively CDTI is implemented Availability of detailed record showing in integration with other health activities. effective CDTI implementation in an integrated manner. Who to take action: ZHDH and ZOFP 24
Deadline for completion: April 2010 Leadership: Priority: HIGH . Indicators of success: Ensure that the zone provide effective Availability of enough staff for proper leadership for the project by having supervision and effective delegation of enough trained personnel that can support CDTI activities. the implementation CDTI activities. Who to take action: ZHDH and ZOFP Deadline for completion: October 2009. Monitoring and Supervision: Priority : MEDIUM Ensure that reports of Supervisory Indicators of success: activities are written and filed properly Availability of report for supervision .Appropriate feedback should equally be conducted and feed back given. given to those concerned. Who to take action: ZHDH and ZOFP Deadline for completion: May 2010 Mectizan Supply: Priority: HIGH Mectizan ordering and distribution should Indicators of success: be made based on updated census to a) Adequate supply of Mectizan at all avoid shortage. Zonal mangers should levels. communicate to the lower level that CDD census updating is part of distribution b) Availability of accurate census data at activities. all levels for Mectizan ordering and Effort should be made to record Batch and distribution. Lot numbers of Mectizan received at Who to take action: Zonal level as well as the expiration dates. ZOFP and the Zonal Pharmacist Deadline for completion: April 2010 Training & HSAM Priority: HIGH Conduct training on need assessment Indicators of success: and focus future training plans on areas Availability of targeted training report. of need. Reports of communities sensitised Identify communities that need Who to take action: sensitisation and sensitise them ZOFP Deadline for completion: October 2009 Finance Priority: HIGH The Zonal Government should fund CDTI Indicators of success: activities by increasing its budgetary a) Increased amount budgeted and allocations and ensure timely releases for released for CDTI activities at the Zone. its implementation. Evidence of Government actual release b) Documentary evidence of Zonal from the Health budget for CDTI activities Government support to CDTI available. 25 should be available Who to take action: Zonal Administartor,ZHD Head and ZOFP Deadline for completion: October 2009 Human Resource: Priority: HIGH Increase the number of trained personnel Indicators of success: available to support CDTI implementation a) Availability of sufficient trained Zonal in the Zone. team for effective CDTI implementation. Zonal project staff should be trained on Who to take action: need assessment, data management and Head of Zonal Health Desk documentation. Deadline for completion: January 2010
.
Transport and other Material Resource: Priority: HIGH Project can request APOC to consider the Indicators of success: replacement of their old Zonal project a)Availability of replaced vehicle and other vehicle, Motorcycles, desk top computer capital equipments. and photocopier. b)Availability sufficient IEC/MIS materials and other materials needed for CDTI The Zone should begin now to put plan in activity implementation. place a Government plan to replace in Who to take action: the future some of the capital equipments ZOFP,APOC Management, ZHDH and for CDTI implementation. LFTW Effort should be made to estimate other Deadline for completion: materials needed like IEC/MIS materials, October 2010 and ensure that they are adequately budgeted and sourced for from dependable sources before the project get to year 6. Coverage Priority: HIGH Sustain 100% geographic and achieve Indicators of success: 80% therapeutic coverage by ensuring REMO map for the zone that REMO map, soft and hard copies of List of endemic communities list of endemic communities and their populations are available for coverage Verified treatment coverage data data verification Who to take action: ZOFP,NOTF and LFTW Deadline for completion: October 2009 26
3.2 Sustainability at the Woreda Level
Figure 2. Average Performance of Each group of indicator at Woreda level
4 4 4 4 3.4 3.5 3.5 3.3
e 3 3 3 3 g 3 a r
e 2.5 v
a 2
d 2 e t
h 1.5 g i
e 1 W 0.5 0 t g n p n ly M e r e e e in io i io p c o rc g g n t h is p A n p u a a ra rs v u S a s o r r n g e r S H n n s e e la e d e i a e v v P t a p n & F r r o o In e u a g T n .C .C L s tiz in a T & c n G g e i m n ra u ri M T H ito n o M Group of indicators
Planning (3.0): There is a CDTI plan which is included in the overall integrated annual health plan at two Woredas visited. CDTI activities are integrated more closely with Communicable disease like Malaria, TB, Polio and other infectious diseases. The Onchocerciasis focal persons are in charge of all communicable disease at the Woredas visited. The plan of the Woredas makes provision for all key CDTI activities and community requirements. According to head of the health office, planning is participatory involving all key stakeholders and approved by the Woreda council, However Evaluators did not find evidence to confirm partners’ participation. Integration (3.0): At this level there is integrated annual health plan where CDTI is incorporated. Though there is no written evidence, the staff combines two or more tasks on a single trip and also combines activities with other health programs. But there is no written evidence to confirm how CDTI is implemented in an integrated manner with other health activities
Leadership (4.0): In the two Woredas visited the management team at this level initiates key CDTI activities evidenced by the presence of a detailed timetable stating the time that each CDTI activity will take place. There is a focal person in charge of CDTI in the Woredas visited. These focal persons are also responsible for other health activities of their respective Woredas. Supervision/Monitoring (3.3) The CDTI activity reports are communicated through the government reporting system. The Communities report to the FLHFs and FLHFs to the Woreda Health Office. The Woreda Health Office then reports to Zonal Health Desk. But there is delay in submitting 27 reports especially from the lower levels (CDDs).The reports being sent contain all necessary data concerning CDTI activities. Financial report of all health activities is available with Wereda finance office. At this level supervision by the Woreda staff is carried out regularly during Mectizan distribution time. Supervision is conducted using supervisory checklist. The woreda supervisory visits mainly target FLHFs; however, spot check visits are sometimes made to the kebeles/communities. CDTI supervision checklist was seen. it is not integrated with other health activities and there is no written supervision report. Problems observed during CDTI supervisory visits or obtained from other sources are addressed promptly in collaboration with the appropriate person in charge at all level. If the problem is not solved by the responsible person in charge of that level, it will be reported to the next level for solution. Written reports were unavailable to confirm how success and feedback were handled.
Mectizan Procurement and Distribution (3.4): Mectizan orders are based on needs and requests from FLHFs and it is also based on the census report of CDDs. Normally, the Woreda Health Office collect Mectizan from Zonal Health Desk. Mectizan is available on time and adequate. No shortages were reported in the 2009 distribution, But in 2008 there was little shortage of Mectizan in both Woredas and was resolved by getting additional tablets from the zone. Mectizan collection, storage and delivery to lower level is within government system and it is effective, uncomplicated and integrated. Woredas collect their Mectizan supply from zonal level. A Government inventory module called module 19 for receiving and module 22 for issuing out is used at this level for drug and equipment control including Mectizan. Transportation cost is covered from the government as Mectizan is transported in integration with other drugs.
Training/HSAM (3.5): Training for both Woreda Health staff and FLHFs workers is given by Zonal Health Desk. The reason the zonal level staff gave for this is for making the training strong. But in 2008 training for Woreda Health staff was given by the Zonal Level health staff and the FLHFs staff were trained by the Woreda Health Staff. Trainings are conducted routinely and not targeted at specific needs of the staff. Training topics does not seem integrated with other health activities. Training is given to both and old staff every year. The trainings focus on signs and symptoms of Onchocerciasis, Its Epidemiology, and Treatment, role of CDDs and supervisors and Census updating. No training module was seen at any of the Woredas visited. Political leaders are well oriented about the CDTI Program and they are supporting the program and are involved in HSAM activities. HSAM activities seem targeted and effective though there is no written report.
Financial resources (2.0): Income statement and ledger showing the amount of money transferred for the program from APOC and Light For the World as well as expenditures were seen at Zonal 28
Finance Unit. Bank Statements were also seen as well as retirement record of disbursed fund. Documentary evidence of Government release of fund for CDTI activities were not seen but the responsible persons claim that since CDTI is included in the overall annual health activity plan it has mandate to utilize funds allocated for the health activities in an integrated manner with other health activities especially for fuelling and maintenance of motorcycle.
The management is aware that APOC funding will stop after five years . But there is no evidence that the management have initiated any sort of plan to mobilize funds from dependable sources. The management is confident that since CDTI is one of the health activities, the program will continue without interruption using government However there is no written commitment or plan yet to support the above claim. For all CDTI related activities at Woreda Health Offices, expenditure approval is given by the head of Health office. There is a strong central control of finance by the Woreda Finance Office. All the support given from Zone to Wereda for CDTI activities is managed centrally by the Woreda finance office. Finance office clearly record allocation of fund. When ever there is income and expenditure the accountants calculate it immediately and record the balance amount. The finance office is transparent. Transport and Other Materials (3.0) There are about 8 motorcycles available for integrated health programmes at the two Woredas visited, two of which were provided by APOC five years ago. The APOC motorcycles are however getting old and not functioning effectively due to difficult train and rough roads. Government regulations permit that the government budget to be used for the maintenance of donated capital equipments, accordingly the motorcycles seem to be maintained on a regular basis. The Woreda Health Offices approves movements of the motorcycles. All available transportation resources are put together in a pool to be used for all health related program. Trips are properly authorized by the Woreda Health Offices. The control mechanism is strong. Though the management at this level are fully aware of the necessity of replacing the existing motorcycles there is no written realistic plan so far made to replace them. According to Head of the Heath Office, since motorcycles available for the Woredas are in a pool to serve all health activities CDTI will not be left out. The Woreda Health Office is unable to replace the motorcycles because government budget allocated for the health activities is very minimal.
Human resources (3.0) Though there is no guaranty for staff to stay in their position for long, the Onchocerciasis focal person and his assistant has been in their position for the last two years. Staff members responsible for CDTI at this level are committed and willing to serve their people. And staff salary is paid regularly every month. They are satisfied by the commendation they get from the beneficiaries. But no one has been rewarded because of CDTI activities. 29
Trained staff available for CDTI activities at this level are inadequate and do not have enough skill on need assessment identification, targeted HSAM, data management /Reporting.
Coverage (4.0) The two Wereda has achieved and maintained 100% geographic coverages over the past three years. Average therapeutic coverage over the past three years was 81% Lecha Dulecha therapeutic coverage was 82 % in year 2008 while Guto Gida had therapeutic coverage of 73%.In 2007,Leka Dulecha had 80% therapeutic coverage and Guto Gida had 81% and in 2006, Leka Dulecha and Guto Gida had 84% and 86% therapeutic coverages respectively.
3.2.1: Recommendation for the Woreda/district Level
Table 3: Recommendation for the Woreda/district Level: Recommendation Implementation Planning: Priority: HIGH The LGA plan should be reviewed Indicators of success: along with relevant partners in order to a)Availability of revised CDTI plan address areas of weaknesses and the b) Availability of minutes of partners Evaluation recommendations. planning meeting . Who to take action: WOFP, WHDH Deadline for completion: October 2009 Integration of support activities: Priority: HIGH Document how CDTI is implemented in Indicators of success: integrated manner with other health Availability of reports on integrated activities as indicated in the Integrated health activities that was used ti Annual Health Plan of the Woreda implement CDTI. Health Office Who to take action: WOFP, WHDH Deadline for completion: April 2010 Leadership: Priority: HIGH Ensure that the Woreda provide Indicators of success: effective leadership for the project by Availability of enough staff for proper putting in place enough trained supervision and effective delegation of personnel that can support the CDTI activities. implementation of CDTI activities. Who to take action: WHDH ,ZOFP Deadline for completion: October 2009. Monitoring and Supervision: Priority: MEDIUM 30
Monitoring and supervision reports Indicators of success: should always be written using the Availability of written monitoring and information obtained from the checklist. supervisory reports. These reports should be properly filed Who to take action: and used for follow up, feedback and WHDH, ZOFP performance review at the Woreda. Deadline for completion: April 2009. Mectizan Supply: Priority: HIGH Mectizan ordering and distribution Indicators of success: should be made based on updated a) Adequate supply of Mectizan at the census to avoid shortage. Woreda Woreda Onchocerciasis focal person should communicate to the lower level that b) Availability of accurate census data CDD census updating is part of at Health Facility and Community distribution activities. levels for Mectizan ordering and Effort should be made to record Batch distribution. and Lot numbers of Mectizan received Who to take action: at Zonal level as well as the expiration WOFP and the Woreda Pharmacist dates. Deadline for completion: April 2010 Training and HSAM: Priority: HIGH The skill of Woreda staff involved in Indicators of success: CDTI implementation need to be built a)Availability of skilled staff for training on how to carry out need assessment and HSAM at the Woreda level for training and HSAM. b)Targeted training and HSAM carried The FLHFs should be empowered to out at Woreda level and the lower be able to address in an integrated levels. manner training and HSAM needs at Who to take action: their level and the community.. WHDH ,ZOFP Deadline for completion: Training and HSAM should not be October 2009 carried out routinely. It should be targeted at addressing needs.
Finance Priority: HIGH Woreda Government should fund CDTI Indicators of success: activities by increasing its budgetary a) Increased amount budgeted and allocations and ensure timely releases released for CDTI activities. for CDTI implementation. Evidence of Woreda actual release b) Documentary evidence of Woreda from the Health budget for CDTI support to CDTI available. activities should be available. Who to take action: Woreda Administrator, WHDH and WOFP 31
Deadline for completion: October 2009 Transport and other material Priority: HIGH resources: Indicators of success: The Woreda should liaise closely with a) Availability of sustainability plan the Zone to ensure they get adequate addressing the transport and other transport and other materials needed material needs of the Woreda. for CDTI implementation in view of b)Availability of capital equipments sustainability. and other Materials for CDTI activities . Who to take action: WOFP, ZOFP ,APOC and LFTW. Deadline for completion: a)October 2009.b)Dec 2010 Human Resource: Priority: HIGH Increase the number of trained Indicators of success: personnel available to support CDTI Availability of sufficient trained Zonal implementation at the Woreda. team for effective CDTI Woreda project staff should be trained implementation. on need assessment, data Reward system in place for hard management and documentation. working staff A reward system should be instituted Who to take action: by the Woreda Head Desk to reward Head of Zonal Health Desk well performing staffs Deadline for completion: (a)January 2010 (b)December 2010 Priority: HIGH . Coverage Indicators of success: Sustain 100% geographic and achieve a)Availability of list of all 80% therapeutic coverage by ensuring Onchocerciasis endemic communities that list of endemic communities and at the Woreda. their populations are available for coverage data verification b)Availability of verified treatment coverage data Who to take action: WOFP and ZOFP Deadline for completion: October 2009 32
3.3 Sustainability at the Health Facility level
Figure 3. Average Performance of Each group of indicator at FLHF level
4 4 4 3.7 4.00 3.3
t 3.50 3 3 h
g 3.00 2.5 i
e 2.50 w 2.00 e
g 1.50 a r 1.00 e
v 0.50 a 0.00
l g n ip n y e a e e n io io l M c ri c g i t sh s p A n r ra n a r i p S a te u e n r e rv u n a o v la g d e S H i s o P te a p & F m re n e n g C I u a n t& n L s iz i r a & t in o g c a p m n e r s u ri M T n H o ra it n T o M Group of indicators Planning (2.5): Written plans for Year 2009 CDTI were seen pasted on the wall of 3 out of the 4 Health Post visited. All activity plans were written in local language (Oromifa).It was only in Bedo Health post that written plans were not seen .The health facility in charge of Bedo Health Post said she had it in her head but forgot to write it on paper and paste. Integration takes place by the same staff in charge of CDTI activities but there no written integrated health plan seen at this level. HFs works in close collaboration with Kebele administrators in carrying out their duties.
Integration (3.0): The health Facility staffs are in charge of all the health programmes in their kebele. There are two Health staffs per facility. CDTI activities are usually carried out in an integrated manner. The HFs equally combines CDTI activities with other health programmes in the area.
Leadership (3.0): The health facility staffs take full responsibility of CDTI activities in the Kebeles under their supervision. In each Health Faculty two female Health extension workers are available and both are responsible for CDTI activities and shares responsibility amongst themselves. In all the Health post visited, Health staff explains that their annual CDTI activities starts with a meeting with their Kebele administrator. They meet to agree on how to mobilise the community through their community leaders. Activities usually discussed and agreed upon include census updating, mobilisation and Health Education, date of distribution, training date for CDDs and any other issues of concern that the Health staff might have. 33
The Community structure is very much aware of CDTI and lends support to the Health Facility staff. However no written report of the planning meeting was seen at all the health post visited.
Supervision/Monitoring (3.7) CDTI supervisory data is being transmitted entirely within the government system. The health Facility summarizes the activities carried out in her kebele and then reports to the focal person at the Wereda who then forwards it to the zone. At all the 4 Heath Facilities visited CDTI data were seen. The record on CDD trained record, Treatment data and Mectizan inventory. Treatment summary for three years were seen pasted on the wall of all the Health post visited. This summary provided information at a glance on total population, Target population, Number of communities target and that treated refusal and absentees etc. With this information, it is easy at a glance to know the performance of the Health facility. The management structure in operation at the Health Facility makes routine supervision of CDTI activities easy and efficient. Each Health Facility in the zone is manned by two female Health Facility staffs. The Health Facility staffs lives in the community where the facility is located .They are appointed for training for that position by their community through the Kebele Administrators but paid by the government . For effective supervision the two health staffs are usually assigned to different areas in the Kebele for CDTI activities and other health programme like environmental sanitation, family planning home visits etc. Movement chart for each of the health staff was seen on the door of the Health post to tell visitors on their where about. The Health Staffs have agreed with the community on the day to be at the health centre or time to attend to antenatal issues, every other time they are in the community supervising and attending to Health needs. Supervisory checklist were used but no summary report of supervision activities was written. Health Facility staff tries to address identified problems on their own since they are part of the community. Those that they are unable to handle they refer to the community leaders or the Kebele administrators. Success and Hard work are usually rewarded, sometimes by using them as guide during polio campaigns or any other activity that requires a committed community volunteer. According to the Health staff , community selecting a CDD gives the CDD a sense of being respected and recognized by the community. Mectizan Procurement and Distribution (4.0): The FLHF staff uses similar inventory stock order for drugs and equipment used by the higher level .They use Model 19 form for receiving Mectizan. FLHFs requests for Mectizan from the Woreda with a request letter written and stamped by the health Facility staff. Copies of these were seen in all the Health Facilities visited. At Bedo and Kewissa health posts they complained, there was excess supply of Mectizan during the 2009 distribution period. This they explained was as a result of teachers and farmers population movement out of their area. 34
The CDDs receive their Mectizan allocation from the FLHFs based on the request calculated using updated census. CDD usually go to the HFs either at home or in the office to collect more Mectizan or return the excess when they complete distribution. According to one Health Facility Staff; “Mectizan is very important to me and my people, because of that I usually carry it home for safety and bring it back when I come to work. This is a temporary arrangement till I have a security guard for the Health post---Mesekerm Wakjira-Health Facility Staff in charge of kitiesa Health Post . Guto Gida Woreda. The Health Facility receives their Mectizan supply from the Woreda
Details like Mectizan Batch number, Lot number, and expiring dates were however not seen in their record.
Training/HSAM (3.3): Training is carried out routinely. FLHFs train CDDs annually in order to refresh their memory on CDTI. All the Health Facility staff visited lacked training need identification skill. No refreshment or transport is provided for the CDDs when they come for training at the Health Post. According to one of the Health Facility Staff; CDDs have accepted this work as a voluntary service to their community, no refreshment nor is transport provided because their homes are not far and they must have eaten at home” The Kebeles support the training by ensuring that every community send their CDDs for the training . The training of CDDs according to the FLHFs takes one day and an average of Eight hours. One striking observation regarding CDD training in the Zone is the inclusion of construction of measuring sticks in the training topics. Evidence of this was seen by the Evaluators when communities were visited. CDDs showed measuring sticks of different designs made with local materials but accurately calibrated. HSAM activities are usually done routinely. Community awareness of the benefits of Mectizan is high and this has led to high treatment compliance. However new issues like amending distribution time to accommodate migrant farmers, or sustained compliance when signs of Onchocerciasis disappears are not addressed. This is because HSAM is carried out routinely. There is lack of skill to identify what the annual objective of any round of HSAM should be.
Financial resources (3.0): There is no budgetary allocation at this level. The Government/Community system used in managing the Health Facilities is functioning effectively. The Health Post don’t manage budget. According to one of the Health Staff she said during the last distribution exercise, APOC provided supervisory allowance for 3 days for them but they continued to work until distribution exercise was completed because activities are implemented in an integrated manner. The FLHF workers do not get additional field allowance from Government only their salary. This is because they are working in their own communities. Below is the comment by one of the Health Staff ; 35
“In managing our Health Post, most of the materials we use are provided by the Woreda, we only use it as instructed by them to serve our people because is our community that appointed us . Little things like pen or pencil for our activities we can buy from our salary - Alganesh Degago, Head of Loko Health Post -Guto Gida. Woreda.
Transport and Other Materials (4.0) There is no transport provision at this level since all Health Staffs lives in the communities where their facility is located. They trek to do their work Activities are integrated, so that when they go to the Woreda for other health activities they equally transmit CDTI data. Training/HSAM materials are available and they are provided by the Woreda. Training/HSAM was sufficient and already translated in the local language.
Human resources (4.0) Front Line Health Facility staff managing the Health Facilities are quite stable as they appointed by the community members and belongs to that specific community. Evaluators were informed that one of the criteria for the appointment is that they remain and continue to work for the community even after marriage.
Coverage (4.0) In all the four visited FLHFs the geographic coverage for the last three years is 100% and the therapeutic coverage is greater than 65%.
3.3.1: Recommendation for the FLHF Level: Table 4: Recommendation for the FLHF Level: Recommendation Implementation: Planning: Priority: HIGH Effort should be intensified by the HFs Indicators of success: at ensuring that a detailed copy of the Availability of integrated health plan. integrated Health plan is available at Who to take action: the Health post. WOFP,HFs. Deadline for completion: August 2009 Integration: Priority: HIGH Ensure that the documentation of Indicators of success: integrated activities is available for Availability of report on integrated effective supervision and monitoring by activities. the higher levels. Who to take action: WOFP,HFs. Deadline for completion: April 2010 Leadership MEDIUM There should be documentary Availability of minutes of meetings of evidence of the HFs meeting with the HFs with Kebele leaders and other Kebele leaders and other key partners. 36
stakeholders Who to take action: WOFP,HFs April 2010 Monitoring and Supervision: Priority : HIGH Target supervision on problem areas Indicators of success: should be given high priority by the Availability of written report and Health Staff. provision of supervision feed back. Checklist for supervision should be Who to take action: summarised and supervisory visit WOFP,HFs report written. Deadline for completion: Record of feedback given based on January 2010 supervisory visit should be made available at the Health Post. Mectizan Supply: Priority: HIGH FLHFs should emphasize to the CDDs Indicators of success: that census updating is part of a) Availability of Community Mectizan distribution activities and should be request based on accurate census. carried out at the same time. This will b) Mectizan data available with batch enable accurate calculation of Mectizan and Lot numbers as well expiring need of the community. dates. . In order to monitor Mectizan use and Who to take action: tracking in the event of pilferage, HFs Mectizan Batch and Lot number as well Deadline for completion: as , expiring dates should be recorded. March 2010
Training and HSAM: Priority: HIGH Training and HSAM should be targeted Indicators of success: at areas with need in view of Availabilty of more efficient CDDs programme sustainability. especially in identified areas of need like census updating and recording of treatment. b) Improved therapeutic coverage due to targeted HSAM. Who to take action: HFs and WOFP Deadline for completion: October, 2010 Coverage Priority: HIGH Sustain 100% geographic and achieve Indicators of success: 80% therapeutic coverage by ensuring a)Availability of verified treatment that coverage data are verified coverage data at the Health Facility Who to take action: HFs Deadline for completion: October 2009 37
3.4 Sustainability at the Community Level
Figure 4. Average Performance of Each group of indicator at Community level
3.9 4 3.9 4 3.7 4 3.5
t 3.5 3 h 2.8 g 3 i e 2.5 W
e 2 g
a 1.5 r e
v 1 A 0.5 0
g n ly s s e n ip o p M e e g i h i p A c c a n s is S n r r n r v u a u e la e r S H n o v d e n & i s o P a p a g F e C e u z n R . L S ti i n T & c in a g e a n M r m ri T u o H it n o M Group of Indicators
Planning (highly, 3.7) The CDDs are enthusiastic about CDTI implementation. They work closely with Health Facility staffs, kebele leaders and village chiefs .The CDDs plan and manage CDTI activities by; Conducting census updating, Requesting for Mectizan based on census population, Informing communities the specific date of Mectizan distribution and reporting back on treatment to the HFs. The village registers were seen at the Health Facility and confirmed that the above activities have taken place.
Leadership (highly, 3.9) Kebele leaders as well as village chiefs have impressive knowledge of CDTI activities. They know the impact of the disease, eligible population, as well its benefits. They also participate in census updating; social mobilization and monitoring Mectizan distributions Surprisingly, most CDDs are appointed as village chiefs so that they are able to solve CDTI problem when it is encountered. If there are problems like shortage of Mectizan and refusals the Community leaders inform the FLHF who lives in the community and ensure it is addressed.
Community members interviewed reported that they were involved in CDD selection and change them when necessary. They are equally involved decision regarding time and place of distribution. 38
Monitoring and Supervision (fully, 4.0) CDDs are reporting their activities to Health Facility staffs, Some CDDs are literate enough to summarize their activities and report it using village summary reporting format, others submit the completed village registers and summary is done by the front line Health Facility staff. In most of villages visited, CDDs have already completed distribution and submitted their register to the Health post. When inquired from the Health Facility on the reason for keeping the Village Registers, One of them had this to say; “It was the decision of the Kebele administrators taken after we lost some registers to communal clashes. Some of them also do not keep it well, they allow their children tamper with it.-Miss Alganha Dagando-Loko district of Guto Gida.
Mectizan Supply (highly, 3.5) Village chiefs and CDDs reported that they received sufficient amount of Mectizan for 2009 treatment period. CDDs requested the drug based upon updated census population. CDDs revealed that unused drugs for absentees and refusals were returned to Health Facility levels. The community register on total population incorporates some invalid data such as migrated household members and some persons that are dead. This affects their therapeutic coverage and drug requirement. In addition, in Kitiesa, Health Facilities of Guto Gida Woreda, there were variations in the amount of Mectizan balance when compared with Woreda report. Mectizan is collected by the CDD from nearby health post not far from their community. They trek down for collection because the Health post is quite close to them.
Training and HSAM (Moderately, 2.8) Comprehensive information was collected from village chiefs and CDDs regarding Health Education. Community members reported that Health education was provided at community meeting and house to house visits before inception of Mectizan distribution. Besides this, community members confirm that CDDs informed them on mode of treatment ,benefits of the drugs and possible side effects. CDDs pledge to maintain health education activities until the disease is eliminated. For effective communication of the CDTI messages to the communities, the CDDs showed their manual and Posters that has translated into Oromifa language (local language) which they appreciated a lot. Community members have positive perceptions towards CDTI. They have given high value to Mectizan. They wish they will be allowed to take Mectizan for unlimited number of years. All community members elaborated that Mectizan has multiple effects on their health. Besides Onchocerciasis, Mectizan kills lice, bugs, relief from persistent itching and expelling intestinal worms. Accordingly, communities are demanding to have Mectizan distributions for every six months. In line with this, communities noted that, “.we need Mectizan for longer period even for our future generation. If the drug distribution is delayed or interrupted we will appeal through our kebele and will fight to get it up to Woreda” Community meeting in Haro village of Kitiesa Health post-Guto Gida Woreda 39
Training is not targeted. It is routinely done without need assessment. CDDs as well as the health extension workers reported that CDDs received routine refresher training every year. The training was conducted at the FLHF levels for one day. The facilitators of the training were the health extension workers. The good thing is that CDDs are knowledgeable on CDTI programmatic activities- know who is eligible for Mectizan, able to express side effects of Mectizan and well acquainted with Onchocerciasis. It seems also that the communities decided not to provide any kind of support to CDDs because they believe that it is a great honor to be appointed to serve their community.
Financing (highly, 3.0) Community incentive to CDDs is not common in communities visited even the CDDs do not expect it. It is evidenced in this statement; “I am treating my community and my people why should I expect incentive from them. The honor of selecting me to serve them and the moral appreciation I get from them is enough.” –Mr. Taye-Tafese from Kambi Village of Kewissa Head post of Lecha Dulecha Wereda. “Even if my community gives me incentive, I will be uncomfortable to accept because from serving them as CDD they have given the honor of becoming their community leader, that is enough for me and I appreciate them.”-Mr Alemu Derge CDD of Wanebo community of Bedo Health Post of Lecha Dulecha. Some community members, invited CDDs to have coffee during distribution time. One CDD however mentioned that incentive is needed for the CDTI work.
Human Resource (highly, 3.9) Data showed that Community members ratio to CDD is about 1:80 across all communities. This shows that one CDD treats about 10-16 households. CDDs are living close to the households that they serve and somehow related. Some are blood relation, others marriage relation and others close neighbors. Most villages are confined together so that CDDs do not travel long distance for distribution. CDDs are willing to continue to distribute Mectizan because they are happy to protect their communities from Onchocerciasis and other diseases like persistent itching, lizard skins and blindness. A few CDDs said if possible APOC should provide small amount of payments during distribution time, like payments received from immunization campaign and other out reach health service programs. Evaluators advised him to learn from other committed CDDs. There is no significant CDD attrition in the areas visited.
Coverage (fully, 4 .0) The partial 2009 distribution data indicated that the therapeutic coverage ranges from 69% to 75% The 2008 data shows that therapeutic coverage ranges from 68% to 80 % across all villages. In 2007 and 2006 treatment data indicated that therapeutic coverage were between 57% to 84% and 72% to 80%, respectively. Therefore, the weighted average of therapeutic coverage for visited communities is 73% from year 2006 to 2009. 40
3.4.1: Recommendation for the community Level
Table 5: Recommendation for the Village Level
Recommendations Implementation Training/HSAM Priority: HIGH Training and HSAM should be need Indicators of Success: based and targeted for cost a)Availability of enough CDDs in the effectiveness and efficiency community . If communities are willingly to select b)High community awareness on the need more CDDs this should be encouraged for long term compliance and training conducted for them. The option of using old and Who to take action: HFs, CDDs and experienced CDDs to train others WOFP should be explored but supervised by Deadline for completion: the HFs. March 2010 HSAM should address the need for continued compliance even when signs of Onchocerciasis are no longer visible Mectizan Supply Priority: HIGH Mectizan should be requested based upon accurate census data. The Indicators of Success: community registration book should not Accurate updated census of endemic contain invalid data like migrated communities. households and died population. Calculation of coverages based on Census updating should form part of accurate census. distribution activities. Who to take action: CDDs, FLHF and Woreda Onchocerciasis focal person
Deadline for completion: February, 2010
Finance Priority: MEDIUM If the kebeles are willing they should be Indicators of Success: allowed to provide refreshment and any Enhanced committed and programme training material they deem fif during ownership by Kebele administrators. CDD training Who to take action: FLHF levels and Woreda health staffs
Deadline for completion: February May, 2010 41
4.0. Comparative Analysis of the Sustainability of the Four Levels
4.1. A quantitative judgement, based on the grades given to individual indicators
The scores given for the various sustainability indicators during the Evaluation of East Wollega CDTI project is shown below.
Table 6: Quantitative indicators at different levels of the project
Levels Groups of Indicators Average
Planning Integratio Leaders Monitorin Mectizan Trainin Finances Transport Human Coverage n of hip g & supply g & Resource support Supervisi HSAM s activities on Zone 2.3 3.5 3.0 3.1 3.5 2.0 2.0 2.3 3.0 4.0 2.8 Woreda 3.0 3.0 4.0 3.3 3.4 3.5 2.0 3.0 3.0 4.0 3.3 FLHF 2.5 3.0 3.0 3.7 4.0 3.3 0 0 4.0 4.0 3.4 Commu 3.7 0 3.9 4.0 3.5 2.8 3.0 0 3.9 4.0 3.6 nity Average project sustainability score 3.3
Figure 5. Quantitative judgment scores at different level
4 3.6 3.4 3.3 3.3 3.5 2.8 3 Average weight 2.5 2 1.5 1 0.5 0 Zone Woreda FLHF Community Overall (Project level) Project level
The average overall grading of the nine groups of indicators for the entire project is 3.3. The Community scored highest in terms of CDTI implementation with an average score of 3.6.followed by the Health Facility scoring 3.4. 42
This makes the community to have a “Full” sustainability potential using the 9 group of indicators. The Wereda and the zone scored 3.3 and 2.8 respectively and this performance groups them into the rating of having “High” sustainability potential.
Figure 6: Average performance of the different indicators for the entire project
From the above figure it is clear that finance, followed by transport then Training/HSAM and planning are the group of indicators that are weak in the East Wollega project. Coverage, Mectizan supply, Monitoring and supervision,leadership and Human resources were seen to be strong. Figure7: Average performance of the different indicators at different levels
Figure A. Quantitative Indicators by project by project levels 4 4 4 4 4 4 4 4 4.0 4 3.9 3.9 3.7 3.7 3.6 3.5 3.5 3.5 3.5 3.5 3.5 3.5 3.5 3.4 3.3 3.3 3.2 3.1 3 3 3 3 3 3 3 3 3 3 2.9 2.9 2.8 2.7 2.5 2.5 t
h 2.3 2.3 2.3 g i e
w 2 2 2 2 e g a r e v
A 1.5
1
0.5
0 0 0 0 0 Planning Integration Leadership Monitoring Mectizan Supply Training &HSAM Finance Transport Human resource Coverage &supervision Groups of Indicators
Zone Woreda FLHF Community At project level (Average) 43
At all the levels coverage was very good as well as Mectizan supply. Planning was best at the community level. Integration was not applicable at the community level, best at the zone with a score of 3.5.Leadership was best at the Woreda level with a score of 4.Monitoring and supervision of CDTI activities were best practiced at the community by the health extension workers. Training and HSAM was best at the Woreda level. Transport was not applicable at the FLHF and community in this project but was better at the Woreda than the zone. Finance was best at the community, non applicable at the FLHF and had the same score of 2.3 for both Woreda and the zone. Human Resource is best at the FLHF and the community.
4.2. Qualitative judgment of project
Table 7: The performance of different aspects of sustainability Judgment: To what extent is this aspect helping or Aspect blocking sustainability in this project? Integration East Wollega CDTI project was judged to be highly integrated into the routine running of health care services at all levels especially the Health Facility ,this is highly helping sustainability. Resources The Health Facilities and community had sufficient committed number of human resources needed for CDTI .This is helping sustainability.
Human, Material and financial resources available at the Woreda and the Zone were inadequate for CDTI activities and this is blocking sustainability
Efficiency CDTI project in the zone is run in a cost efficient manner especially at the community and Health Facility level, This is highly supported by Government policy on integration of health programmes. This is helping sustainability.
However there is skill gap at all levels amongst the Health workers involved in CDTI activities especially skills required to carry out targeted activities identified through efficient monitoring and supervision. Documentation skill is equally poor. This is blocking sustainability Simplicity CDTI is implemented at all levels using very simple Government routines and procedures integrated into Primary Health care. This is fully helping sustainability Health staff At all levels, there was sufficient evidence to show that acceptance Health Staff are committed and will be willing to continue 44
(Attitude of the to implement CDTI as part of their routine activities with or health staff) without additional reward. This is helping sustainability Community Communities visited in East Wollega during the ownership Evaluation makes most of the decisions in the implementation of CDTI activities and indicated that in the event of any interruption they will seek help for continuation at the highest possible level. Mectizan® is highly appreciated by communities in this project because of its benefits enumerated severally like, deworming, clearing of hair lice, Eczema, itching etc.
Communities are aware that they own the programme and Mectizan is for their benefit. This is highly helping sustainability Effectiveness Geographic coverage rate has been high as well as therapeutic which is one of the expected outcomes of CDTI programme implementation. The Evaluators therefore concluded that the programme is implemented in an effective manner. This is helping sustainability.
Grading the seven aspects of sustainability, Evaluators concluded that five of the aspects were completely helping sustainability. These are integration, Community ownership, Staff attitude, simplicity and Effectiveness. Resources partly help sustainability especially at the Health Facility and community levels whereas it blocks sustainability at both the Woreda and the Zone.Efficiency is partly helping sustainability because CDTI is implemented routinely in an integrated manner alongside other Health Programmes. Efficiency is also blocking sustainability because of the competency of most of the health workers involved in CDTI implementation who have skill gap in areas like documentation and targeted activities. If a percentage performance is used ,the aspects scored about 85% in terms of its support to sustainability.
4.3. Judgment using the five key aspects of the project – ‘critical elements’ of sustainability Table 8: CRITICAL ELEMENTS OF SUSTAINABILTY Money: Is there sufficient money NO available to undertake strictly necessary tasks, which have been carefully thought through and planned? (Absolute minimum residual activities). NO Transport: Has provision been made for the replacement and repair of vehicles? Is there a reasonable assurance that vehicles will continue to be available for 45
minimum essential activities? (Note that YES ‘vehicle’ does not necessarily imply ‘4x4’ or even ‘car’). Supervision: Has provision been made YES for continued targeted supportive supervision? (The project will not be sustained without it). Mectizan® supply: Is the supply system dependable? (The bottom line is that YES enough drugs must arrive in villages at the time selected by the villagers). Political commitment: Effectively demonstrated by awareness of the CDTI process among policy makers (resulting in tangible support); and a sense of community ownership of the programme.
4.4. Conclusion
Based on the seven aspects of sustainability and in line with the guideline for grading the whole project using the five critical elements of sustainability ,the Evaluation Team concludes that the East Wellega CDTI project is MAKING SATISFACTORY PROGRESS TOWARDS SUSTAINABILITY. Transport and money are challenged at the moment and not dependable.
Supervision is good at all levels using the integrated Health system of the government.. Mectizan supply is carried out efficiently using Government system which is dependable and provides enough supply and available on time.
There is good political commitment at all levels including community resulting in a high sense of ownership however there is poor political awareness on the need for a written plans to continue to make CDTI programme function effectively in foreseeable future without external funding. The aspects blocking sustainability of the programme in the Zone is Resources. There is no written evidence that sufficient money will be available from Government to undertake strictly necessary tasks, which have been carefully thought through and planned to sustain CDTI programme in the Zone.
In addition no provision has been made for the replacement of vehicles and motorcycles, although at the moment, maintenance is effectively done using Government dependable resources but replacement is obvious in the foreseeable future and plans are not yet in place for replacement.
The Evaluators also concluded that the quantitative score of 3.3 for the pooled groups of indicators at the four levels supports the above qualitative score. 46
5.0. Feedback/sustainability planning plan development workshops
Zonal The East Wollega Zonal feedback meeting was held on the 27th of May 2009 at the Office of the Deputy Head of Health desk. Those in attendance were the deputy Zonal Administrator, The deputy head of the Health desk, the Zonal Onchocerciasis focal person and three additional staff from the Health desk pool. List of participants is attached Annex-1
The Evaluation objectives, Methodology and Findings of the East Wollega evaluation was presented to the Zonal team as well as the conclusion that East Wollega CDTI project was making satisfactory progress towards sustainability
The Zonal team sought for clarification regarding the scores and the results presented. Appropriate explanations were given. At the end the Zonal team agreed with the findings as a true reflection of the present state of CDTI implementation in the zone. The Zonal Onchocerciasis focal person was happy about the impressive performance of the Health Facility level and the community adding that it shows that they are strengthening the lower level who should own the programme. He promised that the Zone and Woreda will ensure that there is improvement in the highlighted weak areas. The Zonal Administrator assured the Evaluators that the Zonal team will hold further discussion and agree on the way forward which will be in cooperated in their sustainability plan. He thanked the Evaluation team for their hard work and the feedback.
After the feedback meeting, there was a briefing of the Zonal team on the development of sustainability plan. The Evaluators went through the sustainability plan guidelines with the Zonal team and the plan development commenced and continued during the remaining days of the planning meeting.
Woreda The Woreda level feedback meeting commenced on the 28th of May 2009. A total of 36 participants were in attendance. Participants wer/e drawn from the 10 CDTI Woredas and two zonal offices. Participants were Woreda Onchocerciasis focal persons, Head of health desk of the 10 Woredas and their Woreda administrators. East Wollega deputy Zonal administrator, Deputy Head of health for East Wollega and Head of Health desk of West Shoa . East Wollega Onchocerciasis focal person and some other experts from the East Wollega Health desk unit (see lists of participants in Annex -2).
The meeting started with an opening remark by Mr. Sitotaw Fito, the deputy Zonal Administrator of East Wollega. In his remark, he reminded participants that this opportunity of being part of the feedback /planning meeting for the zone should be used as a training opportunity for them and ensure that it is done carefully so that accurate plans are available at the end of the workshop. 47
The Evaluation findings were presented by the Evaluators, the following were the questions asked by the participants; a) There are some Woredas that are at risk of Onchocerciasis but were not included during the launch of CDTI, what is fate of these Woredas? Can they request for Mectizan? b) Why was community score low regarding training &HSAM indicator? c) How can Woredas assign a focal person for Onchocerciasis in the era of BPR( Business process re-engineering ). BPR does not support the assignment of a single person for specific program .
The relevant persons provided further clarifications to the above concerns.
The participants were pleased with the findings and were very willing to be part of the group work for the SWOT analysis and the development of the sustainability plans.
Group work on different SWOT analysis for the different levels commenced and was followed by presentations. Comments and suggestions were made on the presentations and the different groups took note of it for amendment. The SWOT analysis is attached as Annex 4 .
The sustainability plan development continued on day three for the Zones and the different Woredas. A draft presentation on their group work so far was made before Tea break in order to identify areas of weaknesses in the plan development. It was obvious that participants needed to know more about on the guideline for the development of a post APOC sustainability plan. Mr. Chukwu Okoronkwo was asked to make a presentation on Key issues to note when developing a post APOC sustainability plan, this was quite useful for the participants.
Group work continued for the Zone and the 10 Woredas ,First to produce the three years plan and later the remaining 2 years plan.
On the fourth day, group work continued till Tea break time, after which presentation of draft plans commenced. Comments and clarifications were given after each presentation and the different groups were requested to make the necessary amendments on their plans. As the different Woredas were presenting their budgets, the attention of the Woreda administrators were drawn to the amount needed for CDTI and they all agreed that is realistic and they can make it available.
The sustainability planning meeting ended at about 4.30 pm. The deputy Zonal administrator of the East Wollega zone in his closing remark thanked all the participants and the Evaluators for their hard work and requested that since the Ethiopian financial 48 year usually ends in June, effort should be made by all concerned to make sure the budget proposed is in cooperated in the next financial year made is available on time for CDTI activities. Mrs. Olamiju thanked the Deputy Zonal Administrator, and the Woreda administrators for their commitment and participation throughout the 3-4 days workshop. She also thanked the Health team from the 2 zones and the Woredas and encouraged them to fine tune their plans as agreed and submit before they leave.
It was agreed that the Evaluation report and their draft five years sustainability plan will come back to them shortly for final input and signing. 49
ANNEXES
Annex 1--List of Participants at Zonal Feedback meeting S. No. Full Name Address Responsibility
1 Sitotaw Fufa Nekemte-Zone Deputy Zonal adminstartor
2 Bedasa Fite Nekemte-Zone Deputy head Zonal health
3 Ensermu Jeldu Nekemte-Zone Zonal Oncho focal person
4 Getahun Zewudu Nekemte-Zone Zonal CDC
5 Habtamu Siyum Nekemte-Zone Zonal Planning
6 Basha Nemomisa Nekemte-Zone Zonal CDC
7 Abata Tilahun Addis Ababa Evaluator
8 Francisca Olamiju Nigeria Evaluator
9 Firew Ayalew Addis Ababa Evaluator
Annex 2--List of Participants at Woreda Feedback meeting/sustainability plan development workshop
S. No. Full Name Address Responsibility
1 Endalew Geleta Guto Gida Guide
2 Busho Kumbi Guto Gida Onchocerciasis Focal Person
3 Abrham Bekele Guto Gida Woreda Adminstrator
4 Tilahun Dessisa Guto Gida Head, woreda health
5 Sitotaw Fufa Nekemte-Zone Deputy Zonal adminstartor
6 Bedasa Fite Nekemte-Zone Deputy head Zonal health
7 Ensermu Jeldu Nekemte-Zone Zonal Oncho focal person
8 Getahun Zewudu Nekemte-Zone Zonal CDC
9 Habtamu Siyum Nekemte-Zone Zonal Planning
10 Basha Nemomisa Nekemte-Zone Zonal CDC Woreda Onchocerciasis focal 11 Abiyot Bekele Diga person
12 Samuel Tolla Diga Woreda health head
13 Yonas Terfasa Diga Woreda Adminstrator 50
Woreda Onchocerciasis Focal 14 Temesgen Tollesa Sasiga Person
15 Getachew Asfaw Sasiga Woreda health head
16 Habtamu Tolla Sasiga Woreda Adminstrator
17 Lemesa Terefe Sirbu Sire Woreda health head
18 Chala Gemeda Sirbu Sire Woreda Adminstrator Woreda Onchocerciasis focal 19 Birhanu Meseret Sirbu Sire person
20 Shiferaw Kebede Jima Arjo Woreda Adminstrator
21 Kefalew Adnew Jima Arjo Woreda oncho focal person
22 Diriba Bekele Jima Arjo Woreda health head
23 Kitesa Mossisaa Leka Dulecha Woreda health head
24 Amsalu Tesema Leka Dulecha Woreda Adminstrator Woreda Onchocerciasis focal 25 Korsa Eba Leka Dulecha person
26 Ahmed Yimana Boneya Boshe Woreda health head
27 Gobena gemechu Boneya Boshe Woreda Adminstrator Woreda Onchocerciasis focal 28 Fromsa Hinkosa Boneya Boshe person
29 Busha Tesfaye Wama Agelo Woreda health head
30 Midekisa Gemechu Wama Agelo Woreda Adminstrator Woreda Onchocerciasis focal 31 Fikire Desalegn Wama Agelo person
32 Tariku Dengiya Wayu Tuka Woreda health head Woreda Onchocerciasis focal 33 Oljira Berkissa Wayu Tuka person
34 Abera Fita West Shoa-Ambo Zonal CDC Zonal CDC -Onchocerciasis focal 35 Gudisa Deyas West Shoa-Ambo person West Shoa-Bako Woreda Onchocerciasis focal 36 Sichala Kore Tibe person
37 Abata Tilahun Addis Ababa Evaluator
38 Francisca Olamiju Nigeria Evaluator
39 Firew Ayalew Addis Ababa Evaluator
40 Chukwu Okoronkwo Nigeria Evaluator 51
Annex 3-SUSTAINABILITY OF EAST WOLLEGA CDTI PROJECT FEDBACK/PLANNING MEETING
AGENDA
DAY ONE Item Activities Time Facilitator 1 Registration of participants 8:30-9:00 Ensermu 2 Introduction of participants 9:00-9.05 Ensermu 3 Welcome and opening remarks 9:05-9:15 Honorable guest 4 Introduction to the Workshop; What are the 9:15- Olamiju objectives and what is sustainability 9:10:00
Methodology for Evaluation Abate 5 Tea break 10:00- 10:30 6 Feedback on achievements, issues and lessons from the evaluation on sustainability of East Wollega CDTI project 10:30- Zone level 11:15 B Woreda level Olamiju HF level Abate Community level Olamiju Firew
7 SWOT analysis 11:15- Ensermu 11:30 8 Group work
Discussions on Problems identified and the 11:30- Olamiju, solutions to these problems using SWOT 12:15 Abate and analysis in Groups Firew Planning/integration/leadership/monitoring and supervision Mectizan/Finances/training& HSAM Transport/human/coverage 9 Report from groups discussions 12:15- Group 13:00 representative 10 Lunch 13:00- All 14:30 11 Report from groups discussions continues 14:30- 17:30 52
Day 2
Item Activities Time Facilitator 1 Registration of participants 8:30-9:00 Ensermu 2 Roles of the different levels and partners 9:00-9:30 Ensermu 3 Steps in planning for the sustainability in this 9:30- Olamiju project and grouping 10:30 4 Tea Break 10:30- 11:00 5 Group work on Development of 11:00- All sustainability plans 13:30 6 Lunch 13:30- 14:30 7 Group work on Development of 14:30- All sustainability plans continues 15:30 8 Presentation of Group work 15:30- Group leaders 17:00 9 Presentation on Key issues to note when 17:00- Chukwu developing a post APOC sustainability plan 17:30 Okoronkwo
Day three
Item Activities Time Facilitator 1 Registration of participants 8:30-9:00 Ensermu 2 Finalization of group work on Development 9:00- All of sustainability plans 11:00 3 Tea Break 11-11:30 4 Group presentation of draft sustainability 11:30- All plans 13:00 5 Lunch 13:00- 14:30 6 Submission of draft plans by the different 14:30- All Zones and Woredas 15:30 7 Administrative issues 15:30- Scout 15:30 8 Closing Remarks 15:30- Deputy Zonal 15:45 Administrator 9 Vote of thanks 15:45- Olamiju 16:00 10 Departure 16:00 53
Annex 4:SWOT ANALYSIS
A.SWOT Analysis for East Wollega CDTI Programme
S INDICAT STRENGTH WEAKNESS OPPORTU THREAT SOLUTION WHO N OR NITY PERFORM
1 planning -CDTI programme is -poor recording - -Shortage of Provide Training on HMIS -ZHDH and included in annual plan & Availability resources ZOFP -Utilization of pool Reporting of of (fund, system and BPR Annual, quarter supportive materials…) implementation & health -Active participation of Monthly plan policy and all partners HEW in the community
2 Integratio -proper utilization of Lack of detailed -presence Work-over Provide training on NOTF n resources document of political load documentation Chairman, commitme ZHO Head nt and focal (Zone- person community ) 3 Leadershi -sufficient awareness of - Poor staffing The -Turn-over of - Employ enough staff -Zonal p leaders regarding CDTI widening leader at administrator of colleges different level and WHO to produce Head educated manpower 54
4 Monitorin -Frequent supportive -Lack of written - Strong -Work over -Provide training on report - NOTF g supervision report and feed structure load writing and feed back Chairman, & -utilization of back from zone -un-stability ZHDH and Supervisi standardized to (uncertainty) ZOFP on Checklist community -Conducting annual level review meetings 5 Mectizan -Timely distribution of -Unable to -MERK -Mectizan Follow up the proper - Mectizan register supply the shortage or registration ZHDH,ZOFP Batch No.& product interruption Of batch No.& expiry date Head and expiry date freely zonal Pharmacist 6 Training -Attempt to train health -Note based on -presence -Provide training on Need - ZHDH and & HSAM workers of different level need of qualified assessment ZOFP -Translation of formats, assessment staff to guidelines to local -Un-targeted train language.
7 Finance -Utilization of -poor liquidation - Strong follow up & timely ZHDH, government of allocated substitutio Liquidation Finance Expenditure models money at district n of office and -Timely utilization level resources ZOFP 9 Human -Committed staff -High turnover -Presence Make staff stay on their - Zonal resource of staff of HEW & position for long administratio Committed n and ZHDH CDD 55
B.Solution for the observed weakness of East Wollega CDTI Zone SE. INDICATOR WEAKNESS SOLUTION WHO NO PERFORM 1 planning poor recording & Reporting of Provide training on HMIS -ZHDH and APOC Annual., quarterly &Monthly plan (HMIS) 2 Integration Lack of detailed document Re-organization of documents -ZHDH and ZOFP
3 Leader Ship poor staffing at WoHO & ZHO - Employ additional employee Oromia Health Bureau and ZHDH
4 Monitoring Lack of written feed back - Follow up and confirm the provision ZHDH and ZOFP &supervision of required feedback to supervision conducted 5 Mectizan Unable to register Confirm the registration of -ZHDH Batch No.& expiry date Of batch No.& expiry date of Mectizan - Zonal Pharmasist 6 Training & Note based on need assessment and - Provide training on need assessment ZHDH and ZOFP HSAM not integrated. - Confirm that training is organized based on need assessment. 7 Finance Poor liquidation of allocated money at Follow up & timely Liquidation funds ZHDH and ZOFP and district level Zonal accountant 8 Transport Poor handling of Provide training on how to handle Motor cycles Motor cycles ZHDH 9 Human Shortage of manpower Employee additional workers ZHDH and Zonal resource Administrator 10 Coverage -Un-reliable population number Utilization of up-dated census ZOFP 56
C.SWOT Analysis and solution for the weaknesses of West Shoa zone/ Bako Tibe woreda
SE. INDICATO STRENGTH WEAKNESS OPPORTUNITY THREAT SOLUTION WHO NO R PERFORM 1 planning -CDTI program is included -poor recording -Relevant health - Shortage of Training on -ZHDH in annual plan & policy fund and high document -Utilization of pool system Reporting of -presence of turn of staff handling -BPR implementation Annual. quarter FHEW & involved in -Participation of all & CDD in the planning concerned body;- Monthly plan community (ZHO,DHO,HFS,Partner) (HMIS) 2 Integration -Proper utilization of Lack of detailed -presence of Work-over load Re- -ZHDH resources document political High turn over organization Head commitment Low of & Good policy collaboration documents Encouragem ent Strengthenin g Stake holders. 3 Leader - Awareness & commitment -Poor staffing -The availability of - Turn-over of Decreasing -ZHDH Ship of staffs HEWs leader staff turnover of staff
4 Monitoring -Frequent supportive -Lack of -presence of -Work over load - Strong - &supervisi supervision documented supervisory team -un-stability follow up for ZHDH and on -utilization of standardized feed at (uncertainty) the provision ZOFP Checklist Back At some Zone,district level of feed -Conducting annual review HFs & community back meetings supervisors 57
5 Mectizan -Timely distribution after -Unable to -MERK supply the -Mectizan *proper ZHDH and Receiving from MOH register product freely shortage or registration ZOFP -Collecting left over Batch No.& interruption Of batch mectizan expiry date No.& expiry date 6 Training & -Attempt to train health -Note based on -presence of -Attrition of Provide ZHDH and HSAM workers of different level need qualified staff to trained personal training on ZOFP assessment train unamendable need -Un-targeted Budget assessment (non-flexible)
7 Finance -Utilization of government -poor liquidation -substitution of - If APOC Follow up & Expenditure models of allocated resources interrupt timely ZHDH,ZOF -Timely utilization money at district donation Liquidation P Zonal level and woreda Acountants 8 Transport -utilization of available -poor handling of -Good inter sect -Interruption of *Proper ZHDH and vehicles fuel oral donation utilization of ZOFP collaboration fuel
9 Human -Committed staff -Lack of skill -Presence of HEW -High turn *Training ZHDH resource & over Committed CDD 10 Coverage -Good Geographical -Un-reliable -collection of -Interruption of Provide ZHDH coverage (100%& population Mectizan from Mectizan supply training on ZOFP therapeutic coverage number neighboring zone updating -HFS (72.4%) census. 58
D. Woreda Level SWOT Analysis for East Wollega
Indicators Strength Weakness Opportunities Threat 1 Planning - Planning included in annual plan - Not based on the - availability of skilled - Work overload of - Availability of written documents former achievement person at woreda focal person level 2 Integration - Collaboration with other health - Unable to prioritize Inspectoral Not participating programs (malaria, Epl .) this programs collaboration equality 3 Leadership Awareness of woreda leaders at - Less attention to the Formation of task - Work over load woreda level program force at woreda level & Instability of focal persons 4 Monitoring - Using check list - Lack of documented Presence of Work over load and - Presence o schedule feed back supervision team at & Instability of focal supervision woreda level persons 5 Mectizan - Timely requesting of the supply Inaccurate census - Freely supplied Interruption of supply from zonal update multi the supply Mectizan supply - timely distribution to H/F 6 Training and - Training of Health staff annually Training and HSAM Presence of qualified Attention of trained HSAM are not targeted on the staff person specific need. 7 Finance - Utilization of govt. expenditure No allocated budget Substitution of model from government available resource 8 Transport and - Utilization of public transport - unable to undergo - Intersectoral - Interruption of other - Ability of staff waking on foot motorcycle collaboration donation materials traveling on animal back etc maintenance - Hard to reach areas. 9 Human - Committed staff - Less commitment at - Awareness CBOs - Refusal of CDD in resource - Assigned focal person on CDTI woreda level and community distribution of drugs leaders 10 Coverage - Good geographical coverage and Poor census update Using suitable - Untimely therapeutic coverage season for supplementation of distribution drugs 59
E. Woreda Level Solution for the observed weaknesses of East Wollega
S. INDICATOR WEAKNESS SOLUTION RESPONSIBLE PERSON NO Assessing the former - Woreda Administrators 1 Not based on the former Panning achievement based on - Woreda Finance office achievement M and E, Data checklist - Woreda Focal person - Training should be given How CDTI is implemented in regarding integration and how integration is not clearly - District health office focal 2 Integration to make it practical. mentioned in the Annual Health person - Integration should be Plan of the WoHO planned Strict follow up for the Monitoring and Lack of document and feed availability of written report WHDH 3 supervision back for supervision and feed back for conducted WOFP supervision. - Supervise census activities WHDH 4 Mectizan supply Inaccurate census update - Provide training on census Woreda administrator and how to update it. WOFP Training s/d be based on Training not based on need WHDH 5 Training and HSAM need of H/staff by H and E assessment WOFP data reviewing check list WHDH No allocated budget from Discuss on review meeting WOFP,WoHo planning and 7 Finance government about allocating Govt budget budgeting team focal person Wo Administrator 8 Transport and other Unable to undergo motor cycle Follow up the regular and WoHo management team materials maintenance timely maintenance of motor WoFD purchasing and finance cycles team. 9 Human resource Less commitment at woreda - Rewarding and initiating of WoHo management team level H/staff Woreda administrator 60
F.FLHF Level SWOT Analysis for East Wollega CEDI Project
Indicator Strength Weakness Opportunity Threat
Planning - Writing plan for CDIT activity available - One HF had no written - Plane carried out in integrated manner plan - H/F staff working closely with kebele administrator Integration - HF staff do CDTI activities with all other -No written report of - FLHF staff are in charge of health programs in an integrated manner. planning meeting is all the health programs available - High awareness of community Leadership - The FLHF staff are responsible for CDTI - They initiate annual CDTI activity - High awareness of the Communities Supervision - CDTI data transmitted using government and monitoring system - Summary of treatment data availavle on the wall. - HF staff routinely supervise CDTI activities in No summer report of Successful and hard workers Shortage an integrated manner supervision are rewarded of Mectizan Mectizan - Mectizan is distributed using government Mectizan Batch number, procurement system and expiring date not and distribution - CDD Mectizan request received from HF in recorded by HF staff the community based on the data census. Training and - Kebeles support training by certain - training is not targeted CDD’s willing to come for HSAM awareness. HSAM is not targeted training with refreshment and - Community awareness is high through transport. HSAM Transport and - Accomplish their activities traveling on foot Training and HSAM materials other materials CDTI is integrated with other Health activities available in local language 61
supplied by woreda. Human - Health staff are stable and appointed by Health staff are stable and resource community appointed by community Coverage Geographical and therapeutic coverage is high in the past system 62
G.FLHF Level Solution for the observed weaknesses of East Wollega Project
Weakness Activity to address weakness (Solution) Who will do it? When to be done No written plan on - Provide training on data management and WOFP and FLHF staff January 2010 health facility planning
No written report of -Appropriate training and refreshment training WOFP January 2010 planning meeting - Documenting the planning meeting when ever available necessary (by demonstrating how to document) No summary report for - Provide training on report writing and feedback WOFP February 2010 supervision No record of Mectizan - Conducting appropriate training on Mectizan HF health workers. February 2010 batch number, Lot stock management including recording exp date, lot number and expiry no and batch number. date. Need assessment not - Provide training on need assessment. WOFP January 2010 done Training is not targeted - Provide training on need assessment WOFP January 2010 HSAM is not targeted - Provide training on need assessment. FLHWs & WOFP Before January 2010 63
H. Community Level SWOT Analysis for East Wollega CDTI Project
Indicator Strength Weakness Opportunities Threat
Planning - FLHF Workers plan with No regular meeting to Structure of community High turn over of Kebele leaders discuss on planning in a manageable way. CDTI workers. - Conducting census updating and activities Supportive Govt. policy - Request Mectizan based on Human resource (CDD) census Leadership - Committed kebele leaders for No regular feed back Employed kebele Work load CDTI report to H/facilities managers Availability of HEWs Monitoring - Reported Mectizan CDD do not have Close follow up from - Work land supervision - Distributed census updating report writing skills HEW occupied with other before distribution Developing skills from health activities immunization and others Monitoring Community leaders conducted ------monitoring during Meet distribution Mectizang supply - Requesting Mectizan based - Poor census quality Trained CDDs Risk of security on census population - Variation in the Support from HEW - Returning of unused drug to amount of Mectizan Community support FLHF balance Structured community - On time distribution of Ex settlement areas Mectizan Training and - Provided on time training - Routine training not CDDs are committed -Routine training HSAM - Conducted health education need based. with no payment monotonous training Integration of community for CDDs gathering Human resource - CDDs selected by No materials supply CDD living with in the Turn over of CDDs communities from community community -No shortage of CDDs Coverage >65% coverage No mop-up for House-to-house Security problems 64
Distribution is on time absentees distribution Migration of the people
I.Community Level Solution for the observed weaknesses of East Wollega Project
Weakness Activities of solution When to do Responsible person or organization No regular meeting to - Encouraging the community to have September to October or HEW discus on planning regular meeting together with community when necessary Community leaders leaders Focal person - Advocacy meetings No regular feed back Conducting training on how they report January 2010 - FLHF workers report to HF from CDDs especially on reporting format CDDs do not have writing Selecting additional educated CDDS to During Mecizan distribution Community members skill assist in writing May to June - Poor census quality Provide training to CDDs on census and January 2010 FLHF workers - Variation of Mectizam Mectizan Stock management balance No mop-up for absentees - CDDS must revisit the absentees After distribution for a month CDD and community - Conduct supportive supervision during time. leaders. Mectizan distribution time. 65
ANNEX-5 : Key Persons Met/Interviewed in the East Wellega CDTI Evaluation
Zonal LEVEL/ PARTNERS No Name Address Position 1 Mr. Ensermu Jeldu East Wolega Zone Zonal Oncho Coordinator 2 Mr. Deressa Knoo East Wollega Zone Zonal Health Desk Head 3 Mr. Kitessa Debello East Wollega Zone Zonal Pharmacist 4 Mrs. Abebech East Wollega Zone Zonal Finance Desk Accountant Asfaw 5 Mr. Shiferaw Bekele East Wollega Zone Zonal Finance Desk Accountant 6 Mr Kitesa Debelo East Wollega Zone Zonal Pharmacist 7 Deribie Mekonnen Light for the World Country Representative,Ethiopia
Wereda LEVEL No Name Address Position 1 Mr. Endale Geleta Guto Gida Woreda Woreda CDC Team Leader 2 Mr. Busho Kumbi Guto Gida Woreda Woreda Oncho Coordinator 3 Mr. Tilahun Desissa Guto Gida Woreda Woreda Health Office Head 4 Mr Tesfa Tesema Leka Dulecha Woreda Vice Woreda Administrator 5 Mr. Kitessa Mosisa Leka Dulecha Woreda Woreda Health Office Head 6 Mr. Korsa Eba Leka Dulecha Woreda Woreda Oncho Coordinator
HF No Name Address Position 1 Ms.Meskerem Wakjira Kitessa Kebele HEW 2 Ms. Alganesh Degago Loko Kebele HEW 3 Ms.Fasikie Debebe Loko Kebele HEW 4 Ms.Wubalem Bedassa Kewissa Kebele HEW 5 Ms. Askale Taye Kewissa Kebele HEW 6 Ms.Tsehay Taraegn Bedo Kebele HEW 7 Ms.Warkinsh Bedo Kebele HEW Wakeshum
COMMUNITY No Name Address Position 1 Mr, Wolda Deriessa Haro Village CDD 2 Mr Wakjira Fufa Haro village Village leader 3 Mr. Getahum Kitessa Kebele Kebele leader Feyisa 4 Mr. Temesgen Derartu Village CDD 66
Terefe 5 Mr. Kebede Dembel Derartu Village CDD 6 Mr. Taye Tafesse Kambi village Village Leader and CDD 6 Mr. Fikadu Leta Mayibasi Village Village Leader and CDD 7 Mrs. Alganesh Lafto village Village Leader and CDD Alemu 8 Mr. Eba Mideksa Legetsebela Village CDD 9 Mr. Mulatu Etana Legetsebela Village Village Leader and CDD 10 Mr. Merga Geleta Burka Woligelti Village Village Leader 11 Mr. Birhanu Negasa Burka Woligelti Village Village Leader and CDD 12 Mr. Tesfaye Kejela Boa Oda Village CDD 13 Mr. Wagari Mideksa Tume village CDD 14 Mr Alemu Derge Wanebo village Village Leader and CDD 67
Annex 6: Sustainability Evaluation of astt Wollega CDTI Project Evaluation Schedule
Date Activity 16th May 2009 Arrival of External Team Members
17th May 2009 Preliminary discussion on plans by team membersive Matters
18th May 2009 Briefing at the Carter Centre, Team reshuffling & Departure to the project sites Arrival at Nekemt 19th May 2009 Meeting with the scout & sub-division of team Briefing of East Wollega zone on the Evaluation 20th May 2009 Zonal level interviews and documents review 21st May 2009 Guto Gida Woreda interviews and document review. Visit to Kitiesa Health post and its selected communities. Visit to Loko Community and its selected communities.
22nd May 2009 Leka Dulecha Woreda interviews and document review. Kewissa and Bedo Health post interviews ans document review 23rd May 2009 Visit to selected communities in Leka Dulecha 24th May 2009 Team meeting on data collation, entry and analysis 25th May 2009 Team meeting on data collation, entry and analysis continues. Report writing.
26th May 2009 Review of plans and presentations for the feedback/sustainability plan development workshop 27th May 2009 Zonal Feedback meeting/Commencement of zonal sustainability planning meeting
28th May 2009 Woreda feedback meeting/development of sustainability plans. 29th May 2009 Sustainability plan development continues 30th May 2009 Finalization of draft sustainability plans/submission. Report writing 31st May 2009 Report Writing continues
1st June 2009 Departure to Addis Ababa 68
2nd June 2009 Review of draft Report. NOTF Debriefing
3rd June 2009 Departure of External Evaluators