World Health Organisation African Programme for Onchocerciasis Control
Mid-term assessment of the sustainability of the Malawi 'Extension' CDTI project:
Blantyr€, Chikwawa, Chiradzulu, Mulanje and Phalombe districts
November 2002
Sebastian Baine ,--*--..F.*n'e*&qi ! Samuel Mugisha il Spy Munthali 2 [ l!lit i:iri;, .,
Detlef Prozesky .t rI ., *....",""..'"=l Index
Page
Abbreviations/ acronyms and acknowledgements J
Executive summary 4
Introduction and methodology 6 1. Introduction 6 2. Methodology 9
Findin gs and recommendations 13 1. National level 13 2. District level 20 3. Health centre catchment area level 26 4. Village level 30 5. Overall self-sustainability grading for the project JJ 6. A new insight 34
Advocacy activities and feedback/ planning workshops 35 l. Advocacy activities 35 2. Feedback/planning workshops 36
Appendix 1 Report on meeting with WR 37 Appendix 2 The district level feedback/ planning workshop 38 Appendix 3 District sustainability plans 40 Appendix 4 Suggested changes to the 'Sustainability' instrument 42 Appendix 5 Timetable of the evaluation visit 44
.) Abbreviations/ acronyms
APOC African Programme for Onchocerciasis Control CDA community development assistant CDD community directed distributor (of ivermectin) CDTI community directed treatment with ivermectin CHAM Christian Hospital Association of Malawi CHSU Community Health Sciences Unit DC district commissioner DEHO distnct environmental health officer DHO district health officer DIP di strict implementation plan DOC district onchocerciasis coordinator EHT environmental health team FLHF first line health facility HC health centre HMIS health management information system HQ headquarters HSA health surveillance assistant HSAM health education/ sensitisation/ advocacy/ motivation IEF International Eye Foundation K Malawian kwacha MoHP Ministry of Health and Population NGDO non-governmental development organisation NOC national onchocerciasis ooordinator NOCP National Onchocerciasis Control Programme NOCT national onchocerciasis control team NOTF National Onchocerciasis Task Force REA rapid epidemiological assessment REMO rapid epidemiological mapping of onchocerciasis TA Traditional Authority wHo World Health Organisation
Acknowledgements
We would like to thank the following persons for their help: ' The staff at the Headquarters of the African Programme for Onchocerciasis Control (APOC) in Ouagadougou: Dr S6k6t6li, Dr Amazigo, Mr Aholou. . Staff of NOCP/ MoHP in Blantyre, for undertaking all the affangements: Mr Tambala (NOC), Mr Sitima (Deputy NOC), Mr Mkhoma (training and monitoring supervisor), Mr Kwizombe (accountant), Ms Jere (secretary), Messrs Kamwana and Kaliati (drivers) ' Political and traditional leaders, health workers and community members in the Blantyre, Chikwawa, Chiradzulu, Mulanje and Phalombe districts.
3 Executive summary
The African Programme for Onchocerciasis Control (APOC) has been supporting the Malawi 'Extension' CDTI (community directed treatment with ivermectin) project for the past three years. At a meeting of national onchocerciasis task forces (NOTFs) in Abuja in June 2002,it was decided that the sustainability of the projects which APOC supports should henceforth also be assessed midway through the funding period; the present evaluation is the first such evaluation. It was carried by a team of four evaluators - two from Uganda and one each from Malawi and South Africa. The evaluation followed on an evaluation in May 2OO2 of the first phase of the Malawi project (covenng the Thyolo and Mwanza districts), which found that there were considerable problems with sustainability after five years of funding.
This second evaluation was carried out over a period of twelve days. Information was collected by document study, interview and observation, at sampled sites at four levels of the health service: national, district, first line health facility (FLHF) and village. The newly revised set of 'sustainability' instruments was used and found to be suitable for the Year 3 evaluation (although a few refinements are suggested).
The following are the principal findings of the evaluation: . Coverage: The rate of implementation of the project has been slower than was originally planned. Mectizan has not yet been distributed in Chikwawa district and part of Blantyre district: a few CDDs (community directed distnbutors) remain to be trained, and many others await the Mectizan which finally arrived during the evaluation. Training has yet to commence in Chiradzulu district (the NOTF first wishes to clarify the endemicity situation there). Therapeutic coverage is lower than expected: several possible reasons for this state of affairs were uncovered, but more detailed research is needed to clarify them. . Planning: Planning at national level is thorough, but the National Onchocerciasis Control Programme (NOCP) is not yet included in the yearly cycle of planning, with equivalent programmes, at the level of the Community Health Sciences Unit (CHSU) in Lilongwe. Similarly CDTI was only included in the district implementation plan (DIP) in one of the five districts. There is a strong and successful tradition of integrated yearly and monthly planning at the FLHF level, in which CDTI is present to varying degrees. The national onchocercasis co-ordinator (NOC) has clear ideas about obtaining resources for CDTI after APOC support is withdrawn, but these have not yet been written down in a systematic plan. . Supervision and monitoring: Data from the village level move effectively to the national level, but in some cases the district health management team (DHMT) does not get to see it. There is an excellent system of review meetings after each distribution. Supervision as it is being conducted at present is not as efficient as it could be (it is being done routinely rather than being targeted, and a team of two is not always needed), nor as effective (NOCP team members are not really aware of the reasons for the low coverage rates, and although there are checklists available supervising staff repon not using them). There is almost certainly a problem with the way in which rates are being calculated at present, which needs to be rectified. There are at present no indicators for CDTI in the national health management information system (HMIS), but the relevant manager at Headquarters (HQ) level at the Ministry of Health and Population (Mo[IP) in Lilongwe would welcome suggestions. I and HSAM rh motivati Training targets are being exceeded in terms of numbers - this ts due to an under-estimation of the numbers of separate villages (and therefore CDDs) in the original project submission. In
-1 general training is conducted efficiently, but there are no specific courses available for staff at the national level. Excellent HSAM materials are available but are in short supply, largely due to their expense. Much advocacy work has been done. but some traditional authorities (TAs) and district commissioners (DCs) have not yet been approached. Mectizan supply: This is working well, and takes place entirely within the government system. Integration of support activities: These are highly integrated at the district and FLHF levels - this is one of the great strengths of the Malawi programme. Available transport and other resources are routinely shared between programmes, and workers perceive themselves to be a team of multi-skilled professionals. Finances/ fundingl transport and equipment: The government's contribution is at present limited almost entirely to salaries - everything else is funded by APOC (transport, equipment, stationery, allowances, HSAM matenal). There is a small budget for onchocerciasis control in the WHO country representative's (WR's) biennial budget, but the methods and extent of government funding post-APOC remain to be spelt out. There is no doubt though that some funding will be available - MoHP has created posts for the programme, and in the local system this implies a commitment to providing the wherewithal for the programme to function. The use of transport is in general well controlled. Human resources: One of the great strengths of the Malawi programme is the fact that there are MoFIP employees at village level - the health surveillance assistants (HSAs) - charged with implementing approved preventive and promotive programmes. These persons were found to be knowledgeable, reasonably committed and stable. They are supported by specific staff at FLHF and district level. In some instances however too few CDDs are being trained.
The overall judgement of the team is that the Malawi 'Extension' CDTI project is not far from being sustainable. The health system in Malawi is functioning at a level which is strong enough to carry CDTI; it is highly integrated; and it reaches effectively right to village level. There is still time to achieve maximal integration of the project into the MoHP system at all levels, and there is a reasonable assurance that MoHP will be able to supply the resources the programme needs to function when APOC withdraws.
Detailed recommendations were drawn up, based on the findings of the evaluation. The recommendations were prioritised, and indicators and deadlines were suggested for each. The most important recommendations concern : . Entering the Government planning and HMIS systems more fully, and drawing up a detailed 'Sustainability plan' based on current ideas. . Bringing Chiradzulu district into the project immediately, and completing the necessary training in Blantyre district. . Researching the reasons for low therapeutic coverage, and correcting mistakes in calculating coverage rates. . Planning supervisory visits more efficiently.
Advocacy activities were carried out, mostly at the district and TA levels. Two feedback/ planning workshops were held - one for the national onchocerciasis control team (NOCT) and one for the DHMTs of the five districts where the 'Extension' project is operating. In each case the evaluation team gave feedback on its findings, which were discussed in depth. In the case of the district level workshop the evaluators then guided the participants to draw up reahstic 'Plans for self-sustainability' for their areas of operation.
5 Introduction and methodology
1. Introduction
1.1 Onchocerciasis control in Malawi
It is estimated that 750 000 persons in Malawi are infected with the parasite Onchocerca volvulus, and altogether 1,5 million persons are at risk of contracting the disease. Efforts to control onchocerciasis in the country had started well before APOC began to offer its support. Between 1984 and 1987 a mission hospital determined the prevalence of the scourge in the Mwanza and Thyolo districts, by means of skin snip surveys. This was followed by the distribution of ivermectin on a trial basis. The International Eye Foundation (IEF), a non- governmental development organisation (NGDO) based in the United States, joined this programme in 1991, and mass distribution of Mectizan to affected communities began in earnest. The activity programme was however limited to two endemic districts, Thyolo and Mwanza. With the advent of APOC and its strategy of CDTI the decision was made to focus all control efforts around the new strategy, and to expand the programme in due course to cover all hyper-endemic and meso-endemic communities in the seven most affected districts in the southern part of the country. These areas were determined after a rapid epidemiological survey for onchocerciasis (REMO) has been conducted for the whole country, in 1997.
In the Malawi CDTI programme the partners were to be MoHP, IEF and the Christian Hospitals Association of Malawi (CHAM). The first CDTI project approved by APOC for funding covered the districts of Thyolo and Mwanza, and funding began in 1997. The sustainability of this initial project was evaluated in May 2002, after five years of APOC funding. The overall finding was that the project was 'potentially sustainable, but will require re-thinking and mobilisation of high-level support to get it on the road to sustainability'. The report cited the following obstacles to sustainability: . CDTI did not appear in the national health plan, and there were no budgetary allocations from MoFIP for the programme. Key directors in MoHP were unaware of the CDTI programme, and there were no explicit plans to assure the sustainability of the programme after the withdrawal of APOC funds. . At distnct level funds were also not being budgeted for the CDTI programme, which was consequently dependent on outside funding for its existence. . At FLHF level key managers had insufficient knowledge of CDTI activities; transpoft was inadequate; CDTI activities were not budgeted for; and HSAs (the actual trainers of CDDs) were themselves inadequately trained. . At village level both political and traditional leadership were not part of the mainstream CDTI programme; villagers were not organised to take final responsibility for the programme; and some CDDs were unable to perform key tasks. It should be noted that the present evaluation team disagrees with some of these findings, considering them not to be valid in the light of the nature of the health service in Malawi.
As a country Malawi is at present facing a number of serious national problems. The HIV/ AIDS problem is very great, and is having a profound effect on society. A series of poor harvests and mismanagement of the national grain reserve has led to an acute food shortage, with widespread repofts of starvation. In addition a number of donor countries are
6 withholding substantial amounts of overseas development aid - and such funds normally make up to one third of the national budget. All of these factors may affect the NOCP.
1.2 The present evaluation
At a meeting of representatives of the NOTFs of APOC member countries in Abuja in June 2002, it was decided that it was not enough to evaluate the sustainability of projects funded by APOC after five years of funding. Since this gave them very little time to undertake remedial action, an additional evaluation after only three years was decided upon. The present evaluation in Malawi is the first such evaluation. It also follows on a revision of the instrument used to evaluate project sustainability, and therefore presents an opportunity to test the instrument and assess its suitability for such a mid-term evaluation of sustainability.
1.3 Background to the Malawi 6Extension' project
The scientific basis for the project
Only the 1997 REMO report was available, to plan the 'Extension' areas that need treatment This provided the following information, in relation to the districts to be covered:
Nodule prevalence No. of communities 2407o 20-39Vo l0-l9Vo District <107o examined 'hyper' tmeso' 'hypo' Blantyre 10 0 I 4 J Chiradzulu 4 0 I 2 I Mulanje/ Phalombe l5 0 7 2 I Chikwawa t4 0 0 5 I
Note that no hyper-endemic communities were found in these districts; the sample for Chiradzulu was extremely small; and only hypo-endemic communities were found in Chikwawa. Not surprisingly the 1999 project application planned to conduct a rapid epidemiological assessment (REA) to complement this sparse information.
The project application: targets
The 1999 application to APOC for the 'Extension' set the following targets
(a) For treatment: Communities to be under treatment by: No. of District Endemicity 1999 2000 2001 2002 2003 communities Mulanje Hyper 98 60 98 98 98 98 Meso r33 80 133 133 t33 Phalombe Hyper 54 54 54 54 54 54 Meso 80 40 80 80 80 Blantyre Meso 63 JJ 63 63 63 Chikwawa Meso JJ 33 35 JJ JJ Chiradzulu Meso 90 45 90 90 90
1 Note that this table assumes that hyper-endemic communities exist. The REMO however only found meso- and hypo-endemic communities (see first table in this section).
(b) For CDD trairtirtg: CDDs to be trained per year: Total no. of CDDs District 1999 2000 2001 2002 2003 to be trained Mulanje/ Phalombe 399 553 326 t218 Blantyre 116 105 221 Chikwawa 116 1r6 Chiradzulu r58 158
Progress in achievinq the tarqets
Training actually proceeded as follows
Year Total Overall District 2000 2001 2002 achieved target xMulanje/ Villages covered 67 t7l 57 295 231 Phalombe CDDs trained 311 593 261 t277 t278 Villages covered 113 113 63 Blantyre CDDs trained 565 565 22r Villages covered 10 10 JJ Chikwawa CDDs trained 263 263 116 Villages covered 90 Chiradzulu No training has yet taken place CDDs trained 158
Targets therefore appear to have been met or exceeded. However progress in implementing CDTI in the five districts of the 'Extension' has been considerably slower than originally planned. Actual distribution of Mectizan has only been taking place in Mulanje (all villages indicatd by REMO), Phalombe (all villages) and Blantyre (about half of the villages). In Chikwawa CDDs were only recently trained for all villages, but no Mectizan has yet been distributed; and in Charadzulu no training has taken place at all, at any level. The reasons for apparently exceeding the targets, and for the delay in implementation, are discussed in the body of the report.
A brief word needs to be said about the control programme in the tea estates in Mulanje. These farms are independent entities, which cooperate with the district health service. They are however able to supplement what the government is able to provide. In this situation NOCP trains HSAs (who are employed by the estates) in CDTI; and these HSAs, assisted by volunteers, conduct the Mectizan distribution.
8 2. Methodology
2.1 Sampling
Four of the five districts could be examined down to village level, but in the case of Chiradzulu only the district level would be examined. In the other four drstricts traditional authonty (TA) areas were sampled as follows: . Blantyre: two TA areas with their two health centres (HCs) (the only ones included in the programme). . Mulanje: two TA areas with their two HCs (the only ones included in the programme). . Phalombe: one TA area, with its two HCs (the only ones included in the programme). One of the HCs is run by MoHP and the other by CHAM. . Chikwawa: two HCs out of four were randomly selected (no coverage data available yet). These two centres serve three TA areas.
In view of the fact that the evaluation team of four persons had to cover five districts, the decision was taken to sample only two villages per FLHF catchment area. In the ZYz districts where coverage rates were available, the villages were selected according to the criterion of 'therapeutic coverage' - selecting one village with good and one with poor coverage. In the other lVz districts the criterion of geographical spread/ distance from the HC was used to select the two villages. This resulted in the following sample:
District Traditional authority HC/ FLHF Community *Chiwembe (near FLHF) Kapeni *Limbe +Jumbe (far from Blantyre FLHF) Chunga (good coverage) Somba Mpemba Mpingo (poor coverage) *John (far, hill country) Maseya/ Katunga xMaperera *Muyaya (near, hill country) Chikwawa xGangu (near, plain) Makhuwira *Makhuwira +Singano (far, plain) Chiradzulu Mbyelioma (high coverage) Mabuka Mimosa Katute (low Mulanje coverage) Monda (high coverage) Njema Muloza Mkhumba (low coverage) Nanyalo (high coverage) Nkhulambe (MoHP) Likhura (low coverage) Phalombe Nazombe Naliya (high coverage) Sukasanje (CHAM) Siyankhuni (low coverage) 5 districts 7 TAs 8 HCs L8 communities * No treatments given yet
9 2.2 Protocol
Research questiort'. How self-sustainable is the Malawi 'Extension' CDTI project? ! Desi grt: Cross-sectional, descriptive. Populatiort: The Malawi 'Extension' project, including:the national team in Blantyre; its NGDO partner (IEF); its five district, with all staff involved in onchocerciasis control in them; the project villages, with their leaders and CDDs. Instrument: * A record sheet, structured as a series of indicators of self-sustainability. The indicators are grouped into nine categories/ groups. These groups represent cntical areas of functioning of the programme. * The instrument assesses sustainability at four levels of operation. * The instrument guides the researcher to collect relevant information about each indicator, from a variety ofrelevant sources. Source of infurmation: * Verbal reports from persons interviewed. * Documentary evidence and observations. Analysis: * Data from all sources is aggregated, according to level and indicator. {. A qualitative summary of the situation regarding each indicator at each level is made. This is aggregated and summarised for each category of indicator, for each level. * Based on the information collected, each indicator is graded on a scale of 0-4, in terms of its contribution to sustainability. x The average 'self-sustainability score' for each group of indicators is calculated, for each level. Recommendations'. * These are strictly based on the findings of each area of research.
2.3 Team composition
The core team members were the following
1. Sebastian Baine x Institute of Public Health, Makerere University, PO Box 7072, Kampala, Uganda * T: *25671925 861;F: *25641531807 * sobaine@ c o.za or [email protected] 2. Samuel Mugisha * PO Box 33613, Kampala, Uganda * T: * 25677 582 505 * [email protected] or [email protected] 3. Spy Munthali * Economics Department, I]niversity of Malawi, Box 280, Zomba, Malawi x T: * 265 8 829 599 * [email protected] or [email protected] 4. Detlef Prozesky (team leader) * Faculty of Health Sciences, University of Pretoria, PO Box 667 ,Pretoia 0001, South Africa * T: * 27 12354 ll47;F: + 27 12354 l'758 * [email protected] or [email protected]
t0 Each team member became the head of a small sub-team, which was made up of one core team member and two community development assistants (CDAs) drawn from the four districts where field work was to be done. These persons were used to collect village level data, and were trained on the job. Each sub-team was also accompanied by a member of NOCT, to act as guide and facilitator. On the first two days of the exercise the core team members met to familiarise themselves with the revised instrument, and to complete planning of the evaluation process.
2.4 Advocacy visits and 'Feedback/ planning' meetings
Advocacy visits were to be paid to relevant persons at each level, as many as possible, and officials were to be debriefed at the end of the field visits - again if possible.
Finally, meetings were to be conducted for relevant officials at the national and district levels During these planning meetings the evaluation team would give feedback on its findings, and the national and district teams would be asked to develop sustainability plans, based on the findings.
2.5 Limitations
The evaluation team was not sufficiently aware of the unique nature of the tea estates, as a separate entity with a different modus operandi. They were therefore conceptually lumped together with Mulanje district, and never actually visited. Future evaluations should see them as an additional district, to be sampled and evaluated separately.
2.6 The performance of the revised 'sustainability' instrument
The revised instrument proved easy to use, and relatively unambiguous. The following should however be noted: . The instrument is designed for use in projects in 19 countries, which differ considerably in the way in which their health services are structured. Some of the indicators and sub- indicators may therefore appear irrelevant for a particular situation (for example, in Malawi financial planning at the FLHF level doesn't take place). Evaluators should be aware of this, and the core team needs to include comments about this issue in the 'Guidelines'. . There was a tendency to focus more on the present situation in each indicator, with a relative neglect of enquiry into the reasons for it. This is not a defect of the instrument, but rather of the way in which it was used. . There were a few indicators where the wording was still found to be unclear. Suggested changes are given in Appendix 4. . There were a few cases where the wording was not entirely appropnate use of the 'third year' evaluation. Suggested changes are also given in Appendix 4. . The instrument as it is structured at present leaves quite a lot of room for data to be written down on it: * This makes it possible to use one copy to record data from several sources - e.g. from a few interviews, from documentation etc. * Even if the allocated space is full, the blank page to the left can be used for additional data recording - and this space could even be formatted for this purpose:
ll It you need more space lor relaled lo 'nlormatron lhe rndrcaior on the taqng page, wnte rl here Make sure you rndrcale lhe source ol rnlormalron clearly
Having all the information collected in one place makes the analysis easier, since there are not so many different pieces of paper to work from.
t2 Findings and recommendations
1. National level
1.1 Overall grading (on a scale of 0-4)
Malawi 'extension' project: sustainability at national level
4
3.5
! 2.5 .9 q) 2 =o) o) G 't.5 o o 1 0.5 0 ^."q a\o' ..tt" ".".." d*"ry a.""" ood ".."t"* "-". a"e group of indicators 1.2 Main findings and recommendations 1.2.1 Planning Yearly planning is thorough and detailed, and is compiled as paft of the yearly application to APOC for renewed funding. The plan varies from year to year, since the programme is still expanding. All NOTF members are given the opportunity to take part in drawing up the plan (which not all do). Roles of the different partners are clearly spelt out in the plan. The area of HSAM is relatively under-emphasised though. There is a yearly planning exercise at the level of CHSU at MoHP HQ in Lilongwe. The sections which make up the Unit are all supposed to take part in this exercise, but to date NOCP has not been involved. It does however forward its annual plan to CHSU. The NOCP leadership has clear ideas about moving towards sustainability when APOC funding ends. These have however not yet been written down, nor have they started being implemented. l3 ! Reasons for not being part of the HQ planning process, and not writirtg down the 'sustainabili4, pktn' : * For years the programme has been officially sited within Thyolo district, for the purposes of administration. There is no tradition of planning together with the other CHSU programmes. * The programme is physically located far from HQ in Lilongwe, which makes its inclusion into routine meetings there more difficult. * The whole question of 'sustainability planning' is entirely new. Recommendations :'Planning' Implementation I. An explicit plan for onchocerciasis control/ Priority: CDTI should be integrated into the yearly 1,2,3.. HIGH National Health Plan, together with those for Indicators of success'. the other programmes falling under CHSU. 1. 'Onchocerciasis control' appears in the 2. A written plan to achieve a state of financial next CHSU year plan. sustainability needs to prepared. This plan 2. The written sustainability plan exists. should include specific steps to: 3. The next year plan (the one submitted to * Increase likely income and resource APOC) includes more specific, targeted allocation for the programme. HSAM activities. * Contain costs, to a level which equals Who to take action'. that of expected income (see also 1.2.6 1. NOC, NOTF, Director CHSU below). 2. NOC and NOTF. 3. The detailed year plan should contain 3. NOC and NOTF. specific targeted HSAM activities, to remedy D e adline fo r c ompl eti on'. shortages noted at present. This includes 1. February 2003 (or when CHSU planning plans for printing HSAM flipcharts in larger takes place). numbers, for distribution to the different 2. June 2003. levels of the programme. 3. As soon as possible. t.2.2 sion and monitorin Monitonng data is generally received in time from the districts, and is collected into district reports. Data related to onchocerciasis control is however not included in the national HMIS. The director of CHSU is of the opinion that this should be done. Staff at this level have developed routines for responding effectively to problems shown up by supervision and monitoring. The review meetings held after each distribution effectively identify problems, which are immediately handed to those present to solve. The written reports sent to districts after each distribution clearly present problems identified by the data, for districts to attend to. Staff in the periphery report that NOCP staff appreciates their efforts - a positive reinforcement for them. I In general the supervision conducted by staff from this level is targeted at the district level. There are indications however that supervision takes place more frequently than is operationally necessary, that visits last too long and that supervising teams are too large. In addition, supervision frequently continues up to the HC level. The visits are therefore not as efficient as they might be. I There is a supervisory checklist available, which NOCT members can use when they conduct supervision at district level. It has also been passed on the district lcvel trainers to use at sub-district level - but ut all levels staff report rnt usirtg tlrcnt. -t There are at present no indicators of CDTI in the national HMIS. The CHSU director is however willing to consider their inclusion. l4 I Reasons for 'over-supervising' and not preparing checklists: * It is difficult to move from a situation at the beginning of a programme, where supervision has to be more intense, to one where lower level staff members are helped to take on that responsibility, in a more mature situation. * Since MoFIP salaries are very low, field allowances form an important salary supplement. * NOCP staff members are highly expenenced, and may not feel a need for checklists. Recommendations:'Supervision and monitoring' Implementation l. Supervision checklists should be prepared for the Priority: following situations: I,2,3.. HIGH * NOCP staff supervising districts. 4: MEDIUM x District environmental health team (EHT) staff Indicators of succe ss'. supervising HSAs. 1. The checklists exist and are used. The relevant staff should then be briefly trained in 2. The 'Sustainability plan' indicates the use of these checklists during supervision. steps that will make supervision 2. Supervision by NOCP needs to be carefully more efficient. Annual reports of planned and controlled: the number of visits distributions show that these steps undertaken; the number of staff involved per visit; have been implemented. the duration of the visits; and the level at which 3. Reports of training in data supervision is conducted. This is in view of the management exist. need to empower staff at district and sub-district 4. A CDTI indicator is included in the level to do their own supervision. and also the national HMIS. need for cost containment. Wto to take actiort'. 3. Additional training in data management should be 1,2,3.. NOC, NOCP offered to all those involved in programme 4. NOC, NOTF. implementation, to enable them to analyse census Deadline for contpletion'. and distribution data correctly. 1. By the next distribution. 4. NOTF should study the present national HMIS; 2. With immediate effect. decide on an indicator that will help CHSU to 3. At the next round of review monitor CDTI nationally (probably'Therapeutic meetings. coverage'); and make a submission to Director 4. June 2003. CHSU to include it in the HMIS. 1.2.3 Mectizan supply I Mectizan supply is managed entirely by MoFIP employees, using a unique system. The system is simple and effective and supplies sufficient quantities for the districts' needs, usually at the right time. 1.2.4 Training and HSAM NOCP staff train members of the district EHT, and supervise these members of the EHT when they in turn train HSAs. There are however a few reports of NOCP staff actively taking part as tutors in HC level training, in such a way that district level trainers take an inappropriate back seat. Training is properly targeted, but there are indications that the number of trainers is at times more than is operationally necessary. I NOCP has in the past planned and carried out necessary and effective HSAM activities at district level, but is now delegating those to district staff. l5 Although HQ level staff at Lilongwe have a high level of awareness of the programme, they do not as yet see the need to involve Programme leadership in routine planning and budgeting exercises. -I Reasons wlry training is rtot as efficient as it might be, and whv Lilongwe HQ staff is not involving NOCP as much as other sections: * These are discussed under 'Planning' and 'Supervision and monitoring' above. Recommendations:'Training and HSAM' Implementation 1. The number of NOCP trainers taking Priority: part in training needs to be carefully 1. HIGH controlled, as well as the level at 2. MEDII.]M which they conduct training. This is in Indi cators of succes s'. view of the need to empower staff at . The 'Sustainability plan' and the annual plans district and sub-district level to reflect the need for efficient, targeted training. undertake training themselves, and . NOTF has made a decision about having an also the need for cost containment. additional office in Lilongwe. 2. In view of the relatively low profile Who to take action'. that 'Onchocerciasis control' has at . NOC, NOCP CHSU in Lilongwe, NOTF should ' NOTF. seriously debate the advantages and Deadline for compl etion'. disadvantages of the programme ' June 2003 - or before the next training round. having an office at CHSU (in addition . The next NOTF meeting - early in2003. to the one in Blantyre). 1.2.5 Integration of support activities There are numerous examples of integration of support activities: the review meetings. combining supervisionl Mectizan supply, supervision/ HSAM etc. I.2.6 Finances and funding The costing of CDTI activities is routinely done in detail. The NOC has a good perception of funds that will be available to the programme the following year, and budgets within that. There is however no sign yet of deliberate cost containment, in preparation for the leaner years ahead. The relative budgetary contributions of the partners are clearly spelt out in the budget. MoFIP funds salaries, but has not yet begun to provide funding for other NOCP activities: APOC still funds these entirely. Plans to bridge this shortfall eventually are still embryonic, and no commitment in this direction has yet been obtained. I Although other national programmes have budget lines within CHSU, this is not yet the case for NOCP. Funds available to the programme are carefully and efficiently managed. Reasons why MoHP is not contributing funding; wlry cost containmert has yet to begin: * MoHP probably perceives that, since the programme is at present adequately funded by APOC, it does not need State funding. x The 'distance factor' (from Lilongwe) also operates here. * Since the programme is still in its growth phase, it is difficult to start thinking of ways of diminishing expenditure for specific activities. 16 Recommendations:'Finances and funding' Implementation 1. The 'Sustainability plan' mentioned in 1.2.1 Priority: above should indicate clearly how costs are 1. HIGH going to be contained, in such a way that 2. HIGH sustainable routines are established before Indicators of success'. APOC funding comes to an end. l. The 'Sustainability plan' indicates 2. NOCP should request to be present at CHSU's how costs are to be contained. annual planning and budgeting meeting: 2. NOC has attended the 2003 CHSU * During this exercise NOCP should make a planning meeting, and NOCP appears submission, reflecting the amount of in the CHSU budget for 2003-2004. counterpart funding agreed to by MoHP at Who to take action: the time of the original project agreement. 1. NOC. x The Onchocerciasis Control Programme is 2. NOC and Director CHSU to be allocated a line in the CHSU budget Deadline for compl etion submission, similar to that of the other 1. June 2003. CHSU programmes. 2. January-May 2003. t.2.7 and material Sufficient transport is available but it is dependent on APOC for its running costs (i.e. not sustainable in the long run). Since the vehicles should seldom be required to go beyond the district level (and in fact seldom do), they may be heavier and mechanically more complex than is required - and will consequently be more expensive to run. I Office equipment is sufficient and in good working order. Logbooks are scrupulously kept for all vehicles. All trips have to be authorized, but the entries are not regularly checked. Vehicle maintenance is recorded in the logbooks, but is not strictly planned according to a schedule. Office equipment is repaired rather than maintained. At present APOC bears the cost of maintenance and repair of vehicles and equipment entirely (again not sustainable in the long run). NOC has a clear plan for the replacement of vehicles and equipment, when this becomes necessary. There are procedures to be followed within MoHP, with which he is fully familiar, and which in his experience have proved effective. Reasons wlty MoHP is not contributing to running/ maintaining vehicles and equipment: * These are discussed under'Finances and funding' above. Recommendations: Implementation 'Transport and material resources' 1. The issue of achieving Priority: sustai nable transport and I. HIGH equipment for all levels of the 2. MEDIUM programme needs to be fully Indicators of success: addressed in the 'Sustainability I. These items are included in the 'Sustainability plan' plan' described in 1.2.1 above. 2. The task of reconciling logbooks monthly is written 2. A senior NOCP staff member in the relevant officer's lob description. should reconcile logbooks Who to take action: monthly. 1. NOC and NOTF 2. NOC. Deadline for compl etiort l. June 2003. 2. Immediately. 11 1.2.8 Human resources The number of staff members at this level is appropnate, and they have appropnate knowledge and skills. They are highly stable and appear committed and hard working. In the nature of the APOC/ CDTI system they have had no formal training for the work in which they are engaged. They also report needing more general management and data management skills. Recommendations:'Human resources' Implementation L Selected members of NOCP should undergo formal training Priority: in the following areas: l. Low x Data management. This course needs to be carefully 2. IVIEDILTM selected, to include both common epidemiological indices Indicators of suc ce s s'. and their mathematical aspects, and appropriate software 1. Staff members have (e.g. Epilnfo). successfully completed * Training methodology. Again a carefully selected, very the short courses. practical course, dealing with improved lectures; 2. The advanced course preparing and using visual aids; conducting practicals; and exists. assessment of learning. Who to take action'. * General management: A very practical course, aimed at l. NOC (to find courses), mid-level managers. APOC (funding). 2. An advanced course in 'CDTI management' or 'Managing 2. APOC. onchocerciasis control' needs to be developed by a suitable D e adl ine fo r c ompl etion institute of higher learning, to enable existing and especially 1. June 2003. new managers of CDTI programmes to learn the ropes. 2. End of 2003. 1.2.9 Coverage I Geographical coverage : x In the districts of Mulanje, Phalombe and Chikwawa all communities have now been covered (although in some villages in Chikwawa treatment has still to follow on the recently completed training). About ZOVo of the target area in Blantyre remains to be covered. * Project implementation has been behind schedule. Training for all five districts was supposed to have been completed by the end of 2001. * The programme has not yet been launched in Chiradzulu district - this after three years of funding of the 'Extension'. t Reasons for the delay in intplementation: x The slow pace of completing training may be due to: - The fact that NOCP has to work through the districts and their EHTs, and is unable to force their pace. - NOCP's ntodus operandi, which has been to let districts start training in one area, and to make sure that that works before going on to other areas. * The reasons for not tackling Chiradzulu appear to be: - NOTF has always been unceftain about the intensity and scope of the infection in Chiradzulu, for a variety of reasons. Only four district villages were sampled during the REMO. The actual boundary (between areas to be treated/ not to be treated) was never finally demarcated following the REMO. The known spread of l8 the disease in neighbouring districts, combined with certain local geographical features, cast doubt on the way the demarcation line started being drawn. - It appears to NOTF that there is considerable danger that many hypo-endemic villages might be treated, unless the present situation is first clarified by a REA. The project apparently erred previously in Mwanza distnct, by starting treatment in some hypo-endemic villages. - The reply to an application for funding for the REA was late in coming from APOC. The application was turned down. - Pending clarification of the situation by a REA (for which funds have yet to be raised) NOTF decided to concentrate on the four other districts, which appeared to present a much higher treatment priority. t Therapeutic coverage: the village level rates vary considerably, but tend to be lower than in comparable projects elsewhere. Only Zl%o of villages achieve rates of 657o or more. In the Mulanje tea estates the rate of coverage is higher than in the districts - up to 81.87o in 2001. This may well be because distribution is carried out by HSAs employed by the estates. Possible reasons for the low therapeutic coverage: * Seasonal movement of people. * Fear and uncertainty: the measuring stick as a sign of death, fear of 'family planning'. * Religious objections to taking medicines. * Some members of the EHT not working in a systematic way, or not being committed to the programme. x Incorrect calculation of coverage rates by HSAs. * Relative unconcern since the disease is not intense ... and so on. Recommendations:'Coverage' Implementation 1. A rapid epidemiological assessment Priority: exercise must be carried out, to clarify T,2,3,. HIGH the endemicity situation in Chiradzulu Indicators of success: district. Funds need to be raised for this 1. The REA report for Chiradzulu is ready. exercise, as soon as possible. 2. The research report on low therapeutic 2. Research must be conducted into the coverage in Phalombe and Mulanje is ready reasons for low therapeutic coverage in 3. All villages in the endemic area in Blantyre the villages of Mulanje and Phalombe. are fully enrolled in the programme. This could be coupled with a Wlto to take action comparison between these villages, and 1. NOC, NOCP. the villages in Blantyre and the Mulanje 2. NOC, APOC. tea estates, where the coverage is higher 3. NOC, NOCP. The remaining 3. villages in the endemic D e adline for c ompl et ion: area in Blantyre district must be fully I. 6 months after funding has been obtained. enrolled in the programme. 2. April2003. 3. June 2003. t9 2. District level 2.1 Overall grading (on a scale of 0-4) Malawi'extension' project: sustainability at district level 4 3.5 J .C 2.5 cD o 2 o= E') G 1 o (E 0.5 0 .*"'"" $..'c ,€$ ..C .oa"'u" C ".'"* "... ".,'"" a""" ".,"t" group of indicators 2.2 Main findings and recommendations 2.2.I Planning Although all districts made fairly detailed year plans for CDTI (usually in the form of a Gantt chart) such plans were only included in the DIP in one case. In some cases several members of the health team participated in drawing up plans. Such planning is not however taught in the training NOCP provides. Reasons for failing to integrate the CDTI plan into the overall year plan are: * The programme is still new. * The DHMT sees it as externally funded, and it therefore does not need funding from the Ministry of Health and Population (MoFIP). * The capacity of the district to fund new programmes is low. 20 Recommendations:'Planning' Implementation 1. The district health management Priority: teams (DHMTs) should henceforth l, 2: HIGH incorporate CDTI activities in the Indicators of success'. DIP, in the same format that is l. The DIPs for 2003-2004 include CDTI activities used for the other control 2. Training reports exist, which show that DHMTs programmes. have been trained to draw up suitable plans. 2. Where this has not yet been done, Who to take actiort'. NOCP should specifically train l. DHMTs, DEHOs and distnct EHTs. di strict envi ronmental health 2. NOC and NOCT. officers (DEHOs) and DHMTs to D e adlin e fo r c ompl et iort draw up a plan for onchocerciasis l. April 2003. control, that will fit into their DIPs 2. March 2003. 2.2.2 lEadership I In each district a district onchocerciasis coordinator (DOC) has been appointed. The yearly Mectizan distribution process in the districts is being initiated by messages from the NOCP (which is normal for this early stage of a project's life); thereafter the DOC and her/ his colleagues in the EHT proceed more or less unaided with the arrangements preceding the distribution. The other members of the DHMT are relatively uninformed about the programme - so too the district commissioners (DCs). Reasons for the relative lack of involventent of the DHMT: * It has delegated responsibility for the programme to the DEHO. * Since the programme is new it is normal for NOCP to take the lead at this stage. Recommendations:'Leadership' Implementation 1. Whenever members of NOCT visit the Priority: districts they should make a point of 1, 2: HIGH seeing and informing members of the Indicators of success'. DHMTs. This should be done in the 1, 2: Reports of next supervision visits by company of the DEHOs and DOCs. NOCT members to the districts. 2. NOCT members should visit DCs at Wo to take action: least once ayear, to discuss progress l, 2: NOC and NOCT, with DEHOs and DOCs and possible problems. D e adline fo r complet iort'. l. 2: At next routine supervision visit. 2.2.3 Monitonne supervlslon ! Data and requests are being channeled from civil servant (DOC/ DEHO) to civil servant (NOCP). The district health officer (DHO) is however being bypassed, which is not ideal - the more members of DHMT are kept informed, the more likely they are to understand the programme's budgetary needs. Supervision is efficient, with the EHT routinely supervising their HSAs (with occasional spot checks in the actual villages). Like all this team's work the supervision is also integrated - at each visit a supervisor will check several of the HSAs' many tasks. Problems uncovered from monitoring data and supervision visits are dealt with, and at the appropriate level: HSAs are always involved. The routine use of review meetings after distributions is excellent. In one case only success was specifically noted and praised. 2t Recommendations:'Monitoring and supervision' Implementation 1. Reports of CDTI activities at distnct level should Priority: routinely be passed on to the DEHO and DHMT, 1, 2: HIGH for them to forward to NOCP. This can be either Ind icators of sttcc e s s'. of the following: 1. NOCP receives reports from DHO * The full reports required by APOC. 2. Coverage reports are am accurate * The more limited information that will go into reflection of what happens on the the HMIS (see Recommendation 4 in 1.2.2 ground. above). Who to take action: 2. The district CDTI team should ensure quality data l, 2: NOCT, DOC, DEHO, DHO from lower levels, before compiling and Deadl ine for compl etion: submitting reports to NOCP. This means that the l. Immediately after the next accuracy with which CDDs and HSAs work with distribution. data needs to be checked, and further practical 2. Before the 2003 distribution training done if necessary. 2.2.4 Mectizan supply Orders for Mectizan are usually made on the prescribed forms. In half of the districts there were repofts of the Mectizan arriving late, which caused some irritation. The supply process is entirely managed by civil servants; however in some cases NOCT delivered the drug to districts, rather than asking them to fetch it. The DOCs usually store/ control the Mectizan without involving the district pharmacy: this new system may be more efficient. Reasons for the drug arriving late, and NOCT taking Mectizan to the districts: * DOC waited for all the district census results to be handed in, before doing the ordenng. This consumed a lot of time. x I consignment was held up at the local airport, due to a misunderstanding about tax exemption. * If NOCT is visiting a district anyway it makes sense to deliver the Mectizan as well. Recommendations:'Mectizan supply' Implementation 1. NOCT should set a deadline for the Priority: districts to hand in census forms. If 1. MEDIUM these are very late orders should be Indicators of succ e s s'. made based on the previous year's 1. NOCT makes orders in time for distribution. figure, plus l0%. Wo to take actiorz: 1. NOC and NOCT D e a dline fo r c ontp I e t i ort'. l. Immediately, for the 2003 order 2.2.5 Trainins HSAM I The EHT trains HSAs, and also supervises the training of CDDs by HSAs (one member of the EHT per training - the HSAs do the bulk of the work). This is reasonable since the programme is still relatively new in these districts. The team is able to give in-service training to newly arrived team members, but still depends on training organised by NOTF. 22 The training of HSAs is efficient - both the initial training, and the refresher/ review meeting after subsequent distributions. New HSAs are trained according to need. The training is not integrated with that for other programmes, but integration of the initial and review training is not desirable. HSAM activities are only partially successful. Both planning and implementation is lacking in important respects - e.g. some DCs and traditional leaders have not been approached. Drama groups for IEC have been trained but have no transport. ! Reasons for failure to implement HSAM: * The district teams have been focusing on implementation, and on the levels below them - so this has been an oversight. Recommendations:'Training and HSAM' Implementation l. DOCs and their teams should keep all Priority: civic leaders and decision makers (e.g. I. MEDILJM DCs and traditional authorities) Indicators of succ es s'. informed about progress in CDTI. 1. Civic leaders have up-to-date information about CDTI in their areas. Who to take action'. l. NOCT, DOCs and their teams (especially CDTI supervisors). Deadline for compl etion: 1. January 2003 and yearly thereafter 2.2.6 Finances and funding The DOCs/ DEHOs do not produce costed annual plans - rather they submit budgets for specific CDTI activities to NOCP. There is only partial evidence of efforts to limit expenditure. The DEHO and DOC have only a vague idea of the amount of funding that might be available to them from MoHP, but they do know what NOCP/ APOC is likely to provide, and work within that - but sometimes not all of it is received during a given year. I The MoHP contribution at this level is in terms of salaries, and this is clearly spelt out (but the document is at national level). There is no evidence of MoFIP allocation to CDTI at this level. Because of the highly integrated nature of the EHT's work, some MoHP funding is however supporting supervision where CDTI is also covered. Disbursed funds (both MoFIP and NOTF/ APOC) are tightly and efficiently managed. Reasons for inadequate budgeting practice and MoHP budgets: * The MoFIP budgeting mechanisms are not programme specific, and the outcome in terms of delivering funds is uncertain. * Since NOCP/ APOC is going to supply their needs, there is not much incentive for DEHO/ DOC to prepare budgets for the MoHP system - which may not provide anything at all. They have in any case not been trained to do so. * DHMT knows that NOCP/ APOC is supporting this programme for the time being, so will tend to allocate scarce resources to programmes without outside donor funding. * The DHMT is responsible for many programmes. It is not gorng to make special efforts to raise funds from other sources, to control a disease which is not so severe. 23 Recommendations:'Finances/ funding' Implementation 1. The onchocerciasis control plans in the Prioritl,: DIPs must be fully costed - limiting 1, 2: HIGH expenditure as much as possible. This Indic at o rs of suc c e s s'. is so that districts will begin to attract l. The DIPs for 2003-2004 include budgets for attention for budget allocations from CDTI activities. MoF{P. 2. Training reports exist, which show that 2. Where this has not yet been done, DHMTs have been trained to budget for CDTI NOCP should specifically train Who to take action'. DEHOs and DHMTs to prepare l. DHMTs, DEHOs and district EHTs. careful, targeted, minimal budgets for 2. NOC and NOCT. their onchocerciasis control plans, to D e adline fo r c ompl et iort fit into their DIPs. l. Apnl 2003. 2. March 2003. 2.2.7 Transport and material resources I Transport at present is adequate for the programme's needs. MoF{P is meeting all of the maintenance costs and provides some of the fuel, but NOTF/ APOC provides fuel for specific activities. Training materials (the two Malawi manuals) are sufficient - photocopies are made locally - but HSAM materials are in short supply in some cases. I Some maintenance on vehicles is carried out but it is not routine. MoFIP meets all maintenance and repair costs, but repair is very often delayed. The EHT copes with broken down vehicles, partly because of its habit of using transport in an integrated way. I Transport is used efficiently, as an integrated pool (although motorcycles are allocated to persons) and at the right level. Logbooks are universally available and completed, but their content is not checked. Management is aware that the present motorcycles will have to be replaced sooner or later. They will apply to MoHP when the time comes, and (on past experience) have a reasonable hope that something will materialise. There are no plans at present to approach other donors. Reasons for unsystentatic maintenance, delays in repair, poor logbook control, lack of HSAM materials: * Routine servicing is not possible due to lack of MoFIP funds - this is also the reason why repairs are delayed. * The probable reason why logbooks are not checked is that managers are too busy. * They depend on NOCP to provide HSAM materials. 24 Recommendations :'Transport Implementation and material resources' Planning: Priority: l. The issue of achieving I. HIGH sustainable transport and 2. MEDIUM equipment for this level of the Indic at o rs of suc c e s s'. programme needs to be fully l. These items are included in the 'Sustainability plan' addressed in the 'Sustainability 2. The task of reconciling logbooks monthly is written plan' described in 1.2.1 above. in the relevant officer's iob description. 2. The district transport control Who to take action'. officer should reconcile 1. NOC and NOTF, in consultation with Director logbooks monthly. CHSU and DHMTs. 2. DHO, district transport officer. Deadline for contpl etion l. June 2003. 2. Immediately. 2.2.8 Human resources The EHT is knowledgeable and skilled. Staff is generally stable. Unskilled persons joining the team are trained by NOCP, but the EHT judges that it would be capable of doing that itself. Salanes are regularly paid but they are very low. Staff satisfaction is generally high, as far as we could ascertain (this positive attitude may however be influenced by the fact that they have been receiving allowances on several occasions, due to their involvement in programme activities). There is no real incentive system operating. Recommendations :'Human resources' Implementation 1. The DHMT should assume full Priority: responsibility of training incoming/ 1. MEDIUM unskilled CDTI staff. Indicators of success: l. Incoming staff members are trained on the iob by other members of district EFITs. Who to take action: l. DEHOs and DOCs. Deadline for completion: l. Ongoing. 2.2.9 Coverage Geographical coverage: in three out of the four districts under discussion all villages indicated by the 1997 REMO survey have now been involved (although in some of them the process is not yet complete - census and/ or distribution remains to be done). I Therapeutic coverage: this is discussed at the 'national' level above. 25 3. Health centre catchment area level This area is taken to mean: . Primarily, the EHT working in the HC catchment area: village level HSAs and their supervisor, who is based at the HC. . Less importantly, the medical assistant and nurse/ midwife working in the HC. These persons are only peripherally involved in CDTI. 3.1 Overall grading (on a scale of 0-4) Malawi 'extension'proiect: sustainability at health centre level 4 3.5 .E 2.5 '6C') 2 o= C') G 1 5 o G 1 0.5 0 csa ^c ^"q -rQ" €$ a"" _c- {.di ..."e uod "..""". ."C a"-t' "-'"- "-""" group of indicators 3.2 Main findings and recommendations 3.2.1 Planning HSAs and their supervisor draw up monthly work plans, in which CDTI activities appear at the appropriate time. In one district a year plan (in the form of a Gantt chart) was seen, in which the activities relating to all EHT programmes at this level were set out - including CDTI. The HSAs (supervisor and village) are not however specifically taught to plan the yearly CDTI events in a sensible sequence. No written 'recommended package' for this level was seen - although HSAs have a set of activities that they routinely carry out. 26 Recommendations :'Planning' Implementation l. HSA supervisors need to be Priority: trained practically to 1,2: MEDIUM produce written year plans Indicators of success'. for their areas of operation. 1. The wntten year plans are produced. 2. NOTF should motivate 2. The written official 'recommended package' includes CHSU to have CDTI CDTI. officially included in the Wlrc to take action'. 'recommended package of l. NOCT, DOCs, EHTs, HSA supervisors activities' for HSAs, for the 2. NOTF, Director CHSU. six districts involved in the Deadline for compl etiort programme. l. End ofJanuary 2003 2. June 2003. 3.2.2 Leadership -t The EHT in the HC catchment areas have taken the programme on board, as simply another programme for which they are responsible. They start CDTI activities when the district level EHT asks them to do so, every year. The medical assistants in charge of the HCs all support the programme, but the understanding is (rightly) that it falls in the domain of the EHT. Leadership is therefore undertaken by the HSA supervisor. In two districts traditional authorities had been oriented about the programme, but not in the other two. Reasons why the traditional authority has not been oriented: * EFITs are focused on programme implementation, and the need to approach the TA has not been a priority for them. 3.2.3 Supervision and monitoring The HSAs submit their census and coverage data to their supervisor routinely and efficiently. S/he in turn has no problem in submitting reports and data to the district level. All of this takes place entirely within the MoFIP system. CDTI activities are effectively and efficiently supervised. In addition to primary visits to get CDTI launched for the year, other visits may follow if problems arise. Visits by HSAs to villages (and visits to the HSAs by their supervisor) are generally integrated, in the sense that more than one programme gets attended to per visit. Problems arising in the CDTI process are rapidly identified and dealt with, by involving village level HSAs, CDDs and the village leadership appropriately in solving the problems. The use of positive reinforcement as a management tool is limited to occasional verbal appreciation. Reasons why positive feedback is seldom used: * The mindset of managers of this new programme is on identifying problems to solve them, in order to achieve good results. 27 Recommendations:'Supervision and monitonng' Implementation 1. Steps must be taken to ensure that HSAs and Priority: their supervisors handle data and reporting I. HIGH 1007o correctly. In particular the denominator Indicators of success that is used must be correct. 1. All reports Who to take ctction'. 1. NOCT, DEHOs, DOCs, EHTs, HSAs. D e adl in e fo r c ompl etion'. l. June 2003, then ongoing 3.2.4 Mectizan procurement and distribution Quantities ordered were correctly calculated, and there were no reports of shortages. In three cases out of four correct order forms were used; ordering was timed to fit in with the preferred community timing for distribution; and the drugs arrived on time. The Mectizan is collected, stored and administered entirely within the MoFIP - using a parallel system to that used for other drugs. The Mectizan is sometimes fetched from the district by the HSA supervisor, and is sometimes delivered to the HC by the DOC. The drug is stored in the HC, where the control system is simple and effective. HSAs may fetch it from the HC, or the HSA supervisor may deliver it to them. In all these cases transport is readily available. 3.2.5 Training and HSAM All training can be justified. The initial training of CDDs is clearly needed, and subsequent 'training' at the review meetings deals with problems identified during the preceding distribution. Training is economical: using small numbers of tutors of the appropriate levels; choosing sites close to where CDDs come from; limiting the time spent. The village leadership is actively co-opted by the excellent practice of asking them to attend part of the CDD training: the effect this has on their participation is clear. Otherwise HSAM activities are carried out when the need arises. Traditional authorities have however been neglected in some districts. Recommendations:'Training and HSAM' Implementation l. Targeted, practical training is provided Priority: for HSAs, on data management and 1. HIGH reporting. 2. MEDIL']\4 2. On every occasion when NOCT staff Indicators of success'. visits districts they should briefly visit l. All reports are accurate. the relevant traditional authorities. 2. All traditional authorities are aware of the programme and support it. Who to take action: 1,2: NOCT, DOCs, EHTs, HSAs. Deadline for completion: 1.2: Before the next round of distributions 28 3.2.6 Financing and fundins HCs have no financial capacity and do not make budgets - their finances are managed at the district level, where information regarding the role of finances at the HC level may be sought. 3.2.1 Transport and materials Motorcycles (for HSA supervisors) and pushbikes (for HSAs) are provided - but some motorcycles have broken down. The district provides fuel but not enough - HSA supervisors report that they sometimes use their own money for fuel. Training materials are easily obtainable, but HSAM materials are often in short supply. There is no scheduled maintenance for the motorcycles. MoFIP pays for all maintenance and repairs, but repair is very often delayed. HSAs generally have to fund pushbike repairs themselves. Having said that CDTI is not really held back by these problems - the EHT copes. The use of the motorcycles is controlled by logbooks, which are faithfully kept but never really checked. The use of pushbikes is controlled by a system of personal attachment - in one district HSAs gradually purchase the bikes they use by means of a monthly payment of K 300. All transport at this level is used in an integrated way, each vehicle serving the needs of all programmes. The replacement of vehicles is a district level issue. ! Reasons wlry maintenance is not done, repairs are delayed, HSAM materials are few: x Due to lack of MoHP funds routine servicing cannot be afforded. Repairs similarly have to wait until funds become available. * Providing HSAM matenals is seen to be the responsibility of NOCP. 3.2.8 Human resources Staff is generally knowledgeable and skilled regarding CDTI - with one notable exception: the management of the data supplied by the CDDs. New staff members who have not yet been trained will either join a formal training event at district level or higher, or be shown the ropes by colleagues locally. EHT staff members at this level are fairlv stable. 3.2.9 Coverage Geographical coverage: in those HC catchment areas that are on stream all villages are covered. Therapeutic coverage: this is discussed at the 'national' level above. 29 4. Village/ community level 4.1 Overall grading (on a scale of 0-4) Malawi'extension' project: sustainability at community level 4 3,5 \t J 2,s .9 q) 2 o= cn G 1,5 G) 1 0,5 0 .*oe *."C €$ -""tt .o,"'t C "".C "..'* +)^c Lc- Group of indicators 4.2 Main findings and recommendations 4.2.1 Planning I The CDDs plan their work carefully: allocating areas to each other so that these are close to their homes; selecting times of work convenient to the community etc. They involve community leaders in dealing with problems which anse. In one case they also combine the census and distribution, thus lessening their workload. Reasons wlry the census and distribution are not combined: * This option is not offered to communities when the programme is explained to them. Recommendations :'Planning' Implementation 1. All communities should be Priority: offered the option of doing census 1. HICH and distribution at the same time, Indicators of succe ss'. to reduce the CDDs' workload. l. Village meetings have been held, where this option For villages which accept this is offered. option, Mectizan orders will be Who to take action: calculated from the previous 1. NOCT, DOCs, EHTs, HSAs and their supervisors year's census. D eadl ine fo r c om plet ion: l. Before the next round of distributions. 4.2.2 Leadership and ownership The community leadership is taking responsibility for the programme in the villages. They are aware of the reasons why people are not taking the Mectizan, and help to solve these and other problems related to the distribution: calling meetings, discussing benefits, accompanying CDDs, gathering the villagers for centralised distribution and so on. I The community selects the CDDs at a mass meeting. On that occasion they usually also decide on the mode of distribution, and sometimes also on the time of the year that they would like the distribution to take place. ! Community members are aware of some of the benefits of taking Mectizan (although some tend to exaggerate these). They generally know that the treatment is to be annual, for about 15 years. 4.2.3 Reporting ! Reports (of the census and distribution) are sometimes made by CDDs unaided, and sometimes together with their HSAs. Transport arrangements vary - sometimes the CDDs bring the reports/ registers, and at others the HSAs fetch them from the CDDs. Transporting the reports presents no problems, and reports generally arrive in good time. 4.2.4 Mectizau procurement and distnbution ! CDDs receive enough Mectizan for their communities' needs - there is usually a little surplus, which is returned to the HSAs. Interestingly CDDs do not know how to calculate the number of tablets they need. Sometimes the CDDs fetch the Mectizan from the HCs, and sometimes the HSAs fetch it and deliver it to them. The transport for either process presents no problems. Reasons why CDDs don't know how to calculate their needs: * They are simply not taught this simple calculation. Recommendations:'Mectizan procuremenU Implementation distribution' 1. CDDs should be taught how to calculate Priority: the number of tablets of Mectizan that I. MEDILM they need for their villages. Indicators of succe s s'. L All CDDs are able to make the calculation Who to take action: l. NOTF, DOCs. EHTs, HSAs. Deadline for completion'. 1. At the next round of review meetings 4.2.5 HSAM CDDs and community leaders identify situations where community members need more information, and take steps to promote acceptance and ownership of the programme - e.g. by calling meetings to answer questions, asking those who have taken the drug to speak to others etc. There is generally a shortage of HSAM materials at this level though. 3I 4.2.6 Funding The materials involved cost negligible amountsl transport is catered for; and the volunteer spirit of the programme is still intact. There is at present no place for funding at this level of the programme, and the issue is not a threat to sustainability. 4.2.7 Human resources I In most villages the ratio of CDDs to the population is adequate - but there are some notable exceptions, where CDDs have a heavy workload. Similarly most CDDs do not have to walk very far to do the distribution; a few though have to walk up to an hour, in communities with scattered homesteads. CDDs are generally well trained, although some experience problems with completing their registers, or are unsure about exactly what to do in case of side-effects. HSAs, CDDs and the village leadership generally know what to do to replace CDDs who fall away (some form of formal or on-the-job training). In general CDDs appear willing to continue their work - annual attrition rates are within acceptable limits (1.87o - 7.}Vo). The rate of attrition is however increasing yearly, and it remains to be seen where it will peak. Some of the reasons for attntion are migration and wanting to be paid. t Reasons wlry there are too few CDDs in some contmunities: * HSAs apparently know what the ratio should be - but this knowledge is not practically applied when CDDs are elected. This may be due to a dearth of candidates for the job, in some situations. Recommendations:'Human resources' Implementation 1. HSAs should ensure that the Priority: recommended ratio of CDDs to 1,2: FIIGH community households is adhered to. Indicators of succe ss'. 2. CDDs should be re-trained in drawing l. There is at least one CDD to 20 households up and maintaining their registers (or per 125 population). correctly, and also in the management of 2. The reports of CDD re-training exist. the side-effects of Mectizan. To save Who to take action'. money HSAs should do this during their l, 2: NOCT, DOCs and EHTs, HSAs routine visits to CDDs, at the beginning Deadline for contpletiort: of the next round of distributions. l, 2: Before the next distribution. 4.2.8 Coverage Geographical coverage within villages in Malawi is unlikely to be other than 1007o - the phenomena of hamlets or multi-cultural villages does not exist, and villages are generally small. Therapeutic coverage: this is discussed at the 'national' level above. It is important to note that the CDDs and community leaders of villages sampled for their low coverage claimed that their coverage rates were in fact excellent. 32 5. Overall sustainability grading for the Malawi 6extension' project Following the instructions laid down in the manual, the team analysed the situation in the project as a whole under the following rubrics: , Money - It is going to be essential to formalise as much as possible the ways in which MoHP will fund the running of the programme, at all levels, within the next year or two. The prospects of meaningful funding becoming available appear reasonable. . Transport -The future availability at national level will depend on NOCP's ability to gain access to vehicles provided from time to time by MoHP. The prospects here again are reasonable. At district and HC level the integrated nature of transport use by EHTs augurs well for the future. . Superttision - At NOCP level this will depend above all on the availability of transport, which it is reasonable to assume will still be available. At district and FLHF levels again the integration of EHT activities bodes well. . Mectizan supply - No problems are foreseen in this area. . Political commitntenl - This seems to be adequate at all levels. Following on the analysis above, the level of sustainability of the Malawi 'extension' project is judged to be as follows: Level of Description sustainability High This project is not far from being sustainable. With feedback from the team before departure, the project staff should be able to undertake the required remedial action. As is the case in all CDTI projects that are funded by APOC in its member countries, the most important factor determining future sustainability is probably the strength and degree of development of the district health service. In the case of Malawi this system is well developed (although under-funded). Each district has a well-developed EHT; there are HSAs in each village; there is a well established system of communication and supervision within the different levels of the EHT - all of this augurs well for the future sustainability of CDTI in the country. 33 6. A new insight In the course of the evaluation it became clear that there is a potential threat to sustainability, the effect of which will only become evrdent after APOC funding comes to an end. The present instrument fails to draw attention to it. It operates as follows: . Health workers at almost all levels of the health services in APOC countries receive salaries which do not meet the basic needs of their families, such as housing, food, clothing and education. It therefore becomes a matter of critical importance for them to find ways of supplementing their basic income. This they do in a number of ways, such as doing private practice after hours, running a farm or a small business, and so on. . A useful source of additional income is the allowances that are paid to health workers for their involvement in particular activities or programmes. In situations where workers are responsible for several programmes it is natural therefore that they will pay particular attention to programmes which are more lucrative. At present CDTI in Malawi is such a programme. At national level it offers salary top-ups, as well as per diem allowances for any trip to the field. At district level it offers EHT staff members allowances for activities such as training and supervision - so too for the HSAs at FLTIF level. Not surprisingly therefore staff members are highly motivated to underlake these activities. . When however the financial benefits of involvement in CDTI come to an end, staff members will be obliged to look for other programmes and activities which will supplement their incomes. Workers in the State Onchocerciasis Control Teams (SOCTs) in Nigeria were quite open about this. perceiving an opportunity for themselves in the upcoming lymphatic filariasis control programme, and planning enthusiastically to limit their future involvement in CDTI. Even if managers were aware of the extent to which this phenomenon poses a threat to sustainability, their options for countering it appear limited. Since it is not possible to maintain financial incentives indefinitely, the only alternative appears tobe thorough and timely integration: to have the CDTI programme so firmly embedded in routines and structures at each level, that it cannot be removed or neglected. It is suggested therefore that the instruments be adapted, to highlight the extent to which this factor is operating in a given situation. This can be done by creating an indicator in the 'human resources' category, which investigates the adequacy of workers' salaries, and the different ways in which additional income comes their way - particularly the role played by CDTI programme allowances. The data collected under such a heading will alert evaluators to the extent to which this threat is present. Measurement of the degree of integration of CDTI is already well catered for in the instruments. Its importance however needs to be stressed anew. It is suggested that 'integration' should become the sixth key rubric against which the final assessment of sustainability of a project is made - the others at present being money, transport. supervision, Mectizan supply and political commitment. 34 Advocacy activities and feedback/ planning workshops By now it is routine for project evaluation teams to undertake two additional groups of activities, in addition to the evaluation itself: . Advocacy for the CDTI projects, with civic and government leaders at different levels . Planning meetings with local stakeholders, at different levels. This section describes how the evaluation team went about fulfilling these two additional tasks. 1. Advocacy The team undertook the following advocacy activities 1.1 National level Involvine the National asis Co-ordinator Throughout the evaluation there was constant communication with the NOC. The team drew him in regularly during the planning phase and while data were being analysed - for instance to clarify the puzzling finding of low coverage rates, and the apparent 'neglect' of Chiradzulu. His deputy was also closely involved. The NOC expressed his appreciation of this involvement. Contact with officials in Lilongwe Only one day was available for a visit to Lilongwe, and appointments were made for meetings with the following persons: x Dr Somanje (Director of Disease Control, and chairman of NOTF). * Dr Salaniponi (Director of CHSU) * Dr W Aldis, WHO country representative. Due to a misunderstanding the evaluation team member was not fetched in time at the airport in Lilongwe, and the first two appointments were missed. An in-depth conversation was however held with the WR - a recent appointment - at the end of which he had a very clear understanding of the CDTI programme and its future needs (see Appendix I for a fuller report of the meeting). A telephone call to Dr Salaniponi at a later stage also provided useful information. 1.2 District and Traditional Authority level In each of the four districts where field work was done, efforts were made to inform and motivate the authorities. This was done in the following ways: . Courtesy visits upon arrival in the district, to the DHO, district administrator and DEHO. During the visit information was also collected from these persons. They were also invited to the coming feedback and planning meeting at Lutchenza. . Visits to the District Commissioners and some of the relevant traditional authorities, to inform them about the CDTI programme, the reason for the current evaluation, and the evaluation team's provisional findings. 35 2. Feedback/ planning workshops 2.1 District level This workshop was held on Tuesday 19th November, at a conference centre in Lutchenza,in Thyolo distnct. Attendance was excellent, from all five districts involved in the Extension project (including Chiradzulu). The workshop process developed as follows: . Introductions - the aim of the evaluation. . Feedback on findings at the community, FLHF and district levels, followed by in-depth discussion in plenary. . Each district developing its own plan for achieving CDTI programme sustainability. Since Chiradzulu has no programme yet, persons from this district were distributed amongst the others, to learn from their experience. These documents constitute the first draft of a 'Sustainability plan' for each district. It is the responsibility of NOCP to follow up the formalisation and implementation of these plans. Further details of the workshop are given below: the full programme and attendance list (Appendix 2), and the plans developed by four districts (Appendix 3). 2.2 National level This meeting was held on Wednesday 20th November, at the NOCP office in Blantyre. The entire national and IEF teams were present. It should be noted that the same persons had already attended the meeting at Lutchenza on the previous day, and were therefore up-to-date concerning the evaluation team's findings at the district and lower levels. The meeting lasted about five hours, and consisted of: . Feedback from the evaluation team, on findings at national level. . Discussion around each finding, concerning its accuracy and its implications for the NOCP. The steps that the NOCP now needs to take were discussed in detail - for example concerning significant problems areas such as Chiradzulu and the low therapeutic coverage rates. 36 Appendix 1 Report on meeting with WR We discussed how the country office could support the programme, once APOC funding ceases. The following emerged: WHO is strongly promoting a sector-wide approach among donors, in providing funding. Powerful donors like DFID are doing the same. This means that donors coordinate what they give and how they give it, to fit in with a common plan. Vertical programmes are to be harmonised. Donations are then pooled and used in an integrated way. The WR's ffice will be etlcolt donors to such Implication: Such integrated use of donations is exactly what the CDTI programme needs, e.g. in the critically important area of tr WHO Malawi has recently prioritised its areas of concern: controlling HIV/ AIDS and TB; disease surveillance and outbreak management; rolling back malaria; child survival; and safe motherhood. All programmes aimed at these priority problems require a well functioning distnct health service as their vehicle. The WR's ffice will therefore be working hard to promote such health service developmenf. I was able to point out to him that the district EHTs will be key players in the implementation of activities aimed at achieving these objectives. The EFITs require transport, primarily motorcycles and pushbikes, as a prerequisite for any project they try to implement. I also pointed out that the integrated use of available transport at this level is already a reality - which fits in with the 'sector-wide approach' insistence on shari resources between Implication: Such strengthening of EtITs' work is exactly what the CDTI programme needs - it uses the same infrastructure WHO has a tradition of supporting and husbanding the smaller technical programmes, such as trypanosomiasis, schistosomiasis and onchocerciasis. The WR's ffice in Malawi will continue to romote such ro rammes both technicall and ticall NOCP is assured of both technical support, and advocacy in the highest circles Finally, WHO Malawi has a biennially determined budget. In the 2002-3 version onchocerciasis control was allocated a sum of $ 10 000 (some of which has already been used). A feature of this budget is that it is almost entirely 'projectised' - funding is allocated to carefully defined activities, and may not be used for anything else. However the present WR hopes to change that, and also to obtain extra-budgetary funding which is more flexible. The WR is clearly expecting onchocerciasis control to take its place among the recipients of he his sources di Implication: NOCP is assured of a small sum, which can be used to fund operationally critical activities. Detlef Prozesky 2002-tt-18 37 Appendix 2 The district level feedback/ planning workshop 1. Timetable for feedback meeting - Lutchenza township Date: Tuesday 19 November 2002 Venue: Nali Motel Master of ceremonies: L Sitima Rapporteurs: FNkhoma&EJere Activity Time Facilitators Registration of participants 09.30 - 10.30 a.m. E Jere Opening prayer 10.30 - 10.35 a.m. F Nkhoma Introductions - participants 10.35 - 10.45 a.m. P Tambala/ L Sitima Introduction to the workshop programme 10.45 - 10.50 a.m. L Sitima/ S Munthali Introduction to CDTI 10.50 - 11.20 a.m. P Tambala/ L Sitima Evaluation objectives - what is sustainability? 11.20 - 11.50 a.m. D Prozesky Methodology and limitations 11.50 - 12.00 noon S Baine Summary of findings at community level 12.00 - 12.20 p.m. S Munthali Lunch 12.20 - 01.20 p.m. C Kwizombe Summary of findings at health centre level 01.20 - 01.40 p.m. S Mugisha Summary of findings at district level 01.40 - 02.00 p.m. D Prozesky Tea break 02.00 - 02.15 p.m. E Jere Discussion: the way forward 02.15 - 05.00 p.m. S Mugisha/ S Baine (with group work following) Closing prayer 05.00 - 05.05 p.m F Nkhoma 38 2. Workshop attendance Name District Title Address Kankhuni BNZ Blantyre DOC PlBag66, Blantyre Hausi H Blantyre DEHO PlBag 66, Blantyre Chinula PO Blantyre Pharmacy Tech. P/Bag 66, Blantyre Nyirongo NGC Blantyre Accountant PlBag 66, Blantyre Juma IPM Mulanje DOC Box22'7, Mulanje Mkolombwe RC Mulanje Director of Admin P/Bag 9, Mulanje Namukutiama Mulanje FVA IEC Box 50, Thuchila Mphepo CE Blantyre DCs Human Resources PlBag 97, Blantyre Kandiero J Phalombe DOC Box 79, Phalombe Katunga JL Phalombe Box 79, Phalombe Nyundo PJ Phalombe DEHO Box 79, Phalombe Yali S Phalombe CDTI Supervisor Box 79, Phalombe Kabudula Chiradzulu DHO Box2l, Chiradzulu Boko PEN Chiradzulu HRMO PlBag 1, Chiradzulu Raviwa FF Chiradzulu Pharmacy Tech. Box 21, Chiradzulu Zgambo A Chiradzulu Ag. DEHO Box 21, Chiradzulu Mhone M Chiradzulu PDHSA Box 21, Chiradzulu ZaingaFLD Chiradzulu Deputy DHO Box 21, Chiradzulu Theu MMK Mulanje DEHO Box22J, Mulanje Nguwo RZ Mulanje AEHO Box227, Mulanje Ndhlovu A Phalombe EDO P/Bag 32, Phalombe Khumbo C Phalombe DC PlBag 32, Phalombe Makonde B Mulanje PHSA Box 22J, Mulanje Banda L Mulanje IEC Officer Box22J, Mulanje Chunga E Chikwawa Ag. IEC Officer Box32, Chikwawa Moyo MP Chikwawa Ag. Pharmacist Box32, Chikwawa Mbadzo PJF Chikwawa CO PlBag l, Chikwawa Chisale TL Chikwawa PHSA Box32, Chikwawa Chunga Chikwawa DEHO Box32, Chikwawa 39 Appendix 3 District sustainability plans Blantyre Shortcomings What can we do to Who must By when must Priority improve? do it? we have done it? No ONCHO Include ONCO in DHMT March 2003 I activities in DIP DIP. Uncosted annual I Costing of annual DOC l't week Dec J work plans programme. 2002 -1 Realignment to DOC I't week Dec the end of 2002 financial year. Other members of the Sharing the plans DEMO Ongoing 2 DHMT are not aware with them. of Oncho plans Mectizan bypasses Mectizan to pass NOTF Immediately 4 the District Pharmacy through pharmacy Phalombe Shortcomings What can we do to Who must By when must Priority improve? do it? we have done it? Leadership To orient the TA DOC January 2003 Mectizan To store the drug at Pharmacist January 2003 supply Pharmacy for safety and proper storage. Training and Refresher courses for CDDs DOC February 2003 HSAM be done for 2 days: . More materials. . More sessions of FI/E in communities and FIIC. HSAs Dec.-April 2003 Finances/ To incorporate the NOTF By 2003 funding programme - HMIS All Human To increase training of HSAs NOTF By 2003 resources to 4 days and include data management. Coverage t Intensification of IEC in IEC Ongoing the impact area. Officer I Promotion materials including: * T-shirts for CDDs; if possible for CDTI health workers too. * Carrier bags. 40 Chikwawa Shortcomings What can we do to Who must By when must Priority improve? do it? we have done it? DHMT lacks of Briefing/orientation DOC Dec.2002 I knowledge on oncho DHMT. Lack of knowledge of Orient DDC DOC Dec.2002 2 oncho. by the DC and local leaders (DDC). Inadequate HSAM Procure and distribute DHO Dec.2002 - Jan 5 materials. adequate HSA 2003 materials. CDDs lack of Training/refresh them H.S.A By early Dec J- knowledge on oncho on drugs side effects. 2002 drugs' side-effects. Lack of information Conduct integrated DHO March 2003 4 on the costed budget planning (DHMT and for oncho. NOCP team). Supervision (due to Maintenance of DHO/ Dec.2OO2 - 6 lack of reliable broken down NOTF ongoing transport at district motorcycles. level) National level Conduct quarterly NOC Dec.2002 - 1 supportive national ongoing supervision. Mulanje Shortcomings What can we do to Who must By when must Priority improve? do it? we have done it? DHMT is not fully DHMT should fully DEHO/ Dec.2OO2 Medium supportive of oncho support Oncho. DOC Broken down Maintain NOT/ DOC Jan.2003 High motorbikes (5). Low motivation of Provide them with NOTF/ Ongoing Medium CDDs. incentives i.e. DOC umbrella, T-shirts and supervisors frequent visits. Poor calculation Train H.S.A and NOTF/ April 2003 High capabilities of HSAs/ CDDs. DOC CDDs. Inadequate Introduce multi-layer DOC/ On \going High information on oncho. dissemination of NOTF by communities oncho. messages. 4t Appendix 4 Suggested changes to the 'Sustainability' instrument The following changes are suggested, for the sake of clarity, and also to make the instrument suitable for use at the end of both Year 3 and Year 5 of APOC funding: Instrument l Indicator 2,3, bullet 'd': 'There should be evidence of action taken, based on recommendations in the reports of previous monitoring exercises.' Indicator 4.1, bullet 'a': 'Staff at this level should routinely only train staff at the level immediately below it.' ! Indicator 5.1, last bullet: this bullet should be split into two. Indicator 6.2, bullet 'b': 'The amount that the government has budgeted in one or more specific onchocerciasis control budget lines (e.g. current and capital) should be increasing yearly, as a proportion of total expenses. By the end of Year 5 of APOC funding the bulk of onchocerciasis control expenses at this level should be met from government funds; by the end of Year 3 at least half of it.' Indicator 6.3, bullet 'a': 'Project management at this level should be aware of the shortfall, if one exists, and of its size.' Indicator 7.4, bullet 'c': 'If it is planned that replacement will be from non-government sources, written commitment for this should have been obtained at the highest level in these donor organisations (end of Year 5). or negotiations should have started (end of Year 3).' Indicator 9.1: 'Check whether the projects in the country (or districts/ LGAs in the proiect) have a satisfactory geographical coverage rate.' I Indicator 9.2: 'Check whether the projects in the country (or districts/ LGAs in the proiect) have a satisfactory therapeutic coverage rate.' Indicator 9.2, bullet 'a': 'All projects in the country (or districts/ LGAs in a project) should have a therapeutic coverage rate of 657o or higher.' Instrunrcnt 2 Indicator 6.2, bullet 'b': The amount that the government has budgeted in one or more specific CDTI budget lines should be increasing yearly. By the end of Year 5 of APOC funding the bulk of CDTI expenses at this level should be met from local government funds; by the end of Year 3 at least half of it.' Instrument 3 Indicator 3.2: Check whether health service staff at this level is routinely and efficiently supervising CDTI activity at the communities on site in an integrated manner.' Indicator 6.2, bullet 'c': 'The proportion provided by the government (FLfm and/ or district/ LGA levels) should be the major one by now (end of Year 5) or covering at least half of expenditure (end of Year 3).' Indicator 7.4, bullet 'c': 'If it is planned that replacement will be from non-government sources, written commitment for this should have been obtained at the highest level in 42 these donor organisations (end of Year 5). or negotiations should have started (end of Year 3).' Instrument 4 Indicator 1.1: bullets re-written as follows: a. CDDs choose visiting times and routes which will make the work less burdensome. b. CDDs arrange with the community leadership for help with specific problems, such as families who are not willing to participate in the programme. c. CDDs carry out census and distribution during the same visit (using this census data for the following year's order). 43 Appendix 5 Timetable of the evaluation visit Date Activity Friday 8 November I 'John the Baptist' prepares sample, logistics Saturday-Sunday Final planning of methodology and research plan 9-10 November Joint discussion of instruments. Monday I I November Final preparations: transpoft; money; stationery; communications. Data collection: national level.(Blantyre). Tuesday l2 November I Data collection: district level (4 districts) Wednesday-Friday Data collection:2 HCs, 4 villages (in each of 4 districts) 13-15 November Friday afternoon: visit to Chiradzulu district Saturday-Sunday Data analysis: district, FLfm, community levels 16-17 November Report writing for these levels. Monday 18 November Data collection: national level (Lilongwe and Blantyre) Drawin g up recommendations Preparing for following day's workshop Tuesday 19 November t District level report back workshop at Lutchenza. Wednesday 20 November Data analysis: national level. Report writing for this level. Report back meeting: national level Thursday 21 November a Working on report Thursday-Friday Evaluation team departs for home. 2I-22 November 44