IMaD’s Lessons Learned Luis Benavente MD, MS CORE’s webinar August 29, 2012
Insert title here Country Number of health Health workers trained in topics related to malaria diagnosis as of June 2012 total N facility visits trained Baseline Outreach How to TOT and On-the- job Database National External Malaria all assesmt. of training & perform supervi- training data entry, Malaria competency micros- Diagnostic support laboratory sors during mainte- Slide sets, assess- copy and capabilities supervision assessments OTSS OTSS nance NAMS ments RDTs Angola 5 6 2 18 12 0 0 12 26 70 Benin 11 409 2 30 1671 9 0 0 40 1,752 Burundi 18 0 0 0 0 0 0 0 0 0 DR Congo 0 0 0 23 0 0 0 0 74 97 Ethiopia 4 0 4 0 0 0 13 17 0 34 Ghana 37 1109 37 46 3704 10 10 6 40 3,853 Guinea Con. 11 0 11 0 0 0 0 0 20 31 Kenya 1192 52 1192 66 312 0 0 37 67 1,674 Liberia 8 200 8 35 975 11 0 7 141 1,177 Madagascar 50 31 50 18 18 0 0 0 23 109 Malawi 14 521 14 75 1132 3 0 0 64 1,288 Mali 5 172 5 24 1188 4 0 0 40 1,261 Nigeria 40 0 0 0 0 0 0 26 0 26 Zambia 6 278 6 12 1733 5 0 9 92 1,857 Total 1,401 2,778 1,331 347 10,745 42 23 114 627 13,229 Proportion of febrile episodes caused by Plasmodium falciparum
Source: d’Acremont V, Malaria Journal, 2010 Better testing is needed to assess the impact of malaria control interventions
Parasitemia prevalence after IRS, countries with at least biennial monitoring
80%
Namibia-C Zimbabwe Moz-Maputo 70%
60%
50%
40%
30%
20%
10%
0% 1948 1950 1952 1954 1956 1958 1960 1962 1964 1966 1968 CORE’s Surviving Malaria Pathway
ITNs - Bednet use Caretaker recognizes Caretaker Malaria fever/convulsions provides as Malaria adequate care
Caretaker Wellness continues proper Improved health care and survival
Informal community services
IPT, malaria Caretaker seeks Provider gives outside care prophylaxis, quality care environmental improvements Formal community services
Public Private Caretaker Referral Provider gives accepts level quality care referral facility
Quality Assurance of malaria microscopy (MM) • EQA protocols harmonized with WHO, CDC • Regional accreditation done by AMREF with sponsorship from IMaD and IMaD logistical support (in Angola) • Slide banks being developed in Ghana and Ethiopia with TA from IMaD’s partner HWH • National Public Health Reference Laboratories being strengthened by IMaD • Refresher training in MM, performance testing • EQA protocols being institutionalized based on the Outreach Training& Support Supervision • Supervisory Data entry and analysis
Completing National Malaria Slide sets in a collaborative way
Ethiopia
Nigeria Benin Ghana
Equatorial Guinea Grading of malaria microscopy competence
Current grading of highly qualified microscopists is based only on species identification and parasite quantitation. This may be adequate for Asia/pacific but it is not for SAA countries such as Nigeria:
Distribution of 0 participants in 6 24 GFATM- sponsored MMRTs by level attained.
390 1 2 3 4
We propose a system based on parasite detection (telling apart negatives and positives), species ID and quantitation: Improvement in selected measurements of malaria microscopy (MM) competence among participants in multiple MM refresher trainings, Liberia 2009-2012
100% 92% Pretest first MMRT Postest last MMRT 90%
80%
70% 68% 61% 60% 52% 50% 44% 40%
30%
20%
10% 2% 0% PDetection SpeciesID Density Illustrative managerial decisions based on malaria microscopy competence
Was this person competent in Yes Did this person have an excellent Yes Recommendation: have this person attend a sensitivity, specificity, parasite ID, posttest? training session for trainers and engage as trainer and counting? No and supervisor.
No Recommendation: schedule this person for a refresher training and reassessment in two years.
Is this person competent in sensitivity and Yes Recommendation: assign to examine and report on negatives and positives. Schedule specificity? for refresher training in species identification and counting.
No
Has this person gone through at least three Yes Recommendation: assign new tasks as he/she is not proficient refresher trainings? and is taking too much effort to retrain. No Recommendation: schedule this person for attendance at the next refresher training and OTSS.
bold: under decision rule 90% italics: below average agreement below target (90%) for agreementdenominator less than 12 OTSS visits grouped in pairs Health Facility 23 34 45 56 CHD MONO-COUFFO 95% 100% 85% 80% CHD OUEME-PLATEAU 95% 100% 100% 100% CHD ZOU-COLLINES 100% 100% 100% 100% Cotonou 4 (Aidjedo) 95% 100% 100% 100% CS Abomey 71% 100% 100% 0% CS Adja Ouere 100% 100% 88% 94% CS Adolph Kolping D'Agbanto 100% 100% 95% 95% CS Alibori Gogounou 100% 100% 100% 100% CS Allada 100% 100% 100% 100% CS Godomey 100% 100% ##### ##### CSC ABGBANGNIZOUN 100% 100% ##### ##### CSC ABOMEY - CALAVI 100% 100% ##### ##### CSC ADJARA 100% 100% ##### ##### CSC ATHIEME 85% 80% ##### ##### CSC Ayelawadje 95% 100% 100% 100% CSC BANTE 89% 90% ##### ##### CSC Bembereke 100% 80% 80% 100% CSC Bohicon 40% 100% 83% 90% CSC BONOU 100% 100% ##### ##### CSC BOPA 81% 80% #### ##### CSC Boukoumbe 100% 95% 95% 100% CSC COPARGO 100% 100% ##### ##### CSC Cotonou 1 85% 85% 95% 100% CSC Dangbo 79% 93% 95% 100% bold: under decision rule 90% CSC Djakotomey 90% 100% 100% 100% italics: below average agreement CSC Djougou 100% 100% 100% 100% below target (90%) for agreementdenominator less than 12 CSC Dogbo 83% 95% 95% 90% OTSS visits grouped in pairs CSC Gbegamey 100% 100% 100% 100% Health Facility 23 34 45 56 CSC Glazoue 83% 100% 100% 100% CHD MONO-COUFFO 95% 100% 85% 80% CSC Grand Popo 95% 95% 90% 95% Quality Assurance ofCHD malariaOUEME-PLATEAU microscopy95% 100% 100% 100% (MM) using CSC Houenoussou 88% 100% 100% 100% CHD ZOU-COLLINES 100% 100% 100% 100% CSC Houeyogbe 70% 75% 80% 80% Cotonou 4 (Aidjedo) 95% 100% 100% 100% CSC Ifangni 95% 100% 100% 100% LQAS sampling at healthCS Abomey facility71% level,100% 100% Benin0% CSC KEROU 2009-201195% 100% ##### ##### CS Adja Ouere 100% 100% 88% 94% CSC Ketou 100% 100% 100% 95% CS Adolph Kolping D'Agbanto 100% 100% 95% 95% CSC Kpomasse 100% 86% 90% 100% Note: this analysis usesCS Alibori Gogounou slide as100% observation100% 100% 100% CSC NDALI unit 100% 100% ##### ##### CS Allada 100% 100% 100% 100% CSC OGANLA 75% 80% 60% 65% CS Godomey 100% 100% ##### ##### CSC Ouake 95% 100% 100% 100% CSC Ouesse 100% 100% 80% 80% CSC ABGBANGNIZOUN 100% 100% ##### ##### CSC OUIDAH 85% 90% ##### ##### CSC ABOMEY - CALAVI 100% 100% ##### ##### CSC Ouinhi 80% 85% 80% 80% CSC ADJARA 100% 100% ##### ##### CSC Parakou 100% 100% 100% 100% CSC ATHIEME 85% 80% ##### ##### CSC PEHUNCO 100% 90% 90% ##### CSC Ayelawadje 95% 100% 100% 100% bold: under decision rule 90% CSC Perere 70% 70% ##### ##### CSC BANTE 89% 90% ##### ##### CSC Seme Kpodji 80% 70% 75% 90% italics: below average agreement CSC Bembereke 100% 80% 80% 100% CSC Sô-Ava 77% 73% 88% 100% below target (90%) for agreementdenominator less than 12 CSC Bohicon 40% 100% 83% 90% CSC Tanguieta 95% 95% 94% 100% CSC BONOU 100% 100% ##### ##### CSC Toffo 91% 100% 100% 88% OTSS visits grouped in pairs CSC BOPA 81% 80% #### ##### CSC Tori-Bossito 100% 95% 95% 100% Health Facility 23 34 45 56 CSC Boukoumbe 100% 95% 95% 100% CSC Toviklin 85% 90% 95% 95% CSC COPARGO 100% 100% ##### ##### HZ Abomey Calavi 100% 100% 100% 100% CHD MONO-COUFFO 95% 100% 85% 80% CSC Cotonou 1 85% 85% 95% 100% HZ Adjohoun 100% 94% 90% 95% HZ Aplahoue 95% 90% 85% 90% CHD OUEME-PLATEAU 95% 100% 100% 100% CSC Dangbo 79% 93% 95% 100% CSC Djakotomey 90% 100% 100% 100% HZ Banikoara 90% 100% 100% 100% CHD ZOU-COLLINES 100% 100% 100% 100% CSC Djougou 100% 100% 100% 100% HZ Bassila 95% 100% 100% 100% HZ Come 90% 95% 90% 75% CSC Dogbo 83% 95% 95% 90% Cotonou 4 (Aidjedo) 95% 100% 100% 100% HZ Cove 100% 100% 100% 100% CSC Gbegamey 100% 100% 100% 100% HZ Dassa 95% 94% 94% 95% CS Abomey 71% 100% 100% 0% CSC Glazoue 83% 100% 100% 100% HZ Kandi Alibori 95% 100% 95% 95% CSC Grand Popo 95% 95% 90% 95% CS Adja Ouere 100% 100% 88% 94% HZ Klouekanme 90% 95% 95% 95% CSC Houenoussou 88% 100% 100% 100% CS Adolph Kolping D'Agbanto 100% 100% 95% 95% HZ Kouande 74% 95% 100% 100% CSC Houeyogbe 70% 75% 80% 80% HZ Lokossa 80% 85% 80% 70% CS Alibori Gogounou 100% 100% 100% 100% CSC Ifangni 95% 100% 100% 100% HZ Malanville 95% 100% 100% 100% CSC KEROU 95% 100% ##### ##### CS Allada 100% 100% 100% 100% HZ Natitingou 95% 95% 85% 80% CSC Ketou 100% 100% 100% 95% HZ Ouidah 85% 82% 56% 63% CS Godomey 100% 100% ##### ##### CSC Kpomasse 100% 86% 90% 100% HZ Pobe 100% 100% 100% 94% CSC NDALI 100% 100% ##### ##### HZ Sakete 90% 90% 100% 100% 100% 100% ##### ##### CSC ABGBANGNIZOUN CSC OGANLA 75% 80% 60% 65% HZ Savalou 100% 100% 100% 100% CSC ABOMEY - CALAVI 100% 100% ##### ##### CSC Ouake 95% 100% 100% 100% HZ Save - Ouesse 83% 79% 88% 94% CSC Ouesse 100% 100% 80% 80% HZ Suru Lere 90% 100% 100% 95% CSC ADJARA 100% 100% ##### ##### CSC OUIDAH 85% 90% ##### ##### HZ Tchaourou 90% 100% 100% 100% St. Michel 94% 100% 100% 95% CSC ATHIEME 85% 80% ##### ##### CSC Ouinhi 80% 85% 80% 80% Average agreement all labs 92% 94% 93% 94% CSC Ayelawadje 95% 100% 100% 100% CSC Parakou 100% 100% 100% 100% CSC PEHUNCO 100% 90% 90% ##### CSC BANTE 89% 90% ##### ##### CSC Perere 70% 70% ##### ##### CSC Bembereke 100% 80% 80% 100% CSC Seme Kpodji 80% 70% 75% 90% CSC Sô-Ava 77% 73% 88% 100% CSC Bohicon 40% 100% 83% 90% CSC Tanguieta 95% 95% 94% 100% CSC BONOU 100% 100% ##### ##### CSC Toffo 91% 100% 100% 88% CSC Tori-Bossito 100% 95% 95% 100% CSC BOPA 81% 80% #### ##### CSC Toviklin 85% 90% 95% 95% CSC Boukoumbe 100% 95% 95% 100% HZ Abomey Calavi 100% 100% 100% 100% HZ Adjohoun 100% 94% 90% 95% CSC COPARGO 100% 100% ##### ##### HZ Aplahoue 95% 90% 85% 90% CSC Cotonou 1 85% 85% 95% 100% HZ Banikoara 90% 100% 100% 100% HZ Bassila 95% 100% 100% 100% CSC Dangbo 79% 93% 95% 100% HZ Come 90% 95% 90% 75% CSC Djakotomey 90% 100% 100% 100% HZ Cove 100% 100% 100% 100% HZ Dassa 95% 94% 94% 95% CSC Djougou 100% 100% 100% 100% HZ Kandi Alibori 95% 100% 95% 95% CSC Dogbo 83% 95% 95% 90% HZ Klouekanme 90% 95% 95% 95% HZ Kouande 74% 95% 100% 100% 100% 100% 100% 100% CSC Gbegamey HZ Lokossa 80% 85% 80% 70% CSC Glazoue 83% 100% 100% 100% HZ Malanville 95% 100% 100% 100% HZ Natitingou 95% 95% 85% 80% CSC Grand Popo 95% 95% 90% 95% HZ Ouidah 85% 82% 56% 63% CSC Houenoussou 88% 100% 100% 100% HZ Pobe 100% 100% 100% 94% CSC Houeyogbe 70% 75% 80% 80% HZ Sakete 90% 90% 100% 100% HZ Savalou 100% 100% 100% 100% CSC Ifangni 95% 100% 100% 100% HZ Save - Ouesse 83% 79% 88% 94% CSC KEROU 95% 100% ##### ##### HZ Suru Lere 90% 100% 100% 95% HZ Tchaourou 90% 100% 100% 100% CSC Ketou 100% 100% 100% 95% St. Michel 94% 100% 100% 95% CSC Kpomasse 100% 86% 90% 100% Average agreement all labs 92% 94% 93% 94% CSC NDALI 100% 100% ##### ##### CSC OGANLA 75% 80% 60% 65% CSC Ouake 95% 100% 100% 100% CSC Ouesse 100% 100% 80% 80% CSC OUIDAH 85% 90% ##### ##### CSC Ouinhi 80% 85% 80% 80% CSC Parakou 100% 100% 100% 100% CSC PEHUNCO 100% 90% 90% ##### CSC Perere 70% 70% ##### ##### CSC Seme Kpodji 80% 70% 75% 90% CSC Sô-Ava 77% 73% 88% 100% CSC Tanguieta 95% 95% 94% 100% CSC Toffo 91% 100% 100% 88% CSC Tori-Bossito 100% 95% 95% 100% CSC Toviklin 85% 90% 95% 95% HZ Abomey Calavi 100% 100% 100% 100% HZ Adjohoun 100% 94% 90% 95% HZ Aplahoue 95% 90% 85% 90% HZ Banikoara 90% 100% 100% 100% HZ Bassila 95% 100% 100% 100% HZ Come 90% 95% 90% 75% HZ Cove 100% 100% 100% 100% HZ Dassa 95% 94% 94% 95% HZ Kandi Alibori 95% 100% 95% 95% HZ Klouekanme 90% 95% 95% 95% HZ Kouande 74% 95% 100% 100% HZ Lokossa 80% 85% 80% 70% HZ Malanville 95% 100% 100% 100% HZ Natitingou 95% 95% 85% 80% HZ Ouidah 85% 82% 56% 63% HZ Pobe 100% 100% 100% 94% HZ Sakete 90% 90% 100% 100% HZ Savalou 100% 100% 100% 100% HZ Save - Ouesse 83% 79% 88% 94% HZ Suru Lere 90% 100% 100% 95% HZ Tchaourou 90% 100% 100% 100% St. Michel 94% 100% 100% 95% Average agreement all labs 92% 94% 93% 94% % of labs attaining 90% agreement in parasite detection by department, Benin 2009-12
OTSS visits grouped in blocks of 4 1234 2345 3456 4567 5678 Alibori 89% 94% 100% 100% 100% Atakora 89% 85% 84% 88% 86% Atlantique 72% 71% 76% 85% 87% Borgou 67% 59% 73% 71% 73% Collines 73% 78% 86% 86% 94% Donga 85% 80% 80% 87% 80% Kouffo 79% 88% 94% 94% 93% Littoral 70% 73% 86% 81% 86% Mono 91% 81% 77% 75% 62% Ouémé 70% 73% 79% 88% 85% Plateau 86% 81% 81% 88% 86% Zou 92% 96% 95% 90% 83% Grand Total 79% 79% 84% 86% 84%
Visits aggregated in blocks of four, at departmental level, unit of analysis is laboratory. Three out of four target departments (yellow) were initially under national level, and they were all above country average in the last block of four supervisory visits.
Average agreement vs % attaining target
MSF moved away from reporting average agreement (2009) to report the % of intervened laboratories attaining 95% agreement (2011).
Why?
• a 0-100% scale in the vertical axis renders a virtually flat line in Fig 1. • 90% average agreement is too easy to obtain if most HFs contributing MMQA data are hospitals with qualified technicians. • the second % identifies if too many Hfs have been left behind, an overall agreement will hide heterogeneity in impact attained.
Priorities for resource allocation (infrastructure, equipment, training, EQA) to public health facilities based on malaria risk, caseload and diagnostic capabilities as MOP mandates focus on endemic districts
Health facility is No Lower priority than facilities in malaria-endemic located in a districts. May still benefit from early warning district of systems to detect outbreaks, place some small high malaria stock of RDT to confirm imported cases. risk?
Yes
40 or more Has a Has a t Priority for Has enough suspected Yes laboratory Yes Yes least two Yes EQA efforts. with functioning malaria trained lab Has basic adequate microscopes cases/ day technicians diagnostic infrastructure ? ? ? ? capacity.
No No No Use RDTs No Priority for infrastructure development/ improvement (by other agency) Priority for equipment. Priority for training.
Needs a microscope (has Has a Yes Low priority none, it’s trained lab Yes broken, they are from DOMC technician insufficient or perspective. needs another ? for TB)?
No Needs a microscope when any of those conditions apply 1) lab has none No and caseload justifies having one; 2) lab has one it’s not in working order; No priority No priority 3) lab needs another because of high malaria caseload and sufficient lab technician; 4) lab needs another to diagnose other conditions (i.e.TB) Malaria Microscopy QA Priorities During outbreaks Continuous During malaria At referral, teaching (SPR=27%) (45 and 22%) season (26%) hospitals (20%)
Distribution of Kenyan laboratories by workload (malaria slides/month) and epi zones, 2008
100% 800+ 90% 601-799 80% 401-599 70%
201-399 60%
50% 1-200 Over 80% of laboratories have a low 40% average demand of malaria diagnostic
30% services. At this level of utilization, it makes more sense to use RDTs than 20% microscopy
10%
0% Highland epidemic Lake endemic Coast endemic Semiarid seasonal Low risk OTSS checklist to be completed by the laboratory supervisor:
•Health Facility / Laboratory identification, contact information •Laboratory equipment, supplies and consumables •Minor laboratory equipment •Malaria reference materials •Internal Quality Assurance (QA) •External QA •Malaria slide blind re-checking •Rapid Diagnostic Test QA as per observation •Observation: preparation of thick and thin blood films •Observation: staining and reading of thick and thin blood films •Issues identified and recommendations •Signatures Slide preparation and reading scores RDT observation score External QA of Malaria Microscopy using blind re-checking during OTSS % agreement on parasite detection, Benin OTSS rounds 1 to 4
Round 1 Round 2
Round 3 Round 4 Prescriber adherence to negative test results Credits and acknowledgements
• IMaD is funded by the U.S. Agency for International Development (USAID) under contract number GHS-A-00-07-00022-00. • MCDI staff –including past and current in-country coordinators- in alphabetical order: Luis Benavente, Hannan Bestman, Joseph Carter, Petros Chirambo, Mamadou Diouf, Glenn Edosoa, Sean Fennell, Tobias Johnson, Seraphine Kutumbakana, Timothy Nzangwa, Isaac Osei-Owusu Bediako, Chris Petruccelli, Saliou Ramani, Saye Renion, Khadidia Waye, Nicole Whitehurst and Matt Worges. • AMREF’s: Jane Carter, Bill Yaggy and Emmanuel Yamo Ouma. • HWH’s: Bob Jordan, W. Roy Prescott and Georganna Prescott.
We also thank the dedicated and expert assistance from Dr Daouda Ndiaye, UCAD (U. of Dakar) , and national counterparts, too many to name.