Meyu Muluke Woreda, ETHIOPIA July 19Th to 29Th 2013 Inés ZUZA

Meyu Muluke woreda, ETHIOPIA th th July 19 to 29 2013

Inés ZUZA SANTACILIA

ACKNOWLEDGEMENTS

International Medical Corps (IMC) and Coverage Monitoring Network extend its deep gratitude to all those who have contributed to this study including: the authorities in Meyu Muluke woreda (district) and East Hararghe province (Oromia Region), Ethiopia and to all the health personnel and village residents for their hospitality and cooperation. Avery special thanks to the mothers and caregivers of severely acute malnourished children.

A very special thanks to the IMC team in Harar, to the Nutrition Officer (ALMAZ TASISA) for his support coordinating the SQUEAC at field level and for his contribution on the improvement of this report. Thanks you to the IMC team in Addis Ababa; the National Nutrition Manager-IMC (Beka TESHOME) for his collaboration.

Thank you to the Federal Ministry of Health (MoH) for their zeal, support and motivation. The East Hararghe Nutrition Focal Person (Mesfin WORKU) for his support during the Harar training. To the Nutrition Focal Person (Daniel SISAY), the Extended Program of Immunization Focal Person (Iskender Mohamed), the Nutritional Survey Field Worker (Abdulahi AHMED) and the TB/HIV Focal Person (Ashenafi DOLEBO) in Meyu Muluke woreda.

Thank you also to Ezana TESFAYE ZEMO from the Autralian MoH for all his support and commitment with the SQUEAC.

This study would not have been possible without the hard work and commitment of everyone involved.

Lastly, thank you to the Office of Foreign Disaster Assistance (OFDA) through GOAL for financing this project.

EXECUTIVE SUMMARY

Meyu Muluke woreda (district) is one of the 19 woredas in East Hararghe province in the Oromia Zone of Ethiopia. It is composed of approximately 124 villages (divided into 19 kebeles) and an estimated population of 54,496 inhabitants (14.8 % of children between 6 and 59 months). The Dinkas and Luo tribes are the predominant ethnic group inhabiting Meyu Muluke woreda. Communities are pastoralist and agro pastoralist who continue to face food insecurity from re-occurring drought and subsequent livestock losses. International Medical Corps (IMC) was running a program to contribute to the reduction of morbidity and mortality related to acute malnutrition and improves nutrition practices in three woredas (Meyu Muluke, Kumbi, and Fedis) in East Hararghe Zone from January to July 2013. IMC has been working in food-insecure woredas of East Haraghe Zone since 2005, responding to the emergency nutrition needs. In Ethiopia, the nutrition services are delivered by the MoH. In July 2013 there were 13 OTP sites functioning in the Meyu Muluke woreda. The other six OTPs were having security problems. There are also three stabilization centres. Since the beginning of 2012 the MoH has created the Health Development Army (HDA). It is a team of community-level volunteers engaged in screening and mobilizing children under 5 and pregnant and lactating women. They can detect cases of a defined number of diseases (including malnutrition) and refer them to the health facilities. There is one HDA per 4 households. The Health Extensions Workers are formal salaried workers within the health system. They provide treatment for Severe Acute Malnutrition (SAM) as part of a Health Extension Program. Regarding the nutritional situation, no data of Meyu Muluke woreda is was available for the previous years. But data from Midega Tolla woreda, the nearest woreda available data (East Hararge Zone) in November 2012 was available the Global Acute Malnutrition and SAM rates which were respectively 10.0 % (7.4 -13.3 95% C.I.) and 0.2 % (0.0 -1.3 95% C.I.).

Resume of coverage assessment The coverage assessment was conducted to evaluate access and coverage of the Community based Management of Acute Malnutrition programme for children aged 6 to 59 months with SAM. It was conducted between July 19th and 29th 2013 and it was the first of its kind for the area. It was conducted at the beginning of the rainy season and the Ramadan.

The SQUEAC methodology used consisted of 3 stages, applying the principles of triangulation (by source and method) and sampling to redundancy. The coverage investigation conducted in Meyu Muluke woreda showed a period coverage of 90.5% (95% IC: 81.6% - 99.4%)

The table below presents the main barriers on which the program must act to improve coverage as well as specific recommendations how to do so:

Barriers Recommendations - Long distances 1. Advocacy on including the WHO standards in - Previous Rejection the national guidelines - Stigma (mother ashamed) 2. Implement mechanism to reduce impact of the - Wrong Admission and Discharge distance and inaccessibility Criteria (MUAC at discharge < 110) 3. Strengthen community sensitization - Insecurity 4. Reinforce supervision and improve data quality and follow up 5. Think over IMC strategy during the periods of no programme implementation / sustainability 6. Repeat the SQUEAC in six months or one year // before IMC support

CONTENTS

1. INTRODUCTION ...... 7 1.1 CONTEXT ...... 7 1.2 International Medical Corps in Meyu Muluke woreda ...... 11 2. OBJECTIVES...... 13 3. METHODOLOGY ...... 14 3.1. GENERAL OVERVIEW ...... 14 3.2. STAGES ...... 15 3.3. ORGANIZATION OF THE EVALUATION...... 20 3.4. LIMITATIONS ...... 21 4. RESULTS ...... 22 4.1. STAGE 1 ...... 22 4.1.1. Quantitative data analysis ...... 22 4.1.2. Qualitative data analysis ...... 30 4.2. STAGE 2 ...... 31 4.3. STAGE 3 ...... 33 A. The prior ...... 33 B. The likelihood ...... 34 C. The posterior ...... 35 5. DISCUSION ...... 38 6. RECOMMENDATIONS ...... 41 Annex 1 : Survey questionnaire for current SAM children NOT in the program ...... 44 Annex 2: Meyu Muluke wereda SQUEAC plan, July 2013 ...... 45 Annex 3 : SQUEAC Survey team ...... 46 Annex 4 : Terminology in Oromifa used to describe malnutrition and RUTF. Meyu Muluke woreda. Ethiopia. SQUEAC July 2013...... 47 Annex 5: Weighted BBQ, Meyu Muluke woreda SQUEAC, Ethiopia. July 2013 ...... 48

ABBREVIATIONS

BBQ Barriers, Boosters and Questions CI Credible Interval CHD County Health Department CMAM Community based Management of Acute Malnutrition CMN Coverage Monitoring Network CSAS Centric Systematic Area Sampling EPI Expanded Program on Immunization FDA Food Distribution Agents GAM Global Acute Malnutrition HC Health Centers HDA Health Development Army HF Health Facility HP Health Post IMC International Medical Corps INGO International Non-Governmental Organisation LoS Length of Stay LP Land Preparation MAM Moderate Acute Malnutrition MoH Ministry of Health MUAC Mid-Upper Arm Circumference ODPPC Oromia Disaster Prevention and Preparedness Commission OFDA Office of Foreign Disaster Assistance OTP Outpatient Therapeutic Programme RHB Regional Health Bureau RUTF Ready to Use Therapeutic Food SAM Severe Acute Malnutrition SC Stabilization Centre SFP Supplementary Feeding Program SSI Semi Structure Interview SQUEAC Semi Quantitative Evaluation of Access and Coverage TSFP Targeted Supplementary Feeding Programmes UNICEF United Nations Children’s Fund WHO World Health Organisation

1. INTRODUCTION

1.1 CONTEXT

4.1.1. Overview of the area

Meyu Muluke woreda (district) is one of the 19 woredas in East Hararghe province in the Oromia Zone (region with 180 woredas) of Ethiopia. Meyu Muluke woreda is composed of approximately 124 villages (divided into 19 kebeles) and an estimated population of 54,496 inhabitants1. With 14.8 % of children between 6 and 59 months (8,087 children). The Dinkas and Luo tribes are the predominant ethnic group inhabiting in the woreda. The altitude of this woreda ranges from 500-1700 meters above sea level. Figure 1: Ethiopia, Oromia Zone and East Hararghe province2.

East Hararghe province

Meyu Muluke woreda lies between 70 32' and 80 54' N latitude and 410 39' and 420 11' E longitude to the south of Harar town. It is bordered by Girawa and Bedeno woredas to the North, Gola Oda woreda to the West, Fedis woreda to the East and Bale zone and Somali regional state to the south.

1 From woreda health office. 2 From wikipedia : http://en.wikipedia.org/wiki/ (visited on September 2013)

The woreda has a total area of 4,988.66 Km 2 accounting for about 22.02% of the total area of East Hararghe Zone. It is located at a distance of 150km from Harar town to the south direction. There are two rainy seasons in the area, meher (June-September), used for crop production, pasture and water harvest and the short belg rains (February- May), mainly used for land preparation, planting of long cycle crops collected after the meher rains, small scale production, and improving water and pastures. While central and western parts of Oromia Region generally received normal to above- normal rains during the kiremt season this year, drought-prone areas in the east including East Hararghe Zone received insufficient rainfall; inadequate for crop development. The communities living in Meyu Muluke woreda are pastoralist and agro pastoralist who continue to face food insecurity from re-occurring drought and subsequent livestock losses.

4.1.2. Nutritional situation

Regarding the nutritional situation, International Medical Corps (IMC) in conjunction with Region’s Oromia Disaster Prevention and Preparedness Commission (ODPPC) has conducted a Nutrition and retrospective mortality survey (SMART survey3) in Midega Tolla woreda of East Hararge Zone in November 2012 (post-harvest periods). The Global Acute Malnutrition (GAM) and Severe Acute Malnutrition (SAM) rates were 10.0 % (7.4 -13.3 95% C.I.) and 0.2 % (0.0 -1.3 95% C.I.). No national data of Meyu Muluke woreda is available for the previous years. Currently in Oromia Regional State, with the exception of a few areas (where NGOs carry out repeated nutrition surveys), there is no nutrition surveillance /no time series data on nutritional status to inform program planning and management. Just to fill this gap ODPPC has planned to undertake nutrition surveillances together with (RHB). There are various ways to implement nutrition surveillance; these include growth monitoring programs, longitudinal anthropometric data systems, community or institution-based sentinel sites systems and repeated cross-sectional surveys. For start- up, the region has conducting six cross-sectional surveys that expected to be repeated bi-annually with the support of United Nations Children’s Fund (UNICEF) in six woredas. Midega Tolla is one of the targeted woreda for the above mentioned purposes4. An OTP coverage assessment using the Centric Systematic Area Sampling (CSAS) methodology was conducted in November-December 2010 in Babile woreda, Oromia

3 based on NCHS growth reference 1977. 4 IMC and ODPPC. Nutrition and retrospective mortality survey. November 2012.

region by IMC. The assessment showed an overall OTP period coverage of 55.3% (95% CI: 46.2% - 64.3%) and OTP point coverage of 46.0% (95% CI: 38.7% - 53.4%). No coverage assessment has been done in Meyu Muluke woreda.

4.1.3. Health access in Meyu Muluke woreda

Services are delivered both at Health Posts (HP), which are closer to community level and deliver primary health care and at Health Centers (HC), which are located within three kebele of Meyu Muluke woreda (Husse, Alola and Chella kebele). Since the beginning of 2012 the MoH has created the Health Development Army (HDA). It is a team of community-level volunteers engaged in screening and mobilizing children under 5 and pregnant and lactating women. They can detect cases of a defined number of diseases (including malnutrition) and refer them to the health facilities. There is one HDA per 4 households. The community has chosen them with the participation of the MoH. In many cases they have included traditional healers or traditional birth attendants into this HDA or community base volunteers (community- level volunteers previously working with IMC). Nevertheless not all of them are trained. They have attended discussion in HF about some diseases (especially in children). Mostly of them do not have MUAC tapes. Only the ones that were previously community base volunteers with IMC have it. The Health Extensions Workers (HEWs) are formal salaried workers within the health system (in HP). They provide treatment for SAM as part of a Health Extension Program containing 16 packages on topics such as hygiene, family health, disease prevention and control, and health education. Each HEW supervises along of 100-200 HDA. In Meyu Muluke woreda except insecurity area all kebeles have HP and at each HP there are two health extensions workers (HEWs), so in Meyu Muluke woreda there are 13 HP and 3 health centers (HC).

4.1.4. Nutrition services5

In Ethiopia, the nutrition services are delivered by the MoH. In July 2013 initially there were 13 OTP sites functioning in the Meyu Muluke woreda. Seven OTPs were inaccessible due to security problems (one of them became insecure during the investigation). Finally only 12 OTP sites were functioning in Meyu Muluke woreda at the end of the month. There were two HEWs workers per HP. They have been first health assistants and after one year training became HEW.

There are also three stabilization centres (SC) in Meyu Muluke woreda: Husse SC, Alola SC and Chella SC

Since 2004, the MoH started to integrate the in-patient and out-patient management of severe acute malnutrition into hospitals and HC (i.e., at regional and woreda levels). In 2008, the out-patient management of SAM was further decentralized to HP (i.e., at the kebele level). The objective was to ensure access to and coverage of malnutrition services by bringing the service closer to the community. It benefits families by reducing opportunity costs of accessing treatment. It also benefits the health system through capacity building and acts as the catalyst for strengthening nutrition activities within health facilities and at the community level, for treatment and prevention of malnutrition. The programme is in line with the first component of the National Nutrition Programme (NNP), with its focus on “Supporting Service Delivery” which includes “Increased Access for the Management of SAM.” The Federal MoH Protocol for the management of SAM is from 2007. The admission criteria are Weigh for Height < 70% (or <-3 Z-score using the WHO-2006 standards), MUAC < 110 mm with (with length > 65 cm) or presence of bilateral pitting oedema. In Meyu Muluke woreda the admission criteria is based on national protocol for SAM management. At HP level the activity implemented by health extension worker where the admission criteria is MUAC <110mm(with length >65cm) and/or bilateral pitting oedema. At HC level the activity implemented by nurse or health officer where the admission criteria is Weigh for Height < 70% (or <-3 Z-score using the WHO-2005 standards), MUAC < 110 mm with (with length > 65 cm) and/ or presence of bilateral pitting oedema. UNICEF provides the Ready to Use Therapeutic Food (RUTF) and medicines for SAM treatment.

5 IMC. Therapeutic Feeding Programme Coverage Assessment Report. Babile Woreda, Oromiya Region, Ethiopia December 2011.

For Moderate Acute Malnutrition (MAM) management, International Medical Corps manages cases of MAM in collaboration with the World Food Program (WFP), DPPC office, health offices and the community. Supplementary food was provided by the WFP. Based on discussions with the WFP and DPPC during implementation, International Medical Corps managed Extended Outreach Service (EOS) beneficiaries (beneficiaries who were identified by MoH mass screening each three month (Child Health Day (CHD) screening). Including also non-EOS beneficiaries who were newly identified by International Medical corps through the Targeted Supplementary Feeding Programmes (TSFP) staff screenings, MAM beneficiary and children graduated from the OTP. Most of the time delay of SFP food after screening is commonly due to transportation issue from government. SFP food is distributed by International Medical Corps SFP staff and Food Distribution Agents (FDA).

1.2 International Medical Corps in Meyu Muluke woreda

International Medical Corps (IMC) was running a program to contribute to the reduction of morbidity and mortality related to acute malnutrition and improve nutrition practices in three woredas (Meyu Muluke, Kumbi, and Fedis) in East Hararghe Zone from January to July 2013. IMC has been working in food-insecure woredas of East Haraghe Zone since 2005, responding to the emergency nutrition needs caused by recurring failed seasonal rains, which have negatively impacted the nutritional well-being, food security, and general health status of the population. Between emergency nutrition programs there are usually funding/support gaps. More recently until October 2012, IMC, with funding from the Office of Foreign Disaster Assistance (OFDA) through GOAL has been implementing the “Emergency Nutrition Support Program” in four woredas (Midega Tolla, Meyu Muluke, Kumbi and Gursum) to respond to the nutrition crisis and reduce the disaster risk for targeted populations using the community-based management of acute malnutrition (CMAM) approach. This programme included capacity building of Ethiopian Ministry of Health and other stakeholders through technical training, medical supply and equipment provision and nutrition education to improve responsiveness and promote behavioral change during the programme. The support from January to July 2013 was composed of the four components of the CMAM approach including: 1) Targeted Supplementary Feeding Program (TSFP) for MAM, 2) SFP for MAM pregnant and lactating women and other vulnerable groups, 3) Outpatient Therapeutic Program (OTP) and Stabilization Centers (SC) for SAM and 4) community mobilization and outreach activities.

At OTP/SC level IMC build the capacity of MoH staff and conduct close supportive supervision to ensure the quality of the service. UNICEF has provided the RUTF to MoH and IMC has provided logistical support to transport the RUTF from the zonal warehouse to woreda stores and from woreda stores to the health facilities. HEWs and health center staffs have managed the malnourished children at health posts and health centers. The emergency nutrition program is implemented in close collaboration with the MoH, DPPC, UNICEF, WFP as well as other stakeholders.

2. OBJECTIVES

Main objective

The main objective of this study was to evaluate access and coverage of the Community based Management of Acute Malnutrition (CMAM) for children aged 6 to 59 months with SAM in Meyu Muluke woreda, in East Hararghe province in the Oromia Zone of Ethiopia, using the Semi-quantitative evaluation of access and coverage (SQUEAC) methodology.

Specific objectives

- To develop capacity of various stakeholders on undertaking program coverage assessments using SQUEAC methodology - To determine baseline coverage for CMAM - To identify boosters and barriers influencing CMAM program access and coverage - To develop feasible recommendations to improve CMAM program access and coverage

Photo 2 : Stage one training at Harar town (Winta Hotel) for data collection, Ethiopia.

3. METHODOLOGY6

3.1. GENERAL OVERVIEW

The Semi-Quantitative Evaluation of Access and Coverage (SQUEAC) is a coverage assessment method developed by Valid International, FHI 360/FANTA, UNICEF, Concern Worldwide, World Vision International, Action Against Hunger, Tufts University, and Brixton Health. The methodology is “semi-qualitative” in nature, meaning that it draws from a mixture of both quantitative data from routine program monitoring activities as well as qualitative data collected on the field. This mixed methods approach combines data sources to estimate program coverage and to develop practical measures that can improve access and coverage. - Quantitative data came mainly from routine monitoring information that the program already collected including: admissions, defaulting, recovery, middle upper arm circumference (MUAC). Routine program data was coupled with “complementary data” like agriculture, labor, and disease calendars, anthropometric nutritional surveys, and agricultural and food security assessments. - Qualitative data collected came from interviews, focus groups and questionnaires with various key informants. Together, the data were triangulated by source and method to formulate hypotheses about coverage and access. Data triangulation is a powerful technique that helped validate our findings through cross verification. Hypotheses were then tested with small-area surveys and small sample surveys. Then, a wide area survey was conducted in the community to determine the point coverage estimate. Lastly, the results from the quantitative and qualitative analyses and the wide-area likelihood survey were combined and the overall global coverage estimate was calculated using Bayesian statistical techniques.

6 2012. SQUEAC and SLEAC Technical Reference. FANTA. Available at http://www.fantaproject.org/sites/default/files/resources/SQUEAC-SLEAC-Technical-Reference- Oct2012_0.pdf

The coverage study was conducted between July 19th and 29th 2013. It was the first of its kind for the area for SAM. It was conducted at the beginning of the rainy season and the Ramadan. The SQUEAC methodology used consisted of 3 stages, applying the principles of triangulation (by source and method) and sampling to redundancy.

3.2. STAGES

Stage 1: Identification of potential areas of high and low coverage and access barriers

Identification of potential areas of high and low coverage using routine program data; in this stage, triangulation of data was done by various sources and methods as highlighted below. 1. Quantitative data (February– June 2013) Quantitative, routine program data helped to evaluate the general quality of CMAM service, to identify admission and performance trends and to determine if the program adequately responds to need. It also helped point out problems in screening and admission. Lastly, routine program data analysis provided the first insights into variation in program performance between OTPs. Routine program data analysis included the following (for 13 OTP) - Global (OTP and SC) trends of admission and defaulters over time and compared to the agricultural calendar, the lean period, child epidemics and diseases, workload, weather patterns and key events - Admission: admission by OTP and SC - OTP and SC program performance indicators over time (recovery, default, death, non-response). - Discharged o Cured: length of stay (LoS) and MUAC at discharge. - Stock break out data. Complementary data from children card (for 11 or 12 OTP7) - MUAC at the time of admission (12 OTP) - Length of Stay for discharged cured (11 OTP)

7 Mojo weldia OTP area started to have security problems during the SQUEAC in July 2013 and some data could not be obtained. Goro Neyeda OTP data for length of stay could not being colected.

- The village lists populations belonging to each OTP and distance to OTP. Admissions per village Defaulter: there were no reported defaulters in the period, Not available - Admissions and other OTP indicators previous to February 2013.

2. Qualitative data

Qualitative data was collected to investigate program operations, to unravel the opinions and experiences of personnel involved in CMAM and to identify any potential barriers to access. The following methods were used: focus groups, semi-structured interviews, structured interviews, case studies and observation. Interviews and focus groups were conducted with key informants either directly or indirectly involved in the CMAM program. These included: women’s and men’s community, program personnel of IMC, local authorities (not religious leaders due to the Ramadan period) HDA, HEWs, caregivers of SAM children, health authorities. Informal caregivers (traditional healers and traditional birth attendants) were not found because mostly were integrated in the HDA. The BBQ framework. Throughout the investigation, the data are going to be organized, analyzed and triangulated using the Barriers, Boosters and Questions (BBQ8) framework. It is a tool that facilitates iterative data collection that is then categorized into one of three categories. The various data organized within the BBQ framework, when combined, will help providing information about where coverage is likely to be satisfactory as well as where it is likely to be unsatisfactory. Additionally, the BBQ provided information about likely barriers to services access that exists within the CMAM program.

Stage 2: Confirms the location of areas of high and low coverage

The goal of stage 2 is to test the hypotheses about coverage and access elaborated in stage 1. These hypotheses usually take the form of identifying areas where the combined data suggest that coverage is likely to be either high or low. The small-area surveys method was used to test the hypotheses for CMAM high and low coverage areas.

8 ‘Barriers’ are negative findings that deter from program coverage and complicate access to service. Conversely, ‘boosters’ contribute to a higher coverage and facilitate access. Lastly, ‘questions,’ are those findings elements to be further investigated, and either become a barrier or booster or remain inconclusive 16

The active and adaptive case-finding methodology was used to find SAM cases. Data surveys will be analysed using simplified lot quality assurance sampling (LQAS). The LQAS classification technique analyses data using the following formula:

⌊ ⌋

where

the threshold value number of cases found coverage standard

If the number of covered cases found (that is, those cases in the program) is greater than then then the coverage of the surveyed area is classified as being greater than or equal to the coverage standard . If the number of covered cases found (that is, those cases in the program) is less than then then the coverage of the surveyed area is classified as being less than or equal to the coverage standard The threshold chosen is 40%. The Centric Systematic Area Sampling (CSAS) coverage survey done in Babile woreda in November 24th to 5th of December 2011 by IMC was the guide to establish this threshold. If the number of covered cases found (that is, those cases in the program) is less than then then the coverage of the surveyed area is classified as being less than or equal to the coverage standard .

Stage 3: Wide area survey conducted to estimate overall coverage.

The goal of stage three is to calculate the overall coverage estimate. This is done using a Bayesian statistical technique called “beta-binomial conjugate analysis.” Conjugate analysis begins with a beta distributed, probability density called the “prior.” The prior is then combined with a binomial distributed, likelihood function called the “likelihood.” The likelihood is going to be determined by a wide-area coverage survey that will be conducted across the entire program catchment area; the mode of the likelihood was, in fact, the point coverage estimate from the survey. Because the prior and the likelihood are mathematically expressed in similar ways (as probability distributions) they can be combined through conjugate analysis, the result of which is the posterior probability density—the “posterior.” The mode of the posterior is the final coverage estimate. 1. The Prior The prior was constructed by combining the results from stages 1 and 2, that is: routine program data, qualitative data and all relevant findings from the small-area and small sample surveys. The prior was the result of combining two modes: 1) The weighted BBQ : a score from 1 to 5 was attributed to each element. The score reflected the relative importance or likely effect that the element had on coverage. The coverage estimate was calculated by the method explained above. 2) The histogram prior : During a participatory working group, the investigation team designed a histogram representing the prior mode. This was done realistically and democratically. The mode, minimum and maximum coverage values were chosen credibly. 2. The likelihood A wide-area “likelihood survey” was conducted over the entire program catchment area to calculate the coverage estimate. The active and adaptive case-finding methodology was used to identify the SAM cases. The case definition used for the coverage survey was defined as “a child matching the admission criteria of the programme”. The admission criteria of the Ethiopian CMAM programme included children aged between 6 and 59 months with at least one of the following criteria: 1) a MUAC of <11.0 cm and 2) bilateral pitting oedema A simple structured interview questionnaire was used to caregivers of non-covered cases for SAM in Annex 1.

The sample size required was calculated by using the following equation:

⌈ ( )⌉ .

1. Mode: prior value expressed as a proportion. 2. α et β: shape parameters of the prior. 3. Precision: desired precision. In the present case the precision used was 0.14 (14%). 4. SAM prevalence: 0.5% was chosen after stage 2 results to be the possible prevalence in the area. Because no available data of Meyu Muluke wereda was available. In the near Midhaga Tolla woreda of East Hararge Zone in November 2012 the SAM rate was at 0.2 % (0.0 -1.3 95% C.I.). 5. Average village population: 401 people in Meyu Muluke woreda (based on woreda health office data) 6. Population between 6 and 59 months : approximately 14.8% And the minimum number of villages needing to be sampled to achieve the sample size was calculated using the following equation:

⌈ ⌉ X

The number of required villages was randomly selected with ENA for SMART software9 from the list of accessible villages in Meyu Muluke woreda. 3. Overall Coverage Estimate The point or period coverage estimate was chosen for SAM coverage. By method of Bayesian beta- binomial conjugate analysis the prior probability density was combined with the coverage estimate from the likelihood survey to calculate the mode of posterior probability density. The Posterior Probability is the estimate of the overall coverage: it represents the synthesis of the prior probability and likelihood generated by the calculator with Bayes credible interval (CI) of 95%. Recommendations and Action Plan: A final important step is the development of an action plan that clearly identifies the actions to be undertaken, indicators, evaluation methods and deadlines.

9Available at: http://www.nutrisurvey.de/ena/ena.html [Accessed: November 2013]

3.3. ORGANIZATION OF THE EVALUATION

3.3.1 CMN technical support

The IMC team and the Ethiopian MoH from East Hararghe province received the technical support of the Coverage Monitoring Network (CMN). The CMN Project is a joint initiative by ACF, Save the Children, International Medical Corps, Concern Worldwide, Helen Keller International and Valid International. The programme is funded by ECHO and USAID. This project aims to increase and improve coverage monitoring of the CMAM programme globally and build capacities of national and international nutrition professionals; in particular across the West, Central, East & Southern African countries where the CMAM approach is used to treat acute malnutrition. It also aims to identify, analyse and share lessons learned to improve the CMAM policy and practice across the areas with a high prevalence of acute malnutrition. The technical and methodological support was provided by a Regional Coverage Advisor (RECO) Inés ZUZA SANTACILIA. During the evaluation CMN support was conducted in three phases:

- 1st phase: remote technical support for the planning and preparation of the evaluation with the CMN RECO. - 2nd phase: in field technical support in Meyu Muluke woreda. The CMN RECO was deployed to support training on the use of the SQUEAC methodology and the implementation of the evaluation until stage 2. - 3rd phase: remote support for the completion of the investigation, analysis of results and report writing.

SQUEAC plan in Annex 2.

3.3.2 Team training, logistic organization and evaluation development

The investigation team (described in Annex 3) was composed of members of IMC from Harar and Addis Ababa team, MoH staff (from East Hararghe province and Meyu Muluke woreda), one partner (MoH of Australia) and one nutritional survey field worker recruited for the SQUEAC.

The SQUEAC was conducted in the field by the CMN RECO in collaboration with the Monitoring and the Harar Nutrition Officer (AIMAZ TASISA).

A two days training in the SQUEAC methodology was made by the CMN RECO in Harar. This training targeted people that integrated the evaluation team and other people who might be interested in the methodology. The East Hararghe Nutrition Focal Person from the Ethiopian MoH participated on this two days training.

After the team was deployed to Meyu Muluke woreda (150km away from Harar) in the mountains. Some of the investigation team members stayed in the Husse capital of Meyu Muluke woreda while the RECO and the other members of the investigation team were based in Girawa (70 km from Harar

in Meyu Muluke woreda). This was due to the security and the logistic conditions in Meyu Muluke woreda. During the SQUEAC all the investigation team was also deployed to Girawa (to avoid daily travel from Meyu Muluke to Girawa)

For the three steps the investigation team was divided in five teams, composed by normally two people each.

3.4. LIMITATIONS

The evaluation was limited by the following elements:

- The security situation didn’t allow accessing one OTP and its area (of the 13 functioning). - The mountain roads and bad weather conditions (rain and fog) made the logistical coordination, communication and deployment of the teams difficult. - There was no telephone network in Meyu Muluke woreda. In Giraw, the telephone network had problems (frequent cut-offs) and no internet was available in both areas. The combination of these factors made impossible to share daily information from all the teams involved in the SQUEAC. Appointments in Girawa or Meyu Muluke woredas were done for sharing the information and continue the training. Also the RECO was unable to communicate and give technical support to the investigation team between the beginnings of stage 2 until the finalization of the SQUEAC (when the Harar Nutrition Officer was travelling back to Harar). - Initially technical support from National Nutrition Manager was going to be given for the investigation team along all the SQUEAC. Finally it could not go more than stage 2. - No data was available from the IMC programme or the MoH from before February 2013 for the SQUEAC.

Photo 3: Training on stage two at Girawa woreda.

4. RESULTS

4.1. STAGE 1

4.1.1. Quantitative data analysis

a. Needs response : admissions and defaulters trends compared to seasonal and key events calendar

Figure number 2 shows the OTP admission over a 4-month period (February –June 2013). This graph is aligned with a seasonal and key event calendar developed by the investigation team (weather patterns, seasonal calendar of human diseases associated with SAM in children, food availability, and workload). Together these two figures helped evaluate to what extent the program responds to seasonal needs. There were no defaulters along these months’. The Ethiopian calendar (Ge'ez Calendar10) starts in from the 29th August. The period showed in figure 2 corresponds to the Ge’ez calendar 2005 year. Since January to June 2013, 601 SAM children have been admitted to OTPs with a mean of 120.2 children admitted per month. Zero defaulters were notified during the period. Data quality issues were detected in one OTP along the register revision in Stage 1. The SAM admission trends are reflecting few months of the year trends. The hunger gap is from December to March, with a peak in February. Nevertheless, trends of admissions show the increase in the number of cases along the 4-month period. Comparison with the trends of admissions in other period should be necessary to extract reliable conclusion of these data. Normally however, the combination of prone diarrhea, food prices and hunger gap should make February the peak month for admissions. However, water shortage and displacement of families looking for water have made OTPs less accessible for the community.

10 The Ge'ez Calendar is the official calendar in Ethiopia. It is based on the Coptic calendar with a leap day, every four years. The Ethiopian Calendar has twelve months with 30 days each and a thirteenth month called Pagume with five or six days depending on the year.

Figure 2. OTP admission patterns over time compared with seasonal event calendar, Meyu Muluke woreda, East Hararghe province, Ethiopia. February-June 2013

200

150

100

admitted 50

0 Number of SAM cases SAM of Number Fev March April May June

2012 2013 S O N D J F M A M J J A Dry Dry bit rain Dry Rainy season Hot * ** *** spring summer autumn winter Seasonality Birra Bonna Afrassa Gana

Hunber gap

Diarrhea ARI

Malaria* Food Price

Sowing Activities LP** LP Weeding Harvest Grazing Water shortage Travel*** (looking for Water) Mass Screening

*Half of the Woreda (around eight Kebeles) are malaria endemic areas. The other areas are less endemic. In April-May there are areas where is very prone. June-August is prone for all the places.

** LP: Land Preparation *** The whole family displaces to look for water, especially in six Kebeles. It happens every year.

b. OTP vs. SC admissions

The percentage of children admitted to the SC could be an indicator of the timeliness of admissions. It is directly related to the percentage of SAM cases that arrive at the OTP with associated medical

complications. Children remaining untreated for long periods with declining nutritional status develop medical complications and end up needing SC care. A high percentage of SAM cases with medical complications could often be the product of a late presentation and uptake of services. In Meyu Muluke woreda, the proportion of program admissions requiring inpatient care from February to June 2013 is only 4.8%. This percentage is less than the 5% recommended for established programs therefore this can indicate early admission of SAM children in OTP services. Figure 3. OTP admission compared with SC admissions. Meyu Muluke woreda, East Hararghe province, Ethiopia. February-June 2013

4.8%

Admissions OTP Admission SC

95.2%

c. Admissions by OTP

Figure 4 shows the number of SAM cases admitted per OTP over the 4-month period (February –June 2013). Alalo OTP is the one that received more cases during the period (161 SAM admissions). Figure 4 : SAM admissions per OTP site . Meyu Muluke woreda, Oromia region, East Hararghe province, Ethiopia. February-June 2013.

180

160 140 120 100 80 60 40 Number of casesof Number 20 0

OTP

Alalo OTP is the nearest OTP to the areas that have insecurity problems, in the border (with no OTP functioning in the area). Go Sodoma OTP however is the one that received the fewer number of cases (20 SAM admissions). IMC team explained that this can be related to the river passing through the kebele that is used for irrigation therefore allowing the community to be less food insecure. Figure 5 shows the percentage of SAM cases admitted per OTP and the percentage of population of the catchment area per OTP over the 4-month period (February –June 2013). Figure 5: Percentage of SAM admissions per OTP and percentage of population catchment area. Meyu Muluke woreda, East Hararghe province, Ethiopia. February-June 2013

30%

25% 20% 15% 10% Percentage 5% Percentage population 0% Percentage admissions

OTP

Alalo OTP is the one that received proportionally much more percentage of cases than expected for their catchment area. Husse OTP has also more percentage of admissions that percentage of population in the health area. In the opposite, Gebibeda and Mojo Weldia OTP have a lower percentage of admissions compared to their percentage of population in the catchment area.

d. Admissions MUAC

Admission MUAC is an indicator for late /early presentation and service uptake at the OTP level. It can be a measure of direct coverage failure because late admissions are those non-covered SAM cases that went untreated for a significant period of time. Late admissions almost always require inpatient care and are associated with prolonged treatment, defaulting and poor outcomes. Figure 6 reports the MUAC distribution for SAM cases admitted by MUAC from February to June 2013. The admission MUAC criteria is < 110 mm. The MUAC median at admission was 107 mm (in red). That means 50% of the children arrive with a MUAC less than 107 mm. 97 mm is the inferior value for MUAC admission along the period. This can indicate early admission/detection of SAM children in the OTP programme. Yet there is still room for improvement because many cases have been admitted with MUAC < 105 mm (with very high risk of mortality). The median at admission by in general is very similar in all OTP. The OTP with less median MUAC at admissions were Biko and Goro Negeya (105 mm). And the one that had the better median MUAC was Gebibeda OTP with 109 mm.

During the analysis of MUAC data, an over-representation of rounded values (i.e. 105 mm, 100 mm, etc.) was observed, indicating imprecision in the MUAC measurement. Figure 7: MUAC at OTP admission. Meyu Muluke woreda, East Hararghe province, Ethiopia. February- June 2013

120

100 median 80

40 Number of cases of Number 20

99 98 97

102 101 109 108 107 106 105 104 103 100 MUAC (mm)

e. Admission by type

In Meyu Muluke woreda admissions are based on the presence of MUAC < 110 mm with (with length > 65 cm) or presence of bilateral pitting edema. In Meyu Muluke Woreda OTP 27.6% of the admissions were done by bilateral pitting edema (for the period of February to June 2013). These results will need further investigation to understand better the reasons for this high percentage of admissions by edema11. Looking in figure 8 at the percentage of admissions per OTP there are differences. Chella and Borka Jenta OTP had 46.9% and 40.0% of admissions by edema respectively while Chira OTP had 8.1% of admissions.

11 More information about Kwashiorkor: Briednd A; Myatt M; Dent N; Brown R. Putting kwashiorkor on the map. CMAM Forum. Available at URL: http://www.cmamforum.org/Pool/Resources/Putting-kwashiorkor-on-the-map- CMAM-Forum-2013.pdf [Accessed: November 2013]

Figure 8: Percentage of admissions by bilateral pitting edema per OTP admission. Meyu Muluke woreda, East Hararghe province, Ethiopia. February-June 2013

50% 40% 30% 20%

Percentage 10% 0%

OTP

f. Performance indicators

The performance indicators per OTP over the 4-month period (February –June 2013) indicate that 100% of children were cured, The performance indicators for the two SC showed that 100% of children were stabilized along the period.

g. Discharged cured

The length of stay before recovery provides helpful insight into the duration of the treatment episode (e.g. the time from admission to discharge). In figure 9 the OTP length of stay (LoS) for 12 OTPs (February to June 2013) has a median duration of 6 weeks. Usually international standards define typical LoS may be 30-40 days (4 to 6 weeks) and of maximum 8 weeks. In this case the maximum length of stay was 10 weeks, with 97.5% of the cases having ≤ 8 weeks of stay.

Figure 9: Length of stay for discharge cured. Meyu Muluke woreda, East Hararghe province, Ethiopia. February-June 2013

120 median

100

40 Number of childrenof Number 20

1 6 2 3 4 5 7 8 9 10 weeks

Figure 10 shows the percentage of MUAC at discharged cured under 110 mm (in 12 OTP) for the period February-June 2013. In general 13.9% of the discharged cured (65 cases) had MUAC at discharge < 110 mm. From them 41.5% had 109 mm, 29,2% 108 mm and 3.1% 107 mm. This could indicate early discharge of SAM children from the OTP programme. The reasons for it should be studied to try to avoid this situation. It can be observed that Borka Jenta had 54.3% of discharged cured with MUAC < 110 mm. Some OTP had more than 20.0% of the discharged cured with MUAC < 110 mm (Goro Negeya, Ifabase and Gugufe OTP). While others in almost all cases the discharge criteria were above 110 mm (Goro Sodoma and Alalo). Figure 10: Percentage of SAM children discharged cured with MUAC < 110 mm. Meyu Muluke woreda, East Hararghe province, Ethiopia. February-June 2013

60.00%

50.00%

40.00%

30.00%

Percentage 20.00%

10.00%

.00%

OTP

Distance influence on OTP admissions

Figure 11 reports the percentage of admissions compared to the percentage of population per Km of distance to the OTP (January 2012 – Mai 2013) at 5 weeks. It is appreciated that most of the population live in the 5 km range around OTP. Distance does not seem to have an impact on admissions. Populations from farther areas (≥ 4 km) seem to have more admissions compared to the estimations of populations in the area. Figure 11: Percentage of OTP admissions and percentage of population per distance in Km to the OTP. Meyu Muluke woreda, East Hararghe province, Ethiopia. February-June 2013

30%

25%

20% 15% % OTP admissions

Percentage 10% % Population 5% 0% 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 > 15 Distance to the OTP (Km)

4.1.2. Qualitative data analysis

The qualitative methods used included focus groups, semi-structured and structured interviews, cases studies and observations. Doing so revealed boosters and barriers. Interviews and focus groups were conducted in villages across the Meyu Muluke woreda. Questionnaire guides were adapted and oriented to facilitate the collection of data pertinent to program coverage and access. The investigation team also elaborated a list of terminology in the local languages (Annex 4) related to malnutrition and the RUTF. Qualitative data was triangulated by both method and source. All findings were indexed daily into the three-pane BBQ framework (complete BBQ can be found in Annex 5). Table 1 lists the sources and methods used during qualitative data collection. Questions ("Q") that appeared along stage one were analysed and resolved within days. Table 1. SQUEAC BBQ framework legend. Meyu Muluke woreda, East Hararghe province, Ethiopia. July 2013 Code Source Code Method 1. Community of Women A. Group Discussion 2. Community of Men B. Semi Structured Interview 3. Community Leader/ Religious Leader C. Case Study 4. Mother/Caretaker SAM D. Observation 5. Health Extension Workers (OTP), Health Workers (SC) E. Data Analysis 6. Health Development Army (HDA) F. Small area survey 7. Traditional Healer 8. Women Leaders 9. Staff of Health Center/ Head of HC 10. Woreda (woreda)MOH 11. IMC project Staff 12. FDA

Table 2 details the principal factors that either negatively or positively influenced program coverage and access during the qualitative data analysis; these are the main barriers and boosters.

Table 2: Main program barriers and boosters after qualitative data analysis. Meyu Muluke woreda, East Hararghe province, Ethiopia. July 2013. Barriers Boosters Insecurity: Lack of OTP in some Kebele Awareness on Malnutrition Cost of the time for staying in the SC as a family Awareness on OTP & Appreciation IMC support (Logistic, material , training & Rejection in OTP that is not their Kebele supervision) Not reporting some defaulters Understanding of schedule of treatment Refuse to go OTP because they assume they will Community network for OTP referral need to go to SC ( Occupation, many children at home, looking for water for the family) Stock-out RUTF ( medicines) High level of engagement/ Commitment of MOH

Not acceptance of the HDA referral because not Follow up of children by HEW (Referred to SC, recognised absente) Distance ( Pastoralist looking for water, HF far ) Good relation HDA,HEW, nurses (Weekly meeting) Lack of MUAC of many HDA ( no possibility of First Seeking Behaviour HP screening) nor training Mother sick OTP site is close to the communities Relating SAM with poverty; refuse to go to OPT of rich families Not aware that OTP is free, they go late Discontinuity of the IMC support

4.2. STAGE 2

This stage confirms the location of areas of high and low coverage and the reasons for coverage failure identified in Stage 1 using small studies, small surveys, small-area surveys. The routine program, quantitative and qualitative data collected in stage one, when combined, helped identify areas within the intervention zone where coverage was likely to be either satisfactory or unsatisfactory. This information was used to formulate hypotheses about coverage that were tested. Small-area surveys methodology were used to test this hypotheses. It was difficult for the team to identify areas of low coverage because areas with insecurity were the ones identified as low coverage but they were not accessible. Analysis of number of admissions and discharged cured with MUAC < 110 mm was finally chosen to be the factors for identifying low coverage area.

Table 3: Small-area survey selected villages for, Meyu Muluke woreda, East Hararghe province, Ethiopia. July 2013 Discharged Number of Other Villages OTP MUAC <110 admissions comments Low coverage areas Lami, Anano, B/qalla, B/guda, A/Hasan, Solom CHIRRA 28.6% Low High coverage areas Health Center. Hargaya, Mussa, Challa, CHELLA 4.3% High Staff very Lucha professional. The Lot Quality Assurance Sampling (LQAS) classification technique was used to analyze the data. The threshold value « p » that was 40%12. - Low coverage: thirteen SAM cases were found (n=13); one case was not covered. Zero cases were found in the process of recovery. d = (13 x (40/100) =5.2 ~ 5. So 12 > 5 » Not confirmation of hypothesis of low coverage area. - High coverage: seven SAM cases were found (n=7), two cases were found not covered. Four cases were found recovering. d = (7 x (40/100) =2.8 ~ 2. 5 > 2. Confirmation of hypothesis of high coverage area. The high coverage area was confirmed. The low coverage area was not confirmed. The investigation team decided to continue the investigation due to the difficulty of stabilizing hypothesis of low coverage apart from the insecurity areas. Like this the final idea was that coverage in all accessible areas could be homogenous and with high coverage.

12 The Centric Systematic Area Sampling (CSAS) coverage survey done in Babile woreda in November 24th to 5th of December 2011 by IMC was the guide to establish this threshold.

4.3. STAGE 3

A. The prior

As explained in the methods, the prior mode for the SAM program was calculated using the mean of the two coverage estimates: 1. The Weighed BBQ; 2. The histogram. Table 4 details the calculation of the prior mode.

Table 4: SAM program prior probability mode calculation. Meyu Muluke woreda, East Hararghe province, Ethiopia. July 2013

Method Boosters Barriers Calculation Résultat Weighted BB(Q) 62 74 (62+ (100-74))/2 44.0% (Annex 513) Histogram N/A 55.5%

Prior mode 49.8%

Next, using the equations presented in methodology 3, the shape parameters and were calculated with a prior mode of 20.3% about which the range of uncertainty was –20% and +25%.

was 22.0 and was 21.5. The distribution of the prior probability density has a mode at 49.8% and a 5% “credible interval” (i.e. the Bayesian equivalent of the 95% confidence interval) from 29.8% to 74.8%, shown in figure 12.

Figure 12. SAM prior coverage (binomial probability density). Meyu Muluke woreda, East Hararghe province, Ethiopia. July 2013

13 Annex 5: Weighted BBQ, Meyu Muluke woreda SQUEAC, Ethiopia. July 2013

Note: During the prior calculation due to a formula mistake the prior was designed as 52.8%. Both prior results (the one with the mistake and the one presented) were very similar -48.8% vs 52.8%-. Alpha and beta values were also very close ( 23.2 instead of 22.0 and 20.1 instead of 21.5). It has been chosen to include in the report the correct prior calculation, which had no repercussion in the likelihood sampling. All the rest of parameter (precision, selection of the villages) were the same.

B. The likelihood Sample size

The sample size was calculated using the equation described in methodology (for the “n likelihood”). In the present program the sample size for the likelihood survey used a precision of 0.14 (14%). The minimum number of children to be sampled was 7.6 ~ 8 children. The sample size was then translated into the minimum number of villages needing to be sampled to achieve the sample size using the equation of “n villages” described in the methodology part. With an estimated SAM prevalence of 0.5% in Meyu Muluke woreda and an average village population of 401.0 inhabitants (14.8% of which approximately are between 6 and 59 months), the minimum number of villages to be sampled was 25.4 (~25). They were randomly selected (described in methodology). Active case-finding

The 25 selected villages were divided up among the investigation team. Stage 3 lasted three days. Villages selected in Stage 2 were taken away from the random selection. In total, 29 SAM cases were identified. Twenty five of these children were covered and in an OTP. Four children were non-covered cases. Thirteen recovering cases were found. It is important to note that 5 recovering cases reported a stock break out in their OTP

Table 5: Results of the SAM active case-finding. Meyu Muluke woreda, East Hararghe province, Ethiopia. July 2013

SAM covered SAM not Recovering SAM cases cases covered cases caes 29 25 4 13 A questionnaire was administered to caregivers of the 4 non-covered cases to find out the reason (Annex 2). Of the 4 caregivers questioned, 100.0% realized their children were malnourished and were aware of the OTP program, yet they choose not to bring their children to it as explained in figure 13.

Figure 13. Barriers to service uptake found by the likelihood survey. Meyu Muluke woreda, East Hararghe province, Ethiopia. July 2013

Previous Rejection Stigma (ashamed) Distance

0 1 2 3 Number of cases

None of the four non-covered cases were in the CMAM programme before. The final precision from the likelihood of the 25 SAM children found was better than planned at 11.7% and not 14.0% as planned.

C. The posterior The period coverage estimate was selected as the most appropriate indicator for this investigation. The main reasons for this selection were 1) adequate length of stay in the OTP service, 2) suspected early seeking behavior (107 mm median admission MUAC with no cases above 97 mm at admissions, awareness on malnutrition and OTP program, etc). In a first time the prior (α 22.0 and β 1.5) and likelihood (38 SAM (covered cases + recovering) and 42 (total SAM cases + recovering)) values have been calculated through the Bayesian beta-binomial conjugate using the “SQUEAC coverage estimate calculation”. The posterior estimation has been estimated at 70.7% (IC 95%: 60.3%- 79.4%). Figure 14 is a graph of the three probability densities. It shows the prior (blue curve), posterior (red curve) and the mode of the likelihood survey (green curve). Figure 14. Program posterior coverage (Bayesian beta-binomial conjugate). . Meyu Muluke woreda, East Hararghe province, Ethiopia. July 2013

Visually, there is practically no overlap between the distributions of the prior and the likelihood suggesting a conflict between the prior and likelihood. A z-test revealed that indeed there was a conflict between the two (z = -4.0 and p = 0.0001), indicating that the combined analysis to calculate the probability post is not appropriate. Therefore the posterior calculations are not valid and cannot be considered. This finding has been analyzed by RECO and Harar Nutrition Officer (SQUEAC field coordinator) and some reasons have been highlighted: - In the prior calculation: o Information coming from areas not assessed in Stage 3 was included: insecurity. There are 19 Kebeles and 12 had functioning OTP at the moment of the SQUEAC. Seven areas were not accessible due to security problems (not OTP available in these area and expected low coverage areas). o Information about the periods of IMC not supporting the area was also included (Discontinuity of the IMC support, Stock-out RUTF ( medicines)) - The BBQ doesn’t include the positive information obtained in stage : high coverage expected in all accessible areas. - In RECOs opinion, the weighted BBQ give a lot of weight to some barriers that are important but should not have an impact in coverage to receive punctuations of 4 or 5. No technical support could however be given during the weighing process because no internet/telephone network was available. Other factors could have also influenced this. The ones however identified are those described above. It is important to remember the objectives of the SQUEAC investigation: to identify boosters and barriers influencing CMAM program access and coverage to undertake specific action plan and recommendations. In addition, to develop capacity of stakeholders on undertaking program coverage assessments using SQUEAC methodology. The SQUEAC investigation in Meyu Muluke woreda has achieved those goals. In this situation, if the sample size permits, it is recommended to use only the data of the likelihood evidence for estimating the posterior coverage. This can be done in two ways: - Method 1: Using the SQUEAC Bayes calculation using values 1:1 for α and β. This makes prior curve non informative. Like this 38 and 42 are the values for likelihood curve. Figure 15 shows this calculation.

Figure 15: CMAM program posterior coverage with prior non informative Meyu Muluke woreda, East Hararghe province, Ethiopia. July 2013.

The coverage estimation in this case is 90.5% (95% IC: 77.9% - 96.0%). - Method 2: Direct calculation of coverage and CI 38/42 * 100 = 90.5% and IC calculation: p ± 1.96 * √((p * (1 - p) / n) = 81.6% and 99.4% Method 1 should be chosen if sample size is < 60 and when coverage is near to 50%. Both results are very similar, but because of the previous explained reasons the selected method is number 2 and coverage estimation for Meyu Muluke woreda is 90.5% (95% IC: 81.6% - 99.4%). Other notes, recommendations for the next SQUEAC have been developed. - Barriers and boosters included should represent the situation of the area that will be assessed in stage 2 and the likelihood survey. If insecurity areas won’t be included in stage and 3, the insecurity and other related barriers better not appear in the BBQ or with a high weight. Nevertheless, the discussion and recommendations part will include the detected insecurity problems. - Barriers and boosters should reflect the actual situation of the CMAM programme in the woreda. Barriers that appear when IMC is not supporting the woreda should not be introduced in the BBQ. But the presence of these barriers may indicate the existence of coverage differences between the periods of IMC support and no IMC support. Discussions and recommendations should be also addressed at the issue. Maybe supporting a coverage assessment when IMC is not present in the area could help to identify and plan sustainable interventions in the woreda. - Technical support in the key stages (selection of Stage 2 and Stage 3) should be given to the investigation team. Presently or by internet/telephone, ensuring the field coordinator has net access in the area.

5. DISCUSION

This SQUEAC investigation in Meyu Muluke woreda gave a period coverage estimate of 90.5% (95% IC: 81.6% - 99.4%) for the accessible areas in the woreda (12 from 19 Kebeles). A combination of positive factors identified during the SQUEAC allows the coverage and access in the woreda to be very high: - Ethiopian MoH commitment: o National level: the MoH is implementing the innovative approach of the HDA at community-level that are integrated in the health system strategy (one HDA per 4 households). This strategy allows SAM case detection and community mobilization. They have also prioritize the inclusion of t traditional healers in the HAD. However there are still windows for improvement: training, providing MUAC for all, etc. o Woreda level: engagement of the health managers of the woreda. Active implication on the CMAM programme and supporting IMC activities in the area (including high level of participation in the SQUEAC). o Number of health facilities and health posts per habitant: 1 per approximately 3,000 habitants in the secure areas. - IMC support: at logistic, material, training & supervision level. - Positive Awareness on malnutrition and the CMAM program by the community. On the other side the periods without IMC support in the woreda have been identified by the MoH, OTP and communities as periods where CMAM work is more difficult. It seems logistics could be the biggest challenge for the MoH them while IMC is not supporting the area. This involves for example RUTF transport in the area. As mentioned in limitations the mountain area and the roads situation makes very difficult the deployment of stocks in the area without IMC support. Maybe advocacy for making the Meyu Muluke woreda more accessible (road access, electricity) could help (but this is medium/long term option not easy to do). Other problems for the periods where IMC is not supporting were related to the decrease in supervision and trainings of the CNW (while IMC works in the area they seem more engaged and motivated). Strategies to avoid stock breakouts and other problems (reduced number of supervisions) during the periods where IMC is not present in the area could be discussed to avoid the decrease in coverage and quality of attention along these periods. The use of donkey cars (for RUTF transport) has been proposed by IMC as a possible solution (already two donkeys have been donated to two HC on Meyu Muluke woreda). Advocacy could be used to inform about the importance of this factors to achieve high coverage and quality standards. It would be a importantto have available data of admissions and performance indicators along the years in the MoH and/or IMC database. IMC could support MoH on doing trends of admissions and performance indicators. This could help in planning and prevention (for example in emergency situations).

The performance indicators (cured, death, non-responders and defaulters) are all well above their corresponding SPHERE standards. During stage 1 the investigation team was surprised and skeptical about the 100% cured rate and stabilization of all OTP and SC cases. Some data quality issues have been detected in one OTP. Generally though, the quality of data seems to be reflecting the real situation. A good quality of care appears to be given to the SAM cases. Nevertheless, the discharged cured with MUAC < 110 mm was a problem detected during the SQUEAC that needs to be addressed. The number of bilateral pitting edema cases in the woreda appeared to be very high (27.6% of the total OTP admissions). The investigation team tried to assess if this could be due to a late seeking behavior or to the malnutrition type in the area etc. The finding suggests that is was more due to the type of malnutrition in the area. It could be interesting to undertake an investigation of the causal factors underlying this edema prevalence. A note which was not verified during the SQUEAC but mentioned during it (IMC team and some members in the community), described some cases of children with no admission criteria but with some signs of malnutrition (i.e. changes in hair color ) that lately became edema cases without MUAC < 110m. Some of these cases were referred by the caregivers to the HF/HP but were not admitted initially. During the SQUEAC it was discussed with IMC the fact that national guidelines are not using the new WHO standards. The lead investigation of the report suggests revising the strategy and proposes working on an advocacy strategy to adapt the admission criteria to the WHO 2005 standards (to be able to reach all SAM children). Barriers detected in the likelihood survey of stage 3 were distance, previous rejection and stigma. Distance appeared mostly in the qualitative data collection related to the pastoralist population looking for water or to distance to the health facility (but not in the quantitative data analyses related to the village distance and admissions). Some mechanism could be used to reduce the impact of distance, for example double ration (especially in specific periods of the year where pastoralist population moves looking for water). Previous rejection also appeared in stage 1. Some OTPs rejected cases from other areas, and also some caregivers brought the children to the OTP and “felt” they were rejected. The application of new WHO standards (which will be able to reach and treat more SAM children) and working on improving the communication between health staff and caregivers could reduce the impact of “rejection” or “perception of rejection”. Stigma that causes shame of the caregiver appeared in one case of non-covered cases of Stage 3. In stage 1, relating SAM with poverty was identified as a factor that could make caregivers refuse to go to the OTP. Community sensitization or discussion on the topic could help to reduce this perception. The biggest problem in the woreda already known by IMC and MoH (not on for the nutritional but for all health interventions) is insecurity. From 19 woredas, 13 woredas have a functioning HF/HP in their areas. Measures could be taken to support the HF/OTP in the insecurity borders. For example, Alalo OTP is receiving many SAM cases from the insecure areas (where no OTP is running). Extra support can be deployed to this area. The measures could be defined and developed with the participation of all the key actors of the area (HAD, community, HF/HP staff, MoH and IMC). Also,

advocacy for peace keeping in the area could be interesting to facilitate health access of all the population in the woreda.

6. RECOMMENDATIONS

Based on the results above there are some factors that could contribute negatively to program coverage in Meyu Muluke woreda. The following recommendations were developed to address them.

Recommandations Justification 1. Advocacy on including the WHO Feeling of rejection at the OTP makes first seeking behavior late standards in the national guidelines (Edema) 2. Implement mechanism to reduce Distance ( Pastoralist looking for water, HF far ) impact of the distance and Population of insecure areas goes to border OTP sites for treatment inaccessibility Work load in OTP due to people coming from other areas with no OTP

3. Strengthen community sensitization Stigma relating SAM to poverty, therefore some families refuse to go to the OTP centres by fear of being perceived as poor. Feeling of rejection on OTP makes first seeking behavior late (Edema) 4. Reinforce supervision and improve Wrong Admission and Discharge Criteria (MUAC at discharge < 110) data quality and follow up Incomplete patients history card ( routine medication data, disease occurrence) Not reporting some defaulters Rejection in OTP that is not their Kebele 5. Think over IMC strategy during the Discontinuity of the IMC support periods of no programme Stock-out RUTF ( medicines) * when IMC is not supporting implementation / sustainability 6. Repeat the SQUEAC in six months or one year // before IMC support

It is important to share the results of the investigation SQUEAC with the MoH and partners involved in CMAM. IF possible a presentation of the results should be presented to the IMC staff, MoH and partners. And giving a feedback to the HAD could help to improve their work. The action plan defined for implementing the recommendations (with indicators) will help to improve the coverage after this assessment. The proposed recommendations should be worked with the MoH and other actors in the field.

ACTION PLAN OF SQUEAC RECOMENDATIONS

Recommendati Verification Time Activities Follow up indicators Resp. ons sources line 1. Advocacy on - Conduct internal discussion on - Advocacy - Inclusion of the WHO IMC including the the importance/ viability of document 2006 standards in the WHO advocacy for change the national nutrition agenda standards in guidelines to the new WHO - National guidelines standards (2006). including WHO 2006 the national - Work on advocacy strategy to standards guidelines include on the malnutrition national guides the WHO2006 standards 2. Implement -Introduce double ration in - Monthly - Double ration Dec MoH & mechanism to special cases (long distance / report implemented in all OTP (if 2013 IMC reduce impact looking for water periods) and necessary) of the distance study other mechanism and - Study with CMAM actors (MoH, -Meeting - Number of ideas Dec MoH & IMC, OTP health staff and minutes proposed 2013 IMC inaccessibility leaders/HDA) ideas to increase - Number of ideas applied access of populations in insecurity areas. And on how to reduce the pressure in the border OTP (increasing staff, working days, etc.). 3. Strengthen - Involve HDA and OTP/SC staff - Activity -Number of sessions and Feb MoH, local community on sensitization and fight against reports people engages 2014 leaders & sensitization stigma and assotiation SAM with (monthly) IMC poverty. Reinforce the training on - Photos of malnutrition causes. the sessions -Engage local leaders in - SQUEAC formulating sensitization reports messages and disseminating these messages. - Reinforce the admissions in OTP Feb MoH & and referral to other OTP to avoid 2014 IMC “rejections of children from other OTP” - Encourage OTP/SC staff on the - Activity Feb MoH & importance of communicating to reports 2014 IMC avoid “rejection feeling”. - SQUEAC reports 4. Reinforce - Reinforce supervision on - MUAC at - 0% children discharged Jan MoH & supervision discharge criteria discharge with MUAC < 110 mm 2013 IMC ,improve data study quality and follow up - Improve defaulter’s notification. - Supervision - Performance indicators Jan MoH & reports 2013 IMC

- Support/ encourage MoH to - Excel/ - Monthly follow up Feb MoH & have a database with admissions graphs of available (for admissions 2014 IMC and performance data for follow follow up and other indicators) up. 5. Think over - IMC supports allow Meyu -Meeting - Proposed ideas Dec MoH & IMC strategy Muluke woreda to have a high minutes 2013 IMC during the level of coverage (logistics periods of no support, training and supervision programme of CNW and OTP). Are there more implementation ideas to mitigate the gaps of IMC / sustainability support in the area? - Advocacy for making Meyu - Document of - Advocacy strategy Fev IMC Muluke woreda more accessible advocacy developed 2014 (road access, electricity) shared with partners/ MoH 6. Repeat the - Assess if it could be interesting MoH & SQUEAC in six to do a SQUEAC (stages 1-3 or IMC months or one stages 1-2) in Meyu before IMC year // before support to assess the coverage IMC support without IMC support. - Repeat SQUEAC investigation

Annex 1 : Survey questionnaire for current SAM children NOT in the program 1. DO YOU THINK YOU CHILD IS SICK? ____ if yes: WHICH DISEASE?______2. DO YOU THINK YOUR CHILD IS MALNOURISHED (local word to describe it)? YES NO (STOP) 3. DO YOU KNOW OF A PROGRAM THAT CAN HELP MALNOURISHED CHILDREN? YES NO (STOP) If yes, what is the name of the program? ______

4. WHY YOUR CHILD IS CURRENTLY NOT ENROLLED IN THE PROGRAM? Do NOT prompt Ask “Anything else?” Several answers are possible

Answers Tick Notes

. No time/ Too busy (what is the caretakers’s occupation? ______)

2. OTP site too far away (how long does it take to walk? ______)

3. There is no one else who can take care of the other siblings

4. No money for the treatment

5. The child has been previously rejected (When? ______approximately) 6. Has been to the clinic but the child was not referred (When? ______approximately)

7. I do not think the program can help the child (prefer traditional healer, etc.)

8. Waiting time too long

9. Mother feels ashamed or shy about coming

10. Mother sick

11. Spouse does not allow

12. Other reasons (specify) :

5. WAS YOUR CHILD PREVIOUSLY ADMITTED TO THE OTP PROGRAM? YES NO (→ stop !) If yes, why is he/ she not enrolled anymore ? Defaulted : When ? ______Why ?______Condition improved and discharged by the program : When ? ______Discharged while he has not recovered : When ? ______Other : ______

Thank the caretaker and give a referral slip. Inform the caretaker of the OTP and date to attend

Annex 2: Meyu Muluke wereda SQUEAC plan, July 2013

Month Date Activities July SQUEAC methodological plan and organization and additional data collection. Inés arrival to Harar. 19 F Stage 1 (in Harar) 20 S SQUEAC training and workshop on data analysis 21 Su Collecting additional qualitative data in health facilities and communities 22 M (interviews, et). (in Meyu Muluke Woreda) 23 T Completing BBQ and Active and adaptive case finding training 24 W Stage 2 : Small area survey. 25 T Inés travel back to Harar and Spain 26 F Stage 3 : Data synthesis. Prior calculation. Sampling and preparation of wide area survey 27 S 28 Su Finalization of Stage 3 29 M

Report and recommendations processing

Annex 3 : SQUEAC Survey team

SQUEAC coordinators (2 women, 100% of women)

 INES ZUZA SANTACILIA, Regional Advisor, CMN Project  AIMAZ TASISA, Nutrition Officer, Harar, IMC

Evaluation team (1 women, 9 men, 90% of men)  SHEWANGIZAW MOTUMA, Meyy CMAM supervisor, IMC  DANIEL SISAY, Nutritional Focal Person in Meyu Muluke woreda health office, MoH Ethiopia  ISKENDER MOHAMED, EPI focal person Meyu Muluke woreda health office, MoH Ethiopia  ABDULAHI AHMED, Chella Health Center, MoH Ethiopia  ANEDENET ESHETU, Nutritional Survey Field Worker  ASHENAFI DOLEBO, TBI/HIV focal Person Meyu Muluke woreda health office, MoH Ethiopia  MUSSA ISMAE, Targeted Supplementary Feeding Programmes (TSFP) Team Leader, IMC  BODENA FEDA, T.S.F.P. Team Leader, IMC  MEAZA GARDIE, T.S.F.P. Team Leader, IMC  MISRAC BEKELLE, T.S.F.P. Team Leader, IMC

People involved in the evaluation team that have participated at some stages (2 men, 100% of men)

 BEKA TESHOME, National Nutrition Manager, IMC  EZANA TESFAYE ZEMO, MoH Australia

Annex 4 : Terminology in Oromifa used to describe malnutrition and RUTF. Meyu Muluke woreda. Ethiopia. SQUEAC July 2013.

Word in Oromifa English description Hanqina nyaataa Shortage of food Faadido (for wasting) Child very thin and retarding(wasting) Furfuraa (for Oedema) Bilateral pitting oedama Chunfa (For RUTF Juice Wanie Jolle Problem of Children Betetie RUTF - Gruel , Watery porridge Tutta RUTF Watery porridge

Annex 5: Weighted BBQ, Meyu Muluke woreda SQUEAC, Ethiopia. July 2013

Pts Boosters Source Meth Pts Barrier Source Meth OTP site is close to the 1 4 1,3 A,B 4 Stock-out RUTF (medicines) 1,5,4 A, D,C communities 1,3,4,6,2,  Feeling of rejection on OTP makes first seeking 2 5 Awareness on Malnutrition A,B,C 3 3,4,6 B,C 8 behaviour late (oedema)

Awareness on OTP & 1,2,3,4,5,  Work load in OTP due to people coming from other 3 5 A, B, C 3 9 B Appreciation 6,8 areas with no OTP

 Refuse to OTP because they assume they will need to 4 4 First Seeking Behaviour HP 1, 4,5,12 A,B,C 5 go to SC ( occupation, many children at home, looking 4,5,6 B,C for water for the family) Good Interface health staff OTP  Not acceptance of the HDA referral because not 5 3 D 4 6 B &beneficiary recognised Sensitization/outreach by 6 2 1,4,5,12 A,B 4 Distance ( Pastoralist looking for water, HF far ) 1,2,4 A,C HEW,FDA

 Lack of MUAC of many HDA ( no possibility of 7 3 Referral by HDA of SAM cases 6 B 4 5,6 B screening) nor training

Good relation HDA,HEW, nurses 8 4 5,6 B 5 Rejection in OTP that is not their Kebelle 2 A (Weekly meeting)

Good relation HDA- Community  Incomplete patients history card ( routine medication 9 3 3,6 B 3 D leaders. (they know the children) data, disease occurrence)

IMC support (Logistic, material , 10 5 1,5,10,11 B,D 4 Mother sick 1 A training & supervision

Understanding of schedule of  Relating SAM with poverty; refuse to go to OPT of 11 5 4 C 4 5 B, treatment rich families Follow up of children by HEW 12 4 5 B 4 Not aware that OTP is free, they go late 5 B (Referred to SC, absentee)

 Low communication between HEW & beneficiary 13 5 Screening every 3 months 12 B 3 5, D discharge

Community network for OTP 14 5 4 C 5 Not reporting some defaulters D referral

High level of engagement/ 15 5 11 B 4 Discontinuity of the IMC support 11 B Commitment of MOH

16 5 Insecurity: Lack of OTP in some Kebelle 10,11 B

17 5 Cost of the time for staying in the SC as a family 10 A, B, C

18 5 Children discharged cured with MUAC < 110 mm E

Tot 62  74 al