Evaluation of Community-based Management of Acute Malnutrition Programme supported by UNICEF in DPR Korea 2015–2017 Evaluation Report Commissioned by: UNICEF Country Office, Democratic People’s Republic of Korea July 2018 Action Against Hunger Paul Binns (Evaluator) and Saul Guerrero (Technical Director) 1 Acknowledgements We would like to thank the Government of the Democratic People’s Republic of Korea (DPRK) for its logistical and technical support in the design and implementation of the evaluation of the Community- based Management of Acute Malnutrition (CMAM) programme. Particular thanks go to: • The National Coordination Committee • Choe Suk Hyon, Deputy Director, External Affairs Department, Ministry of Public Health (MoPH) • Choe Sun Hui, Director, Child Treatment and Prevention Department, MoPH • Paek Hyang Ok, Section Chief, Child Health Management Section, Okryu Children’s Hospital (CMAM Master Trainer) • Pyo Hye Suk, Section Chief of Women’s Health Management Section, Pyongyang Maternity Hospital • Dr. Ri Hye Ok, Researcher and Doctor, Institute of Child Nutrition (ICN) • Dr. Han Myong Chol, Researcher and Doctor, ICN • Kim Sol Rim, Officer, External Affairs Department, Central Bureau of Statistics (CBS) • Ri Un Hui, Data Analyst and Statistician, CBS • Hye Ran, Officer and Researcher, External Affairs Department, Academy of Medical Science • Kang Nam Il, Director, Population Centre • Relevant Peoples Committees. Thanks also to the valuable contributions in quality assurance and ensuring the utility of the evaluation from the stakeholder reference group comprising the Directors of MoPH, ICN, CBS and the Child Data Management Unit; and Christiane Rudert (Regional Nutrition Advisor), Riccardo Polastro (Regional Evaluation Advisor, United Nations Children’s Fund (UNICEF) Regional Office), Hiroaki Yagami (Evaluation Officer, UNICEF Regional Office). Special thanks also to the evaluation management team led by Oyunsaihan Dendevnorov (UNICEF Representative, DPRK), Murat Sahin (UNICEF Deputy Representative, DPRK) and Silas Rapold (UNICEF Monitoring and Evaluation Specialist, DPRK). We would also like to thank all of the members of the provincial and county People’s Committees, the hospital directors, paediatricians, nurses, pharmacists, ri (village) paediatricians and doctors, and other support staff for their insights and discussions. Their contributions are the essence of this evaluation. This evaluation would not have been possible without the contributions of some extraordinary people. We would like to express sincere gratitude to Dr. Wisam Hazem (Chief of Nutrition, UNICEF DPRK) and Ajwang Fatuma (Nutrition Specialist, UNICEF DPRK) for their endless patience and valuable insights into the CMAM programme in DPRK. We are immeasurably indebted to evaluation team members Hyon Chol Jong (Nutrition Officer, UNICEF DPRK), Hye Gyong Kim (Master Trainer, UNICEF DPRK) and Pak Gyong Chol (Driver) for their ceaseless efforts, enthusiasm and endurance throughout the evaluation, and their seemingly infinite capacity for professionalism, understanding and hospitality. Last but not least, we would like to express our heartfelt thanks to the people of DPRK who directly or indirectly participated in, contributed to or supported the evaluation in any way and who, without exception, were kind and hospitable. 2 Contents Abbreviations and acronyms ............................................................................................................ 5 1 Executive summary .................................................................................................................... 6 Main findings and key recommendations .............................................................................................. 7 Conclusions: .......................................................................................................................................... 10 2 Country context and background ............................................................................................. 11 3 Evaluation purpose, objectives, scope and questions .............................................................. 15 3.1 Purpose ........................................................................................................................................ 15 3.2 Objectives .................................................................................................................................... 15 3.3 Scope of the evaluation ............................................................................................................... 15 3.4 Questions from the ToR .............................................................................................................. 16 3.5 Changes to the ToR ..................................................................................................................... 16 4 Evaluation methodology .......................................................................................................... 17 4.1 Limitations ................................................................................................................................... 19 4.2 Ethical considerations for participants ........................................................................................ 23 5 Main findings and analysis ....................................................................................................... 24 5.1 Relevance / appropriateness ....................................................................................................... 24 5.2 Effectiveness ................................................................................................................................ 35 5.3 Coverage ...................................................................................................................................... 39 5.4 Efficiency and quality of services ................................................................................................ 45 5.5 Impact .......................................................................................................................................... 56 5.6 Sustainability ............................................................................................................................... 59 6 Lessons learned ........................................................................................................................ 61 7 Conclusions .............................................................................................................................. 62 8 Recommendations ................................................................................................................... 67 Annex 1: Terms of Reference – Evaluation of CMAM programme UNICEF DPRK ............................ 72 Annex 2: Profile of CMAM programme sites, DRPK 2015................................................................ 84 Annex 3: Expansion of the CMAM programme into 60 more counties (total 149) in 2015 ............. 86 Annex 4: Expansion of the CMAM programme into 40 more counties (total 189) in 2016 ............. 87 Annex 5: UNICEF's Theory of Change for nutrition activities in DPRK ............................................. 88 Annex 6: Schematic Representation of UNICEF's geographical focus .............................................. 89 Annex 7: Evaluation matrix ............................................................................................................. 90 Annex 8: CMAM admissions data 2015–2017 ................................................................................. 93 Annex 9: Data-collection formats ................................................................................................... 96 Annex 10 : Technical Evaluation Team composition ...................................................................... 131 Annex 11: Technical evaluation sampling framework for health facilities implementing CMAM . 133 Annex 12: List of people met by the technical evaluation team during the CMAM evaluation & Summary of interviews ................................................................................................................. 134 Annex 13: LQAS team designation, location and distance from hospital ...................................... 146 Annex 14: LQAS coverage evaluation sampling framework .......................................................... 148 Annex 15: LQAS procedure ........................................................................................................... 149 Annex 16: LQAS data collection forms .......................................................................................... 151 3 Annex 17: LQAS classifications of coverage .................................................................................. 157 9 Bibliography ........................................................................................................................... 159 Tables Table 1: Amount and cost of RUTF distributed in DPRK in 2017 .......................................................... 47 Table 2: Cost savings from reductions in use of therapeutic products ................................................ 47 Table 3: Performance indicators, CMAM national guidelines, DPRK.................................................... 49 Figures Figure 1: Seasonal calendar of risk factors for acute malnutrition in DPRK ......................................... 25 Figure 2: Graphical representation of the