Evaluation of the Bringing Nutrition to Scale Project in Iringa, Mbeya and Njombe Regions (2013–2017) Evaluation Report 23 April 2018 Prepared by Stephen Turner (Team Leader) Bjorn Ljungqvist Joyce Kinabo Jim Grabham Proposal contacts: Evaluation of Bringing Nutrition to Scale in Iringa, Mbeya and Njombe ACKNOWLEDGEMENTS AND DISCLAIMER The evaluation team are grateful to the nutrition colleagues at the UNICEF Country Office in Dar es Salaam and the Sub-office in Mbeya for all their support in providing information and facilitating meetings throughout this assignment. Quality support for the evaluation has been provided by the QS team assigned to the evaluation: Stephen Anderson (Food Economy Group) and Stephen Lister (Mokoro Ltd). The authors take full responsibility for the contents of this report. The designations employed, maps and the presentation of the material in this document do not imply the expression of any opinion whatsoever on the part of UNICEF concerning the legal status of any country, territory, city or area, or of its authorities, or concerning the delineation of its frontiers or boundaries. ii Evaluation of Bringing Nutrition to Scale in Iringa, Mbeya and Njombe Contents Summary v 1. Introduction __________________________________________________ 1 1.1. Evaluation purpose and scope ___________________________________ 1 1.2. Country context ______________________________________________ 1 1.3. Nutrition in Tanzania and the project area __________________________ 2 2. The BNTS and ASRP projects _____________________________________ 5 2.1. Project description ____________________________________________ 5 2.2. Key stakeholders and linkages ___________________________________ 8 2.3. Summary of reported performance ______________________________ 10 3. Approach and methods _________________________________________ 11 3.1. Evaluation approach __________________________________________ 11 3.2. Evaluation methods __________________________________________ 11 3.3. Evaluation activities __________________________________________ 12 4. Findings ____________________________________________________ 12 4.1. Relevance __________________________________________________ 12 4.2. Effectiveness _______________________________________________ 16 4.3. Efficiency __________________________________________________ 30 4.4. Sustainability _______________________________________________ 33 4.5. Equity _____________________________________________________ 41 4.6. Gender ____________________________________________________ 41 5. Conclusions _________________________________________________ 43 5.1. Relevance __________________________________________________ 43 5.2. Effectiveness _______________________________________________ 43 5.3. Efficiency __________________________________________________ 47 5.4. Sustainability _______________________________________________ 48 5.5. Equity _____________________________________________________ 48 5.6. Gender ____________________________________________________ 49 6. Recommendations ____________________________________________ 50 Annex A. Terms of Reference __________________________________________ 54 Annex B. BNTS and ASRP design diagrams ________________________________ 62 Annex C. Evaluation matrix ____________________________________________ 64 Annex D. Research instruments ________________________________________ 79 Annex E. Stakeholder analysis _________________________________________ 86 Annex F. Nutrition sector data _________________________________________ 89 Annex G. Project performance data _____________________________________ 102 Annex H. Maps_____________________________________________________ 116 Annex I. Evaluation mission schedule___________________________________ 121 Annex J. Persons and ‘counselling groups’ met ___________________________ 123 Annex K. Bibliography _______________________________________________ 127 Abbreviations ________________________________________________________ 134 iii Evaluation of Bringing Nutrition to Scale in Iringa, Mbeya and Njombe Tables Table 1 BNTS and ASRP key results, outcomes and outputs................................................. 6 Table 2 Project Regions and their districts......................................................................... 9 Table 3 Other nutrition stakeholders in MINS .................................................................. 34 Table 4 Recommendations ........................................................................................... 50 Table 5 Stakeholder analysis ......................................................................................... 86 Table 6 Nutrition scorecard: Mbeya Region: Q3, 2017 ...................................................... 89 Table 7 Nutrition scorecard: Songwe Region: Q2, 2017..................................................... 90 Table 8 BNA IYCF determinants, indicators and calculation methods ................................... 92 Table 9 BNA: IYCF: data by Region................................................................................ 93 Table 10 BNA: SAM treatment determinants, indicators and calculation methods .................... 94 Table 11 BNA: SAM treatment of children: data by Region .................................................. 95 Table 12 BNA: VAS determinants, indicators and calculation methods ................................... 96 Table 13 BNA: VAS: data by Region................................................................................. 97 Table 14 BNA: IFAS determinants, indicators and calculation methods .................................. 98 Table 15 BNA: IFAS: data by Region ................................................................................ 99 Table 16 BNTS results matrix, fourth annual report: May 2017 .......................................... 102 Table 17 BNTS results matrix, quarterly update 6: November 2017 .................................... 110 Table 18 Evaluation mission schedule ............................................................................ 121 Table 19 List of officials met for interviews or group meetings ........................................... 123 Table 20 Meetings with ‘counselling groups’ .................................................................... 126 Figures Figure 1. Tanzania scorecard: SUN MEAL assessment ................................................................. 3 Figure 2. BNTS impact pathways ............................................................................................ 62 Figure 3. ASRP logic model ................................................................................................... 63 Figure 4. Example of stunting data recorded at a Village Health Day ......................................... 100 Figure 5. Example of stunting data recorded at a Village Health Day ......................................... 101 Maps Map 1. Regional Map of Tanzania 116 Map 2. Njombe Region 117 Map 3. Mbeya Region 118 Map 4. Iringa Region 119 Map 5. Songwe Region 120 iv Evaluation of Bringing Nutrition to Scale in Iringa, Mbeya and Njombe Summary Introduction 1. UNICEF has commissioned an evaluation of the Bringing Nutrition to Scale in Iringa, Mbeya and Njombe Regions (BNTS) project, covering the period from its launch in 2013 to the time of the evaluation mission in November 2017. The project is funded by Irish Aid (IA). The BNTS project has, for operational purposes, been merged with the Accelerated Stunting Reduction Project (ASRP), funded by the United Kingdom Department for International Development (DFID) through the Addressing Stunting in Tanzania Early (ASTUTE) project. 2. This exercise has served as a mid-term evaluation of the ASRP, which recently completed its second year of operations. Following approval of a matrix of 16 evaluation questions submitted in an inception report, it has reviewed progress across all four Regions in which ASRP is being implemented: Mbeya, Iringa, Njombe and Songwe (MINS). In answering the 16 questions, the evaluation is intended to be formative, with a view to generating evidence and lessons to strengthen programme design and accelerate the achievement of results. It reports on the ASRP as a whole, with specific reference to the BNTS where appropriate. It is based on detailed assessment of available documentation and a two-week evaluation mission in November 2017, which included visits to communities and Local Government Authorities (LGAs). 3. The ASRP has a broad scope of ambition ranging from national policy and systems to support at household level. It has made equally broad contributions, which are mainstreamed in overall national efforts to combat stunting – and has the potential, subject to available resources, for further creative support across the spectrum of those national efforts. The evaluation therefore offers equally broad observations on the progress that those national efforts are making, although its focus is on the performance of the ASRP, and its recommendations specifically concern the project itself. 4. Levels of malnutrition have declined significantly in Tanzania since the 1960s, but not far enough. In MINS, some child nutrition indicators remain poor, despite the progress in commercial agriculture that is being achieved in some areas. Stunting is a particular concern, with MINS among the Regions that are furthest above the national average. Tanzania’s National Multisectoral Nutrition Action Plan (NMNAP) has continued the emphasis of the Government of Tanzania (GOT) on