©UNICEF/UN056405/Thuentap THAILAND CHILD SUPPORT GRANT CONTACT US Facebook: facebook.com/unicefthailand Twitter: twitter.com/unicef_thailand Impact Assessment Endline Report UNICEF Thailand IG: @UNICEF_Thailand (CSG) 19 Phra Atit Road LINE: UNICEF Thailand Pranakorn, Bangkok 10200 Youtube: youtube.com/unicefthailand 9 June 2019 Thailand Website: www.unicef.or.th Telephone: +66 2 356 9499 To donate Fax: +66 2 281 6032 Phone: +66 2 356 9299 Email: [email protected] Fax: +66 2 356 9229 Email: [email protected] Thailand Child Support Grant (CSG) Impact Assessment Endline Report Table of Contents 1 Overview 1 1.1 Introduction 1 2 Object of Evaluation 3 2.1 Scale and Complexity 3 2.2 Logical Model and Theory of change 5 2.2.1 Purpose and Goal of the Child Support Grant 5 2.2.2 Evaluation Theory of Change as a Basis for Evaluation Design 5 2.3 Key Stakeholders 11 3 Evaluation Purpose, Objectives, and Scope 13 3.1 Purpose of the Evaluation 13 3.2 Evaluation Objectives 13 3.3 The scope of the Evaluation 14 3.4 Objectives of the Endline Report 15 4 Evaluation Methodology 17 4.1 Quantitative Methods for Impact Assessment 17 4.2 Evaluation Framework 19 4.3 Sampling Strategy 21 4.4 Ethical Considerations 22 4.5 Data Collection 23 4.5.1 Quantitative Survey Instruments 24 4.5.2 Qualitative Survey Instruments 24 4.6 Data Analysis 26 4.6.1 Quantitative Data Analysis 26 4.6.2 Qualitative Data Analysis 28 5 Endline Findings 31 5.1 Impact 31 5.1.1 Increase in Household Income and Expenditure 31 5.1.2 Improved Dietary Diversity and Food Intake 34 Thailand Child Support Grant (CSG) Impact Assessment Endline Report 5.1.3 Increased Take-up of Health Care Services 39 5.1.4 Increased Time Allocated to Children 40 5.1.5 Improved Developmental Outcomes for Children 42 5.1.6 Spill-over effects 43 5.1.7 Women’s Empowerment 44 5.2 Efficiency 46 5.2.1 Complexity of targeting criteria 47 5.2.2 Lack of training and information 47 5.2.3 Unclear Division of Labor 48 5.2.4 Other Reasons 49 5.2.5 Quantitative targeting assessment 50 5.2.6 Ratchet Survey Findings 50 5.2.7 Inclusion Error 50 5.3 Effectiveness 52 5.3.1 Appropriateness of the Benefit Value 52 5.3.2 Effectiveness of Implementation 53 6 Conclusions 67 6.1 Impact 67 6.1.1 Household Expenditure 68 6.1.2 Nutrition and Access to Essential Services 68 6.1.3 Time allocation for children 68 6.1.4 Spill-over effects 68 6.1.5 Women’s Empowerment 69 6.2 Efficiency 69 6.2.1 Targeting Efficiency 69 6.3 Effectiveness 70 6.3.1 Appropriateness of the Benefit Value 70 6.3.2 Awareness and Knowledge 71 6.3.3 Application and Enrolment 71 6.3.4 Payments 72 6.4 Relevance 72 6.5 Sustainability 73 Thailand Child Support Grant (CSG) Impact Assessment Endline Report 7 Recommendations 75 7.1 Efficiency 75 7.2 Effectiveness 76 7.3 Models for Evidence Building 77 8 Bibliography 78 Annexure 80 A. Means-Test/Eligibility Criteria for the CSG 80 B. Evaluation Questions 81 C. Integration of qualitative and qualitative components 84 D. Sampling and Power Analysis – A Technical Note 85 D1 Power Analysis 85 D2 Sampling Strategy 88 E. Ethical Review Process 89 E1 EPRI Institutional Review Board 89 E2 TDRI Ethical Review Board 91 F. Quantitative Survey Instruments 93 F1 Household Questionnaire 93 F2 Caregiver Questionnaire 116 F3 Child Questionnaire 123 G. Qualitative Survey Instruments 140 H. Quantitative Sample and Data Collection 150 I. Qualitative Sample and Data Collection timeline 151 J. Technical Annex I 152 K. Technical Annex II 159 L. Technical Annex III 164 M. Technical Annex IV 177 N. Technical Annex V 193 Thailand Child Support Grant (CSG) Impact Assessment Endline Report List of Tables Table 1: Summary of Data Collection Tools Used and their Purpose 23 Table 2: Abbreviations for Qualitative Data Source 29 Table 3: Impact on food share of total expenditure 32 Table 4: Impact on breastfeeding practices 35 Table 5: Impact on proportion of eligible children achieving minimum meal frequency 37 Table 6: Impact on eligible children’s number of post-natal care visits 38 Table 7: Impact on number of child development activities 39 Table 8: Impact on weight-for-height (wasting) 41 Table 9: Women’s decision-making power on food expenses within the household 42 Table 10: Women’s decision-making power on children’s health care 44 Table 11: Exclusion Error by Poverty Threshold 45 Table 12: Source of Awareness about the CSG 50 Table 13: Application and Eligibility Rates 59 Table 14: Reasons for Not Applying (% of respondents who were aware of the CSG) 60 Table 15: Key Research Questions 61 Table 16: Key Research Questions 81 Table 17: HAZ 86 Table 18: WAZ 87 Table 19: WHZ 87 Table 20: Endline data collection rollout 150 Table 21: Characteristics of FGD participants 151 Table 22: Characteristics of KII participants 151 Table 23: Qualitative Data Collection Schedule for Endline 151 Table 24: Variables used in propensity score matching, by model and income group sample 152 Table 25: Baseline characteristics by Child Support Grant Receipt 157 Table 26: Propensity Score Matching Probit Model Results 159 Table 27: Prevalence of wasting, expanded estimation results 164 Table 28: Proportion of eligible children breastfed for at least six months, expanded estimation results 165 Table 29: Caregivers’ knowledge of best feeding practices, estimation results 166 Table 30: Use of infant formula, estimation results 166 Table 31: Frequency of use of infant formula, estimation results 167 Thailand Child Support Grant (CSG) Impact Assessment Endline Report Table 32: Impact of minimum meal frequency, expanded estimation results 168 Table 33: Impact on food share of total expenditure, expanded estimation results 169 Table 34: Number of post-natal care visits received by eligible children, expanded estimation results 170 Table 35: Impact on number of child development activities, expanded results 171 Table 36: Impact on share of children owning at least three books, expanded estimation results 172 Table 37: Caregivers’ decision-making on food expenses, expanded estimation results 173 Table 38: Caregivers’ decision-making on own health care, estimation results 174 Table 39: Caregivers’ decision-making on the use of his/her own money, estimation results 174 Table 40: Caregivers’ decision-making on children’s health care, expanded estimation results 175 Table 41: Siblings’ exclusion from early childhood education, expanded estimation results 176 Table 42: Primary model balance test - Prevalence of wasting in households with income below THB 1500 177 Table 43: Primary model balance test - Prevalence of wasting in households with income below THB 3000 178 Table 44: Primary model balance test - Prevalence of wasting in households with income below THB 6000 179 Table 45: Primary model balance test - Prevalence of wasting in all households 180 Table 46: Primary model balance test - Proportion of eligible children breastfed for six months in households with income below THB 1500 181 Table 47: Primary model balance test - Proportion of eligible children breastfed for six months in households with income below THB 3000 182 Table 48: Primary model balance test - Proportion of eligible children breastfed for six months in households with income below THB 6000 183 Table 49: Primary model balance test - Proportion of eligible children breastfed for six months in all households 184 Table 50: Primary model balance test - Number of post-natal care visits received by eligible children in households with income below THB 1500 185 Table 51: Primary model balance test - Number of post-natal care visits received by eligible children in households with income below THB 3000 186 Table 52: Primary model balance test - Number of post-natal care visits received by eligible children in households with income below THB 6000 187 Table 53: Primary model balance test - Number of post-natal care visits received by eligible children in all households 188 Table 54: Primary model balance test - Caregivers’ decision-making on food expenses in households with income below THB 1500 189 Table 55: Primary model balance test - Caregivers’ decision-making on food expenses in households with income below THB 3000 190 Table 56: Primary model balance test - Caregivers’ decision-making on food expenses in households with income below THB 6000 191 Table 57: Primary model balance test - Caregivers’ decision-making on food expenses in all households 192 Thailand Child Support Grant (CSG) Impact Assessment Endline Report Table 58: Impact of food share of total expenditure, nearest-neighbour estimator 193 Table 59: Impact on weight-for-height (wasting), nearest-neighbour estimator 193 Table 60: Impact on breastfeeding practices, nearest-neighbour estimator 193 Table 61: Impact of caregivers’ knowledge of best feeding practices, nearest-neighbour estimator 194 Table 62: Impact on eligible child's number of postnatal care visits, nearest-neighbour estimator 194 Table 63: Impact on use of infant of formula, nearest-neighbour estimator 194 Table 64: Impact on frequency of use of infant formula, nearest-neighbour estimator 195 Table 65: Impact on minimum meal frequency, nearest-neighbour estimator 195 Table 66: Impact on number of child development activities, nearest-neighbour estimator 195 Table 67: Impact on share of children owning at least three books, nearest-neighbour estimator 196 Table 68: Impact on women's decision-making on food expenses, nearest-neighbour estimator 196 Table 69: Impact on women’s decision-making power on children’s health care, nearest-neighbour estimator 196 Table 70: Impact