Mcgovern-Dole International Food for Education and Child Nutrition Program Beoog Biiga II Midline Performance Evaluation Report

McGovern-Dole International Food for Education and Child Nutrition Program Beoog Biiga II Midline Performance Evaluation Report

July 2017

Authors: Elnaz Safarha, M.S. Sarah Pedersen, M.P.P. Edoxi Kindané, M.S. Paula Mian, M.P.P

Submitted to: Submitted by:

Chief of Party: Mourad Aidi Project Director: Elnaz Safarha Catholic Relief Services – Burkina Faso IMPAQ International, LLC ACKNOWLEDGEMENT

The authors would like to thank Catholic Relief Services (CRS) and the U.S. Department of Agriculture (USDA) for their financial support. The authors also would like to thank the Ministry of Education of Burkina Faso, its provincial offices in the central north region, and CRS staff for supporting and facilitating the rollout of the midline data collection. We have very much enjoyed our collaboration with CRS and would like to thank Mourad Aidi, Abdoulaye Bikienga, Myriam Dems, and Hamadou Kanazoe from CRS. The authors acknowledge Sara Borelli, Lucy Cutting, and Rachael Ward, researchers from IMPAQ International. Special thanks go to Aude Diarra for her technical insights and support on classroom observations. We also thank Zakaria Ourdraogo for his support in the field work. The authors thank the Action, Gouvernance, Intégration, Renforcement team for its professionalism and flexibility with data collection, including Paul-André Somé, Ahmed Barro, and all the enumerators for their excellent work in the field. The findings, interpretations, and conclusions expressed herein are entirely those of the authors. They do not necessarily represent the views of IMPAQ, CRS, or USDA.

IMPAQ International, LLC Page i Beoog Biiga II Midline Report July 2017 TABLE OF CONTENTS

ACKNOWLEDGEMENT ...... I TABLE OF CONTENTS ...... II TABLE OF EXHIBITS ...... VI ACRONYM LIST ...... VIII EXECUTIVE SUMMARY ...... IX Outcomes for Key Indicators ...... x Additional Key Findings ...... xii Recommendations for the Project ...... xiii SECTION 1. INTRODUCTION ...... 1 1.1 Program Background ...... 1 1.2 Evaluation Background ...... 3 SECTION 2. EVALUATION METHODOLOGY ...... 4 2.1 Quantitative Approach ...... 4 2.1.1 Quantitative Research Questions ...... 4 2.1.2 Quantitative Sampling Design ...... 7 2.1.3 Quantitative Data Sources ...... 8 2.2 Qualitative Approach ...... 10 2.2.1 Qualitative Research Questions ...... 11 2.2.2 Qualitative Data Sources ...... 14 SECTION 3. FIELD WORK AND ANALYSIS ...... 16 3.1 Quantitative Fieldwork and Analysis ...... 16 3.1.1 Quantitative Data Collection...... 16 3.1.2 Quantitative Analysis ...... 17 3.2 Qualitative Fieldwork and Analysis ...... 17 3.2.1 Qualitative Data Collection ...... 17 3.2.2 Qualitative Analysis ...... 18 SECTION 4. EVALUATION SAMPLES ...... 19 4.1 Students ...... 19 4.2 Parents ...... 20 4.3. Household Environment ...... 21

IMPAQ International, LLC Page ii Beoog Biiga II Midline Report July 2017 4.3.1 Household Composition ...... 21 4.3.2 Household Access to Basic Services ...... 21 4.4 Teachers ...... 22 4.5 Classroom Observations ...... 23 4.6 Parent-Teacher Associations ...... 24 4.7 School District Administrators ...... 24 SECTION 5. QUANTITATIVE OUTCOMES ...... 25 5.1 School Outcomes ...... 28 5.2 Student Outcomes ...... 28 5.2.1 Nutrition Knowledge ...... 29 5.2.2 Hygiene Knowledge and Practices ...... 30 5.2.3 Food Security ...... 32 5.2.4 Student Attendance ...... 34 5.2.5 Literacy Activities ...... 34 5.2.6 Student Reading Outcomes ...... 36 5.3 Parent Outcomes ...... 38 5.3.1 Nutrition knowledge ...... 38 5.3.2 Attitude towards Girls’ Education ...... 39 5.3.3 Parents’ Engagement in School ...... 40 5.4 Teacher Outcomes ...... 42 5.4.1 Nutrition Knowledge ...... 42 5.4.2 Attendance ...... 43 5.4.3 Classroom Practices and Teaching Techniques ...... 44 5.4.4 Classroom Observations ...... 48 5.4.5 Attentiveness ...... 53 5.5 PTA Outcomes ...... 53 5.6 School District Administrator Outcomes ...... 55 SECTION 6. QUALITATIVE FINDINGS ...... 57 6.1 Relevance ...... 58 6.1.1 Strengths and Weaknesses of Project Design...... 58 6.1.2 Alignment with Economic, Cultural, and Political Contexts ...... 60

IMPAQ International, LLC Page iii Beoog Biiga II Midline Report July 2017 6.1.3 Alignment with Burkina Faso and U.S. Government Priorities ...... 60 6.1.4 Stakeholder Satisfaction ...... 60 6.2 Effectiveness ...... 61 6.2.1. Achievement of Objectives ...... 61 6.2.2 Successes and Challenges of Program Implementation ...... 66 6.2.3 Internal Collaboration and Management ...... 67 6.2.4 Collaboration with Project Stakeholders ...... 68 6.3 Efficiency ...... 68 6.3.1 Efficiency of Use of Project Resources ...... 68 6.3.2 Achievement of Project Timeline ...... 68 6.3.3 Response to Factors That Could Hinder Implementation ...... 69 6.4 Perceived Impact ...... 69 6.4.1 Perceived Impact of Program ...... 70 6.4.2 Activities with the Greatest and Least Impact ...... 70 6.4.3 Respondents’ Recommended Strategies to Increase Impact ...... 71 6.5 Sustainability ...... 72 6.5.1 Sustainability Factors ...... 72 6.5.2 Current Sustainability Activities ...... 72 6.5.3 Respondents’ Recommended Strategies to Sustain Activities ...... 73 SECTION 7. CONCLUSION ...... 74 7.1 Key Findings ...... 74 7.1.1 Students’ Outcomes ...... 74 7.1.2 Parents and Parent-Teacher Associations’ (PTA) Outcomes ...... 75 7.1.3 Teachers’ Outcomes ...... 75 7.1.4 Qualitative Findings ...... 76 7.2 Limitations ...... 76 7.3 Recommendations ...... 77 7.3.1 Recommendations for the Project ...... 77 7.3.2 Recommendations for the Evaluation ...... 79 7.3.3 Recommendations for the Evaluation with Budget Implications ...... 80 REFERENCE LIST ...... 81

IMPAQ International, LLC Page iv Beoog Biiga II Midline Report July 2017 APPENDICES ...... 82 APPENDIX A. BBII KEY PERFORMANCE INDICATORS ...... 83 APPENDIX B. BBII KEY PERFORMANCE INDICATORS ...... 89 APPENDIX C. ADDITIONAL TABLES AND COMPLEMENTARY OUTCOMES ...... 97 APPENDIX D. ASER-READING TEST ...... 102 APPENDIX E. ASER-READING TEST INSTRUCTIONS ...... 104 APPENDIX F. QUESTIONNAIRES ...... 106 APPENDIX G. PERFORMANCE EVALUATION SCHOOLS ...... 189

IMPAQ International, LLC Page v Beoog Biiga II Midline Report July 2017 TABLE OF EXHIBITS

Exhibit 1. Bam and Sanmatenga Provinces ...... 2 Exhibit 2. Quantitative Research Questions ...... 4 Exhibit 3. Performance Evaluation Sample ...... 7 Exhibit 4. ASER-Reading Test Structure ...... 9 Exhibit 5. Qualitative Research Domain Descriptions ...... 10 Exhibit 6. Qualitative Research Questions ...... 11 Exhibit 7. Respondent Types and Relevant Research Domains ...... 14 Exhibit 8. Sample Distribution by Province and Type of Respondent ...... 19 Exhibit 9. Sampled Student Gender Distribution by Grade ...... 20 Exhibit 10. Parents’ Educational Attainment ...... 20 Exhibit 11. Household Size ...... 21 Exhibit 12. Household Access to Basic Services ...... 21 Exhibit 13. Classroom Size...... 22 Exhibit 14. Teachers’ Educational Attainment ...... 23 Exhibit 15. Teacher Training ...... 23 Exhibit 16. Overview of Baseline and Midline Levels of Key Performance Indicators ...... 26 Exhibit 17. Student Nutrition Knowledge ...... 29 Exhibit 18. Student Hygiene Practices ...... 30 Exhibit 19. Students’ Knowledge of Handwashing versus ...... 31 Exhibit 20. Students’ Food Intake ...... 33 Exhibit 21. Minimum Acceptable Diet ...... 33 Exhibit 22. Student Attendance ...... 34 Exhibit 23. Student Reported Classroom Activities ...... 35 Exhibit 24. Student Reading Groups ...... 36 Exhibit 25. Distribution of Reading Skills for Second Grade Students ...... 37 Exhibit 26. Reading Proficiency at Second Grade Level ...... 37 Exhibit 27. Parent Nutrition Knowledge ...... 38 Exhibit 28. Parents’ Opinions of Girls’ Schooling ...... 39 Exhibit 29. Parents’ Reasons for Supporting Girls’ Schooling ...... 40 Exhibit 30. Parents’ Involvement in School and in Savings Cooperatives...... 41 Exhibit 31. Parents –Teacher Individual Meetings ...... 41 Exhibit 32. Teachers’ Nutrition Knowledge ...... 43 Exhibit 33. Teacher’s Attendance over the Past Three Months ...... 44 Exhibit 34. Utilization of Classroom Activities for the Day of Data Collection ...... 45 Exhibit 35. Teachers Using New Techniques or Tools ...... 46 Exhibit 36. Time Spent by Teachers on BBII Literacy Instruction ...... 47 Exhibit 37. Implementation of Literacy Activities ...... 47 Exhibit 38. Observed BBII Techniques Using Reading Groups ...... 49 Exhibit 39. Observed Reading Lesson Elements ...... 49 Exhibit 40. Key Elements Observed Reading/Decoding Lessons by Grade ...... 50 Exhibit 41. Other Observed Teacher Practices by Grade ...... 51 Exhibit 42. Students’ Attentiveness Reported by Teachers ...... 53 Exhibit 43. Parent-Teacher Association Meetings ...... 54 Exhibit 44. PTA Representative Involvement in the Past 12 Months ...... 54 Exhibit 45. Support for Canteens ...... 55

IMPAQ International, LLC Page vi Beoog Biiga II Midline Report July 2017 Exhibit 46. Important Reading Instructions Reported by District Administrators ...... 56 Exhibit 47. Gender Composition of Students ...... 97 Exhibit 48. School District Staff’ Educational Attainment ...... 97 Exhibit 49. Minimum Acceptable Diet Reported by Parents ...... 98 Exhibit 50. Teacher Reported Utilization of Classroom Activities for this Week ...... 98 Exhibit 51. ASER-Reading Test Results, Disaggregated by Sex (Grade 2) ...... 100 Exhibit 52. Reading Proficiency at Second Grade Level ...... 101

IMPAQ International, LLC Page vii Beoog Biiga II Midline Report July 2017 ACRONYM LIST

AGIR Action, Gouvernance, Intégration, Renforcement ASER Annual Status of Education Report BBI Beoog Biiga I BBII Beoog Biiga II BEPC Brevet d'études du Premier Cycle (Certificate of Studies of the First Cycle) CCEB Chef de Circonscription d’Education de Base (School District Administrator) CRS Catholic Relief Services FAO Food and Agriculture Organization of the United Nations FAVL Friends of African Village Libraries IMPAQ IMPAQ International, LLC MENA Ministère de l’Education Nationale et de l’Alphabétisation (Ministry of Basic Education and Literacy) OCADES Organisation Catholique pour le Développement et la Solidarité PMP Performance Monitoring Plan PTA Parent-Teacher Association RCT Randomized Control Trial SILC Savings and Internal Lending Community TOR Terms of Reference USDA United States Department of Agriculture

IMPAQ International, LLC Page viii Beoog Biiga II Midline Report July 2017 EXECUTIVE SUMMARY

This report presents the midline results of the performance evaluation of the Beoog Biiga II (BBII) McGovern-Dole International Food for Education and Child Nutrition program in the provinces of Bam and Sanmatenga in Burkina Faso. In this report, we measure changes in outcomes related to BBII core objectives, examine implementation of the BBII program thus far, and provide recommendations to strengthen the program and the evaluation. BBII is a four-year program (2014 – 2018) implemented by Catholic Relief Services (CRS) and funded by the United States Department of Agriculture (USDA). In BBII, CRS aims to improve the literacy1, health, and dietary practices2 of primary school-age children in Burkina Faso. BBII implementation activities include provision of school meals and take-home rations; distribution of deworming pills, vitamins, and minerals; health and nutrition training; provision of school supplies and materials; school district administrator training; capacity building at the local, regional, and national levels; mentorship of girls; teacher training; creation of community saving and lending groups; establishment of a standardized reading assessment; establishment of libraries; and efforts to raise awareness of the importance of education.

CRS selected IMPAQ International, LLC (IMPAQ) to conduct impact and performance evaluations of the BBII program. The performance and impact evaluations were designed in parallel to maximize comparability in the outcome indicators and findings, but they follow different timelines. The impact evaluation, which measures the causal effect of the mentoring intervention on literacy of school-aged girls in a subset of BBII program schools, spans only two data collection periods: baseline (2015) and follow up (2018). The performance evaluation spans three data collection periods: baseline (2015), midline (2017), and endline (2018). The baseline, midline, and endline rounds of the performance evaluation are structured to measure changes in outcomes over time and to inform the overall evaluation results on BBII core objectives. To accurately capture program performance over time, IMPAQ measures the same program indicators at all three data collection points.

The key research questions for performance evaluations are:

 To what extent has students’ knowledge of nutrition and dietary practices changed?  To what extent have students improved their hygiene-related practices?  To what extent has students’ knowledge of health and hygiene practices changed?  To what extent do the program interventions help address student hunger and attentiveness?  What percentage of children receiving a minimum acceptable diet has changed, if any, compared to baseline?  What is the current student attendance rate? Has the percentage of students (girls and boys) that regularly (80%) attend schools changed compared to the baseline?  What percentage of students (boys and girls) have increased their reading comprehension compared to baseline?  Have parents who are members of saving and internal lending communities used their savings on education costs?  Has the parents’ level of contribution to the school canteen changed?  To what extent has teachers’ attendance changed?  How much time per day do teachers devote to literacy instruction?

1 Strategic Objective 1: Improved Literacy of School-Age Children 2 Strategic Objective 2: Increased Use of Health and Dietary Practices

IMPAQ International, LLC Page ix Beoog Biiga II Midline Report July 2017  To what extent are teachers implementing literacy teaching techniques acquired through the project?

To answer evaluation questions and provide evidence addressing the McGovern-Dole indicators, IMPAQ conducted the following quantitative data collection activities:

. Student survey including a reading assessment (the Annual Status of Education Report, ASER3) . Parent survey . Teacher survey . Parent-teacher association (PTA) leader survey . School district administrator survey

The survey questionnaires were designed to ask questions about health, nutrition, literacy, and community involvement in schools. IMPAQ also integrated a complementary qualitative analysis by conducting key informant interviews and focus group discussions to address some of the limitations of the quantitative methods, to provide contextual understanding of the program, and to aid in interpretation of the quantitative results. The qualitative study assessed the following principles: (1) the relevance of all interventions, (2) the effectiveness of implementation strategies and activities, (3) the efficiency of the project, (4) the impact of the project, and (5) the likely sustainability of initiated actions.

In addition, the team conducted classroom observations to assess implementation and uptake of project activities and triangulated this data with the qualitative and survey findings.

This report presents the midline levels of key project indicators and measures progress in their achievement over time. At midline, IMPAQ collected data in May 2017 on more than 500 variables from 258 primary school students in grades 2 to 6, 262 parents, 105 teachers, 22 PTA members, and 30 school district administrators. The data provide interesting insights into student, parent, and teacher knowledge of nutrition and hygiene, student academic performance, and community involvement. The data also point to the need for projects like this one to improve food security, hygiene and nutrition knowledge, and student literacy. Key findings are summarized below.

Outcomes for Key Indicators

Below is a snapshot of the key outcomes for midline performance evaluation. More details are provided in Section 5, and Section 6.

Student Outcomes  The proportion of students who achieved a passing score on a food nutrition test increased by 2 percentage points from baseline to midline (statistically significant at the 5 percent level) but remained low (3 percent). Surveyed data reported by teachers show that teachers are well- informed on knowledge necessary to increase students’ nutrition awareness. However, low

3 The Annual Status of Education Report (ASER) Center pioneered in 2005 a nationwide survey, composed of a reading and math test, to measure the achievements of children in primary school in rural India in reading and math. Since their inception, adapted versions of the tests have been implemented in Pakistan, Kenya, Mali, Senegal, Tanzania, Uganda, Ghana, and now Burkina Faso (Pratham 2015; USAID 2012).

IMPAQ International, LLC Page x Beoog Biiga II Midline Report July 2017 nutrition knowledge of students may suggest that teachers need support on how to teach it effectively.  There was a slight (4 percentage point) increase in the proportion of children who reported washing their hands at critical moments compared to baseline although the change was not statistically significant. The proportion of children who achieved a passing score on a test of good health and hygiene practices remained relatively low with no significant change at midline (4 percent). The lack of significant changes from baseline to midline could be explained by the fact that some schools lack reliable water sources and handwashing devices sometimes fall into disrepair, as reported by school-level interviewees.  At midline, the proportion of children who reported that they were hungry during the school day (after eating breakfast before going to school) remained low (7 percent) with only a slight increase (3 percentage points) from baseline. However, the change was not statistically significant. Although this increase is trivial, it could be explained by fewer children (4 percentage points) eating before going to school (not statistically significant at any level) at midline. Students most commonly cited insufficient food (81 percent) as the reason why they could have eaten more after breakfast before going to school at midline.  The percentage of children who consumed a minimum acceptable diet increased significantly from baseline to midline. Students reported an 11 percentage point increase, and parents reported a 20 percentage point increase (statistically significant at the 1 percent level). The increase in consumption of a minimum acceptable diet was slightly higher for boys than for girls. This increase in the diversity of household’s diet might be affected by potential different socioeconomic backgrounds between baseline and midline samples.  Almost all students attended school regularly (at least 80 percent of the time). However, there was a slight decrease in children’s regular attendance from baseline to midline (by 1 percentage point); the result was marginally statistically significant (Statistically significant at the 10 percent level) at conventional levels. Boys’ attendance decreased by 2 percentage points at midline, a slightly greater decrease than in girls’ attendance. The fact that student attendance data were collected from teachers who might not record attendance consistently could confound measurement of the attendance rate.  The proportion of second-graders with reading proficiency at grade level increased by 9 percentage points from baseline to midline. The improvement in reading proficiency was higher for girls (15 percentage points) than for boys (4 percentage points). However, the increase was not statistically significant at conventional levels for either group.

Parent and Parent-Teacher Association Outcomes  At midline, parents’ participation in community savings groups established as part of the BBII program increased by 32 percentage points compared to baseline (statistically significant at the 1 percent level). Although more parents reported at midline that they used their savings to cover school expenses, fewer parents said that the savings helped “a lot” with school expenses, with a significant 33 percentage point decrease compared to baseline. On the other hand, the proportion of those who said that the savings helped “some” increased by 38 percentage points. Both findings are statistically significant at the 5 percent level. A possibility of increase in education expenses or a decrease in parents’ purchasing power to afford school costs might be potential explanations to this shift in parents’ responses.

IMPAQ International, LLC Page xi Beoog Biiga II Midline Report July 2017  The total number of months in which the canteen was covered by MENA, CRS, parents, and other responsible parties decreased by 2 months. This finding is statistically significant at the 10 percent level at midline. This outcome should be interpreted with caution, because there is a possibility that canteen operations were misrepresented at baseline Teacher Outcomes  The proportion of teachers who attended at least 90 percent of school days did not change significantly from baseline to midline, increasing only slightly by 3 percentage points.  Teachers in target schools employed new teaching techniques at a lower rate: 44 percent at midline compared to 53 percent at baseline. This slight, statistically insignificant decrease should be interpreted with caution, because the training component of the project had not started at baseline. More than half of the surveyed teachers at baseline reporting using new techniques may suggest that teachers were already aware of these techniques even in the absence of the training. It is also important to note that teachers at midline were less educated compared to teachers at baseline. As discussed in Section 6.2, teachers with limited education found the training materials more difficult to understand, which may have led them to continue using traditional approaches— approaches they were more accustomed to using.  A larger proportion of teachers at midline than at baseline reported that they devoted at least 45 minutes a day to literacy instruction, specifically reading instruction (including, for example, phonological awareness, phonetics, and vocabulary). However, this increase (11 percentage points) was not statistically significant at any conventional levels.  Teachers identified a slight increase (3 percentage points) in the proportion of students who were attentive during class from baseline to midline. This change was not statistically significant at conventional levels. However, given the difficulty and subjectivity of measuring attentiveness, this outcome should be interpreted with caution.

Additional Key Findings

Classroom observation outcomes were more or less in line with the results of student and teacher surveys. In observed lessons, teachers mostly focused on reading skills, especially decoding.

Below is a summary of the main findings by the qualitative research domain.

IMPAQ International, LLC Page xii Beoog Biiga II Midline Report July 2017 Summary of Main Qualitative Findings Effectiveness Respondents reported that immediate outcomes were reached in most dimensions of the project and that the interventions have contributed to expected results. Teachers reported that they have observed improvements in students’ literacy skills and motivation to learn. Teachers and students also reported that hygiene education has changed students’ behavior, as they are now washing their hands more frequently. Students appreciated the school meals and take-home rations received from the canteens, and teachers believed school attendance was closely tied to the presence of canteens. Though challenges exist, respondents said that CRS has effectively managed partners and that the program has coordinated and collaborated effectively with other stakeholders. Efficiency Respondents reported that the project resources have been sufficient to implement scheduled activities and, for the most part, objectives have been achieved on time. Some activities, including the provision of food in canteens and the provision of training materials, have been delayed due to slow administrative processes. Further, respondents noted that partners’ competing agendas and priorities sometimes exacerbate these delays. Despite challenges, respondents reported that CRS has adequately responded to internal and external factors that have hindered the efficient implementation of project activities. Perceived Impact Respondents believed that the project is on target to meet its medium- and long-term goals, particularly those related to students’ literacy outcomes, and that the effects are due to the project interventions. As for hygiene-related outcomes, respondents noticed positive changes at some schools related to students’ hygiene habits. Respondents believed that, if delivered properly, the educational activities related to hygiene and nutrition will strengthen students’ skills and knowledge. However, respondents described that limitations such as teachers’ lack of motivation to teach hygiene and nutrition knowledge and the lack of clean water sources and latrines may result in less significant outcomes. Respondents recommended several strategies to improve the impact of the project. Recommendations related to literacy included modifications to the teacher trainings to ensure that they are long enough to cover all content and allow sufficient time for practice, development of a motivational framework for teachers that focuses on incentives to implement new literacy techniques, and provision of sufficient materials and resources to enable teachers and students to implement these techniques. Recommendations related to health activities included engaging more teachers in health trainings to further promote the spread of information and ensuring that infrastructure, such as handwashing stations, are operational. Sustainability Respondents believed that the engagement of communities and the government are key factors to sustainability and that CRS and its partners have successfully engaged these parties. There is still work to be done, however, to ensure project activities can be sustained after USDA funding ends. Respondents reported a need to organize a workshop with all relevant players to develop sustainability strategies and define how all parties can play a role in sustainability.

Recommendations for the Project

Below is a summary of our recommendations to CRS based on both lessons learned from our experience in the field and our findings after analyzing the collected data. The recommendations are grouped by category. A complete list of recommendations is provided in Section 7.

IMPAQ International, LLC Page xiii Beoog Biiga II Midline Report July 2017 Project Management and Sustainability  Reevaluate how project activities are planned and communicated to ensure that relevant partners are available to conduct activities. Some project implementers noted that there have been instances in which project activities have been delayed because a partner is simultaneously rolling out other activities and is therefore unavailable.  Reevaluate staffing plan to ensure enough capacity to implement and monitor projects effectively and efficiently. Some implementers reported that the limited size of their staff sometimes impacted their ability to roll out field activities and effectively monitor their activities. When developing activity roll-out and monitoring plans, implementers should take into account the realities of staffing and resource limitations.  Provide additional training to parents on use of student performance tools and to SILC groups on management. Respondents described that students’ parents experience challenges with the use of performance tools distributed by teachers through a pilot activity at five of the BBII schools. Though these parents received training to read and use these tools to support their children, respondents explained that many of these parents still do not know how to interpret the tools. If CRS plans to sustain and/or expand this activity to other schools, they should consider enhancing the training to ensure that parents receive the support they need to effectively interpret and use the tools. Respondents also reported that SILC groups often struggle with management of group activities and require further support to run the groups effectively. Implementers should reevaluate their community sensitization and support structure to ensure that stakeholders are receiving the support and guidance they need to effectively initiate and manage project activities.  Reevaluate the sustainability strategy and develop a robust plan to engage all stakeholders. Respondents believed that the engagement of communities and the government are key factors to sustainability and that CRS and its partners have successfully engaged these parties. However, there is still work to be done to ensure that project activities can be sustained after USDA funding. CRS should continue engaging stakeholders and organize additional workshops with all relevant players to develop sustainability strategies and define how all parties can play a role in sustainability.

Literacy Activities  Explore methods to increase teacher motivation and engagement. Interview respondents reported that teacher motivation is lacking in many schools. In order to motivate teachers to implement new teaching techniques, stakeholders could consider developing a framework in which teachers are incentivized for using the new techniques and tools in their classrooms. For example, during observation visits, school district administrators should continue evaluating teachers’ use of the techniques using a standardized evaluation process and assessing the level of students’ literacy skills using a standardized test. Teachers who excel could be offered awards for their performance. Further, some respondents reported that some teachers have refused to participate in CRS trainings because they believe the per diem payment for training is inadequate to cover all of their expenses. Implementers could consider distributing a short survey to obtain feedback directly from teachers about factors that motivate or dissuade teachers from participating in trainings, and use this feedback to modify practices and policies.  Consult with teachers to reevaluate length, timing, and content of teacher training. Interview respondents reported that trainings are too dense, causing teachers to be overwhelmed by the amount of information. Implementers may want to consider pilot testing a different training approach, in which teachers participate in two or three mini-training sessions that focus on different topics or techniques so as not to burden teachers with information overload. Teachers

IMPAQ International, LLC Page xiv Beoog Biiga II Midline Report July 2017 desired more time to practice the literacy techniques with their peers during trainings, and felt the trainings were lecture heavy. Some teachers also reported that they received literacy training late in the school year, giving them little time to apply the new approaches in their classrooms. Further, interview respondents noted that some teachers have a difficult time understanding the training materials and therefore are not able to successfully implement the new techniques and tools in their classrooms. Implementers should obtain teachers’ input to reevaluate the length, timing, and content of teacher training to ensure that the trainings meet teachers’ needs.  Collaborate with MENA to ensure that teacher observations are fully meeting their objectives. For example, more real-time, hands-on support to teachers is needed through the use of more frequent student testing and summarized feedback. Teachers reported that they appreciated the feedback they received from administrators during observations but desired more detailed, data- driven feedback. Implementers could consider implementing the ASER test more frequently and use the data to inform action plans with teachers that focus on ways to use the new teaching techniques and tools and support to teachers who face challenges. Implementers could consider having school-level administrators play a larger role in supporting teachers on a day-to-day basis. Further, the quantitative findings showed that teachers’ use of new techniques decreased by 9 percentage points over time, though the finding was not statistically significant. Implementers may want to use the observations as an opportunity to understand why teachers are not using the techniques, and how trainings and observations can be strengthened to better support teachers. Additionally, observation visits must occur early in the school year, as opposed to toward the end of the year, to give teachers adequate time to adjust based on the received feedback.  Reassess material distribution plan and further investigate why teachers are reluctant to use some tools. Teachers reported that they often did not have enough supplies to effectively use them in their classrooms. Implementers should ensure that teachers have more adequate quantities of materials to properly implement techniques. For example, teachers should receive one slate for every 5-7 students. Further, none of the teachers in classroom observations used Bananagrams, and word strips, and a very few of them used wood cubes, or phonemic tables as part of their lessons. In interviews, teachers reported only using large slates and blackboards. During trainings and observations by school district administrators, implementers may want to explore if and why teachers may be reluctant to use some tools and how trainings and observations can be strengthened to support teachers’ use of these materials.  Investigate why teachers’ use of new teaching techniques has decreased over time. At midline, teachers were less likely than at baseline to apply or plan to apply at least five of the seven teaching techniques on the day of data collection. Although the difference is not statistically significant, CRS might want to investigate why this may be the case and adjust trainings and teacher support accordingly.  Reconsider design of reading clubs with a greater focus on local context. Interview respondents described several challenges with the reading club design, including the difficulty to find qualified volunteers to facilitate groups and the fact that many parents require that their children help with domestic activities rather than attend reading club meetings. Implementers should reassess how the reading club component can be better tailored to the culture and stakeholders’ priorities.  Exhort school administrators to keep consistent records of teacher attendance and monitor data collection. Although the quantitative data showed high teacher attendance, the quality of the attendance data is uncertain given the sensitive nature of teacher absenteeism at schools. Some school district administrators or school administrators do not keep a record of absentees, and respondents reported varying levels of absenteeism. Given the uncertainty of the attendance

IMPAQ International, LLC Page xv Beoog Biiga II Midline Report July 2017 data, implementers should work with schools to develop more robust tracking systems and conduct spot checks to confirm that schools are collecting the data. 

Health and Dietary Activities  Work with schools to ensure that schools’ hand washing stations are in working order. Some respondents noted that hand washing stations are sometimes out of use and there is no effort made from the school nor the PTA to replace them. Implementers could develop a system whereby a designated teacher or the school-level administrator is responsible for managing the hand washing stations and reporting to CRS and its partners when it is out of order.  Further investigate why students’ hygiene and nutrition knowledge remains poor despite training. Very few children achieved a passing score on the test of hygiene knowledge and students’ level of nutrition knowledge remains very low. This area requires further probing by CRS to explore why student knowledge of good health and hygiene practices and nutrition are so low despite the training. A deeper analysis of why knowledge is so low would inform improvements to the program’s hygiene and health interventions, ideally improving students’ knowledge over time.  Investigate health issues related to dietary supplements. A few interviewed students reported that they stopped taking the micronutrients that teachers gave them because it caused them to experience nausea and/or headaches. Because the intensity of this issue is unknown, implementers should consult with students and parents to better understand why students are reacting this way to the dietary supplements and adjust as needed.

IMPAQ International, LLC Page xvi Beoog Biiga II Midline Report July 2017 SECTION 1. INTRODUCTION

Boog Biiga II (BBII) is a four-year project (2014 – 2018) implemented in the Bam and Sanmatenga provinces in Burkina Faso and funded by the United States Department of Agriculture (USDA) McGovern-Dole International Food for Education program. The project extends and expands on the previously implemented Beoog Biiga I (BBI) project. The purpose of this report is to assess the progress of BBII in achieving the desired project outcomes. The report also describes potential additional questions to be asked at endline for both the performance and impact evaluations and provides details about the implementation of BBII.

This report consists of seven sections. This introduction (Section 1) provides a brief overview of the program context for the performance evaluation. Section 2 outlines the performance evaluation methodology, including research questions, sampling design, and data tools. In Section 3, we describe field work for data collection and data analysis. Section 4 describes the sample respondents and their key characteristics. Sections 5 and 6 present the quantitative and qualitative outcomes. Section 7 concludes with lessons learned, study limitations, and recommendations.

1.1 Program Background

Catholic Relief Services (CRS), in partnership with the government of Burkina Faso and local organizations, implemented the BBI program, Tomorrow’s Child, from 2011 to 2014 in response to recurrent food crises, high levels of malnutrition, and low and unequal levels of education in the central north region of Burkina Faso. The program was funded by the International Child Nutrition and Food for Education program, which is managed by the Foreign Agricultural Service of the USDA.

BBI has been expanded to BBII, a four-year project (October 2014 – August 2018) valued at $21,601,854 million, which includes the delivery of 8,910 metric tons of commodities: soy-fortified bulgur, lentils, cornmeal, and vegetable oil. In BBII, CRS continues to provide assistance to improve the literacy, health, and dietary practices of school-age children in Burkina Faso. BBII serves all primary schools (more than 900) and community-led preschools (bissongos) in the provinces of Bam and Sanmatenga in the central north region of Burkina Faso (Exhibit 1).

The project is based on a participatory approach that reinforces the linkages between communities and schools. This approach aims to ensure program sustainability and to contribute to the achievement of the Burkina Faso education strategic plan and of the United Nations’ Millennium Development Goal objectives.

IMPAQ International, LLC Page 1 Beoog Biiga II Midline Report July 2017 Exhibit 1. Bam and Sanmatenga Provinces

For the implementation of BBII, CRS is partnering with the Ministry of Basic Education and Literacy (MENA), Organisation Catholique pour le Développement et la Solidarité (OCADES), and Friends of African Village Libraries (FAVL); it is also collaborating with the Burkina Faso Ministry of Health and Ministry of Social Action. CRS is leading the project’s implementation activities, including provision of school meals and take-home rations; distribution of deworming pills, vitamins, and minerals; health and nutrition training; provision of school supplies and materials; school district administrator training; capacity building at the local, regional, and national levels; mentorship of girls; teacher training; creation of community saving and lending groups; establishment of a standardized reading assessment; establishment of libraries; and efforts to raise awareness of the importance of education. CRS expects to reach approximately 387,000 direct beneficiaries by implementing these activities over the life of the project, distributed as follows:

 270,000 primary and preschool students (between 170,000 and 200,000 per year)  3,600 teachers  90 preschool caregivers  9,900 parent-teacher association (PTA) members, school management committee members, and cooks  51,120 savings and internal lending community (SILC) members  2,250 community mentors  20,000 community members (using libraries)  40 local government officials (i.e., county mayors and deputy mayors)  30,000 parents  80 school administrators: district-level inspectors and pedagogical advisors

IMPAQ International, LLC Page 2 Beoog Biiga II Midline Report July 2017 1.2 Evaluation Background

The performance and impact evaluations were designed in parallel to maximize comparability in the outcome indicators and findings, but they follow different timelines. The impact evaluation measures the causal effect of the mentoring intervention on literacy of school-aged girls in a subset of BBII program schools. It spans only two data collection periods: baseline (2015) and follow up (2018). The performance evaluation spans three data collection periods: baseline (2015), midline (2017), and endline (2018). The baseline, midline, and endline rounds of the performance evaluation are structured to measure changes in outcomes over time and to inform the overall evaluation results on BBII core objectives. To accurately capture program performance longitudinally, IMPAQ measures the same program indicators at all three data collection points.

This report describes the midline findings for the key performance indicators and studies their change since baseline (May–September 2015). The key performance indicators are the following:

 Percentage of students in target schools, by gender, who demonstrate that they can read and understand the meaning of grade-level text by the end of two grades of primary schooling  Percentage of teachers who devote an average of at least 45 minutes a day to literacy instruction  Percentage of teachers in target schools who attend and teach school at least 90 percent of scheduled school days each year  Percentage of school district administrators who demonstrate use of new techniques or tools as a result of USDA assistance  Percentage of teachers, educators, and teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance  Percentage of students in target schools who are identified as attentive during class  Percentage of students in target schools who indicate that they are hungry or very hungry during the school days  Percentage of students, by gender, who regularly (80 percent of the time) attend USDA-supported classrooms and schools  Percentage of school-age children, by gender, who receive a minimum acceptable diet  Percentage of students who wash their hands at critical moments  Percentage of students in target schools who achieve a passing score on a test of good health and hygiene practices  Percentage of students in target schools who achieve a passing score on a test on food nutrition and dietary practices  Number of months of community and/or government support for school canteens

To evaluate these key indicators and measure progress toward achieving them, IMPAQ collected survey data using the same instruments used at baseline. In addition, IMPAQ used qualitative key informant interviews, focus group data, and classroom observations to assess the relevance of BBII interventions; the appropriateness of BBII strategies; project management structures; and the effect of activities on project effectiveness, efficiency, impact, and sustainability. The same quantitative indicators will be collected and reported at endline, but some qualitative performance evaluation questions about lessons learned and sustainability will be different at endline. At midline, the qualitative research questions focused on learning what has occurred to date, including a formative assessment and suggestions for program process improvement. At endline, the qualitative research questions will focus on learning more about the potential for program sustainability and about promising practices and lessons learned.

IMPAQ International, LLC Page 3 Beoog Biiga II Midline Report July 2017 SECTION 2. EVALUATION METHODOLOGY

2.1 Quantitative Approach

The performance evaluation measures changes in outcomes related to BBII core activities over the life of the project. To accurately reflect program performance over time, IMPAQ will measure the program indicators outlined in Section 1, using the same methodology and sampling strategy at all three data collection points — baseline, midline, and endline — and then compare their values.

2.1.1 Quantitative Research Questions The research questions for the quantitative performance evaluation of BBII focus on literacy, health, and nutrition outcomes for various beneficiaries, including teachers, students, and PTA members, with an emphasis on improving outcomes for girls. These research questions are derived from the program objectives, as well as guidelines from the McGovern-Dole International Food for Education program.

To answer the research questions, the performance evaluation needs to develop evidence addressing the indicators described in Exhibit 2. Quantitative Research Questions. Exhibit 2 maps the strategic objectives (SO) of the CRS performance monitoring plan (PMP) to the quantitative research questions, their corresponding indicators, and the sources from which IMPAQ will collect data in accordance with the approved terms of reference (TOR). This exhibit summarizes the research questions; Appendix A4 provides a detailed table mapping all the research questions to the strategic objectives.

Exhibit 2. Quantitative Research Questions Strategic Objectives Project Performance Research Questions Data Source and Results Indicators

Percentage of students (boys and . What percent of students girls) who, by the end of two Student literacy SO1. Improved (boys and girls) have grades of primary schooling, assessment. Literacy of School-Age increased their reading demonstrate that they can read Children comprehension and understand the meaning of compared to baseline? grade-level text. a

4 In the approved terms of reference and performance monitoring plan (Appendix A) the data collection is referred to as “evaluation” when IMPAQ is collecting the data through qualitative and/or quantitative methods and as “CRS/monitoring” when CRS is responsible for collecting monitoring data on performance indicators. There are overlaps in some questions for which both IMPAQ and CRS will collect data.

IMPAQ International, LLC Page 4 Beoog Biiga II Midline Report July 2017 Strategic Objectives Project Performance Research Questions Data Source and Results Indicators

. To what extent are Number of teachers, educators, teachers implementing and teaching assistants in target literacy teaching schools who demonstrate use of techniques acquired new and quality teaching through the project? techniques or tools as a result of USDA assistance. a . How much time per day do teachers devote to Number of teachers who devote Teacher survey, literacy instruction? an average of at least 45 minutes student survey, a day to literacy instruction. a district

IR1.1. Improved administrator . To what extent teachers’ Number of teachers in target Quality of Literacy survey, and attendance has changed? schools who attend and teach Instruction classroom school at least 90% of scheduled observations. . To what extent are school school days per year. a administrators

implementing training Number of school administrators teaching techniques and and officials who demonstrate pedagogical use of at least one new accompaniment and technique, skill, or tool as a result teacher of USDA assistance. a monitoring/feedback techniques acquired through the project?

Percentage of students in target schools who indicate that they . To what extent do the are hungry or very hungry during IR1.2. Improved program interventions school daysa. Attentiveness Output Student survey, help address student 1.2.1. Reduced Short- teacher survey. hunger and Percentage of students in target Term Hunger attentiveness? schools who are identified as attentive during class/instruction by their teachers.

. What is the current student attendance rate? Number of students (boys and . Has the percent of girls) who attend USDA- students (girls and boys) supported classrooms and Student that regularly (80%) schools at least 80% of the time IR1.3. Improved attendance logs, attend schools changed (boys and girls).a Student Attendance Parent survey, compared to the PTA survey baseline? Number of group members that . Have parents who are spend money from SILC on members of saving and education costs a. internal lending communities used their

IMPAQ International, LLC Page 5 Beoog Biiga II Midline Report July 2017 Strategic Objectives Project Performance Research Questions Data Source and Results Indicators savings on education costs?

Output. 1.3.5. . Has the parents’ level of Number of months of community Increased Community contribution to the school and/or government support for PTA survey. Understanding of canteen changed? school canteens a. Benefits of Education

Number of students in target . To what extent has schools who pass a test of good students’ knowledge of health and hygiene practices by health and hygiene naming at least 4 situations in IR2.1 Improved practices changed? which they should wash their Student survey, Knowledge of Health hands a. classroom and Hygiene Practices . To what extent have observations. students improved their Percentage of schoolchildren hygiene-related who wash their hands at critical practices? moments: before eating and after using the latrine a.

Number of students in target . To what extent has schools who pass a test on students’ knowledge of nutrition and dietary practices by nutrition and dietary naming at least one food with IR 2.3 Increased practices changed? iron and one with vitamin A and Knowledge of . What percent of children by naming one benefit of each a. Student survey. Nutrition receiving a minimum acceptable diet has Percentage of school-aged changed, if any, children (boys and girls) who compared to baseline? receive a minimum acceptable diet (boys and girls). a

. To what degree has IR 2.5 Increased Percentage of students who have students’ knowledge Access to heard of vitamin A and iron, can about vitamin A, iron and Student survey. Preventative Health cite a benefit of each, and know a deworming medication Interventions food that contains each. changed since baseline?

FR 4: Output1.4.4 & . To what extent are PTAs 2.7.4 Increased holding regular meetings? Number of general assemblies Engagement of Local To what degree is this and PTA council meetings held in PTA survey. Organizations and frequency different from the past school year. Community Groups baseline?

Source: BBII Terms of Reference (TOR) and Performance Monitoring Plan (PMP); a Key performance indicators.

IMPAQ International, LLC Page 6 Beoog Biiga II Midline Report July 2017

To complete the analysis of the key research questions, we also examined some non-key indicators as follows:  Number of libraries observed during the data collection nearby schools  Percentage of observed lessons in classrooms where teachers use learning materials in their teaching sessions.  Percentage of students participating in reading clubs  Percentage of schools observed with appropriate food storage conditions.  Percentage of parents reporting educating girls improves living standards

2.1.2 Quantitative Sampling Design

At baseline, in light of budget and time constraints and with USDA’s approval, IMPAQ and CRS agreed to overlap the sample for the impact and performance evaluations and to limit the performance evaluation sample to the 22 control schools in the impact evaluation sample. The impact evaluation uses a randomized control trial (RCT) design in which 22 treatment schools (out of 44 schools) have been assigned to receive the BBII mentoring program. The performance sample excludes these treatment schools, so that the changes in outcomes examined in the performance evaluation can be associated only with the other BBII components and not the mentoring program.

Since the performance evaluation focuses only on progress toward general outcomes such as literacy, nutrition, and hygiene, IMPAQ is not concerned with tracking individual students over time. Unlike the sampling strategy for the impact evaluation, for the performance evaluation we select a new student sample of the same size at all three data collection points (baseline, midline, and endline). Exhibit 3 shows the performance evaluation sample.

Exhibit 3. Performance Evaluation Sample

Number of Total Number Individuals Category Grade 2 Grade 3 Grade 4 Grade 5 Grade 6 of Individuals in Selected per All 22 Schools School Girls 2 1 1 1 1 6 132 Boys 2 1 1 1 1 6 132 Parents 4 2 2 2 2 12 264 Teachers 1 1 1 1 1 5 110 PTA 1 1 22

As shown in Exhibit 3, for the midterm evaluation we randomly selected students in grades 2 to 65 in all 22 control schools from school lists shared by CRS. (See Appendix G for the full list of school names.) We included four second-graders (two girls and two boys) as well as one boy and one girl from each higher grade, for a total of six boys and six girls in each school. To ensure a large enough sample size, we also randomly sampled extra students (four boys and four girls for grade 2, and two girls and two boys for

5 Based on the baseline field test, IMPAQ and CRS decided to exclude first-graders, since they were emotionally and cognitively unable to answer a survey.

IMPAQ International, LLC Page 7 Beoog Biiga II Midline Report July 2017 grades 3 to 6) to serve as replacements for students who might not be in school on the day of data collection. The sample also includes parents of each of the children, including the replacements. In addition, teachers of students in grades 2 to 6 in all 22 schools, as well as one PTA representative6 in each school, were included in sample.

However, before the data collection started, CRS informed us about the possibility that some selected students would be unavailable for reasons such as school dropout or relocation to a new village. After contacting the school principals and checking the availability of selected students on our randomized list, we decided to re-randomize the problematic grades when none of the selected students from the main or the replacement lists were available. In some situations, we learned about this issue a day before the data collection. In these cases, in collaboration with CRS and school principals, we re-randomized students with a lottery process before the school visit.

2.1.3 Quantitative Data Sources

To measure progress toward the key performance indicators, outlined in Section 1, at midline, we collected and analyzed data from six sources (included in Appendix F): (1) student survey, including the Annual Status of Education Report (ASER) reading assessment; (2) parent survey; (3) teacher survey; (4) PTA representative survey; (5) classroom observations; and (6) school district administrator survey. We used the baseline survey tools, which were translated and adapted to the Burkina Faso context, and updated them with minor changes after pilot testing before the midline data collection. Using the same instruments as at baseline provides us with consistent data to measure progress toward program objectives relative to the baseline values.

Surveys Students in grades 2 to 6 completed the same questionnaire. Questions covered reading habits, attendance, health, hygiene, and dietary practices. Whenever possible, we also observed hygiene practices of students at the school to assess to what extent they had adopted good handwashing practices.7 Students’ parents also completed a survey that included the same questions as the student questionnaire regarding their knowledge of nutrition and hygiene. The parent survey also had some questions regarding parents’ involvement with the school and attitudes toward girls’ education. All teachers completed the same teacher questionnaire, regardless of the grade level they taught. Teacher questions focused on teaching practices and attendance. In addition, one PTA leader from each school completed a questionnaire with questions related to meetings held by PTA members in the past year, the functioning of the school canteen, and the extent of community support for the school.

Consistent with our baseline data collection, we collected data from district administrators or Chef de Circonscription d’Education de Base (CCEB) of the 22 sampled schools in Bam and Sanmatenga. In this survey, we collected data on the administrators’ knowledge about training teachers in classroom management methods and literacy instruction.

6 In our sample, we planned to survey the PTA leaders as the board representatives so that they could respond to questions about both school involvement and managerial activities. 7 Enumerators were instructed to survey students before their break or their lunch to have an opportunity to observe their handwashing practices. But only a few students ate or used the latrine after being surveyed, leaving us with few observations.

IMPAQ International, LLC Page 8 Beoog Biiga II Midline Report July 2017 ASER Reading Assessment At baseline, we developed, fielded, and used an adapted version of the ASER reading test8 to measure second-grade students’ reading levels. In collaboration with CRS staff, IMPAQ conducted an adaptation workshop and a pretest in 2015 to ensure that the test was culturally appropriate and consistent with Burkina Faso’s learning standards for each grade level in primary school. After the baseline data collection, CRS also used the same ASER test to collect monitoring data. Given the possibility of bias in reading outcomes — for example, students could have access to the test from older students or teachers could become aware of the test and start to prepare students for it — we updated the test content. We ensured that the updated test had the same level of complexity and difficulty as the one used at the baseline so we would be able to compare students’ reading skills between baseline and midline. We used four earlier versions previously calibrated to Burkina’s context at baseline and updated the midline ASER by borrowing some of their items rather than creating untested items. For example, at the sound level the appearance of the sounds was changed but we kept all key sounds. At the simple word level, we reshuffled some of the words and replaced the others with similar words from those validated versions. To further ensure the appropriateness and comparability of the updated test, at midline we conducted another one-day adaptation workshop; we also convened a group of local reading, curriculum, and assessment experts from the Ministry of Basic Education and Literacy (MENA), as well as the CRS educational team.

In addition, we pretested the updated ASER in two schools in Ouagadougou whose characteristics were similar to those of the rural schools in the evaluation sample. The final version of the test included 11 levels (A – K), which roughly correspond to the reading standards for each grade level (see Appendix C). Exhibit 4 presents the structure of the ASER reading test, including the test’s levels and corresponding grades and reading skills.

Exhibit 4. ASER-Reading Test Structure

Level Corresponding Grade Reading Skills Level 0 None None Level A Grade 1 (CP1) – Lower level Identify letters Level B Grade 1 (CP1) – Upper level Read simple sounds Level C Grade 2 (CP2) – Lower level Read complex sounds Level D Grade 2 (CP2) – Upper level Decode simple words (1-2 syllables) Level E Grade 3 (CE1) – Lower level Decode complex words (2-3 syllables) Level F Grade 3 (CE1) – Upper level Read simple sentences Level G Grade 4 (CE2) – Lower level Read complex sentences Level H Grade 4 (CE2) – Upper level Read simple stories Level I Grade 5 (CM1) – Lower level Answer reading comprehension questions on simple stories Level J Grade 5 (CM1) – Upper level Read complex stories Level K Grade 6 (CM2) Answer reading comprehension questions on complex stories Source: IMPAQ.

8 In 2005, the ASER Center pioneered a nationwide survey, composed of a reading and math test, to measure the achievements of children in primary school in rural India. Since inception, adapted versions of the tests have been implemented in Pakistan, Kenya, Mali, Senegal, Tanzania, Uganda, Ghana, and now Burkina Faso (Pratham 2015; USAID 2012).

IMPAQ International, LLC Page 9 Beoog Biiga II Midline Report July 2017 Classroom Observations We used BBII literacy training materials to develop a classroom observation tool. An IMPAQ education expert, in collaboration with our fieldwork assistant, tested the tool in two schools in Bamako, Mali, and two schools in Ouagadougou, Burkina Faso. We worked closely with the CRS education program manager and specialist to finalize the instrument, fully adapting it to the local context. The final version of the classroom observation tool can be found in Appendix F. The tool was designed to observe instruction in any subject, including literacy, math, and human science, in grades 2 to 6.

The instrument enabled us to observe whether trained teachers have been implementing the new teaching techniques and using the material kits provided by the program.9 It also helped us to correlate the outcomes of classroom observation with the findings from the teacher survey and teacher key informant interviews.

2.2 Qualitative Approach

Our qualitative study assesses both retrospectively and prospectively, the following principles:10 (1) the relevance of all interventions, (2) the effectiveness of implementation strategies and activities, (3) the efficiency of the project, (4) the impact of the project, and (5) the likely sustainability of initiated actions. A description of each research domain is presented below in Exhibit 5.

Exhibit 5. Qualitative Research Domain Descriptions

Research Description Domain The extent to which the project interventions are meeting the needs of the project beneficiaries and are aligned with Burkina Faso’s and the U.S. government’s Relevance development goals, objectives, strategies; the extent to which the project considers economic, cultural, and political context; and the extent to which stakeholders are satisfied with their participation in the project. The extent to which the project is achieving its objectives, the interventions Effectiveness contribute to expected results or objectives, and the program coordinates and collaborates with other stakeholders and manages partners. The extent to which project resources have led to the achieved results, the objectives Efficiency have been achieved on time, and the project has responded to any internal or external factors that have hindered the efficient implementation of project activities. The extent to which the project will achieve medium- to long-term effects, both Perceived intended and unintended, and the extent to which the effects are due to the project Impact intervention or other factors. The extent to which the project has planned for the continuation of project activities, Sustainability developed local ownership of the project, and developed sustainability partnerships

9 We initially planned to record videos in the classrooms so we could score teaching techniques and evaluate the pedagogical instructions as a back-up to our collected data. However, after practicing this approach in the pilot testing, we learned that video recording could reduce our efficiency and create distraction in class. 10 These five principles were developed by the Organisation for Economic Co-operation and Development. More information can be found at http://www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm.

IMPAQ International, LLC Page 10 Beoog Biiga II Midline Report July 2017 Research Description Domain and the extent to which factors (and which ones) will likely influence achievement or non-achievement of project sustainability. Source: BBII Terms of Reference (TOR).

2.2.1 Qualitative Research Questions The IMPAQ team focused assessment of efficiency, project effectiveness, and perceived impact on the strategic objectives and results shown in Exhibit 6. The domain descriptions in Exhibit 5 and the key questions in Exhibit 6 guided the design of the qualitative methodology and the development of interview and focus group instruments. Data sources are defined in Section 2.2.2 below.

Exhibit 6. Qualitative Research Questions Strategic Objectives Key Questions Data Source and Results

SO1. Improved Teacher and school . What factors contributed to an increase in reading Literacy of School-Age administrator comprehension? What factors inhibited improvement? Children interviews

. Have teacher literacy instruction trainings been completed as per the project timeline and budget? . To what extent are teachers implementing literacy teaching techniques acquired through the project? . Are changes in classrooms starting to be evident (e.g., teachers see improved reading abilities of students)? If Implementer, IR1.1. Improved not, why not? What needs to change? teacher, and school Quality of Literacy . What challenges do teachers face in using the new administrator Instruction literacy teaching techniques? interviews . What aspects do they find most useful and why? . In what way has the quality of education improved as a result of the adoption of technical trainings for teachers? . Where libraries are in place, to what degree do community members (including students and teachers) have access?

. What are the greatest inhibiting factors to teachers’ Output. 1.1.1. More Teacher and school attendance? Consistent Teacher administrator . What project interventions influenced the improvement Attendance interviews of teacher attendance?

. To what extent have school supplies and materials been Implementer, Output. 1.1.2. Better distributed as per the project timeline and budget? teacher, and school Access to School . What materials have been supplied? administrator Supplies & Materials . Which school supplies do teachers find most useful and interviews; student why? Which supplies and materials provided do students focus groups like and why?

IMPAQ International, LLC Page 11 Beoog Biiga II Midline Report July 2017 Strategic Objectives Key Questions Data Source and Results . What other supplies would teachers and students prefer? . Are material kits being used as intended? . Do teachers and students need additional training to better use these materials?

. To what extent have literacy instructional materials been distributed as per the project timeline and budget? . Do the teachers consider literacy instructional materials to be an improvement over what they previously had? How? . How are teachers using the materials provided? . What, if any, other materials would they prefer? Implementer, Output. 1.1.3. . What do students like and dislike about using the literacy teacher, and school Improved Literacy materials that have been provided? administrator Instructional Materials . To what extent have reading groups been established? interviews; . Who provides oversight of reading groups most often student focus groups (teachers or older students)? . Why do students join or not join reading groups? . Have students or teachers noticed any change since students have joined reading groups? (e.g., personal confidence, better grades, more social cohesion among students, etc.)

. To what extent have teachers been trained as per the project timeline and budget? . In what ways has the quality of teaching improved based Implementer, Output. 1.1.4. on the tools and techniques used by teachers? teacher, and school Increased Skills and . What aspects of the trainings were not widely adopted administrator Knowledge of and why? interviews; student Teachers . How can the trainings have greater impact? focus groups . What additional training topics would help the teachers be even more effective in literacy instruction?

. To what extent have school administrators been trained as per the project timeline and budget? . To what extent do school administrators find the Implementer, Output. 1.1.5. classroom observation technique useful? teacher, and school Increased Skills and . How is the technique received by teachers? administrator Knowledge of School . Have the observations led to constructive feedback? If so, interviews Administrators has the feedback been received by teachers and affected

their teaching techniques? . In what way has the quality of education improved based on techniques used by the administrators?

Implementer IR1.2. Improved . To what extent have school meals been distributed as per interviews, student Attentiveness Output the project’s budget and timeline? focus groups

IMPAQ International, LLC Page 12 Beoog Biiga II Midline Report July 2017 Strategic Objectives Key Questions Data Source and Results 1.2.1. Reduced Short- . How do students appreciate the ration size? Term Hunger

. To what extent have take-home rations been distributed Output 1.2.1.1/1.3.1.1 Implementer as per the project’s timeline and budget? Increased Access to interviews, student . How do students like the commodities provided for Food (School Feeding) focus groups school meals?

IR1.3. Improved . What are the greatest pull factors for students, regarding Student focus groups Student Attendance attendance?

. To what extent has the girls mentoring activity been implemented as per the project’s timeline and budget? Output. 1.3.1 . Have the mentors been recruited and trained as per the Increased Economic & project’s timeline and budget? Implementer Cultural Incentives (Or . Have the mentors conducted regular mentoring activities interviews Decreased as planned? Disincentives) . Have the SILC activities been implemented as per the project’s timeline and budget?

Output 1.3.4. . Which factors have facilitated or have been obstacles to Teacher interviews, Increased Student enrollment? student focus groups Enrollment

Output. 1.3.5. . To what degree have the awareness-raising activities on Increased Community Implementer the importance of education been completed as Understanding of interviews planned? Benefits of Education

SO2. Increased Use of Health and Dietary . To what extent have students improved their hygiene- Implementer Practices IR2.1 related practices? interviews, student Improved Knowledge . To what extent has the project supplied handwashing focus groups of Health and Hygiene stations to schools as planned? Practices

IR 2.3 Increased . To what extent have PTA members, school management Implementer Knowledge of committee members, and food preparers been trained in interviews Nutrition good nutrition and dietary practices as planned?

IMPAQ International, LLC Page 13 Beoog Biiga II Midline Report July 2017 Strategic Objectives Key Questions Data Source and Results

IR2.5 Increased Access . To what extent has the project distributed micronutrients Implementer to Preventative Health to students as planned? interviews Interventions

Source: Terms of Reference (TOR) and Performance Monitoring Plan (PMP).

2.2.2 Qualitative Data Sources

To address the research domains, IMPAQ conducted 45 – 60-minute interviews with individuals from six stakeholder groups: (1) USDA; (2) BBII implementers including CRS, OCADES, FAVL, and MENA; (3) county mayors; (4) teachers; (5) school administrators; and (6) students. Key informant interviews with these stakeholders probed the relevance of interventions, the effectiveness of implementation strategies and activities, the efficiency of the project, the impact of the project, and the likely sustainability of initiated actions.

Five of the 22 control group schools in the performance evaluation sample (which is the same as the control group for the impact evaluation) were randomly selected for qualitative data collection. See Appendix G for a full list of selected schools.

IMPAQ selected a small sample of teachers and students who completed the survey to participate in key informant interviews. In each of the five randomly selected schools, we conducted interviews with two teachers. We had at least one teacher representative from each grade and aimed for an even mix of male and female teachers. We conducted one focus group with five to eight students from one grade at each school, for a total of five focus groups, one per grade.11 We also interviewed the school principals at three of the five schools and two district administrators, one from each province.

In addition to school-level interviews, IMPAQ interviewed key program stakeholders and implementers. We conducted one interview with a USDA representative; nine interviews with representatives of project implementers, including MENA; and two interviews with county mayors, one from each province.

Exhibit 7 presents the number of interviews conducted per respondent type and the relevant research domains addressed by each group.

Exhibit 7. Respondent Types and Relevant Research Domains

Respondent Type Number of Interviews Research Domain(s) Project-Level Respondents Relevance, effectiveness, efficiency, perceived USDA 1 impact, sustainability Relevance, effectiveness, efficiency, perceived Implementers 9 impact, sustainability

11 We conducted focus group discussions with students in order to obtain detailed information about personal and group perceptions and opinions. Compared to individual interviews, focus groups allowed us to speak to a greater number of students at each school, given time and resource limitations.

IMPAQ International, LLC Page 14 Beoog Biiga II Midline Report July 2017 Respondent Type Number of Interviews Research Domain(s) Project-Level Respondents 4 from CRS, 2 from OCADES, 1 from FAVL, 2 from MENA (1 per province) 2 Relevance, effectiveness, perceived impact, County Mayors (1 per province) sustainability School-Level Respondents 10 Effectiveness, efficiency, perceived impact, Teachers (2 per school) sustainability 5 School 2 school district administrators (1 Effectiveness, efficiency, perceived impact, Administrators* per province) and 3 school sustainability principals 5 focus group discussions Students Effectiveness, perceived impact 1 per school, 35 students in total Source: IMPAQ. *School administrator refers to both district-level administrators/inspectors and school-level administrators/pedagogic advisors. When applicable, school district administrator is used to distinguish between the district-level administrator and the school director.

Appendix F presents the qualitative instruments developed and used for each respondent group.

IMPAQ International, LLC Page 15 Beoog Biiga II Midline Report July 2017 SECTION 3. FIELD WORK AND ANALYSIS

According to U.S. Department of Health & Human Services, this study was exempt from Institutional Review Board (IRB) because:  There is no bio biomedical testing involved in this research, and  This research is being conducted in an established or commonly accepted educational setting, “involving normal education practices”12.

This section outlines the quantitative and qualitative fieldwork and subsequent data analysis for this evaluation.

3.1 Quantitative Fieldwork and Analysis

3.1.1 Quantitative Data Collection In collaboration with Action, Gouvernance, Intégration, Renforcement (AGIR), our data collection partner, we recruited and trained 16 enumerators to collect the midline data in May and June 2017. To enhance the efficiency of midterm data collection, AGIR was able to re-hire five enumerators of the 16 who collected baseline data. Enumerator training consisted of three days of theoretical indoor training, one day of pilot testing in two schools in Ouagadougou, and one day of debriefing and instrument refinement. Enumerators received training on the rationale behind each survey and learned how to collect data on paper and tablets. They used tablets to conduct the in-person surveys13 and electronically submitted the surveys periodically during the field work.

With AGIR, we organized 14 enumerators into two teams of five individuals and one team of four. Each team had a team leader who was involved in the baseline data collection. The team leaders were already familiar with the project, the schools, and implementation of the ASER reading assessment. In addition to leadership tasks, they were mainly responsible for surveying second-graders and administering the ASER. Team leaders also contacted school principals to coordinate their arrival before visiting the school; ensured that randomly selected students were still enrolled and available to be surveyed; and, finally, asked principals to call on selected students’ parents (specifically mothers) to be present on a day of data collection to give their consent and be surveyed. The other two enumerators from the baseline data collection were responsible for collecting attendance data and surveying district administrators. Two IMPAQ local consultants, one in each province, closely followed the teams of enumerators daily to oversee the quality of the data that enumerators collected and provide them with technical support.

All enumerators regrouped with IMPAQ’s local consultants in their village several times during the data collection to debrief, submit daily data collection logs, submit electronic surveys, and review and plan for the next days of data collection. The team completed field work in 10 days.

In addition, one of IMPAQ’s local consultant led the classroom observation activity for the midline evaluation after receiving training from IMPAQ’s classroom observation expert in Bamako, Mali. Classroom observation training consisted of three days of training in real classrooms in Bamako, one day of pilot testing in two schools in Ouagadougou, and a day of debriefing and instrument refinement. During

12 Exemption 45 CFR 46.101(b)(1). 13 There were only a few cases in which enumerators had to survey students on paper due to programming issues on the first day of data collection. That night, the team leaders transferred the information from paper to tablets.

IMPAQ International, LLC Page 16 Beoog Biiga II Midline Report July 2017 the whole process of training, piloting, and debriefing, we worked closely with the CRS educational team to ensure that the tool was well adapted to the Burkina Faso context and to BBII teacher training components.

3.1.2 Quantitative Analysis After the field activities were completed we conducted a final review of the collected data to check: . Data completeness, . Duplicate entries, . Final dispositioning of records, . Skip pattern logic, and . Data cleaning accuracy.

We then compiled the survey responses into a master file for the analysis.

In this report, we provide summary statistics (percentages and averages) at midline using data from individual or multiple survey items. We also outline subgroup analyses by grade, student gender, and province, highlighting emerging patterns. To measure the progress of BBII toward its core objectives, we conducted descriptive analyses comparing mean outcomes at baseline with those at midline by running t-tests and using p-values.

3.2 Qualitative Fieldwork and Analysis

3.2.1 Qualitative Data Collection With oversight from IMPAQ and logistical support from CRS, one of IMPAQ’s local consultants, an experienced qualitative researcher, led the qualitative data collection effort in Burkina Faso in May 2017. IMPAQ’s in-house qualitative expert conducted the USDA interview in Washington, DC, before the field qualitative data collection effort began. IMPAQ coordinated with CRS to identify appropriate individuals for interviews and worked closely with CRS to schedule and organize the data collection effort.

In addition to supporting logistics and leading the in-field data collection effort, the local qualitative lead translated the local data collection guides into French and ensured that they were culturally appropriate. The qualitative lead moderated the interviews and focus group discussions, which were audio recorded, with consent, for note-taking and analysis purposes.

Prior to each interview and focus group discussion, the qualitative lead asked each participant to read and sign an informed consent form. IMPAQ worked closely with the qualitative lead to obtain consent forms for those under 18 year old from guardians prior to the focus group discussion. The form informed participants (and their guardians, if applicable) of the following:

. Their participation and the information that they disclose will be kept private.

. Their name will not be used in any reports. The interviewers will be taking notes during the discussions about what was said and report aggregate responses and opinions.

. Their participation is voluntary and they may choose not to answer a question if they feel uncomfortable.

. With their permission, the entire session will be audio recorded for report writing and analysis purposes only. Only the evaluation team will have access to the audio recording.

IMPAQ International, LLC Page 17 Beoog Biiga II Midline Report July 2017 Contact information for the study’s organizers was provided on the consent form and participants were encouraged to reach out to the organizer if they have any questions after the interview/focus group discussion.

Further, the team adhered to the following data collection protocol throughout the project:

 Interviews incorporated a degree of flexibility, and the evaluation team allowed additional questions in order to capture any information relevant to the research questions and domains.

 The evaluation team followed a consistent data collection approach with each respondent and at each school while allowing for limited variation according to the cultural practices in each locality.

 The evaluation team kept confidential all information and opinions expressed during individual interviews and focus groups. To the extent possible, only principal parties were present during the interviews.

3.2.2 Qualitative Analysis At the end of each data collection day, the qualitative lead documented the main points and key themes from each discussion and shared these insights with the IMPAQ team. These debriefings helped IMPAQ to: . Identify what topics and issues needed further probing . Determine how to adapt the guides in real time, if needed, to obtain more meaningful data . Ensure that the research team shared a common understanding and interpretation of the main points and themes . Establish quick-turnaround findings as warranted . Build a strong framework for additional analyses to be conducted after the site visit

After the data collection was complete, the team reviewed and analyzed the debriefing notes, supplemented by interview recordings and transcripts, to identify recurring patterns pertaining to the five research domains. Our analytical approach captured salient themes for each research domain and any important similarities and key differences that may inform the quantitative results.

IMPAQ International, LLC Page 18 Beoog Biiga II Midline Report July 2017 SECTION 4. EVALUATION SAMPLES

In this section, we provide summary statistics about students, parents, households, teachers, classrooms, and PTA representatives, and school district administrators. Since we drew a new sample at the midline, we also provide balance checks for key demographic characteristics pertaining to students, parents, households, and teachers to ensure the midline sample is observationally equivalent to the baseline sample.

To measure the midline values for performance indicators, shown in Exhibit 2 in Section 2.1.1, and to measure progress toward the desired outcomes over time, we collected data from the 22 control schools in the impact evaluation sample14 for the performance evaluation.

We followed the sampling strategy explained in Section 2 to randomly select students from the school list shared by CRS. However, as soon as the field operations started, as we expected, we found that some of the students on both the main lists and replacement lists were entirely unavailable for reasons such as relocation, grade repetition, and/or dropping out of school. In these situations, we followed our re- randomization plan to ensure a large enough random sample.

With our re-randomization strategy for students, we ended up with a random sample of 258 students, grades 2 to 6, and 262 parents. In each of the 22 schools, we also surveyed teachers of grades 2 to 6, as well as the PTA leader. In addition, we observed one course each in grades 1 to 5 in five randomly selected schools.15 All respondents, including students, parents, teachers, and PTA members, gave us their consent to proceed.16 Exhibit 8 shows the distribution of sampled respondents by province.

Exhibit 8. Sample Distribution by Province and Type of Respondent Type of Respondent Province Schools Students Parents Teachers PTA Leaders Bam 11 130 134 50 11 Sanmatenga 11 128 128 55 11 Total 22 258 262 105 22 Source: Surveys of students, parents, teachers, PTA members

4.1 Students

To maintain a balanced boys-to-girls ratio that reflects the population of beneficiary students, in each school, we randomly selected two boys and two girls in grade 2 and one boy and one girl in grades 3 to 6. In total, we surveyed 258 students in grades 2 to 6: 125 boys and 133 girls. As Exhibit 9 shows, the

14 In April 2016, CRS informed us about their mistake in the control group list, in which two schools in the treatment group were swapped with control schools, so they were included in the baseline performance evaluation data collection. However, for the midline data collection, we collected data from the right list of control schools. Descriptive analysis in this report is based on the correct list of schools and the update baseline values. 15 We observed one lesson in a classroom of each grade (grades 1 through 5). We included first grade because it provides the base of learning in primary school. 16 In accordance the U.S. Department of Health and Human Services guidelines on Human Subjects Research (45 C.F.R. § 46), we asked all respondents for their consent to proceed with the survey.

IMPAQ International, LLC Page 19 Beoog Biiga II Midline Report July 2017 proportion of girls to boys sampled in each grade is largely balanced (51.6 to 48.4 percent). Exhibit 47 in Appendix C also shows that the gender composition from baseline to midline has not changed.

Exhibit 9. Sampled Student Gender Distribution by Grade Male Female Grade Total Number of Total Number of Total Percent Percent Observations Observations Grade 2 (CP2) 49% 43 51% 45 88 Grade 3 (CE1) 51% 21 49% 20 41 Grade 4 (CE2) 45% 20 55% 24 44 Grade 5 (CM1) 49% 21 51% 22 43 Grade 6 (CM2) 48% 20 52% 22 42 Total 48.4% 125 51.6% 133 100% Source: Student Survey; IMPAQ calculations.

4.2 Parents

School principals were asked to call students’ parents — specifically mothers, who are usually the most informed about students — to be at school on the day of data collection to be surveyed and to give consent for their children to be surveyed. If siblings were included in the sample, we surveyed the parent separately for each sibling, to ensure that we collected data from parents for every student in our sample. This process enabled us to account for the possibility that children in one household could be treated differently, especially in the distribution of food. Almost all of the 258 surveyed parents were biological mothers (89 percent) or other female primary or secondary caregivers (10 percent).

Exhibit 10 shows the differences between educational attainments of parents obtained from the parent’s questionnaire at baseline and midline. Parents at midline are less illiterate (9 percentage points), and more educated with primary education (7 percentage points) (statistically significant at the 1 percent level). We should compare literacy outcomes of children in Section 5 with caution since children are from families with a slightly higher level of educations at midline compared to baseline.

Exhibit 10. Parents’ Educational Attainment Baseline Midline Difference in Means Level of Education Percent Percent (p-value test) -9*** None 95% 86% (0.00008) 7*** Primary education (Grades 1 to 6) 3% 10% (0.00016) 2 Secondary education (Grade 7 and above) 2% 4% (0.19087) Source: Parent Survey; N=343 (baseline) and N=262 (midline); IMPAQ calculations. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

IMPAQ International, LLC Page 20 Beoog Biiga II Midline Report July 2017 4.3. Household Environment

We also collected data from parents on household characteristics:

 Household composition: household size, percentage of children under 16, and percentage of household members above 16  Household access to basic services

4.3.1 Household Composition

At midline, the average household size of about 21 household members was similar in both provinces. About half of the household members (11 people) were 16 years old or younger (Exhibit 11). The data indicate a large variation in reported household size, from 4 to 72. This finding suggests that respondents reported on an extended family when answering the questions about household size. Compared to baseline, household size is significantly bigger by 8 members on average (p < 0.01) (four more children under 16 and four more adults).

Exhibit 11. Household Size

Baseline Midline Difference in Means Household Size Percent Percent (p-value test) 8*** Household size 13 21 (0.0000) Household members under 16 years old, including the 4*** 7 11 respondent in the household (0.0000) Household members over 16 years old, including the 4*** 6 10 respondent in the household (0.0000) Source: Parent Survey; N=343 (baseline) and N=262 (midline); IMPAQ calculations. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

4.3.2 Household Access to Basic Services

Exhibit 12 shows parents’ responses about their households’ access to basic services at baseline and midline. As shown in Exhibit 12, households at midline have better access to electricity, latrine, and metal rooves. However, accessibility to water for households at midline was very low (only 5 percent), compared to baseline (76 percent). Almost all households have access to electricity (96 percent); 67 percent had a latrine at home. However, some regional variation exists in the data. 72 percent of households in Bam had latrines versus 62 percent of households in Sanmatenga. Over half of the households (55 percent) had metal roofs: 49 percent in Bam and 62 percent in Sanmatenga.

Exhibit 12. Household Access to Basic Services

Baseline Midline Difference in Means Indicator Percent Percent (p-value test) 38*** Access to any source of electricity at home 58% 96% (0.0000)

IMPAQ International, LLC Page 21 Beoog Biiga II Midline Report July 2017 Baseline Midline Difference in Means Indicator Percent Percent (p-value test) 18*** Access to any type of latrine at home 49% 67% (0.0000) -71*** Access to water at home for washing 76% 5% (0.0000) Households with different types of roofs 0 Thatch Roof 19% 19% (0.92131) 9** Metal Roof 46% 55% (0.03140) -6* Combination of straw and metal roof 29% 23% (0.07573) -2 Roof made of some other material 5% 3% (0.10174) Source: Parent Survey; N=341 - 35517 (baseline) and N=262 (midline); IMPAQ calculations. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

4.4 Teachers

In each of the 22 schools, we surveyed teachers in grades 2 to 6, for a total of 105 teachers. Only five teachers in three schools were absent for medical reasons at the time of the survey. The gender composition of teachers is similar across the two provinces, and between baseline and midline; 59 percent of teachers at midline and 50 percent at baseline were female. The average class size was 54 students at midline which was not significantly different from the average class size at baseline.

Exhibit 13. Classroom Size Baseline Midline Difference in Means Indicator Mean Mean (p-value test) 4 Average number of students per classroom, all grades 50 54 (0.13169) 6 Average number of students in grade 2 classrooms 55 61 (0.38203) 11* Average number of students in grade 3 classrooms 51 62 (0.08890) 6 Average number of students in grade 4 classrooms 46 52 (0.28112) 1 Average number of students in grade 5 classrooms 47 48 (0.79204) -1 Average number of students in grade 6 classrooms 46 45 (0.94560) Source: Teacher survey; N=101 (baseline) and N = 105 (midline); IMPAQ calculations. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

17 Variation in the total number of observations is due to variation in the number of parents who answered the related question.

IMPAQ International, LLC Page 22 Beoog Biiga II Midline Report July 2017 On average, teachers at midline were slightly less experienced (by 6 months) compared to teachers at baseline with 3.5 years of experience, marginally significant at the 10 percent level. As Exhibit 14 shows, teachers at midline were also less educated compared to teachers at baseline. Most teachers (83 percent) had only a Brevet d’Etudes du Premier Cycle (BEPC), which corresponds to a tenth-grade level of education. Also, more teachers in Bam (14 percent) than in Sanmatenga (7 percent) had baccalaureate18 degrees (not shown).

Exhibit 14. Teachers’ Educational Attainment Baseline Midline Difference in Means Level of Education Percent Percent (p-value test) 15** BEPC 68% 83% (0.01484) -9* Baccalaureate 19% 10% (0.0908) 0 Bachelor`s degree 5% 5% (0.9501) 1 Master 1 0% 1% (0.32791) -7 Other 8% 1% (0.01432) Source: Teacher Survey; N=101 (baseline) and N = 105 (midline); IMPAQ calculations; None of the teachers had master 2 degrees neither at baseline nor at midline. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

As shown in Exhibit 15, 99 percent of teachers received teacher training while 95 percent of teachers received literacy training. Exhibit 15. Teacher Training Baseline Midline Difference in Means Level of Education Percent Percent (p-value test) 3 Teachers that have received teacher training 96% 99% (016234.) 79 Teachers that have received literacy training 16% 95% (0.0000) Source: Teacher survey; N=101 (baseline) and N = 105 (midline); IMPAQ calculations. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

4.5 Classroom Observations

We implemented a classroom observation instrument in five randomly selected schools in the performance evaluation sample19. (See Appendix G for the full list of schools.) We observed 25 classes, five grades per school.20 The average class size was 68 students per grade with a balanced proportion of girls to boys (49 to 51 percent). Almost all teachers (96 percent) taught in French. The only teacher who was observed using the local language was teaching first grade. Over 75 percent of the teachers were

18 Baccalauréat de l’Enseignement Secondaire Général corresponds to high school diploma at the end of grade 13. 19 Agreed between CRS and IMPAQ not to observe the classrooms during class break. 20 In one of the selected schools, Kougsable, the first-grade teacher was not available during the data collection. To keep the sample size consistent, we observed a first-grade class in a nearby school (Foulou) in the same CEB.

IMPAQ International, LLC Page 23 Beoog Biiga II Midline Report July 2017 women (19 out 25). All the teachers had completed initial training, in either public or private teacher training schools.

4.6 Parent-Teacher Associations

The IMPAQ team completed twenty-two surveys with PTA leaders. We received one survey from each school and 11 per province. Out of 22 surveyed PTA leaders, only three were women (14 percent). This gender composition is consistent with the baseline data, in which PTA leaders were predominantly men.

4.7 School District Administrators

We surveyed 30 school district level staff (Chef de Circonscription d’Education de Base, CCEB), including 15 pedagogical advisors, and 15 school district administrators. CCEB staff were predominantly men, with only one female member, with an average of 4.5 years of experience. Exhibit 48 in Appendix C shows approximately 63 percent of the CCEB staff had some sort of college education.

IMPAQ International, LLC Page 24 Beoog Biiga II Midline Report July 2017 SECTION 5. QUANTITATIVE OUTCOMES

In this section, we analyze data from the student, parent, teacher, PTA, and school district administrator surveys and from classroom observations. We examined all the data by gender, grade, and province (if relevant) but highlight the differences here only when descriptive differences exceed 5 percentage points. Appendix C provides additional details. Self-reported data, especially those on culturally and socially sensitive topics such as food security, should be interpreted with caution due to social desirability bias.

In analyzing performance data from student, parent, teacher, and PTA surveys, we compare mean outcomes at baseline and midline by using t-tests and p-values to highlight statistically significant differences. Such analysis can only suggest a correlation between the observed changes in outcomes and BBII interventions such as school feeding or teacher training. It cannot determine conclusively whether the interventions caused the changes. Other factors could have led to the observed changes over time, given that, at each data collection point (baseline and midline), we selected new samples of individuals to survey. There might be systematic differences in the two sets of samples that affected the outcomes. For example, improvements in literacy outcomes at midline relative to baseline could mean either that BBII teacher trainings were effective or that the students selected at midline came from better socioeconomic backgrounds. In addition, a simple difference in outcomes between baseline and midline could be caused by other general trends that affected all the schools.

This section also presents classroom observation outcomes to correlate their results with the other qualitative and quantitative findings. These outcomes should be interpreted with caution because of the small sample21 of observations (25 classrooms). However, the outcomes presented can provide useful suggestions for further observations at endline.

This section also presents midline outcomes from the school district administrator survey. Because the baseline sample was small, we did not conduct any descriptive analysis for these indicators.

Exhibit 16 provides an overview of the baseline and midline levels of the key McGovern-Dole evaluation performance indicators described in detail in Section 2.1.1, as required by the terms of reference. The Data Source column shows which survey data we used to measure each indicator. As we did at baseline, we calculated the total numbers by applying the proportion to the total population of beneficiaries22 in the Bam and Sanmatenga provinces. This calculation included 201,852 students (105,894 boys and 95958 girls), 3,993 teachers, 81 school district administrators, and 30,745 SILC members.23 Each of these indicators and other findings are discussed in detail further in this section. See Appendix B for detailed breakdown of this table.

21 Due to budget limitations, and with CRS and USDA approval, we kept the sample small at midline. 22 We used the percentages as basis for extrapolation to estimate the number of beneficiaries in each indicator. 23 This method is described in the CRS performance monitoring plan and was approved by USDA.

IMPAQ International, LLC Page 25 Beoog Biiga II Midline Report July 2017 Exhibit 16. Overview of Baseline and Midline Levels of Key Performance Indicators

Baseline Midline Key Performance Data Total 95% Total 95% Indicators Source Percentd Estimated Confidenc Percentd Estimated Confidence Number e Interval Number Interval 1. Percentage of students (boys and girls) Boys: 12% 12,707 0 – 31% 16% 16,943 5 – 28% who, by the end of two grades of primary ASER reading schooling, demonstrate test that they can read and understand the meaning Girls: 17% 16,313 1 – 33% 32% 30,707 17 – 46% of grade-level text. 2. Percentage of teachers who devote at Teacher least an average of 45 55% 1,795 45 – 65% 66% 2,635 57 – 75% Survey minutes a day to literacy instruction. 3. Percentage of teachers who attend and District teach school at least Administrator 85% 2,775 NAa 88% 3,514 NAa 90% of scheduled school s days per year. 4. Percentage of school administrators and officials who District demonstrate use of at Administrator 61% 49 NAa 97% 79 NAa least one new technique, s skill, or tool as a result of USDA assistance.b 5. Number of preschool Pre- and post- teachers bissongo testing will be caregivers) in target conducted preschools who N/A N/A N/A N/A N/A N/A before and demonstrate use of at after BBII least one new teaching training technique, skill, or tool. 6. Percentage of teachers/educators/ teaching assistants in Teacher target schools who questionnaire demonstrate use of new 51% 1,677 42 – 61% 44% 1,757 34 – 53% Student and quality teaching questionnaire techniques or tools as a result of USDA assistance. 24 7. Percentage of Teacher 45% 74,400 35 - 54% 47% 94,870 37 - 56% students in target questionnaire

24 Teachers who declared using at least 5 of the 7 activities.

IMPAQ International, LLC Page 26 Beoog Biiga II Midline Report July 2017 Baseline Midline Key Performance Data Total 95% Total 95% Indicators Source Percentd Estimated Confidenc Percentd Estimated Confidence Number e Interval Number Interval schools who are identified as attentive25 during class/instruction. 8. Percentage of students in target Student schools who indicate questionnaire 4% 8,075 2 - 7% 7% 14,130 4 - 11% that they are hungry or Parent very hungry during the questionnaire school days. 9. Percentage of Boys: 100 - students regularly (80%) 105,894 98% 103,776 96 - 100% 100% 100% attending USDA School supported registries Girls: classrooms/schools 95,958 99 - 100% 99% 94,998 98 - 100% 100% (boys and girls). 10. Percentage of group members that spend Parent/PTA 48% 14,758 24-71% 63% 19,369 53-72% money from SILC on questionnaire education costs. 11. Percentage of girl students who cite mentors as one of the Student N/A N/A N/A N/A N/A N/A top 3 reasons for their questionnaire success or improvement in school.c 12. Percentage of school-age children Boys: 56% 59,301 49 - 64% 69% 73,067 61 - 77% Student receiving a minimum questionnaire acceptable diet (boys and girls). Girls: 63% 60,454 56 - 70% 74% 71,009 67 - 82% 13. Percentage of school children that wash their hands at critical Student 78% 157,445 74 - 83% 82% 165,519 77 - 87% moments: before eating questionnaire and after using the latrine. 14. Percentage of students in target schools who achieve a Student passing score on a test 5% 10,093 3 - 8% 4% 8,075 1 - 6% questionnaire of good health and hygiene practices by naming at least 4

25 We asked teachers on a scale of 1 to 10, where 1 is not attentive at all, to rate their students’ attentiveness. Passing attentiveness score defined as a score of 7 or more on a scale of 1 to 10.

IMPAQ International, LLC Page 27 Beoog Biiga II Midline Report July 2017 Baseline Midline Key Performance Data Total 95% Total 95% Indicators Source Percentd Estimated Confidenc Percentd Estimated Confidence Number e Interval Number Interval situations in which they should wash their hands. 15. Percentage of students in target schools who achieve a passing scoreon a test on food nutrition and Student 1% 2,019 0 - 1% 3% 6,056 1 - 5% dietary practices by questionnaire naming at least one food with iron and one with vitamin A and by naming one benefit of each. 16. Number of months of community and/or PTA 5 months N/A 4 - 7% 3 months N/A 2 – 5% government support for questionnaire school canteens. Source: Surveys of students, parents, teachers, PTA members; IMPAQ calculations. a the small sample size of district administrators does not permit statistical inferences and a confidence interval is not meaningful. b In the BBII project, this indicator focuses on district-level administrators. c This indicator is irrelevant at the midline performance evaluation because the sample includes only the 22 control schools of the impact evaluation which has not received the mentoring intervention. d Percent columns show the percentages with respect to the surveyed respondents for each indicator at baseline and midline. These percentages are used as basis for extrapolation to estimate the total number.

5.1 School Outcomes

School-level outcomes involve the presence of libraries and food storage for the school canteen. Both were determined by observation.

As part of the BBII project, CRS and its partner FAVL established community libraries in the capital town of each county. We did not observe any of the schools to have a library nearby. This is in line with our qualitative findings described in Section 6—interviewed teachers and school administrators noted that libraries are often long distances from schools and therefore inaccessible to students and teachers. However, during interviews, teachers reported a desire to have access to books and resources through libraries and were disappointed with the location of the libraries.

As observed, 21 out of 22 schools were equipped with food storage. In the school with no food storage, parents stored the food for the canteen at home. Storage spaces were mostly in good condition. All 21 had a metal roof and 20 had cement walls. The other building’s walls were made of mud. Two storage buildings had holes in the roof through which rain could penetrate.

5.2 Student Outcomes

This section presents student outcomes, collected from the student survey, in the following six areas at midline in comparison with baseline:

IMPAQ International, LLC Page 28 Beoog Biiga II Midline Report July 2017  Nutrition knowledge  Hygiene knowledge and practices  Food security  Attendance  Literacy activities  Reading outcomes

5.2.1 Nutrition Knowledge

As we did at baseline, to capture students’ nutrition knowledge, we asked children whether they had heard of vitamin A and iron. If they had heard of those nutrients, then we asked them to name benefits of vitamin A and iron, as well as foods that contain them. We also defined a passing score on a test of nutrition as the ability to name at least one food with iron, one food with vitamin A, one benefit of iron, and one benefit of vitamin A.

Exhibit 17 shows that the proportion of students with the desired level of nutrition knowledge improved over time, although it is still low. The proportion of students who had heard of vitamin A and of iron increased 17 and 10 percentage points, respectively. The differences in means are statistically significant at the 1 percent level in both cases. In spite of these improvements, the proportion of students who could cite one benefit of vitamin A and one type of food that contains it has not substantially increased. However, the proportion of students who could cite one benefit of iron increased by 26 percentage points (from 32 percent to 58 percent) compared to baseline. The difference in means is statistically significant (p < 0.05). The proportion of students who achieved a passing score by naming benefits and foods for both nutrients increased by 2 percentage points at midline, with a statistically significant difference in means (p < 0.05), but this proportion still remains low.

In interviews and focus group discussions, both teachers and students reported that dietary teachings about vitamins and minerals were useful, though it was unclear whether or how students apply these lessons to their everyday lives.

Exhibit 17. Student Nutrition Knowledge

Difference Baseline Midline in Means Indicator Total Number Total Number of Percent Percent of (p-value) Observations Observations 17*** Students who have heard of vitamin A 32% 324 49% 258 (0.0000) Students who can cite one benefit of 5 41% 103 46% 126 vitamin Aa (0.4273) Students who can cite a food 2 23% 92 25% 107 containing vitamin Aa (0.6940) 10*** Students who have heard of iron 9% 324 19% 258 (0.0004) Students who can cite one benefit of 26** 32% 28 58% 48 irona (0.0276) Students who can cite a food 8 29% 28 37% 48 containing irona (0.4354)

IMPAQ International, LLC Page 29 Beoog Biiga II Midline Report July 2017 Difference Baseline Midline in Means Indicator Total Number Total Number of Percent Percent of (p-value) Observations Observations Students who achieve a passing score 2** on a test of food nutrition and dietary 1% 324 3% 258 (0.0220) practices Source: Student Survey; IMPAQ calculations. a Sample comprises only students who said they had heard of the nutrient. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

5.2.2 Hygiene Knowledge and Practices

To measure students’ knowledge and practice of hygiene, we asked students to identify instances in which they should wash their hands (for example, before eating, after touching animals, before preparing food). We defined a threshold of hygiene knowledge as the ability to identify at least four situations in which people should wash their hands. We also asked students about their hand and body washing practices.

The data show no significant changes from baseline to midline in children’s self-reported hygiene practices (Exhibit 18). At midline, almost all students (95 percent) reported washing their hands and more than half (63 percent) reported washing their bodies; these results are similar to baseline findings. Interestingly, in focus group discussions, children reported enjoying the handwashing practice. Teachers and school administrators at some schools reported that students’ hygiene practices have changed since the start of the project; handwashing has become a habit, they report. The lack of significant changes from baseline to midline could be explained by the fact that some schools lack reliable water sources and handwashing devices sometimes fall into disrepair, as reported by school-level interviewees, and explained in more detail in Section 6.2.

Very few children achieved a passing score on the test of hygiene knowledge. The one percentage point decrease in means is not statistically significant. However, at midline almost all children (98 percent) reported their teachers taught them the importance of washing hands. These contradictory findings should be interpreted with caution due to social desirability bias (i.e., children did not want their teachers to look bad)26.

Exhibit 18. Student Hygiene Practices

Difference in Baseline Midline Means Indicator Total Number Total Number Percent of Percent of (p-value) Observations Observations Students who say that they washed 1 94% 324 95% 258 their hands today (0.5575) Students who say that they washed -3 66% 324 63% 258 their body today (0.4722)

26 This indicator was added at the midline, therefore no comparison in mean of outcomes with baseline is available.

IMPAQ International, LLC Page 30 Beoog Biiga II Midline Report July 2017 Difference in Baseline Midline Means Indicator Total Number Total Number Percent of Percent of (p-value) Observations Observations Students who achieve a passing score -1 on a test of good health and hygiene 5% 324 4% 258 (0.4356) practices

Source: Students Survey; IMPAQ calculations. Note: Excluded observations from baseline are students who did not answer these questions. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

We also asked children whether they washed their hands at critical moments, defined as before eating and after using the latrine. Then we compared these self-reported handwashing practices with students’ knowledge of the two critical moments. The green bars in Exhibit 19 show that the proportion of students who reported they washed their hands at critical moments slightly increased by four percentage points from baseline (78 percent), though the results are not statistically significant. Likewise, as shown by the red bars of Exhibit 19, knowledge about handwashing practices changed by only 4 percentage points from baseline to midline; the difference is not statistically significant. The inconsistency between children’s knowledge and their actual practices suggest a possibility of over-reporting by students on their hygiene practices. Therefore, these results should be interpreted with caution, because they derive from self- reported data from young children. Also, as mentioned earlier in this section, the changes in outcomes can be due to external factors that are not necessarily associated with BBII.

Exhibit 19. Students’ Knowledge of Handwashing versus Self-Reported Practices of Handwashing at Critical Moments

90% 82% 78% 80% 70% 60% 50% 40% 26% 30% 22% 20% 10% 0% Baseline Midline

Knowledge Self-reported Practices

Source: Student Survey; IMPAQ calculations. Note: for baseline, N=36, 414 (Knowledge) and N=129,104 (Self-reported Practices); for midline, N=43,035 (Knowledge) and N=135,725 (Self-reported Practices)27. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01, *shows the difference in means between self-reported practices and hygiene knowledge.

27 The calculations are based on the total number of responses to different options that were selected for all that applied.

IMPAQ International, LLC Page 31 Beoog Biiga II Midline Report July 2017 We also attempted to observe students’ handwashing practices at critical moments at school. However, almost 90 percent of students did not eat or use the latrine at the time of data collection. Out of the remaining 9 percent, 7 percent were observed washing their hands before eating and about 2 percent were observed washing their hands after using the latrine. As discussed in Section 7, this approach needs refinements at the endline to observe more students’ handwashing practices.

5.2.3 Food Security

To measure food security among students, we looked at two factors:

1. Children’s food intake during the day (breakfast and lunch). We asked students whether they ate breakfast and lunch a day before data collection, where they ate those meals (at home or at the canteen), and whether they felt full after consuming each meal. 2. Minimum acceptable diet. We followed the UN Food and Agriculture Organization (2010) diversity index, recommended by the USDA Foreign Agricultural Service, to calculate the minimum acceptable diet in terms of diversity, first using student data and then using parent data for purposes of comparison. Using a list of 15 types of food, we defined a threshold for an acceptable diet as including at least five different foods per day. 28

The data show, as at baseline, that most children who ate breakfast and lunch felt full after eating. Exhibit 20 shows that the proportion of children who reported at midline that they were hungry during the school day, defined as whether they felt full after their breakfast using the previous day as reference, was as low as at baseline with a slight increase, but not statistically significant at any conventional level. Although this increase is not statistically significant,29 it could be explained by fewer children (4 percentage points) eating before going to school (not statistically significant at any level) at midline. Students most commonly cited insufficient food (81 percent) as the reason why they could have eaten more after breakfast before going to school at midline.

In addition, the proportion of students that had their lunch at the canteen increased to 80 percent at the midline, statistically significant at the one percent level. This outcome, however, should be interpreted with caution. It is because although 26 percent of surveyed students at baseline reported eating lunch at the canteen, the canteen component of the project had not yet started at that time. One potential explanation is that students might have been confused with the definition of the canteen about having one in their school at the baseline30. During focus group discussions, students expressed much appreciation for the food they received through the canteens. They stated that the cooked food is tasty and that they often take some home to share with their siblings and parents. In fact, respondents described that, from their observations, the presence of canteens is directly associated with attendance; when canteens are providing food, students go to school.

28 IMPAQ removed spices from the list of 16 food groups in the index, because spices are not relevant for child nutrition. 29 The change that is not statistically significant could be attributed to chance. In better words, there’s always a chance that the differences we observe when measuring a sample of users is just the result of random noise. 30 Community (parents, school staff, etc.) provided “endogen canteen” for children at school which was feeding them during lunch before school canteens, provided by BBII, started functioning.

IMPAQ International, LLC Page 32 Beoog Biiga II Midline Report July 2017 Exhibit 20. Students’ Food Intake

Difference in Baseline Midline Means Total Number Total Number Indicator Percent of Percent of (p-value) Observations Observations Proportion of children that ate -4 90% 325 86% 258 breakfast before going to school (0.1624) Children that felt full -3 after the meal s/he ate 96% 293 93% 207 (0.1816) before going to school. Proportion of students in target schools who indicate that they are 3 4% 293 7% 223 hungry or very hungry during the (0.1816) school day. Source: Student Survey; IMPAQ calculations Note: Excluded observations are students who did not answer these questions. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

Exhibit 21 shows that the proportion of children who received a minimum acceptable diet increased from baseline to midline, by 11 percentage points according to student reports and by 20 percentage points according to parents. Both findings are statistically significant, particularly the parent reports (p < 0.01). This increase in the diversity of household’s diet might be explained by different socio-economic backgrounds between baseline and midline samples. There are differences between what parents and students reported and between boys and girls. However, the gaps have diminished over time. As shown in Exhibit 21, the proportion of boys receiving a minimum acceptable diet increased by 13 percentage points, which is slightly more than the increase for girls (11 percentage points); both changes are statistically significant (p < 0.05). See Appendix B for the breakdown of diverse foods received by girls and boys; those data show that girls ate more vegetables (of any type), while boys ate slightly more protein.

Exhibit 21. Minimum Acceptable Diet

Difference in Baseline Midline Means Indicator Total Number Total Number Percent of Percent of (p-value) Observations Observations Students reported receiving a 12*** 60% 324 72% 258 minimum acceptable diet (0.0029) Boys received minimum 13** 56% 156 69% 125 acceptable diet (0.0336) Girls received minimum 11** 63% 168 74% 133 acceptable diet (0.0361) Parents reported that 20*** students received a minimum 35% 343 56% 262 (0.0000) acceptable diet Source: Student Survey; Parent Survey; IMPAQ calculations. Note: Missing observations represent respondents who did not answer these questions. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

IMPAQ International, LLC Page 33 Beoog Biiga II Midline Report July 2017 5.2.4 Student Attendance To measure students’ attendance, IMPAQ and CRS agreed to select three representative months across the agricultural crop cycle and the academic school year: November, January, and April. Then, we collected attendance data for each child in the sample from teachers using school records. Overall, almost all students attended school at least 80 percent of the time (Exhibit 22).

Compared to the baseline, the midline value showed a slight decrease in the proportion of students with regular attendance of 1 percentage point, a statistically significant finding at the 10 percent level. Boys’ attendance decreased by 2 percentage points at midline, somewhat higher than the decrease in girls’ attendance. However, it is important to note that the sample of student at midline for attendance data was smaller than the baseline attendance sample by nearly half.31

Moreover, our experience in the field showed that some teachers did not usually record attendance data and that, when they did, we could not ascertain how rigorously they reported, especially at baseline. Even with the best intent, it is possible that teachers neglected to record missing children. Therefore, the average attendance rate is likely overestimated.

Exhibit 22. Student Attendance Difference Baseline Midline in Means Indicator Total Total Number Percent Number of Percent of (p-value) Observations Observations Students attending class at least 80% of -1* 100% 510a 99% 261 the time (0.0814) Boys attending class at least 80% of the -2* 100% 219 98% 125 time (0.0628) Girls attending class at least 80% of the -1 100% 204 99% 134 time (0.7648) Source: Student Attendance Data; IMPAQ calculations. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01; a See footnote 31

5.2.5 Literacy Activities

Activities in Class We asked students about their reading and writing activities in class. We correlated the outcomes with teachers’ responses, discussed in detail in Section 5.4.2, to assess implementation of BBII teaching techniques in classrooms. Specifically, we asked students whether they had read alone or aloud, listened to other students reading, and worked on writing or words during class on the day before the survey.

Exhibit 23 shows that the proportions of students who spent time reading alone and listening to reading in class increased by 7 and 8 percentage points, respectively; these changes are statistically significant. There were no statistically significant changes in time spent reading to someone else or doing word work

31 At baseline, the sample size for attendance did not exactly match the sample size of the performance evaluation because incomplete records were dropped from the performance evaluation data, and IMPAQ could not match the student data and attendance data.

IMPAQ International, LLC Page 34 Beoog Biiga II Midline Report July 2017 in class. The only activity on which a smaller proportion of children reported spending time at midline than at baseline is writing; however, the decrease is not statistically significant.

Exhibit 23. Student Reported Classroom Activities

Difference in Baseline Midline Means Indicator Total Number Total Percent of Percent Number of (p-value) Observations Observations Spent time reading alone in class 7* 63% 325 70% 245 yesterday (0.0638) Spent time reading to someone 1 62% 325 63% 245 else in class yesterday (0.7485) Spent time writing in class -2 71% 325 69% 245 yesterday (0.6627) Spent time doing word work in 6 43% 325 49% 245 class yesterday (0.2191) Spent time listening to reading in 8** 72% 325 80% 245 class yesterday (0.0376) Source: Student Survey; IMPAQ calculations. Note: Excluded observations from baseline are students who did not answer these questions. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

Activities Outside of Class We also looked at children’s literacy activities outside of their classrooms. We asked children whether they are part of a reading club, defined as a small group of children who read together outside of the school, as part of BBII program activities. In addition, we asked parents whether they take their children to the closest library.

Exhibit 24 shows that the proportion of students who participated in a reading club decreased by 34 percentage points, from 60 percent at baseline to 26 at midline; the finding is statistically significant at the 1 percent level. However, there has been no statistically significant change in whether the clubs consist of whole school classes. These findings should be interpreted with caution since the reading club activity had not started when the baseline data collection was conducted (This activity started in second year of program32). There is a possibility that children at baseline confused the definition of a reading club with a reading group, a small group of children who read together in the classroom.

The proportion of students visiting libraries with their parents remained very low, with no change from baseline to midline. Again, this is not surprising given the findings described above in Section 5.1.

32 CRS monitoring data from 2017 showed that 682 reading clubs were registered across all project schools during the 2016-2017 school year. However, this does not mean that the reading clubs convened, only that they formed.

IMPAQ International, LLC Page 35 Beoog Biiga II Midline Report July 2017 Exhibit 24. Student Reading Groups

Difference in Baseline Midline Means Indicator Total Number Total Number Percent of Percent of (p-value) Observations Observations Students who are in a reading -34*** 60% 325 26% 245 club (0.0000) Reading clubs that consist of -2 19% 195 17% 63 the entire class (0.7893) Students who visit the library 0 with their parents (reported by 2% 343 2% 262 (0.6371) parents) Source: Student and Parent Survey; IMPAQ calculations. Note: Excluded observations are students that did not remember what they had done the day before. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

5.2.6 Student Reading Outcomes

We used the ASER literacy assessment to measure students’ grade-level reading competencies. We determined the thresholds for an acceptable reading level at each primary school grade according to the Burkina Faso curriculum guidelines and the calibration workshop that IMPAQ and CRS held in May 2017 (Exhibit 4 in Section 2.1.3). The outcome of the calibration is that level C in the ASER test, the ability to read complex sounds, is the minimum acceptable level at the end of grade 2.

Exhibit 25 shows the distribution of the ASER literacy assessment results for baseline and midline. The minimum acceptable threshold, level C, is represented with a vertical orange line. Yet, at midline the majority of students are below the threshold for their grade level, showing that they have limited basic reading skills. Girls showed slightly higher reading proficiency scores than boys. (See Exhibit 51 in Appendix C).

IMPAQ International, LLC Page 36 Beoog Biiga II Midline Report July 2017 Exhibit 25. Distribution of Reading Skills for Second Grade Students 70

60 55

50 45

30 Percent 22 20 17 12 9 9 7 7 10 5 3 0 1 0 2 2 0 1 0 Level 0 Level A Level B Level C Level D Level E Level F Level G Level H Level I Level J Level K

Baseline Midline

Source: Student Survey: IMPAQ calculations. N (baseline) = 40; N= (midline) = 87.

Although the majority of students did not achieve grade level reading competencies, the proportion of second grade students with reading proficiency at grade level increased by 9 percentage points, but not statistically significant at any conventional levels (Exhibit 26). These outcomes are in line with CRS monitoring data in 2016 that showed improvements in children’s literacy outcomes. The improvement in reading ability at grade level was higher for girls (15 percentage points) compared to boys (4 percentage points). Lowering the threshold to Level B (ability to read simple sounds), the acceptable level at the beginning of grade 2, most students’ reading ability is still below their grade level however, it has improved over time (See Exhibit 52 in Appendix C). However, it is important to note that because of small sample sizes, the results only provide suggestive descriptive evidence for the changes in the indicators of interest and must be interpreted with caution.

Exhibit 26. Reading Proficiency at Second Grade Level (Students who scored an ASER Level C (Read Complex Sounds) or higher)

Difference in Baseline Midline Means Indicator Total Number Total Number Percent of Percent of (p-value) Observations Observations Second grade students 9 demonstrating reading ability 15% 40 24% 87 (0.2458) at grade level or above Male students demonstrating 4 reading ability at grade level or 12% 16 16% 43 (0.7253) above Female students 15 demonstrating reading ability 17% 24 32% 44 (0.1811) at grade level or above Source: Student Survey; IMPAQ calculations. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

IMPAQ International, LLC Page 37 Beoog Biiga II Midline Report July 2017 These improvements are in line with school-level stakeholders’ perception of the project. All teachers and school administrators who participated in interviews believed that the project is improving literacy skills among children, due to the new teaching techniques. Teachers reported that they have observed improvements in students’ literacy skills and motivation to learn.

5.3 Parent Outcomes

To look at children’s household environments, we surveyed their parents, specifically their mothers. Using this survey, we compared parent outcomes at baseline and midline in the following three areas:

 Nutrition knowledge  Attitude towards girls’ education  School involvement

5.3.1 Nutrition knowledge

A parent is a first child’s teacher. Parents play an important role in children’s knowledge and learning. To gain a deeper understanding of children’s nutrition knowledge, we asked parents the same nutrition questions that we asked students: whether parents had heard of vitamin A and iron and, if so, whether they could name benefits of and foods containing each nutrient.

Exhibit 27 shows changes in parents’ nutrition knowledge over time. Overall, the proportion of parents who had heard of vitamin A and iron increased significantly by 10 and 7 percentage points, respectively, from baseline to midline. This increase in parents’ responses confirms the increase in proportion of children who heard of these nutrients. A significantly smaller proportion of parents were able to name a food containing vitamin A at midline than at baseline (a difference of 23 percentage points, p < 0.01). However, a larger proportion of parents could cite a food containing iron (a difference of 25 percentage points, p < 0.05). Also, there was a significant 20 percentage point decrease in the proportion of parents who named one benefit of iron. In general, the data show improvements in the percentages of parents and of students who have heard of iron and vitamin A. However, their knowledge of the benefits of these nutrients or the foods that contain them are still low. These findings are in line with their children’s low knowledge of nutrition, suggesting children did not have any other sources except teachers to learn about this knowledge.

Exhibit 27. Parent Nutrition Knowledge

Difference in Baseline Midline Means Total Number Total Number Indicator Percent of Percent of (p-value) Observations Observations Parents who have heard of 10** 35% 343 45% 261 vitamin A (0.0141) Parents who can cite one -5 60% 120 55% 117 benefit of vitamin Aa (0.4117) Parents who can cite a food -23*** 59% 120 36% 117 containing vitamin Aa (0.0003)

IMPAQ International, LLC Page 38 Beoog Biiga II Midline Report July 2017 Difference in Baseline Midline Means Total Number Total Number Indicator Percent of Percent of (p-value) Observations Observations Parents who have heard of 7*** 8% 343 15% 262 iron (0.0026) Parents who can cite one -20* 78% 23 58% 40 benefit of irona (0.0991) Parents who can cite a food 25** 28% 29 53% 40 containing irona (0.0390) Source: Parent Survey; IMPAQ calculations a Sample comprises only parents who had heard of the nutrient. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

5.3.2 Attitude towards Girls’ Education To measure parents’ attitude towards girls’ schooling, we asked parents whether they think schooling is good for girls. We also asked those with positive opinions to select from a list of reasons for supporting girls’ education. Exhibit 28 shows the changes in parents’ opinions about girls’ schooling. At midline, virtually all parents (100 percent) showed positive attitudes toward girls’ education; this represents a 21 percentage point increase from baseline, a statistically significant finding (p < 0.01). The exhibit also shows that the majority of parents mentioned that educating girls is good because it improves living standards, an increase of 14 percentage points over baseline (p < 0.01). The percentage of parents who said that girls’ schooling improves their health decreased by 6 percentage points, and the percentage of those who said that education allows girls to find better work decreased by 8 percentage points; both findings are significant (p < 0.01).

Exhibit 28. Parents’ Opinions of Girls’ Schooling

Difference in Baseline Midline Means Indicator Total Total Percent Number of Percent Number of (p-value) Observations Observations Parents who indicated that girls’ 21*** 79% 343 100% 262 schooling is good (0.0000) Parents who indicated that girls’ -2 2% 343 0% 262 schooling is bad (0.1195) -20*** Parents who did not know the answer 20% 343 0% 262 (0.0000) Source: Parent Survey; IMPAQ calculations * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

IMPAQ International, LLC Page 39 Beoog Biiga II Midline Report July 2017 Exhibit 29. Parents’ Reasons for Supporting Girls’ Schooling

Difference in Baseline Midline Means Indicator Total Total Percent Number of Percent Number of (p-value) Observations Observations Parents who say educating girls 14*** 54% 270 68% 261 improves living standards (0.0009) Parents who say educating girls -6*** 7% 270 1% 261 improves their health (0.0002) Parents who say educating girls -1 improves the health of their future 5% 270 4% 261 (0.5984) children Parents who say girls should be able to 2 16% 270 18% 261 fulfill themselves (0.6841) Parents who say educating girls allows -8*** 14% 270 6% 261 them to find better work (0.0009)

Source: Parent Survey; IMPAQ calculations. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

5.3.3 Parents’ Engagement in School

We asked parents about their participation in school as PTA members and in the village SILC. We also asked those who do belong to SILCs whether they used their savings to cover school expenses. Exhibit 30 shows that more parents said they were involved in their children’s school at midline than at baseline. The proportion of parents who were members of the PTA increased significantly, by 15 percentage points (p < 0.01). Exhibit 30 also shows a considerable increase in the percentage of parents who were SILC members; the 32 percentage point difference is significant at the 1 percent level.

Of parents who were SILC members, a larger proportion at midline than at baseline reported that they used their savings to cover school expenses, but the change was not statistically significant. Of the parents who said that they used their savings for school expenses, the proportion of those who reported that the savings helped “a lot” with school fees decreased by 33 percentage points from baseline to midline. The proportion of those who said that the savings helped “some” increased by 38 percentage points. Both findings are statistically significant. A possibility of increase in education expenses or a decrease in parents’ purchasing power to afford school costs might be potential explanations to this shift in parents’ responses.

We asked parents about their participation in school as a PTA member, as well as in SILC group to measure the extent to which the savings are used to cover school expenses. Exhibit 30 shows parents were more involved in their children’s schooling at midline. The proportion of parents who served as a member of a PTA more than doubled over time with statistically significant increase of 15 percentage points (p < 0.01). The exhibit also provides information about parents’ involvement in a savings group, SILC. From baseline to midline, there was a considerable increase in SILC membership from 9 percent to 41 percent, statistically significant at the 1 percent level. A larger proportion of parents reported that they used their savings to cover school expenses at midline, but the change was not statistically significant. However, at

IMPAQ International, LLC Page 40 Beoog Biiga II Midline Report July 2017 midline, fewer parents reported that the savings helped a lot with school fees, while more of them thought that the savings only helped some.

Exhibit 30. Parents’ Involvement in School and in Savings Cooperatives Difference in Baseline Midline Means Total Total Indicator Percent Number of Percent Number of (p-value) Observations Observations Parents who are members of the 15*** 13% 343 28% 262 PTA (0.0000) Parents who are members of a 32*** 9% 220 41% 262 savings group (0.0000) Parents who have used their 15 48% 21 63% 107 savings for school expenses a (0.2023) Parents indicating the savings -33** helped “a lot” with school fees 44% 9 12% 67 (0.0116) and tuition a Parents indicating the savings 38** helped “some” with school fees 11% 9 49% 67 (0.0309) and tuition a Parents indicating the savings -6 helped “a little” with school fees 44% 9 39% 67 (0.7492) and tuition a Source: Parent Survey; IMPAQ calculations. a Sample comprises only parents who are members of a savings group. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

We asked parents whether they meet individually with teachers of their children to assess their involvement in school. We asked the same questions from teachers to triangulate their responses with parents’. Exhibit 31 shows the level of one-on-one meetings held between parents and teachers reported by both types of respondents. At midline, more parents (20 percentage points) reported they met individually with teachers in the past 12 months, statistically significant at the 1 percent level. Their responses were more or less consistent with what teachers reported. Also, more teachers (15 percentage points) reported they met one-one-one with parents at least 3 times in the past year at midline. However, this increase was lower when reported by parents (4 percentage points), but not statistically significant at any levels. The primary reason reported by both parents and teachers to meet with each other was “child’s performance” at both baseline and midline, with no statistically significant changes.

Exhibit 31. Parents –Teacher Individual Meetings Difference Baseline Midline in Means Indicator Total Total Percent Number of Percent Number of (p-value) Observations Observations Teacher met individually with parents of 9 74% 54c 83% 105 students in the past 12 months a (0.1931)

IMPAQ International, LLC Page 41 Beoog Biiga II Midline Report July 2017 Difference Baseline Midline in Means Indicator Total Total Percent Number of Percent Number of (p-value) Observations Observations Teacher met with parents of students about student 15 48% 40 62% 87 performance at least 3 times in (0.1249) the past 12 months a Parent or another adult in household 20*** met individually with teacher over the 45% 343 65% 262 (0.0000) past 12 months b Parent or another adult in household met individually with 4 28% 154 32% 170 teacher over the past 12 (0.4189) months b Source: a Teacher & b Parent Survey; IMPAQ calculations; c Excluded observations from baseline are students who did not answer these questions * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

5.4 Teacher Outcomes

From the teacher survey, we gathered baseline and midline data on teacher outcomes in the following three areas:  Nutrition Knowledge  Attendance  Classroom practices and teaching techniques In addition, this section also provides information on  Classroom observations, and  Students’ Attentiveness reported by teachers

5.4.1 Nutrition Knowledge To assess teachers’ nutrition knowledge, we asked them the same questions about vitamin A and iron that we asked students and parents. Exhibit 32 shows changes in teachers’ knowledge of nutrition from baseline to midline. Overall, since teachers seemed to be knowledgeable about nutrition at the baseline, changes were expected to be small. The data show that a larger proportion of teachers had heard of vitamin A at midline than at baseline, a 4 percentage point difference that is significant at the 10 percent level. However, their knowledge of vitamin A’s benefits or the foods containing this nutrient have not changed significantly. By contrast, although a slightly smaller proportion of teachers had heard of iron at midline among those teachers who had heard of iron, larger proportions could name a benefit and a food containing iron. The differences of 18 and 15 percentage points, respectively, are statistically significant (p < 0.01). This confirms that most teachers seem to have the knowledge necessary to increase students (and community) nutrition awareness, but may need support on how to teach it effectively. This may explain the reason why students’ nutrition knowledge (Section 5.2.1) is still significantly low despite a slight increase over time.

IMPAQ International, LLC Page 42 Beoog Biiga II Midline Report July 2017 Exhibit 32. Teachers’ Nutrition Knowledge Difference in Baseline Midline Means Indicator Total Number Total Number Percent of Percent of (p-value) Observations Observations Teachers who have heard of 4* 95% 101 99% 105 vitamin A (0.0888) Teachers who can cite one 0 78% 96 78% 104 benefit of vitamin Aa (0.9675) Teachers who can cite a food 4 71% 77 75% 104 containing vitamin A a (0.5929) Teachers who have heard of -2 98% 101 96% 105 iron (0.4375) Teachers who can cite one 18*** 69% 87 87% 101 benefit of irona (0.0023) Teachers who can cite a food 15*** 75% 69 90% 101 containing irona (0.0097) Source: Teacher Survey; IMPAQ calculations. Note: Excluded observations from baseline are teachers who did not answer these questions. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01 a Sample comprises only of teachers who had heard of the nutrient.

5.4.2 Attendance

To measure the number of teachers who teach “regularly,” at least 90 percent of normal school days during the year, we collected attendance data for a sample of 110 teachers of grades 2 to 6 in the 22 targeted schools. The data were collected from school district administrators during the months of February, March, and April 201733.

Exhibit 33 compares the three-month attendance rate of teachers at baseline and midline. On average, a somewhat larger proportion of teachers attended and taught at least 90 percent of school days at midline (88 percent) than at baseline (85 percent), but the increase is not statistically significant. Teacher attendance data should be interpreted with caution. Attendance is a direct indicator of teacher performance that teachers and school principals can be reluctant to share, as our field experience shows.34 In addition, we observed a high absence rate (25 percent) in March 2017, at midline. This high absence rate may have occurred due to a terrorist attack in the northern part of Burkina Faso during the reference period. This event could have led teachers to leave their schools or be afraid to report the following days.

33 We used this timeframe for collecting teacher’s attendance based on the performance monitoring plan (PMP), approved by USDA. Enumerators collected teacher attendance by asking school District Administrators to call principals. 34 At baseline, two school principals did not share the teacher attendance data.

IMPAQ International, LLC Page 43 Beoog Biiga II Midline Report July 2017 Exhibit 33. Teacher’s Attendance over the Past Three Months

Difference in Baseline Midline Means Indicator Total Total Percent Number of Percent Number of (p-value) Observations Observations Teachers who taught at least 90% of 3 school days in February, March, and 85% 95 88% 110 (0.3) April Source: Teacher attendance data; IMPAQ calculations. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

5.4.3 Classroom Practices and Teaching Techniques

Applying a variety of activities in class, including some that are teacher-centered, student-centered, and group-centered, is an effective way to teach children with diverse learning styles (Montgomery S. M. et al, 1998). To measure the extent to which teachers applied or planned to apply new teaching techniques and activities, we asked them about classroom practices they used or planned to use in a day or a week, including asking children to work in groups, to write, and to check each other’s work.

Exhibit 34 outlines teachers’ responses about whether they had applied or planned to apply the seven techniques taught in BBII training on the day of data collection. It also shows changes from baseline to midline. The proportion of teachers who had already done each type of activity decreased significantly (p < 0.01) from baseline to midline with one exception: asking students to check their own work. The largest decrease (41 percentage points) was in asking students to write solutions on a slate to show to the class.

By contrast, there were significant increases in the proportions of teachers who were planning to use several teaching techniques; however, pairing children with the same skill level was planned by a smaller proportion of teachers at midline (an 11 percentage point decrease). Findings on proportions of teachers who did not plan on using a particular activity are heterogeneous among the techniques. The activities with the largest increases in proportions of teachers who did not plan to use them were pairing children with same skill level (28 percentage points) and asking students to write on a slate for the class (25 percentage points). These findings are significant at the 1 percent level.

Assuming that teachers’ schedules might be different from one day to another, we also asked teachers about the activities they used, planned to use, or did not plan to use during the week. The results, reported in Exhibit 50 in Appendix C, were more or less the same as the one-day findings shown in Exhibit 34. Most teachers reported that they were planning to use most of the teaching techniques. However, there were significant decreases from baseline to midline in the percentage of teachers who had done certain activities, including having students check each other’s work (a 23 percentage point decrease).

IMPAQ International, LLC Page 44 Beoog Biiga II Midline Report July 2017 Exhibit 34. Utilization of Classroom Activities for the Day of Data Collection

Difference in Baseline Midline Means Utilization Total number Total number p-value of the Percent of Percent of test Observations Observations Individual students check their own work and give themselves a mark or comments. -8 Already used 23% 101 15% 105 (0.1692) 14** Planning to use 13% 101 27% 105 (0.0130) Not used and not planning to -6 64% 101 58% 105 use (0.3591) Students check each other' s work -25*** Already used 46% 101 21% 105 (0.0002) 14** Planning to use 22% 101 36% 105 (0.0229) Not used and not planning to 10 33% 101 43% 105 use (0.1333) Whole class checks the work of one student. -21*** Already used 43% 101 22% 105 (0.0014) 12* Planning to use 25% 101 37% 105 (0.0551) Not used and not planning to 8 33% 101 41% 105 use (0.2203) Students write solutions on a slate and show to the teacher and class. -41*** Already used 64% 101 23% 105 (0.0000) 16*** Planning to use 19% 101 35% 105 (0.0079) Not used and not planning to 25*** 17% 101 42% 105 use (0.0001) Students of different skill levels are paired together. -19*** Already used 50% 101 31% 105 (0.0051) 4 Planning to use 29% 101 33% 105 (0.4762) Not used and not planning to 14** 22% 101 36% 105 use (0.0229) Students of the same skill level are paired together. -17*** Already used 27% 101 10% 105 (0.0012) -11** Planning to use 22% 101 11% 105 (0.0270)

IMPAQ International, LLC Page 45 Beoog Biiga II Midline Report July 2017 Difference in Baseline Midline Means Utilization Total number Total number p-value of the Percent of Percent of test Observations Observations Have not and not planning to 28*** 52% 101 80% 105 use (0.0000) Teacher asks group of 3 or more students to work together on a project and later provides feedback on group performance. -26*** Already used 39% 101 13% 105 (0.0000) 5 Planning to use 21% 101 26% 105 (0.4060) Have not and not planning to 20*** 41% 101 61% 105 use (0.0033) Source: Teacher survey; IMPAQ calculations. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

After examining individual classroom activities in the survey data (Exhibit 34), we determined a threshold to indicate thorough adoption of new teaching techniques: applying or planning to apply at least five of the seven different activities on the day of data collection. Exhibit 35 shows that, at midline, teachers were almost 9 percentage points less likely than at baseline to reach this threshold; however, this difference was not statistically significant. There are some potential explanations for this decrease. As mentioned in Section 4.4, teachers at midline were less educated compared to teachers at baseline. As discussed in Section 6.2, teachers with limited education found the training materials more difficult to understand, which may have led them to continue using traditional approaches—approaches they were more accustomed to using. . Another potential explanation could be the timing of training session. As explained in Section 6.2.2, some teachers reported that training sessions were held at the end of the school year, which did not give them enough time to prepare and apply the techniques in the same academic year. This decrease, however, should be interpreted with caution since the training component of the project had not yet started at baseline. However, more than half of the surveyed teachers (53 percent) at baseline reporting use of new techniques35 may suggest that teachers were already aware of these techniques even in the absence of the training.

Exhibit 35. Teachers Using New Techniques or Tools

Baseline Midline Difference in Means Total Number Total Number Indicator Percent of Percent of (p-value) Observations Observations Teachers who -9 demonstrate use of new 53% 101 44% 105 (0.2153) techniques or tools Source: Teacher Survey; IMPAQ calculations. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

35 CRS monitoring data in 2016 also showed more or less the same value (52 percent) as the baseline value of this indicator.

IMPAQ International, LLC Page 46 Beoog Biiga II Midline Report July 2017 We assessed whether teachers were applying the BBII literacy instructional techniques to teach reading skills by asking teachers to provide examples, without enumerator prompting, of an activity or an instruction they used for teaching reading, such as phonological awareness, phonetics, vocabulary, and so on. Then, we calculated the average time that teachers reported they spent on all the literacy instruction per day. This indicator reflects both teachers’ understanding of and their practice of literacy instruction. Exhibit 36 shows that a larger proportion of teachers devoted at least 45 minutes a day to literacy instruction at midline than at baseline (11 percentage points), but the increase was not statistically significant.

Exhibit 36. Time Spent by Teachers on BBII Literacy Instruction

Difference in Indicator Baseline Midline Means Total Number Total Number Percent of Percent of (p-value) Observations Observations Proportion of teachers who 11 devote at least 45 minutes 55% 91 66% 103 (0.1160) a day to literacy instruction Source: Teacher Survey; IMPAQ calculations. Note: Excluded observations from baseline are teachers who did not answer this question. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

In addition, to understand the extent to which teachers applied literacy practices without focusing on specific techniques, we asked teachers how frequently they have students read by themselves, read to others, listen to others’ reading, write, and play word games in class. Teachers were also asked how much time they spent on each activity per week. The results are shown in Exhibit 37. There were significant increases in the proportions of teachers who had students listen and read to others (10 and 7 percentage points, respectively) at least once a week. There was no change in teachers focusing on children’s writing. Teachers’ responses were more or less the same as students’, especially in practicing listening to others read aloud.

Exhibit 37. Implementation of Literacy Activities

Baseline Midline Difference in Means Total Number Total Number Indicator Percent of Percent of (p-value) Observations Observations Students read by -4 themselves more than 94% 92 90% 38 (0.4397) once a week Students read to others 7* 87% 83 94% 89 more than once a week (0.0859) Students listen to others 11*** read aloud more than once 87% 77 98% 90 (0.0071) a week Students write more than 0 92% 99 92% 101 once a week (0.9785) Students play word games -2 83% 81 81% 91 more than once a week (0.8132)

IMPAQ International, LLC Page 47 Beoog Biiga II Midline Report July 2017 Baseline Midline Difference in Means Total Number Total Number Indicator Percent of Percent of (p-value) Observations Observations Average number of minutes spent per week on 2 60 91 62 103 all literacy-related (0.7435) activities Source: Teacher Survey; IMPAQ calculations. Note: Excluded observations from baseline are teachers who did not answer these questions. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

5.4.4 Classroom Observations

To observe whether trained teachers have been implementing the new teaching techniques and using the material provided by the BBII program, we observed 2536 classrooms in the five randomly selected schools. We also observed teachers’ activities to see if they were implementing other instructional practices beyond BBII techniques.

BBII Teaching Techniques

As explained in Section 2.1.3, we used BBII literacy training materials to develop a classroom observation tool. BBII pedagogical instructions covers two literacy components:

1. Reading (lecture) skills: decoding (déchiffrage), defined as translating written words into the sounds and meanings of spoken words (often silently), or reading comprehension (compréhension). 2. Writing (écriture) skills: spelling (orthographe) or other conventions of written texts, such as grammar; written production.37

Teachers are trained to apply decoding or reading comprehension to develop children’s reading skills. Writing and reading sessions are expected to be taught separately. Teachers usually cover a complete lesson in 6 sessions over two days. In addition, teacher training also encourages teachers to implement all the techniques by asking children to work in groups, especially for reading/decoding lessons. Reading groups were structured to let students choose a member who could read the text; pay attention; look up new words in a dictionary; answer comprehension questions; exercise syllabic awareness; and identify punctuation marks in collaboration with other group members.

The observational data showed that across all five grades in five schools, teachers mostly focused on using decoding techniques (21 lessons out of 25), particularly in grades 1 to 3. In none of the observed classrooms did the teacher use any of the writing techniques. Reading comprehension techniques were

36 We observed one teaching lesson per grade in each of the schools. The observation took between 40 to 50 minutes to ensure that the whole lesson was taught completely by the teacher. 37 Written production enables students to develop and communicate their ideas, feelings, interests, and concerns. This form of communication requires development of skills and strategies that children gradually master over time through formal education.

IMPAQ International, LLC Page 48 Beoog Biiga II Midline Report July 2017 observed in only four lessons,38 one lesson in grade 4 and three in grade 5. These findings suggest that teachers were mostly comfortable using decoding techniques.

Exhibit 38 shows the proportion of observed lessons in each grade in which teachers asked student to work in reading groups to implement BBII techniques. In 48 percent of observed lessons (12 lessons out of 25), students worked in groups focusing on reading skills. Teachers used reading groups mostly to practice decoding; reading groups were used less frequently in earlier grades.

Exhibit 38. Observed BBII Techniques Using Reading Groups All lessons Grade Reading/Comprehension Reading/Decoding observed Grade 1 (CP1) 0% 20% 20% Grade 2 (CP2) 0% 20% 20% Grade 3 (CE1) 0% 60% 60% Grade 4 (CE2) 0% 100% 80 a % Grade 5 (CM1) 67% 50% 60% All Grades 50% 50% 48% Source: Classroom observation tool; IMPAQ`s Calculations. N= 25 observed lessons (5 lessons per grade= 5*5). In 5 observed lessons in 4th grade (1 reading comprehension and 4 decoding lessons), only 4 were observed to be done in groups.

As shown in Exhibit 39, decoding and comprehension are each identified with a range of key elements that we observed in the classrooms.

Exhibit 39. Observed Reading Lesson Elements

 Combination of letters and sounds  Reading aloud  Letters recognition  Respect of liaisons Decoding Elements  Articulation of words  Respect of punctuation  Identification of breath groups  Morphemes and tool words  Fluency  Use of strategies to find answers in a text  Definition of new words  Comprehension of a whole sentence  Explicit teaching of comprehension (connectors, substitutes, punctuation, and Comprehension Elements so on)  Implicit comprehension and inference  Use of dictionary  Construction of hypothesis Source: Classroom observation tool.

38 In one of these four lessons, the teacher used comprehension techniques to remind children what was taught in the previous session.

IMPAQ International, LLC Page 49 Beoog Biiga II Midline Report July 2017 In four observed lessons which applied reading comprehension instruction, using strategies to find answers in a text (for example, searching for familiar words) was the element observed most frequently, in three out of four lessons. Asking students to make hypothesis about the reading was not observed in any lessons. In one lesson, we observed that the teacher both taught comprehension explicitly by explaining the connectors’ chronological order and by asking questions that required implicit comprehension.

Exhibit 40 shows the percentages of observed lessons in each grade in which teachers implemented key decoding elements (21 observed lessons). We observed variation in implementing the key decoding elements across grades. Teachers used a greater variety of decoding elements in the higher grades. As expected, practice with letter recognition and combination of letters and sounds were observed more frequently in lower grades. First- and second-graders did not read aloud in any of the observed lessons. Fluency was observed only in fifth grade; instruction in morphemes was not observed in any of the 21 lessons.

Exhibit 40. Key Elements Observed Reading/Decoding Lessons by Grade

Grade 1 Grade 2 Grade 3 Grade 4 Grade 5 All grades Decoding Key Element (5 lessons) (5 lessons) (5 lessons) (4 lessons) (2 lessons) (21 lessons)

Combinations 100% 80% 80% 25% 50% 71% Reading aloud 0% 0% 80% 100% 100% 48% Letter recognition 80% 80% N/Aa N/Aa N/Aa 80% Respect of liaisons 0% 0% 20% 50% 50% 19% Articulation of words 0% 40% 40% 25% 0% 24% Respect of punctuation 0% 0% 40% 0% 50% 14% Identification of breath 0% 0% 0% 25% 50% 10% groups Morphemes and tool 0% 0% 0% 0% 0% 0% words Fluency 0% 0% 0% 0% 50% 5% Source: Classroom observation tool. IMPAQ’s Calculations; a This activity is not relevant to higher grades.

These findings are in line with the qualitative data. Many teachers reported using decoding activities and reading group sessions during class useful. Teachers appreciated the reading group technique because they found that, during these sessions, students with stronger reading abilities mentor students with less- developed skills. Another useful technique from teachers’ perspective is the inference technique, in which they invite students to answer questions after reading a text. This technique allows teachers to assess students’ comprehension and their creative and critical thinking abilities. For example, teachers can ask students to create a title for a body of text or to imagine the rest of the story. Teachers reported using this technique less than the reading group sessions. Teachers reported that they used the inference technique less frequently than others, such as reading group sessions, because some students do not have the skills necessary to understand the exercise, especially those in lower grades.

IMPAQ International, LLC Page 50 Beoog Biiga II Midline Report July 2017 Other Classroom Activities and Practices To understand teacher classroom activities beyond BBII techniques, we observed other instructional practices that teachers used to develop various reading skills. While only four observed lessons out of 25 exclusively applied BBII reading comprehension techniques, the observations show that teachers integrated the comprehension elements into reading/decoding lessons, reinforcing the interconnection between the two reading skills (Exhibit 41). In 72 percent of the total observed lessons, teachers asked simple comprehension questions; in 60 percent, they asked detailed comprehension questions.

However, generally teachers did not carry out any writing activities, either in groups or individually, including spelling or written production. In 36 percent of total observed lessons, teachers asked students to encode (spell) a word or sentence. However, they did not ask students to write on a free topic or improve their own written production.

Exhibit 41. Other Observed Teacher Practices by Grade Activity Grade 1 Grade 2 Grade 3 Grade 4 Grade 5 All a. Worked with students on recognition of letters or mastery of the complete alphabet 100% 80% 20% 0% 20% 44%

b. Carried out activities on recognition of sounds of 100% 100% 40% 0% 0% 48% letters and letter combinations c. Had students decode an unknown word, sentence, or text, without repetition or 60% 80% 80% 60% 20% 60% memorization d. Had students encode a word, sentence, or text using knowledge of sounds and letters, regardless of 100% 40% 40% 0% 0% 36% spelling or other conventions of written texts e. Developed students’ lexical awareness or vocabulary: finding new words on a given subject, deducing new words’ meaning using their structure 0% 20% 20% 40% 60% 28% or the context, using tools such as dictionaries or vocabulary sheets, and so on f. Had students read a word, a sentence, or a text, 60% 60% 60% 80% 80% 68% aloud or not g. Had students read a sentence or a text with fluency and expression (respecting punctuation, not 0% 0% 80% 100% 100% 56% stopping at each word, reading for meaning) h. Asked students to answer simple comprehension question or to locate answers in a text 20% 60% 80% 100% 100% 72%

i. Asked students to answer more subtle comprehension questions or questions of implicit 0% 60% 60% 80% 100% 60% comprehension involving, for example, causes and consequences, chronology, or riddles j. Asked students to make a hypothesis about a 0% 0% 0% 0% 20% 4% text k. Asked students to establish links from a text to their real life or to another text 0% 20% 0% 20% 0% 8%

IMPAQ International, LLC Page 51 Beoog Biiga II Midline Report July 2017 Activity Grade 1 Grade 2 Grade 3 Grade 4 Grade 5 All l. Asked students to write a word or a sentence, without copying or reciting but perhaps with 0% 0% 0% 0% 0% 0% guidance such as a pattern or “mad lib” m. Asked students to write on a free topic 0% 0% 0% 0% 0% 0% n. Asked students to deduce or practice rules of 0% 0% 0% 20% 0% 4% spelling, grammar, or conjugation o. Helped students to improve their own written production for style, coherence, spelling, 0% 0% 0% 0% 0% 0% punctuation, and so on Source: Classroom observations tool; IMPAQ’s calculations.

Again, these findings are in line with what we heard during interviews with teachers. Teachers mentioned using these techniques infrequently. Teachers reported using these techniques infrequently for the following reasons:  Preparing lessons containing the new techniques can be time consuming and teachers may not have (or prioritize) the time needed to properly prepare for lessons.  Applying some of the new techniques in the classroom can be time intensive so teachers are reluctant to incorporate them into their lessons.  Some students, especially those in lower grades, do not have the skills required to fully comprehend and participate in some of the new techniques.  The training did not emphasize practice of the new techniques so some teachers feel ill-equipped to apply them in the classroom.  Some teachers did not fully understand the purpose and method of some of the techniques so have shied away from using them during lessons.

School Supplies and Learning Materials We noted what school supplies and learning materials were used during the 25 observed lessons. In all 25 classrooms, teachers used blackboards. After blackboards, small slates (used in 60 percent of observed classrooms) and large slates (44 percent of classrooms) were the most commonly used learning materials, particularly in the earlier grades as expected39. Again, interviews with teachers confirmed teachers’ preference in using blackboards and slates.

With teachers focusing more on reading/decoding techniques (21 observed lessons out 25), we expected to see teachers using words learning materials more frequently. However, observations show otherwise. Reading books were used most often in fourth-grade and never in first- or second-grade lessons. Only second-grade classes (three out of five) used reading text posters or pictures. With teachers focusing on reading/decoding techniques (21 observed lessons out of 25), we expected to see frequent use of materials to facilitate learning words, but this was not the case. Learning materials such as letter cubes and sounds tables were not used frequently during the observed lessons (only once). We did not observe teachers using Bananagrams40 or word strips (étiquettes mobiles)41 in any of the 25 classrooms.

39 Small and large slates were mostly distributed to first and second grades. 40 Bananagrams is a word game in which lettered tiles are used to spell words. 41 Translated as Étiquettes mobiles.

IMPAQ International, LLC Page 52 Beoog Biiga II Midline Report July 2017 In general, classroom observation outcomes were more or less in line with the results of student and teacher surveys. Teachers focused on reading skills, especially decoding, and children’s reading skills seem to have improved. All three data sources (classroom observation, student surveys, and teacher surveys) showed less emphasis on writing skills. This area needs further research study to probe reasons that teachers are not emphasizing writing skills as much as reading skills. If this area of focus is added to the scope of work and budget, we can increase the number of classroom observations to find out whether training was not sufficient to encourage teaching of writing skills or whether teachers are not comfortable applying what they learned.

5.4.5 Attentiveness Together with CRS, IMPAQ decided how to best measure student attentiveness. IMPAQ and CRS noted that it is very hard to judge objectively the degree to which a given child is attentive. Some children are very restless when they are attentive, while others, on the contrary, are very still. Conversely, a very still student might be sleepy and an active one might only be distracted. Hence, even classroom observations may not be a good measure of attentiveness. Moreover, classroom observations are tricky because they must be completed in exactly the same circumstances in each classroom in order to get quality, comparable data. IMPAQ and CRS, therefore, decided that the best measure of a child’s attentiveness was the teacher’s own assessment of his or her students’ attentiveness. To measure this indicator, we collect data from teachers who are best able to judge the relative degree of the students’ attentiveness. We asked teachers on a scale of 1 to 10, where 1 is not attentive at all, to rate their students’ attentiveness. Exhibit 42 shows that according to teachers, students’ attentiveness have not changed over time. However, given the difficulty and subjectivity of measuring attentiveness, this outcome should be interpreted with caution.

Exhibit 42. Students’ Attentiveness Reported by Teachers Baseline Midline Difference in Means Total Total Indicator Percen Perce Number of Number of (p-value) t nt Observations Observations Proportion of attentive students 2 45% 95 47% 105 identified by teachers (.8433) Source: Teacher survey; IMPAQ’s calculations.

5.5 PTA Outcomes

To measure the level of community involvement with children’s schooling, we surveyed the PTA leader from each school. We asked these leaders questions related to PTA meetings held in the past year, the functioning of the school canteen, and the extent of community support for the school. We also asked them about the number of months in which the school canteen was covered by MENA, CRS, parents, and other responsible parties. It is important to note that the results presented in this section rely on a very small sample of PTA leaders; estimates for the indicators and their change over time have to be taken with caution.

In general, the data do not suggest much change in the number of meetings held by PTA members over time. Exhibit 43 shows that there was no change in the average number of general assemblies or PTA council meetings, as expected. At midline, less schools (8 percentage points) held at least three general

IMPAQ International, LLC Page 53 Beoog Biiga II Midline Report July 2017 assembly meetings compared to baseline but not statistically significant at any conventional levels. A smaller proportion of PTA members attended general assemblies at midline than at baseline, but the decrease of 9 percentage points was not statistically significant.

Exhibit 43. Parent-Teacher Association Meetings

Baseline Midline Difference in Means Total Total Indicator Mean/ Mean/ Number of Number of (p-value) Percent Percent Observations Observations Number of general assembly 0 3 21 3 22 meetings held in past 6 months (0.3481) Schools that held at least 3 8 general assembly meetings in 71% 21 68% 22 (.8220) past 6 months PTA representatives who -9 100% 25 91% 22 attended a general assembly (0.1288) Number of PTA council meetings 0 4 21 4 21 held past school year (0.9452) Source: PTA Survey; IMPAQ Calculations * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

As shown in Exhibit 44, the proportion of PTA representatives who were involved in many school- community activities decreased from baseline to midline. Having a meal with students at the school canteen decreased by 4 percentage points, participating in a school project by 1, and visiting a classroom by 12; however, the changes were not statistically significant. A marginally statistically significant decline (p < 0.10) is seen in the proportion of PTA representatives who served as cooks or storekeepers, a decrease of 25 percentage points. One potential explanation to this significant decrease is that PTA leaders might focus more on leadership responsibilities at midline, while these roles were filled by other PTA members compared to baseline. These results should be interpreted with caution given the small sample of PTA members. We recommend qualitative research at endline in this area to probe deeper into potential issues. The one area in which the proportion of PTA representatives involved in the school increased significantly (p < 0.10) is seeing a performance by the children, a difference of 29 percentage points.

Exhibit 44. PTA Representative Involvement in the Past 12 Months

Difference in Baseline Midline Means Indicator Total Total Percent Number of Percent Number of (p-value) Observations Observations PTA representatives who have -4 had a meal with students at 4% 25 0% 22 (.35) school PTA representatives who have participated in school- -1 92% 25 91% 22 community projects such as (.9) cleaning

IMPAQ International, LLC Page 54 Beoog Biiga II Midline Report July 2017 Difference in Baseline Midline Means Indicator Total Total Percent Number of Percent Number of (p-value) Observations Observations PTA representatives who have -25* helped the school as cook or 52% 25 27% 22 (.09) storekeeper PTA representatives who have 9 helped watch over a reading 0% 25 9% 22 (.13) group PTA representatives who have -12 76% 25 64% 22 visited a classroom (.37) PTA representatives who have 29* attended a performance put 44% 25 73% 22 (.05) on by students PTA representatives who have -3 8% 25 5% 22 helped in other ways (.64) Source: PTA Survey; IMPAQ Calculations. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

Exhibit 45 shows PTA representatives’ reports on the total number of months in which the canteen was supported by MENA, CRS, parents, and other responsible parties. There is a significant decrease (by 2 months) at midline. This outcome should be interpreted with caution, because there is a possibility that canteen operations were misrepresented at baseline. PTA members over-reported at baseline the total number of months during which the government supported the canteen.

Exhibit 45. Support for Canteens

Baseline Midline Difference in Means Total Number Total Number Indicator Mean of Mean of (p-value) Observations Observations Number of months of community -2* and/or government support for 5 14 3 15 (.06) canteens Source: PTA Survey; IMPAQ Calculations. Excluded observations are the PTA members who did not know the answer. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

5.6 School District Administrator Outcomes

In the BBII program, school district administrators have been responsible for training teachers on pedagogical practices. To gauge their level of confidence in and knowledge of teacher training programs, we asked district administrators about the importance of each literacy instruction subject they taught, the number of training sessions they held, and any supervision or observational activities they conducted. Due to the small sample at the baseline, the results presented below are merely descriptive; no statistical analysis (t-test of difference in means) can be conducted.

Exhibit 46 shows which reading instructions district administrators said were most important. Compared to baseline, a larger proportion of administrators reported that phonemics (hearing, identifying, and

IMPAQ International, LLC Page 55 Beoog Biiga II Midline Report July 2017 manipulating individual sounds) is an important reading technique. Phonemics is considered to be the basis for learning phonics. By contrast, the proportions of administrators who mentioned phonics and fluency as important reading instructions decreased considerably.

Exhibit 46. Important Reading Instructions Reported by District Administrators

Baseline Midline Total Number Total Number Technique Percent of Percent of Observations Observations Phonemic awareness 62% 13 73% 30 Phonics 100% 13 27% 30 Fluency 100% 13 67% 30 Vocabulary 62% 13 53% 30 Comprehension 85% 13 90% 30 Other techniques N/A* N/A * 13% 30 None of the Proposed Techniques N/A * N/A * 10% 30 *Option not available in baseline. Source: School district administrator survey; author’s calculations. At midline, almost all (29 out of 30) district administrators said that they held at least one teacher training on reading skills, in the previous school year. IMPAQ defined this indicator, demonstrating use of new techniques or tools, as school district mangers transferring skills to others through trainings. They reported an average of four training sessions with 122 teacher trainees. The number of district administrators using new techniques improved over time; only 8 out of 13 administrators surveyed held such training sessions at baseline.

Furthermore, nearly all of the administrators (29) said they had observed, supervised, and/or monitored teachers in their classes during the previous academic year, conducting an average of 26 visits lasting an average of 85 minutes each. At baseline, only 9 out of 13 administrators reported that they had observed classrooms. At midline, administrators reported that they had seen 81 percent of teachers using at least one of the BBII literacy activities (writing, listening to others reading, reading to others, reading alone, and playing with words). This proportion is similar to what administrators reported about classroom observations at baseline.

In interviews, teachers reported that they appreciate the teacher observations conducted by school district administrators, as they receive important technical support on teaching techniques and feedback on techniques that they have not yet mastered. Teachers also said that observations prompt them to focus on the quality of their teaching and follow the practices and standards they learned from the literacy trainings.

IMPAQ International, LLC Page 56 Beoog Biiga II Midline Report July 2017 SECTION 6. QUALITATIVE FINDINGS

Below is a summary of the main findings by the qualitative research domain. More detail regarding these results is elaborated in the narrative that follows.

Summary of Main Qualitative Findings Relevance Overall, respondents reported satisfaction with the project, saying they that believed the project interventions are meeting the needs of beneficiaries and are aligned with Burkina Faso’s and the U.S. government’s development objectives. Respondents also believed that the project adequately considers economic, cultural, and political contexts. According to respondents, the primary strengths of the project design include its holistic approach to child development, its focus on evidence-based literacy techniques and provision of teaching resources, and its focus on community and government engagement. The primary weaknesses, according to stakeholders, include little ability to deal with lack of motivation on the part of teachers, lack of key equipment and resources for teacher monitoring and support, and the design of reading clubs. Effectiveness Respondents reported that immediate outcomes were reached in most dimensions of the project and that the interventions have contributed to expected results. Teachers reported that they have observed improvements in students’ literacy skills and motivation to learn. Teachers and students also reported that hygiene education has changed students’ behavior, as they are now washing their hands more frequently. Students appreciated the school meals and take-home rations received from the canteens, and teachers believed school attendance was closely tied to the presence of canteens. Though challenges exist, respondents said that CRS has effectively managed partners and that the program has coordinated and collaborated effectively with other stakeholders. Efficiency Respondents reported that the project resources have been sufficient to implement scheduled activities and, for the most part, objectives have been achieved on time. Some activities, including the provision of food in canteens and the provision of training materials, have been delayed due to slow administrative processes. Further, respondents noted that partners’ competing agendas and priorities sometimes exacerbate these delays. Despite challenges, respondents reported that CRS has adequately responded to internal and external factors that have hindered the efficient implementation of project activities. Perceived Impact Respondents believed that the project is on target to meet its medium- and long-term goals, particularly those related to students’ literacy outcomes, and that the effects are due to the project interventions. As for hygiene-related outcomes, respondents noticed positive changes at some schools related to students’ hygiene habits. Respondents believed that, if delivered properly, the educational activities related to hygiene and nutrition will strengthen students’ skills and knowledge. However, respondents described that limitations such as teachers’ lack of motivation to teach hygiene and nutrition knowledge and the lack of clean water sources and latrines may result in less significant outcomes. Respondents recommended several strategies to improve the impact of the project. Recommendations related to literacy included modifications to the teacher trainings to ensure that they are long enough to cover all content and allow sufficient time for practice, development of a motivational framework for teachers that focuses on incentives to implement new literacy techniques, and provision of sufficient materials and resources to enable teachers and students to implement these techniques. Recommendations related to health activities included engaging more teachers in health trainings to further promote the spread of information and ensuring that infrastructure, such as handwashing stations, are operational.

IMPAQ International, LLC Page 57 Beoog Biiga II Midline Report July 2017 Summary of Main Qualitative Findings Sustainability Respondents believed that the engagement of communities and the government are key factors to sustainability and that CRS and its partners have successfully engaged these parties. There is still work to be done, however, to ensure project activities can be sustained after USDA funding ends. Respondents reported a need to organize a workshop with all relevant players to develop sustainability strategies and define how all parties can play a role in sustainability.

6.1 Relevance

Through interviews with USDA, MENA, project implementers, and county mayors, the research team assessed the relevance of BBII interventions. Interview topics focused on stakeholders’ perspectives on the strengths and weaknesses of the project design and the extent to which the project considers economic, cultural, and political contexts. Stakeholders were also asked to share their perceptions on the extent to which the interventions are meeting the needs of beneficiaries and are aligned with Burkina Faso’s education and development strategies and with the U.S. government’s development objectives. Lastly, stakeholders shared their satisfaction with intervention activities. Below is a summary of the qualitative findings on the relevance of BBII.

6.1.1 Strengths and Weaknesses of Project Design

Stakeholder respondents identified several strengths and weakness of BBII’s design. Below is a summary of their responses.

Strengths of the Project Design Interviewees described several strengths that are helping BBII to meet its goals.

Holistic approach of the project. Respondents believed that the project’s theory of change holistically and adequately captures factors, inputs, and activities that will lead to the intended long-term outcomes of the project. MENA representatives and project implementers said that the project design appropriately addresses the cyclical nature of student learning and health outcomes and, most importantly, involves the actors who are most vital to its success, such as teachers, PTA leaders, and community members. For example, understanding that students’ school performance will be negatively affected if they are hungry or sick due to hygiene-related diseases, the project encourages parents and community members to play an active role in encouraging their children to go to school, where they participate in literacy and health- related education and receive supplemental food.

Building on the BBI framework. Some project implementers thought that a strength of the BBII project is that the design relies on the findings and lessons learned from the evaluation of its predecessor, BBI. Building on best practices from BBI enabled CRS to hit the ground running and enhance its previous work to improve outcomes for children, respondents believed.

Focus on literacy and teaching approach. All types of respondents reported that the most important strengths of the project design are the activities associated with improving students’ literacy abilities and the approach used to teach these literacy skills to students. Respondents believed that focusing on literacy abilities will have a positive impact not only on students’ ability to read and write but also on their ability to think critically across all disciplines. For example, as students improve their ability to decode and

IMPAQ International, LLC Page 58 Beoog Biiga II Midline Report July 2017 understand text, they will be better able to comprehend disciplines like history, geography, science, and mathematics. Thus, focusing on student literacy is an effective way to improve overall learning outcomes, respondents explained.

A related strength noted by teachers and school administrators is the approach the project uses to teach literacy, which respondents believe is more innovative and applicable than what is typically implemented by teachers. CRS and its partners use evidence-based techniques that come from the findings of education experts. The approach takes into account differences in students’ learning aptitudes. Further, the techniques promote the use of examples from students’ daily life to make reading and writing exercises relevant and tangible to students. Teachers found that these real-world examples actively engage students in learning, leading to improved outcomes.

Alignment with education ministry strategic priorities. Another strength project implementers and MENA staff frequently mentioned was the project’s alignment with the strategic priorities of MENA. Respondents stated that the project effectively embraces MENA’s education objectives and actively encourages MENA’s involvement and participation. In partnership with CRS, MENA leads many field activities. For example, the training of teachers on the ground is conducted by MENA’s school district administrators. All types of respondents, including the USDA representative, believed MENA’s involvement in the implementation of teacher training — a key component of the project — is an important strength of the project’s design, as this partnership is important to sustainability.

Teacher monitoring and support by school district administrators. MENA representatives mentioned that the BBII project does not limit its interventions to rolling out activities only; instead, subsequent follow-up is an integral part of the project design, particularly for the literacy component. In the design, school district administrators follow up and support teachers in the field as they implement the new literacy teaching techniques. Project implementers, MENA representatives, and teachers perceived this continued engagement of school district administrators as an important strength of the design, as it actively engages teachers in integrating the techniques, improves the quality of classroom management and instruction through quality monitoring, and provides hands-on support to teachers who experience challenges.

Provision of learning materials. Respondents, particularly teachers, school administrators, and students, stated that the provision of training materials to teachers and learning materials to students is an aspect of the project that is valued by teachers and students alike. Respondents noted that such materials were greatly needed in classrooms, as they help facilitate teaching and learning.

Weaknesses of the Project Design Respondents also identified weaknesses in the project design that may impede progress toward project goals. Those weaknesses relate to weak motivation of key actors, lack of key equipment and supplies for teacher monitoring and support, and the design of reading clubs.

Weak motivation of key actors. During interviews, all types of respondents recognized the benefit of the project to communities and families and the project’s thoughtful design and approach to addressing relevant educational outcomes. However, respondents reported that some activities suffer from a lack of commitment from teachers. For example, some respondents noted that, unless teachers recognize the benefit of the new teaching techniques, they may not integrate the techniques into their daily lessons. Respondents believed that the project design did not adequately take into account teachers’ potential lack of motivation and fidelity to the model.

IMPAQ International, LLC Page 59 Beoog Biiga II Midline Report July 2017

Lack of key equipment and supplies for teacher monitoring and support. Respondents reported that school district administrators sometimes lack the equipment they need to conduct scheduled school visits. For example, in some districts, mopeds dedicated to visits are frequently out of service, so that school district administrators must use their own means of transportation or postpone the trip. It is important to note that these supplies are provided by MENA and not directly via the BBII project. However, respondents described that resource limitations sometimes impact their ability to fulfill their BBII-related mission in the field.

Further, respondents reported that teachers sometimes lack classroom materials, limiting their ability to implement new teaching techniques. For example, some teachers said they do not have enough books and teaching tools, such as large slates, to accommodate all of their students.

Design of reading clubs. Teachers expressed that they have experienced challenges in finding volunteers willing and able to organize and facilitate reading clubs in the villages. Though the reading groups were intended to be led by teachers, secondary school students, or community mentors, teachers found that secondary school students do not have the skills necessary to facilitate these groups. Further, teachers reported that some parents do not allow their children to attend reading clubs, as they do not understand the importance of the activity and prefer that their children take care of domestic chores when not in formal school. Respondents believed that the design of the reading clubs did not anticipate the impact of these challenges on the clubs’ success or fully take into account the local context, including stakeholders’ priorities.

6.1.2 Alignment with Economic, Cultural, and Political Contexts Many respondents believed that the project effectively takes into account economic, cultural, and political contexts. For example, respondents explained that, for every BBII field activity, CRS field staff initiate talks with local communities to inform them of the activity and obtain their feedback and suggestions for an appropriate and effective roll-out of the activity in the local context. Furthermore, because the targeted areas continuously suffer from drought and food shortage, respondents said that the provision of food by CRS through school canteens is pertinent and important.

6.1.3 Alignment with Burkina Faso and U.S. Government Priorities When asked if the BBII project successfully aligns with Burkina Faso’s “The BBII project clearly aligns strategic goals related to education and development, project with [the] MENA strategic implementers and MENA representatives stated that the BBII project priority … to provide high- embraces the strategic priorities of Burkina Faso’s education ministry quality, inclusive education of and found this alignment to be a strength of the project. The USDA good quality for all.” representative believed the project closely aligns with USDA and U.S. –MENA Representative government priorities. The literacy and school feeding components of the project are directly in line with priorities set by the Food for Education program at USDA.

6.1.4 Stakeholder Satisfaction CRS and its implementing partners reported high satisfaction with the BBII project, noting that it is well designed and implemented. Implementing partners reported satisfaction with CRS’ performance in implementing the project and managing stakeholders. All interviewed stakeholders emphasized that they hope the project continues, as they believe it has had a positive impact on children and communities, as discussed in more detail in Section 6.4 below.

IMPAQ International, LLC Page 60 Beoog Biiga II Midline Report July 2017

“I wanted to congratulate CRS Teachers reported mixed satisfaction with various aspects of the for their work on the ground. project. Teachers reported high satisfaction with the content of the They should work to ensure that literacy trainings and liked the design and content of the new teaching municipalities understand their approaches and techniques. However, many teachers reported low concepts and endorse the satisfaction with the conditions of the literacy trainings, describing that project. They can count on our the trainings were too short for the amount of content, the training engagement.” spaces were inadequate (for example, the seats were for children, not –County Mayor comfortable for adults), and the per diem allowance, for many, was not enough to cover all expenses, particularly for “residents” as described in detail in Section 6.2.2.

Teachers also reported mixed levels of satisfaction with the teacher observations. Several teachers described that the feedback given by the school district administrators during their visits was helpful and prompted them to prepare lessons and use techniques learned through the literacy trainings. However, some teachers also reported that these visits often disrupted their classroom agendas, primarily due to lack of communication about when the visits would take place. Often, teachers did not learn about visits until that day, when it was too late for teachers to ensure that their daily agenda was planned smoothly.

Overall, teachers said that they appreciated the project’s immediate impact on their students’ behavior and literacy outcomes. As explained in Section 6.2 below, teachers described anecdotally positive changes they had seen in students’ literacy skills, changes that they believe will improve students’ performance in other disciplines.

Students reported high satisfaction with the canteens, handwashing stations, and literary materials and tools. They appreciated and enjoyed the food received through the canteens and enjoyed using the handwashing stations and classroom materials. As described in the limitations section (Section 7.2), students did not discuss the new literacy techniques in detail. When we asked questions about teaching techniques, students were shy and unwilling to describe their experiences. Although students’ satisfaction was generally high, a handful of students from one focus group did report dissatisfaction with the micronutrients, saying that they experienced headaches and sometimes vomiting when taking them.

6.2 Effectiveness Through interviews with the stakeholder groups described in Section “I give my congratulations to 2.2, the research team assessed the effectiveness of implementation CRS for its vision about the strategies and activities. Interview topics focused on the extent to project and for all the support they brought us, and encourage which the project is achieving its objectives, successes and challenges CRS to continue in that way. of program implementation, and the effectiveness of management They have as priority the future arrangements and stakeholder collaboration. Below is a summary of of kids … whom they will the qualitative findings on project effectiveness. positively impact.” –MENA Representative 6.2.1. Achievement of Objectives Respondents from CRS and its partners (MENA, OCADES, and FAVL) reported that the project has achieved its immediate goals in terms of implementation activities and is on track to achieve its longer-term objectives. Project implementers and school-level actors, including teachers and school administrators, reported observing positive changes, such as improvements in students’ literacy skills and behavior related to hygiene, such as handwashing.

IMPAQ International, LLC Page 61 Beoog Biiga II Midline Report July 2017

A summary of findings for activities associated with the two primary strategic outcomes—improved literacy and increased use of health and dietary practices—is presented below.

Strategic Objective 1: Improved Literacy of School-Age Children All teachers and school administrators who participated in interviews believed that the project is meeting its goals in terms of improved literacy of children. Teachers maintained that the new training techniques have a beneficial effect on student literacy. Because the teaching approach joins reading and writing skills together, respondents reported that it is both efficient and effective: Students learn how to read and write at the same time, whereas, in the traditional practice, the two tasks were taught separately. Interestingly, however, classroom observations and the quantitative results suggest that teachers pay less attention to writing skills and are more focused on reading skills. They may do so because the ASER test focuses on reading abilities, so they are more concerned with improving students’ reading skills than writing abilities.

Furthermore, as mentioned in Section 6.1.1, the techniques use a real-life approach, which teachers believe is beneficial for students. Teachers are trained to use common, real-world examples when teaching students how to pronounce and write letters and words. Teachers appreciate that the new training techniques focus on the process of student learning and encourage students to learn by engaging in the lessons through play and practice. Interestingly, the classroom “I use Bananagrams. I write a observations showed otherwise; few teachers used real-life examples word and ask them to copy this during lessons. word on their slates. After, I ask them to try building other words Teacher respondents shared with these letters using the “After a reading, I do a techniques they found to be the Bananagrams.” dictation, and I do oral most useful in their classrooms; –Teacher in Bam Province expression, asking them to build their responses align with the sentences with words. I usually invite them to copy some words findings of the classroom observations. Many teachers reported using after reading.” a decoding activity that consists of assisting students to recognize and –Teacher in Bam Province read letters. Another widely applied technique that teachers found useful was reading group sessions during class. This activity consists of organizing students into small groups and asking them to take turns reading a text. Teachers appreciated this technique because they found that, during these sessions, students with stronger reading abilities mentor students with less-developed skills.

Another useful technique from teachers’ perspective is the inference technique, in which they invite students to answer questions after reading a text. This technique allows teachers to assess students’ comprehension and their creative and critical thinking abilities. For example, teachers can ask students to create a title for a body of text or to imagine the rest of the story. Teachers reported using this technique less than the reading group sessions.

Other techniques, including substitution, were mentioned less often than the techniques described above. Substitution is an exercise in which students are asked to find appropriate words to replace subjects or other words in a text. A technique that was not mentioned by any of the teachers was systematic screening, which consists of a weekly assessment of students’ reading ability. This technique, in principle, should enable teachers to identify students who need more practice and support and to create a plan for improvement. However, this technique was not applied by the teachers we interviewed. Classroom observations confirmed that these techniques were less commonly used by teachers.

IMPAQ International, LLC Page 62 Beoog Biiga II Midline Report July 2017 A sister activity to classroom reading group sessions is the reading clubs designed as part of the BBII project and implemented in targeted schools. In these reading clubs, groups of students meet in the village after school hours to practice reading under the guidance of a volunteer facilitator. Teachers said that the reading clubs enable students with more advanced literacy skills to mentor students with less developed skills. Teachers reported that students who participate in a reading club that is facilitated by someone other than a teacher feel more comfortable participating and asking questions.

Teachers also reported challenges with the reading clubs. Teachers explained that a lack of specific kinds of books to facilitate reading groups is a problem and that using the same books in reading clubs and classrooms does not motivate students, since the books teachers received through the project for classroom work were more like textbooks than reading books. Teachers noted that students would be more motivated to join reading clubs if there were different types of books to read, such as fictional novels and short stories. Further, teachers reported that most reading club members are not able to access the libraries due to long distances between villages and the capital town where libraries were established.

Another limiting factor of reading clubs is the lack of quality facilitators. Teachers reported difficulty in finding facilitators from the communities because it is a volunteer activity; also, community members are often illiterate. When volunteers do accept the role, they often do not show up for the reading clubs, abandoning the students. Finally, respondents reported that some parents do not allow their children to attend reading clubs as they do not understand the importance of the activity and prefer that their children take care of domestic activities when not in formal school.

Teachers said that classroom literacy activities are facilitated by valuable training materials. Teachers reported that they were particularly fond of the large slates and blackboards. Classroom observations confirmed that teachers use these materials more frequently than others. For example, large slates are generally used during reading group sessions, and their usefulness was emphasized by all teacher respondents. In fact, the slates are the most commonly used material, according to teachers. From teachers’ perspective, the large slates facilitate group work much better than the small slates that individual students use. The new large slates can display many words and sentences, and everyone in the group can see what is written. Additionally, teachers say that the larger slates are often a good substitute for chalkboards, as they are portable. Students who are toward the back of large classrooms sometimes struggle to see what the teacher has written on the chalkboard, so they have difficulty in decoding letters and words.

In addition to the large slates, teachers noted that they appreciate the reading books that CRS distributes. Reading books contain texts, such as stories, and comprehension activities. Teachers reported that these materials are beneficial because students are able to use them to practice reading individually or in groups and because students are generally interested in using them. Teachers’ responses suggest that tools such as Bananagrams, letter cubes,42 and dictionaries are less frequently used than the tools mentioned above. According to teachers, these tools require more time to use in the classroom than blackboards and slates. These findings are in line with classroom observations.

Teachers reported, however, that lack of teaching resources, including the limited number of large slates and reading books, is a challenge. For example, teachers described classes of 60 students with only six reading books and classes of 120 students with only five large slates. Because teachers lack sufficient supplies, students cannot take full advantage of learning practices. Further, respondents reported that

42 Respondents reported that cubes have not yet been distributed to all schools.

IMPAQ International, LLC Page 63 Beoog Biiga II Midline Report July 2017 reading books and other teaching materials, like dictionaries, letter cubes, and Bananagrams sets, are often stored in school administrators’ offices instead of being stored in classrooms where students can have easy access to them.

As mentioned before, teachers also discussed the teacher observations conducted by school district administrators. Overall, teachers reported that, through this monitoring, they receive important technical support on teaching techniques and receive feedback on techniques that they have not yet mastered. Teachers also said that observations prompt them to focus on the quality of their teaching and follow the practices and standards they learned from the literacy trainings.

Because teachers found the close monitoring of school district administrators to be helpful, some teachers regretted that school district administrators sometimes conduct their visits several months after the start of school year and even, in some cases, near the end of the school year. Further, in contrast to implementers’ suggestion to perform random observations, teachers suggested that they receive notification several days in advance of observation visits, as these visits often disrupt their classroom agendas. Again, it is important to note that school district administrator observations are overseen and funded by MENA; CRS and its partners work with MENA to reinforce these activities.

Another constraint to the achievement of literacy objectives reported by respondents is the distance to libraries. School-level respondents stated there are no libraries in their communities. In the rare cases that libraries are present, they are located in the main town, not in the village itself, making it difficult to use these places for learning purposes, especially since some villages are 50 kilometers or more from the main town. Interviews with teachers revealed that libraries are not often visited by teachers or students for educational purposes unless the school is close to the library, which is uncommon. While it is important to note that CRS and FAVL intended for libraries to be located in the capital town and not close to all schools, many teachers reported a desire to have access to libraries and believed libraries would serve as a useful tool to teacher development and in classroom teaching.

Conscious of this difficulty, FAVL has initiated mobile libraries and reading camps to increase access to library services. Mobile libraries consist of FAVL staff visiting schools with a package of books and other reading materials for students to browse on location. Reading camps are organized activities during school breaks that are dedicated to reading in a fun and playful way. Reading camps include activities such as competitions to reward outstanding readers. Teachers and FAVL staff reported that students enjoy the reading camps, which incite interest in reading. As one respondent noted, “Everyone wants to be valued in the village as the best reader.” During interviews, teachers and students asked that reading camps be continued. They emphasized that, even if FAVL could not continue to provide participants with breakfast as it has done in the past, students would attend the camps anyway. Furthermore, teachers reported that some students who are usually late for school are the first ones to arrive at reading camps.

Teachers and school administrators reported having observed meaningful positive results as a result of the new techniques and materials implemented for this project. Teacher respondents emphasized that the project has improved students’ motivation and interest in reading and writing. Teachers said that decoding techniques have enabled younger students from lower grades to recognize letters of the alphabet and correctly spell basic words. For those in the higher grades, application of the techniques permitted students who previously were not able to read to start becoming more comfortable with reading and writing. From the teachers’ perspectives, students have become more enthusiastic about reading and writing. Students who struggled to read are now motivated to perform as well as the best readers in their class.

IMPAQ International, LLC Page 64 Beoog Biiga II Midline Report July 2017

Strategic Objective 2: Increased Use of Health and Dietary Practices In this project, teachers have been taught health, hygiene, and dietary practices to transfer to students. Teacher respondents who attended the trainings reported that the trainings were useful. Some teachers recognized that they did not know how to wash their hands properly prior to the training and appreciated the opportunity to learn the technique and teach it to students. Children reported enjoying the handwashing practice; they liked going to the handwashing station during classroom breaks and before and after eating. Some schools reported that students’ health practices have changed since the start of the project. An example is the school of Foulou in Bam Provence, where respondents expressed that students have adopted handwashing as a reflex.

Teachers reported that dietary teachings about vitamins and minerals are taught together with the handwashing practices. Both teachers and students found these teachings to be useful, though it was unclear from the interviews whether and how students apply these lessons to their everyday lives.

Regarding the canteens, respondents reported that regular school meals and take-home rations are an important aspect of the project. Respondents described delays in commodities getting to schools; some canteens received foodstuffs late in the school year, limiting their ability to feed students as planned. Respondents reported that these delays were due to delays in government payment to food carriers. CRS addressed the issue, however, and worked with the government to allay future transportation payment delays. Regardless of the delays, students expressed much appreciation for the food they received through the canteens and did not mention delays. They stated that the cooked food is tasty and that they often take some home to share with their siblings and parents.

Many types of respondents, including teachers, county mayors, and school administrators, reported limitations of the health and dietary activities that may be preventing the achievement of better results. The first limitation is related to teachers’ motivation to teach handwashing, hygiene, and nutrition knowledge. At each school, two people are trained to educate students about health and dietary practices: the school administrator and one teacher who has been designated as the “cleanliness lead” by the school. Because only two individuals per school are trained, respondents reported, sometimes teachers who were not trained do not want to participate in health and dietary activities. According to some respondents, these teachers are “boycotting” the activities because they did not attend the training and therefore did not receive per diem payments.

As long as facilities are available at the school, respondents reported, students apply handwashing techniques and use latrines. But some schools lack water sources and latrines. In the same vein, the handwashing devices may fall into disrepair, and neither the school nor the PTA members replace them. Finally, the socioeconomic status of some parents limits students’ ability to practice positive health and dietary behaviors if they do not have clean water, soap, or appropriate latrines at home. However, respondents reported that students sometimes positively influence their parents’ behavior. For example, some students pass on information about the benefits of washing hands with soap to their parents and request that their parents buy soap and practice with them.

Regarding dietary supplements, as mentioned in Section 6.1.4, a handful of interviewed students from one focus group reported that they stopped taking the micronutrients that teachers gave them because the supplements caused nausea or headaches.

IMPAQ International, LLC Page 65 Beoog Biiga II Midline Report July 2017 6.2.2 Successes and Challenges of Program Implementation Respondents described successes and challenges in effectively implementing BBII. Below is a summary of their responses.

Successes of Program Implementation Respondents emphasized that the involvement of communities in specific project activities is an important and resounding success of the project to date. Mentoring activities were seen as particularly successful by implementers. Mentoring consists of community women supporting girls by providing advice and serving as an advocate for them with their communities and families. Respondents reported that, because of the mentoring program, many girls who became pregnant remained in school.

In addition to community involvement in mentoring activities, another success reported by stakeholders is communities’ engagement in SILC activities, in which community members are trained to set up financial cooperative groups. The funds that these groups raise enable them to better plan and pay for school- related expenses. Project implementers reported that several parents have been able to buy school supplies and materials, such as books or a bag, for their children with money from SILC activities.

According to respondents, an important factor that has helped the project reach success related to health and hygiene practices and attitudes is the dual implementation of BBII and another CRS project called Koom Yilma, which supports the provision of water sources and toilets in schools. Respondents including county mayors and MENA staff stated that the BBII schools that also participate in Koom Yilma have seen “significant positive results toward practice and attitudes of students,” as one MENA staff member put it.

Challenges of Program Implementation The main challenge of the BBII project to date has been gaining sustained buy-in and engagement of the project’s central actors—the teachers. Respondents reported that, unless teachers fully recognize the benefits of the new teaching techniques, they are unlikely to use them in the classroom and instead will rely on the traditional approach. Some teachers reported that the new techniques are too time consuming in terms of preparation and application and that students are not yet comfortable with these practices. Further, stakeholders noted that some teachers do not consistently prepare for class and because the new teaching techniques require preparation, these teachers do not incorporate the techniques into their daily classroom lessons.

Another challenge is related to teacher training. All interviewed “The workload for the training is teachers and some school district administrators said that the workload too much. It doesn’t enable during the training is too large but the training is too short. In seven teachers to learn.” days, many topics are taught: literacy, hygiene, health and nutrition, –School District Administrator community capacities, and reading groups, among others. Teachers reported that a week is not enough time to enable them to fully understand all the topics in order to adequately integrate them when they return to their schools. A related problem is the conditions in which training takes place. Some teachers reported that the training classrooms, which are the same classrooms used for the students, are cramped and the seats are too small for adults, making it difficult to concentrate. Additionally, teachers said that the trainings sometimes occur late in the school year; they would prefer to receive training at the beginning of the school year in order to implement the techniques immediately.

A challenge reported by some school district administrators is that the concepts and techniques of the teacher training are too advanced for some teachers. Because teachers do not all have the same skills and

IMPAQ International, LLC Page 66 Beoog Biiga II Midline Report July 2017 aptitudes, school district administrators described how trainers sometimes have difficulty ensuring that all teachers understand the topics discussed during the trainings.

According to respondents, teachers are not the only group who need additional training and support; though parents receive some guidance, respondents reported that parents need additional training through the PTA and school management committees, as well as SILCs. For example, respondents said that students’ parents experience challenges with the use of performance tools distributed by teachers. The performance tools include canteen tracking files from the PTA management and performance records from teachers. Though parents receive training to read and use these tools to support their children, respondents explained that many parents still do not know how to interpret the tools. Respondents also reported that SILC leaders often struggle with management of group activities and SILC methodology and require further support to run the groups effectively.

Further, respondents reported that some teachers have refused to participate in CRS trainings because they believe the per diem payment for training is inadequate to cover all of their expenses. For example, CRS determines whether teachers are “residents” or “non-residents” based on the distance between their homes and the training site. Teachers designated as “residents” do not receive per diem payments for accommodations. It is important to note that CRS’s per diem policy and rates align with the government’s policy and rates and that CRS chose these rates for sustainability purposes. However, according to some respondents, some teachers who are designated as “residents” live too far from the training site to return home each night and, therefore, are not able to attend the trainings due to financial constraints. It is not clear how many teachers have refused to participate in CRS trainings because of financial limitations.

Teacher absenteeism may also present challenges to implementation of BBII activities. Although the quantitative data showed high teacher attendance, the quality of the attendance data is uncertain given the sensitive nature of teacher absenteeism at schools. Further investigation is needed to understand the scope of teacher absenteeism, as the field visits did not present a clear picture of the issue. Some school district administrators or school administrators do not keep records of absentees, and respondents reported varying levels of absenteeism.

6.2.3 Internal Collaboration and Management From respondents’ perspectives, collaboration and management of the project among internal stakeholders at CRS has been successful and effective. CRS holds regular meetings to coordinate activities, address issues as they arise, and manage day-to-day operations. Field issues are brought to top-level management through regular monitoring activities conducted by CRS’s Monitoring, Evaluation, Accountability, and Learning division. This division helps project partners to quality-check the work implemented on the ground and make sure the activities are reaching their targets. According to implementers, this mechanism has been a valuable tool to help implement the project.

Respondents reported that all CRS staff are actively engaged in project implementation and monitoring and believe that, if staffing continues to remain stable, positive results will be achieved.

IMPAQ International, LLC Page 67 Beoog Biiga II Midline Report July 2017 6.2.4 Collaboration with Project Stakeholders Respondents reported that collaboration among project stakeholders, including CRS, FAVL, OCADES, and MENA, has been positive. The main concern project implementers shared is related to overlapping agendas and lack of communication or miscommunication among stakeholders. Some project “CRS activities are conflicting implementers described instances in which project activities were with our regular tasks/activities. delayed because a partner was simultaneously rolling out other There is a need for better activities and was therefore unavailable. There is a need for the communication of their partners to have a mutual understanding about the agenda of activities agenda.” to ensure that relevant partners are available to conduct activities. –School Administrator Respondents, including implementers, MENA, and county mayors, believed the program has been monitored effectively and appreciated the project’s participative approach, which involves all partners in monitoring activities. For example, local MENA representatives periodically travel to schools with county mayors to observe project activities. CRS gives county mayors funds for field monitoring, such as cash for gasoline. A few implementer respondents noted that the field reports could be improved by including qualitative data on county mayors’ and local MENA staffers’ perspectives on project progress, successes, and challenges, among other topics.

6.3 Efficiency

Through interviews with USDA, MENA, implementers, teachers, and school administrators, the research team assessed the efficiency of the project. Interview topics focused on efficiency in the use of project resources, the timeliness of project activities, and the project’s responses to internal and external factors that may hinder implementation. Below is a summary of the qualitative findings on BBII efficiency.

6.3.1 Efficiency of Use of Project Resources The issue related to project resources that respondents most commonly expressed was delays in project implementation experienced by stakeholders in the field due to slow internal administrative processes. In some cases, field staff had to use their own means to perform activities and get reimbursed later because their host institution was unable to gather the funds in a reasonable time. Delays were particularly likely when the partner organization’s bank was different from CRS’s bank. Respondents reported that these delays were frustrating and hindered their ability to initiate some activities on time.

Further, respondents from OCADES and FAVL emphasized that the limited size of their staffs sometimes affected their ability to roll out field activities and effectively monitor them.

6.3.2 Achievement of Project Timeline When asked whether the project has remained on schedule, implementers said that the project has reached its immediate results on schedule: number of trainings performed, number of schools reached, number of SILCs created, and number of libraries provided.

Respondents from OCADES reported that they have completed their planned activities on time; in fact, they have initiated more SILCs than originally planned at this point in the project. The same holds for FAVL’s activities; respondents from FAVL reported that they have completed their activities as planned. However, FAVL experienced delays in library activities due to political instability in 2014.

IMPAQ International, LLC Page 68 Beoog Biiga II Midline Report July 2017 As described above, respondents reported that the project has experienced delays in the provision of some project activities and supplies, such as commodities for school canteens. Respondents reported that these delays were due to delays in payment to food carriers. Additionally, training materials and packages (books, letter cubes, Bananagrams sets, and so on) arrived at schools later than scheduled; some teachers did not receive materials until the end of the school year. Furthermore, some teachers did not receive literacy training until several months into the school year.

Finally, some respondents reported that some school district administrators were deployed to the field for monitoring visits later in the school year than originally planned. Some teacher trainings by school district administrators were also delayed.

6.3.3 Response to Factors That Could Hinder Implementation Respondents reported that commodity management has been one of “I feel CRS has done a really the most significant hindrances to efficiency throughout the project but good job of getting the that CRS and its partners have effectively managed the provision of communities involved and project services and materials given the challenges described above. having them be supportive of Respondents stated that CRS’s priority to engage government the commodities being in stakeholders, including MENA, has been beneficial not only to the roll- school, regardless of where out of project activities but also to project sustainability. they’re coming from.” –USDA Representative In addition to the internal factors, respondents shared external factors that could challenge implementation moving forward. First, climate variation threatens food security in the region. From one year to another, populations could experience food shortages that would limit parents’ ability to provide foods in canteens. Second, because the northern region has been subjected to terrorist attacks, some targeted schools may be affected or may be excluded from the project for security reasons. Finally, county municipalities are under a new mandate to increase staff wages. This budget pressure may threatening county municipalities’ ability to take on more responsibility on projects such as BBII.

6.4 Perceived Impact

Through interviews with all stakeholders described in Section 2.2, the research team assessed the perceived impact of the project on intended outcomes. Interview topics focused on stakeholders’ perception of the effect of the project on children’s education and health outcomes and the activities with the greatest and least medium- and long-term effects. Stakeholders were also asked to provide recommendations to improve the success of the project. Below is a summary of the qualitative findings on the perceived impact of BBII.

IMPAQ International, LLC Page 69 Beoog Biiga II Midline Report July 2017 6.4.1 Perceived Impact of Program Respondents of all types expected the new literacy approach to have an impact on student reading abilities if the teacher literacy trainings are sustained and if teachers apply the delivered techniques in their classrooms. Teachers believe the new framework of reading activities has stimulated students’ motivation to learn because being “When I came into the class, able to read correctly in front of others is perceived by students as a only three pupils were able to social recognition in the community. Reading group activities break read. But now, with the new techniques, most of them can down social barriers between girls and boys, respondents believed, as read a text.” boys and girls are becoming more comfortable participating in group –Teacher in Bam Provence activities together. Teachers also stated that students are becoming more confident in speaking and reading. Finally, teachers explained that the new teaching methods improve students’ cognitive abilities and generate positive effects in “We visited a school and were other disciplines including mathematics, geography, and history. really impressed with the cleanliness. The school In the schools researchers visited, most of the interviewed teachers environment was so nice and noticed variation in students’ attendance during the year; at the clean and, as a surprise, the beginning of the school year, before canteens deliver food, many community was also a clean students miss school. But when canteens begin functioning, teachers place. This is one of the most said, students come to school more regularly. To respondents, it is successful hygiene outcomes obvious that the rise in attendance is due to the presence of canteens. I’ve seen.” –MENA Representative As for hygiene-related outcomes, respondents including MENA staff and county mayors noticed positive changes in students’ hygiene habits during their field visits. In schools where water is available, students have changed their habits, developed positive attitudes toward hygiene practices, and adopted handwashing before and after meals. However, respondents believe there is still room for improvement.

6.4.2 Activities with the Greatest and Least Impact In general, respondents believed that the project has been more successful with literacy activities than with activities associated with hygiene, health, and nutrition.

Many respondents, including teachers, school administrators, implementers, and county mayors, stated that it is difficult to determine which activities of the BBII project had the greatest impact, as each aspect of the project is important to reaching key outcomes. Overall, the respondents welcomed the new literacy techniques, recognizing them as one of the activities that is most likely to generate important and positive effects on students’ literacy and comprehension skills.

Teachers, school administrators, and implementers believed that, if delivered properly, the educational activities related to hygiene, health, and nutrition will strengthen students’ skills and knowledge. As described in Section 6.2.1, respondents said that limitations such as teachers’ lack of motivation to teach handwashing, hygiene, and nutrition knowledge and the lack of clean water sources and latrines may result in less significant outcomes.

All respondents believed that no activities are insignificant; every activity plays a role in the project and contributes to improvements in children’s education and health outcomes.

IMPAQ International, LLC Page 70 Beoog Biiga II Midline Report July 2017 6.4.3 Respondents’ Recommended Strategies to Increase Impact Implementers, county mayors, teachers, and school administrators recommended ways to increase the impact of BBII activities based on issues met during implementation.

Teachers and school administrators emphasized that the duration of the teacher literacy trainings was too short for the amount of information that was presented. A strategy recommended by teachers was to lengthen the literacy trainings, based on pilot testing, to appropriately reflect the amount of time needed to accomplish all training objectives outlined in the agenda. Teachers also suggested operating two separate sessions: one dedicated to literacy training and another dedicated to hygiene, nutrition, and health. Some teachers believed that putting both together is too cumbersome for participants. Also, respondents suggested that the training focus more on practice than lecture. One school district administrator suggested that techniques like “discussion” and working group sessions must be emphasized because many teachers have not mastered the literacy techniques yet. Respondents also suggested that the per diem payments for “resident” teachers be increased to cover all of their expenses.

Because involvement of teachers is vital to success of the project, respondents also provided suggestions to improve teacher engagement based on their experience implementing BBII thus far. Respondents recommended that trainings include educating teachers about how their actions influence student success and how the new teaching techniques can yield improvements in the classroom. Teachers also recommended that trainers provide standardized sheets for lesson preparation as part of the training.

Furthermore, some respondents suggested that CRS develop a motivational framework that provides incentives for teachers who use the new teaching techniques and disincentives for those who do not. For example, CRS could initiate a competition to reward the best students in reading as well as their teachers. Respondents believed that school district administrators should also be involved in reinforcing integration of the new teaching techniques in the classroom. To do so, administrators would need regular access to means of transportation and other relevant equipment. Finally, respondents suggested that MENA school district administrators promote encouraging teachers to engage in continuous training using frameworks like groupe d’animation pedagogique, in which schools are grouped based on location and teachers practice teaching techniques and share best practices with each other.

CRS respondents also provided recommendations about the teacher observation process. In the traditional teacher observation approach, the evaluation of teachers by school district administrators consists of a process evaluation to observe whether teachers are following the recommended techniques or not. Respondents noted that this approach has limitations, because teachers may alter their behavior during visits in order to receive positive feedback. CRS respondents therefore suggested that school district administrators conduct observations at random without announcement. This procedure would require teachers to implement the new techniques consistently, resulting in better student outcomes.

Additionally, respondents suggested that administrators should revise the observation methodology to focus on student literacy outcomes and promote real-time hands-on support for teachers. For example, respondents suggested that administrators ask a random sample of students to read a given text to assess their reading skills. This technique would allow administrators to observe patterns and work with teachers to develop an action plan to solve issues and improve student performance over time.

To improve observation feedback, teacher respondents suggested that teachers receive a copy of the full feedback sheet that administrators prepare as part of their visit. Currently, teachers receive only a

IMPAQ International, LLC Page 71 Beoog Biiga II Midline Report July 2017 summary of recommendations and advice; however, teachers suggested that the full review would help them better understand the feedback and action plan.

Because materials and tools are important in facilitating teachers’ instruction, respondents recommended that teachers receive enough materials to support their large classrooms and that they receive them at the start of the school year. For example, teachers should receive a sufficient number of large slates to support small group work, ideally one large slate for every five to seven students. The most important training materials, respondents believed, were large slates and reading books. Respondents suggested that communities get more involved in the provision of training tools. For example, slates could be made with local materials.

To increase access to libraries, respondents recommended that the mobile library concept be implemented more broadly, especially in remote areas with little access to permanent libraries. Respondents also suggested that schools and communities get more involved in providing books. Some suggested that, at the beginning of every year, schools could set aside funds to purchase students’ preferred books to grow their internal library. Others suggested that schools initiate book collection campaigns, encouraging older students and community members to bring books they no longer use.

6.5 Sustainability

To assess the sustainability of project interventions, the research team asked all stakeholders, except students, to identify major factors that are likely to influence the achievement or non-achievement of project sustainability and to describe current sustainability activities implemented by CRS and its partners. Respondents were also asked to recommend strategies to sustain activities after funding ends. Below is a summary of the findings.

6.5.1 Sustainability Factors Project implementers and the USDA representative reported that one of the most important factors that will determine whether BBII activities will be sustained beyond USDA funding is the communities involved in implementation. Respondents reported that the dedication of communities, and specifically of PTAs, to continuing support of key project activities, such as canteens, mentoring, and SILC, will play a significant role in sustainability. For example, PTA members dedicate their own time to ensure that canteens function properly, serving as volunteer cooks, commodity watchers, and managers. Further, the communities supplement the government’s commodities to supply much of the food the canteens offer.

Another important factor to ensure sustainability is engagement of local stakeholders, most notably MENA and local governments. Respondents said that, if local partners do not take responsibility for project activities after USDA funding ends, then the project will not continue.

6.5.2 Current Sustainability Activities Understanding the importance of stakeholder and community involvement, CRS has already begun to coordinate closely with MENA stakeholders and with municipalities, which are involved in all aspects of the project. For example, municipalities have taken over several activities including the provision of library managers’ wages, school materials, and other subventions, which the municipalities receive from the central state. Municipalities also sometimes take over light expenses, such as the expenses of school management committees.

IMPAQ International, LLC Page 72 Beoog Biiga II Midline Report July 2017 CRS has also worked closely with local communities to involve them in implementing activities such as mentoring, SILCs, and canteens. The involvement of communities in these activities will be important for sustainability. Project implementers and county mayors stated that communities have fully endorsed and been actively engaged in project activities, including mentoring, which requires extensive community involvement. Furthermore, after OCADES provided training on SILCs, communities have continued to deliver the activities in their villages through prestataires de services privés or private service providers. These groups are village community members who are trained by OCADES to create SILCs.

6.5.3 Respondents’ Recommended Strategies to Sustain Activities Though it is too early to determine whether the project will be sustained after USDA funding ends, all types of respondents suggested that CRS and its partners continue to work closely with government stakeholders and communities on a sustainability plan. Throughout the interviews, respondents commonly referred to the project as “CRS’s project.” For this reason, respondents recommended organizing a workshop in order to emphasize the responsibility of each actor and obtain buy-in and ownership from stakeholders and communities. This workshop could serve as an opportunity to discuss sustainability strategies and how each stakeholder can play a role in sustainability. Respondents suggested that all stakeholders come to the table for this discussion, including community organizations and municipalities.

IMPAQ International, LLC Page 73 Beoog Biiga II Midline Report July 2017 SECTION 7. CONCLUSION

This report presents the midline results of the performance evaluation of the Beog Biiga II program, highlights changes in outcomes related to BBII core objectives, and informs potential additional questions to ask at endline in the provinces of Bam and Sanmatenga in Burkina Faso. To accurately reflect program performance over time, we followed the same performance evaluation methodology and sampling strategy used during the baseline to generate an appropriate comparison with the midline using USDA guidelines. For this midline performance evaluation report, we employed a variety of data collection methods including: 1) surveys of students, parents, teachers, PTA members, and school district administrators, 2) classroom observations, 3) focus group discussions, and 4) key informant interviews. We collected survey data from 258 primary school students grades two to six, 262 parents, 105 teachers, 22 PTA members, and 30 school district administrators; interview data from 27 stakeholders, including implementers, county mayors, teachers, and school administrators; and focus group data from 35 students grades two to six. We also collected attendance data for all the teachers and students in our sample. This section summarizes key findings in response to the main research questions, highlights study limitations, and provides recommendations for the evaluation and the overall project.

7.1 Key Findings

7.1.1 Students’ Outcomes  The proportion of students who achieved a passing score on a food nutrition test increased by 2 percentage points from baseline to midline (statistically significant at the 5 percent level) but remained low (3 percent). Surveyed data reported by teachers show that teachers are well- informed on knowledge necessary to increase students’ nutrition awareness. However, low nutrition knowledge of students may suggest that teachers need support on how to teach it effectively.  There was a slight (4 percentage point) increase in the proportion of children who reported washing their hands at critical moments compared to baseline although the change was not statistically significant. The proportion of children who achieved a passing score on a test of good health and hygiene practices remained relatively low with no significant change at midline (4 percent). The lack of significant changes from baseline to midline could be explained by the fact that some schools lack reliable water sources and handwashing devices sometimes fall into disrepair, as reported by school-level interviewees.  At midline, the proportion of children who reported that they were hungry during the school day (after eating breakfast before going to school) remained low (7 percent) with only a slight increase (3 percentage points) from baseline. However, the change was not statistically significant. Although this increase is trivial, it could be explained by fewer children (4 percentage points) eating before going to school (not statistically significant at any level) at midline. Students most commonly cited insufficient food (81 percent) as the reason why they could have eaten more after breakfast before going to school at midline.  The percentage of children who consumed a minimum acceptable diet increased significantly from baseline to midline. Students reported an 11 percentage point increase, and parents reported a 20 percentage point increase (statistically significant at the 1 percent level). The increase in consumption of a minimum acceptable diet was slightly higher for boys than for girls.

IMPAQ International, LLC Page 74 Beoog Biiga II Midline Report July 2017 This increase in the diversity of household’s diet might be affected by potential different socioeconomic backgrounds between baseline and midline samples.  Almost all students attended school regularly (at least 80 percent of the time). However, there was a slight decrease in children’s regular attendance from baseline to midline (by 1 percentage point); the result was marginally statistically significant (Statistically significant at the 10 percent level) at conventional levels. Boys’ attendance decreased by 2 percentage points at midline, a slightly greater decrease than in girls’ attendance. The fact that student attendance data were collected from teachers who might not record attendance consistently could confound measurement of the attendance rate.  The proportion of second-graders with reading proficiency at grade level increased by 9 percentage points from baseline to midline. The improvement in reading proficiency was higher for girls (15 percentage points) than for boys (4 percentage points). However, the increase was not statistically significant at conventional levels for either group. 7.1.2 Parents and Parent-Teacher Associations’ (PTA) Outcomes  At midline, parents’ participation in community savings groups established as part of the BBII program increased by 32 percentage points compared to baseline (statistically significant at the 1 percent level). Although more parents reported at midline that they used their savings to cover school expenses, fewer parents said that the savings helped “a lot” with school expenses, with a significant 33 percentage point decrease compared to baseline. On the other hand, the proportion of those who said that the savings helped “some” increased by 38 percentage points. Both findings are statistically significant at the 5 percent level. A possibility of increase in education expenses or a decrease in parents’ purchasing power to afford school costs might be potential explanations to this shift in parents’ responses.  The total number of months in which the canteen was covered by MENA, CRS, parents, and other responsible parties decreased by 2 months. This finding is statistically significant at the 10 percent level at midline. This outcome should be interpreted with caution, because there is a possibility that canteen operations were misrepresented at baseline. 7.1.3 Teachers’ Outcomes  The proportion of teachers who attended at least 90 percent of school days did not change significantly from baseline to midline, increasing only slightly by 3 percentage points.  Teachers in target schools employed new teaching techniques at a lower rate: 44 percent at midline compared to 53 percent at baseline. This slight, statistically insignificant decrease should be interpreted with caution, because the training component of the project had not started at baseline. More than half of the surveyed teachers at baseline reporting using new techniques may suggest that teachers were already aware of these techniques even in the absence of the training. It is also important to note that teachers at midline were less educated compared to teachers at baseline. As discussed in Section 6.2, teachers with limited education found the training materials more difficult to understand, which may have led them to continue using traditional approaches— approaches they were more accustomed to using.  A larger proportion of teachers at midline than at baseline reported that they devoted at least 45 minutes a day to literacy instruction, specifically reading instruction (including, for example, phonological awareness, phonetics, and vocabulary). However, this increase (11 percentage points) was not statistically significant at any conventional levels.

IMPAQ International, LLC Page 75 Beoog Biiga II Midline Report July 2017  Teachers identified a slight increase (3 percentage points) in the proportion of students who were attentive during class from baseline to midline. This change was not statistically significant at conventional levels. However, given the difficulty and subjectivity of measuring attentiveness, this outcome should be interpreted with caution. 7.1.4 Qualitative Findings The qualitative study revealed that stakeholders are generally satisfied with the project and believe the project is having a positive impact on beneficiaries. While the quantitative results show mixed results, teachers reported that they have observed improvements in students’ literacy abilities and students’ motivation to learn these skills. Teachers and students also reported that hygiene education has changed students’ behavior as they are now washing their hands more frequently. As discussed, the quantitative data, however, did not show improvements in handwashing. Students appreciated the school meals and take home rations received from the canteens, and teachers believed school attendance was closely tied to the presence of canteens. Though challenges are present, respondents described that CRS has effectively managed partners and the program has coordinated and collaborated effectively with other stakeholders.

According to respondents, the primary strengths of the project are its holistic approach to child development, its focus on evidence-based literacy techniques and provision of teaching resources, and its focus on community and government engagement. The primary weaknesses are lack of motivation by some key actors, lack of key equipment and resources for teacher monitoring and support, and the design of reading clubs. While it is too early to determine if the project will be sustained after USDA funding ends, all types of respondents suggested that CRS and its partners continue to work closely with government stakeholders and communities on a sustainability plan.

7.2 Limitations

There are some limitations of the study that are worth noting:  An important limitation of the study is that it relied on self-reported data for a number of socially and culturally sensitive subjects, such as food consumption, hygiene practices, and student attendance. This self-reported data should be interpreted with caution and is particularly susceptible to social desirability bias. To help counter biases related to hygiene practices, we integrated observational data of children’s handwashing practices (at critical moments) at the school. However, this method has its limitations as many children did not eat or use the latrine during our school visit.  As described in Section 5, at each data collection point (baseline and midline), we selected new samples of individuals to survey. A limitation of this approach is that there might be systematic differences in the two sets of samples that could confound the observed changes in the outcomes of interest over time. Inspection of the data indicate that the baseline and endline samples were similar along many demographic covariates, except for household size, and accessibility to basic services, parents’ and teachers’ educational attainment, and literacy training received by teachers.  Another limitation is related to the small sample sizes for some indicators, in particular those related to the PTA leaders and survey administrators outcomes. It is important to note that the results presented in those sections rely on very small sample sizes which reduce the accuracy of

IMPAQ International, LLC Page 76 Beoog Biiga II Midline Report July 2017 the indicators measured and the magnitude and direction of observed changes over time needs to be interpreted with caution.  Lastly, another limitation arises from the relatively small number of classroom observation due to budget limitations. Hence, these observations should be interpreted with caution without making a strong connection between students’ outcomes and instructional practices. In addition, we had some difficulties getting information from young students (grades 1-3) during focus group discussions as they were shy and hesitant to respond to many of our questions. 7.3 Recommendations

We present the following recommendations to CRS based on both lessons learned from our experience in the field and our findings after analyzing the collected data. The recommendations are not intended to address all challenges identified through the study; instead they focus on the main drivers of project success. The recommendations are grouped by category.

7.3.1 Recommendations for the Project

Project Management and Sustainability  Reevaluate how project activities are planned and communicated to ensure that relevant partners are available to conduct activities. Some project implementers noted that there have been instances in which project activities have been delayed because a partner is simultaneously rolling out other activities and is therefore unavailable.  Reevaluate staffing plan to ensure enough capacity to implement and monitor projects effectively and efficiently. Some implementers reported that the limited size of their staff sometimes impacted their ability to roll out field activities and effectively monitor their activities. When developing activity roll-out and monitoring plans, implementers should take into account the realities of staffing and resource limitations.  Provide additional training to parents on use of student performance tools and to SILC groups on management. Respondents described that students’ parents experience challenges with the use of performance tools distributed by teachers through a pilot activity at five of the BBII schools. Though these parents received training to read and use these tools to support their children, respondents explained that many of these parents still do not know how to interpret the tools. If CRS plans to sustain and/or expand this activity to other schools, they should consider enhancing the training to ensure that parents receive the support they need to effectively interpret and use the tools. Respondents also reported that SILC groups often struggle with management of group activities and require further support to run the groups effectively. Implementers should reevaluate their community sensitization and support structure to ensure that stakeholders are receiving the support and guidance they need to effectively initiate and manage project activities.  Reevaluate the sustainability strategy and develop a robust plan to engage all stakeholders. Respondents believed that the engagement of communities and the government are key factors to sustainability and that CRS and its partners have successfully engaged these parties. However, there is still work to be done to ensure that project activities can be sustained after USDA funding. CRS should continue engaging stakeholders and organize additional workshops with all relevant players to develop sustainability strategies and define how all parties can play a role in sustainability.

IMPAQ International, LLC Page 77 Beoog Biiga II Midline Report July 2017 Literacy Activities  Explore methods to increase teacher motivation and engagement. Interview respondents reported that teacher motivation is lacking in many schools. In order to motivate teachers to implement new teaching techniques, stakeholders could consider developing a framework in which teachers are incentivized for using the new techniques and tools in their classrooms. For example, during observation visits, school district administrators should continue evaluating teachers’ use of the techniques using a standardized evaluation process and assessing the level of students’ literacy skills using a standardized test. Teachers who excel could be offered awards for their performance. Further, some respondents reported that some teachers have refused to participate in CRS trainings because they believe the per diem payment for training is inadequate to cover all of their expenses. Implementers could consider distributing a short survey to obtain feedback directly from teachers about factors that motivate or dissuade teachers from participating in trainings, and use this feedback to modify practices and policies.  Consult with teachers to reevaluate length, timing, and content of teacher training. Interview respondents reported that trainings are too dense, causing teachers to be overwhelmed by the amount of information. Implementers may want to consider pilot testing a different training approach, in which teachers participate in two or three mini-training sessions that focus on different topics or techniques so as not to burden teachers with information overload. Teachers desired more time to practice the literacy techniques with their peers during trainings, and felt the trainings were lecture heavy. Some teachers also reported that they received literacy training late in the school year, giving them little time to apply the new approaches in their classrooms. Further, interview respondents noted that some teachers have a difficult time understanding the training materials and therefore are not able to successfully implement the new techniques and tools in their classrooms. Implementers should obtain teachers’ input to reevaluate the length, timing, and content of teacher training to ensure that the trainings meet teachers’ needs.  Collaborate with MENA to ensure that teacher observations are fully meeting their objectives. For example, more real-time, hands-on support to teachers is needed through the use of more frequent student testing and summarized feedback. Teachers reported that they appreciated the feedback they received from administrators during observations but desired more detailed, data- driven feedback. Implementers could consider implementing the ASER test more frequently and use the data to inform action plans with teachers that focus on ways to use the new teaching techniques and tools and support to teachers who face challenges. Implementers could consider having school-level administrators play a larger role in supporting teachers on a day-to-day basis. Further, the quantitative findings showed that teachers’ use of new techniques decreased by 9 percentage points over time, though the finding was not statistically significant. Implementers may want to use the observations as an opportunity to understand why teachers are not using the techniques, and how trainings and observations can be strengthened to better support teachers. Additionally, observation visits must occur early in the school year, as opposed to toward the end of the year, to give teachers adequate time to adjust based on the received feedback.  Reassess material distribution plan and further investigate why teachers are reluctant to use some tools. Teachers reported that they often did not have enough supplies to effectively use them in their classrooms. Implementers should ensure that teachers have more adequate quantities of materials to properly implement techniques. For example, teachers should receive one slate for every 5-7 students. Further, none of the teachers in classroom observations used Bananagrams, and word strips, and a very few of them used wood cubes, or phonemic tables as part of their lessons. In interviews, teachers reported only using large slates and blackboards. During trainings and observations by school district administrators, implementers may want to

IMPAQ International, LLC Page 78 Beoog Biiga II Midline Report July 2017 explore if and why teachers may be reluctant to use some tools and how trainings and observations can be strengthened to support teachers’ use of these materials.  Investigate why teachers’ use of new teaching techniques has decreased over time. At midline, teachers were less likely than at baseline to apply or plan to apply at least five of the seven teaching techniques on the day of data collection. Although the difference is not statistically significant, CRS might want to investigate why this may be the case and adjust trainings and teacher support accordingly.  Reconsider design of reading clubs with a greater focus on local context. Interview respondents described several challenges with the reading club design, including the difficulty to find qualified volunteers to facilitate groups and the fact that many parents require that their children help with domestic activities rather than attend reading club meetings. Implementers should reassess how the reading club component can be better tailored to the culture and stakeholders’ priorities.  Exhort school administrators to keep consistent records of teacher attendance and monitor data collection. Although the quantitative data showed high teacher attendance, the quality of the attendance data is uncertain given the sensitive nature of teacher absenteeism at schools. Some school district administrators or school administrators do not keep a record of absentees, and respondents reported varying levels of absenteeism. Given the uncertainty of the attendance data, implementers should work with schools to develop more robust tracking systems and conduct spot checks to confirm that schools are collecting the data.

Health and Dietary Activities  Work with schools to ensure that schools’ hand washing stations are in working order. Some respondents noted that hand washing stations are sometimes out of use and there is no effort made from the school nor the PTA to replace them. Implementers could develop a system whereby a designated teacher or the school-level administrator is responsible for managing the hand washing stations and reporting to CRS and its partners when it is out of order.  Further investigate why students’ hygiene and nutrition knowledge remains poor despite training. Very few children achieved a passing score on the test of hygiene knowledge and students’ level of nutrition knowledge remains very low. This area requires further probing by CRS to explore why student knowledge of good health and hygiene practices and nutrition are so low despite the training. A deeper analysis of why knowledge is so low would inform improvements to the program’s hygiene and health interventions, ideally improving students’ knowledge over time. Investigate health issues related to dietary supplements. A few interviewed students reported that they stopped taking the micronutrients that teachers gave them because it caused them to experience nausea and/or headaches. Because the intensity of this issue is unknown, implementers should consult with students and parents to better understand why students are reacting this way to the dietary supplements and adjust as needed.

7.3.2 Recommendations for the Evaluation  The enrollment list at participating schools should be updated on a more regular basis. After the randomization was complete and before midline data collection, we found some schools with unavailable students on the school lists due to relocations or because they dropped out of school. Collecting and quality-checking monitoring data more frequently (monthly or quarterly) would ensure that school lists are up to date.  Further research should be conducted to explore the reasons why some new teaching techniques, more specifically BBII writing techniques, are not being implemented by teachers.

IMPAQ International, LLC Page 79 Beoog Biiga II Midline Report July 2017 Unlike what teachers mentioned in the qualitative section, we did not observed any teachers in any grades using BBII writing techniques, or reporting using them in the survey. Improvement in literacy outcomes should be measured by progress in both skills. It is worth further investigation through qualitative study at endline to explore why teachers are not applying writing skills as intently as reading techniques: whether they are not comfortable implementing them; training was not enough for them; and/or readings skills were the only component that been emphasized by the trainers. In sampling lessons to observe at endline, we will also be mindful to purposely observe writing lessons. Though we selected lessons to observe at midline at random, it could be that we coincidentally observed reading lessons only.  Observations of health-related activities should be considered. Since we experienced difficulty obtaining useful, detailed information from young students (grades 1-3) during focus groups, our observation of handwashing practices needs improvement, and we observed a drop in hygiene- related practices among children and no change in knowledge of good health and hygiene practices, we may want to consider focusing our effort and resources on conducting additional observations of health-related activities, such as handwashing, canteens, and vitamin distribution, at select schools during the endline. Shifting our focus would allow us to obtain richer data on health-focused interventions that we can triangulate with survey, interview, and focus group data.

7.3.3 Recommendations for the Evaluation with Budget Implications  The scope of work and budget should be amended to include an additional subtest in the ASER reading assessment. BBII teaching techniques include both reading and writing skills together, and in order to assess children’s literacy outcomes we should consider both components together. Adding a subtest to ASER can provide additional data on writing while providing the same comparable data for the reading component.  Larger samples for classroom observations should be considered across treatment and control schools at the endline. We should enhance the size of classroom observations, to a) get a better understanding of how the new techniques are being implemented, b) study the correlation between teacher characteristics, instructional practices, and student’s literacy outcomes, and c) to be able to draw better conclusions from the data. If added to the scope of work and budget, we could study the connections between the qualitative findings, quantitative outcomes, and our observations in a more rigorous way.  Interviews with additional stakeholder groups should be considered. To broaden our understanding of the project’s impact on all BBII stakeholders, we may want to consider conducting interviews with stakeholders that we did not speak with during the midline, such as SILC members and PTAs. Quantitative data showed no changes in PTA activities including holding meetings or supporting canteens, and in some instances PTA support decreased over time. Obtaining data from these stakeholders will allow the team to provide a more comprehensive picture of intervention implementation and the perceptions, opinions, and experiences of all types of stakeholders.

IMPAQ International, LLC Page 80 Beoog Biiga II Midline Report July 2017 REFERENCE LIST

Abadzi, H. (2011). Reading fluency measurements in EFA FTI partner countries: Outcomes and improvement prospects (GPE Working Paper Series on Learning, No. 1). Washington, DC: World Bank.

Catholic Relief Services. (2015). IMPAQ International, LLC Consulting Agreement and Statement of Work. Ouagadougou, Burkina Faso.

Food and Agriculture Organization (2010). Guidelines for Measuring Household and Individual Dietary Diversity. Rome, Italy: United Nations.

Montgomery, S. M., & Groat, L. N. (1998). Student learning styles and their implication for teaching (Vol. 10). Centre for Research on Learning and Teaching, University of Michigan.

Pratham. (2015). Pratham Around the World. Retrieved from http://www.prathamusa.org/programs/pratham-around-world.

Simons, K.A. (2012). Discussion Document #1: Multi-Country Assessments of Learning. Washington, DC: Center for Universal Education at the Bookings Institution.

United States Department of Agriculture (USDA). (2014). Food for progress and McGovern-Dole indicators and definitions. Washington DC: Food Assistance Division, Office of Capacity Building, Foreign Agricultural Service.

United States Department of Health and Human Services (USDHHS). Human Subject Regulations Decision Charts. Retrieved from http://www.hhs.gov/ohrp/policy/checklists/decisioncharts.html (2016, February 16).

United States Agency for International Development (USAID). (2012). 2011 USAID education strategy reference materials. Washington, DC.

Vagh, S.B. (2009). Evaluating the reliability and validity of the ASER testing tools. New Delhi: ASER Centre.

Vanmeenen, G. (2010). Savings and Internal Lending Communities (SILC) - Voices from Africa. Nairobi: Catholic Relief Services.

IMPAQ International, LLC Page 81 Beoog Biiga II Midline Report July 2017 APPENDICES

A. BBII Key Performance Indicators B. Additional detailed data breakdown C. Additional Tables and Complementary Outcomes D. ASER-Reading Test Instructions E. ASER-Reading Test Instructions F. Questionnaires G. Performance Evaluation Schools

IMPAQ International, LLC Page 82 Beoog Biiga II Baseline Report July 2017

APPENDIX A. BBII KEY PERFORMANCE INDICATORS

Data Strategic Objectives Key Questions Collection Data Source and Results Method

Student literacy What percentage of students (boys and girls) have increased their reading SO1. Improved Literacy assessment (ASER- comprehension compared to baseline? What factors contributed to this? What Evaluation of School-Age Children reading test) & factors inhibited this? Teacher interviews.

. Have teachers’ literacy instruction trainings been completed as per the project timeline and budget? CRS program data . To what extent are teachers implementing literacy teaching techniques (training reports), acquired through the project? Teacher surveys & . Are changes in classrooms starting to be evident (e.g they see improved Teacher key informant reading abilities of students)? If not, why not? What needs to change? interviews. CRS/ . How much time per day do teachers devote to literacy instruction? IR1.1. Improved Quality Monitoring . What challenges do teachers face in using the new literacy teaching Classroom of Literacy Instruction & techniques? observations. Evaluation . What aspects do they find most useful and why? . In what way has the quality of education improved as a result of the adoption Observations of of technical trainings for teachers? libraries (as feasible) . Have community library targets been met? How many exist? Where they are & FAVL staff in place, to what degree do community members (including students and interviews. teacher) have access?

To what extent has teachers’ attendance improved in schools compared to the Key informant Output. 1.1.1. More baseline? If so, why? What are the greatest inhibiting factors to teacher’s interviews with school Consistent Teacher Evaluation attendance? What project interventions influenced the improvement of teacher administrators & Attendance attendance? Teacher interviews.

IMPAQ International, LLC Page 83 Beoog Biiga II Baseline Report July 2017

Data Strategic Objectives Key Questions Collection Data Source and Results Method

. To what extent have school supplies and materials been distributed as per the Distribution reports project timeline and budget? (CRS program data). . What materials have been supplied? . Which school supplies do teachers find most useful and why? Which supplies CRS/ Classroom Output. 1.1.2. Better and materials provided do students like and why? Monitoring observations. Access to School . What other supplies would teachers and students prefer? & Supplies & Materials . Are material kits being used as intended? Evaluation Teacher surveys, . Do teachers/students need additional training to better use these materials? Teacher key informant interviews, & Student . What has been the contribution to the education process of these materials? interviews.

. To what extent have literacy instructional materials been distributed as per the project timeline and budget? Do the teachers consider these materials to be an improvement over what they previously had? How? How are teachers Distribution reports using the materials provided? What, if any other materials would they prefer? (CRS program data). What do students like and dislike about using the literacy materials that have Output. 1.1.3. Improved been provided? CRS/Monitoring Classroom Literacy Instructional . To what extent have reading groups been established? What are the & observations. Materials characteristics of a successful reading group? Are the reading groups Evaluation segregated by gender, grade level, etc.? Who provides oversight of the Teacher key informant reading group most often (teachers or older students)? Why do/ do not interviews & Student students join reading groups? Have students or teachers noticed any change interviews. since students have joined reading groups? (Personal confidence, better grades, more social cohesion among students, etc.)

To what extent have teachers been trained as per the project timeline and Output. 1.1.4. Increased CRS/Monitoring budget? What percent of teachers demonstrate use of new teaching techniques Training reports (CRS Skills and Knowledge of & and knowledge? In what ways has the quality of teaching improved based on the program data). Teachers Evaluation tools and techniques used by teachers? What aspects of the trainings were not widely adopted and why? How can the trainings have greater impact? What

IMPAQ International, LLC Page 84 Beoog Biiga II Baseline Report July 2017

Data Strategic Objectives Key Questions Collection Data Source and Results Method additional training topics would help the teachers be even more effective in Teacher survey & Key literacy instruction? informant interviews.

Classroom observations.

To what extent have school administrators been trained as per the project CRS program data timeline and budget? What percent of school administrators demonstrate use of (training reports). new techniques or tools? To what extent do school administrators find the Output. 1.1.5. Increased CRS/Monitoring classroom observation technique useful? How is the technique received by Skills and Knowledge of & School District teachers? Have the observations led to constructive feedback? If so, has the School Administrators Evaluation Administrators feedback been received by teachers and affected their teaching techniques? In interviews & Teacher what way has the quality of education improved based on techniques used by the interviews. administrators?

To what extent have school meals been distributed as per the project’s budget IR1.2. Improved and timeline? What percent of students in target schools indicate that they are Distribution reports, Attentiveness Output hungry during the school days? How do students appreciate the ration size? What Evaluation student surveys & 1.2.1. Reduced Short- percent of students eat the entire school lunch at school and what percent bring a student interviews Term Hunger portion home for family members?

Distribution reports (CRS program data). To what extent have Take Home Rations (THR) been distributed as per the Output 1.2.1.1/1.3.1.1 CRS/Monitoring project’s timeline and budget? How do students like the commodities provided for Increased Access to & Student interviews & school meals? What percent of school level warehouses demonstrate appropriate Food (School Feeding) Evaluation warehouse storage of commodities? observations (as feasible).

IMPAQ International, LLC Page 85 Beoog Biiga II Baseline Report July 2017

Data Strategic Objectives Key Questions Collection Data Source and Results Method

What is the current student attendance rate? Has the percent of students (girls Student attendance IR1.3. Improved Student and boys) that regularly (80%) attend schools changed compared to the baseline? Evaluation logs & Student Attendance If so, what are the greatest pull factors for students? interviews.

CRS program data Output. 1.3.1 Increased To what extent has the girls mentoring activity been implemented as per the (training reports, Economic & Cultural project’s timeline and budget? Have the mentors been recruited and trained as CRS/Monitoring activity reports). Incentives (Or per the project’s timeline and budget? Have the mentors conducted regular & Decreased mentoring activities as planned? Have the SILC activities been implemented as per Evaluation OCADES key informant Disincentives) the project’s timeline and budget? interviews.

To what extent has the enrollment of school‐age students (girls and boys) changed Student registers & Output 1.3.4. Increased compared to the baseline? If so, how? Which factors have facilitated or have been Evaluation Teacher/school key Student Enrollment obstacles towards enrollment? informant interview.

Output. 1.3.5. Increased To what degree have the awareness raising activities on the importance of CRS/Monitoring CRS program data Community education been completed as planned? Has the parents’ knowledge of the & Understanding of importance of education changed compared to baseline? Has the parents’ level of Evaluation Parent survey. Benefits of Education contribution to the school canteen changed?

SO2. Increased Use of To what extent have the use of standard health practices and dietary diversity Student survey & Health and Dietary Evaluation among students increased from baseline? Parent survey. Practices

IMPAQ International, LLC Page 86 Beoog Biiga II Baseline Report July 2017

Data Strategic Objectives Key Questions Collection Data Source and Results Method

CRS program data What percent of students have increased their knowledge of health and hygiene (distribution reports) IR2.1 Improved practices compared to baseline? To what extent have students improved their CRS/Monitoring Knowledge of Health hygiene-related practices (what percent of school children wash their hands at & Student surveys and and Hygiene Practices critical moments)? To what extent has the project supplied hand washing stations Evaluation observations (as to schools as planned? feasible).

IR 2.2 Increased Knowledge of Safe Food To what extent has the project completed trainings for food preparers as Evaluation Training reports. Prep and Storage planned? Practices

Student survey & What percent of students have increased their knowledge of nutrition and dietary Parent survey. practices compared to baseline? What percent of children receiving a minimum IR 2.3 Increased acceptable diet has changed, if any compared to baseline? Evaluation PTA/COGES/Food Knowledge of Nutrition preparers training To what extent have PTA, COGES members, and food preparers been trained in reports (including good nutrition and dietary practices as planned? post-test results).

Sample of student distribution records IR2.5 Increased Access To what extent has the project distributed micronutrients to students as planned? (school level), student to Preventative Health To what degree has students’ knowledge of Vitamin A, Iron and deworming Evaluation surveys, CRS program Interventions medication changed since baseline? data/distribution reports.

IMPAQ International, LLC Page 87 Beoog Biiga II Baseline Report July 2017

Data Strategic Objectives Key Questions Collection Data Source and Results Method

IR2.6 Increased Access to Requisite Food Prep To what extent has the project distributed food preparation and storage supplies Evaluation Distribution reports. and Storage Tools and as planned to preschools? Equipment

FR 1: Output 1.4.1. & 2.7.1 Increased To what extent have local government officials been trained as planned? Evaluation Training reports. Capacity of Government institutions

FR 2: Output 1.4.2.& To what extent have committees been formed and started drafting policy Committee member 2.7.2 Improved Policy or Evaluation improvements in education related policies? interview. Regulatory Framework

FR 4: Output1.4.4 & 2.7.4 Increased To what extent are PTA’s holding regular meetings? To what degree is this Engagement of Local Evaluation PTA survey. frequency different from baseline? Organizations and Community Groups

Source: Terms of Reference (TOR)

IMPAQ International, LLC Page 88 Beoog Biiga II Baseline Report July 2017

APPENDIX B. BBII KEY PERFORMANCE INDICATORS

Outcome Variable Mean Observations

Indicator 1: Percent of students, who by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text Average reading level of second graders 3.1443 87 Proportion of boys that can read and understand the meaning of grade level text 0.16 43 Proportion of girls that can read and understand the meaning of grade level text 0.32 44 Indicator 2: Number of teachers who devote at least an average of 45 minutes a day to literacy instruction Average number of minutes teachers spent on phonemics 25.14 22 Average number of minutes teachers spent on phonics 19.20 20 Average number of minutes teachers spent on fluency 27.61 67 Average number of minutes teachers spent on vocabulary 16.46 24 Average number of minutes teachers spent on comprehension 19.75 60 Average number of minutes teachers spent on identification of key words in a sentence 19.79 28 Average number of minutes teachers spent on identification of key sounds 22.78 9 Average number of minutes teachers spent on ability to repeat words, sounds and 18.35 40 syllables Average number of minutes teachers spent on reading punctuation 19.69 26 Proportion of that teachers didn't know more than one techniques 0.61 105 Average number of minutes students read by themselves today 9 65 Average number of minutes students usually read by themselves during class 11.16 38 Average number of minutes students read to others today 11.16 38 Average number of minutes students usually read to others during class 16.75 89 Average number of minutes students listened to others read today 15.66 68 Average number of minutes students usually listen to others read during class 16.11 90 Average number of minutes students wrote today 14.20 66 Average number of minutes students usually write during class 21.99 101 Average number of minutes students did word play today 15.34 50 Average number of minutes students usually do word play during class 14.67 91 Indicator 3: Number of teachers in target schools who attend and teach school at least 90% of scheduled school days per year Proportion of teachers in target schools who attend and teach school at least 90% of 0.72 49 scheduled school days per year

43 This value corresponds to between reading levels B (level 2) and C (level 3).

IMPAQ International, LLC Page 89 Beoog Biiga II Baseline Report July 2017

Outcome Variable Mean Observations Indicator 4: Number of school administrators in target schools who demonstrate use of new techniques or tools as a result of USDA assistance Proportion of school administrators in targeted districts who demonstrate use of new 0.61 13 techniques or tools as a result of USDA assistance Indicator 5: Number of preschool teachers (bissongo caregivers) in target preschools who demonstrate use of at least one new teaching technique, skill or tool Proportion of preschool teachers (bissongo caregivers) in target preschools who N/A N/A demonstrate use of at least one new teaching technique, skill, or tool44 Indicator 6: Number of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance

Proportion of teachers who have received literacy training in the last 12 months 0.95 105 Proportion of teachers who have had students check their own work this week 0.35 105 Proportion of teachers who are planning to have students check their own work this week 0.38 105 Proportion of teachers who have not and are not planning to have students check their 0.27 105 own work this week Proportion of teachers who have had students check each other’s work this week 0.44 105 Proportion of teachers who are planning to have students check each other’s work this 0.43 105 week Proportion of teachers who have not and are not planning to have students check each 0.13 105 other’s work this week Proportion of teachers who have had the whole class check the work of a student this 0.48 105 week Proportion of teachers who are planning to have the whole class check the work of a 0.38 105 student this week Proportion of teachers who have not and are not planning to have the whole class check 0.14 105 the work of a student this week Proportion of teachers who have checked students' work this week 0.49 105 Proportion of teachers who are planning to check students' work this week 0.33 105 Proportion of teachers who have not and are not planning to check students' work this 0.18 105 week Proportion of teachers who have paired students of different skill levels to work together 0.49 105 this week Proportion of teachers who are planning to pair students of different skill levels to work 0.35 105 together this week Proportion of teachers who have not and are not planning to pair students of different skill 0.16 105 levels to work together this week Proportion of teachers who have paired students of the same skill levels to work together 0.10 105 this week Proportion of teachers who are planning to pair students of the same skill levels to work 0.15 105 together this week Proportion of teachers who have not and are not planning to pair students of the same skill 0.75 105 levels to work together this week

44 Will be collected with pre and post testing at the time of training provided by BBII

IMPAQ International, LLC Page 90 Beoog Biiga II Baseline Report July 2017

Outcome Variable Mean Observations Proportion of teachers who have had students work together in groups this week 0.30 105 Proportion of teachers who are planning to have students work together in groups this 0.41 105 week Proportion of teachers who have not and are not planning have students work together in 0.30 105 groups this week Proportion of teachers using the phonemics technique 0.21 105 Proportion of teachers using the phonics technique 0.19 105 Proportion of teachers using the fluency technique 0.64 105 Proportion of teachers using the vocabulary technique 0.23 105 Proportion of teachers using the comprehension technique 0.57 105 Proportion of teachers using the identification of key words in a sentence technique 0.27 105 Proportion of teachers using the identification of key sounds technique 0.08 105 Proportion of teachers using the ability to repeat words, sounds and syllables technique 0.38 105 Proportion of teachers using the reading punctuation technique 0.25 105 Indicator 7: Number of students in target schools who are identified as attentive during class/instruction Average students' attentiveness on a scale of 1 (the lowest) to 10 (the highest), as reported 6.18 105 by teachers Proportion of students identified as 7 or above on the attentiveness scale, as reported by 0.47 105 teachers Indicator 8: Number of students in target schools who indicate that they are hungry or very hungry during the school day Proportion of children that felt full after the meal s/he ate before going to school 0.93 223 Proportion of students that could have eaten more before going to school 0.07 223 Proportion of students that cite lack of food as the reason for not having a meal before 0.81 16 going to school Proportion of students that mentioned there was no food that they liked before school 0.12 16 Proportion of students indicating another reason why they did not eat more before school 0.6 16 Proportion of children that felt full after eating lunch at home 0.94 150 Proportion of students that could have eaten more for lunch at home 0.06 150 Proportion of students that cite lack of food as the reason why they did not eat a meal for 0.78 9 lunch at home Proportion of students that mentioned there was no food that they liked for lunch at home 0.11 9 Proportion of students indicating another reason why they did not eat more for lunch at 0.11 9 home Proportion of children that felt full after eating lunch at the school canteen 0.88 207 Proportion of students that could have eaten more for lunch at the school canteen 0.12 207 Proportion of students that cite lack of food as the reason why they did not eat a meal for 0.8 25 lunch at the school canteen Proportion of students that mentioned there was no food that they liked for lunch at the 0.04 25 school canteen Proportion of students indicating another reason why they did not eat more for lunch at 0.16 25 the school canteen Proportion of children that felt full before going to bed 0.98 246 Proportion of children that could have eaten more before going to bed 0.02 246

IMPAQ International, LLC Page 91 Beoog Biiga II Baseline Report July 2017

Outcome Variable Mean Observations Proportion of children that did not eat a meal before going to bed for the lack of food 0.67 6 Proportion of children that mentioned there was no food they liked before going to bed 0.33 6 Proportion of students indicating another reason why they did not eat more before going 0.00 6 to bed Proportion of parents that indicated 1 or more of their children ate less because there was 0.34 355 no food Proportion of parents that indicated their children did not eat because there was no food 0.21 355 Proportion of parents that indicated 1 or more of their children went a full day without 0.08 355 food Indicator 9: Number of students regularly (80%) attending USDA supported classrooms/schools Proportion of students attending regularly (80%) 0.99 559 Proportion of boys attending regularly (80%) 1 238 Proportion of girls attending regularly (80%) 0.99 229 Indicator 10: Number of SILC group members that spend money from SILC on education costs Proportion of parents that are members of SILC 0.09 240 Proportion of parents that have used a portion of their savings accumulated in SILC for 0.50 24 school expenses Indicator 11: Number of girl students who cite mentors as one of the top 3 reasons for their success or improvement in school Proportion of girl students who cite mentors as one of the top 3 reasons for their success 0 0 or improvement in school. Indicator 12: Percent of school-age children receiving a minimum acceptable diet Proportion of boys that ate cereals (e.g. pasta, rice,noodles, etc) 0.97 125 Proportion of girls that ate cereals (e.g. pasta, rice,noodles, etc) 0.99 133 Proportion of boys that ate white roots like white potato 0.03 125 Proportion of girls that ate white roots like white potato 0.04 133 Proportion of boys that ate vitamin A rich vegetables (e.g. carrot, etc) 0.00 125 Proportion of girls that ate vitamin A rich vegetables (e.g. carrot, etc) 0.03 133 Proportion of boys that ate dark green leafy vegetables like spinach 0.62 125 Proportion of girls that ate dark green leafy vegetables like spinach 0.72 133 Proportion of boys that ate other vegetables (e.g. eggplant, tomato) 0.51 125 Proportion of girls that ate other vegetables (e.g. eggplant, tomato) 0.61 133 Proportion of boys that ate vitamin A rich fruit like mango & papaya 0.47 125 Proportion of girls that ate vitamin A rich fruit like mango & papaya 0.46 133 Proportion of boys that ate other fruit (e.g. watermelon, coconut) 0.26 125 Proportion of girls that ate other fruit (e.g. watermelon, coconut) 0.26 133 Proportion of boys that ate internal organs (e.g. liver, heart) 0.02 125 Proportion of girls that ate internal organs (e.g. liver, heart) 0.04 133 Proportion of boys that ate flesh meat (e.g. chicken, pork, beef) 0.27 125 Proportion of girls that ate flesh meat (e.g. chicken, pork, beef) 0.23 133

IMPAQ International, LLC Page 92 Beoog Biiga II Baseline Report July 2017

Outcome Variable Mean Observations Proportion of boys that ate any bird's eggs 0.06 125 Proportion of girls that ate any bird's eggs 0.03 133 Proportion of boys that ate seafood (e.g. shrimp, fish) 0.48 125 Proportion of girls that ate seafood (e.g. shrimp, fish) 0.47 133 Proportion of boys that ate nuts and seeds (e.g. lentils, beans) 0.76 125 Proportion of girls that ate nuts and seeds (e.g. lentils, beans) 0.80 133 Proportion of boys that ate dairy products (e.g. milk, cheese) 0.05 125 Proportion of girls that ate dairy products (e.g. milk, cheese) 0.06 133 Proportion of boys that ate oils and fats like butter 0.76 125 Proportion of girls that ate oils and fats like butter 0.79 133 Proportion of boys ate any sweets (e.g. sugar, honey) 0.38 125 Proportion of girls ate any sweets (e.g. sugar, honey) 0.37 133 On the scale of 1 to 15, FAO dietary diversity index for boys 5.65 125 On the scale of 1 to 15, FAO dietary diversity index for girls 5.9 133 Indicator 13: Percent of school children that wash their hands at critical moments Proportion of school children that wash their hands at critical moments 0.82 258 Proportion of children that washed their hands before eating 0.91 258 Proportion of children that did not wash their hands before eating 0.00 258 Proportion of children that did not wash their hands because they did not eat that day 0.09 258 Proportion of children that washed their hands before preparing food 0.38 258 Proportion of children that did not wash their hands before preparing food 0.07 258 Proportion of children that did not wash their hands because they did not prepare food 0.55 258 that day Proportion of children that washed their hands before giving food to others 0.30 258 Proportion of children that did not wash their hands before giving food to others 0.08 258 Proportion of children that did not wash their hands because they did not give food to 0.62 258 anyone that day Proportion of children that washed their hands when they were dirty 0.46 258 Proportion of children that did not wash their hands when they were dirty 0.25 258 Proportion of children that did not wash their hands because they were not dirty that day 0.29 258 Proportion of children that washed their hands after touching dirt 0.34 258 Proportion of children that did not wash their hands after touching dirt 0.26 258 Proportion of children that did not wash their hands because they did not touch dirt that 0.40 258 day Proportion of children that washed their hands after using the latrine 0.39 258 Proportion of children that did not wash their hands after using the latrine 0.12 258 Proportion of children that did not wash their hands because they did not use the latrine 0.49 258 that day

IMPAQ International, LLC Page 93 Beoog Biiga II Baseline Report July 2017

Outcome Variable Mean Observations Indicator 14: Number of students in target schools who achieve a passing score on a test of good health and hygiene practices Proportion of students that believe they should wash their hands before eating 0.97 258 Proportion of students that believe they should wash their hands before preparing food 0.15 258 Proportion of students that believe they should wash their hands before feeding others 0.01 258 Proportion of students that believe they should wash their hands if they are dirty 0.08 258 Proportion of students that believe they should wash their hands after touching dirt 0.07 258 Proportion of students that believe they should wash their hands after using the latrine 0.28 258 Indicator 15: Number of students in target schools who achieve a passing score on a test of food nutrition and dietary practices Proportion of children that have heard of vitamin A before 0.49 258 Proportion of children that say vitamin A provides good vision 0.01 126 Proportion of children that say vitamin A protects the body against diseases 0.25 126 Proportion of children that say vitamin A helps kids to grow up healthy 0.23 126 Proportion of children that say vitamin A keeps the skin healthy 0.02 126 Proportion of children that specify something else vitamin A does 0.37 126 Proportion of children that were not able to specify what vitamin A does 0.25 126 Proportion of children that say liver has vitamin A 0.01 126 Proportion of children that say small whole fish has vitamin A 0.08 126 Proportion of children that say dairy products have vitamin A 0.08 126 Proportion of children that say egg yolks have vitamin A 0.06 126 Proportion of children that say carrot has vitamin A 0.01 126 Proportion of children that say sweet potato has vitamin A 0.00 126 Proportion of children that say squash has vitamin A 0.01 126 Proportion of children that say dark green leafy vegetables have vitamin A 0.10 126 Proportion of children that say red palm oil has vitamin A 0.00 126 Proportion of children that say papaya has vitamin A 0.02 126 Proportion of children that say mango has vitamin A 0.15 126 Proportion of children that say Nere flour (leaves of a local tree) has vitamin A 0.01 126 Proportion of children that say green pepper has vitamin A 0.00 126 Proportion of children that say melon has vitamin A 0.02 126 Proportion of children that specify another food having vitamin A 0.28 126 Proportion of children that were not able to specify a food containing vitamin A 0.51 126 Proportion of children that knew there is iron in the food they eat 0.19 258 Proportion of children that say iron provides good vision 0.06 48 Proportion of children that say iron protects the body against diseases 0.31 48 Proportion of children that say iron helps kids to grow up healthy 0.12 48

IMPAQ International, LLC Page 94 Beoog Biiga II Baseline Report July 2017

Outcome Variable Mean Observations Proportion of children that say helps the muscle function 0.06 48 Proportion of children that say iron helps the brain function 0.00 48 Proportion of children that say iron regulates the body temperature 0.00 48 Proportion of children that say iron prevents fatigue 0.00 48 Proportion of children that say iron is the oxygen carrier in the body 0.00 48 Proportion of children that say iron maintains healthy skin 0.00 48 Proportion of children that say iron helps hemoglobin formation 0.17 48 Proportion of children that specify something else iron does 0.21 48 Proportion of children that were not able to specify what iron does 0.25 48 Proportion of children that say fish has iron 0.08 48 Proportion of children that say the poultry family has iron 0.00 48 Proportion of children that say red meat has iron 0.15 48 Proportion of children that organ meat has iron 0.00 48 Proportion of children that say beans have iron 0.08 48 Proportion of children that say dry peas have iron 0.04 48 Proportion of children that say lentils has iron 0.00 48 Proportion of children that say cowpeas has iron 0.02 48 Proportion of children that say dark green leafy vegetables have iron 0.21 48 Proportion of children that say eggs have iron 0.00 48 Proportion of children that say potatoes have iron 0.02 48 Proportion of children that say tofu have iron 0.00 48 Proportion of children that say green beans have iron 0.00 48 Proportion of children that say nuts (sesame, cashew) have iron 0.00 48 Proportion of children that say green pepper have iron 0.00 48 Proportion of children that say watermelon have iron 0.00 48 Proportion of children that say tomatoes have iron 0.04 48 Proportion of children that specify another food having iron 0.27 48 Proportion of children that were not able to specify a food containing iron 0.44 48 Indicator 16: Number of months of community and/or government support for school canteens Number of months of community and or government support for school canteen 3 15 Proportion of parents that contribute wood every day that the canteen operates 0.43 14 Proportion of parents that contribute wood most of the days that the canteen operates 0.43 14 Proportion of parents that rarely contribute wood of the days that the canteen operates 0.14 14 Proportion of parents that never contribute wood of the days that the canteen operates 0 14 Proportion of parents that contribute utensils every day that the canteen operates 0.64 14 Proportion of parents that contribute utensils most of the days that the canteen operates 0.07 14 Proportion of parents that rarely contribute utensils every day that the canteen operates 0 14

IMPAQ International, LLC Page 95 Beoog Biiga II Baseline Report July 2017

Outcome Variable Mean Observations Proportion of communities that pay women to cook for the canteen 0.23 22 Proportion of students that contribute food to the canteen every day it operates 0.5 14 Proportion of students that contribute food to the canteen most of the days it operates 0.36 14 Proportion of students that rarely contribute food to the canteen when it operates 0 14 Proportion of students that never contribute food to the canteen when it operates .14 14 Proportion of parents that never contribute utensils every day that the canteen operates 0.29 14 Proportion of parents that contribute to the storage of food every day the canteen operates 0 4 Proportion of parents that contribute to the storage of food most of the days the canteen 0.29 14 operates Proportion of parents that rarely contribute to the storage of food when the canteen 0.14 14 operates Proportion of parents that never contribute to the storage of food when the canteen 0.29 14 operates Source: Surveys of Student, Parent, Teacher, and PTA

IMPAQ International, LLC Page 96 Beoog Biiga II Baseline Report July 2017

APPENDIX C. ADDITIONAL TABLES AND COMPLEMENTARY OUTCOMES

Exhibit 47. Gender Composition of Students Baseline Midline Difference in Means Total Total Percent Number of Percent Number of (p-value test) Observations Observations Proportion of 0 52% 343 52% 258 Female Students (0.82207) Source: Student survey; IMPAQ calculations.

Exhibit 48. School District Staff’ Educational Attainment 40% 37% 35% 30% 27% 25% 20% 15% 13% 13% 10% 7% 5% 3%

0% PercentofCounselors/Managers

Source: School district administrator survey; IMPAQ calculation; N=30.

IMPAQ International, LLC Page 97 Beoog Biiga II Baseline Report July 2017

Exhibit 49. Minimum Acceptable Diet Reported by Parents

Outcome Variable Percent Observations

Household receiving a minimum acceptable diet The proportion of parents that offered their child cereals (e.g. pasta, rice, noodles, etc.) 98% 262 The proportion of parents that offered their child white roots like white potato 2% 262 The proportion of parents that offered their child vitamin A rich vegetables (e.g. carrot, 262 2% etc.) The proportion of parents that offered their child dark green leafy vegetables like 262 84% spinach The proportion of parents that offered their child other vegetables (e.g. eggplant, 262 64% tomato) The proportion of parents that offered their child vitamin A rich fruit like mango & 262 34% papaya The proportion of parents that offered their child other fruit (e.g. watermelon, 262 17% coconut) The proportion of parents that offered their child internal organs (e.g. liver, heart) 5% 262 The proportion of parents that offered their child flesh meat (e.g. chicken, pork, beef) 16% 262 The proportion of parents that offered their child any bird's eggs 2% 262 The proportion of parents that offered their child seafood (e.g. shrimp, fish) 64% 262 The proportion of parents that offered their child nuts and seeds (e.g. lentils, beans) 47% 262 The proportion of parents that offered their child dairy products (e.g. milk, cheese) 5% 262 The proportion of parents that offered their child oils and fats like butter 48% 262 The proportion of parents that offered their child any sweets (e.g. sugar, honey) 25% 262 On the scale of 1 to 15, FAO dietary diversity index reported by Parents 5% 262 Source: Parent Survey; Author’s calculation

Exhibit 50. Teacher Reported Utilization of Classroom Activities for this Week

Baseline Midline Difference in Means Utilization Percent Observations Percent Observations P- value Individual student checks his or her own work and gives himself/herself a mark/comments 8 Already used 27% 101 35% 105 (0.1890) 18*** Planning to use 20% 101 38% 105 (0.0037) Have not and not -27*** 54% 101 27% 105 planning to use (0.0001) Students check each other' s work -23*** Already used 67% 101 44% 105 (0.0006) 26*** Planning to use 17% 101 43% 105 (0.0000)

IMPAQ International, LLC Page 98 Beoog Biiga II Baseline Report July 2017

Have not and not -3 16% 101 13% 105 planning to use (0.6120) The whole class checks the work of a student -14** Already used 62% 101 48% 105 (0.0335) 17*** Planning to use 21% 101 38% 105 (0.0064) Have not and not -3 17% 101 14% 105 planning to use (0.6161) Students write solutions on a slate and show to the teacher and class -28*** Already used 77% 101 49% 105 (0.0000) 15** Planning to use 18% 101 33% 105 (0.0108) Have not and not 13*** 5% 101 18% 105 planning to use (0.0032) Students of different skill levels are paired together -20*** Already used 69% 101 49% 105 (0.0024) 11* Planning to use 24% 101 35% 105 (0.0719) Have not and not 9** 7% 101 16% 105 planning to use (0.0385) Students of the same skill level are paired together -31*** Already used 41% 101 10% 105 (0.0000) -1 Planning to use 16% 101 15% 105 (0.9054) Have not and not 31*** 44% 101 75% 105 planning to use (0.0000) Teacher asks group (3 or more) of students to work together on a project and later provides feedback on group performance -18*** Already used 48% 101 30% 105 (0.0077) 22*** Planning to use 19% 101 41% 105 (0.0005) Have not and not -4 34% 101 30% 105 planning to use (0.5251) Source: Teacher Survey; IMPAQ calculations. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

IMPAQ International, LLC Page 99 Beoog Biiga II Baseline Report July 2017

Exhibit 51. ASER-Reading Test Results, Disaggregated by Sex (Grade 2) (Reading Level C: Read Complex Sounds)

Male Students 70 60 56 49 50

40 % 30 25 23

20 12 10 6 5 6 7 6 0 0 0 0 2 2 0 Level 0 Level A Level B Level C Level D Level E Level F Level G Level H Level I Level J Level K

Baseline Midline

Source: Student Survey; authors’ calculations. N=16 (baseline); N=43(midline).

Female Students 70 60 54 50 41

40 % 30 25 20 20 14 7 8 8 7 10 4 5 0 2 0 0 2 0 2 0 Level 0 Level A Level B Level C Level D Level E Level F Level G Level H Level I Level J Level K

Baseline Midline

Source: Student Survey; authors’ calculations. N=24 (baseline); N=44 (midline).

IMPAQ International, LLC Page 100 Beoog Biiga II Baseline Report July 2017

Exhibit 52. Reading Proficiency at Second Grade Level (ASER Level B: Reading Simple Sounds)

Baseline Midline Difference in Means Indicator Perce Perce Observations Observations (p- value) nt nt Second grade students 14 demonstrating reading ability at 32% 40 46% 87 (0.1550) grade level or above Male students demonstrating 21 reading ability at grade level or 19% 16 40% 43 (0.1384) above Female students demonstrating 10 reading ability at grade level or 42% 24 52% 44 (0.4106) above Source: Student Survey; authors’ calculations. * P-value < 0.1, ** p-value < 0.05, *** p-value < 0.01

IMPAQ International, LLC Page 101 Beoog Biiga II Baseline Report July 2017

APPENDIX D. ASER-READING TEST

IMPAQ International, LLC Page 102 Beoog Biiga II Baseline Report July 2017

IMPAQ International, LLC Page 103 Beoog Biiga II Baseline Report July 2017

APPENDIX E. ASER-READING TEST INSTRUCTIONS

IMPAQ International, LLC Page 104 Beoog Biiga II Baseline Report July 2017

IMPAQ International, LLC Page 105 Beoog Biiga II Baseline Report July 2017

APPENDIX F. QUESTIONNAIRES

Student Questionnaire

MIDLINE EVALUATION CRS FOOD FOR EDUCATION BURKINA FASO Student Survey

Basic Information

Last Name: …….. First Name: ID : Enum ……… Supervisor Who is your supervisor? Date Date (JJ/MM/AAAA) 1. Bam I__I Prov 2. Santamega Com Write the commune ‘s name Write the CEB’s name CODE CEB I__I__I__I__I__I__I__I__I Write the school name CODE School I__I__I__I__I__I__I__I__I Student ID CODE StudentID I__I__I__I__I__I__I__I__I preloadlastname What is the last name of the student? preloadfirstname What is the first name of the student? 1. Male I__I preloadgender 2. Female What grade/class are the student in? I__I 1. CP2 2. CE1 preloadgrade 3. CE2 4. CM1 5. CM2

IMPAQ International, LLC Page 106 Beoog Biiga II Baseline Report July 2017

Parent Consent: Has the caregiver given consent for her/his child to participate in this survey? parentconsent 1. Caregiver consented  parentname I__I 2. No consent  STOP- Consult with your supervisor, and proceed to the next child on your list. If you don’t have the parent’s consent terminate the survey. Consult with your supervisor, and proceed to the next child on your list. parentname What is the last name of the caregiver giving consent? parentfirstname What is the first name of the caregiver giving consent?

Dear student: You have been selected to participate in a survey about health, nutrition, and education for the Cantine Scolaire project. Your participation in this interview is entirely voluntary. If, at any time, you wish to discontinue participation, you may do so without penalty. If you accept, please respond to all questions as candidly as possible. If you do not know the answer to a question, you may simply say so. All responses will be kept strictly confidential. Do you accept if I ask you some questions? I__I *Select only one option 1. Yes  fname Consent 2. No  thanks 3. Not found  thanks

If child says No or Not found, thank him/her, terminate the survey and proceed to the next child on your list. N.B: If the respondent refuses to answer any question mark an "R" for the answer and proceed to the next question.

Personal Information Great! Now I want to ask you some questions about you …. fname What is your first name? lname What is your last name? Record gender I__I *select only one gender 1. Male option *Ask only if 2. Female necessary Do you know your Date of Birth? I__I *select only one dob 1. Yes option

IMPAQ International, LLC Page 107 Beoog Biiga II Baseline Report July 2017

2. No  yearborn

What is your Date of Birth? Day/Month/Year : [If answered → lsintro] dob1 [Jan 2000- Dec 2013] Do you know what year you were born? I__I *select only one yearborn 1. Yes option 2. No  qage yearborn What year were you born? [If answered → lsintro] 1 [Jan 2000- Dec 2013] Do you know your age? I__I *select only one qage 1. Yes option 2. No  lsintro How old are you? …… record number ageq . >=5 & =<17

Literacy Thank you! Now I would like to ask some questions about what you do in class… I__I *Select only one option I would like you to think about yesterday. Do *If No, talk to child about you remember what you did yesterday? yesterday - did you go to

school yesterday, did you lsintro 1. Yes play with your friends, 2. No etc. to help them remember. Did your class teacher make you read on your I__I * Select only one option own in class yesterday? ls1 1. Yes 2. No Did you spend time reading to someone else in I__I * Select only one option class yesterday? ls2 1. Yes 2. No Did you spend time writing in class yesterday? I__I * Select only one option ls3 1. Yes 2. No

IMPAQ International, LLC Page 108 Beoog Biiga II Baseline Report July 2017

Did you spend time doing word work in class I__I * Select only one option yesterday? ls4 1. Yes 2. No Did you spend time listening to someone’s I__I * Select only one option reading in class yesterday? ls5 1. Yes 2. No A reading group is a small group of students I__I * Select only one option who regularly meet outside of school for reading, vocabulary and reading comprehension ls6 activities. Do you take part in a reading group? 1. Yes 2. No  fs2 Does your reading group have a group notebook? Ls6f 1. Yes 2. No Does your reading group consist of all the I__I * Select only one option children in your class or a smaller number of ls6a children? 1. Everyone→ ls6c 2. Just some Does your reading group consist of <> or both? ls6b 1. Just boys/girls 2. Both How long have you been part of a reading I__I * Select only one option group? 1. Less than this school semester 2. All of this semester ls6c 3. All of this semester and part of last 4. Both this semester and last 5. Longer than 2 semesters 6. Can't remember/don't know How many times do you participate in the Ls6d reading group’s activities per month? 1. Once

IMPAQ International, LLC Page 109 Beoog Biiga II Baseline Report July 2017

2. Twice 3. Tree or more times 4. Don’t know Do you have an adult who gives you support for the reading groups? Ls6e 1. Yes 2. No  fs2 Who is the adult that supports you? 1. A teacher Ls6g 2. A mentor 3. Another member of the community

Food Security Now, I would like you to think about all the meals you ate yesterday. Now thinking about what you did yesterday, did you I__I * Select only one option eat something at home before coming to school? fs2 1. Yes 2. No→ fs3 Did you feel full after eating this meal or could you I__I * Select only one option have eaten more? fs2a 1. I felt full → fs3 2. I could have eaten more Why did you not eat more food? I__I * Select only one option 1. There wasn't any more food fs2b 2. There wasn't anything I liked; 3. Something else (specify) ______Did you go home to eat during lunch break yesterday? I__I * Select only one option fs3 1. Yes 2. No → fs4 Did you feel full after eating lunch at home or could I__I *Select only one option you have eaten more? fs3a 1. I felt full →fs4 2. I could have eaten more Why did you not eat more food? * Select only one option fs3b 1. There wasn't any more food I__I 2. There wasn't anything I liked; 3. Something else (specify) ______

IMPAQ International, LLC Page 110 Beoog Biiga II Baseline Report July 2017

Did you eat at the canteen while you were at school I__I * Select only one option yesterday? fs4 1. Yes 2. No → fs5 Did you feel full after eating at the canteen or could I__I *Select only one option you have eaten more? fs4a 1. I felt full → fs5 2. I could have eaten more Why did you not eat more food? I__I *Select only one option fs4b 1. There wasn't any more food 2. There wasn't anything I liked; 3. Something else (specify) ______Did you have dinner yesterday? I__I * Select only one option fs5 1. Yes 2. No Now think about when you went to bed last night, I__I * Select only one option did you feel full or did you feel you could eat more fs6a food? 1. I felt full →fs7 2. I could have eaten more Why did you not eat more food? I__I * Select only one option 1. There wasn't any more food fs6b 2. There wasn't anything I liked; 3. Something else (specify) ______We just talked about all the meals you ate yesterday. Now, I want you take a minute and think of all the different foods you ate yesterday. fs7 [INTERVIEW AID: USE FLASH CARDS WITH PICTURES OF EACH FOOD TYPE, so children can easily recognize what we are asking them about] Are you ready? Okay. Did you eat any: a. Cereals: porridge, bread, noodles, corn, millet, I__I * Select only one option sorghum, rice, cake, macaronis, boule d’acassa (foura), zoom-koom? fs7a 1. Yes  fs7a1 2. No

IMPAQ International, LLC Page 111 Beoog Biiga II Baseline Report July 2017

Did you eat it at home, school or at both places? I__I * Select only one option 1. Home fs7a1 2. School 3. Both b. White roots and tubers: Potatoes, yams, cassava, I__I * Select only one option manioc? fs7b 1. Yes  fs7b1 2. No

Did you eat it at home, school or at both places? I__I * Select only one option 1. Home fs7b1 2. School 3. Both c. Vitamin A rich vegetables and tuber: Carrots, I__I * Select only one option orange sweet potatoes, squash? fs7c 1. Yes  fs7c1 2. No

Did you eat it at home, school or at both places? I__I * Select only one option 1. Home fs7c1 2. School 3. Both d. Dark green leaves from manioc, baobab, sorrel, or I__I * Select only one option spinach? fs7d 1. Yes  fs7d1 2. No

Did you eat it at home, school or at both places? I__I * Select only one option 1. Home fs7d1 2. School 3. Both e. Other vegetables like eggplant, local eggplant, okra, I__I * Select only one option peppers, onions, tomatoes? fs7e 1. Yes  fs7e1 2. No

IMPAQ International, LLC Page 112 Beoog Biiga II Baseline Report July 2017

Did you eat it at home, school or at both places? I__I * Select only one option 1. Home fs7e1 2. School 3. Both f. Vitamin A rich fruit : Ripe mangoes, papaya, melon? I__I * Select only one option fs7f 3. Yes  fs7f1 4. No

Did you eat it at home, school or at both places? I__I * Select only one option 1. Home fs7f1 2. School 3. Both g. Other fruits: Watermelon, oranges, coconut, I__I * Select only one option tamarind, néré, shea, patagium (kaga)…. Or other wild fruits and their juices? Bissap juice and monkey fs7g bread (teudo) 1. Yes  fs7g1 2. No

Did you eat it at home, school or at both places? I__I * Select only one option 1. Home fs7g1 2. School 3. Both h. Organs: Internal organs: liver, heart, intestines, I__I * Select only one option etc.? fs7h 1. Yes  fs7h1 2. No

Did you eat it at home, school or at both places? I__I * Select only one option 1. Home fs7h1 2. School 3. Both i. Meat, such as: chicken, mutton, goat, duck, rabbit, I__I * Select only one option pork, beef, turkey, guinea fowl, or other fs7i poultry/birds? 1. Yes fs7i1 2. No

IMPAQ International, LLC Page 113 Beoog Biiga II Baseline Report July 2017

Did you eat it at home, school or at both places? I__I * Select only one option 1. Home fs7i1 2. School 3. Both j. Eggs from chicken, ducks, or other birds? I__I * Select only one option fs7j 3. Yes  fs7j1 4. No

Did you eat it at home, school or at both places? I__I * Select only one option 1. Home fs7j1 2. School 3. Both k. Fish, shrimp, or another type? I__I * Select only one option fs7k 1. Yes  fs7k1 2. No

Did you eat it at home, school or at both places? I__I * Select only one option 1. Home fs7k1 2. School 3. Both l. Lentils, beans, nuts, peanuts, groundnuts, sesame, I__I * Select only one option boule d’arachide (Moore : mougoudougou / Fulfulde : sorondobo), touto d’arachide (kourakoura), beignet fs7l de haricot (gonre)? 1. Yes  fs7l1 2. No

Did you eat it at home, school or at both places? I__I * Select only one option 1. Home fs7l1 2. School 3. Both m. Milk, cheese, yogurt? I__I * Select only one option fs7m 1. Yes  fs7m1 2. No

Did you eat it at home, school or at both places? I__I * Select only one option fs7m 1 1. Home

IMPAQ International, LLC Page 114 Beoog Biiga II Baseline Report July 2017

2. School 3. Both

n. Oil, butter and other fats? I__I * Select only one option fs7n 1. Yes  fs7n1 2. No

Did you eat it at home, school or at both places? I__I * Select only one option 1. Home fs7n1 2. School 3. Both o. Sweets: sugar, honey, candy, chocolate, cookies? I__I * Select only one option fs7o 1. Yes  fs7o1 2. No

Did you eat it at home, school or at both places? I__I * Select only one option 1. Home fs7o1 2. School 3. Both

Nutrition Knowledge Let’s continue to talk about the meals you had yesterday. Let’s continue to think about meals. Have you I__I * Select only one heard of Vitamin A? option vita 1. Yes 2. No → iron What do you think Vitamin A does? *Select all that apply I__I 1. Good vision I__I 2. Protects against diseases I__I vita1a 3. Helps with growth I__I 4. Keeps skin healthy I__I 5. Other (specify) ______I__I 6. I don’t know Can you name a food that has Vitamin A? *Select all that apply I__I vita2a 1. Liver I__I 2. Small whole fish I__I 3. Dairy products (milk, yogurt) I__I

IMPAQ International, LLC Page 115 Beoog Biiga II Baseline Report July 2017

4. The egg yolk I__I 5. Carrot I__I 6. The orange-fleshed sweet potato I__I 7. Squash (orange) I__I 8. Vegetable edible dark green leaves I__I 9. Red palm oil I__I 10. Papaya I__I 11. Mango I__I 12. Néré four I__I 13. Green bell pepper I__I 14. Melon (yellow and orange) I__I 15. Other (specify)______I__I 16. I don’t know We just finished talking about vitamin A. Do you I__I * Select only one know that there is iron in the food we eat? option iron 1. Yes 2. No→ Nuttimeout

What do you think iron does? I__I * Select all that apply I__I 1. Good vision I__I 2. Protects against diseases I__I 3. Helps with growth I__I 4. Muscle function I__I 5. Brain function I__I iron2a 6. Regulates body temperature I__I 7. Prevents fatigue I__I 8. Oxygen carrier in the body I__I 9. Maintains healthy skin I__I 10. Hemoglobin formation I__I 11. Other (specify) ______12. I don’t know Can you name a food that has iron? Any others? * Select all that apply

1. Fish I__I 2. Poultry I__I 3. Red meat I__I iron3a 4. Organ meats (liver) I__I 5. Beans I__I 6. Dried peas I__I 7. Lentils I__I 8. Cowpeas I__I

IMPAQ International, LLC Page 116 Beoog Biiga II Baseline Report July 2017

9. dark green leafy vegetables (baobab leaf, I__I spinach) I__I 10. Eggs I__I 11. Potatoes I__I 12. Tofu I__I 13. Green beans I__I 14. Nuts (sesame, cashew) I__I 15. Green bell pepper I__I 16. Watermelon I__I 17. Tomato I__I 18. Other (specify) ______I__I 19. I don’t know Hygiene Health Okay, we are almost done! Now I have some questions about hygiene. Did you wash your body this morning before coming I__I * Select only one to school? option body 1. Yes→bwash 2. No Did you wash your body yesterday? I__I * Select only one body1 1. Yes option 2. No→hand

What did you use to wash your body? I__I * Select all that apply I__I *If they say Water, 1. Water I__I PROBE – Did you use bwash 2. Ashes I__I anything else in 3. Sand I__I addition to water 4. Soap *Do not read the list 5. Other (Specify) ______Did you wash your hands today? I__I * Select only one 1. Yes option hand 2. No→handwash 3. Don’t know

What did you use to wash your hands? I__I * Select all that apply I__I *If they say Water, 1. Water I__I PROBE – Did you use hand1 2. Ashes I__I anything else 3. Sand *Do not read the list 4. Soap 5. Other (Specify) ______

IMPAQ International, LLC Page 117 Beoog Biiga II Baseline Report July 2017

In your opinion, when do you think a person should I__I * Select all that apply wash their hands? I__I I__I 1. Before eating? I__I handwas 2. Before touching or preparing food? I__I h 3. Before giving food to someone else? I__I 4. When you have dirt on your hands? 5. After touching something dirty? 6. After using the latrine? Did you wash your hands today? I__I * Select only one a. Before eating? option hand2a 1. Yes 2. No 3. Did not eat today

b. Before touching or preparing food? I__I * Select only one 1. Yes option hand2b 2. No 3. Did not touch/prepare food today c. Before giving food to someone else? I__I * Select only one 1. Yes option hand2c 2. No 3. Did not give food to others today d. When you had dirt on your hands? I__I * Select only one 1. Yes option hand2d 2. No 3. Did not have dirt on hands today e. After touching something dirty? I__I * Select only one 1. Yes option hand2e 2. No 3. Did not touch something dirty today f. After using the latrine? I__I * Select only one 1. Yes option hand2f 2. No 3. Did not use latrine today teachwash Did your teacher ask about washing your hands I__I * Select only one 1 yesterday? option

IMPAQ International, LLC Page 118 Beoog Biiga II Baseline Report July 2017

1. Yes→teachwash3 2. No 3. Can’t remember/don’t know Did your teacher ask about washing your hands I__I * Select only one this in the last 5 school days? option teachwash 1. Yes 2 2. No 3. Can’t remember/don’t know Did your teacher teach you about the importance I__I * Select only one of washing your hands? option teachwash 1. Yes 3 2. No  water 3. Can’t remember/don’t know Did the teacher tell you about the critical moments I__I * Select only one when to wash your hands? option Teachwash 1. Yes 4 2. No 3. Can’t remember/don’t know Do you have access to water at home for washing? I__I * Select only one water 1. Yes option 2. No

If the child is in grade 2 (CP2) got to readassess, otherwise thanks the students and terminate the survey.

Reading Assessment: Here is the final set of questions! I would like to play a little game with you... What level did the student read at? I__I * Select only one 1. 0 option 2. A * Only if they are in second grade 3. B readassess 4. C 5. D 6. E 7. F 8. G 9. H

IMPAQ International, LLC Page 119 Beoog Biiga II Baseline Report July 2017

10. I 11. J 12. K 13. No Assessment Why didn’t the student take the test? I__I * Select only one readassess 1. Not CP2 option 2 2. The CP2 child refused or could not do it (redo the survey with a replacement child) thanks Thank you so much for answering my questions. Candy for everyone!

OBSERVATION S

OBSERVATION : Hand washing Which critical moment did you observe: I__ 1. Before eating I *If 3, finish the observations Wash1 2. After using the latrines 3. Did not eat or use the latrines (Specify * Select only one why:______) option

How did the student wash his/her hands? I__ 1. Water I * Select only one Wash2 2. Water and soap option 3. Other (specify)(______) 4. Did not wash his/her hands

IMPAQ International, LLC Page 120 Beoog Biiga II Baseline Report July 2017

MIDLINE EVALUATION CRS FOOD FOR EDUCATION BURKINA FASO Parent Survey BASIC INFORMATION Last Name: …….. First Name: ID : enum ……… supervisor Who is your supervisor? date Date (JJ/MM/AAAA)

prov 3. Bam I__I 4. Santamega com Write the commune ‘s name ceb Write the CEB’s name CODE I__I__I__I__I__I__I__I__I school Write the school name CODE I__I__I__I__I__I__I__I__I

Studgrade 1. CP2 2. CE1 3. CE2 4. CM1 5. CM2 studentid Student’s name CODE I__I__I__I__I__I__I__I__I

Dear mother: Hello, my name is . You have been selected to participate in a survey about health, nutrition, and education for the Cantine Scolaire project. Your participation in this interview is entirely voluntary and you have no obligation to participate. If, at any time, you wish to discontinue participation, you may do so without penalty. If you accept, please respond to all questions as candidly as possible. If you do not know the answer to a question, you may simply say so. All responses will be kept strictly confidential.

Are you the parent or primary caregiver of I__I *Select only one <>? option parent 3. Yes, parent 4. Yes, primary or secondary caregiver 5. No  STOP – thanks them and terminate the survey. Consult with your supervisor,

IMPAQ International, LLC Page 121 Beoog Biiga II Baseline Report July 2017

and proceed to the next mother on your list.

If the respondent is not the parent or caregiver of the student on your list, thank them and terminate the survey. Consult with your supervisor, and proceed to the next mother on your list. Can I talk to you for a few minutes? I have some I__I * Select only one questions to ask you about some of the things option you do every day. Is it okay if I ask you some questions? Consentp 3. Yes 4. No  thanks them and terminate the survey, and proceed to the next mother on your list.

If the parent or caregiver of the student says NO, thank him/her, terminate the survey and proceed to the next mother on your list. N.B: If the respondent refuses to answer any question mark an "R" for the answer and proceed to the next question.

Personal Information Great! Now I want to ask you some questions about you …. parentnam What is your first name? e parentlna What is your last name? me Ask only if needed I__I * Select only one genderp 1. Male option 2. Female Do you know your date of birth? I__I * Select only one dobp 3. Yes option 4. I don’t know→yearbornp What date were you born? Day/Month/Year *If answered →kid dop1 [Jan 1916 – Dec 2004] Do you know what year you were born? I__I * Select only one yearbornp 1. Yes option 2. I don’t know →qagep yearbornp *If answered →kid In what year were you born? 1 *[1916 -2002] Do you know your age? I__I * Select only one qagep 1. Yes option

IMPAQ International, LLC Page 122 Beoog Biiga II Baseline Report July 2017

2. I don’t know→ kid ……. *record age agep How old are you? . >=13 & =<99

Household Environnent Good! Now I would like ask you a few questions about your household How many children 16 years old or younger live ……. >=0 record number kid with you? . =<25 How many of the children 16 years old or ……. >=0 record number younger who live with you are your own . =<25 ownkid children? cannot exceed kid

……. >=0 record number kid05 How many of the children living with you are 5 . =<25 years or younger? cannot exceed kid ……. >=0 record number kid612 How many of the children living with you are . =<25 between the ages of 6 and 12? cannot exceed kid How many people 17 years or older, including ……. >=0 record number adult yourself, live in the place where you live? . =<25 Is the roof in the place where you live made of I__I * Select only one thatch, metal or something else? option roof 1. Thatch 2. Metal 3. Combination for straw and metal 4. Something else elec2 What energy do you use for lighting? City provided electricity I__I * Select only one elec2a 1. Yes option 2. No Solar I__I * Select only one elec2b 1. Yes option 2. No Generator I__I * Select only one elec2c 1. Yes option 2. No Car battery I__I * Select only one elec2d option 1. Yes

IMPAQ International, LLC Page 123 Beoog Biiga II Baseline Report July 2017

2. No Other (specify) I__I * Select only one elec2e 3. Yes option 4. No Do you have a latrine at your home? I__I * Select only one latrine 1. Yes option 2. No

I__I * Select only one option What type of latrine is it? *Have flash cards with 1. Flush or pour pictures of different 2. Ventilated pit latrine types of latrines, so lattyp 3. Pit latrine with slab respondent can easily 4. Pit latrine without slab identify the type. Also, 5. Bucket latrines include description for interviewers to use, if needed. stuff Do you have any of the following items in the place where you live? Table or desk I__I * Select only one stuffa 1. Yes option 2. No Telephone or mobile phone I__I * Select only one stuffb 1. Yes option 2. No Bicycle I__I * Select only one stuffc 1. Yes option 2. No Motorcycle I__I * Select only one stuffd 1. Yes option 2. No Radio I__I * Select only one stuffe 1. Yes option 2. No

IMPAQ International, LLC Page 124 Beoog Biiga II Baseline Report July 2017

Television I__I * Select only one stufff 1. Yes option 2. No Books, newspapers, or magazines I__I * Select only one stuffg 1. Yes option 2. No A separate room for the children I__I * Select only one stuffh 1. Yes option 2. No Can you read French or in any local language? I__I * Select only one Would that be French, a local language, or option both? read 1. Yes, local 2. Yes, French 3. Yes, both 4. No What about writing? Can you write in French I__I * Select only one or any local language? Would that be French, a option local language or both? write 1. Yes, local 2. Yes, French 3. Yes, both 4. No What is the highest grade in school that you I__I * Select only one have completed? option

0. None 1. 1-4th grade 2. 5-6th grade edu 3. 7-10th grade 4. 10th-13th grade 5. Some college 6. Completed Bachelor’s 7. More than Bachelor’s 8. Vocational or professional school

IMPAQ International, LLC Page 125 Beoog Biiga II Baseline Report July 2017

What is your main job or work? I__I * Select only one option 1. Paid worker: Farming/livestock 2. Own Farming/Livestock 3. Teacher 4. Government job jobtyp 5. Stay at home, take care of the home and children 6. Self-employed - business, craft and trade 7. Mining 8. Other (specify) ______livestock Which of the following livestock do you work with? Chicken/poultry I__I * Select only one 1. Yes option livestocka 2. No Goats/sheep I__I * Select only one 1. Yes option livestockb 2. No Cows I__I * Select only one 1. Yes option livestockc 2. No Donkeys I__I * Select only one 1. Yes option livestockd 2. No Pigs I__I * Select only one livestocke 1. Yes option 2. No Are you a member of a savings group? I__I * Select only one ceci 1. Yes option 2. No→ meet Have you used a portion of your savings in the I__I * Select only one savings group for school expenses? option ceci2 1. Yes 2. No→ meet

IMPAQ International, LLC Page 126 Beoog Biiga II Baseline Report July 2017

Did the saving help for schooling and tuition I__I * Select only one fee? option ceci3 1. A little 2. Moderately 3. Significantly

School Interaction Thank you! Now, I would like to ask you a few questions about your participation in school activities In the last 12 months, did you, or another adult I__I * Select only one in your household, meet one-on-one with option <>’s teacher for any reason? meet 1. Yes, I met 2. Yes, another adult 3. Yes, me and another adult 4. No  act How many times did <> meet one-on-one with option <>'s teacher in the last 12 meetnum months? 1. Once 2. Two or three times 3. More than three times What were the reasons why <> met one-on-one for I__I Periodic school scheduled parent-teacher I__I conference with <>'s I__I teacher in the last 12 months? I__I meetwhy I__I 7. Child’s performance 8. Disciplinary issues 9. Just touching base 10. Other (specify) ______

IMPAQ International, LLC Page 127 Beoog Biiga II Baseline Report July 2017

In the last 12 months, did you or another adult I__I * Select all that apply in your household: I__I I__I 1. Attend a class while the teacher was I__I teaching? I__I

2. Have a meal with <> while <> was at school I__I 3. Attend a PTA general assembly (APE/ I__I AME) I__I 4. Participate in a school community act project (such as cleaning or community fields) 5. Help the school as cook or store keeper 6. Help watch over a reading group 7. Visit the classroom 8. Attend a performance put on by children in <>'s class? 9. Something else? (specify)______Are you a member of the PTA at ’s I__I * Select only one school? option PTA 1. Yes 2. No In the past it was mainly boys who went to I__I * Select only one school. Nowadays both boys and girls are going option to school. In your opinion, is this a good thing girl or a bad thing?? 3. Good →girlgood 4. Bad→girlbad 5. Don’t know→lib Why do you think it’s a good thing for girls to go I__I * Select only one to school? option 3. Educating girls improves living standards of the whole family girlgood 4. Educating girls improves their health 5. Educating girls improves the health of the children they will have; 6. Girls should also be able to fulfill themselves;

IMPAQ International, LLC Page 128 Beoog Biiga II Baseline Report July 2017

7. Educating girls allows them to find better work 8. Other (specify) ______

Why do you think that it’s a bad thing for girls I__I * Select only one to go to school? option 1. Girls are supposed to stay at home; 2. Girls don’t need school; 3. Girls don’t work outside the home, so what is the point? girlbad 4. Girls should not be out in public; 5. There are no school for girls only and they should not be going to school with boys 6. Other (specify) ______Do you know where the library closest to where I__I * Select only one you live is located? option lib 3. Yes 4. No Have you ever visited a library? I__I * Select only one libvisit 1. Yes option 2. No→libkid What kinds of books/publications do you borrow from the library most of the times? 1. Comic book 2. Children’s book 3. Books in local language libborrow 4. African literature 5. European literature 6. Other (specify) ______7. Don’t know Now, think back to the last six months - that I__I * Select only one would be last <>. Did you visit a option library within the last six months that is since libfreq <>? 1. Yes 2. No→libkid

IMPAQ International, LLC Page 129 Beoog Biiga II Baseline Report July 2017

How many times in the last six months did you ……. *record number liblastvisit visit a library that is since <>? >=0 & <=50 Has <> ever visited a library I__I * Select only one either with you, with the school or with option libkid someone else? 1. Yes 2. No

Food Security and Diet Now I want you to take a minute and think about all the food being prepared in your household… Are you the person who makes the decision I__I * Select only one regarding what children eat in your household? option decide 3. Yes→ cook 4. No Who in your household makes decisions regarding what the children eat? decwho (enumerator: record what the relationship to that person is, as well as his/her name) Are you the person who prepares or cooks the I__I * Select only one food for the household? option cook 1. Yes→ nunmeal 2. No cookwho Who in your household prepares or cooks food? (record response) How many meals did children 16 years and …… *record number nummea younger living in your household eat yesterday? >=0 & <=9 l

Now, I want you take a minute and think of all the food you or another adult offered fs7P <> yesterday. Did you serve: a. Cereals: porridge, bread, noodles, corn, I__I * Select only one millet, sorghum, macaronis, cake, acassa, rice..? option fs7Pa 3. Yes→ fs7Pa1 4. No How many times did you serve this to her/him ……. *record number [0-9] fs7Pa1 yesterday? b. White roots and tubers: potatoes, yams, I__I *Select only one cassava...? option fs7Pb 1. Yes→ fs7Pb1 2. No

IMPAQ International, LLC Page 130 Beoog Biiga II Baseline Report July 2017

How many times did you serve this to her/him ……. *record number fs7Pb1 yesterday?

c. Vitamin A rich vegetables and tuber: carrots, I__I * Select only one orange sweet potatoes...? option fs7Pc 1. Yes→ fs7Pc1 2. No How many times did you serve this to her/him ……. *record number fs7Pc1 yesterday? d. Dark green leaves of cassava, baobab, I__I * Select only one manioc, baobab, sorrel, spinach … ? option fs7Pd 1. Yes→ fs7Pd1 2. No How many times did you serve this to her/him ……. *record number fs7Pd1 yesterday? e. Other vegetables as in eggplant, gombo, bell I__I * Select only one peppers, tomatoes, onions..? option fs7Pe 1. Yes→ fs7Pe1 2. No How many times did you serve this to her/him ……. *record number fs7Pe1 yesterday? f. Vitamin A rich fruits: ripe mangoes, papaya, I__I * Select only one melon...? option fs7Pf 1. Yes→ fs7Pf1 2. No How many times did you serve this to her/him ……. *record number fs7Pf1 yesterday? g. Other fruits: watermelon, oranges, coconut, I__I * Select only one tamarind, néré, karité, patagium (kaga), or option fs7Pg other wild fruits and their juice? 1. Yes→ fs7Pg1 2. No How many times did you serve this to her/him ……. *record number fs7Pg1 yesterday? h. Organs: liver, heart, intestines…? I__I * Select only one fs7Ph 1. Yes→ fs7Ph1 option 2. No

IMPAQ International, LLC Page 131 Beoog Biiga II Baseline Report July 2017

How many times did you serve this to her/him ……. *record number fs7Ph1 yesterday?

i. Meat of chicken, sheep, goat, duck, rabbit, I__I * Select only one pork, cow, turkey, or other poultry? option fs7Pi 1. Yes→ fs7Pi1 2. No How many times did you serve this to her/him ……. *record number fs7Pi1 yesterday? j. Eggs, from chicken, ducks, or other birds? I__I * Select only one fs7Pj 1. Yes→ fs7Pj1 option 2. No How many times did you serve this to her/him ……. *record number fs7Pj1 yesterday? k. Fish, shrimp, or other seafood? I__I * Select only one fs7Pk 1. Yes→ fs7Pk1 option 2. No How many times did you serve this to her/him ……. *record number fs7Pk1 yesterday? l. Lentils, beans, groundnuts, other nuts, ground I__I * Select only one peas, peanut balls? (Moore : mougoudougou / option Fulfulde : sorondobo), touto d’arachide fs7Pl (kourakoura), beignet de haricot (gonre)? 1. Yes→ fs7Pl1 2. No How many times did you serve this to her/him ……. *record number fs7Pl1 yesterday? m. Milk, cheese, yogurt? I__I * Select only one fs7Pm 1. Yes→ fs7Pm1 option 2. No How many times did you serve this to her/him ……. *record number fs7Pm1 yesterday? n. Oil, butter and other fats? I__I * Select only one fs7Pn 1. Yes→ fs7Pn1 option 2. No How many times did you serve this to her/him ……. *record number fs7Pn1 yesterday?

IMPAQ International, LLC Page 132 Beoog Biiga II Baseline Report July 2017

o. Sweets: sugar, honey, candy chocolate, I__I * Select only one cookies? option fs7Po 1. Yes→ fs7Po1 2. No How many times did you serve this to her/him ……. *record number [0-9] fs7Po1 yesterday? Now, I want you to think about the last month. I__I * Select only one Was there a time in the last month, when you option were worried that you did not have enough pfs1 food to feed your family? 1. Yes 2. No Was there a time in the last month when you I__I * Select only one or another adult in your household ate a option smaller meal because there wasn't enough pfs2 food? 1. Yes 2. No Was there a time in the last month when you I__I * Select only one or another adult in your household did not eat option pfs3 a meal because there wasn't enough food? 1. Yes 2. No Was there a time in the last month, when one I__I * Select only one or more of your children ate a smaller meal option pfs4 because there wasn't enough food? 1. Yes 2. No Was there a time in the last month, when one I__I * Select only one or more of your children did not eat a meal option pfs5 because there wasn't enough food? 1. Yes 2. No Was there a time in the last month, when there I__I * Select only one was no food in your household because you option pfs6 could not afford to buy any? 1. Yes 2. No

IMPAQ International, LLC Page 133 Beoog Biiga II Baseline Report July 2017

Was there a time in the last month, when you I__I * Select only one or another adult in your household went a option whole day without eating because there wasn't pfs7 enough food? 1. Yes 2. No Was there a time in the last month, when one I__I * Select only one or more of the children in your household went option a whole day without eating because there pfs8 wasn't enough food? 1. Yes 2. No

Nutrition Knowledge That's quite a lot of different foods you ate yesterday. Let’s continue to think about meals. Now, let me ask you, have you heard of Vitamin A? I__I * Select only Vitap 3. Oui one option 4. Non→ironp What do you think Vitamin A does? *Select all that I__I apply 1. Good vision I__I 2. Protects against diseases I__I Vitap1a 3. Helps with growth I__I 4. Keeps skin healthy I__I 5. Other (specify) I__I 6. I don’t know Can you name food that has Vitamin A? I__I I__I 17. Liver I__I 18. Small whole fish I__I 19. Dairy products (milk, yogurt) I__I 20. The egg yolk I__I 21. Carrot I__I Vitap2a 22. The orange-fleshed sweet potato I__I 23. Courge (orange) I__I 24. Vegetable edible dark green leaves I__I 25. Red palm oil I__I 26. Papaya I__I 27. Mango I__I 28. Néré flour I__I

IMPAQ International, LLC Page 134 Beoog Biiga II Baseline Report July 2017

29. Green bell pepper I__I 30. Melon (yellow/orange) I__I 31. Other (specify)______I__I 32. I don’t know We just finished talking about vitamin A. [Have you I__I * Select only heard of iron?] Do you know that there is iron in the one option food that we eat? Ironp 3. Yes 4. No→ handp

What do you think iron does? * Select all that I__I apply 13. Good vision I__I 14. Protects against diseases I__I 15. Helps with growth I__I 16. Muscle function I__I 17. Brain function I__I Ironp2a 18. Regulates body temperature I__I 19. Prevents fatigue I__I 20. Oxygen carrier in the body I__I 21. Maintain healthy skin I__I 22. Hemoglobin formation I__I 23. Other (specify) ______I__I 24. I don’t know Can you name food that has iron? I__I 20. Fish I__I 21. Poultry I__I 22. Red meat I__I 23. Organ meats (liver) I__I 24. Beans I__I 25. Dried peas I__I 26. Lentils I__I 27. Cowpeas I__I Iron3a 28. dark green leafy vegetables (baobab leaf, I__I spinach) I__I 29. Eggs I__I 30. Potatoes I__I 31. Tofu I__I 32. Green beans I__I 33. Nuts (sesame, cashew) 34. Green bell pepper 35. Watermelon

IMPAQ International, LLC Page 135 Beoog Biiga II Baseline Report July 2017

36. Tomato 37. Other (specify) ______38. I don’t know

Hygiene Health Okay, we are almost done! Now I have some questions about washing now. Did you wash your hands today? I__I * Select only one Handp 3. Oui option 4. Non→handbwashP What did you use to wash your hands? [Do not give * Select all that examples or read the options to the respondent] I__I apply I__I *Do not give Handp1 6. Water I__I examples or read 7. Ashes I__I list to respondent 8. Sand 9. Soap In your opinion, when do you think a person should I__I * Choose all that wash their hands? I__I apply I__I 7. Before eating? I__I Handwash 8. Before touching or preparing food? I__I p 9. Before giving food to someone else? I__I 10. When you have dirt on your hands? 11. After touching something dirty? 12. After using the latrine? Handp2 Did you wash your hands today? a. Before eating? I__I * Select only one 4. Yes option Handp2a 5. No 6. Did not eat today b. Before touching or preparing food? I__I * Select only one 4. Yes option Handp2b 5. No 6. Did not touch/prepare food today c. Before giving food to someone else? I__I * Select only one 4. Yes option Handp2c 5. No 6. Did not give food to others today

IMPAQ International, LLC Page 136 Beoog Biiga II Baseline Report July 2017

d. When you have dirt on your hands? I__I * Select only one 4. Yes option Handp2d 5. No 6. Did not have dirt on hands today e. After touching something dirty? I__I * Select only one 4. Yes option handP2e 5. No 6. Did not touch something dirty today f. After using the latrine? I__I * Select only one 4. Yes option handP2f 5. No 6. Did not use latrine today Do you have water at home for washing? I__I * Select only one waterP 3. Yes option 4. No

How far do you have to go to get water? I__I * Select only one 1. Less than 1/2 km option 2. 1/2 km to 1 km source 3. 1 to less than 2 kms 4. 2 to less than 3 kms 5. 4 to less than 5 kms 6. 5 kms or more thank2 Thank you so much for answering my questions. This test was approved by the supervisor I__I * Select only one APPROB 1. Yes option 2. No

IMPAQ International, LLC Page 137 Beoog Biiga II Baseline Report July 2017

Teacher Questionnaire

MIDLINE EVALUATION CRS FOOD FOR EDUCATION BURKINA FASO Teacher Questionnaire

BASIC INFORMATION

Enum Last Name: …….. First Name: ……… ID :

date Date (DD/MM/YYYY)

Supervisor Who is your supervisor?

Prov 5. Bam I__I 6. Santamega Com Write the commune ‘s name

CEB Write the CEB’s name CODE I__I__I__I__I__I__I__I__I School Write the school name CODE I__I__I__I__I__I__I__I__I studentid CODE I__I__I__I__I__I__I__I__I

preloadfnameo Student’s first name

Preloadlnameo Student’s last name

1. CP2 I__I 2. CE1 Preloadstudgrade 3. CE2 4. CM1 5. CM2 Select only one option 1. Male preloadstudgender 2. Female Is the person the teacher of the student on your list? Teacherconfirm 1. Yes 2. No

IMPAQ International, LLC Page 138 Beoog Biiga II Baseline Report July 2017

Dear Teacher : You have been selected to participate in a survey about health, nutrition, and education for the Cantine Scolaire project. Your participation in this interview is voluntary. If, at any time, you wish to discontinue participation, you may do so without penalty. If you accept, please respond to all questions as candidly as possible. If you do not know the answer to a question, you may simply say so. All responses will be kept strictly confidential.

Do you accept participation in this survey? I__I * Select only one 1. Yes  lastname option 2. No  STOP – thank them and terminate Consent the survey. Consult with your supervisor, and proceed to the next teacher on your list.

If the response to “consent” is NO, thank the respondent and terminate the survey and proceed to the next teacher on your list.

Personal Information Great! Now I would like to ask some questions about you… lastname What is your last name? ______name What is your first name? ______Ask only if necessary I__I * Select only one genderq 3. Male option 4. Female Do you know your date of birth? I__I * Select only one dob 5. Yes option 6. I don’t know →yearborn What is your date of birth? Day/Month/Year If answered →gradeteach dob1

Do you know what year you were born? I__I * Select only one option yearborn 3. Yes 4. I don’t know →qagep yearborn What year were you born? If answered →gradeteach 1 *[1916 -2002] Do you know your age? I__I * Select only one qage 1. Yes option

IMPAQ International, LLC Page 139 Beoog Biiga II Baseline Report July 2017

2. I don’t know →gradeteach ……. *record age Ageq How old are you? >=13 & =<99 In which grade do you mainly teach? I__I * Select only one 1. CP1 option 2. CP2 gradetea 3. CE1 ch 4. CE2 5. CM1 6. CM2 What grade do you currently teach? I__I * Select only one 1. CP1 option gradeno 2. CP2 w 3. CE1 4. CE2 5. CM1 6. CM2 How many children are in your main class? …… *record number>=0 kid … and =<150 How long have you been working as a teacher? I__I * Select only one 1. Less than a year option 2. 1-2 years teachlen 3. 3-5 years 4. 6-10 years 5. 11 years or more How long have you been teaching this grade? I__I * Select only one 1. Less than a year option 2. 1-2 years gradelen 3. 3-5 years 4. 6-10 years 5. 11 years or more How long have you been teaching your main I__I * Select only one class? option 1. Less than a year classlen 2. 1-2 years 3. 3-5 years 4. 6-10 years 5. 11 years or more

IMPAQ International, LLC Page 140 Beoog Biiga II Baseline Report July 2017

Do you have any administrative duties, other I__I * Select only one than those for your classes that you do regularly option admin for the school? 3. Yes 4. No→edu What percent of your time, on average, I__I * Select only one throughout the school year, would you estimate option that you spend on these administrative duties? adminpct 1. 0% - 25% 2. 26% - 50% 3. 51 % - 75% 4. 76% - 100% What is the highest level of education you have I__I * Select only one completed? option 1. BEPC 2. Baccalaureat edu 3. Bachelor’s degree 4. Master’s degree 5. Master 1 6. Master 2 7. Other (specify) ______Have you ever received formal training in I__I * Select only one teaching children such a completing a teaching option certification program or a course in teaching? 1. Yes, I have a certificate or degree in teachcert teaching 2. Yes, I have taken classes or courses but do not have certificate or degree 3. No Have you ever received formal training in I__I * Select only one teaching children to read and write such as option completing a certification program or a course in teaching these techniques? readcert 1. Yes, I have a certificate or degree in teaching 2. Yes, I have taken classes or courses but do not have a certificate or degree 3. No

IMPAQ International, LLC Page 141 Beoog Biiga II Baseline Report July 2017

Literacy Instruction Thank you! Now I would like to ask some questions about teaching and literacy instruction I__I * Select only one Do you know what “learning style” means? option style 1. Yes 2. No→lect1 In your own words, can you please describe I__I *Select all that apply what learning style means to you? I__I *do not read the I__I options styledesc 11. Listening/feeling I__I 12. Watching/observing 13. Thinking 14. Doing On average, how much time do you spend I__I * Select only one teaching reading each week? option 1. 0-10% lect1 2. 11%-20% 3. 21%-40% 4. 41%-60% 5. 61%-80% 6. 81%-100% Please give me an example of an activity or I__I *Select only one technique that you use to teach reading. option 1. Phonological awareness *Do not read the list 2. Phonetics or give examples 3. Standard reading 4. Vocabulary lectex1 5. Comprehension 6. Identification of key sounds in another sentence 7. Identification for key sounds 8. Repetition of words, sounds, and syllables 9. Reading with punctuation symbols 10. Don’t know→vocab How much time, in minutes, do you spend on …… *Record time in lectex1a this activity, on average, every day? … minutes (0-600) Please give me another example of an activity I__I *Select only one that you use to teach reading. option lectex2 1. Phonological awareness *Do not read the list 2. Phonetics or give examples

IMPAQ International, LLC Page 142 Beoog Biiga II Baseline Report July 2017

3. Standard reading 4. Vocabulary 5. Comprehension 6. Identification of key sounds in another sentence 7. Identification for key sounds 8. Repetition of words, sounds, and syllables 9. Reading with punctuation symbols 10. Don’t know→vocab How much time, in minutes, do you spend on …… *Record time in lectex2a this activity, on average, every day? … minutes (0-420) Please give me another example of an activity I__I *Select only one that you use to teach reading. option 1. Phonological awareness *Do not read the list 2. Phonetics or give examples 3. Standard reading 4. Vocabulary 5. Comprehension lectex3 6. Identification of key sounds in another sentence 7. Identification for key sounds 8. Repetition of words, sounds, and syllables 9. Reading with punctuation symbols 10. Don’t know→vocab How much time, in minutes, do you spend on …… *Record time in lectex3a this activity, on average, every day? … minutes (0-600) Please give me another example of an activity I__I *Select only one that you use to teach reading. option [Do not read the list or give examples] *Do not read the list 1. Phonological awareness or give examples 2. Phonetics 3. Standard reading lectex4 4. Vocabulary 5. Comprehension 6. Identification of key sounds in another sentence 7. Identification for key sounds 8. Repetition of words, sounds, and syllables

IMPAQ International, LLC Page 143 Beoog Biiga II Baseline Report July 2017

9. Reading with punctuation symbols 10. Don’t know→vocab How much time, in minutes, do you spend on …… *Record time in lectex4a this activity, on average, every day? … minutes (0-420) Do you include vocabulary in your I__I * Select only one reading/writing activities? option vocab 1. Yes→comp 2. No 3. I don’t know→comp Why do you not use vocabulary in your *Select all that apply I__I reading/writing activities? I__I

I__I 1. Not appropriate for the grade I__I 2. Student should already know the skill vocab1 I__I 3. I am not sure what it is 4. I don't think students should learn it I__I 5. I don't think that is it is the right technique 6. Other (specify) ______Do you include comprehension in your I__I * Select only one reading/writing activities? option comp 1. Yes→read 2. No 3. I don’t know→read Why do you not use comprehension in your I__I * Select all that apply reading/writing activities? I__I I__I 1. Not appropriate for the grade 2. Student should already know the skill I__I comp1 3. I am not sure what it is I__I 4. I don't think students should learn it I__I 5. I don't think that is it is the right technique 6. Other (specify) ______Are you either planning to or have you already I__I * Select only one had your students read by themselves today? option 1. Yes, I already did it read 2. Yes, I plan to do it 3. No, I haven’t done it and don’t plan to  readno

IMPAQ International, LLC Page 144 Beoog Biiga II Baseline Report July 2017

How long <> read by themselves . of minutes readyes today? Please tell me how many minutes on average. Do you usually have the children read by I__I * Select only one themselves? option readno 1. Yes 2. No→readp How often do you usually have the children I__I * Select only one read by themselves? option 1. Never readfreq 2. Once per week 3. 2-4 times per week 4. Every day 5. Not sure/I don’t know For how long do the children usually read by ……. * Record the number readnoye themselves when they do this activity? Please . of minutes s tell me how many minutes on average a day. Are you either planning to or have already had I__I * Select only one your students read to one or more other people option today? readp 1. Yes, I already did it 2. Yes, I plan to do it 3. No, I haven’t done it and don’t plan to  readpno How long <> read to one or more . of minutes readpyes other people today? Please tell me how many minutes on average a day. Do you usually have the children read to one or I__I * Select only one more people? option readpno 1. Yes 2. No→ listen How often do you usually have the children I__I * Select only one read to one or more people? option readpfreq 1. Never 2. Once per week 3. 2-4 times per week 4. Every day

IMPAQ International, LLC Page 145 Beoog Biiga II Baseline Report July 2017

5. Not sure/I don’t know For how long do the children usually read to ……. * Record the number readpnoy one or more people when they do this activity? . of minutes es Please tell me how many minutes on average a day. Are you either planning to, or have you already I__I * Select only one had your students listen to someone else option reading today? listen 1. Yes, I already did it 2. Yes, I plan to do it 3. No, I haven’t done it and don’t plan to →listenno How long <> listen to someone . of minutes listenyes reading today? Please tell me how many minutes on average. Do you usually have the children listen to one I__I * Select only one or more people reading? option listenno 1. Yes 2. No→ write How often do you usually have the children I__I * Select only one listen to one or more people reading? option 1. Never listenfreq 2. Once per week 3. 2-4 times per week 4. Every day 5. Not sure/I don’t know For how long do the children usually listen to ……. * Record the number Listennoy other people reading when they do this . of minutes es activity? Please tell me how many minutes on average a day. Are you either planning to, or have you already I__I * Select only one had your students write today? option 1. Yes, I already did it Write 2. Yes, I plan to do it 3. No, I haven’t done it and don’t plan to. →writeno

IMPAQ International, LLC Page 146 Beoog Biiga II Baseline Report July 2017

For how long <> write today? Please . of minutes tell me how many minutes on average. Do you usually have the children write? I__I * Select only one Writeno 1. Yes option 2. No→ word How often do you usually have the children I__I * Select only one write? option 1. Never Writefreq 2. Once per week 3. 2-4 times per week 4. Every day 5. Not sure/I don’t know For how long do the children usually write they ……. * Record the number writenoye do this activity? Please tell me how many . of minutes s minutes on average a day. Are you either planning to, or have you already I__I * Select only one had your students do a word play today? option 1. Yes, I already did it Word 2. Yes, I plan to do it 3. No, I haven’t done it and don’t plan to →wordno How long <> do word play . of minutes Wordyes today? Please tell me how many minutes on average. Do you usually have the children do word play? I__I * Select only one Wordno 1. Yes option 2. No→ littrain How often do you usually have the children do I__I * Select only one word play? option 1. Never wordfreq 2. Once per week 3. 2-4 times per week 4. Every day 5. Not sure/I don’t know For how long do the children do word play ……. * Record the number wordnoye when they do this activity? Please tell me how . of minutes s many minutes on average a day.

IMPAQ International, LLC Page 147 Beoog Biiga II Baseline Report July 2017

Did you receive training on teaching literacy in I__I * Select only one the last 12 months? option Littrain 1. Yes 2. No Please tell me if you have already done, or plan to do any of the following TODAY: Each student checks his or her own work and I__I * Select only one gives himself/herself a mark/comments option eval 1. Yes, I already did it *Read the options one

2. Yes, I plan to do it after the other 3. No, I haven’t done it and don’t plan to. Students check each other’s work I__I * Select only one 1. Yes, I already did it option eval1 2. Yes, I plan to do it 3. No, I haven’t done it and don’t plan to. The whole class checks the work of a student I__I * Select only one (E.g.: From work on the blackboard or an oral option answer provided to the teacher with whole eval2 class listening) 1. Yes, I already did it 2. Yes, I plan to do it 3. No, I haven’t done it and don’t plan to. Students write solutions on a slate and show to I__I * Select only one teacher and class option eval3 1. Yes, I already did it 2. Yes, I plan to do it 3. No, I haven’t done it and don’t plan to. Students of different skill levels are paired I__I * Select only one together (so that lower levels learn from option others) eval4 1. Yes, I already did it 2. Yes, I plan to do it 3. No, I haven’t done it and don’t plan to. Students of the same skill level are paired I__I * Select only one together (so that they reinforce each other.) option eval5 1. Yes, I already did it 2. Yes, I plan to do it 3. No, I haven’t done it and don’t plan to.

IMPAQ International, LLC Page 148 Beoog Biiga II Baseline Report July 2017

Teacher asks group (3 or more) of students to I__I * Select only one work together on a project and later provides option feedback to the group on its performance. eval6 1. Yes, I already did it 2. Yes, I plan to do it 3. No, I haven’t done it and don’t plan to. Please tell me if you have already used or plan to use any of the following THIS WEEK, that is between Monday and Saturday THIS WEEK: Each student checks his or her own work and I__I * Select only one gives himself/herself a mark/comments option evalb 1. Yes, I already did it *Read the options one after the other 2. Yes, I plan to do it 3. No, I haven’t done it and don’t plan to Students check each other’s work I__I * Select only one 1. Yes, I already did it option eval2b 2. Yes, I plan to do it 3. No, I haven’t done it and don’t plan to The whole class checks the work of a student I__I * Select only one (E.g., From work on the blackboard or an oral option answer provided to the teacher with whole eval3b class listening) 1. Yes, I already did it 2. Yes, I plan to do it 3. No, I haven’t done it and don’t plan to Students write solutions on a slate and show to I__I * Select only one teacher and class option eval4b 1. Yes, I already did it 2. Yes, I plan to do it 3. No, I haven’t done it and don’t plan to. Students of different skill levels are paired I__I * Select only one together (so that lower levels learn from option others) eval5b 1. Yes, I already did it 2. Yes, I plan to do it 3. No, I haven’t done it and don’t plan to. Students of the same skill level are paired I__I * Select only one eval6b together (so that they reinforce each other.) option 1. Yes, I already did it

IMPAQ International, LLC Page 149 Beoog Biiga II Baseline Report July 2017

2. Yes, I plan to do it 3. No, I haven’t done it and don’t plan to. Teacher asks group (3 or more) of students to I__I * Select only one work together on a project and later provides option feedback to the group on its performance. eval7b 1. Yes, I already did it 2. Yes, I plan to do it 3. No, I haven’t done it and don’t plan to. Did you meet individually with students during I__I * Select only one class time yesterday? option meet 1. Yes 2. No→ meetno For how long did you meet students individually ……. * Record the number meetyes yesterday? Please tell me the total number of . of minutes minutes. Do you usually meet individually with students I__I * Select only one during class time? option meetno 1. Yes 2. No→ grp For how long do you usually meet individually ……. * Record the number meetnoye with students during class time? Please tell me . of minutes s how many minutes on average. Did you meet with small groups of students I__I * Select only one during class time yesterday? option Grp 1. Yes 2. No→ grpno For how long did you meet with each group? ……. * Record the number Grpyes Please tell me how many minutes on average. . of minutes Do you usually meet with small groups of I__I * Select only one students during class time? option Grpno 1. Yes 2. No→ libdist For how long do you usually meet with the ……. * Record the number grpnoyes groups during class time? Please tell me how . of minutes many minutes on average. How far is the closest library? Please tell me ……. * Record in Kilometer Libdist how many kilometers approximately. . *>=0 & =<100

IMPAQ International, LLC Page 150 Beoog Biiga II Baseline Report July 2017

Do you sometimes take your students to this I__I * Select only one library? option Libvisit 1. Yes→libvistf 2. No Why not? *Select all that apply I__I 1. Too far I__I 2. Students don't behave well enough I__I 3. I do not know if they have the I__I appropriate books for my students I__I 4. They do not have books that are I__I appropriate for my students I__I 5. I do not know if they have enough books I__I for my students I__I Novisit 6. They do not enough books for my students 7. I do not have funding to take my students 8. I do not have enough parents willing to come with me to the library 9. Other (specify) ______

After this question, go to “attentive” How long ago did you visit? I__I * Select only one 1. A week ago or less option 2. 2-3 weeks ago 3. A month ago Libvisitf 4. 2-5 months ago 5. 6-11 months ago 6. One year ago or more 7. I don’t remember Does the library have materials that are I__I * Select only one relevant to your students? option matrelev 1. Yes 2. No Does the library have enough materials for your I__I * Select only one matenoug students? option h 1. Yes 2. No

IMPAQ International, LLC Page 151 Beoog Biiga II Baseline Report July 2017

Overall this week, on a scale of 1 to 10, where 1 I__I * Select only one is Not attentive at all and 10 is very attentive, option how would you rate your students’ level of attentiveness?

1. 1 2. 2 attentive 3. 3 4. 4 5. 5 6. 6 7. 7 8. 8 9. 9 10. 10 I__I * Select only one Are any students in your school in a reading option readgrp group? 1. Yes 2. No Have you helped your students set up a reading I__I * Select only one group? option Setup 1. Yes 2. No Do you supervise or help a reading group? I__I * Select only one Sup 1. Yes option 2. No

Interactions with Parents Great! Now I would like to ask you about your interaction with parents In the last 12 months, did you meet I__ * Select only one option individually with the parents of the children I meetp you teach for any reason? 5. Yes 6. No→ act How many times did you meet individually I__ * Select only one option meetnum with parents of the children you teach? I p 1. Once 2. Twice

IMPAQ International, LLC Page 152 Beoog Biiga II Baseline Report July 2017

3. More than three times

What were the typical reasons for meeting I__ *Select all that apply individually with the parents of the children I *Do not read the you teach in the last 12 months? I__ options

I 1. Periodic school scheduled parent- I__ meetwhy teacher conference I p 2. Child's performance I__ 3. Disciplinary issues I 4. Just touching base I__ 5. Other (specify) ______I

In the last 12 months, did any of the parents of I__ * Select only one option the children you teach attend a class while you I were teaching? 1. Yes Act 2. No

Nutrition Knowledge Now I have some questions about nutrition. vita Now, I’d like to ask you a question: have you heard of I__I * Select only vitamin A? one option 5. Yes 6. No→iron vita1a What do you think Vitamin A does? Anything else? I__I *Select all that I__I apply 10. Good vision I__I *Do not give 11. Protects against diseases I__I examples or 12. Helps with growth I__I read the list 13. Keeps skin healthy I__I 14. Other (specify) 15. I don’t know

IMPAQ International, LLC Page 153 Beoog Biiga II Baseline Report July 2017 vita2a Can you name a food that has Vitamin A? * Select all I__I that apply 33. Liver I__I * Do not give 34. Small whole fish I__I examples or 35. Dairy products (milk, yogurt) I__I read the list 36. The egg yolk I__I 37. Carrot I__I 38. The orange-fleshed sweet potato I__I 39. Courge (orange) I__I 40. Vegetable edible dark green leaves I__I 41. Red palm oil I__I 42. Papaya I__I 43. Mango I__I 44. Néré flour I__I 45. Green bell pepper I__I 46. Melon (yellow and orange) I__I 47. Other (specify)______I__I 48. I don’t know iron We just finished talking about vitamin A. Do you know I__I * Select only that there is iron in the food that we eat? one option 5. Yes 6. No → nutteach iron2a What do you think iron does? * Select all I__I that apply 25. Good vision I__I * Do not give 26. Protects against diseases I__I examples or 27. Helps with growth I__I read the list 28. Muscle function I__I 29. Brain function I__I 30. Regulates body temperature I__I 31. Prevents fatigue I__I 32. Oxygen carrier in the body I__I 33. Maintain healthy skin I__I 34. Hemoglobin formation I__I 35. Other (specify) ______I__I 36. I don’t know iron3a Can you name a food that has iron? * Select all I__I that apply 39. Fish I__I * Do not give 40. Poultry I__I examples or 41. Red meat I__I read the list 42. Organ meats (liver) I__I

IMPAQ International, LLC Page 154 Beoog Biiga II Baseline Report July 2017

43. Beans I__I 44. Dried peas I__I 45. Lentils I__I 46. Cowpeas I__I 47. dark green leafy vegetables (baobab leaf, I__I spinach) I__I 48. Eggs I__I 49. Potatoes I__I 50. Tofu I__I 51. Green beans I__I 52. Nuts (sesame, cashew) I__I 53. Green pepper I__I 54. Watermelon I__I 55. Tomato I__I 56. Other (specify) 57. I don’t know Do you teach about nutrition in your classes usually? I__I * Select only one option nutteach 1. Yes 2. No Did you teach, or plan to teach about nutrition in your I__I * Select only classes this week, meaning between this Monday and one option Saturday? nutteach1 1. Yes, I already did it 2. Yes, I plan to do it 3. No, I haven’t done it and don’t plan to. Did you teach or plan to teach about nutrition in your I__I * Select only classes today? one option 1. Yes, I already did it nutteach2 2. Yes, I plan to do it 3. No, I haven’t done it and don’t plan to.→nuttrain How long <>about nutrition in your classes today? . number of nutteach3 Please tell me the number of minutes you minutes <>. Did you receive training on teaching about nutrition in I__I * Select only the last 12 months? one option Nuttrain 1. Yes 2. No

IMPAQ International, LLC Page 155 Beoog Biiga II Baseline Report July 2017

3. I don’t know

Hygiene Health Okay, we are almost done! I have some questions about hygiene now. Stuprop On a typical day, how many of your students, would I__I * Select only one you say, wash their hands before eating at school? option Would you say 5. None 6. Less than half 7. About half 8. More than half 9. Almost all 10. All 11. I don’t know hygteach Do you teach about hygiene in your classes usually? I__I * Select only one 1. Yes option 2. No Did you teach or plan to teach about hygiene in your I__I * Select only one classes this week, meaning between this Monday and option Saturday? 7. Yes, I already did it 8. Yes, I plan to do it hygteach1 9. No, I haven’t done it and don’t plan to. Did you teach or plan to teach about hygiene in your I__I * Select only one classes today? option 1. Yes, I already did it 2. Yes, I plan to do it 3. No, I haven’t done it and don’t plan hygteach2 to.→hygtrain How long <>about hygiene in your classes today? . number of minutes hygteach3 Please tell me the number of minutes you [0-420] <>. Did you receive training on teaching about hygiene in I__I * Select only one the last 12 months? option Hygtrain 1. Yes 2. No 3. I don’t know

IMPAQ International, LLC Page 156 Beoog Biiga II Baseline Report July 2017

PTA Questionnaire

MIDLINE EVALUATION CRS FOOD FOR EDUCATION BURKINA FASO PTA Questionnaire

Basic Information Enum Last Name: …….. First Name: ……… ID : Supervisor Who is your supervisor?

Date Date (JJ/MM/AAAA)

Prov 7. Bam I__I 8. Santamega Com Write the commune ‘s name

CEB Write the CEB’s name CODE I__I__I__I__I__I__I__I__I School Write the school name CODE I__I__I__I__I__I__I__I__I

Dear PTA: You have been selected to participate in a survey about health, nutrition, and education for the Cantine Scolaire project. Your participation in this interview is voluntary. If, at any time, you wish to discontinue participation, you may do so without penalty. If you accept, please respond to all questions as candidly as possible. If you do not know the answer to a question, you may simply say so. All responses will be kept strictly confidential.

Do you accept participation in this survey? I__I *Select only one 1. Yes  PTAT option Consent 2. No→ THANKS 3. Not answered

If No or Not answered, thank the respondent and end the survey.

Personal Information

IMPAQ International, LLC Page 157 Beoog Biiga II Baseline Report July 2017

Great! Now I want to ask you some questions about you …. Are you a member of the school’s Parent Teacher Association? I__ * Select only one PTAT 1. Yes I option 2. No

If No, thank the respondent and end the survey and look for a member of the Parents Teacher Association with the help of the principal. pname What is your last name? pfirstname What is your first name? What is your responsibility within the PTA I__ * Select only one Bureau? I option bureau 1. Secretary 2. Treasurer 3. President Enter the gender I__ * Select only one genderpta 1. Male I option 2. Female Do you know your date of birth? I__ * Select only one dobpta 1. Yes  dobpta1 I option 2. I don’t know→ yearbornpta What is your date of birth? Day/Month/Year : dobpta1 If answered →kidPTA

Do you know what year you were born? I__ * Select only one yearbornpt 1. Yes  yearbornpta1 I option a 2. I don’t know→ agepta year If answered →kidPTA What year were you born? bornpta1 *[1916 -2002] Do you know your age? I__ * Select only one agepta 3. Yes  agepta1 I option 4. I don’t know→ kidpta agepta1 How old are you? …… RECORD AGE >= 15 How many children 16 years old or younger live …… >=0 record number kidpta with you? =<25

IMPAQ International, LLC Page 158 Beoog Biiga II Baseline Report July 2017

…… >=0 record number How many of the children 16 years old or =<20 ownkidpta younger who live with you are your own cannot exceed children? “kidpta” …… >=0 record number =<25 kidpta05 How many of the children living with you are 5 cannot exceed years or younger? “kidpta” …… >=0 record number =<25 kidpta612 How many of the children living with you are cannot exceed between the ages of 6 and 12? “kidpta” How many people 17 years or older, including …… >=0 record number adultpta yourself, live in the place where you live? =<25

PTA Activities Thank you! Now I would like to ask you some questions about parent teacher association activities In the last 6 months, when did the PTA organize I__I * Select only one the last general assembly with parents and option teachers to discuss the school’s life? 1. May 2. April 3. March pta01 4. February 5. January 6. December 7. November 8. No general assembly in the past 6 months How many general assemblies did the PTA I__I * Select only one organize this school year? option 1. 4 times *If it is more than 4 pta02 2. 3 times times, select 4 3. 2 times 4. 1 time 5. Never How many meetings of the APE council took ……. record the number pta03 place this year?

IMPAQ International, LLC Page 159 Beoog Biiga II Baseline Report July 2017

Are you a member of A SILC? I__I * Select only one option pta04 1. Yes  pta04b 2. No  pta06 How many members of the APE are members ……. record the number pta04b of a SILC? Did you use a portion of your savings from your I__I * Select only one SILC group for educational expenses of your option pta05 children? 1. Yes 2. No Does the canteen function? I__I * Select only one option pta06 1. Yes  pta07 2. No→ meetnumt ……. *record the number During this current school year, how many >= 0 & <=12 months did the canteen function? pta07 If « Not applicable » or

« Don’t know », write -

99 ……. *record the number >= 0 & <=12 pta08 =< pta07 During the school year, how many months [Cannot exceed pta08 were the canteen’s expenses covered by the number of months in Ministry of Education? PTA07] If « Not applicable » or « Don’t know », write - 99 ……. *record the number pta09 =< pta07 – pta08 During the school year, how many months *If pta09> (pta07 – was the canteen covered by the parents and pta08) → ask pta09 pta09 the community? again If « Not applicable » or « Don’t know », write - 99 During the school year, how many months ……. *record the number was the canteen covered by CRS (Cathwell)? pta10_cr =< pta10_cr pta07–pta08– pta09 *If pta10_cr >

IMPAQ International, LLC Page 160 Beoog Biiga II Baseline Report July 2017

(pta07–pta08– pta09) → ask pta10_cr again During the school year, how many months ……. *record the number was the canteen covered by other partners? pta10_ot =< pta07– pta08– pta09 – pta10_cr *If pta10_cr > pta07– pta10_ot pta08– pta09 – pta10_cr → ask pta10_cr again If « Not applicable » or « Don’t know », write - 99 When the canteen is functioning, how often do I__I * Select only one parents contribute with wood for the kitchen? option Explain that the pta11 1. All days that it functions parents can contribute 2. Most of the days that it functions with wood even if it is 3. Rarely the children who bring 4. Never it. When the canteen is functioning, how often do I__I * Select only one parents contribute with utensils? option 1. All days that it functions pta12 2. Most of the days that it functions 3. Rarely 4. Never When the canteen is functioning, how often do I__I * Select only one parents contribute with offering storage? option 1. All days that it functions pta13 2. Most of the days that it functions 3. Rarely 4. Never When the canteen is functioning, does the I__I * Select only one community pay women who cook? option pta14 1. Yes 2. No 3. Don’t know How many students contribute in-kind with I__I * Select only one pta17 food for the canteen (e.g. cereals, sorghum, option corn, beans, etc.)?

IMPAQ International, LLC Page 161 Beoog Biiga II Baseline Report July 2017

1. All Students 2. More than 75% 3. Between 50%-75% 4. Less than 50% 5. Don’t know How many times have you met with the I__I * Select only one teacher (of your child) during the last 12 option MEETNUM months? T 1. Once 2. Twice or three times 3. More than three times What are the reasons that you met I__I *Select all that apply individually with the teacher in the last 12 I__I months? I__I MEETWHY 1. Student performance I__I T 2. Disciplinary issues I__I 3. Just touching base 4. Other (Specify) ______

PTA School Participation Thank you! Now, I would like to ask you a few questions about your participation in school activities. In the past 12 months, did you or another adult in your household: Have a meal with students while in school? I__I * Select only one ACT2 1. Yes option 2. No Attend a PTA general assembly? I__I * Select only one ACT3 1. Yes option 2. No Participate in a school community project I__I * Select only one such as cleaning or "champs collectif"? option ACT4 1. Yes 2. No Help the school as a cook or store keeper? I__I * Select only one ACT5 1. Yes option

IMPAQ International, LLC Page 162 Beoog Biiga II Baseline Report July 2017

2. No Help watch over a reading group? I__I * Select only one ACT6 1. Yes option 2. No Visit the classroom I__I * Select only one ACT7 1. Yes option 2. No Attend a performance put on by the children I__I * Select only one in class? option ACT8 1. Yes 2. No Something else? I__I * Select only one ACT9 1. Yes (specify) ------option 2. No In the past it was mainly boys who went to I__I * Select only one school. Nowadays both boys and girls are option going to school. In your opinion, is this a good thing or a bad thing? GIRL 1. Good  girlgood 2. Bad → girlbad 3. Don’t know→ lib Why do you think it’s a good thing for girls to I__I * Select only one go to school? option 1. Educating girls improves living standards of the whole family 2. Educating girls improves their health 3. Educating girls improves the health of GIRLGOOD the children they will have; 4. Girls should also be able to fulfill themselves; 5. Educating girls allows them to find better work 6. Other (specify) Why do you think it’s a bad thing for girls to I__I * Select only one go to school? option GIRLBAD 1. Girls are supposed to stay at home; 2. Girls don’t need school;

IMPAQ International, LLC Page 163 Beoog Biiga II Baseline Report July 2017

3. Girls don’t work outside the home, so what is the point? 4. Girls should not be out in public; 5. There are no school for girls only and they should not be going to school with boys 6. Other (specify) Do you know where the library closest to I__I * Select only one where you live is located? option LIB 1. Yes 2. No Have you ever visited a library or I__I * Select only one “bibliotheque”? option LIBVISITPTA 1. Yes 2. No  THANK2 Now, think back to the last six months - that I__I * Select only one would be since last <>. Did option you visit a library or “bibliotheque” within LIBFREQ the last six months that is since <>? 1. Yes 2. No THANK2 Thank you very much for answering my questions. This test was approved by the supervisor. I__I * Select only one APPROB 1. Yes option 2. No

IMPAQ International, LLC Page 164 Beoog Biiga II Baseline Report July 2017

MIDLINE EVALUATION CRS FOOD FOR EDUCATION BURKINA FASO Chef de Circonscription d’Education de Base (CCEB) Questionnaire This questionnaire is to be used by CCEBs and CPIs (Conseillers Pédagogiques Itinérants/Educational Consultants). For each CEB, both a CCEB and a CPI need to be interviewed, that is two persons per CEB)

School Information Date Date (JJ/MM/AAAA) 9. Bam Prov I__I 10. Santamega Com Commune name CEB CEB name CODE I__I__I__I__I__I__I__I__I School School name CODE I__I__I__I__I__I__I__I__I

Personal Information Firsname What is your first name? Lastname What is your last name? 1. Male Gender 2. Female

CCEB Activities 1. How long have you been CCEB (Chef de Circonscription de l’Education de Base) or CPI (Conseiller Pédagogique Itinérant)?

2. What did you do before?

3. What is your level of education/training?

4. What do you think about the “training style” and how does it manifest itself for the training? (don’t read the list)

The important Level Very Important Sometimes Not No Modality important Important important Opinion

a) Feeling/touching b) Watching/observing c) Thinking d) Doing

IMPAQ International, LLC Page 165 Beoog Biiga II Baseline Report July 2017

5. Which do you think are the most important techniques when teaching reading? a) Phonological awareness b) Phonetic techniques c) Lectures d) Vocabulary e) Comprehension f) Other (specify)

6. Have you led a training for teachers this school year (for reading)?

7. If yes, how many training sessions?

8. How many teachers have you trained to teach reading?

9. Which subjects/themes have you taught? (don’t read the list) a) Phonological awareness b) Phonetic techniques c) Lectures (courante means something like flowing) d) Vocabulary e) Comprehension f) Other

10. When you gave a reading training, how have you supervised/followed-up with the teachers after the training?

11. Have you observed/supervised/monitored teachers in their classes this school year?

11.1 If yes, how many times? 11.2 If yes, how much time did you stay within one class, on average?

12. According to your observations, what proportion of teachers who uses the “package” of the following five activities every day? (enumerators: cite the 5 activities)  Writing  Hear others reading  Read aloud to others  Read alone  Play with words

IMPAQ International, LLC Page 166 Beoog Biiga II Baseline Report July 2017

MIDLINE EVALUATION CRS FOOD FOR EDUCATION BURKINA FASO

Classroom Observation Tool

Section A – The school A. 1. District A. 2. Commune A. 3. School’s name A. 4. School’s ID A. 5. School’s status  Public  Community  Private Section B – The teacher and the class B. 1. Last name: B. 2. First name : B. 3. Gender:  F  M B. 4. Age : ______B. 5. Experience in teaching (years) : ______B. 6. Pre-service training  Public teachers’ college  Private teachers’ college for teaching  No training  Other (specify)______ Trainee teacher  State worker B. 7. Worker’s status  Contracting  Community volunteer B. 8. Grade(s) observed  1st  2nd  3d  4th  5th B. 9. Number of students in the classroom : ______including ______girls

On a notebook, report the progress of the lesson: what is the teacher doing, which materials he and students are using, how are students are reacting. At the end of the lesson, fill the next 3 sections:

SECTION C – Materials C.1. Choose all materials used during the lesson  Black board  Sounds’ reading table  Small slates  Large slates  Illustrated dictionnary  Bananagram or equivalent  Wood cubes  Word strips  Posters of reading text pictures  Grammar, conjugation, spelling

IMPAQ International, LLC Page 167 Beoog Biiga II Baseline Report July 2017

 Manuel de la méthode Boscher  Lire au Burkina  Livre Flamboyant CP1  Livre Flamboyant CE1  Je lis et j’écris, Classiques africains CE2   Je lis et j’écris, Classiques africains CM1  Mamadou et Bineta sont devenus grands

 Other. Specify : ______

SECTION D – Languages In what language the teacher speaks to students most of the time? What language is taught (the language written on the blackboard)?

SECTION E – Practices and activities observed During the lesson, the teacher No Yes a. Carried out activities aiming letters’ recognition or master the complete alphabet or its order b. Carried out activities on letters’ and combinations’ sounds c. Made students decode an unknown word, sentence, or text, without to make them repeat or memorize d. Made students encode a word, a sentence, or a text using their knowledge of sounds and letters, regardless spelling or other conventions of written texts e. Make students develop their lexical awareness or vocabulary (find new words on a given subject, deduce new words’ meaning using their structure or using the context, use tools such as dictionnary or lexical sheets...) f. Made students read a word, a sentence, or a text (aloud or not) g. Made students read a sentence or a text with fluency and expression (respecting punctuation, with expression, without to stop at each word) h. Asked students answering questions of simple comprehension or locating answers in a text read i. Asked students answering more subtil questions of comprehension or questions of implicit comprehension (comprehension of causes/consequences, chronology, more subtil answers, riddles,...) j. Asked students to make hypothesis on a text

IMPAQ International, LLC Page 168 Beoog Biiga II Baseline Report July 2017 k. Asked students to establish links between the text read and their real life or with another text l. Asked students to write a word, a sentence, without to copy or recite (he can provide them a text guidance such as a pattern, mad libs, stairs...) m. Asked students to write on a free topic n. Asked students to deduce or practice rules of spelling, grammar, or conjugation o. Helped students to improve their own written production for style, coherence, spelling, punctuation... Please describe all other activities observed that are not listed above

______

SECTION F – Use of the Beoog Biiga II method The teacher carried out :  A reading/decoding lesson  A reading/comprehension lesson  Writing groups focusing on spelling  Writing groups focusing on production IF HE CARRIED OUT A READING/DECODING LESSON, did he work on :  Letters recognition  Respect of punctuation  Combinations  Respect of liaisons  Morphemes and tool words  Words’ articulation  Reading aloud  Fluency  Identification of breath groups  Detecting students difficulties IF HE CARRIED OUT A READING/COMPREHENSION LESSON, did he work on :  Defining new words  Comprehension of a whole sentence (syntax)  Explicit teaching of comprehension (connectors, substitutes, punctuation...)  Implicit comprehension and inference  Making hypothesis

IMPAQ International, LLC Page 169 Beoog Biiga II Baseline Report July 2017

 Use of dictionnary  Identification of strategies to find answers in a text

Other comments

______

IMPAQ International, LLC Page 170 Beoog Biiga II Baseline Report July 2017

Qualitative Tools

MIDLINE DATA COLLECTION FOR BEOOG BIIGA II (BBII) PROGRAM IN BURKINA FASO Qualitative Instruments

Key Informant Interview: USDA staff Good morning/afternoon. My name is [insert name] and I am with IMPAQ International, a US- based research company. Catholic Relief Services (CRS) has contracted with IMPAQ to carry out an evaluation of the Beoog Biiga II (BBII) program in Burkina Faso. The purpose of this interview is to hear about your experience so far with the BBII program, including your perspective on the relevance, effectiveness, and impact of the program.

With your permission, I will audio record the discussion to assist with note-taking. No one outside the evaluation team will have access to this recording. Do I have your permission to record the conversation?

The interview will last about 60 minutes and will work best if you do most of the talking. Feel free to speak openly and candidly about your experiences and perspectives regarding this project. Your participation in this interview is voluntary. If, at any time, you wish to discontinue participation, you may do so without penalty.

The information provided during the interview will help in understanding the relevance, effectiveness, and impact of the project in order to improve future projects and activities. It will help us to know what works and doesn’t work.

The data gathered through these interviews will be reported in an aggregate manner, highlighting informational points from specific sites and not from particular individuals. You will not be identified by name.

Do you have any questions for me before we begin? Okay, let’s get started.

1. To begin, could you tell me a bit about what your role on the BBII project? If person is new to organization: What is the role of your institution on the BBII project? 2. From your perspective, what are the strengths and weaknesses of the project’s design? 3. How well does BBII align with USDA’s priorities and trends? 4. From your understanding, to what extent does the program effectively consider economic, cultural, and political contexts? Are there any factors that were not considered during design but should be now?

IMPAQ International, LLC Page 171 Beoog Biiga II Baseline Report July 2017

5. What other internal or external factors could have an impact on the program’s success? From your knowledge, have program implementers successfully responded to such factors? 6. What achievements have been made since the project started? Do these achievements align with the intended objectives of the project? Why or why not? 7. Overall, do you think the project is achieving its objectives? Why or why not? 8. What have been some of the most successful elements of the project to date from your perspective? What have been the most challenging? Why have these been challenging? 9. From your perspective, what are the most critical aspects for CRS and its partners to focus on to make sure that this project is a success? 10. To your knowledge, how well does the program coordinate and collaborate with external stakeholders (e.g., government, other NGOs)? 11. From your perspective, how well has CRS managed project partners? (Probe for specific examples) What improvements could be made? 12. From your perspective, has implementation of BBII been effectively monitored by CRS and its partners? What improvements could be made? 13. What strategies could CRS use to increase teachers’ and other stakeholders’ involvement in project activities? 14. Have project objectives been achieved according to schedule and as planned? If not, what have been the obstacles? 15. How well has the Monitoring, Evaluation, Accountability and Learning (MEAL) mechanism helped the implementation of the project? What improvements could be made? 16. What aspects of the program do you think will have the most impact on literacy among school age children? What aspects will have the least impact? What strategies could CRS consider to improve literacy outcomes for children? 17. What aspects of the program do you think will have the most impact on health and dietary knowledge and practices among school age children? What aspects will have the least impact? What strategies could CRS consider to improve health and dietary outcomes for children? 18. With respect to sustainability, to what extend have stakeholders begun preparing for the absence of support from USDA and CRS? (Probe for specific activities) 19. What are the major factors that are likely to influence the achievement or non-achievement of project sustainability? (Probe for government’s capacity, policies, priorities) 20. From your perspective, what strategies should be used to obtain long-lasting support from communities and local/central administration that extends beyond the life of the project?

IMPAQ International, LLC Page 172 Beoog Biiga II Baseline Report July 2017

21. Overall, reflecting on implementation and outcomes of the program so far, what suggestions do you have to improve its success? 22. Is there anything else that you’d like to share with me today? Key Informant Interview: MENA staff

Good morning/afternoon. My name is [insert name] and this is my colleague [insert name]. We are from [insert name of company] and are working with IMPAQ International, a US-based research company. Catholic Relief Services (CRS) has contracted with IMPAQ to carry out an evaluation of the Beoog Biiga II (BBII) program. The purpose of this interview is to hear about your experience so far with the BBII program, including your perspective on the relevance, effectiveness, and impact of the program.

Do you have any questions for me before we begin? Okay, let’s get started.

1. To begin, could you tell me about your role at MENA and about your role on the BBII project? If person is new to organization: What is the role of MENA on the BBII project? 2. From your perspective, what are the strengths and weaknesses of the project’s design? 3. How well does BBII align with MENA’s priorities and trends? 4. From your understanding, to what extent does the program effectively consider economic, cultural, and political contexts? Are there any factors that were not considered during design but should be now?

IMPAQ International, LLC Page 173 Beoog Biiga II Baseline Report July 2017

5. What other internal or external factors could have an impact on the program’s success? From your knowledge, have program implementers successfully responded to such factors? 6. What achievements have been made since the project started? Do these achievements align with the intended objectives of the project? Why or why not? 7. Overall, do you think the project is achieving its objectives? Why or why not? 8. What have been some of the most successful elements of the project to date from your perspective? What have been the most challenging? Why have these been challenging? 9. From your perspective, what are the most critical aspects for CRS and its partners to focus on to make sure that this project is a success? 10. To your knowledge, how well does the program coordinate and collaborate with external stakeholders (e.g., government, other NGOs)? 11. From your perspective, how well has CRS managed project partners? (Probe for specific examples) What improvements could be made? 12. From your perspective, has implementation of BBII been effectively monitored by CRS and its partners? 13. What strategies could CRS use to increase teachers’ and other stakeholders’ involvement in project activities? 14. Have project objectives been achieved according to schedule and as planned? If not, what have been the obstacles? 15. What aspects of the program do you think will have the most impact on literacy among school age children? What aspects will have the least impact? What strategies could CRS consider to improve literacy outcomes for children? 16. What aspects of the program do you think will have the most impact on health and dietary knowledge and practices among school age children? What aspects will have the least impact? What strategies could CRS consider to improve health and dietary outcomes for children? 17. With respect to sustainability, to what extend have stakeholders begun preparing for the absence of support from USDA and CRS? (Probe for specific activities) 18. What are the major factors that are likely to influence the achievement or non-achievement of project sustainability? (Probe for government’s capacity, policies, priorities) 19. From your perspective, what strategies should be used to obtain long-lasting support from communities and local/central administration that extends beyond the life of the project? 20. Overall, do you think MENA’s partnership with CRS on this project has been worthwhile? Why or why not?

IMPAQ International, LLC Page 174 Beoog Biiga II Baseline Report July 2017

21. Overall, reflecting on implementation and outcomes of the program so far, what suggestions do you have to improve its success? 22. Is there anything else that you’d like to share with me today?

IMPAQ International, LLC Page 175 Beoog Biiga II Baseline Report July 2017

Key Informant Interview: Implementers (CRS, OCADES, FAVL) Good morning/afternoon. My name is [insert name] and this is my colleague [insert name]. We are from [insert name of company] and are working with IMPAQ International, a US-based research company. Catholic Relief Services (CRS) has contracted with IMPAQ to carry out an evaluation of the Beoog Biiga II (BBII) program. The purpose of this interview is to hear about your experience so far with the BBII program, including your perspective on the relevance, effectiveness, and impact of the program.

Do you have any questions for me before we begin? Okay, let’s get started.

1. To begin, could you tell me a bit about what your role on the BBII project? If person is new to organization: What is the role of [insert name of organization] on the BBII project? 2. From your perspective, what are the strengths and weaknesses of the project’s design? 3. From your understanding, to what extent does the program effectively consider economic, cultural, and political contexts? Are there any factors that were not considered during design but should be now? 4. What other internal or external factors could have an impact on the program’s success? From your knowledge, have program implementers successfully responded to such factors? 5. What achievements have been made since the project started? Do these achievements align with the intended objectives of the project? Why or why not? 6. Overall, do you think the project is achieving its objectives? Why or why not?

IMPAQ International, LLC Page 176 Beoog Biiga II Baseline Report July 2017

7. What have been some of the most successful elements of the project to date from your perspective? What challenges have you faced in implementing the project and why have you faced this challenge? 8. FAVL only: Could you describe how teachers and students are using the community libraries? From your knowledge, to what degree do community members (including students and teachers) have access to community libraries? How can access be improved? From your perspective, in what ways are the community libraries and library-based activities impacting children’s literacy? (Probe for examples) How can the impact of libraries be increased? 9. To your knowledge, how well does the program coordinate and collaborate with external stakeholders? (e.g., government, other NGOs)? What suggestions do you have to increase collaboration? 10. What lessons have you learned from working with the counties? What is the main difficulty in collaborating with the counties? How can the project’s partnership with the counties be improved? 11. From your perspective, how well has CRS managed project partners? (Probe for specific examples) What improvements could be made? 12. From your perspective, has implementation of BBII been effectively monitored by CRS and its partners? What improvements could be made? 13. Have project objectives been achieved according to schedule and as planned? (Probe for organization specific activities: FAVL – libraries; OCADES – SILC) If not, what have been the obstacles? Have you faced any challenges in reaching your targets? 14. CRS only: How well has the Monitoring, Evaluation, Accountability and Learning (MEAL) mechanism helped the implementation of the project? What improvements could be made? 15. What aspects of the program do you think will have the most impact on literacy among school age children? What aspects will have the least impact? What strategies could CRS consider to literacy outcomes for children? 16. What aspects of the program do you think will have the most impact on health and dietary knowledge and practices among school age children? What aspects will have the least impact? What strategies could CRS consider to improve health and dietary outcomes for children? 17. With respect to sustainability, to what extend has [name of organization] and other stakeholders begun preparing for the absence of support from USDA and CRS? (Probe for specific activities) 18. What are the major factors that are likely to influence the achievement or non-achievement of project sustainability? (Probe for government’s capacity, policies, priorities)

IMPAQ International, LLC Page 177 Beoog Biiga II Baseline Report July 2017

19. From your perspective, what strategies should be used to obtain long-lasting support from communities and local/central administration that extends beyond the life of the project? 20. Overall, reflecting on implementation and outcomes of the program so far, what suggestions do you have to improve its success? 21. Is there anything else that you’d like to share with me today?

IMPAQ International, LLC Page 178 Beoog Biiga II Baseline Report July 2017

Key Informant Interview: County Mayors Good morning/afternoon. My name is [insert name] and this is my colleague [insert name]. We are from [insert name of company] and are working with IMPAQ International, a US-based research company. Catholic Relief Services (CRS) has contracted with IMPAQ to carry out an evaluation of the Beoog Biiga II (BBII) program in your county. The purpose of this interview is to hear about your experience so far with the BBII program, including your perspective on the relevance, effectiveness, and impact of the program.

The interview will last about 30 minutes and will work best if you do most of the talking. Feel free to speak openly and candidly about your experiences and perspectives regarding this project. Your participation in this interview is voluntary. If, at any time, you wish to discontinue participation, you may do so without penalty.

Do you have any questions for me before we begin? Okay, let’s get started.

1. To begin, could you tell me a bit about your county’s role on the BBII project? Could you describe your relationship with CRS and its partners on this project? (Probe for how well the county and the BBII team have worked together) 2. From your understanding, to what extent does the program effectively consider economic, cultural, and political contexts? Are there any factors that were not considered during design but should be now? 3. What other internal or external factors could have an impact on the program’s success? From your knowledge, have program implementers successfully responded to such factors? 4. How well does BBII align with the county’s priorities? 5. My understanding is that you visited schools in your county that are participating in the BBII program. What did you observe at those schools? What are the main challenges faced by

IMPAQ International, LLC Page 179 Beoog Biiga II Baseline Report July 2017

schools in your county? (Probe for issues related to BBII such as literacy, health, nutrition, hygiene) 6. What did you learn from your visit to the schools? 7. From your knowledge, what achievements have been made since the project started? Do these achievements align with the intended objectives of the project? Why or why not? 8. Overall, do you think the project is achieving its objectives? Why or why not? 9. What have been some of the most successful elements of the project to date from your perspective? What have been the most challenging? Why? 10. What strategies could CRS and its partners consider to improve literacy and health outcomes for children? 11. With respect to sustainability, to what extend has the county begun preparing for the absence of support from USDA and CRS? (Probe for specific activities) 12. What actions do you recommend the county and communities take to prepare for the absence of support from USDA and CRS? 13. Overall, do you think the county’s partnership with CRS on this project has been worthwhile? Why or why not? 14. Overall, reflecting on implementation and outcomes of the program so far, what suggestions do you have to improve its success? 15. Is there anything else that you’d like to share with me today?

IMPAQ International, LLC Page 180 Beoog Biiga II Baseline Report July 2017

Key Informant Interview: Teachers Good morning/afternoon. My name is [insert name] and this is my colleague [insert name]. We are from [insert name of company] and are working with IMPAQ International, a US-based research company. Catholic Relief Services (CRS) has contracted with IMPAQ to carry out an evaluation of the Beoog Biiga II (BBII) program. The purpose of this interview is to hear about your experience so far with the program.

Do you have any questions for me before we begin? Okay, let’s get started.

1. Let’s start by talking a little bit about your background. How long have you been a teacher? How long have you been at this school? 2. I’d like to get a better understanding of the literacy instruction trainings and materials you have received through the BBII program. Could you describe the trainings you’ve received and what you’ve learned? 3. Have the literacy instruction trainings been useful? (Probe for examples) How do you use the techniques you’ve learned in the classroom? (Probe for examples) Which techniques have you found to be the most useful? Which techniques have been the least useful? What challenges do you face in using the literacy teaching techniques? How could the trainings be improved? What additional training topics would help you be more effective in literacy instruction? 4. Have you found the literacy instruction materials to be useful? (Probe for examples) How do you use the materials? (Probe for examples) How could they be improved? How do these

IMPAQ International, LLC Page 181 Beoog Biiga II Baseline Report July 2017

materials compare to the materials you previously had? From your knowledge, how have these materials been received by students? Do you have regular access to these materials? Why or why not? Are there materials you would like but do not have? (Probe for examples) 5. Now I’d like to learn about classroom observations by school administrators. How often do school administrators observe your classroom? In what ways have you incorporated observation feedback into your teaching techniques, if any? (Probe for examples) How could the observation process be improved? 6. Next I’d like to learn about student reading clubs. Are you involved or have you observed student reading clubs at your school? From your perspective, what makes a reading club successful? What motivates students join reading clubs? What about students who don’t join, why don’t they participate? Have you noticed any changes since students have joined reading clubs? (Probe for personal confidence, better grades, more social cohesion among students, etc.) At your school, who provides oversight of the reading clubs (mentors, teachers, older students)? Has your school experienced challenges in finding people willing and able to create and facilitate reading clubs? Why or why not? 7. Next I’d like to ask a few questions about your experience educating students about nutrition and health practices. Have the trainings you received about nutrition and health successfully prepared you to teach students about these issues? (Probe for examples) How could the trainings be improved? Which techniques have you found to be the most useful in transferring knowledge about health, nutrition, and hygiene to students? Which techniques have been the least useful? From your observations, what are potential barriers to behavior change among students in adopting good practices? 8. Do you think the BBII program will improve students’ knowledge of and use of health and dietary practices? Why or why not? What aspects of the program do you think have the most impact on improvements in health and dietary practices? What aspects have the least impact? What strategies could CRS consider to improve health and dietary outcomes for children? 9. Is there a library in your community? How often do you use it for teaching purposes? Could you provide examples of how you use it for teaching purposes? (Probe for how teacher encourages students to use library) How could the library program be improved? Do you encourage students to go? How do you do that? 10. How often do you miss a class you’re scheduled to teach? What are common reasons you miss class? Have there been any changes to your attendance over the past year? Why or why not?

IMPAQ International, LLC Page 182 Beoog Biiga II Baseline Report July 2017

11. Now I’d like to learn about your perspective on changes at your school over the last year. Have you observed a change in enrollment at your school over the last year or so? (Probe for increase or decrease) What do you think has influenced this? 12. Have you observed improvements in students’ abilities over the past year? Please elaborate. (Probe for general study skills, social interactions, reading skills, writing skills) What aspects of the BBII program do you think have the most impact on improved literacy for students? What aspects have the least impact? What strategies could CRS consider to improve literacy outcomes for children? 13. Overall, reflecting on implementation and outcomes of the program so far, what suggestions do you have to improve its success? 14. Is there anything else that you’d like to share with me today?

IMPAQ International, LLC Page 183 Beoog Biiga II Baseline Report July 2017

Key Informant Interview: School Administrators Good morning/afternoon. My name is [insert name] and this is my colleague [insert name]. We are from [insert name of company] and are working with IMPAQ International, a US-based research company. Catholic Relief Services (CRS) has contracted with IMPAQ to carry out an evaluation of the Beoog Biiga II (BBII) program. The purpose of this interview is to hear about your experience so far with the program.

Do you have any questions for me before we begin? Okay, let’s get started.

1. Let’s start by talking a little bit about your background. How long have you been a school administrator? For school level: How long have you been at this school? For district level: How long have you been posted in this school district? 2. I’d like to begin our discussion today by getting a better understanding of the literacy instruction trainings and materials that teachers have received through the BBII program. Could you describe the trainings they’ve received? 3. From your perspective, have the literacy trainings been useful? (Probe for examples) How do teachers use the techniques in the classroom? (Probe for examples) From your knowledge, what aspects of the trainings have not been widely adopted by teachers, and why? How could the trainings be improved? What challenges do teachers face in using the literacy teaching techniques? What additional training topics would help your teachers be more effective in literacy instruction?

IMPAQ International, LLC Page 184 Beoog Biiga II Baseline Report July 2017

4. From your perspective, have the literacy instruction materials been useful for teachers? (Probe for examples) How do they use the materials? (Probe for examples) How could the materials be improved? How do these materials compare to the materials teachers previously used? From your knowledge, how have these materials been received by teachers? How about students? 5. In what ways has the quality of teaching changed since teachers received literacy instruction trainings and materials? Which tools or techniques do you think have been the most impactful on the quality of teaching? 6. Now I’d like to learn about classroom observations. Have you participated in classroom observation trainings? Have they been useful to you, why or why not? How often do you or other administrators observe teachers at your school? Have you found the observations to be a useful activity, why or why not? What observation techniques have you found to be the most useful to you and the teachers? Have teachers incorporated feedback from observations into their teaching techniques? (Probe for examples) How could the observation process be improved? 7. In what ways has the quality of teaching changed since the school has implemented classroom observations? Which tools or techniques do you think have been the most impactful on the quality of teaching? 8. Next I’d like to ask a few questions about teachers’ experience educating students about nutrition and health practices. From your perspective, have the trainings teachers received about nutrition and health successfully prepared them to teach students about these issues? (Probe for examples) How could the trainings be improved? Which techniques have been the most useful in transferring knowledge about health, nutrition, and hygiene to students? Which techniques have been the least useful? 9. My next set of questions will be about teachers’ behaviour and changes you’ve observed over the last year. How often do teachers miss a class they’re scheduled to teach? From your understanding, what are common reasons teachers miss class? Have there been any changes to teacher attendance over the past year? Why or why not? 10. Now I’d like to learn about your perspective on changes at your school over the last year. Have you observed a change in enrolment at your school over the last year or so? (Probe for increase or decrease) What do you think has influenced this? 11. Have you observed improvements in reading abilities among your students over the past year? What aspects of the BBII program do you think have the most impact on improved literacy for students? What aspects have the least impact?

IMPAQ International, LLC Page 185 Beoog Biiga II Baseline Report July 2017

12. Do you think the BBII program will improve students’ knowledge of and use of health and dietary practices? Why or why not? What aspects of the program do you think have the most impact on improvements in health and dietary practices? What aspects have the least impact? 13. Overall, have you been satisfied with the implementation of the BBII program? Why or why not? (Probe for comments on management system) 14. Overall, reflecting on implementation and outcomes of the program so far, what suggestions do you have to improve its success? 15. Is there anything else that you’d like to share with me today?

IMPAQ International, LLC Page 186 Beoog Biiga II Baseline Report July 2017

Focus Group: Students Good morning/afternoon. My name is [insert name] and this is my colleague [insert name]. I am here asking some questions from children like you to understand more about a program at your school called the Beoog Biiga II program. I would like to hear about your experience in the program.

I would like to record our conversation so I can listen to you while you talk instead of writing down what you say. I will not share these records with your parents, teachers or anyone. May I do that? There aren’t any right or wrong answers. I want you to feel safe and comfortable answering the questions and to answer as best as you can. Please remember to talk one at a time. If at any time you want to stop or have any questions, just let me know.

Do you have any questions for me before we begin? Okay, let’s get started.

1. I’d like to start by learning more about each of you. As we go around the circle, please tell me your first name and your favorite subject in school. 2. I’d like to start by asking a few questions about what you learn in school. How many of you are learning how to read? (Show of hands) What kind of activities or lessons do you do at school to learn how to read? Can you describe a reading lesson? What kind of materials do you use at school to learn how to read? What do you like about these activities or materials? What don’t you like? 3. Now, how many of you are learning how to write? (Show of hands) What kind of activities or lessons do you do at school to learn how to write? Can you describe a writing lesson? What kind of materials do you use at school to learn how to write? What do you like about these activities or materials? What don’t you like? 4. Have any of you received any materials to help you to learn how to read and write? What do you like about these materials? What don’t you like? Could you describe an activity that your teacher leads using these materials? 5. Next, I’d like to learn about other activities you do at school to learn how to read and write. Does your school have reading clubs? If so, are any of you in a reading club? For those of you who are in a reading club, why did you join? What do you like about the reading club? What don’t you like? For those of you who aren’t in a reading club, why haven’t you joined? 6. Next, I’d like to learn about the food that you receive at school. Do any of you receive meals at your school? Do you like receiving these meals? Why or why not? Is the food good? Are you full after eating the meal?

IMPAQ International, LLC Page 187 Beoog Biiga II Baseline Report July 2017

7. For girls: Do any of you receive food from your school to take home? Do you like receiving this food? Why or why not? Is the food good? Do you share the food with your family - your siblings, your parents? 8. Have any of you ever missed school? Why did you miss school? 9. Now, I’d like to learn about what you know about health and nutrition. What kinds of things have you learned from your teacher about taking care of your body and staying healthy? (Probe for knowledge of key nutrients, healthy foods, how to eat a balanced diet, sanitation, hygiene) How do you stay healthy? Have you shared what you learned with your family? 10. Do you have a hand washing station at your school? With a show of hands, how many of you use it every day? When do you use it? (probe for critical moments) How do you use it? (probe for use of soap/ashes and for appropriate handwashing techniques) Do you like to use it? Why or why not? Has your teacher explained why there is a hand washing station for you to use? What have you learned about washing your hands? Do you think it’s important? Why is it important? 11. That’s all the questions I have for you today. Is there anything else that you’d like to share with me?

IMPAQ International, LLC Page 188 Beoog Biiga II Baseline Report July 2017

APPENDIX G. PERFORMANCE EVALUATION SCHOOLS

Province School Name Data Collection Method Bam Bani Quantitative Bam Billiga Quantitative Bam Boulounga Quantitative Bam Bourzanga B Quantitative Quantitative, and Classroom Bam Foulou Observations Bam Gassongo Quantitative Quantitative, Qualitative , and Bam Kargo Classroom Observations Quantitative, Qualitative , and Bam Kougsabla Classroom Observations Quantitative, Qualitative , and Bam Manegtaba A Classroom Observations Bam Simidougou Quantitative Bam Vousnango A Quantitative Sanmatenga Gouaradji Quantitative Quantitative, Qualitative , and Sanmatenga Guibtenga Classroom Observations Sanmatenga Guiendbila Quantitative Sanmatenga Lebda Quantitative Sanmatenga Ouanobian Quantitative Sanmatenga Sanba mossi Quantitative Quantitative, Qualitative , and Sanmatenga Silmidougou Classroom Observations Sanmatenga Soaga Quantitative Sanmatenga Solomnore Quantitative Sanmatenga Soubeira Natenga Quantitative Sanmatenga Tamasgo Quantitative

IMPAQ International, LLC Page 189 Beoog Biiga II Baseline Report July 2017