Beoog Biiga Program Final Evaluation Report

Authors:

Michaela Gulemetova, Ph.D. Laurence Dessein, Ed.M. Kimberly Garasky

December 2014

Submitted by: Submitted to:

Project Manager: Laurence Dessein Project Manager: Neda Sobhani IMPAQ International, LLC Catholic Relief Services – Burkina Faso 10420 Little Patuxent Parkway, Suite 300 Zone Industrielle Gounghin, 01 BP 469, Columbia, MD 21044 Ouagadougou, Burkina Faso

IMPAQ International, LLC Page i Beoog Biiga Final Evaluation Report ACKNOWLEDGEMENTS

We would like to thank Catholic Relief Services (CRS) and the U.S. Department of Agriculture (USDA) for their financial support. We would also like to thank the Ministry of Education of Burkina Faso, its regional and provincial offices in the Sahel region, as well as CRS’s staff for supporting and facilitating the rollout of the evaluation. We would also like to acknowledge Dr. Jacob Benus, Roman Ponos, Dr. Andrea Beasley, Dr. Jean Paul Petraud, Maria Di Fuccia from IMPAQ International, and Neda Sobhani, Mr. Mamadou Salle, Mr. Souleymane Sogodogo, Ms. Marie Koalga Marie, Mr. Adama Romba, Mr. Fabris Compaoré from Catholic Relief Services for their support and advice. We would also like to thank Dr. Yiriyibin Bambio from the University of Ouagadougou and Edoxi Kindane for coordinating the field work. The findings, interpretations and conclusions expressed herein are entirely those of the authors. They do not necessarily represent the views of IMPAQ International nor do they reflect the views of CRS and USDA.

IMPAQ International, LLC Page i Beoog Biiga Final Evaluation Report TABLE OF CONTENTS

ACRONYM LIST ...... v EXECUTIVE SUMMARY ...... vi CHAPTER 1. OVERVIEW OF THE BEOOG BIIGA PROGRAM ...... 1 CHAPTER 2. EVALUATION APPROACH AND DATA ...... 3 2.1 Research Questions ...... 3 2.2 Data Sources ...... 3 CHAPTER 3. STUDY SAMPLES ...... 5 3.1 Schools ...... 5 3.2 Students ...... 6 3.3 Teachers ...... 7 3.4 Directors ...... 8 3.5 PTAs ...... 8 3.6 Households ...... 8 3.7 Stakeholders/Beneficiary Sample ...... 9 CHAPTER 4. FIELD WORK AND ANALYSIS ...... 10 4.1 Field Work ...... 10 4.2 Quantitative Analysis ...... 10 4.3 Qualitative Analysis ...... 10 CHAPTER 5. FINDINGS ...... 11 5.1 Relevance of the Program ...... 11 5.2 Program Implementation...... 14 5.3 Effects on School Outcomes ...... 18 5.4 Effects on Student Outcomes...... 23 5.5 Effects on Directors ...... 31 5.6 Effects on Teachers ...... 33 5.7 Effects on Parent-Teacher Associations ...... 37 5.8 Effects on Households ...... 43 5.9 Stakeholder Perceptions on Program Successes and Challenges ...... 48 CHAPTER 6. CONCLUSIONS AND RECOMMENDATIONS ...... 52 6.1 Key Findings...... 52 6.2 Stakeholder and Beneficiary Recommendations ...... 53 APPENDIX 1. MAP OF SAMPLE INTERVENTION VILLAGES ...... 55 APPENDIX 2: LIST OF VILLAGES AND NUMBER OF RESPONDENTS BY CATEGORY ..... 56 APPENDIX 3: ATTENDANCE RATES WITH AND WITHOUT CANTEEN ...... 60 REFERENCES ...... 62

IMPAQ International, LLC Page ii Beoog Biiga Final Evaluation Report TABLE OF EXHIBITS

Exhibit 1: Students Using The Tippy Taps In A Program School In Bam...... 1 Exhibit 2: Beoog Biiga Logic Model ...... 2 Exhibit 3: Program School In Sanmatenga...... 4 Exhibit 4: M&E Sample Of Schools ...... 5 Exhibit 5: Sample Replacements ...... 6 Exhibit 6: Student Grades ...... 7 Exhibit 7: Sample Composition ...... 7 Exhibit 8: Qualitative Sample ...... 9 Exhibit 9: Tippy Taps At A Program School In Sanmatenga...... 15 Exhibit 10: Students’ Containers For Lunch In A Program School In Bam...... 17 Exhibit 11: Total Number Of Enrolled Students ...... 18 Exhibit 12: Average School Size ...... 19 Exhibit 13: Monthly Enrollment By School And By Gender ...... 20 Exhibit 14: Percentage Of Students Attending Less Than Half Of The Time By Year ...... 21 Exhibit 15: Percentage Of Students Attending Less Than Half Of The Time By Month ...... 22 Exhibit 16: Students In Class At A Program School In Bam...... 23 Exhibit 17: Students’ Knowledge Of Nutritious Meals ...... 24 Exhibit 18: Students’ Knowledge Of Vitamin A ...... 24 Exhibit 19: Students’ Knowledge Of Iron ...... 25 Exhibit 20: Students’ Hygiene Practices...... 26 Exhibit 21: Students’ Hand-Washing Self-Reported Practices ...... 26 Exhibit 22: Students’ Reasons For Using Latrines ...... 27 Exhibit 23: Observed Practices Of Hand Washing Among Students ...... 27 Exhibit 24: Observed Presence Of Water Infrastructure ...... 28 Exhibit 25: Observed Presence Of Latrines And Cleanliness Of Latrines In Schools ...... 29 Exhibit 26: Latrines At A Program School In Sanmatenga...... 29 Exhibit 27: Schools With Active Clubs ...... 30 Exhibit 28: Knowledge Of And Access To Mentors ...... 31 Exhibit 29: Directors’ Participation In Professional Development ...... 32 Exhibit 30: Directors’ Knowledge Of Education Quality...... 32 Exhibit 31: Schools With Active Clubs ...... 33 Exhibit 32: Teachers’ Participation In Professional Development ...... 34 Exhibit 33: Teachers’ Instructional Practices ...... 34 Exhibit 34: Teachers’ Knowledge Of Education Quality ...... 35 Exhibit 35: Teachers’ Knowledge Of Nutrition...... 35 Exhibit 36: Teachers’ Knowledge Of Hygiene ...... 36

IMPAQ International, LLC Page iii Beoog Biiga Final Evaluation Report Exhibit 37: Teachers’ Hand-Washing Practices ...... 36 Exhibit 38: PTA-Organized General Assemblies In Previous Year ...... 37 Exhibit 39: Number Of Months Over The Year That Schools Had Functioning Canteens ...... 38 Exhibit 40: Number Of Months Various Stakeholders Provided Food For Canteens ...... 39 Exhibit 41: Food Stored At A Program School In Sanmatenga...... 39 Exhibit 42: Schools Where Parents Provided A Defined Level Of Inputs To Canteens ...... 40 Exhibit 43: Schools Where Parents Provided A Defined Level Of Inputs ...... 40 Exhibit 44: Knowledge And Management Practices Of Ptas ...... 41 Exhibit 45: PTA’ Money Management Books With Essential Elements ...... 41 Exhibit 46: PTA’ Food Management Books With Essential Elements ...... 42 Exhibit 47: PTA Implemented Projects Over The Last Two Years ...... 42 Exhibit 48: Existence Of Pta Action Plans ...... 43 Exhibit 49: Household Knowledge Of Benefits Of Girls’ Education...... 44 Exhibit 50: Household Reasons For Hand Washing And Using Latrines ...... 45 Exhibit 51: Household Hand-Washing Practices ...... 45 Exhibit 52: Observed Presence Of Latrines And Cleanliness Of Latrines Among Households .. 46 Exhibit 53: Observed Practices Of Hand Washing Among Households ...... 47 Exhibit 54: Household Knowledge And Participation In Reflect Activities ...... 48 Exhibit 55: Students Study During Lunch At A School In Bam ...... 49

IMPAQ International, LLC Page iv Beoog Biiga Final Evaluation Report ACRONYM LIST

BMI Body Mass Index CRS Catholic Relief Services DPENA Direction Provinciale de l’Education Nationale et de l’Aphabétisation FFE Food For Education IDP Internally Displaced People M&E Monitoring and Evaluation MDMS Midday Meal Scheme MOE Ministry of Education MOH Ministry of Health PTA Parent-Teacher Association SFP School Feeding Program SILC Saving and Internal Lending Communities THR Take-Home Ration USDA US Department of Agriculture WFP World Food Program

IMPAQ International, LLC Page v Beoog Biiga Final Evaluation Report EXECUTIVE SUMMARY

This report presents findings on the Beoog Biiga (Tomorrow’s Child) program implemented by Catholic Relief Services in Burkina Faso during the period 2011–2014. The program was funded under the International Child Nutrition and Food for Education program managed by the Foreign Agriculture Service (FAS) of the United States Department of Agriculture (USDA) and implemented by Catholic Relief Services (CRS). The program provided school meals, take-home rations for girls, health training, nutritional supplements, and capacity building in schools and communities. It was rolled out in over 700 primary schools and preschools in the provinces of Bam and Sanmatenga. Fifty pilot schools received additional services, such as community-led development programs, trainings for teachers and parent-teacher associations, a mentoring program for girls, and community savings and lending activities.

In 2011, CRS collected baseline survey data from a sample of students, teachers, directors, members of parent-teacher associations and households in Bam and Sanmatenga. In 2014, CRS contracted IMPAQ International, LLC (IMPAQ) to conduct the final evaluation for Beoog Biiga. IMPAQ developed and implemented a mixed method impact evaluation to measure the relevance, effectiveness, efficiency, impact, and sustainability of the program. For the evaluation, IMPAQ collected data from a sample of respondents similar to those of the baseline survey, which included 110 schools, 426 students, 234 teachers, 112 directors, 108 PTAs, and 434 households. In addition, IMPAQ conducted 13 key informant interviews at the MOE, school, and CRS level in order to identify best practices to continue and/or scale up, and lessons learned for future interventions. We summarized below the key findings.

Evaluation Relevance While School Feeding Programs (SFP) are generally considered to be effective at improving school participation and health outcomes, the evidence base calls for further research on how school SFP increase school enrollment, improve academic performance and improve student’s health outcomes. In response, our evaluation of the Beoog Biiga program investigates the impact of the program on enrollment and attendance, as well as students’ knowledge of health and hygiene behaviors. This expands our understanding of school feeding in poor economic environments, such as in Burkina Faso and help improve the design and implementation of future programs in similar contexts.

Program Implementation The program was implemented by a staff of dedicated personnel who were able to implement the three-year Beoog Biiga program almost according to plan during the program timeline of 2011– 2014. CRS encountered some challenges while implementing the program, but was able to troubleshoot most issues. Notably, CRS did not start cateens and take-home rations until the second year of the program because of delays and losses in USDA-provided commodities.

IMPAQ International, LLC Page vi Beoog Biiga Final Evaluation Report Moreover many of the intervention schools did not have a nearby water source, prompting CRS to purchase water barrels and carts so that water could be brought from a distance and stored on site.

Effects on School Outcomes

. Total school enrollment was steady in the first and second years, then increased drastically in the third year. Enrollment increased for both boys and girls. . Girls’ enrollment was slightly lower than that of boys, but their proportion slightly improved from 46% to 48% during the three years of the program. . Enrollment was higher in pilot schools on average than in program schools both before and after the program. . Attendance was also higher at pilot schools than in program schools.

Effects on Student Outcomes

. Students’ knowledge of nutrition, vitamin A and iron improved. Specifically, 97% of pilot students and 87% of program students cited two or more foods containing vitamin A compared to 56% before the program. . There was a considerable reduction in the number of students who did not wash their hands after using the latrines or before eating. The change for students using water and soap to wash their hands after using latrines or before eating is less significant than using water only, with an 8 percentage point increase among pilot school students and a 30 percentage point increase among program school students from baseline. . The number of students reporting washing their hands with soap and water also increased – to 86% among program (up from 42%) and to 97% among pilot students (up from 46%). . Students knew of an average of 2 clubs at program schools and 3.4 at pilot schools by the end of the program, compared to practically zero clubs at baseline. . Knowledge and access to mentors spread from girls to boys and from pilot schools to program schools – 80% of girls and 52% of boys from pilot schools received support from a mentor. In program schools, 48% of girls and 30% of boys received support from a mentor.

Effects on School Directors

. Directors received 3.4 training modules on average at program schools and 3.7 training modules at pilot schools. . Directors’ knowledge of education quality improved – 57% of program (up from 26%) and 62% of pilot school directors (up from 22%) cited at least five aspects of education quality. . Directors were engaged in setting up and maintaining student clubs. The average number of active student clubs increased and the percentage of schools without any clubs sharply declined. The most common were hygiene clubs, sanitation clubs and health clubs.

IMPAQ International, LLC Page vii Beoog Biiga Final Evaluation Report Effects on School Teachers

. Teachers received an average of 1.9 training modules at program schools and 3.2 training modules at pilot schools. . Nearly all of them reported that they applied the approaches learned at the training and used participatory techniques in the classroom. . Teachers’ knowledge of education quality also improved – 40% of program and 43% of pilot schools teachers cited at least five aspects of education quality, compared to 13% of them before the program. . Teachers’ knowledge of nutrition improved substantially. After the program, 78% of pilot school teachers (up from 24%) and 69% of the program school teachers (up from 28%) cited three or more foods containing vitamin A.

Effects on Parent-Teacher Associations

. The average number of general assemblies remained the same, about 4 in the previous year. . The schools had functioning canteens on average for 8 months, a 2-month increase since baseline. Nearly all pilot schools and 85% of program schools had functioning canteens for 7–10 months. . The community continued to provide food staples for an average of 2 months while the education authorities contributed on average 3 months, a reduction of one month relative to baseline. CRS provided an average of 3 months of food by the end of the program compared to zero at baseline. . Parent and community contributions to the school canteens increased, particularly in the provision of wood, stocking and storage of food, and paying the canteen cooks. . PTA members’ knowledge of their responsibilities and good management practices improved relative to before the program started. About three-quarters were able to show money management and food management books containing essential elements. . 79% of pilot and 89% of program PTAs by the end of the program had an action plan compared to less than 10% at baseline. . PTAs implemented an average of 2.5 projects in program schools and 3.9 projects in pilot schools over the last two years.

Effects on Households

. Households were aware of more benefits in the education of girls after the program – listing an average of two benefits in comparison to only one at baseline. . Household also cited more reasons for hand washing and for using latrines. A quarter of households said they did not use latrines at the end of the program, a reduction from about a half at baseline. . Observations by enumerators revealed a greater presence and higher standards of latrines. However, enumerators did not observe improvements in hand-washing practices among

IMPAQ International, LLC Page viii Beoog Biiga Final Evaluation Report households. At the end of the program, only 7% of household used soap and water to wash their hands. . Knowledge about REFLECT spread – 80% of households in pilot communities and 46% of households in program communities had heard about it. . 35% of pilot community households and 22% of program community households received training. 45% of pilot and 16% of program community households participated in the implementation of projects. Only 18% of pilot and 9% of program community households started and completed their projects.

Stakeholder and Beneficiary Perceptions on Program Successes, Key lessons and Challenges

Successes and Key Lessons: . Stakeholders and beneficiaries observed noticeable improvements in school attendance and attention levels in the classroom, which they attributed to school meals, micronutrients, and improved hygiene practices. . Stakeholders and beneficiaries praised the increased community engagement in school management and upkeep, which they attributed to specific program activities (SILC, REFLECT, trainings for PTA on developing action plans, and awareness-raising among communities). . Stakeholders commended teacher trainings for improving teachers’ pedagogical practices by training them on new and critical themes, such as gender equity.

Challenges: . Program beneficiaries stated the lack of efficiency of tippy taps and of water availability as major barriers to implementing the positive hygiene practices promoted by the program. . Project stakeholders mentionned the long turnaround time of the school commodity distribution reports and the discreptancies in the numbers of student meal recipients as important challenges to delievering school meals efficiently and adequately. . Beneficiaries mentioned that the program activities relied heavily on the voluntary support of teachers, creating teacher fatigue. . Stakeholders mentioned the limited ability of provincial Ministry of Education staff to provide adequate monitoring of all school activities because of a lack of adequate resources. . Stakeholders mentioned the increase in student-to-teacher ratio in schools as a result of the increased school attendance.

Stakeholder and Beneficiary Recommendations

. Increase the use of technology (such as tablets) by school staff to to report and count students more frequently.

IMPAQ International, LLC Page ix Beoog Biiga Final Evaluation Report . Ensure proper functioning of water and sanitation infrastructures, such as school wells and latrines. . Provide more teacher training to mitigate the effect of the high teacher turnover. . Engage communities early in the programing and implement SILC before other program activities. . Develop a process to redirect the surplus of food across schools. . Continue the strong and flexible collaboration with the MOE. . Focus program efforts on beneficiaries living in local communities, such as women mentors, parent-teacher associations and households. . Establish and maintain systematic school and student-level data to keep track of students, teachers and directors, and to facilitate the construction of indicators.

IMPAQ International, LLC Page x Beoog Biiga Final Evaluation Report CHAPTER 1. OVERVIEW OF THE BEOOG BIIGA PROGRAM

Catholic Relief Services (CRS), in partnership with the government of Burkina Faso and local organizations, implemented the Beoog Biiga I (Tomorrow’s Child) program in response to recurrent food crises, high levels of malnutrition, and low and inequitable levels of education in the Central North region. The program was funded by the International Child Nutrition and Food for Education program managed by the Foreign Agriculture Service (FAS) of the United States Department of Agriculture (USDA).

The three-year, multi-sectorial Exhibit 1: Students using the tippy tap in a program program sought to improve food school in Bam. security through education, health and capacity building. It was implemented in over 700 primary schools and preschools and 50 pilot schools in the provinces of Bam and Sanmatenga. Program activities included daily lunches, take-home rations for girls, deworming and micronutrient supplements, health, hygiene and nutrition trainings for teachers and cooks, and trainings for PTAs. (Exhibit 1 shows students using a Photo: IMPAQ International tippy tap or hand washing station in a program school in Bam).

Pilot schools received supplementary programing, including mentoring for girls, savings and internal lending communities, community-led development programs, and additional trainings for teachers and school administrators. Exhibit 2 displays the logic model of the program and describes how program activities were designed to impact students, teachers, schools, and parents.

CRS conducted a baseline assessment and contracted IMPAQ to conduct the final evaluation of the Beoog Biiga program. The IMPAQ team designed a mixed-methods approach to investigate how well the program goals were met. Using data from surveys, observations, monitoring and evaluation (M&E) records, interviews, and program documents, IMPAQ researchers compared changes in outcomes in 112 participating villages between baseline and endline. The results of the quantitative and qualitative analyses are presented in this report.

IMPAQ International, LLC Page 1 Beoog Biiga Final Evaluation Report Exhibit 2: Beoog Biiga Logic Model

IMPAQ Interanational, LLC Page 2 Beoog Biiga Final Evaluation Report CHAPTER 2. EVALUATION APPROACH AND DATA

2.1 Research Questions

Using a mixed-methods approach, the IMPAQ team investigated how well the program goals were met. In addition, the team generated evidence to address the following evaluation questions:

. How relevant is the program evaluation given the economic, cultural (including gender) and political context in Burkina Faso? . To what extent is the program implemented as designed (e.g., lunches and trainings delivered)? . To what extent is the program effective in increasing student enrollment and attendance? . To what extent is the program successful in improving students’ knowledge and behavior about nutrition, health and sanitation? . To what extent is the program successful in improving school teachers’, school directors’, and local households’ knowledge and behavior about girls’ education, nutrition, health, and sanitation? . To what extent is the program effective in increasing knowledge among PTA members about food preparation and nutrition, student health and sanitation, and girls’ education? . Are there considerable differences across gender of program beneficiaries? . Are there considerable differences across pilot schools and regular schools? . What key lessons, challenges and recommendations from the program implementation can be drawn for future sustainability?

2.2 Data Sources

To answer the research questions and provide evidence, we used data from multiple sources, including monitoring and evaluation (M&E) administrative records, surveys, interviews, and program documents.

M&E Data

CRS collected M&E administrative data on a monthly basis at every school throughout the life of the program. IMPAQ used the data to compute indicators required by USDA. Examples of the indicators included school enrollment, attendance, and dropout and promotion rates for boys and girls.

Survey Data

CRS collected the baseline data in 2011 using five questionnaires: (1) student surveys and observations, (2) teacher surveys, (3) school director surveys, (4) PTA surveys and observations,

IMPAQ International, LLC Page 3 Beoog Biiga Evaluation Report and (5) household surveys and observations. For the final evaluation, the IMPAQ team revised the survey instruments to include the key outcomes from the baseline survey. The follow-up surveys also included new questions to capture additional performance indicators.

Stakeholder and Beneficiary Interviews

In addition to the surveys, we developed key informant interview protocols and conducted interviews to collect information on informants’ perceptions of the program successes, barriers and lessons. In total, we interviewed 13 CRS-recommended stakeholders and beneficiaries within the school system, Ministry of Education (MOE) staff, and CRS staff. We recorded the interviews and collected notes in the field to identify recurrent patterns in the responses.

Document Review

We also reviewed relevant program documentation provided by CRS, such as the biannual Program Status Reports and the Commodity Logistics and Monetization Reports prepared for USDA. These reports contained detailed information regarding the implementation of the program, including various challenges and how they were addressed.

Exhibit 3 shows a program school in Sanmatenga.

Exhibit 3: Program school in Sanmatenga.

Photo: IMPAQ International

IMPAQ International, LLC Page 4 Beoog Biiga Evaluation Report CHAPTER 3. STUDY SAMPLES

In this section we describe the sample of Beoog Biiga beneficiaries and other samples used in the analysis. We first present the quantitative samples and then present the key informant sample from the qualitative interviews.

3.1 Schools

CRS implemented the Beoog Biiga Program in over 700 primary schools, including 50 pilot schools in the provinces of Bam and Sanmatenga. Two different types of quantitative data were collected from schools: M&E data and survey data.

M&E Data

M&E data was collected by CRS from 708 primary schools during three school years, 2011–2014. In order to track changes over time, data from every year of the program was required. Since some of the schools were only monitored for one or two years, we excluded those schools from the sample. The final analysis sample included a total of 489 schools.

Overall, 182 schools were located in Bam and 307 in Sanmatenga. Exhibit 4 displays the geographic distribution of the sample. Specifically, 45 of the 50 pilot schools are included in the analysis sample – 18 from Bam and 27 from Sanmatenga. The remaining 444 schools were program schools – 164 from Bam and 280 from Sanmatenga.

Exhibit 4: M&E sample of schools

Program Pilot Total Bam 164 18 182 Sanmatenga 280 27 307 Total 444 45 489

Survey

To compare changes in individual outcomes over time, we used the same sample of 110 schools in the follow-up survey in 2014 that was used in the baseline survey in 2011. The survey sample included 50 pilot schools and 60 program schools. Appendix 1 presents a map identifying a subset of the schools.1

1 These were all the schools with GIS coordinates.

IMPAQ International, LLC Page 5 Beoog Biiga Evaluation Report In each surveyed school, we collected data from directors, teachers, PTA members, students and households (See Appendix 2 for a detailed list of villages and the number of respondents by category).

The IMPAQ team made a concerted effort to interview the same respondents that were interviewed at baseline in the follow-up survey. In some cases, identifying the same respondents was not feasible. As a result, it was necessary to replace some of the original respondents with new respondents. Exhibit 5 shows the replacement rates for each type of respondent. Below we discuss the replacement rates and the basic demographic characteristics for each group of respondents.2

Exhibit 5: Sample replacements

Type of 2014 Replacement Respondents 2011 Sample Replacements Rate Schools 110 0 0% Students 426 426 100% Teachers 234 168 72% Directors 112 84 75% PTAs 108 31 29% Households 434 101 23% Total 1424 810 57%

3.2 Students

At baseline, students were randomly selected from the sampled 110 schools to represent 10% of the target population in the two provinces. In 2011, 426 students from grades CE1 and CM1 were surveyed. Exhibit 6 shows how these grades compare to US grades and the typical age of the students in each grade.

2 For all adult respondents, we attempted to locate and survey the same individuals who were surveyed during the baseline. If they were difficult to locate or refused to participate, we randomly surveyed new adults in the desired roles. For example, if a school director had changed since baseline, we surveyed the director who was currently serving at the school site. We proceeded similarly with the teachers and the PTA member respondents. In the case of households, when a baseline respondent was difficult to locate, we randomly selected a new household respondent.

IMPAQ International, LLC Page 6 Beoog Biiga Evaluation Report Exhibit 6: Student grades

Burkina Faso Grade US Grade Age CP1 1 6 CP2 2 7 CE1 3 8 CE2 4 9 CM1 5 10 CM2 6 11

In 2014, at the endline, we determined that most of the students who were surveyed in 2011 would have graduated if they were on track in school or might have dropped out of school.3 Since it would have been prohibitively expensive to track down the students after leaving school, we sampled a new cohort of 426 students in the same target grades, CE1 and CM1.

The two samples are very similar in gender and age composition (see Exhibit 7). At the baseline, the sample of students was 49% female and the average age was 11.5. Additionally, 36% of the sample was less than 10 years old, 49% was between the ages of 11 and 13, and 15% was age 14 or older. At the endline, the sample was 46% female and the average age was 11.1. Moreover, 46% of the sample was less than 10 years old, 42% was between the ages of 11 and 13, and 13% was age 14 or older.

Exhibit 7: Sample composition

Baseline Endline Female 49% 46% Average age 11.5 11.1 Under 10 years old 36% 46% Between 11 and 13 years old 49% 42% Over 14 years old 15% 13%

3.3 Teachers

At baseline, a total of 234 teachers were sampled representing a random sample of 10% of the target population of teachers. At the endline, we attempted to survey the same teachers. However, many teachers had been reassigned to different schools during the program years and a few had

3 Students could have gone to another school, but this is less likely as children often walk from nearby villages and the schools are not very close to each other.

IMPAQ International, LLC Page 7 Beoog Biiga Evaluation Report been promoted to directors. Of the 234 teachers surveyed originally, we found 66 and had to replace the remaining 168, or 72% of the sample.

The sample of teachers was 39% female and the average age was 31.5 at baseline. In 2014, the sample was 43% female and the average age was 33.

3.4 Directors

In 2011, a random sample of 10% of the target population of directors was surveyed, for a total of 112 directors. At the endline, we attempted to survey the same directors. However, as with the teachers, many directors had been reassigned to different schools and had moved to different villages. Of the 112 directors originally surveyed, 28 were located. This meant that 84 of them, or 75% of the sample, had to be replaced.

At the baseline, the sample of directors was 98% male and the average age was 36.1 years. At the endline, the sample was 93% male and the average age was 36.5 years.

3.5 PTAs

At the baseline, a random sample of 10% of the target population of PTAs was surveyed, for a total of 108 PTAs. In 2014, we attempted to survey the same PTAs. Of the original 108, we found 77 and we had to replace the remaining 31, or 29% of the sample. PTA members may have moved away, died, or their child may have graduated from school and hence the parents were no longer actively involved in the parent-teacher association as a result.

Similar to directors, PTA members consisted mainly of men. The sample of PTAs was 98% male and the average age was 47.6 years at baseline. In 2014, the sample was also 98% male and the average age was 50.8 years.

3.6 Households

At baseline, 434 households were randomly selected from project communities. At the endline, we attempted to survey the same households. Of the 434, 333 were found and we had to replace the remaining 101, or 23% of the sample.

In 2011, the sample of households consisted of heads of households who were 20% female and their average age was 47.1 years. In 2014, the sample was also 20% female and the average age was 48.9.

IMPAQ International, LLC Page 8 Beoog Biiga Evaluation Report 3.7 Stakeholders/Beneficiary Sample

We interviewed a total of 13 key informants from the Sanmatenga and the Bam provinces. Key informants were selected at the MOE level, the school level and the CRS administration and program level. Specifically, the sample of key informants included: two provincial Ministry of Education directors from the Direction Provinciale de l’Education Nationale et de l’Alphabétisation (DPENA), three school directors, one teacher, and six CRS program staff (see Exhibit 8). The majority of the key informants were male adults with the exception of two female adults. School beneficiaries were selected from both pilot and program schools.

Exhibit 8: Qualitative sample

Title Region Gender Basis for Selection (M/F) MOE Representatives Primary Provincial MOE DPENA Representative Bam M representative Primary Provincial MOE DPENA Representative Sanmatenga M representative School Representatives School Director Bam M Pilot school School Teacher Bam M Program school School Director Sanmatenga M Pilot school School Director Sanmatenga M Program school CRS Representatives CRS SILC Project Manager All M Head of key program activity CRS CMO Director All M Head of key program activity CRS Commodities Project Manager All M Head of key program activity CRS Logistics Manager All M Head of key program activity CRS School Health and Nutrition Head of key program activity All M Project Manager CRS Capacity Building Project Head of key program activity All F Manager CRS Beoog Biiga Program Manager All F Manager of entire program

IMPAQ International, LLC Page 9 Beoog Biiga Evaluation Report CHAPTER 4. FIELD WORK AND ANALYSIS

4.1 Field Work

We recruited 24 experienced data collectors from CRS’ database and provided them with two days of theoretical in-door training, one day of hands-on practice at a nearby school and one day of post-field practice debrief. The enumerators used iPads and iPods to implement the in-person surveys and submitted them electronically during the field work.

The enumerators formed teams of two individuals (male and female, when possible) to act as checks on each other. Due to the greater number of program and pilot school in Sanmatenga, we assigned six teams in Sanmatenga and four teams in Bam. Each day, two supervisory teams supported the enumerator teams by providing them with technical assistance, helping them troubleshoot issues and overseeing the quality of the collected data. The supervisory team consisted of two CRS animators, one local IMPAQ expert and one evaluation expert from headquarters. The field work was completed in a total of 8 days.

IMPAQ directly conducted all 13 key informant interviews with the CRS-recommended stakeholders and beneficiaries. IMPAQ took detailed notes of and recorded all interviews.

4.2 Quantitative Analysis

The quantitative analysis consists mainly of an analysis of the survey data and the secondary M&E data. Specifically, we constructed and computed descriptive indicators (percentages and averages). For some indicators, we combined several survey items into subscales. We linked the baseline and endline responses whenever possible to construct a longitudinal dataset.4 We computed and reported changes over time on the aggregate level. In addition, we conducted subgroup analysis by school type, gender and province, highlighting emerging patterns.5

4.3 Qualitative Analysis

For the qualitative analysis, we developed common themes from the interview notes using constant comparative methods and developed a template that mapped the themes to the relevant research questions. We summarized the interview data to provide a snapshot of program management, future sustainability and identified common patterns across all sites.

4 There are 2 data points for the survey data (2011 and 2014) and 3 data points for the M&E data (2012, 2013, 2014). 5 All survey data has been analyzed by school type and by province. Most adult respondents were men. About half of the student respondents were girls, so the data was also analyzed by gender. However, since there were practically no differences across provinces and gender, we decided to present all the findings in this report by school type. For the M&E datasets and for knowledge and access to tutors among students, we also present findings by gender.

IMPAQ International, LLC Page 10 Beoog Biiga Evaluation Report CHAPTER 5. FINDINGS

5.1 Relevance of the Program

In the last decade, Burkina Faso has experienced persistent food crises, high levels of malnutrition, and low and inequitable levels of education access and quality. The literacy rate is among the lowest in How relevant is the program the world at 29% (UNDP, 2009), and it is even evaluation given the lower in rural areas and among women and girls. economic, cultural (including The majority of households in Burkina Faso rely on gender), and political context? subsistence agriculture. Malnutrition is extremely high across the country, with 35% of children under five experiencing stunting as a result of a poor diet, poor hygiene and illness (UNICEF, 2008).

School Feeding Programs (SFPs) such as Beoog Biiga address health, nutrition and education issues through schools meals6 and/or Take-Home Rations (THRs)7, often with complementary activities focused on health training and nutritional supplements, as well as capacity building in schools and communities (Adelman et al., 2008). In contexts such as Burkina Faso in which primary school enrollment and income levels are extremely low, and food shortages occur frequently, SFPs strive to impact student enrollment because the value of food offered is expected to be a sufficient incentive to attract children to school. Moreover, by providing nutritional meals to schoolchildren who would otherwise go hungry, SFPs can directly impact the health and nutritional status of school aged children, particularly those experiencing malnutrition and stunting. School feeding can also improve students’ cognitive functions and academic performance by reducing absenteeism and increasing attention due to the improved nutritional status.

Few rigorous studies have evaluated the impact of FFE programs on education and health outcomes. In general, FFE programs are considered to be effective at improving school participation and health outcomes, though estimates of the size of the impacts differ considerably by context (Ahmed, 2004; Adelman et al., 2007; Adelman et al., 2008; Alderman et al., 2008; Alderman et al., 2012; Jacoby, 2002; Finan et al., 2010; Kazianga et al., 2009; Kazianga et al., 2012; Kazianga et al., 2014). Moreover, some studies have found positive spillover effects of THR to younger children because the food received by students ends up being shared by other children in the household. The following review of the literature of SFPs is not meant to be exhaustive, but

6 School meals are generally defined as meals (breakfast and/or lunch) served daily at the school (Kazianga et al., 2009). 7 Take-Home Rations are defined as certain amounts of food staples given to students periodically (Kazianga et al., 2009).

IMPAQ International, LLC Page 11 Beoog Biiga Evaluation Report rather presents some of the key features of previous research that sought to measure the impact of school feeding programs.

In Burkina Faso, Kazianga et al. (2009) used a randomized experiment to estimate the impact of two school feeding programs8 in the Sahel region of northern Burkina Faso on health, nutrition, education and labor outcomes. The authors found that both programs increased enrollment for girls by 5 to 6 percentage points, but they found no considerable effects on girls’ raw mathematics scores nor on other measures of cognitive development. Overall, absenteeism increased for children exposed to the program, which explained the mixed impact on learning. In particular, the authors found increased absenteeism for households with low child labor supply, but decreased absenteeism for households with relatively large child labor supply, consistent with the labor constraints. Finally, the authors found that siblings aged between 12–60 months of beneficiaries of THRs had increased weight-for-age by 0.38 standard deviations and weight-for-height by 0.33 standard deviations. However, the authors did not find any considerable effect on the nutrition of younger children.

Using the same datasets, Kazianga et al. evaluated the impact of the two school feeding schemes on health outcomes of pre-school age children (2014). The authors examined spillover effects on younger siblings of the beneficiaries and found that THRs increased weight-for-age of boys and girls under 5 by 0.4 standard deviations. However, school meals did not have any considerable effect on the weights of siblings. The authors provide suggestive evidence that the majority of the gains were realized through intra-household food allocation.

In another study using the same datasets, Kazianga et al. found suggestive evidence that the programs, especially THRs, did not eliminate child labor (2012). However, the authors found that the programs did alter the allocation of child labor, especially for girls, in that children participated less in the type of productive activities that were difficult to combine with school activities.

Previous evaluations of SFPs outside of Burkina Faso found relatively similar results, with impact size estimates varying considerably by context. Adelman et al. (2007) conducted a critical review of the causal evidence on the impact of FFE programs on learning, cognitive development, education participation and attainments, and nutrition using several studies. The authors found that, in general, FFE programs have larger impacts in areas with low school participation and on children with greater initial malnutrition. Several of the reviewed studies showed that in-school meals improved primary school attendance of enrolled students in schools where initial attendance was low. The review also found mixed evidence that school meal programs improved performance on math and literacy tests, as well as on cognitive development. Several reviewed randomized

8 The first school feeding program consisted of school meals in which students were provided with lunch for every school day. The second program consisted of take-home rations where girls were provided 10kg of cereal flour each month, conditional on 90 percent of attendance rate.

IMPAQ International, LLC Page 12 Beoog Biiga Evaluation Report experiments showed that school meals had a positive impact on students’ nutrition status including anthropometry and iron status, depending on the feeding program. The studies did not reveal potential impacts on school participation by children not previously enrolled in school. Few studies evaluated scaled-up THR programs. However, one of the studies from Bangladesh showed a considerable effect of a THR program on school participation.

More recent studies of FFE programs in Uganda and India have shown similar impacts of school feeding schemes, particularly on attendance and students’ nutrition. In northern Uganda, Alderman et al. (2012) investigated the impact of two WFP school feeding schemes, take-home rations and in-school meals on primary school participation by using a prospective, cluster- randomized field experiment in camps for internally displaced people (IDPs). The authors found that there was no impact of either program on enrollment rates, likely because the baseline enrollment was already high at 84.9 percent. However, the programs increased morning attendance of children aged 10– 17 by 8 to 12 percent. There was also a positive impact on afternoon attendance of a similar magnitude (Alderman et al., 2012). In India, Singh et al. assessed the impact that the midday meal scheme (MDMS) had on the health status of primary school students (2014). The authors found that the MDMS increased weight-for-age by 0.60 standard deviations and increased height-for-age by 0.27 standard deviations. These results are consistent with larger short-term effects of changes in nutrition on weight (Singh et al., 2014).

The literature concludes that further research is necessary. Specifically, Kazianga et al. call for more investigation into how school feeding programs can increase enrollment and improve academic performance, and a more direct comparison with conditional cash transfers (2012). Sulemana et al. also calls for further research on the health outcomes of students receiving school meals (2013).

Our analysis adds to the literature in several ways. First, we investigate the impact that the Beoog Biiga program had on enrollment and attendance. We examine health indicators, particularly students’ knowledge of health and hygiene behaviors. We take into account gender differences in all aspects, and also examine the impacts of activities specifically targeted to girls. Moreover, we look at the effects of the program on key stakeholders, including teachers, directors, households and PTAs.

Thus, the relevance of the current evaluation is to expand our understanding of school feeding programs in poor economic environments, such as in Burkina Faso. The information gained in this study can improve the design and implementation of future programs in similar environments and political contexts.

IMPAQ International, LLC Page 13 Beoog Biiga Evaluation Report

5.2 Program Implementation

CRS began implementing the 3-year Beoog Biiga program in October 2011, at the start of the 2011– 2012 academic year. Most activities were implemented according to plan during the program To what extent is the program timeline from 2011–2014. CRS encountered some implemented as designed (e.g., challenges implementing the program, but was able lunches and trainings to troubleshoot most issues. Using the semi-annual delivered)? project status reports submitted by CRS to USDA, we reviewed and summarized below the implementation of the three-year program and discussed the challenges the program encountered, as well as the solutions that were implemented.

Year 1

During the first year, 601 schools with 120,171 students received micronutrients and de-worming medication. Trainings were provided to teachers, PTA members, school administrators, preschool staff, MOE staff, parents and community members. In the pilot schools, teachers, directors, mentors, REFLECT9 supervisors, facilitators, participants, SILC10 field agents, community members and girl students were also trained. Awareness-raising sessions on SILC were conducted in each of the 50 pilot school communities. Low maintenance hand-washing stations known as tippy taps were piloted in 10 schools and all 50 pilot schools received a first- aid kit (CRS, May 2012; CRS, November 2012).

Several challenges emerged during the first year. Canteens and take-home rations were supposed to be supplied to students starting in February 2012. However, the commodities provided by USDA did not arrive in Burkina Faso until May 2012, with losses of 699 metric tons. As a result, only six schools received commodities for one month in the first year. While nearly all 50 SILC field agents were trained during the first year to create groups at the start of the second academic year and begin lending funds, one SILC field agent was not trained in the first year. This delayed the formation of the SILC group in one pilot school community. An assessment of the existing hand washing stations in the schools also found that the majority of stations were out of order due to the lack of maintenance and repair (CRS, May 2012; CRS, November 2012).

9 Community-led projects are developed through the REFLECT approach. This process involves a series of practical trainings in which community members identify and analyze needs, and develop, implement, and monitor projects. The funded projects vary by community, but may include grain banks, market gardening and livestock raising. The projects are put in place first and foremost to provide revenue for the school and the parents. 10 Saving and Internal Lending Communities (SILC) is a holistic microfinance approach that allows low-income community members to mobilize local resources by saving small sums of money and lending out accumulated savings within the group.

IMPAQ International, LLC Page 14 Beoog Biiga Evaluation Report

CRS worked with local craftsmen to design the tippy taps in response to the broken hand-washing stations (see Exhibit 9). CRS also trained 25 district managers in financial resource management (CRS, May 2012; CRS, November 2012), in response to a lack of knowledge among district managers on how to manage USG funds, according to CRS. The training aimed to help district managers understand how to manage funds without leading to questioned costs on audits.

Exhibit 9: Tippy taps at a program school in Sanmatenga.

Photo: IMPAQ International

Year 2

Canteens and distribution of take-home rations began in the second academic year. Additionally, 152,649 students received micronutrients and de-worming medication in the fall, and 148,807 in the spring. The trainings from the first academic year continued and were also expanded. School canteen cooks and caregivers were trained on food safety and nutrition basics. A total of 42 Ministry of Health staff were trained and 483 SILC groups were created, and all 10,584 members were trained. CRS launched the Private Service Provider (PSP)11 approach, and 35 SILC field

11 Private Service Provider (PSP) is a model that prepares agents to become independent service providers operating on a market-led basis, in a bid to ensure the long-term (post-project) availability of SILC services in a given area. Agents are recruited, but the project only pays them for a limited period of time. Successful agents are certified as PSPs, and from that moment, offer their training and support services to communities at a fee. A combination of

IMPAQ International, LLC Page 15 Beoog Biiga Evaluation Report agents were certified. 24 REFLECT projects were funded in the fall, and 26 in the spring. CRS distributed 2,315 tippy taps to 698 schools. Posters, board games, and books promoting health, hygiene, and nutrition were also distributed (CRS, May 2013; CRS, November 2013).

Approximately 124 schools out of 701 did not have a nearby water source, so CRS purchased water barrels and carts so that water could be brought from a distance and stored on site. CRS issued a call for bids to supply additional equipment to the bissongos (preschools), but no craftsmen met the criteria for making the purchase. CRS was able to procure all the necessary equipment after consulting with various equipment providers (CRS, May 2013; CRS, November 2013).

There were also difficulties related to the SILC. Transmissions of financial reports were delayed, there was not enough senior management for oversight activities, there were budget limitations, and SILC supervisors were not systematically evaluated. CRS implemented several solutions to tackle these difficulties. An accountant was recruited to provide financial support, the participation of SILC senior members in capacity building activities was increased, CRS realigned project budgets and an SILC supervisor evaluation model was designed to be implemented in the next reporting period (CRS, May 2013; CRS, November 2013).

Additionally, a few other mid-course corrections were made. First, REFLECT groups no longer received further funding for grain bank projects.12 Of the 19 grain bank projects that were financed, only two were able to sell the products, likely due to the grain subsidy that was implemented by the government of Burkina Faso, which caused grain prices to drop. Therefore, CRS decided that these projects were too risky and not worth the investment. Secondly, since CRS could not find a suitable nutrition book for students, CRS worked with a Peace Corps Volunteer to develop one that was appropriate for the context of the schools. CRS contributed to the story line and content and the PCV did all of the illustrations.

Year 3

In the third and final year, the canteens and rations continued, and MOE provided rice, beans, and oil to schools for four months, as planned. Iron, Vitamin A, and de-worming medication was distributed to all primary school students. CRS trained more stakeholders. As of March 2014, 2,159 cooks, 4,995 PTA members from 705 schools, 69 MOE and MOH staff, 2,764 teachers, and

financial and social/ personal incentives drives the PSPs’ work, which is no longer constrained by the project timeline. PSPs continue forming and supporting groups throughout the project period and beyond, greatly leveraging the initial investment that has been made in them through training and supervision (CRS, 2013).

12 Community grain banks are used to store grain that can later be made available to people at fair prices during times of scarcity. The bank prevents traders from exploiting a difficult situation and provides a fairer way of organizing food supplies.

IMPAQ International, LLC Page 16 Beoog Biiga Evaluation Report

114 preschool caregivers had received training. SILC groups expanded to a total of 30,532 members. Thirty-two new community projects, all focused on livestock fattening, were funded, and USD 3,608 from previous REFLECT projects profits was reinvested in schools (CRS, May 2014). Exhibit 10 displays the containers students used for lunch at a program school in Bam.

The community-run canteen was operational in all 35 preschools. The canteens collected cowpeas, sorghum and millet from the parents of students, thus allowing each canteen to function for about two months. Unfortunately, only two canteens were able to do a second food collection, which allowed the canteen to provide meals for more than two months. The regional advisor from DPENA also made recommendations to CRS on how to improve the use of WASH posters and games, and the management of hand-washing stations, medical kits, latrines and pumps. These recommendations were not described in the project status report (CRS, May 2014).

Exhibit 10: Students’ containers for lunch in a program school in Bam.

Photo: IMPAQ International

IMPAQ International, LLC Page 17 Beoog Biiga Evaluation Report

5.3 Effects on School Outcomes

In this section, we present findings on enrollment and attendance for the 489 schools using CRS’s yearly M&E data. To what extent is the program Enrollment effective in increasing student enrollment and attendance? At the beginning of each month, schools recorded student enrollment. Given the fluctuation in enrollment records over the school months, we calculated yearly enrollments as a monthly average within each year to ensure that the same students were not counted twice each month they remained enrolled.

As shown in Exhibit 11, the total enrollment was steady in the first and second years, but increased by about 20,000 students in the third year to reach about 110,000 students in the two provinces of Bam and Sanmatenga. The growth was due to a higher enrollment of both boys and girls.

In all three years of the program, girls’ enrollment was slightly lower than that of boys. By the third year, the program reached about 52,000 girls compared to 57,000 boys. Despite having program activities that specifically targeted girls, the ratio of girls to boys was similar across pilot and program schools. In fact, there were about 6,000 boys and 5,500 girls enrolled in pilot schools in the third year.

Exhibit 11: Total Number of enrolled students

Program Schools Pilot Schools Total Year 1 Year 2 Year 3 Year 1 Year 2 Year 3 Year 1 Year 2 Year 3 Total 81,056 81,607 98,047 9,738 9,956 11,465 90,794 91,562 109,512 Boys 43,386 42,901 51,180 5,279 5,276 5,947 48,665 48,176 57,127 Girls 37,671 38,706 46,867 4,459 4,680 5,518 42,129 43,386 52,384 Source: M&E Data sets, authors’ calculations

To further analyze the composition of boys and girls at the school level, we looked at the average number of students per school. This measure of average school size allows us to better compare the student enrollment in program and pilot schools since there are very few pilot schools in comparison to program schools. As presented in Exhibit 12, pilot schools on average were larger than program schools every year, likely because the additional activities offered more incentives to students to go to school.13 In the third year, an average pilot school had 255 students compared to 221 students at an average program school.

13 Another possible explanation can be ascribed to how pilot schools were selected.

IMPAQ International, LLC Page 18 Beoog Biiga Evaluation Report

Exhibit 12: Average school size

Program Schools Pilot Schools Total Year 1 Year 2 Year 3 Year 1 Year 2 Year 3 Year 1 Year 2 Year 3 Total 183 211 221 216 255 255 186 215 224 Boys 98 111 115 117 135 132 100 113 117 Girls 85 100 106 99 120 123 87 102 107 Source: M&E Data sets, authors’ calculations

All schools increased in average size. Pilot schools had a higher average school size to begin with and remained more numerous over time. As pilot schools were larger, there were also more girls enrolled in them. This is likely an indication that the girls’ mentoring activities helped motivate girls to enroll and stay enrolled. In the third year, there were 123 girls enrolled in an average pilot school compared to 106 girls enrolled in an average program school.

Furthermore, the ratio of girls to boys slightly improved over the three years in both program and pilot schools. In the first year, 46% of all students were girls, in the second year they were 47% and in the third year the percentage of girls reached 48%.14

Yearly enrollment, however, may hide fluctuations during the school year. Consequently, as presented in Exhibit 13, we analyzed average school enrollment by month to further understand the enrollment trends.

As indicated in the exhibit, enrollment increased at both program and pilot schools during the first year. Average enrollment increased from about 180 to 205 students at the program schools and from about 210 to 240 at the pilot schools, as students likely learned about the availability of the program. It is possible that the expectation15 of the upcoming program activities drew students who would not otherwise enroll in school.

14 The percent of girls is calculated by dividing the total number of girls by the total number of students. For example, in year 1, 85 girls out of 183 students in program schools yielded 46%. Similarly, in pilot schools, 99/216 = 46%. 15 The program was supposed to provide school meals starting in February 2011, but this did not happen because of delays and losses.

IMPAQ International, LLC Page 19 Beoog Biiga Evaluation Report

Exhibit 13: Monthly enrollment by school and by gender

Year 1 Year 2 Year 3

250

250

250

200

200

200

150 150 150 Nov Dec Jan Feb Mar Apr May Jun Nov Dec Jan Feb Mar Apr May Jun Nov Dec Jan Feb Mar Apr May Jun Program Pilot

150 150 150

100 100 100

50 50 50 Nov Dec Jan Feb Mar Apr May Jun Nov Dec Jan Feb Mar Apr May Jun Nov Dec Jan Feb Mar Apr May Jun Boys Girls

*Note: The top three graphs show the differences in enrollment at pilot schools versus program schools. The bottom three graphs show the differences in girls’ enrollment versus boys’ enrollment.16 Source: M&E Datasets, authors’ calculations

In the second year, enrollment was stable at both program and pilot schools. There were about 215 students enrolled at program schools and 255 enrolled at pilot schools. The canteens were fully functional in the second year, which likely motivated students to remain enrolled.

Enrollment continued to be stable in the third year. Program schools had about 220 students enrolled, and enrollment at pilot schools remained at 255. This suggests a possible lasting effect of the program on enrollment.

While boys’ enrollment was slightly higher than that of girls, the trends seem similar, suggesting that new boys and girls were enrolling and staying enrolled equally (see Exhibit 13).

16 The M&E data in year 1 contained monthly data only through March 2012.

IMPAQ International, LLC Page 20 Beoog Biiga Evaluation Report

Attendance

As more students enrolled in the school system, we analyzed their actual attendance.17

The percentage of students attending less than 50% of the time is calculated by dividing the number of students attending less than 50% of each month by the total number of students enrolled that month. As presented in Exhibit 14, over the life of the program, the percentage of students attending less than half of the time was very low, ranging from 1–10 percent.

Exhibit 14: Percentage of students attending less than half of the time by year

Program Schools Pilot Schools Total Year 1 Year 2 Year 3 Year 1 Year 2 Year 3 Year 1 Year 2 Year 3 Total 1% 10% 2% 1% 4% 2% 1% 9% 2% Boys 1% 10% 2% 1% 4% 2% 1% 9% 2% Girls 1% 9% 2% 1% 5% 2% 1% 9% 2% Source: M&E Data sets, authors’ calculations

There was no difference between program and pilot schools in the percentage of students attending less than 50% of the time, except in the second year. In the first and third years, 1% and 2% of students attended less than 50% of the time, respectively, at both program and pilot schools. In the second year, 10% of students attended less than half of the time at program schools, while 4% of students at pilot schools attended less than half of the time. It is unclear why attendance worsened in the second year. This may be related to a local shock (such as floods) captured in the M&E data.

The attendance data by month provides some nuanced insights into the worsened attendance during year two. As shown in Exhibit 15, attendance at pilot schools was much better than attendance at program schools in the second year. This is likely due to pilot schools offering more activities to incentivize students. At the pilot schools, about 3% of students attended less than 50% of the time at the beginning of the year. By the end of the year, closer to 4% of students were attending less than 50% of the time. In the program schools, at the beginning of the year, close to 12% of students

17 The M&E datasets contain schools records of the number of students who attended less than 10 days each month and the number of students who attended less than 50% of the days each month. These indicators overlap, and if there are 20 school days in a month, they essentially provide the same information. We have performed the analyses for both indicators and the findings are very similar. Without knowing the number of school days each month, we present the 50% indicator, as it is more representative of student behavior.

IMPAQ International, LLC Page 21 Beoog Biiga Evaluation Report were attending less than half of the time.18 The attendance rate dropped to about 3% in February and March, but then increased to 5% at the end of the year.19

Every month throughout the life of the program, the percentage of boys and girls attending less than 50% of the time was virtually the same, ranging from 1% to 12%.

Exhibit 15: Percentage of students attending less than half of the time by month

Year 1 Year 2 Year 3

15 15 15

10 10 10

5 5 5

0 0 0

Nov Dec Jan Feb Mar Apr May Jun Nov Dec Jan Feb Mar Apr May Jun Nov Dec Jan Feb Mar Apr May Jun Program Pilot

15 15 15

10 10 10

5 5 5

0 0 0

Nov Dec Jan Feb Mar Apr May Jun Nov Dec Jan Feb Mar Apr May Jun Nov Dec Jan Feb Mar Apr May Jun Boys Girls

Note: The top three graphs show the differences in attendance rates at pilot schools versus program schools. The bottom three graphs show the differences in attendance rates between girls and boys. Source: M&E Datasets, authors’ calculations

18 This pattern was very strong in Bam and less so in Sanmatenga, which may reflect a local shock captured in the M&E data. One possible explanation can be related to the increased influx of new students who had not previously been enrolled in school in expectation of lunches and other program benefits, but who eventually dropped out early after realizing that school requires hard work, time and resources. Thus, early in year 2, as overall enrollment went up, considerable percentages of the newly enrolled students started missing school and eventually attended less than 50% of the time or dropped out completely. 19 The slight increase in the percentage of students attending less than half of the time at the end of the school year may be related to seasonal changes when many children are asked to support their families in agricultural activities starting in April/May.

IMPAQ International, LLC Page 22 Beoog Biiga Evaluation Report

The M&E datasets also provided attendance records of students when the canteen was functional and when it was not. We have computed the attendance rates and provided additional tables and graphs in Appendix 3. Exhibit 16 shows students in a classroom at a program school in Bam.

Exhibit 16: Students in class at a program school in Bam.

Photo: IMPAQ International

5.4 Effects on Student Outcomes

The IMPAQ team analyzed the student survey data separately by school type, province and gender. However, since there were practically no To what extent is the program differences across provinces and gender, in this successful in improving section we present the findings by school type. students’ knowledge and The only indicators with considerable gender behavior about nutrition, differences are for knowledge and access to tutors. health, and sanitation? These are presented at the end of the section. Below, we present student results for the following areas:

. Students’ knowledge of nutrition . Students’ knowledge and self-reported practices of hygiene . Observed hygiene practices of students

IMPAQ International, LLC Page 23 Beoog Biiga Evaluation Report

. Observed school hygiene infrastructure . Presence of and participation in school clubs

Students’ knowledge of nutrition

To measure the effect on students’ knowledge of nutrition, the enumerators asked students questions related to their knowledge and awareness of nutritious foods, vitamin A and iron. While a large portion of students at the baseline could already cite at least two foods that make up a nutritious meal, only a few additional program school students could cite at least two nutritious foods at the endline. Pilot schools students showed slightly better knowledge at the endline compared to the baseline. In fact, as shown in Exhibit 17, there was a one percentage point increase in program school students who could cite two or more foods that make up a nutritious meal at endline, compared to a 10 percentage point increase for pilot school students.

Exhibit 17: Students’ knowledge of nutritious meals

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change % of students citing at least 1 food that makes up a 74% 83% +9% 72% 89% +17% nutritious meal % of students citing 2 or more foods that make up a 66% 67% +1% 63% 73% +10% nutritious meal Source: Student Questionnaire, authors’ calculations

Similarly, compared to students’ awareness at the baseline, only a small number of additional program school students had heard about vitamin A at the endline, but a larger number of additional pilot school students were aware of vitamin A at the endline (see Exhibit 18).

Exhibit 18: Students’ knowledge of vitamin A

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change % of students having heard 57% 64% +7% 54% 71% +17% of vitamin A % of students citing 2 or 3% 24% +21% 2% 30% +28% more benefits of vitamin A % of students citing 2 or 56% 87% +30% 56% 97% +41% more foods with vitamin A Source: Student Questionnaire, authors’ calculations

IMPAQ International, LLC Page 24 Beoog Biiga Evaluation Report

Very few students knew about the nutritional benefits of iron before the program, but increased their knowledge considerably after the program (see Exhibit 19). Additionally, more students in both types of schools were able to cite two or more nutritional benefits of iron and gained substantial knowledge of foods containing iron during the program years.

Exhibit 19: Students’ knowledge of iron

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change % of students having heard 9% 53% +44% 10% 59% +49% of iron % of students citing 2 or 0% 24% +24% 2% 31% +30% more benefits of iron % of students citing 2 or 0% 46% +46% 1% 43% +42% more foods with iron Source: Student Questionnaire, authors’ calculations

Students’ knowledge and self-reported practices of hygiene

To measure student’s knowledge and self-reported practices of hygiene, the enumerators asked students questions about their bathing and hand-washing practices, as well as about their latrine usage.

Nearly all students during both the baseline and the endline reported washing their bodies at least once the previous day and reported using soap (see Exhibit 20). These self-reported hygiene practices remained the same or slightly improved over time. However, there was a dramatic increase in the percentage of students who reported two or more periods in which they washed their hands. Specifically, there was a 35 percentage point increase in the number of program school students citing two or more periods of the day when they washed their hands, and a 44 percentage point increase for pilot school students. While this data is encouraging, it should be interpreted with caution because with any self-reported data respondents may answer based on the answer they believe is expected of them (social desirability bias).

IMPAQ International, LLC Page 25 Beoog Biiga Evaluation Report

Exhibit 20: Students’ hygiene practices

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change % of students who washed their body at least once the 91% 92% +1% 89% 97% +7% previous day % of students who used 87% 97% +10% 89% 97% +8% soap to wash their bodies % of students citing 2 or more periods of the day 18% 53% +35% 16% 60% +44% during which they washed their hands Source: Student Questionnaire, authors’ calculations

Students also reported how they washed their hands and why they used latrines. Students seemed to be shifting away from washing their hands only with water to washing their hands with soap and water (see Exhibit 21).

Exhibit 21: Students’ hand-washing self-reported practices

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change % of students who only used water to wash their 57% 12% -45% 54% 1% -53% hands % of students who used soap and water to wash 42% 86% +44% 46% 97% +51% their hands Source: Student Questionnaire, authors’ calculations

The majority of students reported using latrines before and after the program, though many more pilot school students reported using the latrines at the endline compared to the baseline. However, the use of latrines is still not universal (see Exhibit 22).

IMPAQ International, LLC Page 26 Beoog Biiga Evaluation Report

Exhibit 22: Students’ reasons for using latrines

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change % of students saying they 22% 18% -4% 22% 2% -20% did not use latrines % of students citing at least 77% 78% +1% 78% 84% +6% 1 reason for using latrines % of students citing 2 or more reasons for using 8% 13% +5% 9% 16% +7% latrines Source: Student Questionnaire, authors’ calculations

Observed hygiene practices of students

While the self-reported hygiene practices of students suggest that the program was successful in contributing to a positive change in behavior, it is critical to actually observe the students’ behaviors in practice. Therefore, the enumerators also observed students’ hand-washing behaviors during certain critical moments while administering the survey in order to check whether students washed their hands. The most frequently observed moments at endline for both program and pilot school students were ‘after using the latrines’ and ‘before eating.’

As presented in Exhibit 23, there was a considerable reduction in the number of students who did not wash their hands after using the latrines or before eating. In fact, between baseline and endline, there was a 29 percentage point decrease among program school students and a 37 percentage point decrease among pilot school students who did not wash their hands before these two activities. At the same time, there was an increase in the percentage of students who washed their hands with water only and with soap and water at endline.

Exhibit 23: Observed practices of hand washing among students

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change % of students washing their 0% 19% +19% 1% 30% +29% hands with water only % of students washing their 20% 30% +10% 18% 26% +8% hands with soap and water % of students who did not 80% 51% -29% 81% 44% -37% wash their hands Source: Student Questionnaire, authors’ calculations

IMPAQ International, LLC Page 27 Beoog Biiga Evaluation Report

It is important to note that these findings indicate that the program was effective in changing student hygiene behaviors. While there remains a discrepancy between the self-reported and the observed practices, many students of the Beoog Biiga program implemented the knowledge acquired through the program.

Observed school hygiene infrastructure

To assess changes in hand washing, it is essential that water be present at the school site. Similarly, to observe greater use of latrines, it is important that the latrines be clean and functional. Therefore, to measure the schools’ quality of hygiene infrastructure, the enumerators also observed the presence of water infrastructure and the hygiene of school latrines.

In terms of water infrastructure at schools such as wells, pumps, and containers, a larger number of program schools (78%) and pilot schools (86%) had at least one form of water infrastructure. While this represents a substantial improvement relative to baseline, a number of schools remained without water at their premises (22% of program schools and 14% of pilot schools; see Exhibit 24).

Exhibit 24: Observed presence of water infrastructure

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Baseline with Endline with Endline Baseline with Endline with Endline Water Water without Water Water without Water Water Program Schools Pilot Schools .

Source: Student Questionnaire, authors’ calculations

In terms of latrines, compared to the baseline, the enumerators still observed a number of program schools (17%) at the endline with no latrines. All pilot schools were equipped with a latrine at the endline (see Exhibit 25).

IMPAQ International, LLC Page 28 Beoog Biiga Evaluation Report

Exhibit 25: Observed presence of latrines and cleanliness of latrines in schools

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change % of schools with no latrines 21% 17% -4% 5% 0% -5%

% of school latrines with 55% 61% +6% 80% 47% -33% human feces on the latrines % of school latrines with 46% 20% -25% 62% 22% -40% large presence of flies % of school latrines with no human feces or presence of 19% 21% +1% 13% 35% +22% flies Source: Student Questionnaire, authors’ calculations

Exhibit 26 shows a photo of a latrine at a program school in Sanmatenga.

Exhibit 26: Latrines at a program school in Sanmatenga.

Photo: IMPAQ International

IMPAQ International, LLC Page 29 Beoog Biiga Evaluation Report

Presence of school clubs and students’ participating in school clubs

One aspect of the Beoog Biiga program was to improve the school climate by encouraging active participation in school clubs (see Exhibit 27). According to surveyed students, more pilot schools than program schools had at least some clubs at the endpoint, and pilot schools had more clubs than program schools. There were two active clubs on average at program schools and more than three active clubs at pilot schools.

However, 42% of program students and 18% of pilot students reported that there were no active clubs at their school at endline. While this presents an improvement relative to what students reported at baseline, it also indicates that not every student had a chance to become involved in school club activities. When clubs were present in schools, students participated in them. Half of program school students and 69% of pilot school students said that they belonged to a student club. This demonstrates an important success of the program.

Exhibit 27: Schools with active clubs

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change Average number of active 0.1 2.0 +1.9 0.2 3.4 +3.2 clubs in schools % of schools with 0 active 94% 42% -52% 92% 18% -74% clubs % of schools with 1–2 active 6% 29% +23% 5% 25% +20% clubs % of schools with 3 or more 0% 29% +29% 3% 57% +54% active clubs % students participating in 3% 50% +47% 5% 69% +64% clubs Source: Student Questionnaire, authors’ calculations

Another essential component of the Beoog Biiga program was the involvement of designated mentors whose role was to encourage girls to enroll and attend school regularly. Mentors were available to female students in pilot schools. As presented in Exhibit 28, the majority of girls from pilot schools knew mentors who could help them with school work (80%) and who could receive support from them (80%).

Furthermore, students from program schools where mentors were not available knew of and received support from mentors at high rates. This rather surprising finding suggests that program practices such as mentoring spread to other communities at residents’ own expense.

IMPAQ International, LLC Page 30 Beoog Biiga Evaluation Report

Exhibit 28: Knowledge of and access to mentors

Program Schools Pilot Schools Boys Girls Total Boys Girls Total % who knew a mentor 39% 57% 47% 62% 80% 70% % who received support 30% 48% 38% 52% 80% 65% from a mentor Source: Student Questionnaire, authors’ calculations

5.5 Effects on Directors

In this section, we report findings on the information collected from directors at participating schools. We assess findings in the following three To what extent is the program areas: successful in improving school . Number of trainings received directors’ knowledge and . Knowledge of education quality behavior about girls’ . Presence of active student clubs education, nutrition, health, and sanitation? Number of received trainings

In comparison with CRS’s project agreement, the data shows that CRS trained many more directors than originally planned, and increased the number of training modules as well. In fact, as presented in Exhibit 29, directors from program schools received an average of 3.4 trainings while directors from pilot schools received 3.7 trainings.The percentage of directors who participated in 6–7 training modules was 19% among program schools and 31% among the pilot schools.

IMPAQ International, LLC Page 31 Beoog Biiga Evaluation Report

Exhibit 29: Directors’ participation in professional development

Program Pilot schools schools Average number of received training modules 3.4 3.7 % of directors who received 0–3 training modules 54% 56% % of directors who received 4–5 training modules 28% 13% % of directors who received 6–7 training modules 19% 31% Source: Director Questionnaire, authors’ calculations

Directors’ knowledge of education quality

There was a substantial gain in directors’ knowledge of education quality over time. To measure the effect on directors’ knowledge, the enumerators asked directors to cite criteria important to quality education. In comparison with baseline, far more directors were able to cite relevant aspects at the endline. As shown in Exhibit 30, 57% of program school directors cited five or more important aspects as compared with 26% at baseline. Similarly, in pilot schools, 62% cited five or more important aspects as compared with 22% at baseline. There were few differences across program and pilot schools, as expected, since directors from both types of schools received similar trainings.

Exhibit 30: Directors’ knowledge of education quality

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change % of directors citing 5 or more aspects of quality 26% 57% +31% 22% 62% +40% education Source: Director Questionnaire, authors’ calculations

The most cited aspects of education quality among directors remained unchanged over the life of the program. The top five in both baseline and endline surveys were:

. Presence of school lunches . Teacher support . School health . Infrastructure . Community participation

IMPAQ International, LLC Page 32 Beoog Biiga Evaluation Report

Presence of active student clubs

The presence of active student clubs reflected improvements in the school environment. At baseline, all schools had on average of about one active club, according to school directors. Three years later, there were 1.6 clubs in program schools and 3.2 clubs in pilot schools, on average (see Exhibit 31). The most common student clubs were hygiene, sanitation and health clubs. This finding is encouraging as substantial resources and activities were devoted to hygiene, sanitation and health over the life of the program.

Exhibit 31: Schools with active clubs

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change Average number of active 1.0 1.6 +0.6 1.2 3.2 +2 clubs in schools % of schools with 0 active 69% 33% -37% 50% 4% -46% clubs % of schools with 1–2 active 73% 72% 0% 74% 27% -47% clubs % of schools with 3 or more 6% 20% +14% 6% 71% +65% active clubs Source: Director Questionnaire, authors’ calculations

5.6 Effects on Teachers

In this section, we present teacher findings in four areas:

. Number of received trainings To what extent is the program . Pedagogical practices and knowledge of successful in improving school education quality teachers’ knowledge and . Knowledge of nutrition behavior about girls’ . Knowledge and self-reported practice of education, nutrition, health, hygiene and sanitation?

Number of received trainings

The enumerators asked teachers whether they received training on all eight modules. Similar to directors, the data shows that CRS trained many more teachers than originally planned, and increased the number of training modules as well. As presented in Exhibit 32, teachers from program schools received an average of 1.9 trainings while teachers from pilot schools received

IMPAQ International, LLC Page 33 Beoog Biiga Evaluation Report

3.2 trainings. The higher number of received trainings amount pilot school teachers is as expected as these teachers received seeral additional trainings.

Exhibit 32: Teachers’ participation in professional development

Program Pilot schools schools Average number of received training modules 1.9 3.2 % of teachers who received 1–4 training modules 80% 68% % of teachers who received 5–6 training modules 6% 15% % of teachers who received 7–8 training modules 2% 11% Source: Teacher Questionnaire, authors’ calculations

The most cited training modules that both program and pilot teachers reported receiving were the ones titled ‘instructional techniques’ and ‘gender and education’. The least cited by all teachers were ‘prejudices and stereotypes in the community and in the school’ and ‘gender sensitive classrooms’.

Teachers’ pedagogical practices and knowledge of education quality

To measure whether the professional development training led to changes in teachers’ instructional practices, the enumerators asked teachers whether they practiced the approaches learned in training modules and whether they applied participatory techniques in the classroom. As presented in Exhibit 33, the majority of teachers reported practicing the approaches and techniques in their classroom. These numbers reflect self-reported practices and should be interpreted with caution.

Exhibit 33: Teachers’ instructional practices

Program Pilot schools schools % of teachers who practiced the approaches learned in 89% 93% training modules % of teachers who applied participatory techniques in 98% 99% their classroom Source: Teacher Questionnaire, authors’ calculations

The enumerators also asked teachers to cite criteria important to quality education. In comparison to the teachers’ baseline knowledge, far more teachers were able to cite relevant aspects during the endline (See Exhibit 34). There is a small difference in the gain in knowledge between program and pilot schools since teachers from both types of schools received similar trainings.

IMPAQ International, LLC Page 34 Beoog Biiga Evaluation Report

Exhibit 34: Teachers’ knowledge of education quality

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change % of teachers citing 5 or more aspects of quality 13% 40% +27% 13% 43% +30% education Source: Teacher Questionnaire, authors’ calculations

The top five cited aspects of education quality among teachers at the endline were:

. Presence of school lunches . Teacher support . School health . Infrastructure . Community participation

Teachers’ knowledge of nutrition

To measure the effect of the program on teachers’ knowledge of nutrition, the enumerators asked them to cite at least three foods that contain vitamin A. In comparison to the teachers’ baseline knowledge, far more teachers were able to cite foods containing vitamin A at the endline (see Exhibit 35). Even though teachers from both types of schools received the same trainings, the data shows that teachers from pilot schools gained more knowledge than teachers from program schools. This suggests that pilot school teachers may have benefited from additional program activities, such as setting up and maintaining clubs on similar topics. The most cited foods among teachers remained unchanged over the life of the program. The top three were: green-leaf vegetables, milk products and mangoes.

Exhibit 35: Teachers’ knowledge of nutrition

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change % of teachers citing 3 or more foods containing 28% 69% +41% 24% 78% +53% vitamin A Source: Teacher Questionnaire, authors’ calculations

IMPAQ International, LLC Page 35 Beoog Biiga Evaluation Report

Teachers’ knowledge and self-reported practices of hygiene

To measure the effect on teachers’ knowledge of hygiene, the enumerators asked teachers why there should not be human feces (stools) or animal excrements in the school courtyard. At the endline, teachers were able to cite more reasons than at baseline (see Exhibit 36).

At both the baseline and the endline, the majority of teachers cited ‘to prevent diseases’ as the main reason for keeping human feces and animal excrements out of the courtyard. Many more teachers at the endline, however, also cited the reasons ‘to have a more agreeable school environment’ and ‘to limit the presence of flies’, reflecting a change in overall perceptions of the importance of hygiene among teachers.

Exhibit 36: Teachers’ knowledge of hygiene

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change % of teachers citing 2 or more reasons why human feces and animal excrements 28% 68% +40% 41% 70% +29% should be kept out of the courtyard Source: Teacher Questionnaire, authors’ calculations

Finally, more teachers cited three or more periods of the day during which they washed their hands. The data shows that at the endline teachers reported washing their hands more often (see Exhibit 37).

Exhibit 37: Teachers’ hand-washing practices

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change % of teachers citing three or more periods of the day 31% 61% +31% 39% 67% +27% during which they washed their hands Source: Teacher Questionnaire, authors’ calculations

IMPAQ International, LLC Page 36 Beoog Biiga Evaluation Report

5.7 Effects on Parent-Teacher Associations

In this section, we report the findings related to members of the parent-teacher associations (PTAs) in the project school communities. The enumerators asked questions about the following four areas: To what extent is the program effective in increasing . Functionality of PTAs . Functionality and sustainability of school knowledge of PTA members canteens about food preparation, . PTA support of school canteens nutrition, student health and . PTA knowledge and management practices sanitation, and girls’ education?

Functionality of PTAs

To measure the functionality of PTAs and assess how active they were, the enumerators asked PTA representatives the number of general assemblies (or PTA meetings) organized by PTAs the previous year. As presented in Exhibit 38, there was a slight decline in the average number of general assemblies in program schools, from 4.3 at baseline to 3.8 at endline. There was a similar decline in pilot schools, from 4.7 at baseline to 4.4 at endline.

Exhibit 38: PTA-organized general assemblies in previous year

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change Average number of general 4.3 3.8 -0.5 4.7 4.4 -0.3 assemblies % of schools with 2 or more 93% 98% +5% 98% 98% +0% general assemblies Source: PTA Questionnaire, authors’ calculations

At the end of the program, nearly all program and pilot PTAs (98%) reported organizing two or more general assemblies during the previous year. CRS was slightly below their Year 3 target on the Community Development progress indicator20 of 5.2 organized assemblies for all schools to have parent groups playing a defined role in program management at both program and pilot schools.

20 Catholic Relief Services, Baseline Survey Report, Ouagadougou, Burkina Faso, June 2012.

IMPAQ International, LLC Page 37 Beoog Biiga Evaluation Report

Functionality and sustainability of school canteens

To appraise the functionality and sustainability of school canteens, the enumerators asked PTA representatives how many months the canteens had been operational during the previous year. As shown in Exhibit 39, both program and pilot school PTAs reported that the canteens were functional about eight months out of the year during endline, compared to six months at the time of the baseline. This suggests that all canteens were, on average, operational for two extra months at endline relative to the baseline period.

While there is variation in terms of months across the schools, there is evidence that the majority of canteens were operational for the entire school year. In fact, at endline, the data shows that 85% of program schools and 98% pilot schools had functioning canteens for seven to ten months during the previous academic year (see Exhibit 39).

Exhibit 39: Number of months over the year that schools had functioning canteens

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change Average number of months schools had functioning 6 8 +2 6 8 +2 canteens 0–2 months schools had 11% 2% -9% 6% 2% -4% functioning canteens 3–6 months schools had 40% 13% -27% 52% 0% -52% functioning canteens 7–10 months schools had 49% 85% +36% 42% 98% +56% functioning canteens Source: PTA Questionnaire, authors’ calculations

The enumerators also asked the PTAs about the average number of months during which the different stakeholders provided food for the canteens. At baseline, for the six months during which the canteens were functional, the MOE provided food for four months and the community provided food for another two months (see Exhibit 40). At endline, for the eight months the canteens were functional, the MOE and other stakeholders (mainly CRS) provided food for three months each, and the community provided food for another two months. There was no difference between program and pilot schools.

IMPAQ International, LLC Page 38 Beoog Biiga Evaluation Report

Endline data indicates that the MOE and the community predominantly supported school canteens. Thus, CRS exceeded its Year 3 target on the Graduation/Sustainability progress indicator21 of having 50% of local support.

Exhibit 40: Number of months various stakeholders provided food for canteens

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change Average # of months MOE 4 3 -1 4 3 -1 provided food Average # of months 2 2 0 2 2 0 community provided food Average # of months other 0 3 +3 0 3 +3 donors provided food Source: PTA Questionnaire, authors’ calculations

PTA support of school canteens

To assess the extent of parental and community support Exhibit 41: Food stored at a of school canteens, the enumerators asked PTA program school in Sanmatenga. representatives about contributions and frequencies on five defined levels of inputs: wood, condiments, and utensils for cooking, food stocking and storage, and payment of canteen cooks.

Exhibit 41 shows food stored at a program school in Sanmatenga.

As shown in Exhibit 42, at 58% of program schools and 67% of pilot schools, the parents and community contributed four out of five inputs to the school canteens. For both program and pilot schools, the data shows that CRS exceeded its Year 3 target on its Community Development progress indicator.22 Photo: IMPAQ International

21 Catholic Relief Services, Baseline Survey Report, Ouagadougou, Burkina Faso, June 2012. 22 Catholic Relief Services, Baseline Survey Report, Ouagadougou, Burkina Faso, June 2012.

IMPAQ International, LLC Page 39 Beoog Biiga Evaluation Report

Exhibit 42: Schools where parents provided a defined level of inputs to canteens

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change % of schools where parents 47% 58% +11% 41% 67% +26% provided 4 out of 5 inputs Source: PTA Questionnaire, authors’ calculations

Exhibit 43 presents the breakdown of parental and community contributions to school canteens on the five defined levels of inputs, as reported by PTA representatives.

Exhibit 43: Schools where parents provided a defined level of inputs

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change % of schools where communities contributed 53% 52% -1% 46% 73% +28% wood for 5 or more days during a canteen week % of schools where communities contributed 62% 62% 0% 61% 67% +5% condiments for 5 or more days during the week % of schools where communities ensured 76% 92% +16% 76% 91% +16% stocking and storage of food % of schools where communities provided 81% 94% +12% 88% 91% +4% utensils % of schools where communities ensured the 64% 65% +1% 55% 85% +30% payment of canteen cooks Source: PTA Questionnaire, authors’ calculations

PTA representatives’ knowledge and management practices To evaluate the knowledge of PTA representatives on PTA responsibilities and practices, the enumerators asked representatives to cite the primary responsibilities of PTAs and the characteristics of good practices for stocking food. As shown in Exhibit 44, in comparison to the PTAs’ baseline knowledge, far more PTAs were able to cite at least three main PTA responsibilities at the endline. The most cited responsibility was ‘providing basic social services

IMPAQ International, LLC Page 40 Beoog Biiga Evaluation Report

(such as building maintenance, latrines, water source)’ while the least cited one was ‘hygiene and cleanliness of children’.

Exhibit 44: Knowledge and management practices of PTAs

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change % of PTA members citing 3 or more main 51% 78% +27% 56% 80% +24% responsibilities % of PTA members citing 3 or more characteristics of 42% 71% +29% 43% 78% +35% good management practices Source: PTA Questionnaire, authors’ calculations

To further assess PTA practices, the enumerators directly reviewed PTAs’ various management books to check whether they contained the required items such as quantities, dates and signatures. As presented in Exhibit 45, at endline, very few PTAs did not have their money management book. At endline, 63% of PTA books from program communities and 78% from pilot communities contained four or all five items required for good maintenance of their money management book. The most often observed item in the books were the entries of expenses and the dates. The least observed items in the books were signatures.

Exhibit 45: PTAs’ money management books with essential elements

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change % of PTAs without a book 59% 13% -46% 60% 7% -53% % of books with 3 or less 81% 37% -45% 82% 22% -60% items % of books with 4 or 5 items 19% 63% +45% 18% 78% +60% Source: PTA Questionnaire, authors’ calculations

Similarly, at endline, even more PTAs were able to present their food management books (see Exhibit 46). The most often observed item in the books were the dates, while the least often observed item was the signatures.

IMPAQ International, LLC Page 41 Beoog Biiga Evaluation Report

Exhibit 46: PTAs’ food management books with essential elements

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change % of PTAs without a book 53% 3% -49% 59% 2% -56% % of books with 4 or less 81% 25% -56% 73% 26% -47% items % of books with 5 or 6 items 19% 75% +56% 27% 74% +47% Source: PTA Questionnaire, authors’ calculations

The enumerators also checked how many projects the PTAs conducted over the course of two years and whether they had an action plan. Both types of PTAs also completed slightly more projects by the end of the program compared to the beginning.

Exhibit 47: PTA implemented projects over the last two years

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change Average number of projects 1.4 2.5 +1.1 2.4 3.9 +1.5 among PTAs % of PTAs with 0 projects 32% 13% -20% 14% 0% -14% % of PTAs with 1–2 projects 49% 41% -8% 45% 36% -9% % of PTAs with 3 or more 19% 46% +27% 41% 64% +23% projects Source: PTA Questionnaire, authors’ calculations

Finally, as presented in Exhibit 48, a much larger number of program school PTAs (79%) and pilot school PTAs (89%) had action plans at the endline, compared to baseline.

IMPAQ International, LLC Page 42 Beoog Biiga Evaluation Report

Exhibit 48: Existence of PTA action plans

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Baseline Endline Baseline Endline Program Schools Pilot Schools .

Source: PTA Questionnaire, authors’ calculations

5.8 Effects on Households

In this section, we present findings in five areas, as reported by households representatives in the same communities:

. Household knowledge of the benefits of girls’ education To what extent is the program . Household knowledge of hygiene successful in improving local . Household self-reported practice of hygiene household knowledge and . Observed hygiene practices of households behavior about girls’ . Household participation in REFLECT education, nutrition, health, activities and sanitation?

Household knowledge of the benefits of girls’ education

To measure the effect on household members’ knowledge of the benefits of educating girls, the enumerators asked households to cite the advantages. In comparison to the households’ baseline knowledge, more households were able to cite advantages at the endline. In fact, at the beginning of the program, households were able to cite one advantage on average, while three years later they mentioned about two advantages (see Exhibit 49). About two-fifths of all households at both program and pilot schools and communities cited three or more advantages. The most and least cited advantages between both program and pilot school households remained unchanged over the

IMPAQ International, LLC Page 43 Beoog Biiga Evaluation Report life of the program. The most cited were improvement of girls’ life outcomes and the opportunity to obtain employment. And the least cited were girls’ health and health of girls’ future children.

Exhibit 49: Household knowledge of benefits of girls’ education

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change Average number of cited 1.0 2.0 +1 1.2 1.9 +.7 benefits among households % of households citing 3 or 18% 39% +21% 27% 43% +15% more benefits Source: Household Questionnaire, authors’ calculations

Household knowledge of hygiene

As presented in Exhibit 50, while nearly all households reported washing their hands, some households still reported not using latrines three years later (28% among program communities and 22% among pilot communities). As these practices are not yet universal, to measure the change on household knowledge of hygiene, the enumerators asked members of the household to cite reasons why they washed their hands and why they used the latrines. While more program and pilot school households cited additional reasons for both activities at the endline, the percentage of households that could cite more than three reasons remained low across the board. Thirty-one percent of program school households and 24% of pilot school households were able to cite three or more reasons for washing their hands. The most cited reason among all households before and after the program was ‘to avoid illnesses such as diarrhea’. All other reasons were cited less frequently.

IMPAQ International, LLC Page 44 Beoog Biiga Evaluation Report

Exhibit 50: Household reasons for hand washing and using latrines

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change % of households citing 3 or more reasons for hand 9% 31% +22% 11% 24% +13% washing % of households citing 2 or more reasons for using 19% 34% +15% 22% 35% +14% latrines % of households saying they 46% 28% -18% 51% 22% -29% do not use latrines Source: Household Questionnaire, authors’ calculations

In terms of latrine use, households provided even fewer reasons. In fact, only 34% of program school households and 35% of pilot school households were able to cite at least two reasons why they use the latrines. The most cited reason between both types of households before and after the program was ‘to reduce the transmission of illnesses’.

Household self-reported practices of hygiene

To measure the self-reported household hygiene practices, the enumerators asked household members for daily activities in which they washed their hands. Nearly all households (99%) reported washing their hands before eating at both the baseline and the endline. As shown in Exhibit 51, 37% of households from program communities and 35% of households from pilot communities reported washing their hands for at least three activities.

These top three activities included the almost universally cited ‘before eating’ as well as ‘when they have dirt on their hands’ and ‘after using the latrines’. The fact that more households reported washing their hands after using the latrine became more common across households at both program and pilot communities.

Exhibit 51: Household hand-washing practices

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change % of households citing 3 or more activities for which 11% 37% +26% 15% 35% +28% household members washed their hands Source: Household Questionnaire, authors’ calculations

IMPAQ International, LLC Page 45 Beoog Biiga Evaluation Report

Observed hygiene practices of households

To measure the hygiene practices and behaviors at households, the enumerators collected data on latrine hygiene standards using the following categories:

. Presence of human feces in the latrines . Large presence of flies . No human feces or presence of flies . Clean, but not used . Presence of ashes in the latrines . No walls covering the latrines . No latrines

Compared to baseline, the enumerators observed many more latrines among households. As shown in Exhibit 52, at baseline, 63% of households from program communities and 74% of households from pilot communities had no latrines. Three years later, these percentages fell to 37% and 34%, respectively. This finding is very encouraging as the availability of (clean) latrines is essential to enabling changes in behavior around the use of latrines.

The data also shows considerable improvements in the level of cleanliness of existing latrines. While the data shows that most households maintained their latrines generally clean before and after the program, more households after the program had cleaner latrines.

Exhibit 52: Observed presence of latrines and cleanliness of latrines among households

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change % of household latrines with presence of human feces in 9% 12% +2% 9% 11% +2% the latrines % of household latrines with 18% 7% -11% 19% 9% -9% large presence of flies % of household latrines with no human feces or presence 16% 41% +25% 7% 42% +35% of flies % of households with no 63% 37% -26% 74% 34% -40% latrines present at the home Source: Household Questionnaire, authors’ calculations

IMPAQ International, LLC Page 46 Beoog Biiga Evaluation Report

The enumerators also observed hand-washing behaviors at the household during certain critical moments while administering the survey. The most commonly observed occasion for hand washing at endline for both program and pilot school households was ‘before eating’. As shown in Exhibit 53, although over half of all households washed their hands with water only, a large percentage of households still did not wash their hands (36% of households from program communities and 47% of households from pilot communities).

Moreover, if some hand washing was observed among households, it was most often only done with water. Very few households before and after the program washed their hands with soap and water. In fact, only 7% of households washed their hands with soap and water, suggesting that there is room for improvement (see Exhibit 53).

Exhibit 53: Observed practices of hand washing among households

Program Schools Pilot Schools Baseline Endline Change Baseline Endline Change % of households washing 58% 57% -2% 65% 45% -19% their hands with water only % of households washing their hands with soap and 3% 7% +4% 5% 7% +2% water % of households that did 38% 36% -1% 31% 47% +16% not wash their hands Source: Household Questionnaire, authors’ calculations

Household participation in REFLECT activities To measure household participation in REFLECT activities, the enumerators first asked whether households had heard of the REFLECT program and, if so, which REFLECT activities they had participated in. As indicated in Exhibit 54, awareness of REFLECT spread across the two provinces beyond the pilot schools. In fact, 80% of households in pilot communities heard about it and 46% of households from program communities where REFLECT was not available also knew about it. Exhibit 54 also presents various aspects of participating in REFLECT activities: training on community projects, conception and implementation of community projects, receipt of funding for projects, etc.

IMPAQ International, LLC Page 47 Beoog Biiga Evaluation Report

Exhibit 54: household knowledge and participation in REFLECT activities

Program schools Pilot schools % of households that have heard about REFLECT 46% 80% % of households that received training on community 22% 35% projects or on financial management of projects % of households that participated in conception or 16% 45% implementation of community projects % of households that received funding or technical 19% 40% support for the implementation of their project % of households that started their project 12% 30% % of households that started and completed their 9% 18% projects Source: Household Questionnaire, authors’ calculations

5.9 Stakeholder Perceptions on Program Successes and Challenges

In this section, we present the findings on the programs successes, key lessons and challenges that may be pertinent to future experiences using the qualitative data collected from interviews with What key lessons and various program beneficiaries and stakeholders. challenges from the program implementation can be drawn Successes and Key Lessons for future sustainability?

Improved student attendance and attention. Nearly all of the interviewed program stakeholders and beneficiaries observed noticeable improvements in school attendance and attention levels in the classroom as a result of the school meals, micronutrients, and improved hygiene practices such as hand washing. One beneficiary explained that school meals provided more study time for students because the meals motivated students to stay at school over lunch to study instead of walking several miles back home for lunch. Exhibit 55 is a photo of students studying during lunch at a program school in Bam.

Improved community engagement in school management. Many of the stakeholders and beneficiaries praised the increased community engagement in school management and upkeep, which they attributed to specific program activities including SILC, REFLECT, trainings for PTA on developing action plans, and awareness-raising among communities. In several communities, the beneficiaries stated that the parents had built additional classrooms for the schools. One stakeholder explained that the SILC activities provided an additional source of income for women in particular, which helped households afford to send their children to school and maintain them

IMPAQ International, LLC Page 48 Beoog Biiga Evaluation Report in school. Moreover, by helping communities to create savings accounts, SILC also helped communities support maintenance-related needs of the schools.

Strengthening of teacher and provincial Ministry of Education staff (DPENA) skills and knowledge. Program stakeholders also commended the program’s capacity building activities for teachers and the DPENA. According to stakeholders, teacher trainings improved teachers’ pedagogical practices by training them on new and critical themes such as gender equality. One beneficiary explained that teachers in the school identified high-performing and motivated female students, and gave them more responsibilities in the classroom, which helped make female students feel more valued and motivated to learn. The trainings for and engagement of DPENA staff in the field to monitor program activities helped them gain a better understanding of the issues schools face on the ground, ultimately helping the DPENA design and support more responsive educational policies and programs. The presence of DPENA in the schools also helped encourage

teachers and directors by making them feel more supported.

Exhibit 55: Students study during lunch at a school in Bam

Photo: IMPAQ International

IMPAQ International, LLC Page 49 Beoog Biiga Evaluation Report

Challenges

Lack of adequate water and sanitation infrastructure. Several program beneficiaries stated that the CRS tippy taps were quite time-intensive and could not easily accommodate large numbers of children who needed to wash their hands all at once. As a result, many students ended up not washing their hands. Moreover, beneficiaries and stakeholders alike mentioned the lack of adequate access to water, whether man-made water infrastructure (pumps and wells) or natural sources (river, streams and lakes) in many of the schools. Consequently, though schools were equipped with tippy taps, they could not operate them due to a lack of water. Additionally, project stakeholders explained that many schools lacked latrines, preventing students from putting into practice what they had learned in the program’s awareness-raising activities regarding the importance of using latrines.

Difficulties related to school meal budgeting, allocation and tracking. Project stakeholders stated that the schools’ commodity log reports can take time to return to the CRS team. Though the time delays decreased from one month to 2–3 weeks after the start of the project, this still does not allow CRS to troubleshoot food-related issues in a timely and efficient manner. This problem is compounded by the lack of adequate commodity supervisors. Stakeholders also explained that there are often discrepancies in the number of student recipients between those calculated by the MOE M&E team, the CRS M&E team and the logistics team. As a result, schools sometimes experienced a shortage or surplus of food. CRS and the MOE teams were not always able to redistribute excess food before it expired and it had to be destroyed.

Moreover, the quantity of food for school meals provided by CRS from the USDA was not always reliable, since the USDA based their food provision not on quantity but on monetary value. Since food crop prices and ocean freight increased progressively in the US throughout the life of the program, USDA supplied CRS with about half of the quantity of food originally expected. This meant that the MOE and CRS team were not able to provide adequate food to all schools during the last year of the program and many schools had to end their meal programs early.

Increased teacher responsibility. Some beneficiaries mentioned that the program activities relied heavily on the voluntary support of teachers, creating teacher fatigue. For instance, teachers were responsible for picking up the micronutrients at the sub-provincial level at their own expense. Teachers were also responsible for administrating the micronutrients to children, which took up a lot of class time. Some teachers did not feel adequately trained to do it. According to project stakeholders, many teachers did not know how to fill out the micronutrient register set up by CRS or they filled it out incorrectly. Similar to the commodity log books, CRS must wait until the end of the semester to collect the micronutrient registers since teachers need them during the semester period, which prevents CRS from promptly dealing with issues related to incorrect administration

IMPAQ International, LLC Page 50 Beoog Biiga Evaluation Report of micronutrients to children. Moreover, PTA representatives who were illiterate relied on teachers to fill out the food and management books.

Limitations of provincial MOE staff. Project stakeholders mentioned the limited ability of provincial Ministry of Education staff to provide adequate monitoring of all school activities because of a lack of financial resources. Stakeholders explained that school activities from project partners, like CRS’s Beoog Biiga activities, required additional monitoring, but that the costs for the additional monitoring were not budgeted in the partnership with CRS. While stakeholders did not necessarily expect CRS would take on the added costs, they emphasized the importance of the issue and the need to resolve it.

Increase of student-to-teacher ratio. Other challenges the stakeholders mentioned included the increase in student-to-teacher ratios as a result of increased school attendance. While the program encouraged more children to go to school, the surge in attendance created a lack of space in many schools and put further pressure on scarce educational resources.

IMPAQ International, LLC Page 51 Beoog Biiga Evaluation Report

CHAPTER 6. CONCLUSIONS AND RECOMMENDATIONS

6.1 Key Findings

The collected evidence showed that the Beoog Biiga program reached many of its goals. The M&E data revealed that total school enrollment increased during the life of the program in the two provinces, both for boys and girls. Girls’ enrollment was slightly lower than that of boys, but their proportion within the student body grew slightly. Pilot schools had on average more students in comparison with program schools. Both student enrollment and attendance increased in pilot schools in comparison with program schools.

The Beoog Biiga program seemed successful in increasing students’ knowledge of nutrition, vitamin A and iron. Student behaviors also showed that their knowledge was put into practice – more students washed their hands with water or with soap and water after using the latrines or before eating. Students also participated more actively in school clubs. Finally, students received mentor support at surprisingly high rates, both at pilot and program schools.

Evidence from the directors suggested that the program was successful in building their capacity. There was a substantial gain as a result in directors’ knowledge of education quality over time. Many directors were engaged in setting up and promoting student clubs at the schools. The most common were hygiene clubs, sanitation clubs, and health clubs, clearly related to the types of workshops that CRS provided.

Teachers attended even fewer training modules than the directors. They, too, often changed schools, which challenged the efficacy of the professional development. However, even with partial participation in training, teachers’ knowledge of education quality, nutrition and hygiene improved. While we did not observe their behaviors, their own self-reported hand washing practices and that of their students also improved.

Where the program seemed to have made most impact was at the PTA level. The evidence demonstrates that the program empowered them to be actively involved in running school canteens. After the program, school canteens operated longer during the school year and the local community and parents were increasingly supporting the canteens by providing wood, ensuring stocking and storage of food, and paying the canteen cooks. PTA members maintained money management and food management books in very good standing. Most of them established an action plan and were implementing real projects in their communities.

Households from the nearby communities were also responsive to Beoog Biiga. After the program, households knew more about the benefits of educating girls and cited additional reasons for hand

IMPAQ International, LLC Page 52 Beoog Biiga Evaluation Report washing and for using latrines. More households used latrines. Those that had latrines at their disposal kept them cleaner than they were three years earlier. Still, we did not observe improvements in hand-washing practices. Knowledge of REFLECT spread even to communities where the program was not offered. People learned about it, some were trained, and many more were involved in various stages of their community projects. However, few were able to complete their projects.

6.2 Recommendations

During the field interviews, respondents were asked if they had any suggestions for improving Beoog Biiga based on their experience with the program. We present below their key recommendations. What recommendations from the program implementation Increase the use of technology in data collection. can be drawn for future Project stakeholders suggested that school staff use sustainability? tablets to fill out the commodities reports and micronutrient registers to speed the transfer of information and reduce issues with discrepancies in the number of student recipients. Stakeholders also suggested counting student recipients more often, at least twice a year in November and again in December–January.

Ensure the functionality of water and sanitation infrastructures. Several project stakeholders and beneficiaries mentioned the importance of combining school feeding programs with activities to ensure adequate and functionning water and sanitation infrastructure (wells and latrines). Some schools explained that the program had not supported them with ensuring proper access to water, as their wells had run dry. Successful water infrastructure through pumps and wells can also provide an important source of financing for school. For example, one school that had been equipped with a well had started gardens and a tree nursery to support the school’s needs.

Increase teacher training. Many stakeholders and beneficiaries mentioned the importance of increasing the number of teacher trainings per year, especially because of the high teacher turnover. Teachers receive at most two trainings per year. According to them, teachers are the ones at the forefront of program implementation and hence teacher trainings are key to the success of program activities.

Implement SILC first. Project stakeholders emphasized the benefit of having implemented SILC before the other program activities, as SILC helped engage and organize communities by helping them identify their needs, problems and potential solutions. Stakeholders also emphasized the

IMPAQ International, LLC Page 53 Beoog Biiga Evaluation Report importance of engaging communities early on in the program and suggested SILC as an effective mechanism for achieving strong community engagement.

Redirect surplus food. Project stakeholders suggested that CRS develop a system to redistribute food from schools with surplus food to others suffering from food shortages. Alternatively, stakeholders also suggested that schools that experience a food surplus redistribute the food to girls as take-home meals.

Continue strong and flexible collaboration with MOE. Project stakeholders highly praised CRS’ participative and collaborative approach to implementing the program. MOE staff explained that CRS had made great efforts to integrate the aspirations of program beneficiaries and engage the MOE staff in the planning and budgeting of the program. The MOE staff also commended the strong communication between them and CRS, including the periodic meetings to discuss program progress, challenges and necessary mid-course adjustments.

Furthermore, based on our field observations and our data analysis, we present below two additional recommendations

Focus program efforts on beneficiaries living in local communities. Teachers and directors often change schools, preventing them from fully integrating themselves into the local community. Most teachers and directors come from distant villages and regions, and when they get assigned to a school by the education authorities, they are offered housing on the school premises. Many aspire to change schools to find better conditions or to be closer to their own home villages. However, local stakeholders such as female mentors, PTA members and lunch cooks are adults from nearby villages who are invested in their own community. Focusing program efforts on them to strengthen their capacity may require less refresher courses as they are far less mobile than teachers and directors and help sustain program efforts.

Establish and maintain systematic school-level and student-level data. To keep track of students, teachers and directors, and to facilitate construction of indicators, we recommend building a database with unique school and individual identifiers. The identifiers are essential for merging data from other sources such as commodity logs, training session logs, administrative logs and surveys. Furthermore, collecting the same variables over time will help make comparisons consistent and enable additional analysis to inform on the program’s progress.

IMPAQ International, LLC Page 54 Beoog Biiga Evaluation Report

APPENDIX 1. MAP OF SAMPLE INTERVENTION VILLAGES

IMPAQ International, LLC Page 55 Beoog Biiga Evaluation Report APPENDIX 2: LIST OF VILLAGES AND NUMBER OF RESPONDENTS BY CATEGORY

Provin Circumscri Scho Direct PT Teach Stud Househ ce Department ption Village ols ors As ers ents olds BOURZANG Bam BOURZANGA A ABRA 2 2 2 3 8 4 BOURZANG Bam BOURZANGA A BOURZANGA 0 0 0 0 0 4 BOURZANG Bam BOURZANGA A KIEKA-FULBE 1 1 1 2 4 4 BOURZANG Bam BOURZANGA A MALEOUALE* 1 1 1 2 4 4 BOURZANG Bam BOURZANGA A ZANA* 1 1 1 2 2 4 BOURZANG Bam BOURZANGA A ZANAMOGO 1 1 1 2 4 4 Bam GUIBARE GUIBARE GUIBARE 2 2 2 4 10 4 Bam GUIBARE GUIBARE KOUNDOULA* 1 1 0 3 6 8 Bam GUIBARE GUIBARE SAKOUDI* 1 1 1 2 4 4 KONGOUSSI Bam KONGOUSSI II BADINOGO 1 0 0 0 0 0 4 KONGOUSSI Bam KONGOUSSI II BADINOGO 2 1 1 1 2 4 0 BAM Bam KONGOUSSI KONGOUSSI VILLAGE* 1 1 1 3 4 4 KONGOUSSI Bam KONGOUSSI II BOGONAM* 1 1 1 2 4 4 KONGOUSSI Bam KONGOUSSI II LOAGHA* 1 1 1 3 4 4 KONGOUSSI Bam KONGOUSSI II LOURGOU 1 1 1 3 4 4 KONGOUSSI Bam KONGOUSSI II SANDOURE 1 1 1 1 4 4 KONGOUSSI Bam KONGOUSSI II SARGO 1 1 1 3 4 4 BOURZANG Bam KONGOUSSI A SECTEUR 5 1 1 1 2 4 0 KONGOUSSI Bam KONGOUSSI II WOUSSE* 1 1 1 2 4 4 KONGOUSSI Bam KONGOUSSI II ZINGGUIMA 1 1 1 2 4 4 Bam NASSERE NASSERE NASSERE 1 0 0 3 4 0 ZIMTANGA TAMIGA- Bam NASSERE 1 MOSSI 1 1 1 3 4 4 ZIMTANGA Bam NASSERE 1 TORA* 1 1 1 3 4 4 Bam ROLLO ROLLO BARKANA* 1 1 1 3 4 4 Bam ROLLO ROLLO IGONDEGA 1 1 1 2 2 4 Bam ROLLO ROLLO POGORO* 1 1 1 3 4 4 Bam ROLLO ROLLO ROLLO 0 0 0 0 0 1 Bam ROLLO ROLLO SECTEUR 2 1 1 1 2 4 3 Bam ROUKO ROUKO RAKA 1 1 1 2 4 4 Bam ROUKO ROUKO RILGO* 1 1 1 2 4 4 Bam ROUKO ROUKO YAMANE 1 1 1 2 4 4

IMPAQ International, LLC Page 56 Beoog Biiga Evaluation Report Provin Circumscri Scho Direct PT Teach Stud Househ ce Department ption Village ols ors As ers ents olds Bam ROUKO ROUKO YAOGHIN* 1 1 1 2 4 4 Bam SABCE SABCE IMIOUGOU* 1 1 1 2 4 8 Bam SABCE SABCE KOMBEDEGO 1 1 1 1 2 4 Bam SABCE SABCE LEFOURBA* 1 1 1 2 4 4 Bam SABCE SABCE LOUNGO* 1 1 1 2 4 4 Bam SABCE SABCE SABCE 1 1 1 3 4 4 Bam SABCE SABCE WINTINI* 1 1 1 2 4 4 Bam TIKARE TIKARE ANSOURI 1 1 1 2 4 4 Bam TIKARE TIKARE HAMDALAYE 1 1 1 2 4 4 Bam TIKARE TIKARE OUI* 1 1 1 2 4 4 Bam TIKARE TIKARE RITIMYINGA* 1 1 1 3 6 4 Bam TIKARE TIKARE SANCE 1 1 1 2 4 4 Bam TIKARE TIKARE TIKARE 1 1 1 1 4 4 Bam TIKARE TIKARE ZANO 1 1 1 1 2 4 ZIMTANGA Bam ZIMTENGA 2 MINIMA* 1 1 1 2 4 4 ZIMTANGA Bam ZIMTENGA 2 ZIMTENGA 2 2 1 4 8 8 Sanmate BOUSSOUMA nga SANMATENG BOUSSOUM A A S2 NASSERE 0 1 1 0 0 4 Sanmate nga KAYA KAYA 2 RASLA* 0 1 1 0 0 4 Sanmate nga MANE MANE SAORZI 0 1 1 0 0 4 Sanmate NAMISSIGUI nga MA DABLO BASKONDO 0 1 1 0 0 4 Sanmate nga PIBAORE PIBAORE OUALA 0 1 1 0 0 4 Sanmate BARSALOGH BARSALOG nga O HO BARSALOGHO 3 3 3 7 12 8 Sanmate BARSALOGH BARSALOG nga O HO DARKOA* 1 1 1 2 4 4 Sanmate BARSALOGH BARSALOG nga O HO FOUBE 1 1 1 3 4 4 Sanmate BARSALOGH BARSALOG nga O HO GABOU* 1 1 1 3 4 4 Sanmate BARSALOGH BARSALOG nga O HO KONDIBITO* 1 1 1 1 4 4 Sanmate BARSALOGH BARSALOG nga O HO NAGRAOGO* 1 1 1 2 4 4 Sanmate BARSALOGH BARSALOG nga O HO SOUDOUGOU 1 1 1 3 4 3 Sanmate BARSALOGH BARSALOG nga O HO YIMBOULSA* 1 1 1 3 4 4 Sanmate BARSALOGH BARSALOG nga O HO YIROUGOU 1 1 1 2 4 4 BOUSSOUMA Sanmate SANMATENG BOUSSOUM nga A A/S1 BIGUISSI 1 1 1 3 4 4 BOUSSOUMA Sanmate SANMATENG BOUSSOUM nga A A S2 BOULSIN* 1 1 1 2 4 4

IMPAQ International, LLC Page 57 Beoog Biiga Evaluation Report Provin Circumscri Scho Direct PT Teach Stud Househ ce Department ption Village ols ors As ers ents olds BOUSSOUMA Sanmate SANMATENG BOUSSOUM nga A A S2 FATIN 1 1 1 2 4 4 BOUSSOUMA Sanmate SANMATENG BOUSSOUM OUGA- nga A A/S1 YARCE* 1 1 1 3 4 4 BOUSSOUMA Sanmate SANMATENG BOUSSOUM nga A A S2 PANA* 1 1 1 2 4 4 BOUSSOUMA Sanmate SANMATENG BOUSSOUM nga A A S2 SERA* 1 1 1 3 4 4 BOUSSOUMA Sanmate SANMATENG BOUSSOUM nga A A/S1 SIRGUI* 2 2 2 4 4 8 BOUSSOUMA Sanmate SANMATENG BOUSSOUM nga A A S2 SOAGA 1 1 1 3 4 4 Sanmate nga DABLO DABLO BAWENNE* 1 1 1 1 4 4 Sanmate nga DABLO DABLO DAKE 1 1 0 3 0 0 Sanmate nga DABLO DABLO KOUGPELA 1 1 1 3 4 4 Sanmate nga DABLO DABLO LOADA* 1 1 1 0 4 4 Sanmate nga KAYA KAYA 2 AROUEM 0 0 0 0 0 4 Sanmate nga KAYA KAYA 2 BANGASSE* 1 1 1 2 4 4 Sanmate nga KAYA KAYA 2 BASNERE* 1 1 1 3 4 4 Sanmate nga KAYA KAYA 2 DELGA 1 1 1 3 4 4 Sanmate nga KAYA KAYA 2 GOUNGUIN 1 1 1 0 4 4 Sanmate nga KAYA KAYA 2 KAYA VILLE 1 0 0 2 4 0 Sanmate nga KAYA KAYA 2 KONEAN* 1 1 1 2 4 4 Sanmate nga KAYA KAYA 2 NAPALGUE 1 1 1 3 4 0 Sanmate nga KAYA KAYA 2 SILMIOUGOU 1 1 0 3 4 4 Sanmate nga KAYA KAYA 2 ZABLO 1 1 1 2 4 4 Sanmate KORSIMOR nga KORSIMORO O FOULLA* 1 1 1 3 4 4 Sanmate KORSIMOR IMIOUGOU- nga KORSIMORO O NATENGA* 1 1 1 3 4 0 Sanmate KORSIMOR nga KORSIMORO O KORSIMORO 2 2 2 4 8 7 Sanmate KORSIMOR nga KORSIMORO O OUEDSE* 1 1 1 3 4 4 Sanmate KORSIMOR nga KORSIMORO O OUIDIN 1 1 1 1 2 4 Sanmate KORSIMOR OUITENGA- nga KORSIMORO O POECIN* 1 1 1 3 4 4

IMPAQ International, LLC Page 58 Beoog Biiga Evaluation Report Provin Circumscri Scho Direct PT Teach Stud Househ ce Department ption Village ols ors As ers ents olds Sanmate KORSIMOR nga KORSIMORO O TAMBOKO 1 1 1 2 4 4 Sanmate KORSIMOR nga KORSIMORO O ZAIBTENGA 1 1 1 2 4 4 Sanmate nga MANE MANE KONLEBSE 1 1 1 1 4 4 Sanmate nga MANE MANE MANE-MOSSI* 1 1 1 0 2 4 Sanmate nga MANE MANE TANLALLE* 1 1 1 1 4 4 Sanmate nga MANE MANE YABO* 1 1 1 0 4 4 Sanmate nga PENSA PENSA BADNONGO 1 1 1 2 4 4 Sanmate nga PENSA PENSA DORO* 1 1 1 2 4 4 Sanmate nga PENSA PENSA NAHI-PEULH 1 1 1 2 4 4 Sanmate OUALOGOTEN nga PIBAORE PIBAORE GA* 1 1 1 2 4 4 Sanmate nga PIBAORE PIBAORE PEOTENGA* 1 1 1 2 4 4 Sanmate nga PIBAORE PIBAORE PIBAORE 1 0 0 2 4 0 Sanmate TANYOKO- nga PIBAORE PIBAORE PEULH 1 1 1 2 4 4 Sanmate nga PISSILA PISSILA DIASSA* 1 1 1 2 4 4 Sanmate nga PISSILA PISSILA DIBILOU 1 1 1 2 4 4 Sanmate KIEMNA- nga PISSILA PISSILA YARCE* 1 1 1 2 4 4 Sanmate nga PISSILA PISSILA KOALMA* 1 1 1 3 4 4 Sanmate nga PISSILA PISSILA OUANOBIAN 1 1 1 2 4 4 Sanmate SANBIN- nga PISSILA PISSILA NABITENGA 1 1 1 2 4 4 Sanmate nga PISSILA PISSILA TOUROUM 1 1 1 2 4 4 Sanmate nga ZIGA ZIGA GUIBTENGA 1 1 1 2 4 4 Sanmate nga ZIGA ZIGA KINKIRGO* 1 1 1 2 4 4 Sanmate nga ZIGA ZIGA PISSIGA 1 1 1 2 4 4 TOTAL 110 112 108 234 426 434

IMPAQ International, LLC Page 59 Beoog Biiga Evaluation Report APPENDIX 3: ATTENDANCE RATES WITH AND WITHOUT CANTEEN

Appendix 3.1: Average Attendance Rates with and without Canteen

Program Schools Pilot Schools Total Year 1 Year 2 Year 3 Year 1 Year 2 Year 3 Year 1 Year 2 Year 3 Total 93% 94% 97% 93% 93% 97% 93% 94% 97% With Canteen 97% 98% 98% 98% 98% 99% 98% 98% 98% Without Canteen 87% 84% 93% 86% 81% 93% 87% 83% 93%

Appendix 3.2: Attendance Rates with Canteen

Year 1 Year 2 Year 3

100 100 100

90 90 90

80 80 80

70 70 70

60 60 60

50 50 50 Nov Dec Jan Feb Mar Apr May Jun Nov Dec Jan Feb Mar Apr May Jun Nov Dec Jan Feb Mar Apr May Jun Program Pilot

100 100 100

90 90 90

80 80 80

70 70 70

60 60 60

50 50 50 Nov Dec Jan Feb Mar Apr May Jun Nov Dec Jan Feb Mar Apr May Jun Nov Dec Jan Feb Mar Apr May Jun Boys Girls

IMPAQ International, LLC Page 60 Beoog Biiga Evaluation Report Appendix 3.3: Attendance Rates without Canteen

Year 1 Year 2 Year 3

100 100 100

90 90 90

80 80 80

70 70 70

60 60 60

50 50 50 Nov Dec Jan Feb Mar Apr May Jun Nov Dec Jan Feb Mar Apr May Jun Nov Dec Jan Feb Mar Apr May Jun Program Pilot

100 100 100

90 90 90

80 80 80

70 70 70

60 60 60

50 50 50 Nov Dec Jan Feb Mar Apr May Jun Nov Dec Jan Feb Mar Apr May Jun Nov Dec Jan Feb Mar Apr May Jun Boys Girls

IMPAQ International, LLC Page 61 Beoog Biiga Evaluation Report REFERENCES

Adelman, S., Alderman, H., Gilligan, D.O. and Lehrer, K. (2008). The Impact of Alternative Food for Education Programs on Learning Achievement and Cognitive Development in Northern Uganda [Unpublished manuscript]. Washington, DC: International Food Policy Research Institute.

Adelman, S., Gilligan, D.O. and Lehrer, K. (2007). How Effective are Food for Education Programs? A Critical Assessment of the Evidence from Developing Countries. Washington, DC: International Food Policy Research Institute.

Ahmed, A.U. (2004). Impact of feeding children in school: Evidence from Bangladesh. Washington, DC: International Food Policy Research Institute.

Alderman, H., Gilligan, D.O. and Lehrer, K. (2012). The Impact of Food for Education Programs on School Participation in Northern Uganda. Economic Development and Cultural Change, 61(1), 187–218.

Catholic Relief Services (June 2012). Baseline Survey Report. Ouagadougou, Burkina Faso.

Catholic Relief Services (2013). Private Service Provider Implementation Manual. Baltimore, MD.

Catholic Relief Services (May 2012). Project Status Report, October 2011–March 2012. Ouagadougou, Burkina Faso.

Catholic Relief Services (May 2013). Project Status Report, October 2012–March 2013. Ouagadougou, Burkina Faso.

Catholic Relief Services (May 2014). Project Status Report, October 2013–March 2014. Ouagadougou, Burkina Faso.

Catholic Relief Services (November 2012). Project Status Report, April–September 2012. Ouagadougou, Burkina Faso.

Catholic Relief Services (November 2013). Project Status Report, April–September 2013. Ouagadougou, Burkina Faso

Catholic Relief Services (November 2014). Project Status Report, April–September 2014. Ouagadougou, Burkina Faso.

IMPAQ International, LLC Page 62 Beoog Biiga Evaluation Report Finan, T.J., Rashid, A., Woel, B., Arunga, D., Ocholoa, S., Rutere, S. and Muindi, M. (2010). Impact Evaluation of WFP School Feeding Programmes in Kenya (1999–2008): A Mixed-Methods Approach. Washington, DC: International Food Policy Research Institute.

Jacoby, H.G (2002). Is There an Intrahousehold ‘Flypaper Effect’? Evidence from a School Feeding Programme. The Economic Journal, 112 (January), 196–221.

Kazianga, H., de Walque, D. and Alderman, H. (2009). Educational and Health Impacts of Two School Feeding Schemes: Evidence from a Randomized Trial in Rural Burkina Faso [Policy Research Working Paper 4976]. http://dx.doi.org/10.1596/1813-9450-4976

Kazianga, H., de Walque, D. and Alderman, H. (2012). Educational and Child Labour Impacts of Two Food-for-Education Schemes: Evidence from a Randomised Trial in Rural Burkina Faso. Journal of African Economies, 21(5), 723–760. doi: 10.1093/jae/ejs010

Kazianga, H., de Walque, D. and Alderman, H. (2014). School feeding programs, intrahousehold allocation and the nutrition of siblings: Evidence from a randomized trial in rural Burkina Faso. Journal of Development Economics, 106(2014), 15-24. doi:10.1016/j.jdeveco.2013.08.007

Singh, A., Park, A. and Dercon, S. (2014). School Meals as a Safety Net: An Evaluation of the Midday Meal Scheme in India. Economic Development and Cultural Change, 62(2), 275–306.

Sulemana, M., Ngah, I. and Majid, M.R. (2013). The challenges and prospects of the school feeding programme in Northern Ghana. Development in Practice, 23(3), 422–432. http://dx.doi.org/10.1080/09614524.2013.781127

United Nationals Development Programme (UNDP) (2009). Human Development Report 2009, Overcoming barriers: Human mobility and development. New York, NY.

United Nations Children’s Fund (UNICEF) (2008). The State of the World’s Children 2009. New York, NY.

IMPAQ International, LLC Page 63 Beoog Biiga Evaluation Report