Baseline Study Report

MESA II Project - Better Education and Health Agreement: FFE-524-2017/025-00 Final Evaluation Report Coordinated by Project Concern International (PCI) Nicaragua August/Sept. 2017

Submitted to USDA/FAS Project “Mejor Educación y Salud

(MESA)” - Nicaragua Agreement: FFE-524-2013-042-00

Submitted to: USDA/FAS

Vanessa Castro, José Ramón Laguna, Patricia Callejas

with collaboration from Micaela Gómez

Managua, December 2017

June 4, 2019 Managua, Nicaragua

Acknowledgements

The consultant team appreciates PCI Nicaragua for entrusting Asociación Nicaragua Lee with the completion of this study. In particular, we would like to acknowledge the valuable support provided by María Ángeles Argüello and María Zepeda at PCI Nicaragua-, and by officials from the Ministry of Education (MINED) in Managua and in the departmental delegations of Jinotega and the Southern Caribbean Coast Autonomous Region (RACCS). We also recognize the support given by the officials at the MINED offices in the 11 municipalities participating in the study: Jinotega, La Concordia, San Sebastian de Yali, Santa Maria de Pantasma, Bluefields, Kukra Hill, La Cruz del Río Grande, Laguna de Perlas, Desembocadura Río Grande, El Tortuguero and Corn Island. In particular, we would like to acknowledge the enthusiasm showed by the educational advisors from the aforementioned MINED municipal offices, in the administration of the instruments

Our greatest gratitude and consideration to the actors of this study, the fourth-grade students from the elementary schools included in the sample, who agreed and participated with great enthusiasm. We would also like to thank the third-grade teachers who contributed by answering the questionnaire.

We should also mention and thank the team of supervisors, applicators and data entry personnel, who put much dedication and effort into the collection and processing of the Early Grade Reading Assessment (EGRA) instruments, the questionnaires, and the school and classroom environment observation sheet.

Credits

This study was conducted by Asociación Nicaragua Lee for Project Concern International (PCI) Nicaragua for the beginning of the MESA II Project

PCI Nicaragua authorities:

Carlos Leonel Argüello Director of PCI – Nicaragua María Ángeles Argüello Project Director María del Socorro Zepeda Director of Monitoring and Evaluation

Government partner:

Nora Cuadra Baquedano General Director of Educational Planning and Programming, MINED Natalia Sequeira Moreno Director of the Comprehensive School Nutrition Program of MINED

The findings, opinions, interpretations, and conclusions expressed in this publication are those of the authors and do not necessarily reflect the views of PCI Nicaragua or its local partners.

ii Table of Contents

Executive Summary ...... 1 I. Context and Justification...... 4 II. Conceptual Framework ...... 5 III. Methodology ...... 8 1. Preparatory Stage of the Study ...... 8 2. Limitations to the Study ...... 8 3. Sampling ...... 9 4. Data collection workshop ...... 11 5. Instruments ...... 11 6. Fieldwork - Data collection ...... 13 7. Factor Associated to Learning ...... 15 IV. Results ...... 15 1. Literacy Results ...... 15 a. Oral Comprehension (section 1) ...... 18 b. Beginning Sound Identification (Section 2, Part A) ...... 18 c. Word Segmentation (Section 2, Part B) ...... 18 d. Letter Recognition (Letter Naming) (section 3) ...... 19 e. Reading Simple Words (Section 4) ...... 20 f. Reading Pseudo-words (Section 5) ...... 21 g. Reading Fluency (Section 6, Part A) ...... 22 h. Reading Comprehension (Section 6, part B) ...... 23 i. Dictation (Section 7) ...... 24 2. Analysis of Factors Associated with Learning ...... 25 3. Teacher’s Abilities ...... 29 4. Food Security Results ...... 29 5. Hygiene Results ...... 30 V. Conclusions and Recommendations ...... 31 Recommendations ...... 31 VI. Bibliography ...... 33 Appendix 1 – EGRA...... 34 Appendix 2 – Student Survey ...... 43 Appendix 3: Teacher Questionnaire ...... 47 Appendix 4: Learning Environment Observation Guide ...... 52 Appendix 5 – Global Indicators Collected in Baseline Study ...... 54

iii List of Tables

Table 1. IDEL Standards for Third grade ...... 6 Table 2a. Study Sample Calculation ...... 9 Table 2b. Compliance with Sample ...... 10 Table 3. Alfa Cronbach Test –EGRA All sections ...... 13 Table 4. Total Number of Instruments Administered in Each Municipality ...... 14 Table 5. Routes and Team of Applicators for Fieldwork – February to March 2019 ...... 14 Table 6a. Number and Percentage of Cases of “Stopped Exercise” ...... 15 Table 6b. General Results EGRA Test Administration by gender ...... 16 Table 6c. Statistically Significance Tests ...... 16 Table 6d. EGRA Results by Municipality ...... 17 Table 7. Oral Comprehension by gender ...... 18 Table 8. Beginning Sound Identification ...... 18 Table 9a. Word Segmentation by gender ...... 19 Table 9b. Sounds Showing Greater Error Rates in Words Segmentation ...... 19 Table 10a. Letters Correct Per Minute ...... 19 Table 10b. Level of Difficulty in Letter Recognition ...... 20 Table 11a. Simple Words Read Correctly Per Minute ...... 20 Table11b. Level of Difficulty in Reading Common Words ...... 21 Table 12a. Pseudo Words Read Correct Per Minute ...... 21 Table 12b. Level of Difficulty in Reading Pseudo Words...... 22 Table 13. Average Reading Fluency by Gender ...... 22 Table 14. Questions Answered Correctly ...... 23 Table 15a. Average Dictation Results by Gender ...... 24 Table 15b. Percentage of Correct Items in Dictation ...... 24 Table 16. Variables affecting learning ...... 27

List of Graphs

Graph 1. Model used to identify factors associated to learning ...... 7 Graph 2. Percentage of Students Reaching IDEL Standard ...... 23

iv Abbreviations

ANL Asociación Nicaragua Lee (Nicaragua Reads Association) CCV Consonant-consonant-vowel CCCV Consonant-consonant-consonant-vowel CEDEHCA Center for Autonomous, Citizen and Human Rights EGRA Early Grade Reading Assessment FAS Phonic-Analytic-Synthetic Reading Method FFE Food for Education IDEL Dynamic Indicators of Reading Success IR Intermediate Results MESA Better Education and Health Project MINED Ministry of Education PCI Project Concern International PINE Comprehensive School Nutrition Program RACCS Southern Caribbean Coast Autonomous Region RTI Research Triangle Institute International SO Strategic Objective USDA United State Department of Agriculture WPM Words read correctly per minute

v Executive Summary

Learning to read and write is a complex process that must be well organized. It requires properly trained teachers, access to textbooks and teaching materials, and an efficient use of instruction time. The task of teaching literacy is even more complex when students come from low-schooling households, when the language spoken at home is different from the one used in school, or when they have no access to preschool or early childhood education. For those students, an early and high-quality reading and writing education is of paramount importance, and those who do not receive it tend to drop-out of the educational system. In Nicaragua, the average percentage of students graduating from sixth grade is approximately 60%, (Preal Eduquemos 2007) the lowest percentage in Central America, and among those who come from families in poverty the percentage drops to 21% (Flórez C. A., IEPP, 2012), so these children as a mean tend to have less than three years of primary education (ibid). Meanwhile the “no- poor” children had almost seven years of education as a mean (ibid). Therefore, providing opportunities for children at risk of academic failure is essential to prevent education from further aggravating social inequality in Nicaragua, where the Gini index is higher than 0.50.

Due to reasons external to PCI, one month after the study began, a national socio-political crisis broke out resulting in a delay of almost a year. As a consequence of this longstanding interruption, and the lack of authorization from MINED to continue with the study as it was initially designed, the TOR was revised and a considerable reduction in the scope of the study was made. Despite the difficulties, data was collected, and the study was concluded in May 2019.

Literacy An assessment of fourth grade students’ reading skills was conducted using Early Grade Reading Assessment (EGRA) test, which involves the administration of two questionnaires, (one to students and another to teachers) as well as an observation guide with a checklist to assess the learning environment of the third grade classrooms. The study included a robust sample of 872 students and 52 teachers from the 39 selected schools located in the municipalities of Jinotega, La Concordia, San Sebastian de Yalí, Santa María de Pantasma, Bluefields, Kukra Hill, La Cruz del Río Grande, Laguna de Perlas, Desembocadura del Río Grande, El Tortuguero and Corn Island. Questions about factors related to learning that could help explain student’s performance on the test were included in the questionnaires.

The analysis of the data found that 19.8% of boys and 27.9% of girls read at a 3rd grade level, for 24.1% of the overall study population. A more in depth analysis of the factors that lead to literacy levels focused on analyzing words per minute (WPM) as the outcome of interest, found the following results: • Students who attended preschool read 6.0 correct WPM more than those who did not. • Girls read 6.5 WPM more than boys. • Knowledge and implementation of personal hygiene practices were positively related to students reading abilities. Students with knowledge of personal hygiene practices read about 14.4 WPM more than those without that knowledge. • Students in rural areas read 7.4 WPM less than their peers in urban areas. • Discrepancies between the teacher’s mother tongue and the language she/he uses to deliver instruction, as well as the student’s mother tongue and the language in which instruction is

delivered in school, were negatively related to learning (-11.3 WPM for teachers whose native language is not Spanish). • Teachers who studied another career or took courses in any field at the time of the study played a positive role in student learning (7.2 WPM). • Students learning in cleaner schools and classrooms, classrooms with a literacy-rich environment, and classrooms in which students made greater use of books had higher performance scores. However, these relationships were not statistically significant.

In terms of the capacity of educators to provide literacy instruction, the baseline measured the percentage of teachers in target schools who demonstrate improvement in literacy teaching as identified by their supervisors1 (MGD 1.1). The baseline found that 81.8% of male teachers (9 of 11) and 31.7% of female teachers (13 of 41) met this indicator definition, for 42.3% of the overall study population (22 of 52).

Food Security The baseline measured the percentage of students in target schools who indicate to be “hungry” or “very hungry” during the school day even after eating in school (MGD 1.2.1). The baseline found that 9% of male students (38 of 409) and 5% of female students (23 of 463) met this indicator definition, for 7% of the overall study population (61 of 872) being hungry or very hungry.

Hygiene In terms of hygiene, the baseline measured the percentage of schools whose students wash their hands before eating (SO 2), and the percentage of students who can identify at least three good hygiene practices (MGD 2.1). The baseline found that students in 95% of schools (37 of 39) wash their hands before eating but in looking at individual students only 68% confirmed to do so, and 44% of male students and 46% of female students were able to identify at least three good hygiene practices, for a total of 45% of the overall study population (393 of 872).

Challenges and Recommendations The study found the following key challenges that MESA II will address to improve literacy skills in the schools the project will benefit: • Strong difficulties were found in sections related to phonological awareness (the ability to discriminate the sounds and syllables that make up a word), a required skill for decoding. • Students demonstrated difficulties in decoding (the ability to assign each grapheme or letter its corresponding sound to be able to read the word that contains those letters), which negatively affects their ability to read fluently and understand material. • Children showed problems in pseudo-word reading and dictation sections. Almost 70% of girls and 80% of boys sampled - cannot read according to the standards set by the Dynamic Indicators of Reading Success (IDEL) for the third grade (> = 85 WPM). This is evidence of the decoding difficulties presented by students. The difficulties found in reading were mainly in the rural and multi-grade schools which present a strong challenge for the project to overcome. – Therefore, the project strategies must be carefully analyzed.

1 In the questionnaire teachers were asked about training received in the last four years on strategies to evaluate reading, the FAS method, and reading comprehension. They were also asked if they put training into practice in the classroom

Recommendations emerging from the study are the following:

1. Identify what the schools in San Sebastián de Yalí in Jinotega are doing differently from other schools, as students in these schools - outperformed their peers in all indicators. These students read an average 82.7 WPM surpassing their peers by 30.1 WPM, a statistically significant difference at the 99% confidence level. 2. Understand what is being done in La Cruz del Río Grande, a municipality of RACCS, where students read 74.8 WPM (close to the IDEL goal for 3rd grade) and 22.6 WPM faster than their peers from Laguna de Perlas2. These differences are statistically significant at the 99% confidence level. 3. Determine which new knowledge, methodologies or learning teachers have acquired whose practices have better results than those of their colleagues. Two qualitative case studies and instruction follow-up are proposed to do this. Classroom observations could be done using the Stalling Observation Tool, which was designed to assess teachers’ use of time in the Language and Literature class. 4. Systematize information presented in this report as well as information emerging from the proposed studies to design a comprehensive professional development program for currently working teachers. This could be an “online” based program complemented with previously tested, ready-to-use lesson to be distributed among teachers with fewer professional development opportunities. Specifically, include modules on teaching phonological awareness, given that students showed many weaknesses in this area. 5. Include an effort to expand preschool coverage and/or a community child development service in the project, since attending preschool was associated with positive in learning outcomes. 6. Intensify and extend training in personal hygiene to the community, as it was proven to have positive effects on learning. Further research is advised. 7. Sponsor campaigns to increase the use of books where they already exist, replace them or distribute new ones where schools do not have any.

2 In the model, the omitted variable was Laguna de Perlas

I. Context and Justification

Project Concern International (PCI) implemented the Food for Education (FFE) Better Education and Health Project (MESA) in Nicaragua from September 27, 2013 to September 30, 2017. By the end of 2017, PCI received funds from the United State Department of Agriculture (USDA) to implement a new phase of the FFE project in Nicaragua, called MESA II. MESA II is a comprehensive project with the objective of improving literacy skills in school-age children with a focus on providing schools with food and techniques to improve hygiene and health. The project is being implemented in 11 municipalities of Nicaragua, four of which are in the department of Jinotega and seven of which are in RACCS: San Sebastián de Yalí, Santa María de Pantasma, La Concordia, Bluefields, Desembocadura, Tortuguero, Kukra Hill, Laguna de Perlas, La Cruz del Río Grande, and Corn Island. MESA II is scheduled to last five years, from September 26, 2017 to September 30, 2022, and will have the Ministry of Education (MINED), the Comprehensive School Nutrition Program (PINE), and the Center for Autonomous, Citizen and Human Rights (CEDEHCA) as partners.

MESA II directly supports USDA’s vision of an integrated program that combines both health and educational interventions. - The Strategic Objectives (SO) and Intermediate Results (IR) for MESA II are: Improved literacy of school-age children (MGD SO1), improved quality of literacy instruction (MGD 1.1), improved attentiveness of students in the classroom (MGD 1.2), improved student attendance (MGD 1.3) and increased use of health, nutrition, and dietary practices of the beneficiaries and their families (MGD SO2). Due to reasons external to PCI, this study will focus mostly on the SO1, with less of an emphasis on SO2. MESA II plans to cover a total of 1,115 schools (including 986 elementary schools) in the 11 municipalities, to directly benefit 101,830 students, parents, teachers, and government officials, and to indirectly benefit 310,000 members of the families of the main beneficiaries.

Project activities include: provision of school feeding, teacher training, distribution of school materials and supplies, distribution of books and other teaching and learning materials, construction and / or improvement of school infrastructure, establishment of school gardens and orchards, training for the parents-teachers association, training in food preparation and storage practices, rehabilitation of water systems, rehabilitation of latrines, distribution of deworming medication (in collaboration with the Ministries of Health and Education), training in good health and nutrition practices, awareness-raising on the importance of education, establishment of savings groups, and institutional strengthening.

To measure the benefits of the project, PCI hired the Asociación Nicaragua Lee (ANL), an independent organization specializing in educational research, to conduct a baseline study to answer the following research questions:

1) To what extent will the activities and key strategies of the project be effective in achieving the proposed results? 2) What are the main challenges MESA II faces to improve literacy in the schools benefited by the project? 3) To what extent does MESA II’s design duly include the needs of the beneficiary population?

4) To what extend are the educational strategies in the project designed to contribute to better learning inside and outside the school? 5) What lessons learned are available from the previous phase of the project to help improve the learning of school-age children in the current implementation?

The study design included the administration of questionnaires and guides with open-ended questions to government officials to obtain valuable information to answer the research questions. However, due to reasons external to PCI, one month after the study begun, a national socio- political crisis broke out resulting in a delay of almost a year. As a consequence of this longstanding interruption, and the lack of authorization from MINED to continue with the study as it was originally designed, TOR were revised and a considerable reduction in the scope of the study made. All the qualitative portion of the study was eliminated, and the instruments were significantly reduced which prevented the research questions 1, 3 and 4 to be properly answered. The lack of focus groups interviews for teachers, parents, and PINE officials and individual interviews with officials from governmental and non-governmental institutions that were scheduled have profoundly diminished the possibility of collecting insights on some topics of interest. An Addendum to the contract was written stating the scope of the study as follow:

a) Determine quantitative information that serves as reference for mid-term and end-of- project comparisons. b) Evaluate the knowledge and skills of students and teachers to perfect the activities for capacity building. c) Propose recommendations to adapt the educational strategies of the project that facilitates maximizing strengths and mitigating challenges.

This study will allow PCI to compare current reading skills and teacher capacities to their skills and capacities after the project implementation. The study will also provide recommendations about educational strategies to maximize strengths and mitigate challenges in the implementation of project.

II. Conceptual Framework

Learning to read and write is a complex process that must be well organized, guided by qualified teachers with access to textbooks and teaching materials, and with efficient use of instruction time. The task of teaching reading and writing is even more complex when students come from low- schooling households, when the language spoken by their parents is different from the one used in school, or where they have not had access to preschool or early childhood development. For these students, an early and high-quality education in reading and writing is of paramount importance to avoid dropout from the educational system. In Nicaragua, the percentage of students graduating from sixth grade is approximately 60% (Preal Eduquemos 2007; Flórez C. A., IEPP 2012), the lowest percentage in Central America, and among those who come from families in poverty, the percentage drops to 21% (Flórez C. A., IEPP, 2012). Therefore, providing opportunities for children at risk of failure is essential to prevent education from further aggravating social inequality in Nicaragua, where Gini index is higher than 0.50. In the first grade of elementary school, reading and writing lessons are an "initiation to literacy," where most students, particularly the vulnerable ones, are introduced to written language, promoting the development of unique skills necessary to continue learning. Reading with precision

5 and fluency is essential to understand and develop communication skills. Therefore, the school has an enormous responsibility with the children who enroll with an existing disadvantage due to socio-economic status, race, or lack of access to early childhood education or even worst, a combination of all those factors.

If literacy acquisition happens timely and appropriately a new future opens because reading comprehension allows students to work on mathematical problems, develop analytical and critical thinking, communicate properly, carry out scientific research, and have fun, among many other things (RTI 2009, Castro, Laguna, Mayorga, 2009 Ciases RTI).

Previous studies carried out in Nicaragua in schools with characteristics similar to those in this study have found that opportunities to learn to read depend almost exclusively on the school, as the majority of students come from homes in poverty. In these homes, children have little exposure to written word, as evidenced by the fact that 70% of the children of poor households do not have magazines, newspapers or books in their homes (Castro, V., Laguna, JR, Mayorga N., 2009 CIASES RTI). For this reason, efforts from the school community to assist these children in mastering the precursor skills of reading at an early stage are crucial.

Knowing this, it is key to assess the basic reading skills that lead to reading comprehension. Data from evaluation provides teachers with important information to improve their teaching strategies, also this information can also be used to promote parent´s interest and help them become committed to their child's learning. Evaluation allows learning problems to be detected in a timely manner, and if results are properly used, the data generated can be very useful in the selection of teaching-learning strategies that aim to solve the identified problems.

The standardized test known as EGRA allows for a quick assessment of the foundational skills that are essential in helping students learn to read in the first grades of elementary school. When measuring third graders´reading abilities the standards for establishing success are mainly focused in fluency because the other reading skills are considered foundational, therefore they should be already mastered by the end of third grade. Morever, IDEL –an institute which establishes reading standards for students-- only offer third grade standards for this specific ability: fluency, that is associated with the capacity to comprehend what is read and reflects the ability to properly decodify. The unit used to measure fluency is the number of words that a student can read correctly in one minute (WPM). Students are assessed individually as they read aloud for one minute from an unpracticed text. To evaluate reading fluency results, these results are compared with the international IDEL standards according to the school period in which the EGRA test is applied.

Table 1. IDEL Standards for Third grade Reading Fluency Beginning of Middle of School End-of-School Category School Year Year Year At Risk <50 WPM <60 WPM <65 WPM

Learning >=50 and <60 >=60 and <70 >=65 and <85 WPM WPM WPM Good >=60 WPM >=70 WPM >=85 WPM

The reading ability of the students can be explained by their individual characteristics, by family factors, by the teacher’s abilities, and by their social or economic context a factor external to the educational model. The following graph summarizes the conceptual model used to identify variables that might contribute to reading and writing outcomes; not all of them were studied in this baseline because a hindrance of this study was MINED´s decision to shorten the questionnaires eliminating important questions for students and teachers.

Graph 1. Model used to identify factors associated to learning

•Gender •Support / Help •Age •Motivation •Motivation •Health •Health

Personal Family

Social School

•Security •Value education •Director's role •Teacher's personal and •Stability teching characteristics •Poverty •School's facilities

This evaluation will provide PCI with a valuable baseline against which the efforts made to guarantee equal opportunities to children from households in poverty can be measured. At the same time, these results will allow PCI to know in which direction to train and support directors and teachers so that their efforts to improve reading are increasingly effective.

III. Methodology

1. Preparatory Stage of the Study

To obtain the necessary induction and feedback from PCI, seven preparatory coordination meetings were held for the organization of the study. - In these meetings, requirements, expectations, and plans were specified for the preparation of the instruments, the selection of the sample, the fieldwork, and the preparation of the report.

In 2018 the study began with a document review, through which monitoring, and evaluation tools previously used in MESA were examined along with the plans and methodology that will guide the implementation of MESA II. Once the instruments were designed, reviewed by PCI and validated, they were reviewed by the MINED and approved in 2019. The MINED prioritized the assessing of literacy, the enquire about factors associated with learning and variables which contribute to strengthen institutional technical capacity at the municipal level through the participation of pedagogical adviser personnel in the study. The MINED provided the final approval for the study.

2. Limitations to the Study

Most of the limitation affecting the study were related to the national socio-political crisis that started in April 2018, exactly a month after the beginning of the study. Some of the limitations the study faced were the following:

- The fieldwork was delayed, therefore the hiring and training process of personnel participating in study was interrupted. - The authorization to visit the schools selected to participate in the study was granted from MINED after a ten months wait. - The original questionnaires designed for the study had to be revised and modified by the MINED several times. Many essential questions were eliminated for example, questions related to health and absenteeism, and questions related to socio-economic status were eliminated in the student questionnaire. Data collected showed a relationship between health and learning, nonetheless, a deeper analysis was not possible due to the lack of information. - Inaccuracy between the student enrollment data gathered at the time of selecting the study’s sample and the number of actual student enrollment found at the schools. To maintain student representativity, once in the field, the number of schools was increased from 36 to 39 schools. - EGRA had to be administrated to students who attended third grade in 2018 but were in fourth grade in 2019. This fact limited the possibility to perform cross analysis between data collected from third grade teachers and student’s results in the EGRA. - There was no confirmation that the students in fourth grade who took EGRA in 2019 belonged to the third grade classrooms observed.

Also as a result of the delays and Government of Nicaragua involvement, PCI was not able to collect several of the indicators that were supposed to be collected during the baseline. PCI will collect the following indicators through the annual monitoring process.

- Percent of students in classrooms identified as inattentive by their teachers (MGD 1.2) - Percent of students in target schools who miss more than 10 school days per year due to illness (MGD 1.3.2) - Percent of parents in target communities who can name at least three benefits of primary education (MGD 1.3.5) - Percent of parents in targeted communities who can identify at least three important health/hygiene practices (MGD 2.1) - Percent of mothers or caretakers in targeted communities who can identify at least three important nutrition or dietary guidelines/recommendations (MGD 2.3)

3. Sampling

According to the initial scope of the study, the sample3 size was estimated with a confidence level of 95% and an allowable error of 4%, leading to a sample size of 1,158 students (573 EGRAs plus student questionnaire in the third grade and 585 student questionnaires in grades four to six). However, due to changes in the scope of the study, both instruments were given to fourth grade students4 only. Given that the number of schools remained constant, ANL increased the number of assessments to guarantee the necessary statistical robustness for the different comparisons, and a total of 872 EGRAs and student questionnaires were administrated to fourth grade students.

Two-stage cluster sampling was used for the sampling methodology. In the first stage the schools that participated in the study (36 schools) were randomly selected in proportion to the size of the clusters, and in the second stage, students were chosen. After the 10-month delay in starting fieldwork, all partners agreed to only assess students starting fourth grade in 2019, most of whom completed third grade during the 2018 school year.

In the first stage, the schools were selected proportionally to their size, so the list was ordered in ascending order according to the enrollment of each school. This allowed the study team to build an accumulated value and calculated amplitude value (total enrollment of the municipality divided by the number of schools to be selected), and a random number between one and the amplitude was generated. The following table shows this distribution:

Table 2a. Study Sample Calculation Frame Sample Selection Criteria Municipality Students Schools 3rd grade 4th to 6th School Amplitude Random Bluefields 6,022 191 115 117 7 860 148 Corn Island 883 9 5 6 1 883 105 Desembocadura 542 8 5 5 1 542 502 del Río Grande El Tortuguero 3,013 143 86 88 5 603 121 Jinotega 11,412 143 86 88 5 2,282 813

3 The finite population formula was used to calculate the sample the size 4 Because the baseline was to be applied at the beginning of the school year as a proxy for the learning acquired at the end of third grade.

Kukra Hill 1,372 58 35 36 2 686 374 La Concordia 679 26 16 16 1 679 217 La Cruz de Río Grande 3,233 153 92 94 5 647 385 Laguna de Perlas 1,555 64 38 39 3 518 502 San Sebastián de Yalí 2,716 60 36 37 2 1,358 231 Santa María de Pantasma 4,818 99 59 61 4 1,205 555 Total 36,245 954 573 587 36

The students from each school were selected using the following methodology:

• The fieldwork supervisor obtained the attendance or enrollment list of students in fourth grade. If the grade had 16 students or fewer students, all students were assessed. • If more than 16 students were enrolled in the grade, 16 were randomly selected using systematic random sampling based on the actual number of students in the class. The number of students in the class was divided by 16, and the result was used to systematically select the sample from the list of all students. • If the grade had several sections, students from all sections were added onto one list, and the same systematic random sampling process was carried out. • If the center had Matutina (morning) and Vespertina (afternoon) sessions, the list of all students enrolled in the whole grade was obtained, and the sample was randomly selected using the systematic random sampling strategy.

The final sample consisted of 39 schools, in which EGRA and the student questionnaire were administrated to 872 fourth grade students, and the teacher questionnaire was administered to 52 teachers who taught third grade in 2018. The data is detailed in the following table.

Table 2b. Compliance with Sample Instruments Applied % Application Municipality Students Schools Students Schools Bluefields 159 7 138% 100% Corn Island 31 1 620% 100% Desembocadura del Río Grande 31 1 620% 100% El Tortuguero 87 5 101% 100% Jinotega 195 5 227% 100% Kukra Hill 71 3 203% 150% La Concordia 28 2 175% 200% La Cruz de Río Grande 79 5 86% 100% Laguna de Perlas 75 3 197% 100% San Sebastián de Yalí 58 2 161% 100% Santa María de Pantasma 58 5 98% 125% Total 872 39 152% 108%

Table 2b shows an over-compliance of 108% for the total number of schools where the instruments were administered and an over-compliance of 152% at the student level, as 872 students in fourth

10 grade took the EGRA and the student questionnaire, higher than the initial sample of 573 students. Mixed results were found in reference to finding the expected number of students in the schools5; fewer students than expected were located in the municipality of La Cruz de Río Grande and more than expected were located in Desembocadura and Corn Island. Also, once in the field to guarantee the number of students tested, three complementary schools were chosen increasing the number of schools visited from 36 to 39 school. It should be noted that the application of the EGRA tests and student questionnaires only to fourth grade students made the sample statistically very robust and highly representative for that grade.

4. Data collection workshop

Training for the fieldwork staff was held as originally planned in 2018, but due to the delay caused by the socio-political crisis, a second training was held in 2019.

2018 From April 9 - 13, 2018, a training session was held in Managua for a team of 12 interviewers with the objective of training them on the purpose of the study, presentation, analysis, validation, and subsequently to practice the administration of the study instruments. The EGRA test designed and approved for this study, the questionnaires for students, teachers, and parents, and the observation guide, were all validated at “15 de Septiembre School” located in Managua.

2019 Once the MINED granted authorization to proceed with the study, two workshops were held: one in Jinotega and the other in RACCS. The purpose of the sessions was to train pedagogical advisors personnel from the 11 municipalities in the administration of the instruments and to be a refresher for the 12 interviewers trained in 2018. Workshops took place in Jinotega on February 18th, in Bluefields on February 18th and 19th, and in Corn Island on February 24th. The originally planned questionnaires were revised and reduced due to the modification of the scope in this study.

5. Instruments

Four different instruments were applied. In addition to the EGRA test, students answered a questionnaire composed of 26 questions. Teachers were given a questionnaire to determine their educational level, mother tongue, language they use when teaching, training received, and trainings’ usefulness for teaching. In addition, observations of schools and classrooms were made following a previously prepared and validated guide. (see Appendices 1-4). The description of the instruments follows below. a. EGRA is an easily administrated oral test. EGRA was developed by literacy specialists from the Research Triangle Institute International (RTI), and it assesses the basic skills that students are expected to acquire in the initial grades of primary education. Since 2007, EGRA has been applied internationally to assess reading abilities. EGRA measures skills that should be learned in the first three grades of elementary school. Therefore, students from second grade –as example-- are expected to show greater mastery of these basic competences than first graders. The EGRA used for this study was a version validated in Nicaragua in 2007, but because it was applied to fourth

5 Student enrollment data was not always accurate and absenteeism was high in some schools.

11 grade students, it was adjusted to raise its complexity. The adjustment process was carried out following IDEL guidelines. The EGRA used during this study --previously validated-- served as a tool to measure learning of five basic reading skills: phonological awareness, alphabetical principle (letter naming and decoding), reading fluency, reading comprehension, and oral comprehension.

The EGRA version used in the study has the following seven sections:

• The first section measures the student's oral comprehension through a simple story which was written exclusively for this baseline. The enumerator reads the story to the student, - according to established norms, and then the student answers five questions about the story. • The second section has two components for evaluating students' phonological awareness. In Part "A," the student is asked to discriminate the initial sound of ten words, and in Part "B," the student is asked to decompose 10 words which total 39 sounds between them. • The third section measures alphabetical principle by naming letters in a chart. This section is timed, one minute is allowed to complete the exercise. To evaluate this skill, 100 graphemes are used. The graphemes are chosen from the 27 letters of the alphabet, and distributed randomly, mixing upper and lowercases. • The fourth section assesses reading. A chart with 50 bi-syllabic and commonly used simple words, along with additional more complex words in which combinations of letters known as consonant-consonant-vowel (CCV) or consonant-consonant-consonant-vowel (CCCV) were used. In these type of words their syllables are composed with two or three consonants combined with a vowel; for example: “brazo” (CCVCC), “hambre” (CVCCCV). This is also a timed section, students have one minute to complete the exercise. • The fifth section evaluates decoding skills, or the ability to assign a sound to each grapheme, which is a skill considered a precursor of reading fluency. In this section, 50 meaningless words, also known as pseudo words or invented words, are presented to the student, who in turn is asked to read them correctly to demonstrate mastery in this fundamental ability. Mastering decoding skills allow children to read unknown words for them and to read with greater precision and speed. This section is timed, students have one minute to complete the exercise • The sixth section assesses fluency and comprehension by having the student read a short story. The stories included in this Egra were written specifically for this study. Reading comprehension is evaluated using five questions (three literal – the answer is found in the text, and two inferential -answer must be inferred), for the student to answers after reading the story. This section is also timed, and students have one minute to complete the exercise. • The seventh section evaluates the student's ability to write a simple sentence dictated to him/her three times slowly, according to the established norms.

To confirm the reliability of each section of the test, which were duly validated beforehand, correlations were analyzed using Cronbach's6 alpha coefficient. Between the different sections, there was a Cronbach alpha of 0.8515, which is a high indication that the EGRA instrument used for this baseline is reliable (See Table 3).

6 The alpha coefficient is calculated as a result of correlating the individual items with the overall score of the test. It has been established that a test of high reliability and validity must reach an alpha of 0.75 or more.

Table 3. Alfa Cronbach Test –EGRA All sections

Test scale = mean (standardized items) Item- Item- Inter- test rest item Item Obs Sign corr. corr. corr. Alpha Oral Comprehension 871 + 0.4206 0.2718 0.4448 0.8650 Phonemic Awareness – First Sound 871 + 0.4768 0.3350 0.4325 0.8591 Segmenting Word into Sounds 871 + 0.5426 0.4106 0.4182 0.8519 Correct words per minute 871 + 0.7349 0.6438 0.3764 0.8285 Common words per minute 871 + 0.8347 0.7727 0.3548 0.8148 Pseudo words per minute 871 + 0.8006 0.7280 0.3622 0.8196 Reading Fluency 871 + 0.8397 0.7793 0.3537 0.8140 Reading Comprehension 871 + 0.7360 0.6451 0.3762 0.8283 Dictation 871 + 0.6989 0.5986 0.3843 0.8331 Test Scale 0.3892 0.8515 b. The Student Questionnaire was composed of 27 questions and was administrated to the same students that took the EGRA test. This instrument was used to gather information on factors associated with literacy acquisition and on data related to personal hygiene. c. The Individual Questionnaire for teachers had 35 questions. It was administered to 52 teachers of the sample schools who were teaching third grade, - most of them had taught the students who started fourth grade in 2019--. The questionnaire was designed to obtain information about their academic preparation, their practical experience as a teacher, the training received, the trainings’ usefulness for teaching, and the availability of didactic materials in the classroom. They were also asked about teaching practices related to literacy instruction and about the participation of parents in school life. d. The Learning Environment Observation Guide was designed to collect data about the school environment and third grade classrooms´ atmosphere. The instrument lists 15 items related to the general appearance of the school, the learning environment within the third grade classroom, and other sanitary and hygiene issues. The instrument was completed by observation and supplemented with photographs of the school and the classroom.

6. Fieldwork - Data collection

Fieldwork took place from February 17, 2019 to March 8, 2019. Six teams were formed (see Appendix 8): five of them with two members and the sixth, which covered Jinotega, with four enumerators because they administered the instruments in three municipalities simultaneously. Each team had a set of schools and communities to visit that were organized in a geographical "route" with a calendar to follow. During the data collection process, officials from the eleven MINED municipal delegations accompanied the fieldwork teams during the administration of the instruments in each school.

The following table shows the total number of EGRA tests, student and teacher questionnaires, and observation guides administrated. It is important to note that observations were made at the third grade classrooms, where the students studied in 2018. In some municipalities, the number of classroom observations was higher than the number of schools in the sample (in San Sebastian, Bluefields, Kukra Hill, Laguna de Perlas and La Cruz de Río Grande) because these schools had more than one third grade classroom. For example, two schools and three classrooms observed in Yalí, seven schools and nine classrooms were observed in Bluefields, and five classrooms and three schools, in Laguna de Perlas.

Table 4. Total Number of Instruments Administered in Each Municipality Teacher Student Class Municipality Schools EGRA Questionnaire Questionnaire Observation Jinotega 5 195 11 195 5 La Concordia 2 28 2 28 2 San Sebastián de Yalí 2 58 5 58 3 Santa María de Pantasma 5 58 5 58 5 Bluefields 7 159 7 159 9 Desembocadura 1 31 1 31 1 Kukra Hill 3 71 5 71 5 Laguna de Perlas 3 75 4 75 5 El Tortuguero 5 87 6 87 5 La Cruz del Río Grande 5 79 5 79 6 Corn Island 1 31 1 31 1 Total 39 872 52 872 46

Despite the harshness of the geography and the isolation of some of the schools in the sample, particularly in the municipalities of RACCS, data collection was successful and achieved 108% compliance, as shown in Table 2b. Table 5 displays the municipalities covered by each team and route.

Table 5. Routes and Team of Applicators for Fieldwork – February to March 2019 Fieldwork Municipalities Enumerators Jinotega, La Concordia, San Sebastián de Yalí y Santa María de Route 1 Pantasma. 4 Route 2 Bluefields y Desembocadura 4 Route 3 El Tortuguero 2 Route 4 Kukra Hill y Laguna de Perlas 2 Route 5 La Cruz del Río Grande 2 Route 6 Corn Island 2

7. Factor Associated to Learning

The methodology used for the analysis was based on the typical educational production functions. A linear regression was conducted in which the dependent variable was the result of the Fluency section, taking into account – as it was said before – , that the most important ability to master in third grade is fluency. The independent variables included factors related to students’ individual and family characteristics, as well as aspects of the school context that allowed for the assessment of the impact of these factors on learning to read. The majority of variables included were binary or dummy so, the model could allow for the comparison of how each directly influenced learning results. The dummies variable for the municipality of Laguna de Perlas was omitted from the model because it obtained the lowest average on fluency out of all the municipalities in the study; therefore, the results of the model may differ with respect to whether the student was from Laguna de Perlas or not. Bearing this in mind and using the study’s research questions as a guide, the set of factors that could have a positive or negative impact on learning to read were explored.

IV. Results

1. Literacy Results

The analysis of the data found that 19.8% of boys and 27.9% of girls read at a 3rd grade level, for 24.1% of the overall study population. Comparing students’ results in the EGRA section related to reading fluency against IDEL international standards set for students at the end-of- third grade – 85 WPM – the indicator “Percentage of students who, after completing two elementary school grades, demonstrate that they can read and understand the meaning of texts at their level (MGD # 26)” is calculated by dividing the number of students who obtained a score of 85 WPM or above by the total number of students to whom EGRA was administrated. It is important to note that at the time data was collected, students had completed three years in elementary school instead of the standard two years.

Analysis of EGRA administration are presented by sections of the test disaggregating results by gender and municipality.

EGRA administration guidelines indicate that if the student cannot read the first line of letters or words after three attempts, the exercise must be stopped, and the enumerator must mark the option called "The exercise was stopped because the entire first line was incorrect.” Table 6a shows that the largest concentration of stopped exercises was in the section of segmentation of sounds, where more than 24% of students could not do the exercise. - Difficulties were also found in the identification of beginning sounds, with 12.1% of exercises stopped. Dictation presented the third level of difficulty, with 6.5% of students unable to complete the section. The section of letters recognition was the easiest exercise, which was expected, as students had already finished third grade in 2018. Only 0.8% of students could not perform the exercise.

Table 6a. Number and Percentage of Cases of “Stopped Exercise”

Number Stopped Exercise % Stopped Exercise EGRA Section Male Female Total Male Female Total Oral Comprehension 10 17 27 2.4% 3.7% 3.1% Beginning Sound 58 47 105 14.2% 10.2% 12.0% Segmentation 110 101 211 26.9% 21.8% 24.2% Letter Recognition 5 2 7 1.2% 0.4% 0.8% Simple Words 12 8 20 2.9% 1.7% 2.3% Pseudo Words 19 14 33 4.6% 3.0% 3.8% Reading Fluency 20 13 33 4.9% 2.8% 3.8% Dictation 30 27 57 7.3% 5.8% 6.5%

The following table presents the average results obtained in each section of the EGRA. The results were positive in that on average, students read 40.5 simple words per minute and that slightly over half of the questions in oral comprehension were answered correctly, whereas difficulties were found in phonological awareness. Out of ten beginning sounds to be identified, only 5.3 were correctly identified by the average of the students, and of the 39 phonemes to segment, only 15.1 were completed correctly by the average of the students. There were 10 words containing 39 sounds in this section. It is evident that there are problems in reading fluency when comparing the average students´ results to the IDEL standards for the end of third grade. Students should be able to read between 80 and 85 WPM, but on average, students read 65.8 WPM. Male students showed more difficulties (62 WPM) than girls (69 WPM). Lack of phonological awareness skills negatively affects the ability to decode, which in the test was measured through the reading of invented words and dictation. In both pseudo words and dictation, the results were not positive, and the tendency for girls to do better than boys remained constant.

Table 6b. General Results EGRA Test Administration by gender Average results Egra by section Male Female Total % Questions answered correct by the average of Students 60.7% 58.4% 59.4% Number of letters correct by the average of students 51.3 51.4 51.3 Number of simple words correctly read per minute by average of Students 38.9 41.9 40.5 Number of pseudo-words correctly read per minute by average of students 32.4 34.3 33.4 Reading fluency WPM by the average of students 62 69.2 65.8 Reading comprehension % questions correctly answered by the average of students 45.8% 47.4% 46.7% % of correct items in dictation by the average of students 40.8% 45.9% 43.5%

The following table shows the statistically significance of differences between results in females and males. Girls outperformed boys in simple words, pseudo-words, reading fluency, and dictation. No statistically differences by gender were found in the other sections.

Table 6c. Statistically Significance Tests EGRA Sections Gender N Average Deviation Sig. típ. (bilateral) Difference

Oral Comprehension Male 409 3.03 .31 .199 .114 Female 462 2.92 1.31 Phonological Awareness- First Male 409 5.25 3.38 .833 -.0474 sound Female 462 5.29 3.27 Word segmentation - Male 409 15.27 14.18 .778 .2692 Female 462 15.00 13.92 Letters correct per minute Male 409 51.28 19.57 .964 -.0588 Female 462 51.34 18.44 Words correct per minute Male 409 38.94 19.65 .026 -2.878 Female 462 41.82 18.48 Pseudo-words correct per minute Male 409 32.41 15.08 .057 -1.894 Female 462 34.31 14.29 Reading Fluency Male 409 61.96 34.22 .002 -7.155 Female 462 69.12 32.25 Reading Comprehension Male 409 2.29 1.51 .434 -.0816 Female 462 2.37 1.56 Dictation Male 409 3.26 1.83 .001 -.407 Female 462 3.67 1.83

Table 6d illustrates the average results from in each section. Results highlighted in red are the sections in which the average students obtained the lowest results, and those highlighted in blue were above average. At the municipal level, the two schools located in San Sebastián de Yalí obtained the best results, followed by the two schools in La Concordia and the five in the Municipality of Jinotega. The average scores of students from these municipalities exceeded the results obtained by the rest in all indicators. In these municipalities, the results were above the general average in all sections. It is also important to note that in San Sebastián de Yalí and Jinotega, students read very close to the IDEL goal for the end of third grade. Additionally, these students had better results than the rest and higher than the average of the whole group in the phonological awareness section. Students from Desembocadura and Corn Island displayed disadvantages. The language students are taught in can be a factor and will be analyzed when working the variables that influence how students learn to read.

Table 6d. EGRA Results by Municipality Oral Begin Segm. Letters Simple Pseudo Reading % Municipality Comp. Sounds Sounds Recog. Words Words Fluency Comp Dictation San Sebastián de Yalí 73.8% 8.7 28.0 58.3 50.6 37.5 82.7 59.7% 65.3% La Concordia 79.3% 7.9 21.8 56.5 42.1 34.3 69.3 50.7% 53.6% Jinotega 61.7% 6.5 21.3 57.1 44.8 37.2 75.3 58.3% 50.1% Santa María de Pantasma 69.7% 4.4 9.6 46.9 34.6 29.2 53.1 37.2% 35.3% La Cruz de Río Grande 45.3% 2.6 6.1 53.0 44.8 37.6 74.8 39.0% 36.2% Bluefields 55.8% 5.4 16.6 54.3 42.0 34.5 68.2 49.4% 45.1% Corn Island 53.5% 4.5 9.8 50.0 29.5 24.0 45.4 29.0% 28.6% El Tortuguero 60.9% 2.9 4.7 54.4 44.2 36.0 73.1 54.5% 42.2% Kukra Hill 67.3% 4.3 9.3 45.9 36.3 30.2 55.7 48.5% 37.7% Desembocadura de Río Grande 41.9% 4.7 11.0 38.1 36.6 31.9 51.6 20.6% 40.7%

Oral Begin Segm. Letters Simple Pseudo Reading % Municipality Comp. Sounds Sounds Recog. Words Words Fluency Comp Dictation Laguna De Perlas 50.1% 6.1 18.7 32.2 23.7 21.2 37.3 21.6% 30.5% Total general 59.4% 5.3 15.1 51.3 40.5 33.4 65.8 46.7% 43.5%

a. Oral Comprehension (section 1)

Almost 60% of the questions were answered correctly by students tested. Four of the questions were literal, meaning the answer could be found directly in the story, and only one was an inferential question, meaning the answer had to be inferred by the student which requires a higher level of thinking. Similar to other studies, (Castro V., Mejia J., Cummiskey C., Pressley J., Betts K., and Gove A., 2018; Castro V., Laguna J.R., 2017; Castro V., Laguna J.R., Callejas P., Gómez M., 2015-2017) male students obtained slightly better results than girls by 2.3% percentage points.

Table 7. Oral Comprehension by gender Gender % Correct Number Correct Female 60.7 3 Male 58.4 3.9 Total 59.5% 3

b. Beginning Sound Identification (Section 2, Part A)

On average, students identified more than half of the beginning sounds in the test correctly; girls’ performance was very similar to boys’, surpassing them only in the identification of 0.1 more sound. The standard deviation showed a high dispersion in the results, with an overall coefficient of variation of 62.3%, which is slightly more heterogeneous in the case of male students. As noted, this section saw a high proportion of failures, as more than 100 children in the sample could not initiate the exercise.

Table 8. Beginning Sound Identification Gender Mean Std. Dev Stopped Exercise N Male 5.2 3.4 58 409 Female 5.3 3.3 47 463 Total 5.3 3.3 105 872

Words beginning with the sound / s / (both of S and C) and with the sounds / m /, / f /, and the vowel / a / were easier for the students, with the students average identifying correctly more than 65% of the sounds. The easiest first sound identified was in the word "sala" followed by "cielo". Greater difficulties were found with the sounds of the letters / t /, / k / (or the "q" sound which is the beginning sound for “queso”), / b / and / p / also created difficulties. The average fo students identify correctly less than 35% of these sounds, that result went down to only 29.9% in the case of the sound / t /. In most of the studies in which this team has participated, errors with those letters are very frequent.

c. Word Segmentation (Section 2, Part B)

Segmenting a word in its sounds is the act of sounding out each sound in the word. This is a very complex task but is a skill of great importance when decoding and reading unknown words. The official method of instruction used in Nicaragua is the Phonic, Analytic, Synthetic method (FAS) which is a phonetic method. If a phonic method was used for teaching literacy, students who have already completed third grade should have mastered this skill. However, that section showed many problems, as more than 200 students could not even start the exercise. - This section got the highest “stopped exercise” percentage in the entire test. In this section, students had to segment 10 extremely simple words, two of which were monosyllabic, like “mía” and “aro” and eight of which were bi-syllabic words. Students had to segment a total of 39 sounds in this section. Out of 660 students who were able to complete this section, an average of 15 sounds were pronounced correctly, less than half of the total possible. The standard deviation showed a high dispersion in the results, with an overall coefficient of variation of 92.7%, which was slightly more heterogeneous in the case of male students.

Table 9a. Word Segmentation by gender Gender Mean Std. Dev Stopped Exercise N Male 15.3 14.2 110 409 Female 15.0 13.9 101 463 Total 15.1 14.0 211 872

The / ll / sound, which is a digraph, and the / t / sound presented strong problems, similar to the identification of beginning sounds section. The / k / sound (as in "loca" and "cosa") and the / b/ sound, were also difficult as in the previous section. The easiest sounds for students were those of the vowels and those of the graphemes / s / and / m /. Most of the correct answers were concentrated in these phonemes.

Table 9b. Sounds Showing Greater Error Rates in Words Segmentation Greatest Sound % of error Difficulty 1 ll 76.31 2 t 73.76 3 c 70.64 4 b 69.91 5 g 67.26

d. Letter Recognition (Letter Naming) (section 3)

The section of letter naming, as mentioned at the beginning of the report, consisted of 100 graphemes randomly distributed and presented in uppercase and lowercase. This section was timed, and it would be possible for one student to read the entire chart correctly in less than a minute. However, on average students read correctly little more than half of the letters in one minute. The low number of stopped exercises (less than 0.1%) in this section indicates that this part of the test was easy for the majority of students. The standard deviation showed a smaller dispersion in the results, with an overall coefficient of variation of 37%, although the results were slightly less homogeneous for male students.

Table 10a. Letters Correct Per Minute

Gender Mean Std. Dev Autostop N Male 51.3 19.6 5 409 Female 51.3 18.4 2 463 Total 51.3 19.0 7 872

Letters with the highest level of difficulty, with 50% or more errors by the average students, were uppercase J, with 60.1% mistakes, followed by the uppercase Q, with 50.3% errors. The easiest letters to recognize were the five vowels in both uppercase and lowercase and the letters b, L, T, B, t, n, P, m, and r.

Table 10b. Level of Difficulty in Letter Recognition Greatest Letter % Incorrect Difficulty 1 J 60.18 2 Q 50.25 3 ch 48.90 4 q 45.05 5 Ch 42.50

e. Reading Simple Words (Section 4)

Most of the words included in this section were bi-syllabic and of common use, but some complex words such as "quemado," "Tablero," and "hielo" were added. On average, students read 40.5 WPM, with girls slightly surpassing boys with 2.9 WPM more. The standard deviation showed a greater dispersion in the results, with an overall coefficient of variation of 47%. Male results were more dissimilar, with slightly higher dispersion than the female students.

Table 11a. Simple Words Read Correctly Per Minute Stopped Gender Mean Std. Dev Exercise N Male 38.9 19.6 12 409 Female 41.8 18.5 8 463 Total 40.5 19.1 20 872

Words like "clavo" composed by a “locked syllable” (CCV), and “hielo” (diphthong) at the beginning of the list of words in this section did not cause major difficulties. However, on average students had between 20% and 26% of errors reading "comió" and "trepa" (CCV), also at the beginning of the list, "bajó," a word with an accent mark, and “comió”. This type of error highlighted the difficulty students have with the use of accents marks and their limitations to read "qu" (a digraph with the sound / k /) . This problem was already detected in the grapheme reading

20 section (see Table 11b). The word "duelo," which has a diphthong, also created difficulty, although it is supposed to be simpler than "hielo" which starts with a letter whose sound is "mute or non- existent". It is worth to notice that “hielo” is a common word not so “duelo”, this could hypothetically explain this result.

Table11b. Level of Difficulty in Reading Common Words Greatest Difficulty Word % Incorrect 1 Bajó 25.92 2 comió 24.44 3 trepa 22.34 4 Duelo 21.06 5 Quemado 19.69

The simplest words to read were “tiza,” “que,” “cama,” “verano” (number 50 on the list), “silla” (number 23), “vaso,” “año,” “sopa” (number 48), and “lodo,” all with less than 5% of errors.

f. Reading Pseudo-words (Section 5)

On average, students identified fewer pseudo words than common words, which confirmed their weaknesses in phonemic awareness detected in previous sections. According to IDEL standards for second grade7, students –on average--, are expected to read between 80 and 90 pseudo words correctly per minute (WPM) at the beginning of the school year. Therefore, students who finished third grade in 2018 are lagging behind, as they should read at least 85 WPM. Girls scored slightly better than boys by less than two WPM. The observed standard deviation for pseudo words showed a greater dispersion in the results, with an overall coefficient of variation of 44%, showing that the results were more dispersed for male students.

Table 12a. Pseudo Words Read Correct Per Minute Stopped Gender Mean Std. Dev. Exercise N Male 32.4 15.1 19 409 Female 34.3 14.3 14 463 Total 33.4 14.7 33 872

Words with the greatest number of errors were “Juru”, which starts with a j, a grapheme that obtained more than 60% of errors in the section corresponding to letters naming, “uvu” which although it has two vowels seemed to be difficult to read, perhaps because the similarity between the V and the U, “crate” a word CCV that reached almost 28% of errors, followed by “Llusia,” which begins with the digraph "ll" and caused difficulties for students in the section of segmentation of sounds as in letter naming.

7 No IDEL standards are established for pseudo words in third grade because it is assumed that they already dominate the work of decoding.

Table 12b. Level of Difficulty in Reading Pseudo Words Greatest Word % Incorrect difficulty 1 Juru 35.70 2 uvu 30.96 3 crate 27.92 4 Llusia 27.80 5 tlanco 25.91

The easier pseudo words to read, where less than 10% of errors by the average students were made, were “duba”, “lise”, “lura”, “tuzi”, “caco” and “mise”, all of which were bisyllabic with a CVCV structure.

g. Reading Fluency (Section 6, Part A)

The average student read 65.8 WPM. Comparing this result with IDEL standards (see Table 5), students are at “some risk” level because they already finished third grade and at the time of the test, were starting fourth grade. As it has been said according to IDEL, students should be able to read between 80 and 85 WPM at the end of the third grade school year to learn effectively. Girls scored 7.1 WPM higher than boys. The standard deviation for reading fluency showed a greater dispersion in the results, with an overall coefficient of variation of 50.8%. Results were more dissimilar for male students than for female students.

Table 13. Average Reading Fluency by Gender Gender Std. Stopped Mean Dev. Exercise N Male 62.0 34.2 20 409 Female 69.1 32.2 13 463

Total 65.8 33.4 33 872

Results showed that only 24% of students were at the “out of risk” level of performance. Girls had a higher percentage, with 28% performing at the “learning” level and only 19.8% at the “out of risk” level. These results are used to report against indicator “Percentage of students who, after completing two elementary school grades, demonstrate that they can read and understand the meaning of texts at their level (MGD # 26)”

Graph 2. Percentage of Students Reaching IDEL Standard

30% 27.7% 24.0%

19.8% 20%

10%

Female Male Total

h. Reading Comprehension (Section 6, part B)

On average, students answered slightly fewer than half of the questions correctly in the reading comprehension section. This score is slightly below their oral comprehension performance. Girls outperformed boys, but the difference was negligible. The standard deviation for reading fluency showed a greater dispersion in the results, with a global coefficient of variation of 65.9%; however, no difference in dispersion between the genders were observed.

Table 14. Questions Answered Correctly Stopped Gender Reading Comp % Reading Comp Std. Dev. % Exercise N Male 2.3 45.8% 30.2% 70 409 Female 2.4 47.4% 31.2% 84 463 Total 2.3 46.6% 30.7% 154 872

The test involved five questions, of which two were inferential and three literal. The two inferential questions proved difficult8, especially the last one, which no more than 23.5% of boys gave answered correctly. The easiest question to answer was the first one, “¿quién cantaba?”. More than 77% of girls answered this question correctly, one percentage point higher than boys. The later questions had lower percentages of correct answers, with only 40% giving a proper answer to the question, “¿qué asustó a Rosita?”. These results can´t not be understood without highlighting that on average students were able to read less than 43 correct words per minute. Therefore it was impossible for them to respond correctly to questions linked to the last words of the short story.

Boys generally performed worse than girls, except for two questions. One of them was question

8 The students who did not respond due to poor or lower than expected fluency, for example students who could not read the text corresponding to questions four and five, were included in the percentage of incorrect answers, as these texts should not be difficult for a student at the beginning of the fourth grade.

23 number 2, “¿Qué quería hacer Rosita?,” for this question there were differences between the genders and 63.6% of boys gave the correct answer, meanwhile 61.9% of girls did so.

i. Dictation (Section 7)

This section comprises a set of items coded as dummies9; however, recognizing that the items presented different difficulties, the value of the answers was weighted. Table 15a shows that dictation was difficult, as the average of students could not complete more than 35% of the exercise. In this section, 57 students could not start the exercise, representing 6.5% of the total number of students surveyed. The difficulties in taking dictation are associated with decoding problems. The standard deviation for dictation showed greater dispersion in the results with an overall coefficient of variation of 52.9%, and the results were more dissimilar for the male students compared to the female students.

Table 15a. Average Dictation Results by Gender

Gender Dictation Dictation % Std. Dev. Stopped Exercise N Male 3.3 32.6% 18.3% 30 409 Female 3.7 36.7% 18.3% 27 463 Total 3.5 34.8% 18.4% 57 872

Spelling the words “café” and “Miguel” correctly proved very difficult for students. Fewer than 15% of girls spelled “Miguel” correctly. The average student performance was a success of 14.4%. Only 12.8% managed to spell the word “café” correctly, with girls surpassing boys by 5%. The use of the accent mark was difficult for students, as it was in reading the words “comió” and “bajó” in Section 4.

The easiest task was The use of correct direction from left to right. Over 70% of the students spelled “leche” correctly. This is positive because that same digraph (ch) caused difficulties in Section 3, where 48.9% of errors occurred when ch was presented in lowercase form and 42.5% when it was in uppercase. Girls performed 5% higher than boys when spelling “leche.” It is noteworthy that only 33.5% of boys managed to use capital letter at the beginning of the sentence after having completed third grade.

Table 15b. Percentage of Correct Items in Dictation % correct answer Task Male Female Total 7.1 'Miguel' spelled correctly. 14.2% 14.7% 14.4% 7.2 'leche' spelled correctly 68.2% 73.0% 70.8% 7.3 'café' spelled correctly. 10.0% 15.3% 12.8% 7.4 'huevos' spelled correctly 19.6% 23.8% 21.8% 7.5 The use of spaces between words. 73.1% 79.7% 76.6%

9 A dummy variable or fictitious variable usually indicates the presence (1) or absence (0) of a quality or attribute, such as feminine or masculine, black or white, public or private, etc. See Damodar Gujarati, Basic Econometrics, 3rd ed. (Santafé de Bogotá, Colombia: McGraw-Hill, 1997).

7.6 The use of capital letter at beginning of sentence. 33.5% 40.6% 37.3% 7.7 The use of correct direction from left to right 90.2% 90.7% 90.5% 7.8 The use of marking period at the end of the sentence. 17.6% 29.2% 23.7%

2. Analysis of Factors Associated with Learning

Literature on learning and factors associated with learning identify a set of variables that influence learning achievements. The original design of this study included qualitative and quantitative data collection that would have provided information on some of those factors. However, due to the modifications made by MINED to the original design, not all of these factors were investigated in the study. In particular, no data were collected on effective time of instruction and use of time in language and literature class period, which would have provided important insights for some of the results. However, the questionnaires asked about elements that are proven to influence student performance, among others, hygiene and nutrition.

Previous studies carried out by this same team or by some of the team members (Castro V., Mejia J., Cummiskey C., Pressley J., Betts K., and Gove A., 2018. Castro V., Castro V., Laguna JR 2009a 2009b, Castro V., Laguna JR, Callejas P., Gómez M., 2015, 2017) showed that variables such as being over-age, studying in multi-grade schools, and living and going to school in rural area have a negative effect when learning to read. Girls were generally found to obtain better results in all sections of EGRA, except for oral comprehension, a difference that was statistically significant at the 95% confidence level. Preschool attendance and teacher motivation generally found to have a positive relationship with learning outcomes.

In the regression outputs, the following variables had a statistically significant, positive relationship with students´ literacy acquisition (marked in blue in Table 16):

• Female had better reading and writing results than boys (by 6.5 WPM), and this finding was statistically significant at the 99% confidence level.

• Those who attended preschool read 6.0 WPM more than those who did not attend preschool, and this finding was statistically significant at the 95% 10 confidence level.

• Knowledge of personal hygiene routines was also beneficial, as those with knowledge had a reading fluency of 7.5 WPM more than those who did not know personal hygiene routines.

• The practice of washing hands frequently and/or before eating was associated with a reading fluency up to 6.9 WPM higher than those who did not practice washing hands, and this finding was statistically significant at the 95% confidence level. However, this relationship could be due to lower absenteeism or better health of the student. Further research is needed for this factor.

Factors influencing students’ learning linked to teacher characteristics

10 80.3% of the students who participated in the study attended preschool, with a more favorable difference in the children of the department of Jinotega (88.6%) compared to their peers in the RACCS (75% attended preschool).

• Having a teacher who was still studying was positively associated with better learning results. Students with a teacher actively studying read 7 WPM more than students whose teachers were not studying. Out of 16 teachers who were studying at the time the study was conducted, three were studying language and literature, four pedagogy, two elementary school teaching, two agricultural development, two social sciences, one secondary school, and one beauty. The career concentration did not seem to affect if the teachers delivered better classes, since most of them were not studying subjects related to education. Thus, if continuing studies is taken as a "proxy" for motivation, it was found that teachers with higher motivation levels achieved better learning results among their students.

• Having a male teacher was positive particularly for reading fluency. Students with a male teacher read 9 WPM more than those who had not a male teacher. Among the teacher interviewees, there were 11 men and 41 women.

• Extrinsic motivation, particularly achievement recognition given to students by the teacher, was also positively associated with a higher reading fluency. When the teacher told students that they did well, they scored six WPM more than students who said not to receive that kind of recognition. This association was statistically significant at the 95% confidence level. This factor has also been identified previously in studies cited (Castro V., Laguna J.R., 2009 Castro V., Laguna J.R., Mayorga Nora., 2009, Castro V., Laguna J.R., Vijil J., 2010).

The following variables all of them linked to teacher performance or the educational system were negatively associated with reading results (marked in red in Table 16):

• Studying in a rural school area was related to reading 7.9 WPM fewer than their peers in the urban area. This difference was statistically significant at 95% confidence level.

• Having a teacher who taught in a language different from their mother tongue was negatively associated with fluency. Students who received classes in Spanish from teachers who were not Spanish-speaking natives read on average 16.2 WPM fewer than those students who received classes from a native Spanish speaking teacher. Out of the 52 teachers, 46 reported teaching in Spanish, four in Creole, one in Ulwa and another in Sumu- Ulwa. It was not possible to determine the Spanish fluency level in the teachers that taught in Spanish, or the number of students whose native language was different from Spanish but were instructed in Spanish. These differences could also negatively influence learning.

Generalities

The model also explored another series of variables that were not statistically significant, such as being over-age, repetition and learning in a multi-grade environment. Information such as teacher reads to students in classes, eating at school or getting help from mothers to do homework did not show statistically significant differences either. The relevance of professional development topics was also explored but no statistically significant differences were found. On the contrary, they were excluded from the model because the inclusion of these variables generated problems of multicollinearity in the regression model.

Table 16. Variables affecting learning Factors Associated with Learning Fluency p-value Student Characteristics – Student Questionnaire Is student female? 6.5 0.002 Repeating grade? n.s. 0.537 Over-age? n.s. 0.820 Multi-grade? n.s. 0.246 3. Attended preschool? 6.0 0.024 Stories - 7. What does your teacher read in class? n.s. 0.811 Cartoons - 7. What does your teacher read in class? n.s. 0.256 Stories - 12. And when you read, what do you like to read? n.s. 0.766 Mother - 17. Who helps you with homework? n.s. 0.131 I eat at school everyday. n.s. 0.343 Others - 26. What personal hygiene procedures do you know? 7.5 0.016 Cut your nails frequently – 26. What personal hygiene procedures do you know? n.s. 0.200 Others - 27. When do you wash your hands? 6.9 0.040 After going to the latrine - 27. When do you wash your hands? n.s. 0.544 School Characteristics – Geographic Location Rural -7.4 0.032 BLUEFIELDS 26.0 0.000 CORN ISLAND n.s. 0.326 DESEMBOCADURA DE RIO GRANDE 13.5 0.094 EL TORTUGUERO 16.4 0.019 JINOTEGA 21.7 0.000 KUKRA HILL n.s. 0.149 LA CONCORDIA n.s. 0.300 LA CRUZ DE RIO GRANDE 22.6 0.000 SAN SEBASTIAN DE YALI 30.1 0.000 SANTA MARIA DE PANTASMA 13.0 0.056 Teacher Characteristics – Teacher Questionnaire Teacher is male 9.5 0.008

Factors Associated with Learning Fluency p-value Nicaraguan elementary education certificate - 2. What is your last approved educational level? n.s. 0.420 3. Are you studying at the present? 7.2 0.024 Mother tongue different from Spanish -11.3 0.017 9. How long have you been teaching this or these grade(s)? n.s. 0.648 What helps to teach a class better? n.s. 0.260 Tell them they had good performance - 30. When students get grades 6.2 0.023

No correlation was found between teachers with an elementary education certificate issued by the official Nicaraguan teacher training schools (Escuelas Normales) and learning to read. This was the case in all of the aforementioned studies and likely be attributed to the fact that the Nicaraguan teacher training program consists of nothing more than two years of studies after high school. Despite various efforts to modernize this program, even with economic support from foreign institutions (Luxembourg, for example), the program content is outdated and methodologies old fashioned.

No correlation was found between school food and learning either. This can be explained by the fact that almost 100% of the students received food in school, therefore there was no statistical variance in the sample to find binding associations. The only exception to this 100% coverage in terms of food was in La Cruz de Río Grande, where the percentage of students receiving food was 83%.

Discrepancies in the municipal level results were not correlated statistically with the observations of classrooms and schools, but it is worth highlighting the following:

In San Sebastián de Yalí where students’ fluency met the expected IDEL standard and were more than 26 WPM higher than their peers in the municipality of Laguna de Perlas, two schools and three classrooms were observed. In those schools, there was no garbage in the classrooms, students reported 100% used of the books available in the classroom, 100% used latrines and latrines were clean. Furthermore, 66.7% of the classrooms had mini libraries, placing them well above the average of the other schools and classrooms visited which had mini libraries in only 29.8% of the classrooms. In that municipality 100% of the classrooms had a literacy-rich environment, word- building panels and educational posters displayed.

In Bluefields, the average student read no more than 68.2 WPM; however, this was higher than their peers in the region. In the region, seven schools and nine classrooms were visited. In that city, garbage was found in 11% of these classrooms, 44% of the students were using books available in the classroom, and 22% of the classroom had mini libraries. This percentage was lower than the average, but it should be noted that Bluefield had instructional posters (33%) and bulletin boards promoting reading (22%). The fact that Bluefield is an important city in the South Caribbean Coast may also impact the quality of education, making reading results there better than in Laguna de Perlas, Desembocadura, Kukra Hill, etc. In the past, educational projects

28 implemented by organizations such as Community Action for Reading and Security (CARS), BASE and Save the Children supported schools in Bluefields. It is possible that one or more of the schools in the sample had support from one or more of those projects. In addition, having been part of a first stage MESA project implies that the teachers have received training in food safety, textbooks, and children's literature.

The next municipality with fluency results well above average among the municipalities in RACCS was La Cruz de Río Grande, where five schools and six classrooms were visited. In these classrooms, 100% of books were used, there was no garbage in the classrooms, the students used the latrines, and 67% of the latrines were clean. As a remote rural municipality with a high level of poverty, its results were positive despite the rather adverse conditions. The lack of garbage and the relatively good condition of the latrines contrasted with what was found in El Tortuguero where there was garbage in 40% of the classrooms, or in Jinotega, a nearby and nonrural municipality where only 40% of the latrines were cleaned. Three other elements observed in La Cruz that may have contributed to the municipality’s above average fluency were the literacy-rich classroom environment and, instructional posters displayed along with worksheets and drawings, all of which were promoted in the first phase MESA. Elementary schools in this municipality have not been the beneficiary of any other non-governmental organization’s project.

3. Teacher’s Abilities

The teacher questionnaire inquired about the training teachers received in the past four years, the topics, and the training’s usefulness. The results showed that 88% of teachers received training in learning methodologies and 81% in reading assessment techniques. Comparing the training received by teachers in the municipalities of RACCS with those of teachers in Jinotega, 83% from RACCS said to have received training in FAS method, whereas only 60% in Jinotega. As for training in reading comprehension, 81% of teachers from Jinotega and 69% from RACCS confirmed to have received the training. Teachers also responded that they received training in Food Security (72%) and hygiene (77%).

In regards to putting the trainings into practice some interesting findings were uncovered. 95% of teachers from Jinotega and 80% from RACCS confirmed the use of the literacy training in the classroom. 83% of teachers from RACCS confirmed having received training in FAS method, but only 61% are putting it into practice.

An indicator of the effectiveness of the teaching literacy training was measured by the percentage of teachers that replied “Have been trained in the last four years, and put into practice in the classroom the following topics: strategies to evaluate reading, FAS Method and Reading Comprehension”. Baseline data found that only 42% of teachers were both trained and are putting new knowledge into practice.

According to the teachers, the challenges they face after receiving these trainings is the lack of materials and the difficulties in managing the new strategy. At the regional level, almost half of teachers from Jinotega requested more training, while a third of the RACCS teachers indicated that they have not been able to implement what they have learned.

4. Food Security Results

Questions related to food security were included in the student questionnaire. The baseline measured the percentage of students in target schools who indicate to be “hungry” or “very hungry” during the school day even after eating in school (MGD 1.2.1). The baseline found that 9% of male students (38 of 409) and 5% of female students (23 of 463) met this indicator definition, for 7% of the overall study population (61 of 872). Additionally, 98% of the total number of students interviewed confirmed to receive food in school. However, their answers concerning the frequency in which food was received varies. 74% received food every day, 11% from three to four days a week, 13% one or two days a week, and 2% claimed did not receive food.

5. Hygiene Results

The baseline measured the percentage of schools whose students wash their hands before eating (SO 2), and the percentage of students who can identify at least three good hygiene practices (MGD 2.1). The baseline found through observation that students in 95% of schools (37 of 39) wash their hands before eating, but in looking at individual students, only 68% of students confirmed to do so, and 44% of male students and 46% of female students were able to identify at least three good hygiene practice, for a total of 45% of the overall study population (393 of 872). 88% of all students identified at least one hygiene practice to be healthy, answers from students in Jinotega was slightly higher (90%) compared to answers in RACCS (86%). Healthy practices most identified by students were: washing hands (68%), daily shower (61%), and brushing teeth (51%).

V. Conclusions and Recommendations

EGRA results indicated that the students evaluated had limitations in decoding skills, which had a negative influence on their ability to read fluently. Problems were also found in sections of the test related to phonological awareness, a fundamental skill for decoding, and in reading pseudo-words and dictation.

When comparing the reading fluency averages to the IDEL standards set for third graders at the end- of the -school year, it was found that students read an average of about 20 WPM fewer than they should be. Girls outperformed boys by 6.5 WPM, and boys performed an average of 62 WPM. It is a prerequisite for students in fourth grade to have mastered reading skills to truly learn.

Accordingly, close to 70% of girls and 80% of boys did not reach the IDEL benchmark set for the end of third grade. This problem was identified in four out of the 11 municipalities: San Sebastián de Yalí, Jinotega, La Cruz de Río Grande and El Tortuguero.

Improve instruction in phonological awareness represents a challenge, since 18% of students showed weaknesses in this area. Limitations were even more evident in the segmentation of sounds where 24% of students could not even start the exercise and, where only 16 out of 39 sounds were correctly done.

It is important to remark that factors that affect positive learning were found in the study, so MESA II can include these variables in the project strategies. Factors with a positive effect on learning included having attended preschool and having motivated teachers, with higher achievements were seen in students with teachers who were still studying.

It is also important is to highlight that "knowing and practicing personal hygiene routines,” for example hand washing before eating, impacted learning positively, which confirms the benefit of the interrelationship between learning and health addressed in MESA I.

Recommendations

2. Understand what is being done in La Cruz del Río Grande, a municipality of RACCS, where students read 74.8 WPM (close to the IDEL goal for 3rd grade) and 22.6 WPM faster than their peers from Laguna de Perlas11. These differences are statistically significant at the 99% confidence level.

3. Determine which new knowledge, methodologies or learning teachers have acquired whose practices have better results than those of their colleagues. Two qualitative case studies and

11 In the model, the omitted variable was Laguna de Perlas

instruction follow-up are proposed to do this. Classroom observations could be done using the Stalling Observation Tool, which was designed to assess teachers’ use of time in the Language and Literature class.

4. Design a comprehensive professional development program for currently working teachers that includes personal and professional growth. This plan should preferably be "online," available on smart televisions or on USB. The plan should include training in phonological awareness, fluency and comprehension, and the distribution of previously tested, ready-to-use lessons, to improve teaching instructions in these critical areas. The program should also include socio-affective elements that take the importance of motivation in teaching into account. In addition, these are key elements to overcome the stress generated by the socio- political and economic crisis experienced in Nicaragua.

5. It is recommended that MESA II include an effort to expand preschool coverage, since attending preschool was positively associated with learning outcomes. If this is not possible, topics related to childhood development and the promotion of reading should be included in parents’ workshops o school for mothers and fathers. Evidence has shown that workshops for parents, working at the community level to create reading clubs, reading circles, and holding reading fairs has a positive influence in learning (Castro V., Luna J., 2018)

6. Intensify and extend training in personal hygiene to the community. According to statistical correlation analysis it was proven to have positive effects on learning. Further research is advised.

7. Considering that having books in the classroom and making use of them seem to have a positive effect on learning, it would be advisable to sponsor a campaign to increase the use of books where they already exist, replace broken books, or distribute new books where do not have any.

VI. Bibliography

Castro V., Mejia J., Cummiskey C., Pressley J., Betts K., and Gove A., (2018) Evaluación de Lectura Inicial en El Salvador. web de USAID: https://lnkd.in/evPU4uf

Castro V., Luna J., (2018) “Estudio de Caso sobre aprendizaje en Siuna y el impacto de los círculos de lectura en el rendimiento de estudiantes de primero a tercero” promovido y financiado por Save the Children

Castro V., Laguna J.R., Mayorga N., (2009) Informe para USAID sobre resultados EGRA 2008 en Nicaragua. En web RTI

Castro V., Laguna J.R., (2009) Informe para el Banco Mundial sobre escuelas PROHECO Honduras. En web RTI.

Castro, V., Laguna J.R., Vijil, J., (2010) Informe sobre las habilidades lectoras de estudiantes de la Costa Caribe Nicaragüense. En web RTI

Flórez C, A., (2012) La educación primaria en Nicaragua: condiciones que favorecen u obstaculizan retención y promoción escolar. IEEPP www.ieepp.org

Gujarati, Damodar. 1997. Econometría básica (3a ed.). Santafé de Bogotá, Colombia: McGraw- Hill. Traducción, Gladys Arango Medina; revisión técnica, Martha Misas Arango.

PREAL-EDUQUEMOS (2007), Apostar por la educación. Informe de Progreso Educativo Nicaragua

Appendix 1 – EGRA

Formulario EGRA Proyecto Mejor Educación y Salud - MESA II Evaluación inicial febrero 2019 Instrucciones generales: Establezca una atmosfera relajada para que los estudiantes que participaran en esta valoración se sientan cómodos. Inicie con una conversación trivial y procure conseguir que el estudiante se sienta como que va a jugar. Cuando termine la valoración agradezca al estudiante por su tiempo.

Asegurar el Consentimiento Verbal

Lea el texto siguiente con claridad.

Me llamo ______...trabajo con • Queremos saber cómo aprenden a leer los estudiantes. • Voy a pedir que leas letras, palabras y un cuento corto en voz alta. • Usando este aparatito que mide el tiempo, voy a anotar cuánto tiempo te toma leer. Queremos que leas, sin detenerte, lo mejor que puedas. • Esto no tiene nada que ver con tus notas de la escuela, así es que no te preocupes. • ¿Quieres ayudarnos? Si no quieres, no hay problema, no es obligatorio. • ¿Podemos comenzar?

Chequee la casilla si obtiene el consentimiento oral: SI (Si no se obtiene el consentimiento oral, dele las gracias al niño y siga al siguiente niño usando este mismo formulario)

Nombre de la Escuela: ______Multigrado: ______Matrícula inicial: ______

Regular: ______Matrícula actual: ______Nombre del niño/a: ______Intercultural bilingüe: Número de estudiantes presentes ______cuando se aplica prueba: ______Monolingüe: _____

Código Único de Escuela Grado del Alumno

Nombre del Encuestador Sección del Grado Única o ______

Turno 0=Mañana 1= Tarde Género del Alumno 0=Masculino 1=Femenino

Fecha de Nacimiento del Edad (en Años) Alumno

HORA DE INICIO ______: ______AM/PM HORA DE TERMINAR ______: ______AM/PM

   Sección 1. Comprensión Oral  Voy a leer un párrafo breve una vez, luego te haré unas preguntas. Por favor presta mucha atención mientras leo y luego responde a las preguntas lo mejor que puedas. ¿Listo? Comencemos. [Lea en voz alta y clara UNA VEZ el pasaje a un ritmo constante y pausado.]

 Pon una marca en la caja que corresponde

La abeja Mariela tenía hambre y quería miel. Buscando comida en un jardín con pocas flores, Mariela se encontró una flor pequeña color blanco. Como la flor era chiquita, la abeja Mariela no voló hacia ella pensando que no tenía miel. Mariela buscó una flor grande creyendo que entre más hermosa más miel tendría y vio un jalacate rojo que le gustó. Se acercó y cual fue su susto que no halló miel. Mariela se quedó con hambre pues cuando quiso volar hacia la flor que había despreciado, otra abeja estaba comiendo su deliciosa miel. Esperar 15 segundos para que el-la estudiante de su respuesta Correcto Incorrecto NS/NR

1.1 ¿Qué le pasaba a Mariela? [Tenía hambre/quería miel]

1.2 ¿Qué quería comer? [miel]

1.3 ¿Dónde halló Mariela una flor chiquita? [En un jardín con pocas flores, en un jardín que no tenía muchas flores]

1.4 ¿Cúal fue el susto de Mariela? [que en la flor grande no había miel]

1.5 ¿Por qué se quedó con hambre Mariela? [porque despreció una flor pequeña que si tenía miel por buscar una grande/ porque creyó que entre más grande la flor más miel tenía cualquiera de las dos]

 Sección 2a. Conciencia Fonémica aislar primer sonido  

 Las palabras están hechas de sonidos y cada letra tiene un sonido, por ejemplo, la letra M suena así: /mmm/. Ahora, voy a leerte algunas palabras para que practiquemos su primer  (CONTINUE) sonido. • Si el niño pausa Por favor escucha bien y dime el sonido con que comienza cada palabra. Empiezo con “papá”; en un sonido por /ppp/ el primer sonido de “papá” es /ppp/. Practiquemos ¿Cuál es el primer sonido de papá? 3 SEGUNDOS). [Si lo hace incorrecto, diga:] Practiquemos de nuevo la palabra “papá”, cuyo primer sonido es /ppp/. [Marcar con énfasis el sonido /ppp/]. Dime cuál es el primer sonido de “papá”. [Si dice, /ppp/ diga] ¡Muy bien! El primer sonido suena /ppp/. Continúe con una práctica más.  (PARE) •Si el estudiante no Practiquemos con otra palabra, ¿Cuál es el primer sonido de la palabra “si”? identificó ninguno [Si dice, /s/ diga]: ¡Muy bien!, El primer sonido de “si” es /ssss/. de los primeros [Si lo dice incorrectamente diga]: El primer sonido de “si” es /ssss/. cinco sonidos ¿Comprendes lo que debes hacer? ¿Listo(a)? Te voy a dar la primera palabra. ¿Cuál es el primer correctamente. sonido de la palabra: “…__”? [Repita cada vez estas instrucciones. Marcar correcto o incorrecto para cada palabra. Si el estudiante se equivoca en las primeras cinco palabras, pare el ejercicio. Debe dar 15 segundos máximo por palabra].

 Pon una marca en la caja que corresponde ¿Cuál es el primer sonido de la palabra “_____”? [Marque correcto o incorrecto] mamá /m/ Correcto  Incorrecto  No responde  Sala /s/ Correcto  Incorrecto  No responde  Amor /a/ Correcto  Incorrecto  No responde  tamal /t/ Correcto  Incorrecto  No responde  cielo /sss/ Correcto  Incorrecto  No responde  foca /f/ Correcto  Incorrecto  No responde  luna /l/ Correcto  Incorrecto  No responde  bueno /b/ Correcto  Incorrecto  No responde  pecho /p/ Correcto  Incorrecto  No responde  queso /k/ Correcto  Incorrecto  No responde 

 Descontinuado: (Marque una “ “ en la siguiente casilla, si el estudiante no dijo los primeros cinco sonidos y/o los pronunció incorrectamente)

Sección 2b. Conciencia fonémica-segmentación de palabras en los sonidos que las componen   Voy a decirte una palabra, después de que te la diga, quiero que me digas uno por uno todos los sonidos que tiene. Por ejemplo, te digo oso, y tú dirás /oo/, /sss/, /oo/. Vamos a probar, ahora dime los sonidos de la palabra mamá. Si responde  (CONTINUE) •Si el niño pausa en correctamente /mmm/ /a/ /mmm/ /a/ dígale MUY BIEN. Si no responde un sonido por 3 correctamente separele la palabra mamá en sus cuatro sonidos: /m/ /a/ /m/ /a/, y SEGUNDOS). pídale que los repita. Luego de haber hecho esto dígale: Comencemos! Por favor presta mucha atención mientras te digo cada palabra y separá sus sonidos  (PARE) lo mejor que puedas. Si el estudiante no Lea en voz alta y clara UNA VEZ cada palabra a un ritmo constante y pausado, repita identificó ninguno para cada palabra la misma instrucción. de las primeras cinco palabraas ¿Cuáles son los sonidos de la palabra “_____”? correctamente

 Descontinuado: (Marque con una “ “ en la siguiente casilla si el estudiante no pudo segmentar correctamente las primeras cinco palabras). Tache con una raya ----- los sonidos que el estudiante no reprodujo bien o no pudo reproducir. Ponga corchete después del último sonido ( ] ) pronunciado.

 Al lado se cada sonido, marcar “1” si el estudiante dice el sonido correcto o “0” si lo dice incorrecto.

salta /s/ /a/ /l/ /t/ /a/ Mia /m/ /i/ /a/

aro /a/ /r/ /o/ Cosa /c/ /o/ /s/  /a/

pala /p/ /a/ /l/  /a/ Gana /g/ /a/ /n/ /a/

beso /b/ /e/ /s/  /o/ Tubo /t/ /u/ /b/ /o/

loca /l/  /o/ /c/  /a/ bella /b/ /e/ /ll/  /a/

Sección 3. Conocimiento Nombre Letras  Tarjeta 1  60 segundos  Muestre al estudiante la hoja plastificada de letras. Diga:] Aquí tienes una serie de letras del  (PARE) •Si el tiempo en alfabeto. Por favor decí sus nombres lo mejor que podas. Empiezo con un ejemplo: el nombre de el cronómetro esta letra [señala la C] es “Ce”. Ahora inténtalo con esta otra letra. ¿Cuál es el nombre de esta la se acaba (60 segundos). letra [señala la letra “T” (te]?:

[si el estudiante responde correctamente, diga]: bien, el nombre es “Te”.  (CONTINUE) [si el estudiante no responde correctamente, diga]: el nombre de esta letra es “Te”. • Si un niño pausa en una letra ¿Comprendés lo que debés hacer? Cuando te diga “comienza”, decí los nombres de las letras lo por 3 mejor que podás. Yo me voy a callar y te voy a escuchar, a menos que necesités ayuda. Cuando pase SEGUNDOS). un tiempo voy a decir “alto” para que te detengás. Poné tu dedo debajo de la primera letra. ¿Listo(a)? Comienzá por favor.  (PARE)  (/ ) Incorrecto o sin respuesta Autocorreción • Si el estu-diante ( ] ) Después de la última letra leída no identificó ninguna letra de la primera V l e m S y h ñ L N 10 línea correctamente. l K T D H T a d z w 20 r ch z m U e j G X u 30 g R B Q I f J Z s r 40 B n C d p Y F c a E 50 y s Ll P M v O t n P 60 Z A e x f F r u A t 70 Ch G T b S l g m i I 80 L L o q a N E Y p x 90 N k c D d y b j R v 100

 Tiempo utilizado (en segundos):  Descontinuado: (Marque con una “ ” en la casilla si el estudiante no pudo leer la

primera línea correctamente)

Sección 4.  60  Tarjeta 2 Conocimiento de palabras simples segundos  [Muestre al estudiante la hoja plastificada de palabras. Diga:] Aquí tenés una serie de  (PARE) •Si el tiempo palabras para que las leas. Empiezo con un ejemplo: esta palabra es “mi”. Ahora en el intentálo con esta otra palabra. [señale la siguiente palabra: la] Leéla en voz alta. cronómetro [Si el estudiante dice “la”, diga]: “Muy bien: la”. se acaba (60 segundos). [Si el estudiante no dice “la” correctamente, diga]: Esta palabra es “la”. ¿Comprendés lo que debés hacer? Cuando diga “comienza”, leé las palabras lo mejor  (CONTI- que podás. Cuando pase un tiempo voy a decir “alto” para que te detengás. Poné tu NUE) dedo debajo de la primera palabra. ¿Listo? Comenzá por favor. • Si el niño pausa en una  ( / ) Incorrecto o sin respuesta Autocorrección palabra por 3 ( ] ) Después la última palabra leída SEGUN-DOS). Asar pesa en Roja luna 5  (PARE) Malo nuca pala Mima lucha 10 • Si el estudiante no comió subir trepa eso Duelo 15 contestó ninguna Pela laja Clavo misa Hielo 20 palabra de la Año silla fresa cuna Preste 25 primera línea correc- Mata que casa Esta Quemado 30 tamente.

Salir tacho fácil grasa Preso 35 tablero vaso todo Vaca zapato 40 Ranas cama Bajó Plata Cose 45 Usado pata sopa Tiza verano 50

 Tiempo utilizado (en segundos):  Descontinuado: (Marque con un “ ” en la casilla si el estudiante no pudo leer la

primera línea correctamente)

Sección 5. Decodificación de palabras sin sentido  Tarjeta 3

 [Muestre al estudiante la hoja plastificada de palabras sin sentido. Diga] Aquí tenés  (PARE) una serie de palabras inventadas para que las leás. Empiezo con un ejemplo: esta • Si el tiem-po palabra inventada es “ut”. Ahora intentálo vos con esta otra. Leéla en voz alta en el [señale: dif]. cronóme- [Si el estudiante dice “dif”, diga]: “Muy bien: dif”. tro se aca- [Si el estudiante no dice “dif” correctamente, diga]: Esta palabra inventada es “dif”. ba (60 segundos). ¿Comprendés lo que debes hacer? Cuando diga “comienza”, leé las palabras lo mejor que podás. Cuando pase un tiempo voy a decir “alto” para que te detengás.  (CONTI- Poné tu dedo debajo de la primera palabra. ¿Listo? Comenzá por favor. NUE)  ( / ) Incorrecto o sin respuesta / Autocorrección • Si el niño pausa en ( ] ) Después de la última palabra leída una palabra Lete sipe boja Mise garo 5 por 3 SEGUN- quibe beño nolo Lura Cusa 10 DOS).

muno rite duso Sata dica 15  Luma alti Luda crate Dulte 20 (PARE) • Si el estu- Ledo cosu tresa Lemo molpa 25 diante no identificó Bosa male Flano Trabu Bulo 30 ninguna palabra de uvu arcu Cince Llusia Firta 35 la primera Onti maca Quecho Bana Juru 40 línea correctamente. toba Lise Vodo Tuzi Listu 45 quira Crutir tlanco caco Duba 50

Palabras correctas por minuto  Tiempo utilizado (en segundos):  Descontinuado: (Marque con una “ “ en la casilla si el estudiante no pudo leer la primera línea correctamente)

Sección 6. Lectura y comprensión de un pasaje  Tarjeta 4  60 Segundos

 Aquí tenés un cuento para que lo leás. Quiero que leás en voz alta. Cuando terminés, te haré algunas preguntas sobre el cuento. ¿Comprendés lo que debés hacer? Cuando te diga "comenzá", leé el cuento lo más rápido y mejor que puedas. Cuando pase un tiempo  PARE voy a decir "alto" para que te detengás. Poné tu dedo debajo de la primera palabra. ¿Listo? Comenzá, por favor.

 ( / ) Ponga esta marca a cualquier palabra incorrecta. • Si el tiempo en el cronómetro se ( ] ) Ponga esta marca a la última palabra leída. acaba Autocorrección Ahora te haré algunas preguntas sobre la historia que acabas de leer. Trata de contestar Sin las preguntas lo mejor que puedas. Correcta Incorrecta respuesta

Un zanate negro como la noche cantaba ¿Quién cantaba en el patio de 13 diario en el patio de Rosita. Rosita? [una ave, un zanate] (60 Segundo)

¿Qué quería hacer Rosita? Rosita estaba enamorada de su canto y 26 [Quedarse con el pájaro, ave o se quería quedar con el zanate. zanate] • Si ha marcado como incorrectas Un día lo quiso agarrar con un trapo y ¿Qué asustó a Rosita? [que el pájaro 43 todas las se asustó mucho cuando el ave le habló. o zanate le hablara] palabras de la primera línea sin El ave le dijo: soy libre como el viento y ¿Por qué cantaba el ave? [Porque autocorrecciones por eso canto, si me pones en una jaula 63 era libre] descontinúe el moriré. ejercicio. ¿Por qué volvía el ave todos los días? [Porque sabía que a Rosita le Rosita le pidió perdón y el ave volvió 79 gustaba su canto/ porque ella le todos los días a su patio para cantarle. pidió perdón. una de las dos respuestas o ambas]

 Tiempo utilizado (en segundos):

 Descontinuado: (Marque con una “ ” en la casilla si el estudiante no pudo leer la

primera línea correctamente)

Sección 7. Dictado El estudiante debe escribir en el espacio de la actual página que está libre o en la parte de atrás. Diga: Vas a escribir la oración que te voy a dictar. Escuchá sin escribir. Luego, mientras la escribís, la voy a repetir dos veces más. ¿Listo-a? Comencemos! Haga una primera lectura de la oración a razón de dos o tres segundos por palabra. Luego, entregue el lápiz y repita la oración con igual lentitud. Usted puede repetir la oración una vez más solamente.

Miguel fue a comprar leche, café, pan y huevos para desayunar. Criterio de Evaluación Forma 0=incorrecto 1=correcto correcta

7.1 Escribió “Miguel” correctamente. Miguel con mayúscula y gu 7.2 Escribió correctamente “leche” leche

7.3 Escribió “café” correctamente. café con tilde 7.4 Escribió “huevos” correctamente Huevos con h y el diptongo bien 7.5 Usó espacios entre los textos.

7.6 Usó la mayúscula inicial correctamente. M

7.7 Usó la dirección correcta de izquierda a derecha 7.8 Usó punto de cierre al final de la oración. .

Appendix 2 – Student Survey

Appendix 2 Proyecto Mejor Educación y Salud - MESA II Cuestionario del Estudiante de 4to grado 18 Consentimiento verbal Hola. Mi nombre es ______, estamos realizando para PCINicaragua entrevistas a los niños y niñas sobre la educación, alimentación e higiene en la escuela. Tus opiniones son importantes, apreciaríamos tu participación. Esta información ayudará a mejorar las actividades de lecto escritura y el ambiente higiénico sanitario en el marco del Proyecto MESA con el PINE/MINED. Esta entrevista normalmente tarda _____ minutos.

Cualquier información que nos proporciones es estrictamente confidencial y no se mostrará a otras personas.

Tu participación en esta entrevista es voluntaria y puedes escoger no contestar cualquier pregunta individual o todas las preguntas. ¿Te gustaría participar?

¿Tenés alguna pregunta sobre lo que te dije?

¿Aceptás ser entrevistado-a?: ____Si ____No - Termine la entrevista

Firma del/de la entrevistador-a: ______

Fecha aplicación Día______Mes______Año______

Hora de inicio: ______Hora finalización: ______

Proyecto Mejor Educación y Salud MESA II - Evaluación inicial 2019 Cuestionario del Estudiante 4to grado I. Datos Generales de la Escuela Nombre de la Escuela: ______

Código Único de la Escuela: ______Comunidad:______Municipio:______Departamento:______

II. Datos Personales y Académicos del Estudiante 1 Sexo: _____ Mujer _____ Hombre 3 Edad: ______años 4 Grado :______2 ¿En qué grado estabas el año pasado?

( ) 2do Grado ( ) 3er Grado ( ) Ninguno ( ) NS ( ) NR 3 ¿Fuiste a preescolar? ( ) Sí ( ) No ( ) NS ( ) NR

4 ¿Qué libros tenés para tus clases en la escuela? No lea opciones ( ) Libro de Lengua y Literatura ( ) Libro de Matemáticas ( ) Diccionario ( ) Cuaderno de Trabajo para lecto escritura ( ) Cuentos ( ) Ninguno ( ) otro: ______III. Aprendizaje de la lectura en la escuela 5 ¿ Lee tu maestra en voz alta durante la clase? SI ( ) Si responde NO ( ) ( ) NS ( )

NR Pase a P. 9 6 ¿Cuántas veces lo hizo en la semana pasada?

( ) Nunca ( ) Dos Veces ( ) Tres veces ( ) Cuatro veces ( ) Cinco o más veces 7 ¿Qué les lee tu maestra en la clase? ( ) Cuentos ( ) Otros libros de texto ( ) Libro no de texto ( ) Libro de Lengua y Literatura ( ) Paquines ( ) NS/NR ( ) Historietas ( ) Revistas ( ) Otro, especifique: ______8 ¿Te hace preguntas sobre lo que leen en clase? ( ) Sí ( ) No ( ) NS ( ) NR

9 En la escuela, ¿te dan tiempo para leer? SI ( ) Si responde NO ( ) ( ) NS ( ) NR Pase a P. 13

10 ¿Cuántas veces por semana lees en la escuela? ( ) Todos los días ( )NS ( )NR ( ) 1 vez por semana ( ) 2 a 3 veces por semana ( ) Otro ______

11 ¿Escoges vos lo que lees? ( ) Sí ( ) No ( ) NS ( ) NR 12 Y cuando lees, ¿qué te gusta leer? Marcar todas las que apliquen

( ) Libro de Lengua y Literatura ( ) Otros libros de texto ( ) Libro no de texto ( ) Cuentos ( ) Paquines ( ) Historietas ( ) Revistas ( ) NR ( ) NS ( ) Otros, especifique

13 ¿Llevas libros de clase a tu casa? ( ) Si ( ) No Pase a P.15

14 Cuál? ______15 Cuando lees bien en clase o hacés bien tu tarea, tu maestra o maestro te…… ( ) felicita ( ) da un premio ( ) no hace nada ( ) no se da cuenta cómo sales ( ) otro: ______

IV. Aprendizaje de la lectura en casa 16 ¿Te ayudan con tus tareas en la casa? ( ) Sí ( ) No Pase a P.18 ( )

NS ( ) NR 17 ¿Quién te ayuda con las tareas en tu casa? Marcar todas las que apliquen

( ) Los dos padres ( ) Mamá ( ) Papá ( ) Hermano-a ( ) Abuelo-a ( ) Tío-a ( ) Pariente ( ) Otro-a ______( ) NS ( ) NR 19 ¿Te leen o cuentan cuentos en la casa? ( ) Sí ( ) No Pase a P.20 ( ) NS ( ) NR

20 ¿Quiénes te leen o cuentan cuentos en la casa? Marcar todas las que apliquen ( ) Los dos padres ( ) Mamá ( ) Papá ( ) Hermano-a ( ) Abuelo-a ( ) Tío-a ( ) Pariente ( ) Otro-a ______( ) NS ( ) NR 21 Dedicas tiempo en tu casa para estudiar? Sí ( ) No ( ) NR ( )

22 Qué actividades realizas en tu casa cuando regresas de la escuela? ( ) hacer tareas

( ) leer ( ) jugar ( ) cocino para la familia ( ) trabajo fuera de la casa para ganar dinero ( ) cuido a mis hermanos ( ) Ninguna ( ) otro______( ) NS ( ) NR V Prácticas de higiene y alimentación 22 ¿Te están dando comida en la escuela? ( ) Sí ( ) No Pase a P.24 ( ) NR Pase a P.24 23 ¿Cuántos días a la semana comés en la escuela? ( ) todos los días ( ) tres a cuatro días

( ) uno o dos días ( ) Ninguno ( ) NS ( ) NR

24 Cuando comés en la escuela, ¿Quedás muy lleno, lleno, sin hambre, con hambre o con mucha hambre?

( ) muy lleno ( ) lleno ( ) sin hambre ( ) con hambre ( ) con mucha hambre 25 ¿Conoces alguna medida de higiene personal para estar sano? Puede repetir la pregunta de manera más sencilla si dice que no entiende ( ) Sí ( ) No Pase a P.27 ( ) NS ( ) NR

26 ¿Cuáles medidas de higiene personal conoces? Marcar todas las que mencione ( ) Uso de letrinas o inodoro ( ) Lavarse las manos ( )Bañarse diario ( ) Cortarse las uñas frecuentemente ( ) Cepillarse los dientes ( ) Otros ______

27 ¿Cuándo te lavás las manos?

( ) Antes de comer ( ) Antes de servir la comida ( ) Antes de preparar la comida ( ) Después de ir a la letrina ( ) Otros ______

¡Muchas gracias por tu colaboración!

Appendix 3: Teacher Questionnaire

Proyecto Mejor Educación y Salud - MESA II Cuestionario de Docentes sobre educación y aprendizaje de los escolares

11 Consentimiento verbal Hola. Mi nombre es ______, estamos realizando para PCI- Nicaragua entrevistas en las escuelas sobre la educación y aprendizaje que han recibido los niños y niñas. Sus opiniones son importantes, apreciaríamos su participación. Esta información ayudará a mejorar las actividades de lecto escritura y el ambiente higiénico sanitario en el marco del Proyecto MESA con el PINE/MINED. Esta entrevista normalmente tarda _____ minutos.

Cualquier información que nos proporcione es estrictamente confidencial y no se mostrará a otras personas.

Su participación en esta entrevista es voluntaria y puede usted escoger no contestar cualquier pregunta individual o todas las preguntas. ¿Te gustaría participar?

¿Tiene alguna pregunta sobre lo que le dije?

¿Acepta ser entrevistado-a?: _____ Sí _____No - Termine la entrevista

Firma entrevistador: ______

Fecha aplicación Día______Mes______Año______

Hora de inicio: ______Hora finalización: ______

Proyecto Mejor Educación y Salud - MESA II Cuestionario de Docentes sobre educación y aprendizaje de los escolares I. Datos Generales de la Escuela Nombre de la escuela: ______Código único de la escuela: ______Turno: _____Matutino _____Vespertino Multigrado: ______Regular: ______Intercultural Bilingüe: ______

Comunidad:______Municipio:______Departamento:______II. Datos del docente 1 Sexo: _____ Mujer _____ Hombre 3. Edad: ______años cumplidos 2 ¿Cuál es su último nivel educativo aprobado? Marque la opción que aplica

( ) Bachillerato ( ) Técnico ( ) Maestro Normalista Primaria ( ) Universitario ( ) Maestría ( ) Otro_especifique______

3 ¿Estudia usted actualmente? ( ) Sí ( ) No - Pase a la 5

4 En caso de responder SI preguntar ¿Qué estudia? ______

5 ¿Cuál es su lengua materna? ______

6 Además de su lengua materna, ¿Qué otro idioma hablas? Marque todas las que aplican ( ) Español ( ) Kriol ( ) Miskitu ( ) Sumu- Ulwa ( ) Garífuna ( ) Rama ( ) Ninguna

7 ¿En qué lengua facilitas el aprendizaje en la clase? Marque todas las que aplican

( ) Español ( ) Creole ( ) Miskitu ( ) Sumu- Ulwa ( ) Garífuna ( ) Rama ( ) mezcla

III. Datos sobre Práctica y Experiencia docente 8 ¿En esta escuela, qué grado o grados impartes en este año escolar? Marque todas las que aplican

( ) Multigrado - Detallar la combinación de grados que atienden ______( ) 3ro Grado puro Otro grado? ( ) 4to ( ) 5to ( ) 6to

9 ¿Cuántos años tienes de impartir este grado o grados? ______10. ¿Cuántos años de Experiencia docente tienes? _____

En caso de ser menor a un año marcar 0 IV. Capacitaciones recibidas por el docente 12 ¿En los últimos 4 años, en qué temas has sido capacitado ? Leer y Marcar las respuestas Temas Sí No Comentarios a. Métodología de aprendizaje de la lecto escritura b. Estrategias para evaluar la lectura

c. Método FAS

d. Comprensión lectora

e. Animación lectora

f. Uso de material del medio para el aprendizaje de la lecto escritura g. Alimentación/ Seguridad alimentaria nutricional h. Higiene

Otro, especifique

13 ¿Cuáles de los temas de capacitación antes mencionados han puesto en práctica en el aula de clase? Leer y Marcar las respuestas Sí No

a. Métodología de aprendizaje de la lecto escritura

b. Estrategias para evaluar la lectura

c. Método FAS

d. Comprensión lectora

e. Animación lectora

f. Uso de material del medio para el aprendizaje de la lecto escritura g. Alimentación/ Seguridad alimentaria nutricional

h. Higiene

Otro, especifique

14 ¿Qué beneficios ha obtenido de las capacitaciones recibidas en su práctica pedagógica. Marque todas las que apliquen

( ) que puedo usar esos conocimientos en clase ( ) que siento que mis estudiantes aprenden mejor cuando les enseño usando lo que aprendí en la capacitación ( ) que me ayuda a dar mejor la clase Otro:______( ) NS ( )NR 14 ¿Qué beneficios han obtenido sus estudiantes como resultado de las capacitaciones recibidas? Marque todas las que apliquen

( ) están motivados e interesados en aprender /poner atención ( ) ponen más atención ( ) participan más en las clases Otro:______( ) NS ( )NR

15 De los temas recibidos, ¿qué dificultades ha tenido para mejorar el aprendizaje de lecto escritura de los estudiantes ? Marque todas las que apliquen

( )No tengo materiales interesantes para los niños ( ) Hay niños que no me ponen atención ( ) Me falta capacitación ( ) No he podido integrar a mi trabajo en el aula ( ) No sé como utilizarlos en mi clase Otro:______( ) NS ( )NR

V Materiales didácticos disponibles en el aula de clase 16 ¿Cuenta con material de apoyo didáctico para la clase de Lengua y Literatura? ( ) Sí ( ) No

17 ¿Con cuál material cuenta? ( ) textos ( ) guías didácticas ( ) malla curricular

( ) componedor ( ) cuadricula ( ) Otro, cuál? ______( ) cuentos infantiles ( ) textos informativos

18 ¿Cuántos estudiantes del grado que imparte tienen libro de texto de Lengua y Literatura ? todos ______( ) _____ la mayoría ( ) _____ la mitad ( ) _____menos de la mitad ( ) Otro, especifique ______VI Prácticas relacionadas al desarrollo de la Lectura 19 ¿Pone a leer a los estudiantes durante la clase de Lengua y Literatura?

( ) Sí ( ) No ( ) NS ( ) NR 20 ¿Le lee cuentos a los estudiantes en voz alta durante la clase de Lengua y Literatura? ( ) Sí ( ) No Pase P.23

21 ¿Cuántas veces lo hizo en la última semana de clase? ( ) Ninguna ( ) Dos Veces ( ) Tres veces ( ) Cuatro veces ( ) Cinco o más veces ( ) Otro especifique ______22 ¿Qué hace antes de empezar a leerle a sus estudiantes? NO LEA OPCIONES - Marque todas las que aplican

( ) les pregunto qué saben del tema ( ) identifico con ellos el título para que me digan de que se puede tratar para que me puedan seguir ( ) les entrego copia de lo que leeré para que me puedan seguir ( ) les digo que me pongan atención ( )No hago nada en particular ( )NS ( ) NR ( ) otro, especifique ______23 ¿Qué acostumbra hacer con los estudiantes cuando les termina de leer? Esperar respuesta

( ) Nada ( ) les hago preguntas ( ) les pregunto sobre los personajes de la historia ( ) les pido que me resuman lo que leyeron ( )NS ( )NR ( ) otro

:______24 ¿Asigna ejercicios o lecturas individuales a sus estudiantes como tareas durante la clase de Lengua y Literatura? ( ) Si ( ) No Pase a P. 25

25 ¿Con qué frecuencia?

( ) Algunas veces ( ) Frecuentemente ( ) Todos los días ( ) Otro

______26 En Lengua y Literatura ¿deja tareas para que se hagan en casa? ( ) Si (

) No - Pase P.29 27 ¿Con qué frecuencia?

( ) Algunas Veces ( ) Frecuentemente ( ) Todos los días ( ) Otro ______

28 ¿Le alcanza el tiempo para revisar la tarea de Lengua y Literatura de todos sus estudiantes? ( ) Sí ( ) No

29 ¿Les corrige y explica sus errores? ( ) Sí ( ) A veces ( ) No ( ) NR 30 Cuando los

estudiantes tienen buenos resultados en pruebas o tareas, ¿qué hace?

( ) nada ( ) les digo que salieron bien ( ) les felicito ( ) aplaudimos ( ) otro especifique: ______31 ¿Se apoya en algún juego para mejorar el aprendizaje de la lectura y escritura en su grado?

( ) Sí ¿Cuál? ______( ) No ( ) NR 32 ¿Se reúne con otros docentes de la escuela para discutir problemas de aprendizaje de los estudiantes?

( ) Sí ( ) A veces ( ) No ( ) Nunca ( ) No aplica cuando es único en la escuela 33 ¿Qué considera Usted se podría hacer para mejorar el aprendizaje de los niño-as en la escuela?

VII Participación de padres y madres en vida escolar 34 ¿Participan los papas y las mamas de los niños en actividades de la escuela? ( ) Sí ( ) No Se concluye la entrevista

35 ¿En qué participan ellos/ellas? ( ) Actividades limpieza ( ) Escuelas de Padres

( ) Preparan la merienda escolar ( ) Llegan a recibir boletines ( ) Sustituyen maestro/as cuando estos se enferman ( ) Otro ______

¡Muchas gracias por su colaboración!

Appendix 4: Learning Environment Observation Guide

Proyecto Mejor Educación y Salud - MESA II Guía de Observación Ambiente de Aprendizaje

Introducción El objetivo de este instrumento es conocer las condiciones del ambiente de aprendizaje de la lectoescritura en la escuela a fin de planificar acciones orientadas a apoyar el mejoramiento del aprendizaje. I. Datos Generales de la Escuela Nombre de la Escuela: ______Código único de la escuela: ______. Comunidad: ______Municipio: ______Departamento: ______Grado observado: ______Ubicación de la escuela: Urbana: _____ Rural: ______Modalidades: Puro: ______Multigrado: ______Fecha de la observación: ______II. AMBIENTE DE APRENDIZAJE El aula de 3er grado observado cuenta con: Marcar todas las que apliquen 1 ( ) Rincón de Lectura ( ) Juegos didácticos ( ) Láminas didácticas Lecto escritura ( ) Mini-biblioteca ( ) Murales Promoción lectura ( ) Libros de textos Lecto escritura ( ) Componedores ( ) Aulas ambientadas ( ) Materiales dibujo ( ) Tiza o marcadores ( ) Cuentos ( ) Pizarra -detallar tipo- ______( ) Móviles colgando del techo Otro:______2 ¿Los niños utilizan los libros que están disponibles en el aula? ( ) SI ( ) NO ( ) NA 3 ¿Hay basura en el aula? ( ) Sí ( ) No ¿Qué tipo? ____plástica ___ papeles ____lodo otro______4 ¿Existen suficientes pupitres en el aula? ( ) Sí ( ) No 5 ¿Están los pupitres de los estudiantes en buen estado? ( ) Sí ( ) No 6 ¿Hay pupitres rotos en uso? ( ) Sí ( ) No 7 ¿Hay pupitres rotos sin usar al fondo del aula? ( ) Sí ( ) No 8 ¿Hay estantes para libros en el aula? ( ) Sí ( ) No

II. ELEMENTOS SANITARIOS E HIGIENICOS DE LA ESCUELA 9 ¿Los estudiantes usan la estación de lavado de mano? ( ) Si ( ) No ( ) No hay estaciones de lavado de mano 10 ¿Se sirve merienda escolar en la escuela? ( ) Si ( ) No

11 ¿Los estudiantes se lavan las manos con agua y jabón antes de la merienda escolar?

( ) Si ( ) No

12 ¿Los estudiantes usan la letrina o inodoro? ( ) Si ( ) No ( ) No hay letrina o inodoro

13 ¿Están aseadas o limpias las letrinas? ( ) Si ( ) No ( ) No aplica

14 ¿Hay basura tirada en los patios? ( ) Si ( ) No

15 ¿Hay basura tirada en los corredores? ( ) Si ( ) No

Appendix 5 – Global Indicators Collected in Baseline Study

Indicator Male Female Total Percentage of students who, after completing three Numerator 81 129 210 elementary school grades, demonstrate that they Denominator 409 463 872 can read and understand the meaning of texts at Result their level (MGD # 26). 19.8% 27.9% 24.1% Percentage of teachers in target schools who Numerator 9 13 22 demonstrate improvement in literacy teaching as Denominator 11 41 52 identified by their supervisors12 (custom MGD # Result 1.1). 81.8% 31.7% 42.3% Percentage of students from target schools who Numerator 38 23 61 indicated "hungry" or "very hungry" during the Denominator 409 463 872 school day (custom MGD # 1.2.1). Result 9.3% 5.0% 7.0% Numerator 276 319 595 Percentage of schools whose students wash their Denominator 409 463 872 hands before eating as a result of USDA support Result 67.5% 68.9% 68.2% (custom SO # 2). Schools 94.9%13 Percentage of students who can identify at least Numerator 179 214 393 three good hygiene and health practices (custom Denominator 409 463 872 MGD # 2.1). Result 43.8% 46.2% 45.1%

12 Based on questions 11 and 12 of the teacher questionnaire "In the last four years, teachers have been trained and put into practice in the classroom the following topics: Strategies to evaluate reading, FAS Method and Reading Comprehension. 13 In 37 of the 39 schools students indicated that they wash their hands before eating.

Appendix 7: Additional Tables and Graphs for EGRA Results

Table 1. Oral Comprehension - Percentage of Questions Answered Correct

Oral Comprehension Questions Male Female Total 1.1 ¿Qué le pasaba a Mariela? 52.8% 49.6% 51.1% 90.5% 90.0% 90.2% 1.2 ¿Qué quería comer? 1.3 ¿Dónde halló Mariela una flor chiquita? 46.9% 44.6% 45.7% 1.4 ¿Cuál fue el susto de Mariela? 55.3% 53.0% 54.1% 1.5 ¿Por qué se quedó con hambre Mariela? 57.9% 54.8% 56.3%

I.5.1.1 Graph 1. Oral Comprehension – Percentage of Correct Answers by Gender

Porcentaje de Respuestas Correctas de Comprensión Oral 100 % 90.5 % 90.0 % 90 % 80 % 70 % 57.9 % 60 % 52.8 % 55.3 % 53.0 % 54.8 % 49.6 % 50 % 46.9 % 44.6 % 40 % 30 % 20 % 10 % 0 % Hombre Mujer 1.1 ¿Qué le pasaba a Mariela? 1.2 ¿Qué quería comer? 1.3 ¿Dónde halló Mariela una flor chiquita? 1.4 ¿Cuál fue el susto de Mariela? 1.5 ¿Por qué se quedó con hambre Mariela?

I.5.1.2 Table 2. Success in identification Beginning Sound by Gender

% Beginning Sound Correct Male Female Total 1. "¿Cuál es el sonido de la palabra "mamá" 65.0% 66.9% 66.0% 72.1% 73.8% 73.0% 2. "¿Cuál es el sonido de la palabra "Sala" 3. "¿Cuál es el sonido de la palabra "Amor" 67.5% 71.0% 69.3% 4. "¿Cuál es el sonido de la palabra "tamal" 31.1% 28.8% 29.9% 5. "¿Cuál es el sonido de la palabra "cielo" 70.7% 72.1% 71.4% 6. "¿Cuál es el sonido de la palabra "foca" 65.0% 65.2% 65.1% 7. "¿Cuál es el sonido de la palabra "luna" 49.1% 49.4% 49.3% 8. "¿Cuál es el sonido de la palabra "bueno" 33.7% 34.8% 34.3% 9. "¿Cuál es el sonido de la palabra "pecho" 38.9% 35.9% 37.3% 10. "¿Cuál es el sonido de la palabra "queso" 31.5% 31.6% 31.6%

Table 3. Questions and Percentage of Correct Answers in Reading Comprehension

Questions – Reading Comprehension Male Female Total 1. ¿Quién cantaba en el patio de Rosita? 77.0% 78.1% 77.6% 2. ¿Qué quería hacer Rosita? 63.6% 61.9% 62.7% 3. ¿Qué asustó a Rosita? 37.9% 42.2% 40.2% 4. ¿Por qué cantaba el ave? 26.9% 26.0% 26.4% 5. ¿Por qué volvía el ave todos los días? 23.5% 28.8% 26.3%

I.5.1.3 Graph 6. Percentage of correct answers in Reading Comprehension Porcentaje de Respuestas Correctas de Comprensión Lectora 90 % 78.1 % 80 % 77.0 % 70 % 63.6 % 61.9 % 60 % 50 % 42.2 % 37.9% 40 % 28.8% 26.9 % 26.0% 30 % 23.5 % 20 % 10 % 0 % Hombre Mujer 1 . ¿Quién cantaba en el patio de Rosita? 2 . ¿Qué quería hacer Rosita? 3 . ¿Qué asustó a Rosita? 4 . ¿Por qué cantaba el ave? 5 . ¿Por qué volvía el ave todos los días?

Appendix 8: Supervisor and Enumerator Team

First Name Last Name Position

1 Fanor de los Angeles Padilla Rodriguez Enumerator

2 Clara Josefa López Enumerator

3 Cesar Enrique Guerrero Mendoza Enumerator

4 Yasmina Concepción Moraga Baldelomar Enumerator

5 Roberto Antonio Rivera Martínez Enumerator

6 Sander Givanni Guerrero Espinoza Supervisor/Enumerator

7 Nelson Echevarria Bustamante Supervisor/Enumerator

8 Nandys Antonio Tellez Supervisor/Enumerator

9 Emma del Socorro Mejía López Enumerator

10 Luis Humberto Jeréz García Enumerator

11 Juana Mercedes Aguilar Narvaez Supervisor/Enumerator

12 Nidya del Carmen Reyes Mendoza Supervisor/Enumerator

Appendix 10: Terms of Reference for the Consultancy

Terms of Reference (TOR) for the Food for Education Baseline USDA/FAS Grant Agreement: FFE-524-2017/025-00 Project MESA II Terms of Reference (TOR) for the Food for Education Baseline USDA/FAS Grant Agreement: FFE-524-2017/025-00 Project MESA II

Project Concern International (PCI) implemented the Food for Education (FFE) Project MESA in Nicaragua from September 27, 2013 to September 30, 2017. PCI has received funding from The United States Department of Agriculture (USDA) to implement a new phase of the FFE program in Nicaragua, called Better Education and Health (Mejor Educación y Salud II), MESA II from 2017 to 2022. These terms of reference have been prepared in order to solicit offers from consultancy firms to carry out the baseline study for this program. The proposal document should include a technical as well as financial proposal.

I. Summary and Background of the FFE Program

Name of the Program: Food for Education Project MESA II (Better Education and Health) – FFE

Name of Grantee: Project Concern International (PCI) Nicaragua

Name of the Donor: USDA

Implementing Partners: Ministry of Education (MOE), PINE-MOE and CEDEHCA, covering 4 municipalities in Jinotega Department and 7 municipalities in the South Caribbean Coast Autonomous Region (RACCS)

State Date: September 26, 2017

Completion Date: September 30, 2022

In partnership with PINE-MOE (the Government of Nicaragua’s Program for Integrated Nutrition in Schools managed by the Ministry of Education), CEDEHCA (project implementing partner), and local communities, PCI will implement MESA II, a five-year, USDA funded, integrated school feeding project with the goal of improving literacy of school-age children in eleven municipalities in the Departments of Jinotega and the South Caribbean Coast Autonomous Region (RACCS) in Nicaragua.

MESA II directly supports USDA’s vision of an integrated program that combines both health and educational interventions, responding to the project’s strategic objectives and intermediate results – improved literacy of school-age children (MGD SO1), improved quality of literacy instruction (MGD 1.1), improved attentiveness (MGD 1.2), improved student attendance (MGD 1.3) and increased use of health, nutrition and dietary practices (MGD SO2). (See Project MESA II’s Results Framework in Annex III).

Project MESA II will reach a total of 1,115 schools (including 986 primary schools) across eleven municipalities spanning Jinotega and RACCS, directly benefitting 101,830 students, parents, teachers and government officials and indirectly benefitting an additional 310,000 family members. Project activities include: provision of school feeding, teacher trainings, distribution of school supplies and materials, distribution of books and other teaching and learning materials, construction and/or improvement of school infrastructure, establishment of school gardens and orchards, training for parent-teacher association, training in food preparation and storage practices, rehabilitation of water systems, rehabilitation of latrines, distribution of deworming medication (in collaboration with the Ministries of Health and Education), training in good health and nutrition practices, raise awareness on the importance of education, establishment of savings groups, and institutional strengthening.

II. Purpose and Scope of the Baseline

PCI will hire an independent research organization or university-based research group to conduct a baseline study that will refine project baseline values, establish baseline information for comparison at midterm and endline, refine targets for project indicators in accordance with the findings, identify opportunities and threats to project implementation, develop strategies to maximize strengths and mitigate challenges, assess beneficiaries’ knowledge and skills to refine capacity strengthening activities, generate data for comparative analysis across the life of the project to measure change, and validate project strategies and assumptions, including those related to sustainability readiness. As per USDA’s Evaluation Policy, the baseline will be completed during the first six months of the project and prior to starting activities.

2.1 Methodology

PCI will utilize a mixed-method, cross-sectional design that includes quantitative and qualitative methods. Because MESA II will support all schools in the Jinotega and RACCS regions and because of government restrictions on accessing schools outside of the MESA II program, a control group is not possible for the evaluation. The quantitative data collected through representative surveys will establish baseline values for all project indicators against which results at the midterm and final evaluation can be compared and also contribute to understanding the context for project beneficiaries. Qualitative methods will provide additional information to understand the program’s operating context, triangulate quantitative findings, strengthen credibility and validity and provide a greater level of detail and contextualization of findings. 2.1.1 Sampling strategy To conduct the baseline evaluation, evaluators will select a representative random sample of program participants from the 986 primary schools participating in MESA II to achieve a 95% confidence level and a 5% confidence interval, an 80% sample power14 and a design effect of 2. Schools will be divided geographically by region and separate statistically significant samples will be taken from Jinotega and RACCS. Within each region, PCI proposes a two-stage cluster sampling strategy. At the first cluster, schools will be randomly selected from within PCI’s 13 areas of supervision. At the second level, students will be randomly selected from within the schools, both for implementation of a student survey (with fourth to sixth graders) and the early grade reading assessments (EGRA) with third graders. In addition to the student surveys, PCI will attempt to interview the parents with children currently attending sampled schools for the survey as well as one teacher from each school to

14 PCI will work with the consultant to select an indicator from the phase 1 project to determine sample power.

obtain both quantitative data from a structured survey instrument and qualitative data from semi- structured interviews.

In addition to the two-stage cluster sample, the sample will be stratified to include implementation of the early grade reading assessments (EGRA15) with a statistically significant number of students who have completed two grades of primary schooling.

The number of schools selected at the first cluster and the number student respondents per school selected at the second cluster will be proportional to the populations of the supervision areas and number of students enrolled at the school, respectively. Schools and respondents will be selected randomly using interval sampling to minimize sampling bias and help ensure the validity of the results, with the caveat that no more than one student will be interviewed from the same household. The number of students and schools sampled in each region will be reviewed after data on school enrollment has been obtained from all schools to take into account the size of the student body at schools in each cluster. The sample should also be stratified to compare educational outcomes of children with parents who participate in Women Empowered (WE) savings groups with communities which have no groups. It is expected that the consultant will finalize the sampling strategy in consultation with a statistician and the PCI/International Office (IO) M&E Technical Advisor, and will calculate adequate sample sizes in respondent groups for quantitative indicators that allow the detection of statistically significant changes between baseline, midline and final data collection points. 2.1.2 Data collection The focus of the baseline should be participatory and involve the following key stakeholders, proposed data collection methodologies and examples of key data collected:

Beneficiary Group Proposed Data Key Data Collected Collection Methodologies Elementary school children Student Survey Enrollment and regular attendance in classes from 4th to 6th grade Absences due to illness Frequency of school snacks, by week Satisfaction of the amount of food in the school snack Anti-parasite treatments Access to study materials (books, etc) Health and hygiene knowledge Elementary school children EGRA test Phonemic knowledge from 3rd grade Oral reading ability Reading comprehension Word decoding ability Letter sound knowledge

15 The Early Grade Reading Assessment (EGRA) is an individually administered oral assessment of the most basic foundation skills for literacy acquisition in early grades (https://www.eddataglobal.org/reading/)

Elementary school teachers Teacher Survey Student attentiveness In-depth Interviews Access to sanitary facilities for students Hand washing infrastructure and practice in schools Knowledge and practice of teaching methods Participation and involvement of PTAs, municipal governments, and public and private sector in the schools School parents Parent Survey Focus Knowledge of the benefits of primary education Group Good practice in health, nutrition and hygiene Good practice in food preparation and storage Organization and participation of parents in PTAs and School Feeding Committees (CAEs) in project strategies Officials and/or staff of the In-depth Interviews Management of MOE-PINE strategies (school Integral School Nutrition feeding, SAN training, school gardens and Program – MINED infrastructure) (PINEMINED) Perspective on cooperation with school organizations, with a focus on sustainability Gender perspective in the classroom Learning and recommendations for improving school feeding programs Parent-Teacher Associations Focus Group School feeding, training, behavior change, school (PTAs) Discussions gardens, school infrastructure, PTA and CAE organization and sustainability. Gender perspectives Lessons learnt and recommendations Local government officials and In-depth Interviews Appropriateness, efficiency and effectiveness of local private organization the program officials Sustainability of program results and impact Learning and recommendations for improving school feeding programs PCI and CEDEHCA officials or In-depth Interviews Appropriateness, efficiency and effectiveness of technical staff the program Perspective on cooperation with school organizations, with a focus on sustainability Gender perspective in the classroom Learning and recommendations for improving school feeding programs

USDA Nicaragua officials and In-depth Interviews Appropriateness, efficiency and effectiveness of USDA Washington officials the program Perspective on cooperation with school organizations, with a focus on sustainability Gender perspective in the classroom Learning and recommendations for improving school feeding programs

A standardized survey instrument for students will be developed that will focus on key performance indicators such as literacy, attentiveness, absenteeism, health and hygiene practices, hunger, and literacy using the EGRA methodology. Separate instruments will be developed to use with parents and teachers that focus on key performance indicators around knowledge of education and health, hygiene and nutrition practices. The survey instruments will also be used during midterm and final evaluations so that results can be compared across time and changes in results for the key performance indicators can be identified.

In addition to the student quantitative survey, data collection activities include in-class teacher observations, school administrator in-depth interviews, school checklists to review school records and infrastructure, parent interviews, and application of the sustainability readiness assessment tool at each sampled school. The development of all instruments will be guided by the instruments used in MESA I so that data for the same schools can be compared over a longer period.

The baseline study team will be responsible for collecting both primary data and reviewing secondary source data to establish/validate baseline values and targets for all project indicators16. Where deemed appropriate and necessary, they will make recommendations for refining targets. PCI’s graduation readiness assessment tool will be tested and validated, in order to be finalized and implemented throughout the project period to help inform each school’s sustainability action plan as well as the program’s overall approach to sustainability. 2.1.3 Data Analysis For analysis of the quantitative data it is recommended that statistical packages such as SPSS, STATA, or Excel be used. The results should be presented in tables and graphs with the corresponding interpretation. Baseline results for indicators with mean data should be presented with confidence intervals and confidence levels, noting that chi-squared and/or t-tests of significance will be required during midterm and final evaluations. The minimum detectable difference for EGRA scores based on an 80% sample power will also need to be presented. For analysis of the qualitative data, it is suggested that qualitative analysis techniques such as content analysis be used to capture key topics and determine the frequency of the themes.

2.2 Measurement of early grade reading outcomes

PCI will use the EGRA at baseline to measure changes in early grade reading outcomes. EGRA is a standardized method of measuring early grade literacy involving an oral assessment administered one-onone with children. In addition to giving a picture of student skills, the EGRA also provides a measure of the effectiveness of teacher training and support activities. Each student participating in the EGRA is asked to complete five tasks:

16 Secondary data sources include data from MINED and performance data and reports from MESA I

• Task 1: Phonemic Awareness – The student is read aloud a set of words and asked to identify which word began with a different sound; • Task 2: Letter Sound Knowledge – The student is given a page of letters from the Spanish alphabet and asked to identify the sounds (not the names) of as many letters as possible in 60 seconds; • Task 3: Invented Word Decoding – The student is given a page of “made-up” words and asked to read as many as possible in 60 seconds; • Task 4: Oral Passage Reading – The student is given a short reading passage and asked to read as much of it as possible in 60 seconds; and • Task 5: Reading Comprehension – The student is asked up to five reading comprehension questions relevant to the passage read.

Students are scored on each task on a scale of 0% to 100%, and their total EGRA score is an average across all five tasks.

2.3 Gender analysis

As part of the baseline assessment, the selected consultancy will conduct a gender analysis to ensure that staff and stakeholders understand the gender findings that are most likely to impede or support achievement of the project’s goal. The approach will enable staff and stakeholders to focus on how the project may impact gender relations within schools, homes, and communities and how the project can mitigate potential negative impacts. The findings of the gender analysis will be incorporated into project planning, implementation and strategies. Building on PCI’s experience conducting gender analyses, particularly in Guatemala and Malawi (USAID FFP), PCI will ensure that the gender analysis is participatory, builds staff capacity for gender analysis, and that the results are practically applied and regularly evaluated.

The key objectives of the gender analysis17 are to:

x Evaluate different roles, status, and power dynamics of women/ girls and men/boys within the household, community, and school ecosystem x Review relevant literature x Identify the most prevalence social norms impeding equitable education impacts to inform behavior change strategies x Determine how any identified gender issues or gaps may affect project activities or project outcomes x Determine, beyond gender parity, how the project can ensure gender equitable outcomes for both boys and girls in schools

17 Data on conducting a gender analysis, the key gender analysis questions listed and the gender integration checklist referenced can be found at: https://www.usaidassist.org/sites/assist/files/checklist- integratinggenderthroughouthciprojectphases.pdf

III. Evaluation Questions

3.1 Program Design and Relevance: x To what extent are key project activities and strategies effective in accomplishing proposed results? x What are the main challenges to achieving improved early grade literacy outcomes in the intervention schools? x To what degree have the needs of the beneficiary population been included in the design of the program? 3.2 Sustainability: x To what extent are project sustainability strategies designed to be presented to key stakeholders and allow them to internalize and carry it out (e.g. the emphasis on transferring the school feeding program to local government, private partners, and PINE MOE; are certain elements of the program being promoted; ¿are guaranteeing compliance with certain specific results for education and health, etc.)? What lessons learned are available from previous phases of the project and other similar successful projects to promote and achieve greater sustainability? x To what degree does the project expect key local actors (private sector, community entities/associations, local government agencies, etc.) to take ownership for sustaining the program activities (as demonstrated by physical, resource and in-kind contributions)? x What specific institutional capacity needs, e.g. for PINE MOE should the project address to foster greater host country engagement and accountability? x What elements or considerations should be incorporated into public policy in the municipalities to sustain a school feeding program? x Is the Sustainability Readiness Assessment Tool, designed by PCI, an effective way to measure sustainability readiness and graduate schools? What recommendations do you have to improve the tool?

IV. Duration of the Consultancy

The consultancy should last no longer than 70 business days, specifically between February and May 2018. The table below illustrates the main activities with a suggested number of days for each one.

Activity Responsible Number of days Review of program documentation, results framework, PCI Nicaragua and 10 materials, planning of field work (office). This includes meetings IO M&E staff and with PCI representatives to review instruments, methodology, consultant team time lines, coordination and delivery of documentation. Incorporation of PCI observations and feedback into evaluation Consultant team 10 instruments; standardization of consultant’s technical field team (PCI staff will be present for the standardization) Initial field visits (meeting with local PCI staff), validation and PCI staff, 5 final editing of the data collection instruments (with PCI staff Consultant team participation)

Field work. This includes an exit meeting with key PCI staff to PCI staff, 22 review, clarify, amplify or give feedback on findings or key Consultant team results. Data analysis/preparation and delivery of first draft of report Consultant team 20

Results presentation workshop/final feedback PCI, Consultant 1 team Revision and presentation of final report (including the Consultant team 2 observations of the PCI review team) Total estimate of work days 70

Note. In May 2018, the baseline report will be presented to USDA, including an updated indicator table and the Performance Monitoring Plan (PMP). V. Selection of the evaluation team

The external evaluation team will be selected through a competitive procurement process based on professional competency, experience with the evaluation tasks, independence from the project, avoidance of conflict of interest, experience and knowledge of school feeding programs and understanding of Nicaragua, the RACCS and Jinotega municipalities and programming designed to improve literacy.

VI. Proposal Submission Applicants are requested to submit a proposal for these services by submitting separate Technical and Financial Proposals as detailed below. The standard forms in this proposal document may be retyped for completion but the applicant is responsible for their accurate reproduction. Required formats are hereby attached:

x Appendix I (Technical Proposal) x Appendix II (Financial Proposal)

Preparation of Technical Proposal

The Technical Proposal shall be submitted in the format provided in Appendix I and describe the approach and plans for accomplishing the work outlined in this RFP. The Technical Proposal should be limited to five (5) pages and must contain the following:

x Technical Proposal Cover Sheet (does not count towards the 5-page limit) x A minimum of three letters of reference from organizations with which they have previously conducted evaluation consultancy work or a list of three references with contact information (does not count towards the 5-page limit) x A description of the applicant’s direct prior relevant experience including summary of evaluations previously conducted x Description of Methodology x Explain how the evaluation design addresses the research questions x Sampling strategy

x Approach to primary data collection x Plan for data analysis x Timeline for Activity x Include a timeline of activities and level of effort required for each activity x Team Composition and Roles, including: x Key Team Member List and Core Qualifications x CVs of Key Team Members (does not count toward the 5-page limit) x A minimum of two examples of evaluation reports that have been produced by the applicant (does not count towards the 5-page limit) Preparation of Financial Proposal

The Financial Proposal shall be submitted in the format provided in Appendix II and establish a breakdown of consultancy price according to the table provided.

Applicants are solely responsible for their own costs in preparing the applications

Requirements for Individual Consultant/Consulting Firm x Academic background in social sciences, education, public health or related field with a preferred minimum qualification of Masters. x Experience in designing and conducting evaluations using both quantitative and qualitative techniques. x Knowledge and experience in appropriate sampling strategies and statistical analysis x Experience in conducting evaluations in the areas of social services and international development, preferably with experience in evaluation of school feeding programs x Experience with administering the EGRA tool and analyzing the collected data with concise results and presentation18 x Experience in administering gender analysis tools and analyzing the collected data x Experience with USDA programs or other similar United States government program x Proven capacity to deliver high quality results within the proposed timeframe x Experience conducting evaluation in Central America, including Nicaragua. x Ability to produce a high-quality report in Spanish for English translation. x Must agree to submit all data and not share data without PCI’s permission.

18 EGRA implementation skills and experience must exist within the consultant team, potentially through a subcontracting arrangement