Readers are Leaders Project Early Grade Reading and Mathematics Assessment in the Republic of Macedonia: Study Report January 2016 USAID Readers are Leaders Project Study Report May 2015 Table of Contents: Acknowledgments .................................................................................................................... 10 List of Abbreviations ................................................................................................................ 11 Executive Summary ................................................................................................................. 12 Main Findings and Recommendations ................................................................................. 13 1. Project Background .......................................................................................................... 16 2. EGRA AND EGMA STUDY IN 2015 ............................................................................ 17 2.1 Sample design ............................................................................................................ 18 2.2 Instrument structure ................................................................................................... 20 3. Early Grade Reading Assessment .................................................................................... 21 3.1 The EGRA instrument for Macedonia....................................................................... 22 4. Early Grade Mathematics Assessment ............................................................................. 30 5. Fieldwork and Data Collection Process for EGRA & EGMA ......................................... 36 5.1 Training for Administration of EGRA and EGMA Instruments ............................... 36 5.2 Data collection ........................................................................................................... 37 5.3 Quality Assurance...................................................................................................... 38 5.4 Data analysis and reporting ....................................................................................... 40 6. EGRA & EGMA Baseline Findings ................................................................................ 40 6.1 Characteristics of Sample .......................................................................................... 40 6.2 How Well Are Students Reading in Macedonian and Albanian? ............................. 41 6.2.1 Results according to gender .................................................................................... 43 6.2.2 Results according to education of parents ............................................................... 47 6.2.3 Results according to type of school......................................................................... 49 6.2.4 Results according to attendance of pre-school ........................................................ 51 6.2.5 Results according to availability of books at home ................................................. 52 6.2.6 Results according to reading before going to school .............................................. 53 6.2.6 Results according to borrowing library books ........................................................ 55 6.2.7 Results according practicing reading at home ......................................................... 56 6.2.8 Results according to EGRA tasks ........................................................................... 58 6.3 How Well Are Pupils Doing Basic Mathematics? .................................................... 60 6.3.1 Overall test results according to gender .................................................................. 61 6.3.2 Results according to education of parents ............................................................... 63 6.3.3 Results according to language of instruction .......................................................... 64 6.3.4 Results according to type of school......................................................................... 66 6.3.5 Results according to attendance of pre-school ........................................................ 67 6.3.6 Results according to availability of books at home ................................................. 67 6.3.7 Results according to EGMA tasks .......................................................................... 68 7. EGRA & EGMA Longitudinal Findings ......................................................................... 73 8. FINDINGS FROM THE QUESTIONNAIRES ............................................................... 92 8.1 School Climate .......................................................................................................... 93 8.2 School Resources for Teaching Reading and Mathematics .................................... 107 8.3 Teacher Preparation ................................................................................................. 112 8.4 Classroom Instruction .............................................................................................. 116 8.5 Home Environment Support for Reading Achievement .......................................... 120 9. CONCLUSIONS ............................................................................................................ 134 9.1 Lessons Learned from EGRA AND EGMA application ........................................ 134 2 USAID Readers are Leaders Project Study Report May 2015 9.2 School Climate ........................................................................................................ 135 9.3 School Resources for Teaching Reading and Mathematics .................................... 136 9.4 Teacher Preparation ................................................................................................. 136 9.5 Classroom Instruction .............................................................................................. 137 9.6 Home Environment Support for Reading Achievement .......................................... 138 1. APPENDICES ................................................................................................................ 140 1.1 Appendix 1: List of schools participating in EGRA and EGMA studies ................ 140 1.2 Appendix 2: EGRA and EGMA longitudinal results .............................................. 142 1.3 Appendix 3: Questionnaire for School Director ...................................................... 152 1.4 Appendix 4: Questionnaire for Teachers ................................................................. 158 1.5 Appendix 5: Questionnaire for Parents ................................................................... 165 List of Tables: Table 1. Number of students per grade and language of instruction in the baseline sample ... 19 Table 2. Number of students per grade and language of instruction in the longitudinal sample .................................................................................................................................................. 19 Table 3. Example of Comprehension Questions Task in Macedonian language ..................... 25 Table 4. Example of Comprehension Questions Task in Albanian language .......................... 27 Table 5. Timing and stop rules for EGRA tasks ...................................................................... 29 Table 6. Timing and stop rules for EGMA tasks ..................................................................... 35 Table 7. Number of Grade 2 and 3 students attending pre-school ........................................... 51 Table 8. Number of Grade 2 and 3 students having additional books at home ....................... 52 Table 9. Number of Grade 2 and 3 students reading before starting school ............................ 54 Table 10. Number of Grade 2 and 3 students borrowing library books ................................... 55 Table 11. Number of Grade 2 students reading at home .......................................................... 56 Table 12. Number of Grade 3 students reading at home .......................................................... 57 Table 13. Student results on Letter Identification task ............................................................. 58 Table 14. Student results on Familiar Words Reading task ..................................................... 58 Table 15. Overall student results on Reading Fluency task ..................................................... 59 Table 16. Student results on Reading Fluency per task ........................................................... 59 Table 17. Overall student results on Reading Comprehension task ......................................... 60 Table 18. Student results on Reading Comprehension per task ............................................... 60 Table 19. Number of Grade 2 and 3 students attending pre-school ......................................... 67 Table 20. Number of Grade 2 and 3 students having additional books at home ..................... 67 Table 21. Overall student results on Number Identification task ............................................. 68 Table 22. Overall student results on Number Discrimination task .......................................... 69 Table 23. Overall student results on Missing Numbers task .................................................... 69 Table 24. Overall student results on Addition task .................................................................. 70 Table 25. Overall student results on Subtraction task