NATIONAL ASSESSMENT SURVEY of LEARNING ACHIEVEMENT at GRADE 2 Results for Early Grade Reading and Mathematics in Zambia
Total Page:16
File Type:pdf, Size:1020Kb
NATIONAL ASSESSMENT SURVEY OF LEARNING ACHIEVEMENT AT GRADE 2 Results for Early Grade Reading and Mathematics in Zambia Prepared by RTI International March 2015 NATIONAL ASSESSMENT SURVEY OF LEARNING ACHIEVEMENT AT GRADE 2 Results for Early Grade Reading and Mathematics in Zambia Prepared for Teresiah Wambui Gathenya (PhD), Education Advisor Education Office, United States Agency for International Development (USAID)/Zambia U.S. Embassy Subdivision 694 / Stand 100 P.O. Box 32481 Kabulonga District, Ibex Hill Road Lusaka, District Zambia Prepared by Aarnout Brombacher, Jennae Bulat, Simon King, Emily Kochetkova, and Lee Nordstrum RTI International 3040 Cornwallis Road Post Office Box 12194 Research Triangle Park, NC 27709-2194 RTI International is a trade name of Research Triangle Institute. The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ACKNOWLEDGMENTS The authors wish to acknowledge partnership support from the British Department for International Development (DFID); the United States Agency for International Development (USAID); the Government of the Republic of Zambia, through its Ministry of Education, Science, Vocational Training and Early Education (MESVTEE); and the Examinations Council of Zambia. Many departments and staff from the Ministry of Education and the Examinations Council of Zambia also made important contributions to the development and implementation of this study and helped ensure that the instruments and methodologies were appropriately aligned to the Zambia context. Most importantly, this work could not have succeeded without the cooperation and contributions of the District Education Officers, head teachers, teachers, pupils, and communities included in the study who, for obvious reasons, must remain anonymous. The USAID funding was provided through an Education Data for Decision Making (EdData II) task order, Data Collection Services for the USAID/Zambia Education Project, Contract No. AID-611-M-14-00002 (RTI Task 28). Grade 2 NAS: Results for Early Grade Reading and Mathematics in Zambia | iii CONTENTS PAGE Acknowledgments ................................................................................................. iii List of Figures ........................................................................................................ vi List of Tables ......................................................................................................... ix Abbreviations ........................................................................................................ xii Executive Summary ............................................................................................... 1 1 Background and Purpose of the Study ......................................................... 4 1.1 Background .......................................................................................... 4 1.2 Research Questions ............................................................................. 5 1.3 learning outcomes Measured ............................................................... 5 1.4 Baseline Report .................................................................................... 6 2 Evaluation Approach and Design of the Assessment .................................. 7 2.1 Overview of EGRA ............................................................................... 8 2.1.1 Why Test Early Grade Reading? ................................................ 8 2.1.2 Purpose of EGRA ....................................................................... 8 2.1.3 What EGRA Measures ............................................................... 8 2.1.4 The EGRA Instruments for Zambia ............................................ 9 2.2 Overview of EGMA ............................................................................. 10 2.2.1 Why Test Early Grade Mathematics? ....................................... 10 2.2.2 Purpose of EGMA ..................................................................... 11 2.2.3 What EGMA Measures ............................................................. 11 2.2.4 The EGMA Instrument for Zambia ............................................ 11 2.1 Overview of Complementary instruments .......................................... 13 2.1.1 Complementary Instruments for Zambia .................................. 13 2.2 Instrument Adaptation Process for Zambia ........................................ 14 3 Methodology ............................................................................................... 16 3.1 Sampling Framework ......................................................................... 16 3.2 Descriptive Statistics .......................................................................... 16 3.3 Weighting ............................................................................................ 18 4 Results and Findings .................................................................................. 19 4.1 Instrument Reliability and Validity ...................................................... 19 4.2 EGRA Results .................................................................................... 19 4.2.1 Summary of Scores .................................................................. 20 4.3 EGMA Results .................................................................................... 34 iv | Grade 2 NAS: Results for Early Grade Reading and Mathematics in Zambia 4.3.1 Summary of Scores by Category .............................................. 36 4.3.2 Subtask Analysis ...................................................................... 40 4.4 Results from the Complementary Instruments ................................... 51 4.4.1 Learner Background and Characteristics ................................. 52 4.4.2 Teacher and Classroom Characteristics .................................. 55 4.4.3 Head Teacher and School Characteristics ............................... 62 4.4.4 Factors Associated with Learner Performance ........................ 69 5 Conclusions and Recommendations .......................................................... 81 5.1 Recommendations from the EGRA Study .......................................... 81 5.2 Recommendations from the EGMA Study ......................................... 81 5.3 Recommendations from the Complementary Instruments ................. 82 5.4 Recommendations from the Exercise to Calculate and Compare “Baseline” and Midterm Data .............................................................. 83 5.5 Policy Dialogue Workshop and Recommendations ........................... 83 5.6 Benchmarks and Targets ................................................................... 83 References ........................................................................................................... 84 Annex A: EGRA Results by Local Language Subtask, Including Standard Error Values and 95% Confidence Intervals .............................................. 86 Annex B: EGMA Results By Subtask, Including Standard Error Values And 95% Confidence Intervals .......................................................................... 98 Annex C. Baseline Comparison Report ............................................................. 100 Annex D. Report on the Equating Pilot Study .................................................... 105 Grade 2 NAS: Results for Early Grade Reading and Mathematics in Zambia | v LIST OF FIGURES PAGE Figure 1. Language distributions of scores on Listening Comprehension subtask (local languages) ........................................ 20 Figure 2. Score distributions of Letter Sound Identification .................... 24 Figure 3. Score distributions of Nonword Reading ................................. 24 Figure 4. Language distributions of scores on Oral Reading Fluency subtask (local languages) ..................................................... 27 Figure 5. Language distributions of scores on Reading Comprehension subtask (local languages) ........................................ 30 Figure 6. Performance on the EGMA subtasks ....................................... 35 Figure 7. Performance by gender on the EGMA subtasks ..................... 37 Figure 8. Urban and rural performance on the EGMA subtasks ............. 38 Figure 9. Provincial performance on the Missing Number and Addition and Subtraction Level 2 EGMA subtasks ............................ 39 Figure 10. Performance on the Missing Number and Addition and Subtraction Level 2 EGMA subtasks, by school type ................. 40 Figure 11. Item-level performance on the Number Identification subtask ............................................................................................... 41 Figure 12. Item-level performance on the Addition Level 1 subtask ............................................................................................... 43 Figure 13. Item-level performance on the Subtraction Level 1 subtask ............................................................................................... 44 Figure 14. Item-level performance on Quantity Discrimination subtask ............................................................................................... 46 Figure 15. Item-level performance on the Missing Number subtask ............................................................................................... 47 Figure 16. Item-level performance on