Evaluation of Formative Continuous Assessment Pilot in Amhara and Tigray Regions
Total Page:16
File Type:pdf, Size:1020Kb
Evaluation of Formative Continuous Assessment Pilot in Amhara and Tigray Regions September 2018 A partnership with American Institutes for Research Contract No. AID 663-C-15-00001 USAID Reading for Ethiopia’s Achievement Developed Monitoring and Evaluation (READ M&E) Evaluation of Formative Continuous Assessment Pilot in Amhara and Tigray Regions Main Authors: Belay Tefera, Belen Mekonnen, Daniel Tefera, Solomon Areaya Additional Contributors: Mohammed Elmeski, Rebecca Stone, Mariela Taylor Prepared by: American Institutes for Research 1000 Thomas Jefferson Street, NW Washington, DC 20007 Submitted to: U.S. Agency for International Development/Ethiopia Addis Yigzaw, COR Under Contract No. AID-663-C-15-00001 Disclaimer: This report was made possible by the support of the American People through the United States Agency for International Development (USAID). The contents of this report are the sole responsibility of the American Institutes for Research (AIR) and do not necessarily reflect the views of USAID or the United States Government. Table of Contents Executive Summary ..................................................................................................................... 1 Introduction .............................................................................................................................. 1 Findings and Discussions ......................................................................................................... 1 Conclusions and Recommendations ......................................................................................... 4 I. Introduction ............................................................................................................................... 5 1.1. Background ....................................................................................................................... 5 1.2. The FCA Project ................................................................................................................ 5 1.3. Description of FCA Tools ................................................................................................. 6 II. Evaluation Methods .............................................................................................................. 10 2.1. Research Objectives ........................................................................................................ 10 2.2. Study Design ................................................................................................................... 10 2.3. Sampling of Respondents ................................................................................................ 12 2.4. Data Collection Procedures ............................................................................................. 13 2.5. Coding and Data Analysis ............................................................................................... 14 2.5.1. Qualitative Data Analysis ....................................................................................... 14 2.5.2. Quantitative Data Analysis ..................................................................................... 14 III. Findings and Discussions .................................................................................................... 14 3.1. FCA Tools: Availability, Training and Use .................................................................... 14 3.1.1. Tool Availability and Personnel Training .............................................................. 15 3.1.2. Use of FCA Tools ................................................................................................... 17 3.1.3. Assessment, Record-keeping, Feedback, Follow-up, and Remedial Measures ..... 20 3.2. Contributions of FCA Training and Tools ...................................................................... 26 3.2.1. Teachers’ Improved Understanding of FCA’s Role ............................................... 27 3.2.2. Changes in Students’ Learning and Achievement ................................................. 28 3.2.3. Cascading and Sustainability.................................................................................. 30 3.3. Challenges, Opportunities and Measures ........................................................................ 31 3.3.1. Challenges .............................................................................................................. 31 3.3.2. Opportunities .......................................................................................................... 34 3.3.3. Participants’ Suggestions for Enhancing FCA ....................................................... 36 IV. Conclusions and Recommendations ................................................................................... 39 4.1. Conclusions ..................................................................................................................... 39 4.2. Recommendations ........................................................................................................... 40 References ................................................................................................................................... 42 Annexes Annex 1: List of Names of Regions, Woredas and Schools ..................................................... 45 Annex 2: Common Themes of the Interview Protocols ............................................................ 48 Annex 3: Tool 1: Classroom Observation Guide for READ M&E’s FCA .............................. 49 Annex 4: Tool 2: Classroom Observation Checklist for READ M&E’s FCA Intervention; for grades 3 &4 54 Annex 5: Tool 3: Group discussion with Teachers ................................................................... 59 Annex 6: Tool 4: Group Discussion Questions for Supervisors and Principals, ....................... 61 Annex 7: Tool 5: Interview Guide for Woreda Education Experts, who took the FCA training delivered by READ M&E ........................................................................................................... 63 Annex 8: FCA feedback communication form .......................................................................... 65 Tables Table 1: Key Evaluation Questions .......................................................................................... 11 Table 2: Sampling of Zones, Woredas, and Schools ................................................................ 13 Table 3: Availability of FCA Tools in Classrooms, Grades 1 to 4 ........................................... 15 Table 4: Availability of FCA Tools in Classrooms, Grades 3 and 4 ........................................ 15 Table 5: Number of FCA Trained Teachers in First Cycle Primary Schools ........................... 17 Table 6: Use of FCA Assessment Tools in Classrooms, Grades 1 and 2 ................................. 18 Table 7: Use of FCA Assessment Tools in Classrooms, Grades 3 and 4 ................................. 18 Table 8: Types of Skills Assessed by FCA Tools in Grades 1 and 2 ....................................... 20 Table 9: Types of Skills Assessed by FCA Tools in Grades 3 and 4 ....................................... 21 Table 10: Use of Record-keeping Tools during Observation of Tigrigna and Amharic Speaking Classrooms .................................................................................................................. 21 Table 11: Teachers’ Feedback during Classroom Observations ................................................ 23 Table 12: Teacher Remedial Strategies Used during Classroom Observation ........................... 24 Figures Figure 1. Sample Afaan Oromo flashcards ................................................................................. 7 Figure 2. Example of KWL ......................................................................................................... 8 Figure 3. Example of Mind Map ................................................................................................. 8 Figure 4. Example of Web Map .................................................................................................. 9 Figure 5. Example of sequential graphic organizer ..................................................................... 9 Acronyms AIR American Institutes for Research AMPS Amhara Principal/ Supervisor AMT Amhara Teacher AMWE Amhara Woreda Office Expert CA Continuous Assessment EGRA Early Grade Reading Assessment ESDP Education Sector Development Program FCA Formative Continuous Assessment KWL Know, Want to Know, Learned MoE Ministry of Education MT Mother Tongue MTL Mother Tongue Language TGPS Tigray Principal/ Supervisor TGT Tigray Teacher TGWE Tigray Woreda Office Expert USAID United States Agency for International Development Evaluation of Formative Continuous Assessment Executive Summary Introduction Background: The Ethiopian education sector has documented remarkable strides in the last few decades. However, it has experienced equally significant challenges with education quality. These concerns of quality appear to be critically important for education during the formative years, where foundations are laid down for lifelong learning. The National Learning Assessment (NLA) results consistently show that Ethiopian children are unable to read at a grade-appropriate level because they did not learn to read by the time, they complete the first cycle primary school (MoE, 2008; RTI, 2010; Seid, 2015; Belay & Belay, 2018). Different actors have been working to address