READ FOUNDATION - Baseline Assessment Study
USAID-funded Pakistan Reading Project 1 READ FOUNDATION - Baseline Assessment Study
READ FOUNDATION Baseline Assessment Study (Azad Jammu & Kashmir and Gilgit Baltistan) July, 2018
This report was made possible by the support of the American people through the United States Agency for International Development (USAID) funded Pakistan Reading Project. The contents are the sole responsibility of International Rescue Committee (IRC) and do not necessarily reflect the views of USAID or the United States Government.
USAID-funded Pakistan Reading Project 2 READ FOUNDATION - Baseline Assessment Study
Table of Contents
Glossary ...... 5 Acknowledgement ...... 6 Background ...... 7 Executive Summary ...... 8 Major Findings ...... 9 Study Methodology ...... 12 Baseline Questions ...... 12 Method ...... 13 Participants ...... 13 Languages Spoken by Children Tested in Urdu ...... 14 Instruments ...... 15 1. EGRA ...... 15 2. Teacher Interview ...... 17 3. Teacher Classroom Observation ...... 17 4. Student Interview ...... 19 Process ...... 19 Results ...... 22 What are the levels of literacy skills among grade 1 and 2 children in READ Foundations schools at baseline? ...... 22 Summary of EGRA Scores ...... 22 Zero Scorers of EGRA subtasks: ...... 24 EGRA Subtasks Scores Compared with National Performance Standards: ...... 25 Subtasks by Region ...... 28 Comparison of Regional Results with Drafted National Reading Standards ...... 29 National Reading Standards ...... 29 What are the current demographics and perceptions of teachers regarding teaching in early grades? .. 34 Like reading Urdu book: ...... 35 What teacher does when student made mistake: ...... 35 Provision of Library at School ...... 36 Limitations ...... 36
USAID-funded Pakistan Reading Project 3 READ FOUNDATION - Baseline Assessment Study
List of Table
Table 1.READ Foundation Sample Schools by district proportion ...... 13 Table 2 Number of assessed Students and observed Teachers by grade and region ...... 14 Table 3 EGRA Tool task wise and overall Reliability ...... 16 Table 4 Teacher competencies wise Mean Score and internal reliability statistics ...... 19 Table 5 Both grades students average Percentage scores for different subtasks ...... 22 Table 6 Grade wise % zero scorers for reading subtasks ...... 24 Table 7: Zero Scorer by grades for subtasks ...... 24 Table 8.EGRA Subtasks percentage mean scores and National Performance Standards ...... 25 Table 9: EGRA Untimed Tasks Result by Region ...... 28 Table 10: EGRA timed tasks scores ...... 28 Table 11: Student percentage meeting reading standards ...... 30 Table 12:Task wise students average percentage scores for both regions ...... 31 Table 13: Teacher academic qualification by region ...... 34 Table 14: Teacher professional qualification by region ...... 34
List of Figure
Figure 1: Percentage of language spoken by Students at home ...... 14 Figure 2: Grade 1 students reading skills results by gender ...... 23 Figure 3: Grade 2 students reading skills results by gender ...... 23 Figure 4: Un-Timed Tasks Mean Score by grade ...... 26 Figure 5: Timed Task Mean Scoresby grades ...... 26 Figure 6: Readers’ categorization by grades ...... 27 Figure 8: Grade wise average mean score and percentage of students meeting reading standards for different tasks ...... 31 Figure 9: Student average mean score for passage reading with national standards and percentage of students meeting standards ...... 33 Figure 10: Student average mean score and percentage of student meeting RC standard ...... 33 Figure 11: Teacher average competency score by region ...... 34 Figure 12: Student who like and dislike reading Urdu books Figure 13: Comparison between students like & dislike reading book ...... 35 Figure 14: Teachers actions towards students' mistake ...... 35 Figure 15: Students response on School Library ...... 36
USAID-funded Pakistan Reading Project 4 READ FOUNDATION - Baseline Assessment Study
Glossary
ACRONYMS
AJ&K Azad Jammu and Kashmir BHS Boys High School BPS Boys Primary School CRO Classroom Observer DDE Deputy Director Education DOE Department of Education DPM District Program Manager EGRA Early Grade Reading Assessment GB Gilgit Baltistan HT Head Teacher PRP Pakistan Reading Project RF READ Foundation WL World Learning
USAID-funded Pakistan Reading Project 5 READ FOUNDATION - Baseline Assessment Study
Acknowledgement
We would like to acknowledge the cooperation of READ Foundation’s schools’ management for their openness and welcoming attitude for the USAID-funded Pakistan Reading Project (PRP) to work for the improvement of Urdu reading with their grade one and two students as well as teachers.
PRP also wants to thanks Urdu language teachers and grade 1 & 2 students who participated in the study along with Head Teachers who facilitated PRP enumerators and classroom observers during data collection. Special thanks to the enumerators and classroom observers for collecting assessment data.
Furthermore, we would also like to acknowledge the role of each PRP department who directly or indirectly provided support to conduct this baseline study.
USAID-funded Pakistan Reading Project 6 READ FOUNDATION - Baseline Assessment Study
Background
The USAID-funded Pakistan Reading Project (PRP) was conceived by USAID and the Government of Pakistan to address the reading deficit in Pakistani schools. PRP aims to improve children’s Urdu and Sindhi reading skills in grades 1 and 2 and through pilot interventions improve reading in Pashto. During the seven years of program implementation, approximately 1.3 million children will be targeted for reading skills improvement. Project focus areas are: Azad Jammu & Kashmir (AJ&K), Balochistan, Federal Administered Tribal Areas (FATA), Islamabad Capital Territory (ICT), Khyber Pakhtunkhwa (KP), Gilgit Baltistan (GB) and Sindh. Applying an evidence- based and evidence-generating approach to teaching reading,
PRP will:
1. Deliver a robust in-service training model, including teacher inquiry groups, coaching and face-to-face training, that will help teachers effectively teach the five component skills of reading (phonemic awareness, phonics, vocabulary, fluency, and comprehension, with writing and print concepts integrated); 2. Develop rich, age appropriate children’s literature and supplemental learning materials to engage children in the learning process, spurring both immediate and lasting improvements in student reading ability; and 3. Build on Year 1 pre-service work in select teacher training institutes (TTIs) to strengthen the preparation of Pakistan’s teaching workforce, with a specific focus on reading, and to demonstrate to the Higher Education Commission how to take this work forward.
Improving policies and systems at the provincial and district levels to better enable teachers and students to succeed is also critical to reading achievement outcomes. PRP’s strategy to improve Pakistan’s policies and systems underpinning its national reading initiative is to align the many elements of the in-service and pre-service programs with policy discussions at the national, provincial, district, and school levels.
Based on practical experience and global evidence regarding the importance of community- based support to reinforce children’s learning and reading skills acquisition, PRP will solicit support from a variety of Pakistani NGOs through small community grants and a few larger grants focused explicitly on community-based reading activities in support of PRP’s goals.
USAID-funded Pakistan Reading Project 7 READ FOUNDATION - Baseline Assessment Study
Executive Summary
The USAID-funded Pakistan Reading Project (PRP), implemented by the International Rescue Committee (IRC) and its partners (Creative Associates International, World Learning, and Institute of Rural Management), is a seven-year project with the objective to support the provincial and regional Departments of Education (DoE) and low cost private schools throughout Pakistan to improve reading skills of children in grades one and two.
PRP is interested in exploring cost effective and sustainable options for improvement of reading in the country other than the public sector schools to expand its intervention and increase number of beneficiaries. For this purpose, PRP has initiated the process to coordinate with the management of several foundations working across the country through low fee foundation schools.
PRP successfully negotiated with READ Foundation a collaborative project to implement its reading model in their schools in AJK and GB from second quarter of FY 2016-17. As part of this initiative, a baseline study was carried out in May 2017. Trained enumerators administered EGRA assessments using tablets to a total of 483 students from grade 1 (B=116, G=118) and grade 2 (B=132, G=117) from 39 mixed gender Urdu-medium schools in four districts of AJK (Bagh, Bhimber, Muzaffarabad, Kotli, ) and two districts of GB (Gilgit, Astore). Enumerators also observed a total of 59 Urdu language teachers (22 grade I, 24 grade II and 13 grade I and II) while they were conducting their classes.
USAID-funded Pakistan Reading Project 8 READ FOUNDATION - Baseline Assessment Study
Major Findings
A summary of each subtask’s result follows:
! Orientation to Print - This subtask assessed students’ pre-knowledge before reading. It tested students’ ability to hold a book in the right direction, identify where the title of the story is located, show the direction of the text and identify starting and ending letters or words in a paragraph. Grade I students achieved 52.9 correct percentage average score whereas Grade II achieved 58.40. In grade I boys are slightly better than girls by achieving 53.1 in comparison to 52.60 percentage correct scores. Grade II has better results than grade I where boys achieved 57.2 and girls 59.80 percent correct scores. Hence boys are performing slightly lesser than girls of grade II. READ Foundation’s students have good understanding of orientation to print and there is no student with zero score in both grades.
! Phonemic Awareness – Phonemic awareness refers to the ability to explicitly identify and manipulate the sounds of language. Phonemic awareness has been found to be one of the most robust predictors of reading acquisition, as this skill can be used to put the sounds of letters together to make a simple word. In this subtask, children heard three words sets spoken aloud, and were asked which of the three words began with a different sound. Amongst administered students, 23.3% of grade I students and 18.2% of the grade II students achieved zero scores. Initial sound identification was challenging for students, because on average, grade I students could identify the initial sound of 4.3 (out of 15 sets) words, whereas grade II students could identify the initial sounds of 5.4 words.
! Letter Sound Knowledge – Letter-sound knowledge assessed children’s automaticity in their knowledge of the sounds associated with each letter. This skill is critical, as it enables children to decode, or sound out, new and unfamiliar words. In this subtask, students were given a chart with 100 letters and were asked to produce the sound associated with each letter. More than half of students - 53.7% in grade 1 - and almost
USAID-funded Pakistan Reading Project 9 READ FOUNDATION - Baseline Assessment Study
near to half of the students - 48.2% in grade 2 - did not know the sound of a single letter. Among the students who knew at least one letter sound, students knew the sounds of an average of 11.2 letters within one minute (grade 1, 10.6 CLPM and grade 2, 11.8 CLPM), in both grades, girls are more fluent than boys. In grade I fluency for girls is 11.59 CLPM whereas boys are 9.68 CLPM and in grade II fluency of girls is 14.01 CLPM and boys is 9.85 CLPM. Although girls showed more automaticity in their knowledge of letter sounds than boys, these differences were small.
! Reading Familiar Words – This subtask assessed children’s skill at reading high frequency words. Recognizing familiar words is critical for developing reading fluency. In this subtask, children were given 50 simple familiar words drawn from grade 1 & 2 textbooks of different provinces/regions to read within 1 minute. Approximately 19.1% of students in grade 1 and 5.3% in grade 2 could not read a single word. Among the students who could recognize at least one word, grade I students could read an average of 27.7 words within one minute (boys, 22.05 CWPM and girls, 33.19 CWPM), and students of grade II on average of 45.0 familiar words within one minute (boys, 40.56 CWPM and girls, 50.10 CWPM). Once again, girls could correctly read more familiar words than boys, with a good margin of differences between boys and girls.
! Reading Fluency and Comprehension – This sub-task assessed children’s fluency in reading a passage of grade-level text aloud and their ability to understand what they had read. In this subtask, children first read aloud a short story consisting of 60 words, with a simple plot. Children then answered 5 questions related to the story. Reading the story was a timed task for which the enumerator had to record the number of correct words read per minute.
o Reading Fluency – paragraph reading – Reading the grade-level passage was not very challenging for grade II students whereas it was a bit challenging for grade I students. Almost 12.6% of grade I students and 3.3% grade II students could not read a single word of the passage. Among the students who could recognize at least one word, students could read an average of almost 39.2 words within one
USAID-funded Pakistan Reading Project 10 READ FOUNDATION - Baseline Assessment Study
minute (CWPM) (grade 1, 26.6 CWPM and grade 2, 51.8 CWPM). Once again, girls could correctly read more connected words than boys in grade I (boys 21.28 CWPM, girls 31.80 CWPM) as well as in grade II (boys 43.48 CWPM, girls 61.25 CWPM).
o Reading Comprehension – Given children’s challenges in reading the passage, it is not surprising that almost 15.55 children could not answer any questions about the story. Among these children in grade 1, 21.9% whereas 9.2% grade 2 students got zero scores on the reading comprehension test. Typically, good reading comprehension for a grade II student is thought to occur when a child could answer at least 4 out of 5 comprehension questions correctly. The data shows that, on average, grade 1 student are able to respond 1.91 correct questions (out of 5) whereas grade 2 students are able to respond 2.87 correct questions. Although girls are leading as compared to boys in responding correct answers but difference is small.
o Student Demographics –
o Languages - A majority of the students who were surveyed in Urdu language at READ Foundations schools spoke Pahari (33.1%), Urdu (39.2%), Shina (15.4%), Punjabi (6.1%) and Gojri (3.4%) at home. Almost 2.9% students reported 06 additional languages as their mother tongue (such as Burushaki, English, Kashmiri, Khaur, and Hindko).