Faqs (Frequently Asked Questions) s2

FAQs (Frequently Asked Questions)

What is ASER Pakistan? ASER - The Annual Status of Education Report (ASER) is a survey of the quality of education. ASER seeks to fill a gap in educational data by looking to provide a reliable set of data at the national level, that is comprehensive and, at the same time, easy to understand. The survey’s stated objectives are three fold:  To get reliable estimates of the status of children’s schooling and basic learning (reading and arithmetic level)  To measure the change in these basic learning and school statistics from last year  To interpret these results and use them to affect policy decisions at various levels. The Annual Status of Education Report - ASER Pakistan is the largest annual survey of rural and urban children done by the citizens of Pakistan every year since 2008-09. ASER is facilitated by SAFED and is carried out by a local organization in rural and urban districts in Pakistan. In 2011, the ASER data was collected from 84 rural and 3 urban city districts of Pakistan. The survey covered 2502 villages, 97 urban blocks, 3642 schools, 49793 households and 146874 children. ASER asks children whether they are enrolled in school. ASER also asks children to read a simple text and do some very basic arithmetic tasks.

Objectives The survey’s stated objectives are: . To get reliable estimates of the status of children’s schooling and basic learning (reading and arithmetic level) at the district level; and, . To measure the change in these basic learning and school statistics from last year . To share the results for action regionally and internationally to the global reporting and campaigns for quality education for all. . To leverage the results for appropriate education interventions with in- school and out of school children in Pakistan. . To build capacity for citizen led initiatives to improve understanding and options of: where, what and how well students learn

Alongside the above, there will be rigorous efforts to obtain disaggregated insights into gender trends as well as public and private systems of education

Phases  Phase I : Year I - 2010 – 32 districts across Pakistan  Phase II : Years II – 2011 – 84 rural and 3 urban districts across Pakistan  Phase III : Years III, IV , V all districts across Pakistan

FAQs - ASER 2012 1 FAQs - ASER 2012 2 What is the sampling technique used in ASER? a) ASER is carried out in all five provinces i.e. in Sindh, Balochistan, Punjab, NWFP, Gilgit - Baltistan, and FATA, AJK. b) District selection – The eventual aim of ASER is to be conducted in all the districts but for ASER Pakistan 2011, 84 rural districts were selected using presence of key partners ( where collaborating partners are available) c) The sample size is 600 households per district. The sample design is a two- stage stratified sample. The sample is obtained by selecting 30 villages per district and 20 households per village.

Number of Rural Number of Villages per Number of Households per Districts District Village

84 30 20 d) In rural ASER, the Villages are selected randomly using the village directory of the latest Census (1998). The sampling is done using the Probability Proportional to Size Sampling (PPS) technique. The PPS is a widely used standard sampling technique and is the appropriate technique to use when the sampling units are of different sizes. In our case, the sampling units are the villages. This method allows villages with larger populations to have a higher chance of being selected in the sample. e) In urban localities, blocks are identified with FBS’s technical assistance and systematic sampling is used for household selection

Households PSUs(12 HHs Per City (HHs) PSU) KARACHI 412 34 LAHORE 328 27 PESHAWAR 428 36 Total 1168 97 (t=95%, d=0.05) (PSU’s are the primary sampling units i.e. Blocks) f) In the rural ASER sample, every year, 20 villages from the previous year are retained and 10 new villages are added. Ten villages are randomly dropped each year from the previous year’s list, and 10 more villages are added from the population census village directory. The 10 new villages are selected using PPS. The 20 old villages and the 10 new villages gives a “rotating panel” of villages, which generates more precise estimates of changes. Since one of the objectives of ASER is to measure the change in learning, creating a panel is an appropriate sampling strategy. g) Each district team receives a village list with appropriate information along with the data from the Population Census on total number of households and total population. The village list will also specify which villages are from last year survey list, and which are new villages. The village lists will be final and cannot be replaced under normal circumstances. This is to maintain randomness of the sample and to obtain reliable estimates.

FAQs - ASER 2012 3 h) The households are chosen after making the village map and determination of the number of hamlets. If the hamlets are more than four, chits are made with numbers which are randomly drawn to select four hamlets. Once the hamlets are selected, whatever is the population size, for each of the four sections a rough estimation of the population is made. In each section the surveyors find a central point and then every fifth household beginning from the left hand side is selected. In the ASER survey households where there are no children or no children in the age group 3 – 16 are also selected and their relevant information is taken down. A household closed or a household refusing to participate is noted as ‘closed and no-response. What are ASER’s Learning Assessment Tools? a) Reading Assessment Tool - Urdu and local language: Reading assessment tools have been developed in three languages - Urdu, Pashto and Sindhi. Children can choose to be tested in a language of their own choice. Therefore, the ASER team has a set of tools in any basic language that the child is likely to know. The language tool is divided into four levels of assessment, which are as follows 1. Alphabets (class 1 level text) 2. Words (class1 level text) 3. Easy Paragraph(Class 1 level text) 4. Simple Story(Class 2 level text) b) English Language Tool: (Reading & Comprehension) 1) Reading: The English language tool is divided into four levels of assessment, which are as follows 1. Capital alphabets (Pre- school/class 1 level text) 2. Small alphabets(Pre- school/class 1 level text) 3. Simple words (class 1 level text) 4. Easy sentences (class 2 level text) 2) Comprehension: English comprehension is assessed in following two ways 1. Meaning of words: (child is marked at “word” level) 2. Meaning of sentences: (child is marked at “sentence” level) c) Arithmetic Tool: There are four categories in the arithmetic tool for assessing basic numeracy skills of children. These include: 1. Number recognition 1 to 9 (Class 1 level text) 2. Number recognition 10 to 99 (Class1 / Class2 level text) 3. Subtraction (Class 2 / Class3 level text) 4. Division (Class 3 level text) d) Mothers’ literacy tool Simple sentences in Urdu/ local language are used to test mothers’ literacy

FAQs - ASER 2012 4 e) Survey Tools 1. Village Map 2. Household Survey Form 3. School Survey Form a) Government School: Attendance teachers/students; Facilities and Grants b) Private School: Attendance teachers/students; Facilities and Grants

How many districts will ASER Pakistan cover from 2010-2015?  Year 2010 - 32 Rural districts  Year 2011 - 84 Rural and 3 Urban districts  Year 2012 and thereafter all districts of Pakistan each year until 2015

What are the key characteristics of ASER Pakistan?  ASER is carried out in an inclusive campaign mode as a national movement by citizens.  SAFED involves ordinary citizens in the process of data collection and survey empowering them with an accessible tool for evidence gathering & action  ASER Associates are selected each year to be trained in the practice, theory and research skills of ASER through a formal modular training lasting 20 days.  Workshops are held at the national, provincial and district levels to train volunteers.  Survey is undertaken within 6 weeks and the results are sent to the central secretariat of ASER Pakistan for consolidation of data and report writing.  A communication strategy is developed for the dissemination of ASER findings  The final report is launched at three levels ; at National, Provincial and Districts levels  NEAS/PEC other assessment initiatives are kept in the loop, informing them about ASER Pakistan and actively seeking synergies

How ASER compares with other assessment systems in Pakistan? Recognizing the significance of measuring learning levels, the Pakistan Government has in recent years undertaken large scale assessment initiatives. One such assessment system already in place is the National Education Assessment System (NEAS). Another is the Punjab Examination Commission (PEC) endeavor. However, ASER is different from the above mentioned initiatives in several ways. Firstly, ASER is based on a household survey and thus assesses everyone in a given age group within a household. This allows a comparison of learning differences among those enrolled and not enrolled in school. Secondly, ASER measures more generic and basic numeracy and literacy skills rather than the subject and often curriculum-based specific skills measured by NEAS and PEC

FAQs - ASER 2012 5 for specific grades (Grades 4, 5 and 8). This is important because it allows us to paint a more vivid picture of basic learning than has historically been available.

FAQs - ASER 2012 6 How is ASER different from NEAS and PEC?

National Education Punjab Examination Annual Status of Assessment System Commission (PEC) Education Report (NEAS) (ASER) NEAS was established in PEC is a provincial ASER is a household the late 90s as a country- initiative of the based survey to measure wide initiative to build government of the Punjab reading, comprehension assessment capacity at established in 2004 to and numeracy skills for school, provincial and address the Quality children between the federal levels to measure Challenge. Punjab ages of 3-16 Launched in learning outcomes and Examination Commission India in 2005 by Pratham improve the quality of is an autonomous body to (an NGO) has been education. To date NEAS administer assessment- conducted consecutively has conducted three only examinations for each year (2005-2009) rounds of subject-based grade 5 and grade 8 in all across all districts of assessments: subjects and both public India, with a focus on (www.neas.gov.pk). and private schools. To rural areas. date four rounds of NEAS is planned as a assessments have been Compared to NEAS and sample-and school based held in 2006, 2008, 2009 PEC, ASER is a household national assessment for and 2010 based survey looking at grades 4th & 8th in 4 (www.pec.edu.pk). all types of schools and subjects: Language, more generic skills of Maths. Science, and PEC is school based and numeracy and literacy up Social Studies. is conducted in February to grades 2 & 3, each year as an annual compared to subject To date four rounds of exam covering specific learning levels. assessment have taken approximately 2 million www.safedafed.org place. In 2009 & 2010 plus children from govt. www.asercenter.org NEAS was not and private schools implemented.

FAQs - ASER 2012 7